Indexed GEDCOM Method Detailed Setup Instructions

---

Concept originally created by Tim Doyle. Modified and enhanced by Randy Winch <gumby@edge.net>, with additional scripts added. Further modified and more new scripts added by Doug Macdonald <DAMacdonald@juno.com>.

This file is more than 10 pages long, and that may seem like a lot, or make it appear as if this is a complicated program, but it really isn't. These instructions are very detailed, and should walk you through the process step-by-step. Since so much depends on your server's setup (and every server is different <sigh>), much detail is needed to avoid confusion. My goal is to get you up and running on the first try.

OVERVIEW

The Indexed GEDCOM Method (IGM) is a powerful and flexible method for formatting and presenting genealogical data on the Internet. It employs CGI scripts to operate on an ordinary GEDCOM file in order to index it, and also to format various reports and charts in response to user requests.

A GEDCOM file is a kind of the universal language allowing communication between the growing number of genealogical database programs on the market. Whatever program you use, it should be able to export its data as a GEDCOM file, which can then be read by another program. This means that no matter what program you have used to assemble your data, you should be able to use the IGM method to display it on the Internet.

THe CGI scripts on which the IGM system is based are computer programs which are run on the web server, as opposed to the client or user machine. The IGM scripts are written in the Perl programming language.

The Indexed GEDCOM Method is simple to operate and to use. When a new or updated GEDCOM is to be made available, the program IGMMak is run on it. This program creates a master index file as well as a surname index and several HTML sub-index files. This process takes only about 1-2 minutes to index a 7,000+ person GEDCOM file, although this may vary based upon the system you are using.

Once the GEDCOM is indexed by IGMMak, users can access the automatically generated HTML index files or use the built-in search engine to find the person they are interested in. When they request information on any person in the database, the HTML index instructs the Web Server to run the IGMGet program. IGMGet opens the GEDCOM and goes directly to the person directed. It then loads all of the information for that person and his/her family, generates an HTML file, and sends that HTML file back to the user. The page generated by IGMGet contains links to request customized pedigree or descendancy charts for the subject of interest, to request a downloaded GEDCOM centered on the person, and to add a link to information about this person anywhere else on the web.

The HTML index files allow the user to search the database either by a range of names, or by an every-surname index. The user can also search names and places with the IGMSrch program or the entire GEDCOM for any text with the program IGMFind.

An important thing to note is that the HTML files sent to users are generated on the fly at the time of the request. This approach allows much greater flexibility in formatting information than methods which rely on a fixed set of pre-generated static webpages.

All of the scripts, IGMMak, IGMMake, IGMLivng, IGMGet, IGMFind, IGMSrch, IGMLink, IGMPed, IGMDesc, IGMGed, IGMPedTree, IGMDescTree, IGMLib, and IGMIni are written in the Perl programming language and can be modified, provided that the original copyright information is not removed or modified.

Other exciting features include:

• CGI requires less storage space - a LOT less than static HTML files
• Allows the inclusion and display of photographs
• NAME/PLACE search engine built in
• FREESTYLE TEXT search engine built in (your option)
• Pedigree charts in text or table format (user can specify the number of generations)
• Descendancy charts in text or table format (user can specify the number of generations)
• Source citations and notes displayed
• Handles aliases (your option)
• Owner's name and email address on every page
• Automatically logs every hit and every download
• Easy Linking (built-in) between individuals in remote databases
• Generates custom GEDCOMs for downloading
• Creates a file for use with both John Rigdon's and Gene Stark's indexes

BEFORE YOU BEGIN

There a few things that you'll need to know before you can use these scripts. You should check with your web hosting provider you don't know the following:

1. Does your server allow user-written CGI scripts? This is absolutely necessary in order to run these programs. Not all web hosting providers support CGI. In particular, the large mass market homepage providers like geocities.com and homepage.com do not allow it. Some other providers will only allow you to run scripts which they themselves have written or approved. The reason for this is easy to understand. By allowing user-written CGI scripts, the web service provider is letting you run your own programs on their server. Besides putting an extra load on their processor, it opens the risk of your program crashing and doing damage on their machine. The particular web hosting provider which I use (spaceports.com) not only allows CGI use, but provides 10 MB of server disk space, all for free. The only cost is that they put a popup banner advertisement on your webpages.
2. Do they have a version of Perl installed? Also necessary.
3. What version of Perl is it? You'll need to know if it's version 5 or better. Type "perl -v" to check.
4. Where is Perl located on the server's hard-drive? (What is it's path? - you'll need to be able to call it.) Type "which perl" to check.
5. What is the 'real path' to your home page. (NOT the URL, but the path as seen on the server's hard drive. Usually something like /home/your_name/public_html)
6. Does your server require that you place all CGI scripts in a special directory? If so, what is it's path? This varies from server to server. Some servers require that all CGI scripts be in a subdirectory on your account called cgi-bin. On the other hand, my own web hosting provider (spaceports.com) lets you put them anywhere except a directory called cgi-bin.
7. Does your server require that CGI scripts have any specific extensions, such as *.pl or *.cgi? If so, then you might have to do a little bit of editing of the scripts in addition to renaming the scripts.
8. You'll need to create a directory structure on your server to contain your database(s), one main database directory, and then subdirectories of that - one for each GEDCOM. For example: if your URL looks like this http://www.server.com/~your_name/ and your main HTML directory path looks like this: /home/your_name/public_html, then you'll need a subdirectory of that such as /home/your_name/public_html/gedcom. You don't have to call the new subdirectory 'gedcom'. You can give it whatever name you want. This is the "main database directory" referred to later. The subdirectories off of that (which will hold one GEDCOM each, plus its generated files), should have the same name as the GEDCOM they hold, with the same capitalization. Example: for Macdonald.ged, the directory name would be /home/your_name/public_html/gedcom/Macdonald
   The name of the directory must have the same capitalization as the name of the GEDCOM file. If you plan on adding photographs to your database, you will need a subdirectory of your main database directory called 'photos'.
9. Map out your directory structure on paper. This will help you with the installation. My system is as follows:
   • Your web directory: /home/macdonal/public_html
   • Your main database directory: /home/macdonal/public_html/gedcom
   • Your database directory: /home/macdonal/public_html/gedcom/Macdonald
   • Your photos directory: /home/macdonal/public_html/gedcom/photos
   I have referenced my directories listed above as examples in the steps below. This should help you in determining exactly which directory the steps are referring to.

DOWNLOADING AND UNPACKING THE IGM SUITE

The following paragraphs summarize how to download and unpack the IGM program suite. A "packing list" summarizing the files which comprise it is also provided.

• If you have not already done so, download the IGM programs by clicking on the link below.
  
  It will be most convenient if you create a directory to save them to. The scripts are in the form of a self-extracting ZIP file called igm.exe.
• Unpack the scripts by running the file igm.exe, either by invoking it from the DOS command line or by double-clicking on it from within Windows Explorer. The files making up the IGM program suite will be unpacked into the same directory the ZIP file is in.

The following summarizes the files contained within the IGM program suite. More detailed descriptions are given later. Please check to make sure that you have all of these.

In addition to the above files, you'll need 2 additional files, and an optional third file, which you will supply. The first, obviously, is the GEDCOM itself. The second (required) file is a data file which should have the same root name as the GEDCOM, and the extension '.dat' (e.g. Macdonald.dat). This file is used to put the owner's name and email address on each created page. Details of this file are given later. The third (optional) file is a data file which should have the same root name as the GEDCOM, and the extension '.info' (e.g. Macdonald.info). This file should contain a short text introduction or description of your database, which will be included on the main index HTML page. The contents of this file are also described later.

SERVER DATABASE STRUCTURE

As a result of running igmmak on the server, a set of static (permanent) files will be created. For instance, if the GEDCOM is called Smith.ged, then the files will be as follows:

It is important to note that most of these permanent files are indices into the database. Most of the files sent to a user in response to specific requests are generated at the time of the request.

PREPARING THE SCRIPTS

Depending on the conventions used by your server, you may need to modify some or all of the scripts to make them functional on the server. This is most easily done on your home computer before uploading the files to the server. The editing should be done with a text mode editor such as the DOS Editor or Notepad. The following paragraphs summarize the changes which need to be made to each file.

igmini
This file is called by every one of the IGM scripts. It contains all the main variables and the paths which are used throughout the IGM series. This is also where optional items are set, such as whether or not to include aliases on an individual's page, background colors or graphics, default depths of the displayed pedigrees and descendant charts, etc.

This file will require the most editing, but it's fairly straightforward. The first section deals with defining your URL and path - on your server. The script contains my own personal setup. If it will help any, my online database is located at

     http://www.macdonda.spaceports.com/genealogy/Macdonald/Macdonald.htm

Please compare the above to what you see in the igmini file. It may help you to understand this section.

The next section covers the names of the scripts. Please make sure that the names here match the actual names and extensions of the scripts, including capitalization. In other words, if you renamed the scripts for any reason, be sure to change them in this section, too.

The last two sections of the igmini file contain the options that you can set. They're pretty self-explanatory.

The table below explains this in more detail.

igmlib
This file contains all of the subroutines that may be called from any of the IGM 'modules'. This is an extremely important file, and fortunately, requires no editing.

igmget.cgi
This script does the majority of the work for the web users. When an individual is selected for displaying, this is the script that creates that individual's page. It reads in the GEDCOM, extracts the information, and displays the page. Once another individual is selected, or the user leaves the program, the individual's page no longer exists on your server.

The only lines you may need to edit are the following:

igmdesc.cgi
This script is called whenever anyone chooses to display a text-format descendancy chart.
Changes to this script are the same as for igmget.cgi

igmped.cgi
This script is called whenever anyone chooses to display a text-format pedigree chart.
Changes to this script are the same as for igmget.cgi

igmdesctree.cgi
This script is called whenever anyone chooses to display a tree-format descendancy chart.
Changes to this script are the same as for igmget.cgi

igmpedtree.cgi
This script is called whenever anyone chooses to display a tree-format pedigree chart.
Changes to this script are the same as for igmget.cgi

igmsrch.cgi
Searches the *.gdx file instead of the entire GEDCOM for a particular name or surname, and returns a linked list of all matches.
Changes to this script are the same as for igmget.cgi

igmfind.cgi
Does a FREESTYLE TEXT search whenever anyone choses that option from the Master index page, and returns a linked list of all matches.
Changes to this script are the same as for igmget.cgi

igmged.cgi
This script creates the customized GEDCOMs for downloading.
Changes to this script are the same as for igmget.cgi

igmlink.cgi
This script is called when you want to link an individual in your database to another database containing the same individual.
Changes to this script are the same as for igmget.cgi

igmmak
Executed from the command line while dialed in to your shell account. This script is responsible for reading in the GEDCOM and creating all indexes and other static pages. It is much faster than igmmake.
Changes which may be necessary for this script are the following:

• #!/usr/bin/perl      Change this to the path from step #4 of 'Before You Begin'.
• Modify the path in the 'unshift' statement near the start of the file to point to your cgi-bin directory. You can specify a relative pathname from your main database directory.

igmmake
Like igmmak, this script would be executed from the command line while dialed into your shell account. It is now essentially obsolete, and is included in the IGM distribution for reference. Does basically the same thing as igmmak, but is a lengthier process.
Changes for this script are the same as for igmmak.

igmlivng
Like igmmak, this script is executed from the command line while dialed into your shell account. It should be run before igmmak is run, and will will create a GEDCOM file with the birth information of living individuals removed.
Changes for this script are the same as for igmmak.

SUPPORT FILES

A couple of files are used to automate the process of creating the index files from the GEDCOM. These are both small ASCII text files. You can create these file in any text editor (The DOS Editor or Notepad will be fine.)

DAT File
This file is used to put the owner's name and email address on each created page. It should have the same root name (with the same capitalization) as the GEDCOM, and the extension '.dat' (e.g. Macdonald.dat). This file contains up to four lines, the first two of which are required, and the last two of which are optional.

1. The name of the database owner
2. Their email address (no '<' or '>', just the address.)
   Example: mine is DAMacdonald@juno.com
3. A title for this database if different from the directory name. For instance, my database is called the Gordon-Fletcher-Macdonald database, even though it is derived from a GEDCOM file I just call Macdonald.ged. Therefore, I have the line "Gordon-Fletcher-Macdonald" here. If you don't include this line, the database will have the same name as your GEDCOM file.
4. The WWW address of the database owner's homepage, so that a link to it can be added to the IGM-generated pages. If this is not specified, the HomePage parameter in the igmini file will be used.

INFO File
This file is used to "personalize" the main index HTML page of your database. It should have the same root name (with the same capitalization) as the GEDCOM, and the extension '.info' (e.g. Macdonald.info). It should contain a short text introduction or description of your database, which igmmak will insert on the main index HTML page when it creates it. This file can contain more than just text; it can contain HTML tags and references to images to be included on the index page. Keep in mind, though, that this description should be short enough that it does not push useful links down off the initial browser screen view of the main index page. The INFO file is optional; if you don't want a description on the main index page, you don't need to create this file.

FINAL PREPARATION

Three more steps remain in preparing the scripts for upload to the server. The first step is only necessary if you are going to add photographs to your database. The last two steps are only applicable if the server is running UNIX or one of its variants. This will apply in most cases, however, since most servers where Perl is available are UNIX servers. Some of these steps will modify your GEDCOM file in ways which may may make it incompatible with the programs which created it. Therefore, it is strongly suggested that you make a copy of the GEDCOM in a special pre-upload directory and perform these steps on that copy.

1. If you plan to use photographs in your database, download and follow the instructions for adding photographs. When you follow these instructions, you will create a revised GEDCOM file with special additional tags pointing to your photographs.
2. UNIX is case-sensitive, while DOS and Windows are not. That is, UNIX will see the files Igmget.cgi and igmget.cgi as different files, while DOS and Windows will see them as the same. Windows is often cavalier about changing the case of file names since it makes no difference to it, but it will make a difference when the files are uploaded to the UNIX server.
   The convention used for the IGM scripts is that all script names are in lower case, while the initial letter of the GEDCOM file is capitalized, so you should do a directory listing, and if some filenames are not consistent with this convention, change them with the DOS rename command. Note: Going through DOS is the surest way of doing this; Windows will sometimes not change the name as it should.
3. Finally, another difference between DOS/Windows and UNIX is that they denote the end of a line of text within a file in different ways. UNIX puts a Line-Feed character (LF = ASCII 10) at the end of each line, whereas DOS/Windows ends each line with the two-character combination Carriage-Return (CR = ASCII 13) and Line-Feed. Files and scripts with the DOS convention will be uninterpretable when they are uploaded to the UNIX server. Therefore, a small program called d2u.exe is included with the IGM programs. This file, which runs on a PC, removes the CR characters from the end of each line. You should run this program on any file you are going to upload just before uploading it. To run it, just type e.g.
   
        d2u igmget.cgi
   
   at the DOS prompt. Run this file on all of the CGI scripts, plus the GEDCOM file, and any other files you upload. Since this process makes permanent changes to the files which may make them uneditable on your home computer, it is strongly suggested that you make copies of all the files in your special pre-upload directory before performing this step, and make the changes on these copies. Note that whenever you upload a new GEDCOM file, you will have to run d2u.exe on it first.

SETTING UP THE PROGRAM ON YOUR SERVER

Now you're ready to upload the programs to your server and set up your database. The instructions below assume the GEDCOM database being created is named Macdonald, and that the installation is on a UNIX server. You should modify the procedure accordingly for your setup. Note that the following procedure assumes that you have TelNet access to your account. It is still possible to set things up if this is not true; just see the notes at the end of the precedure.

1. Dial into your shell account using TelNet or any other program you choose. Move to your main HTML directory (probably something like /home/your_name/public_html). Now create your directory structure (See step 8 of 'Before you Begin'). For example:
        makedir gedcom
        cd gedcom
        makedir Macdonald
        makedir photos (if you plan on adding photographs to your database)
   Also, create your cgi-bin directory wherever required by your system.
2. Using FTP, dial into your server and upload the following scripts and files to the following directories:
   **** VERY IMPORTANT **** Do NOT upload using BIN. Use ASCII.
   1. To your "main database directory" (Example: /home/your_name/public_html/gedcom)
      • igmmak
      • igmlivng
      • igmmake (if desired)
      • photo.gif
      • bkgd.jpg
   2. To the subdirectory of the main database directory particular to this GEDCOM (Example: /home/your_name/public_html/gedcom/Macdonald)
      • the GEDCOM
      • the DAT file
      • the INFO file
   3. To the 'photos' subdirectory of the main database directory (Example: /home/your_name/public_html/gedcom/photos)
      • any JPG or GIF pictures referenced from within the GEDCOM (remember the capitalization of their names is significant)
   4. To the CGI directory
      • igmini
      • igmlib
      • igmget.cgi
      • igmdesc.cgi
      • igmped.cgi
      • igmdesctree.cgi
      • igmpedtree.cgi
      • igmsrch.cgi
      • igmfind.cgi
      • igmged.cgi
      • igmlink.cgi
3. Now in TelNet again:
   1. Change to your "main database directory". Make igmmak, igmmake, and igmlivng user-executable by typing the following commands at the UNIX prompt:
           chmod u+x igmmak
           chmod u+x igmmake
           chmod u+x igmlivng
   2. Make the directory in which your GEDCOM resides world-writeable, e.g.
           chmod a+w Macdonald
      This is necessary if you plan to let users access the Link function, which requires them to write small files to this directory.
   3. Change to whatever directory your other .cgi scripts are in and make them world-executable by typing the following command at the UNIX prompt for each of the .cgi files:
           chmod a+x igmget.cgi
           chmod a+x igmped.cgi
           Repeat for all .cgi files
   4. Change back to your "main database directory" and enter the following at the UNIX command prompt:
      
           igmlivng Macdonald
      
      This command will modify the GEDCOM file to exclude birth information for living individuals. This is a good idea for privacy considerations.
      The first part of this command should be the exact name of the 'IGMLivng' script (extension included), followed by a space, then just the name of the GEDCOM - with proper capitalization. If this syntax doesn't work, try typing
           perl igmlivng Macdonald
      If you get an error message saying that the files igmini and igmlib couldn't be found, try typing
           perl -I<cgi-dir> igmlivng Macdonald
      where <cgi-dir> is the full pathname of the directory where the CGI files reside.
   5. Enter the following at the UNIX command prompt:
      
           igmmak Macdonald (or you may have to modify the command as in step 4 above)
      
      At this point, all of the static pages will be created - the Master Index page, all other HTML index pages, the .gdx file, the database index file, igmlog.htm, etc. This process takes only a short time, and depends on the size of the GEDCOM.

Once this step is complete, you'll finally be ready to enjoy your handiwork! You can close down FTP if you haven't already, and you can close down TelNet or whatever other program you used.

Start up your web browser. Enter the URL of the Main Index Page, which will look something like this:

     http://domain/~you/main_database_directory/gedcom_name/gedcom_name.htm

You should see your main index page come up. Check the links on it to be sure that they are all resolved. Go to an individual's data page and check it out. Try doing a search for a name.

THAT SHOULD BE ALL THERE IS TO IT !!!

Although the above procedure assumed that you had TelNet access to your account, this is not really necessary. All of the setup can be done through FTP alone, with only a little more difficulty. All of the directory creations and file permission settings performed above can be done with any reasonably full featured FTP program, for instance WS_FTP from Ipswitch. And the IGMLivng and IGMMak scripts can be run via http requests by simply renaming them as ordinary CGI scripts and uploading them to your cgi-bin directory. Since they take single arguments, they can be run with a command like:

     http://domain/~you/cgi-bin/igmmak.cgi?Macdonald

Once you're sure your site is working, there are a couple of optional steps you can take to make it more visible to the outside world.

• Add a link from your home page to the main index for your database.
• Add your database to Gendex, the master database of genealogical databases, by following the instructions here.

Note that updating your site (for instance after changing the GEDCOM), is substantially easier than this first-time installation. Just make a copy of your GEDCOM on your home PC, run addphotos.exe and d2u.exe on it, then upload it to your server using FTP, and run igmlivng and igmmak on it there.

TROUBLESHOOTING

If something is not working, check first for the following common errors.

1. Having a mismatch in capitalization between the name of a program and a call to it from within another program.
2. Failing to run d2u.exe before uploading a program or GEDCOM.
3. If you get a message indicating the system can't find the required programs igmini or igmlib, you may need to experiment with how you specify their location in the headers of the various CGI files (see the section on igmget.cgi under 'Preparing the Scripts' above. If all else fails, specify the full pathname to igmini and igmlib in each CGI file header.

If you experience any problems, please seek help with your server first. Most servers have user's group mail lists, and you're bound to find a CGI expert there. If all else fails, feel free to contact me

Note that the Indexed GEDCOM Method programs are freeware, which means that you can use and modify them as you see fit, provided that the original copyright information is not removed or modified. If you download them and find them of use, please let me know by email, and I will notify you if there are improvements or upgrades in the programs.

Doug Macdonald <DAMacdonald@juno.com>

Best of Luck to you, and ENJOY !!!