Monday, July 3, 1995 3:55:53 PM
          GenWeb Item
  From:           Gene Stark,gene@starkhome.cs.sunysb.edu,Internet
  Subject:        Re: Indexes of GenWeb Databases
  To:             GenWeb
>  * This takes a lot of disk space and cpu time to process.
>    The current one-line file with all information is about
>    25 Mbytes in size. It takes a lot of cpu to sort 25 Mbytes
>    in 350,000 records and it doesn't scale in a linear fashon.
>    When I double the size of things and get to 700,000 records
>    and 50 Mbyte, the amount of cpu required may be 5 or 10 times
>    as much. (Or 4 or 8 or 16 times as much for those of you
>    in the computer biz. :-)

This is indicative of an inefficient sorting algorithm.  Efficient sorting
algorithms run in time proportional to (N * log N), where N is the size of
the input file.  This means that, using such an algorithm, doubling the
size of the input file will only slightly more than double the time taken
to sort the file.

As an experiment, I just wrote a C program to generate a 350,000 line
file with each line containing 75 random ASCII characters in the range
'A' to 'z', followed by a newline.  The resulting file is 26,600,000 bytes
long.  Using GNU sort (from GNU textutils version 1.9) on my 486/33
system with 16MB of RAM, it took 108 seconds of CPU time (82 user,
26 system) and a total of 4 minutes and 23 seconds of wall clock time
to sort this file and write the output.  The sorting consumed 5MB of
virtual memory, and used temporary file space of about the same size as the
input file.

I wasn't able to clear the 150MB of disk required to run a test on a 700,000
line file (input file, output file, and temporary files), so I ran another
test on a 175,000 line file.  This took 52 seconds of CPU time (39 user,
13 system) and a total of 2 minutes and 3 seconds of wall clock time.  This
is slightly less than half of the time to sort the 350,000 line file,
as I would expect.  Sorting a 700,000 line file using the same method should
take about 10 minutes of wall clock time.

I briefly examined the source code for GNU sort, and noted that it is using
a merge sort algorithm in which on the first pass it reads the input file
in chunks of a size that can be sorted "in core", writes each sorted chunk out
to a separate temporary file, and then enters a merging phase in which it
repeatedly merges 16 sorted temporary files into one until the sorting is
complete.
							- Gene Stark


          Monday, July 3, 1995 4:09:50 PM
          GenWeb Item
  From:           mavrogeorge@genealogysf.com,Internet
  Subject:        Re: Indexes of GenWeb Databases
  To:             GenWeb
your model implies:
1. the central index authority actively maintains its own 

index, rather than
remains a passive archive of submitted information.
2. not explicit but it seems the central authority needs 

the index of the 
individual sites to build an index consisting of all 

individuals.
3. puts the burden of initial notification on the 

individual site, then the central site assumes the burden 

of keeping current information.
 
my model proposes:
1. the central index authority acts as a clearing house 

for information submitted from sites. It is a passive 

repository of information it does not actively seek out 

information for its indexes.
2. the central site uses summary material submitted from 

sites not the entire index for that site.
3. the burden for keeping an individual site's entries 

current at the central site lies with the individual site 

- always. The central site does not have any 

responsibility for the accuracy of the indees. It reflects 

what has been submitted.
4. the one active role the central site assumes is the 

periodic verification that a site link is valid. 
 
---
I prefer an approach which minimizes the central role and 

maximizes the individual ownership and responsibility. 

(sheeesh sound just like a Republican <grin>).



          Monday, July 3, 1995 4:15:42 PM
          GenWeb Item
  From:           Gene Stark,gene@starkhome.cs.sunysb.edu,Internet
  Subject:        Re: Indexes of GenWeb Databases
  To:             GenWeb
Tim Doyle writes:

>Some of the problems that I ran across when doing this index were:

The problems you list were the ones I was trying to address with my
proposal.

>1. Have each database provide a master name index in a common format.  
>When a database comes online, an e-mail message listing the pertinent 
>information (person's name, database name, URL of index file, etc.) would 
>initiate a program which would grab the master index and add the items to 
>the master index.  The same type of message would be sent if the database 
>were updated.  This puts some responsibilty upon the person making the 
>database available.

I would skip the e-mail message and instead start putting entries for the
databases under the domain GenWeb.ORG (actually, say, V1.GenWeb.ORG, for
version 1 GenWeb).  This way, the master indexer could simply do a domain
transfer to get the list of current GenWeb sites, grab the index files for
each one (they should have a standardized name), and update the master index.
The results could be made available in HTML format, and this would all be
completely automatic.  At about 100 bytes per individual, I would figure that
10 million individuals could be indexed in 1 gigabyte.  More could be done if
a non-naive scheme were used to store the indexes.
								- Gene



          Monday, July 3, 1995 4:27:14 PM
          GenWeb Item
  From:           Scott McGee,smcgee@microware.com,Internet
  Subject:        Re: Indexes of GenWeb Databases
  To:             GenWeb
Some of my own thoughts on indexing.

Since sites come and go at unpredictable intervals, it seems that trying
to "go out" and find all genweb sites to index is unlikely to actually
get them all. If a new site comes online, and it registers itself somehow,
then it can be added to a list of sites to index. What to do about sites
that go away depends more on the indexing method. If the index gatherer
is going to go out at frequent intervals to verify the current info it
has on a site, the there is no problem. If it doesn't do so, then we need
some sort of method to invalidate the info it has after a certain time 
unless the site inquestion has confirmed the data again.

Let me try to put it little more clearly. Each genweb site will have an
index that the indexing site can gather and generate an overall index with.
This index should include data info so that if the indexing site desires, it
can check to see if the index has changed.

When a new site comes online, it registers itself so that the indexing site
can obtain it's index and integrate it into the overall index info.

The site should also indicate somehow its permanancy. If the site is not
availible at either regular or irregular times, this should be known at
least to the indexing site so that it can retry later if it fails to get
through. I suppose some indication of downtime intervals and periods would
be needed. Also, some method to indicate how permanant both the site ant
the data is would be needed. The site may contain hardcoded data that will
never change (thus eliminating re-indexing needs) but may be a student 
account that will go away next june. (or on graduation in two years)
In other words, some sites need to be checked for existance fairly often
even though their info is unlikely to change. Other sites could be trusted
to stay around for very long times.

Some decision would have to be made as to wether a site will be re-indexed
by the indexing site, or if re-indexing will only happen if the site in
question informs the indexing site of a change in data. (how much change
justifies re-indexing?) This whole issue is separate from checking to see
the site is still up.

Obviously, if a site undergoes any major change in data (how to define that?)
then it should be able to inform the indexing site of that.

None of the above discussion deals with the format or contents of the indexing
info, not the actual method of obtaining it, but is still important to 
deciding how indexing is to be done.

Thoughts?

Scott

   |-   I never actually SAID that!   -|-   email: smcgee@microware.com   -|
   |------------ http://www.cc.utah.edu/~sam8644/homepage.html ------------|
   Programming in assembly language is a lot like digging a post hole with a
   teaspoon. Of course, you DO have complete control over every bit of dirt!



          Monday, July 3, 1995 4:49:04 PM
          GenWeb Item
  From:           mavrogeorge@genealogysf.com,Internet
  Subject:        GenWeb outline
  To:             GenWeb
What is happening with the GenWeb outline you proposed? 

Has it been fleshed out?



          Monday, July 3, 1995 4:49:19 PM
          GenWeb Item
  From:           JohnR238@AOL.COM,Internet
  Subject:        Re: Indexes of GenWeb Databases
  To:             GenWeb
While I love the concept of a search engine going out and finding anything I
want (in 10 secs. or less), the fact remains that a small percentage of the
data I have is in GEDCOM format, 5%, a larger portion in unformatted etext
records gleaned from lists, postings, etc 10%,  etext family histories, maybe
5%, etext census records and other "lists" which are in computer manipulative
formats other than GEDCOM, 5% which alltogether makes 30%.  That number's
probably a bit high, because the VAST majority of my records are on microfilm
and another sizable set are on paper.  I would hazzard a guess that a mere
fraction of 1% of the available data has been converted to etext and probably
most of it never will be.  While I could conceivably create a consolidated
etext all name index in the forseeable future, anything more than that is
impossible.

Is the scope of our proposal to cover only an index to etext documents of one
kind or another, or to identify the source of data and the email way to
contact that site/person?

John R @ EDR



          Monday, July 3, 1995 4:54:26 PM
          GenWeb Item
  From:           Anders Andersson,andersa@Mizar.DoCS.UU.SE,Internet
  Subject:        Re: Indexes of GenWeb Databases
  To:             GenWeb
Annelise Anderson writes:
>I think I read somewhere that the purpose of the HTML <HEAD> tag was
>to block off part of a document, and that part might (sometime) be
>subject to indexing, whereas the <BODY> part would not.  An computer
>indexing program could then index just those portions of each document.

The purpose of the HEAD element, according to the HTML specification,
is to contain meta-information about the document (such as title and
author), while the BODY element is to contain the document itself.
I don't think the HTML specification says anything about what can be
indexed.  Some WWW crawlers may, at their option, decide to use data
from either element for indexing purposes, but I'm not aware of any
standards for that.

You may have seen something about the INDEX element, which may appear
in the HEAD element.  That's only a marker telling your client software
that the document can be the target of a search operation, and most
clients will react by putting up a search box for you to enter some
keywords.  Upon doing that, the server is supposed to perform the
search for you.  It's not as trivial as searching that particular HTML
document for the text entered, something your client can easily do for
you; the server may return any document it chooses, in HTML or some
other format, in response to your search.  The INDEX element itself
does not contain any information that can be indexed.

When designing generic mechanisms for automatic retrieval of GenWeb
data, I think we should try to avoid becoming dependent on any
particular presentation format.  HTML may be the one dominant
hypertext format on the Internet today, but we may find alternative
ways to present genealogical material in the future, and requiring
all databases to support HTML may impede such development.  Also,
I believe that most of the data which we may want to gather in this
fashion won't be easily found in the numerous HTML files provided by
each server.  We need to define precisely what kind of information
we want to gather before deciding what mechanism to use.
--
Anders Andersson, Dept. of Computer Systems, Uppsala University
Paper Mail: Box 325, S-751 05 UPPSALA, Sweden
Phone: +46 18 183170   EMail: andersa@DoCS.UU.SE



          Monday, July 3, 1995 4:54:30 PM
          GenWeb Item
  From:           Anders Andersson,andersa@Mizar.DoCS.UU.SE,Internet
  Subject:        Re: Indexes of GenWeb Databases
  To:             GenWeb
Bill Minnick writes:
>  I guess I'm saying I think "Searches" are the way to go; "Indexes" seem
>useless to me if a good search engine is also available.  Five years from
>now, when GENWEB has documented 100 million++ people, does anyone here
>really want to still be sight-reading through an old fashioned index?

I agree in general with your thoughts, though I'd prefer not to
ban the "indexing" concept without a thorough understanding of what
an "index" is that everybody agrees is obsolete.  You seem to equate
"index" with "old-fashioned index", and of course I don't want to
read through 100-million entry list of people.  Are you saying that
something smaller and more useful wouldn't be an index?

To me, the contents of an index needn't be spelled out in full for the
index to be useful.  As with an HTML form today, you could enter a
name in a box and have it looked up in a particular index.  Do you
call this a "search" just because you don't see the other names, or
what?  Well, it is a search through an index...

Sometimes, seeing other entries than the one you are looking for may
even be important to you.  The context helps you decide what to look
for, and to estimate your chances of a success (looking up randomly
chosen keywords and only getting "no match" in return is a pretty
boring task, and doubly so if you later discover that there were in
fact less than 50 words available to that particular search engine).

Sometimes I may actually want a full-text search to retrieve anything
at all about a person or a place, but sometimes a more restrictive
approach (like, searching a particular directory for someone) may be
preferred.  I don't want to rule out either option.  Perhaps the human
mind is in fact a little more systematic than we think our ideal,
user-friendly search engines should be?  After all, the computers of
past decades were also designed by humans...

I don't think indexing and searching are distinctly contrasting
concepts, defined so that we can pick one but not the other.  They
are just two facets of the process of locating information, and we
don't even know today what actual mechanisms will be covered by
those facets tomorrow.

We may need some explicit and visual examples of what constitutes
a "search" before we can discuss the relative merits of different
technical approaches in further detail.  Otherwise we might just
waste time disputing each other's terminology.
--
Anders Andersson, Dept. of Computer Systems, Uppsala University
Paper Mail: Box 325, S-751 05 UPPSALA, Sweden
Phone: +46 18 183170   EMail: andersa@DoCS.UU.SE



          Monday, July 3, 1995 4:54:39 PM
          GenWeb Item
  From:           Annelise Anderson,ANDRSN@HOOVER.STANFORD.EDU,Internet
  Subject:        Re: Indexes of GenWeb Databases
  To:             GenWeb
I think I read somewhere that the purpose of the HTML <HEAD> tag was
to block off part of a document, and that part might (sometime) be
subject to indexing, whereas the <BODY> part would not.  An computer
indexing program could then index just those portions of each document.
Probably a compromise.
Annelise




          Monday, July 3, 1995 4:54:49 PM
          GenWeb Item
  From:           Gene Stark,gene@starkhome.cs.sunysb.edu,Internet
  Subject:        Re: Indexes of GenWeb Databases
  To:             GenWeb
Bill Minnick wrote:

>  As a researcher, I want to create complex, conditional queries (in a
>simple, intuitive way), perhaps using a computer-aided form tool; but
>potentially keying on any information that might be found on an individual
>page.  Then I pass the query to a GENWEB Search Engine and (if I'm lucky)
>get back a small list of URLs (of high potential value to me) in a Web page,
>ready for me to to click and read the results.  If the results are good
>enough, I'd even be willing to wait more than 10 seconds for them.  I want
>the GENWEB Search Engine to find my material inspite of name spelling
>differences, place name differences, etc.  

This sounds fine.  However, my concern is whether this is currently doable
within a reasonable amount of storage space, computation time, and real time.
As a computer scientist, I am familiar in a general sense with techniques
for storing, organizing, and searching data.  As a WWW user, I have used
a number of the Web search engines such as Lycos, the WebCrawler, etc.
The Lycos catalog claims to have indexed between 2 and 3 million URLs.
GenWeb alone will easily have this many, and more, if it is to be very
useful.  Unfortunately, I have not looked into the experimental techniques
researcher are using to create these indexes, nor do I know how much storage
space these indexes occupy.  However, I would not be surprised if the Lycos
indexes for the 2-3 million URLs occupied 10 gigabytes of storage (but I
don't know).  For more URLs, one could expect proportionately larger amounts
of storage.  Will GenWeb in the near term be able to support this kind of
massive indexing?

My proposal was geared toward the idea of having GenWeb evolve from something
we can do right now, to the idealized thing we would all like it to be in
the future.  We already have a lot of people with data online, but to find
out what's there you have to contact each individual site and peruse their
indexes, which are in many different formats.  A standard index file would
make it possible for an automatic program that has been primed with a list
of GenWeb sites to maintain an up-to-date index of all individuals in any
currently accessible GenWeb database.  This is easily doable right now.
All we need is to agree on a standard for the index files, and perhaps to
get started entering database names in the genweb.org domain.

							- Gene Stark



          Monday, July 3, 1995 4:56:13 PM
          GenWeb Item
  From:           Beau Sharbrough,beau@connect.net,Internet
  Subject:        CFL:  GENTECH96 Tech Sessions
  To:             GenWeb
I enclose the call for lectures for the Tech Sessions.  It
seems relevant here.
--------------------------------------------------------
 Beau Sharbrough         The Rockets are back.
<beau@connect.net>       Break up the Rockets.
--------------------------------------------------------

GENTECH96
                                       4th Annual National Conference
                                  
             CALL FOR PRESENTATION IN TECHNICAL SESSIONS
                                  
GENTECH96 is proud to invite you to attend the Technical Sessions at our 4th
Annual Conference.  GENTECH is the Technology Conference for Genealogists.  The
Technical Sessions are the Genealogy Meetings for Technologists.  GENTECH96
Technical Sessions will be held in Plano, just north of Dallas, Texas, on
January 25 1996.

            THE GENEALOGY TECHNOLOGY SPEAKING ENGAGEMENT

As a lecturer at the GENTECH96 Technical Sessions, you will establish yourself
and your organization as leaders and innovators among the genealogy community.  
You'll share your ideas about open problems and current issues facing
technologists in this area.  You'll present your peers with your creative
approach to relevant issues.  

Another reason to participate is that addressing Technical Sessions entitles you 
to a reduced registration rate for the conference, as well as waiving the fee 
for attending the Technical Sessions.

The Selection Committee is composed of senior genealogy and technology
volunteers.  Presenters will be selected based on the technical merits of their
proposals and on their contributions to the knowledge base of genealogists
worldwide.  The deadline for informing us of your interest in speaking at these
sessions is August 15, 1995.

            THE REQUIRED BACKGROUND FOR YOUR PRESENTATION

Presenters at the Technical Sessions are asked to submit a short abstract
describing their presentation and its relevance to the genealogy and technology
community.  A suggested guideline is:  

A brief biographical sketch.
The title of your presentation.
Two to three sentence summary for the brochure.
Brief description of your organization, and problems that led you to this
solution.

Sessions are 20 minutes, followed by 10 minute question and answer periods. 
GENTECH will provide an overhead projector, LCD projection panel and screen.  If
you have other requirements, please contact us in advance.  The focus of GENTECH
Technical Sessions is the free exchange of information.  Presentations and
supporting documents must be free of commercialism.

ttach a copy of this form to your proposal.  Address it to:

General Chair:                          Program Chair:
John Wylie (Acting)                          Beau Sharbrough (Acting)
2818 Landershire Lane                        600 Six Flags Dr Ste 456
Garland TX 75044-5974                        Arlington TX  76011
(214) 495-4410                               (817) 633-4522
(FAX) 495-1569                          (FAX) 640-7533
Internet: jwylie@metronet.com                Cserve:     76057,2127
                                        Inet:     beau@connect.net

GENTECH96 Technical Sessions Call for Presentations Form

Name_____________________________________________________________________       
 
                                 
Title ______________________ Company ____________________________________

Street __________________________ City ___________________ ST/Zip _______

Day phone __________________ FAX ___________________ EMAIL ______________

Other information you think we need _____________________________________

_________________________________________________________________________

I acknowledge that I have my organization's approval to make the presentation
described, if applicable.  I have full legal rights to the materials I will
present, and agree that GENTECH INC may publish, audiotape, or videotape all or
part of my presentation.  I also agree to respect the commercial prohibition of
these sessions.

_____________________         _________________
Signature                     Date                            


                                  
August 15, 1995:         Abstracts due
November 15, 1995:       Presenter registration deadline
January 25-27:           GENTECH96 - BE THERE!





          Monday, July 3, 1995 6:06:01 PM
          GenWeb Item
  From:           Mickey Lane,MLANE@CSI.compuserve.com,Internet
  Subject:        Re: Indexes of GenWeb Databases
  To:             GenWeb
Greetings,

With all the talk of indexing multiple sites, I suppose it would
be in order to describe how ROOTSBOOK has been doing this for the
last several years.

If you haven't seen ROOTSBOOK's index, it's at

  http://mlane2.inhouse.compuserve.com:8000/GenWeb.html

The index currently has 350,000 people extracted from some 600
databases. The o/s-server combination is still malfunctioning
so your access may be iffy.

The production of this index has eight steps:

1. Make a list of the databases to be indexed.

2. Run a program that runs through the list in #1 and make
   an extract file. The extract file contains the book name
   followed by entry number, name and, if it exists, date
   and place of birth and death. The format of this file
   is very simple.

3. Run a program that converts the extract file to a one-line
   format file. This operation converts names to last-name-first
   and uses any available birth and death data. The one-line
   format is one line per person.

4. Run a program that slices the one-line file into semi-sorted
   files. It currently makes 3: one with surnames A-H, one with
   surnames I-R and the third with the remaining S-Z.

5. Sort the files produced by #4.

6. Concatenate the results of #5 back into one file.

7. Run a program that changes the one-line format into the
   last name subheadings you see on the ROOTSBOOK server.
   If the end result is intended for the printer, it comes
   out as one huge file. If it's for the web server, it's
   sliced into 26 smaller files, one for surnames beginning
   with A, one for B, etc.

8. For the server environment, the HTML stuff is added by the
   server on the fly. (This is true for all ROOTSBOOK material.)

This works fine if all the data is on one machine. If it's
distributed, you have to modify step #2 so that each machine
runs it's own extract program. You then insert a step between
#2 and #3 that gathers all the extract files and concatenates
them into one file to feed step #3.

Some observations:

  * This takes a lot of disk space and cpu time to process.
    The current one-line file with all information is about
    25 Mbytes in size. It takes a lot of cpu to sort 25 Mbytes
    in 350,000 records and it doesn't scale in a linear fashon.
    When I double the size of things and get to 700,000 records
    and 50 Mbyte, the amount of cpu required may be 5 or 10 times
    as much. (Or 4 or 8 or 16 times as much for those of you
    in the computer biz. :-)

  * As Tim mentioned, the names encountered come in all shapes
    and sizes. There are lots of cases where people have in-
    corporated misleading information, from an indexer's point
    of view, into the person's name. Ex: Mary, wife of Mr. Jones.

  * At 350,000 names, I've about reached the limit of pre-sort
    based on surname 1st letters. I'm about ready to go to some
    files based on the 1st two letters. My largest searchable
    file is just under 3 Mbyte. I consider this too much for a
    single web browser transaction.

  * Having one giant program to do all the work is awkward.
    Breaking the task into little pieces distributed by both
    location and time is much better. There are cases where
    some pecularity of one item to be indexed will break the
    indexing software. Breaking the job into pieces allows
    someone to fix the pecularity without having to re-do
    everything.

This process is transportable to all platforms if we write
programs to replace my step #2. I extract from ROOTSBOOK databases.

It would be easy to write a program to extract directly from
a GEDCOM (or any other) database. It would probably more difficult
to extract from someone's handcrafted HTML document and, as you
might expect, most difficult to extract from random text. The up
side is that once you replace step #2, all the other steps work.
(Step #8 is, of course, subject to discussion.)

If anyone would like sorces to this stuff, let's find an ftp site
where I can upload it. I can't make it available here.

Mickey.



          Monday, July 3, 1995 6:13:31 PM
          GenWeb Item
  From:           JohnR238@aol.com,Internet
  Subject:        Re: Indexes of GenWeb Databases
  To:             GenWeb
Earlier George stated...

>>I prefer an approach which minimizes the central role and 
maximizes the individual ownership and responsibility. 
(sheeesh sound just like a Republican <grin>).

I was thinking more libertarian myself ;)

Your proposal makes a lot of sense, especially as the web has evolved and
continues to develop.

My earlier thoughts were that I/we needed to control the data so that it was
not lost when a link became bad.  That's an impossible task because of the
volume and transitory nature of the data.  While some of us may have much
more data than others, and some links may be much more transitory than
others, I think that a small fraction of the data will be ulitmately lost if
we (1) encourage people to provide alternate (US Snail and phone #) ways of
contacting them, and (2) provide site(s) which are willing to accept data
which can no longer remain on the web for whatever reasons. Speaking for our
project at the Georgia-Lina Historical Society, I'm sure we would be willing
to accept any and all submissions, and there will certainly be many others
including the mormon church.  That in itself presents another dilemna, just
as we are facing with the census project. How does our indexing deal
with/recommend/purge duplicate sets of data?

The fact remains that the volume of data is greater than we can ever
accumulateor index and the growth of data (from a population explosion
standpoint and the growth of the web) will continue to grow exponentially
faster than we can catalog it.  The sooner we agree on a protocol and begin
the work, the better off we'll all be.

John Rigdon
email JOHNR238@AOL.COM



          Monday, July 3, 1995 6:21:47 PM
          GenWeb Item
  From:           Anders Andersson,andersa@Mizar.DoCS.UU.SE,Internet
  Subject:        Re: Indexes of GenWeb Databases
  To:             GenWeb
John R writes:
>Is the scope of our proposal to cover only an index to etext documents of one
>kind or another, or to identify the source of data and the email way to
>contact that site/person?

I don't see that you are referring to any specific proposal, and neither
do I think that we have one yet...  There is no reason to define an
"upper limit" as to the quality or the scope of the various search
mechanisms in the GenWeb.  Since we must live with the fact that not all
information will become available on-line within a limited period of
time, we'll have to cope with partial data at many levels.  If all you
have on-line is a short index, then let's try to make the best possible
out of that, while at the same time not discouraging contributors with
more complete information from providing any index they like.

Identifying the source of each piece of information is an important
issue in itself, and not necessarily related to the search/indexing
issue.
--
Anders Andersson, Dept. of Computer Systems, Uppsala University
Paper Mail: Box 325, S-751 05 UPPSALA, Sweden
Phone: +46 18 183170   EMail: andersa@DoCS.UU.SE



          Monday, July 3, 1995 6:21:47 PM
          GenWeb Item
  From:           Anders Andersson,andersa@Mizar.DoCS.UU.SE,Internet
  Subject:        Re: Indexes of GenWeb Databases
  To:             GenWeb
John R writes:
>Is the scope of our proposal to cover only an index to etext documents of one
>kind or another, or to identify the source of data and the email way to
>contact that site/person?

I don't see that you are referring to any specific proposal, and neither
do I think that we have one yet...  There is no reason to define an
"upper limit" as to the quality or the scope of the various search
mechanisms in the GenWeb.  Since we must live with the fact that not all
information will become available on-line within a limited period of
time, we'll have to cope with partial data at many levels.  If all you
have on-line is a short index, then let's try to make the best possible
out of that, while at the same time not discouraging contributors with
more complete information from providing any index they like.

Identifying the source of each piece of information is an important
issue in itself, and not necessarily related to the search/indexing
issue.
--
Anders Andersson, Dept. of Computer Systems, Uppsala University
Paper Mail: Box 325, S-751 05 UPPSALA, Sweden
Phone: +46 18 183170   EMail: andersa@DoCS.UU.SE



          Monday, July 3, 1995 6:21:51 PM
          GenWeb Item
  From:           Tim Doyle,tdoyle@netcom.com,Internet
  Subject:        Re: Indexes of GenWeb Databases
  To:             GenWeb
On Fri, 30 Jun 1995, Bill Minnick wrote:

> Gene,
>   Since your discussion deals with an interim solution, have you checked out
> Tim Doyle's Home Page?  He's doing a prety good job of indexing the data
> bases right now.  I think he's already automated the process, at least
> partially.

At one time I had manually indexed about five of the GenWeb databases 
manually just to show that it could be done.  This index was available 
from the GenWeb Database Index page up until about a week ago, when I was 
notified that one of the databases I had used for the index had been 
regenerated.  I have now removed this index.

Some of the problems that I ran across when doing this index were:

1. Different formats. Some databases use one HTML per person, some 
combine many people on an HTML.  Some databases are more text or booklike 
in nature.

2. Lack of an index for all databases.  Some databases include their own 
person index, but many do not.  Grabbing one of these index files and 
using it in a master index would be fairly easy, but not if the database 
doesn't have one itself.

3. Index Formats not standardized.  Some of the database indexes are in 
the format of 'Last, First', some are 'LAST, First', some are 'First 
LAST', some are 'First Last', etc.  Some indexes include date ranges, 
some do not.  Some indexes use an unnumbered list <UL> which means that 
each person is preceeded with a <LI> tag, but many do not.

4. Even if all of the databases were in a standardized format, the sheer 
numbers of people would soon overwhelm an indexing program if it were not 
written to handle such a large amount of information.

I have been thinking of a couple of ideas:

1. Have each database provide a master name index in a common format.  
When a database comes online, an e-mail message listing the pertinent 
information (person's name, database name, URL of index file, etc.) would 
initiate a program which would grab the master index and add the items to 
the master index.  The same type of message would be sent if the database 
were updated.  This puts some responsibilty upon the person making the 
database available.

2. I have been looking into the use of Harvest.  I'm not quite sure if it 
will work or not yet, but I'm still looking.  One of the nice things that 
it supposedly is able to do is allow multiple index sites to communicate 
with each other and share the index.  In this way, the indexing would be 
shared among various sites, but each site would have the same 
information.  This is just what we have been looking for - if one site 
goes down, others would still be available.  

The Harvest system needs a certain type of operating system to operate 
on, and my current operating system (Linux) is on the list of 'being 
worked on'.

My partners and I have just purchased a Silicon Graphics 133 MHz machine. 
The Silicon Graphics machines are being used by some of the most heavily 
accessed sites on the web, including Yahoo.  Hopefully this system would 
be able to handle the GenWeb traffic <g>.

> His URL is:
> 
> http://www.doit.com/tdoyle/genweb/genweb.html

actually, the following is just as valid (and shorter):

  http://www.doit.com/tdoyle/genweb/

I hope that I didn't ramble too much in this letter.  Please let me know 
what you think about all of this.

Tim Doyle  / tdoyle@netcom.com / tdoyle@doit.com

WWW homepage:   http://www.doit.com/tdoyle/
ftp directory:  doit.com   pub/tdoyle




          Monday, July 3, 1995 6:24:26 PM
          GenWeb Item
  From:           Gene Stark,gene@starkhome.cs.sunysb.edu,Internet
  Subject:        Re: Indexes of GenWeb Databases
  To:             GenWeb
>  Since your discussion deals with an interim solution, have you checked out
>Tim Doyle's Home Page?  He's doing a prety good job of indexing the data
>bases right now.  I think he's already automated the process, at least
>partially.  His URL is:

I am familiar with his page.  He has a list of links to the GenWeb databases.
He used to have some sort of index of contents of a few of the databases,
but that seems to have been deleted now.  I don't know how he made the
indexes that used to be there, but without having a standard format for
the data I am certain that it would require a good deal of manual intervention.

							- Gene Stark



          Monday, July 3, 1995 6:26:15 PM
          GenWeb Item
  From:           Bill Minnick,svpafug@rahul.net,Internet
  Subject:        Re: Indexes of GenWeb Databases
  To:             GenWeb
Gene Stark writes:


>My proposal was geared toward the idea of having GenWeb evolve from something
>we can do right now, to the idealized thing we would all like it to be in
>the future. 
>
> We already have a lot of people with data online, but to find
>out what's there you have to contact each individual site and peruse their
>indexes, which are in many different formats.  A standard index file would
>make it possible for an automatic program that has been primed with a list
>of GenWeb sites to maintain an up-to-date index of all individuals in any
>currently accessible GenWeb database.  This is easily doable right now.
>All we need is to agree on a standard for the index files, and perhaps to
>get started entering database names in the genweb.org domain.

Gene,
  Since your discussion deals with an interim solution, have you checked out
Tim Doyle's Home Page?  He's doing a prety good job of indexing the data
bases right now.  I think he's already automated the process, at least
partially.  His URL is:

http://www.doit.com/tdoyle/genweb/genweb.html

Regards,

Bill Minnick




          Monday, July 3, 1995 6:27:37 PM
          GenWeb Item
  From:           Gene Stark,gene@starkhome.cs.sunysb.edu,Internet
  Subject:        Re: Indexes of GenWeb Databases
  To:             GenWeb
>This is probably a good idea....but should there also be a "place"
>included for birth and death?  Geographical limitations are helpful
>in narrowing things down.

Yes, I agree, the PLAC substructures of birth and death structures should
probably be included.
								- Gene



          Monday, July 3, 1995 6:27:40 PM
          GenWeb Item
  From:           Bill Minnick,svpafug@rahul.net,Internet
  Subject:        Re: Indexes of GenWeb Databases
  To:             GenWeb
Gene Stark wrote:

>I've been thinking off and on about what should be done in terms of a standard
>for GenWeb indexes . . . . 

Gene,
  Why do you assume that the centuries-old concept of a limited information
separate index is the way to go for GENWEB?  In my opinion, the process of
finding genealogy records on the Web should allow us researchers to make
complex queries that state what facts we know of about a person, and let the
search process sift through the GENWEB information base for us - that is,
taking full advantage of the computer power we have at our disposal.
  To me this requires us to put our energies into tools that effectively
index every word in the GENWEB databases and allow sophisticated "fuzzy"
searches, date range searches, "aka" person and place name substitutions, etc. 
  How much longer do we have to put up with the inefficiencies and forced
manual labor imposed by a "book-like" index ???  Lets set a higher level of
expectation for GENWEB.  
  I've been hoping one of us will investigate the HARVEST software from the
University of Colorado , which, I believe, can be modified to accommodate
most, or at least some of our needs. Another Genwebber suggested this
possibility 3 months ago.  I know of commercial document indexing software
systems (they call their function "automated archiving") which sound like a
perfect solution, except for price.  One such product is LaserFiche (by
Compulink in Torance, CA); they have at least one competitor in Italy.  
  Perhaps we should see if one of these Automated Archiving companies could
be persuaded to allow free use of their product on GENWEB - as a showcase to
attract buyers for commercial applications.
  Let's see if the tools are out there that will give GENWEB a clear-cut
edge over a library full of 19th century genealogy books.
  Gene, you may pursuade me that your proposed index is required as a 1-2
year interim help while we get our act together, but as for me, I'm eager to
see us exploit the Web for all it's worth!   
  As a researcher, I want to create complex, conditional queries (in a
simple, intuitive way), perhaps using a computer-aided form tool; but
potentially keying on any information that might be found on an individual
page.  Then I pass the query to a GENWEB Search Engine and (if I'm lucky)
get back a small list of URLs (of high potential value to me) in a Web page,
ready for me to to click and read the results.  If the results are good
enough, I'd even be willing to wait more than 10 seconds for them.  I want
the GENWEB Search Engine to find my material inspite of name spelling
differences, place name differences, etc.  
  I guess I'm saying I think "Searches" are the way to go; "Indexes" seem
useless to me if a good search engine is also available.  Five years from
now, when GENWEB has documented 100 million++ people, does anyone here
really want to still be sight-reading through an old fashioned index?

  Other than that Gene, thanks for fanning the fires of GENWEB discussion on
this topic.  

Regards,

Bill Minnick




          Monday, July 3, 1995 6:27:46 PM
          GenWeb Item
  From:           mavrogeorge@genealogysf.com,Internet
  Subject:        Re: Indexes of GenWeb Databases
  To:             GenWeb

<---- Begin Included Message ---->
From: Gene Stark <gene@starkhome.cs.sunysb.edu>
One person (I forgot who)
was proposing the use of Tiny Tafel as the index format.  

I'm not in favor
of this as I don't think this is going to be flexible 

enough for what we might
want to do with it in the future.
<---- End Included MEssage ----->

I was the one suggesting the Tiny-Tafel model. I do -not- 

suggest the adoption
of the tiny-tafel "as is". My thinking was that there 

needed to be a place for people to start their search and 

also that the task of keeping that site updated might be 

horrendous.

So I was proposing a two-tier index. That the central site 

contain the surname and the date ranges for that surname 

perhaps locations and a link to the site that has that 

information.  Maybe something like:

Bonner 1660-1810 PA-OH-IN <A link to genealogysf>
Bonner 1670-1750 NC <A link to helms toolbox>
Bonner 1820-1880 IA <A link to whereever>

This top level index would be built automatically from 

submissions from the link sites. E.G. when I decided to 

put my genealogy online I would build this highlevel index 

for my site and then send it to the central index site. 

That site would then add/update the master index of all 

sites. 

Once the person clicked on the link to one of the sites, 

they would then go to the index of individuals at that 

site.

This put the burden for currency of the information with 

the participating site not on the central site. It also 

lessens the burden of maintenance of the central site 

because that site doesn't maintain individual names but 

surnames and the associated date ranges and geographic 

areas.

--
This is based on the tiny-tafel model. Take a tiny-tafel 

and instead of the submitter info being that of a person 

make it a GenWeb site. Submit that tiny-tafel to a central 

Tafel Matching System which archives it and adds is to its 

surname index that same way that the Tafel Matching System 

does now. Then when someone searches the Tafel Matching 

System for a surname and date and/or location match, what 

is returned is the surname line information and the 

submitters info (which in this case would be a link to the 

GenWeb site).

What the central site could do is use a crawler to verify 

that a site was still reachable and if it was not 

automatically delete that site's tt from the systemm and 

rebuild the index.

--

Once again I sort of hesitated to mention the tiny-tafel 

because I didn't want people to get hung up on that 

particular format but instead to think of the Tafel 

Matching System/ Tiny-Tafel concept. And the benefits of 

putting the burden of updating with the individual site 

and the advantage of a two-tier indexing scheme.
 



          Tuesday, July 4, 1995 8:01:49 AM
          GenWeb Item
  From:           mavrogeorge@genealogysf.com,Internet
  Subject:        Re: GENWEB OUTLINE
  To:             GenWeb

<---- Begin Included Message ---->
From: alfred.sanford@chrysalis.org
I don't understand "the GENWEB outline I proposed."  
<---- End Included Message ---->

well my collection of past GenWeb messages isn't complete but at one 

point there was discussion of how to proceed now that there were several 

sites and many topics had been discussed. You had developed an outline of 

the issues/topics that needed to be discussed and decided. I rer a  least 

a few messages about the inclusion of software engineering or 

architecture etc etc. I believe it was you who said you were going to 

take that outline and, from the archives of GenWeb messages, fill in what 

had been discussed and then send it to Gary. The intent was that we would 

then e a roadmap of what we  needed to do and a recap of what we had 

already reached consensus on.

Does that sound familiar. Again, I was disconnected from GenWeb for some 

time so excuse me if I'm going over old business. 



          Tuesday, July 4, 1995 8:33:18 AM
          GenWeb Item
  From:           mavrogeorge@genealogysf.com,Internet
  Subject:        Another GenWeb database
  To:             GenWeb
I have added the Bonner/Rea database using Gene's new version of his 

program. It
was very easy to use and fun to change the parms around and generate the 

indexes in different ways. 

So the Mavrogeorge/Paspalas is an example of extracting from R4 to 

GEDCOM, passing it through the old G2HWIN.EXE, and then using PEDIGREE to 

build the charts.

The Bonner/Rea is an example of extracting from R4 to GEDCOM, and passing 

it through the new G2HWIN.EXE with the options of i (index), w 0 (index 

width of zero - everyname index), and -f %s.html (so the links would be 

to .html not to .htm). 
Still needed to use mymove to rename the files once I got them over to 

the UNIX box.

A couple of observations. 

I did not do a hierarchical index because it seemed to
me that the user would be confused and that an individual index would be 

better. But it resulted in a 93K index.html and that certainly is not 

user-friendly! I sent along an idea to Gene to have the surname.html be 

the entry point and then the nexevel would be all individuals for that 

surname and the third level would be the individual html page (or 

bookmark on the surname page).  

This method is not at all hard nor particularly time-consuming. Given 

that m updates to this GenWeb database will not be frequent, the GEDCOM 

to HTML seems perfectably acceptable. And that sort of seems to me to be 

the core of the G2H/CGI question. If the structure is not subject to 

frequent change what is the penalty for building indexes? and conversely 

what is the benefit of building the structure ad hoc when a query is made 

by a caller?

Is it more an apparent problem than a real one to pre-build indexes? What 

am I 
missing here in the discussion? 



          Tuesday, July 4, 1995 12:20:00 PM
          GenWeb Item
  From:           mavrogeorge@genealogysf.com,Internet
  Subject:        Re: Another GenWeb database
  To:             GenWeb

<---- Begin Included Message ---->
Gene  Stark responding to Brian Mavrogeorge
>Still needed to use mymove to rename the files once I got them over to 
>the UNIX box.
Why did you have to do this?  It should be that all the files and links
would be to ".htm" and everything should work OK without renaming (except 

for
the bug above). -Gene
<---- End Included Message ---->

Well my first attempt was to use the default which is to create files 

with the htm extension for DOS and to make the internal links .htm. But 

when I got the
files up to the UNIX system my browser, Internet in a Box (Mosaic based) 

would not recognize a file ending in .htm as an html file. It displayed 

the document source.

So I ran g2hwin changing the default so it created links with .html. 

Because of DOS even though g2hwin tried to create files with .html they 

came out as .htm. So I had internal links pointing to .html but actual 

files ending in .htm. Once I got
the files up to the UNIX system I simply ran mymove to rename them all to 

.html.

If my browser would have recognized an htm extension I wouldn't have had 

to do that but as I said your program operation is so easy and the 

documentation clear it was a minor inconvenience.




          Tuesday, July 4, 1995 1:30:05 PM
          GenWeb Item
  From:           Anders Andersson,andersa@Mizar.DoCS.UU.SE,Internet
  Subject:        Re: GENWEB OUTLINE
  To:             GenWeb
Brian Mavrogeorge writes:
><---- Begin Included Message ---->
>From: alfred.sanford@chrysalis.org
>I don't understand "the GENWEB outline I proposed."  
><---- End Included Message ---->
>
>well my collection of past GenWeb messages isn't complete but at one 
>point there was discussion of how to proceed now that there were several 
>sites and many topics had been discussed. You had developed an outline of 
>the issues/topics that needed to be discussed and decided. I rer a  least 
>a few messages about the inclusion of software engineering or 

[GenWeb project group: Project Management]

Ok, that's my proposal you are talking about.  I'm sorry that I haven't
maintained the discussion on that, but I have been a little short of
time, and I was hoping for additional input from other list members.
Maybe we can get the ball rolling again.

My proposal is still there, for everyone to accept or reject.  I haven't
modified it since our last mention of it on May 19.  I have now made it
available as <http://www.docs.uu.se/%7Eandersa/genweb/agenda.html> for
anyone's convenience.  Since I don't expect this agenda to change very
often once it has been adopted, I'll volunteer to remain as its editor.

Since I have seen no objections to my proposal, and some support for it,
I believe that we can agree on it.  If Gary Hoffman concludes that the
proposal has indeed been adopted, then I hope this will be indicated by
my agenda page being referred to from his de facto "official" GenWeb WWW
page at <http://demo.genweb.org/gene/genedemo.html>.  Gary, is this ok?

Once the agenda is taken, I'd like to link each topic to appropriate
outlines handled by other editors.  All such parts of the total outline
should ultimately be subject to discussion and consensus on the mailing
list, but I believe the editors should be free to add submitted material
at their own discretion, as long as the distinction between what has
been formally adopted and what hasn't is clearly visible (say, additional
proposals and differing opinions could be placed "below the <HR> line").
--
Anders Andersson, Dept. of Computer Systems, Uppsala University
Paper Mail: Box 325, S-751 05 UPPSALA, Sweden
Phone: +46 18 183170   EMail: andersa@DoCS.UU.SE



          Tuesday, July 4, 1995 5:45:02 PM
          GenWeb Item
  From:           Scott McGee,smcgee@microware.com,Internet
  Subject:        Re:  Another GenWeb database
  To:             GenWeb
With my own genweb site, I generate all reports at the time they are
requested, but the index is static. I can regenerate it at will by 
simply running a LifeLines report program I have for the purpose, but
prefer to keep the index prebuilt and just update it when I make changes
that will effect it.



          Tuesday, July 4, 1995 5:51:10 PM
          GenWeb Item
  From:           Scott McGee,smcgee@microware.com,Internet
  Subject:        Re:  Another GenWeb database
  To:             GenWeb
With my own genweb site, I generate all reports at the time they are
requested, but the index is static. I can regenerate it at will by 
simply running a LifeLines report program I have for the purpose, but
prefer to keep the index prebuilt and just update it when I make changes
that will effect it.

My current index is 280K and, in my opinion, too large. I do, however,
have an anchor at the first of each surname in the list, and a search
page to search the list, and also a list of about a dozen more common
names in my genweb front door html file.

Darn, I wish I could get my stuff on a site where it was visible to 
others. I was hoping by now that my work would have opened up our server
to outside access, but it could be another year the way things are
going!

Anyway, even for dynamically generated genweb sites like my own, I still
prefer a static index.

Scott

PS below is a bit from the front of my index.html.

[begin included file]

<HTML>
<HEAD>
<ISINDEX>
<TITLE> str </TITLE>
</HEAD>
<BODY>
<IMG SRC="http://www.microware.com:8080/~smcgee/pics/genweb.gif" ALT =
""><BR><BR>
<H1> INDEX </H1>
<UL>
<A NAME="ABNER">
<LI><A
HREF="http://www.microware.com:8080/~smcgee/cgi-bin/standardall/INDEX=I368?LookupInternal">
ABNER, Cecil</A> -
<LI><A
HREF="http://www.microware.com:8080/~smcgee/cgi-bin/standardall/INDEX=I366?LookupInternal">
ABNER, Clinton</A> -
<LI><A
HREF="http://www.microware.com:8080/~smcgee/cgi-bin/standardall/INDEX=I362?LookupInternal">
ABNER, Esther</A> - born 1902, USA
<LI><A
HREF="http://www.microware.com:8080/~smcgee/cgi-bin/standardall/INDEX=I367?LookupInternal">
ABNER, Hazel</A> -
<LI><A
HREF="http://www.microware.com:8080/~smcgee/cgi-bin/standardall/INDEX=I369?LookupInternal">
ABNER, Lowell Gene</A> -
<LI><A
HREF="http://www.microware.com:8080/~smcgee/cgi-bin/standardall/INDEX=I364?LookupInternal">
ABNER, Mary</A> - born USA
<LI><A
HREF="http://www.microware.com:8080/~smcgee/cgi-bin/standardall/INDEX=I363?LookupInternal">
ABNER, Roy</A> - born 1904, USA
<LI><A
HREF="http://www.microware.com:8080/~smcgee/cgi-bin/standardall/INDEX=I361?LookupInternal">
ABNER, Smith</A> - born 1881, USA died 1967, USA
<LI><A
HREF="http://www.microware.com:8080/~smcgee/cgi-bin/standardall/INDEX=I365?LookupInternal">
ABNER, Wilbur</A> -
<A NAME="ADAIR">
<LI><A
HREF="http://www.microware.com:8080/~smcgee/cgi-bin/standardall/INDEX=I286?LookupInternal">
ADAIR, Gerald Wayne</A> - born 1900
<A NAME="ADAMS">
<LI><A
HREF="http://www.microware.com:8080/~smcgee/cgi-bin/standardall/INDEX=I1591?LookupInternal">
ADAMS, Hannah</A> -
<LI><A
HREF="http://www.microware.com:8080/~smcgee/cgi-bin/standardall/INDEX=I1543?LookupInternal">
ADAMS, Hepzibah</A> -
<LI><A
HREF="http://www.microware.com:8080/~smcgee/cgi-bin/standardall/INDEX=I613?LookupInternal">
ADAMS, Mary</A> -
<LI><A
HREF="http://www.microware.com:8080/~smcgee/cgi-bin/standardall/INDEX=I1551?LookupInternal">
ADAMS, Thamerson</A> -
<A NAME="ADAMSEN">
<LI><A
HREF="http://www.microware.com:8080/~smcgee/cgi-bin/standardall/INDEX=I1929?LookupInternal">
ADAMSEN, Maren</A> -
<A NAME="AGENSTEIN">
<LI><A
HREF="http://www.microware.com:8080/~smcgee/cgi-bin/standardall/INDEX=I1955?LookupInternal">
AGENSTEIN, Jakob</A> -
<A NAME="AHLSTROM">
<LI><A
HREF="http://www.microware.com:8080/~smcgee/cgi-bin/standardall/INDEX=I1033?LookupInternal">
AHLSTROM, </A> - born 1803 died 1806



          Tuesday, July 4, 1995 5:55:33 PM
          GenWeb Item
  From:           Scott McGee,smcgee@microware.com,Internet
  Subject:        Re: GENWEB OUTLINE
  To:             GenWeb
Anders,
Let me add at least one consenting vote for adoption of your agenda.

Scott


          Tuesday, July 4, 1995 6:41:51 PM
          GenWeb Item
  From:           Phlete Teachout,fteachou@eagle1.eaglenet.com,Internet
  Subject:        Re: Another GenWeb database
  To:             GenWeb


On Tue, 4 Jul 1995 mavrogeorge@genealogysf.com wrote:

> Well my first attempt was to use the default which is to create files 
> with the htm extension for DOS and to make the internal links .htm. But 
> when I got the
> files up to the UNIX system my browser, Internet in a Box (Mosaic based) 
> would not recognize a file ending in .htm as an html file. It displayed 
> the document source.
> 
Brian, I don't believe it is your browser that is forcing the .html 
suffix - it is the client you are using at your web site.  Most UNIX 
based versions of the .http (and related) clients require the .html 
extension.  When the software got ported to DOS machines, the fourth 
letter needed to be dropped to accommodate the DOS operating system.  
There are a number of systems (with web pages) being operated from 
Windows NT systems - with the .htm extension for the web pages.
 
I'm not any sort of expert, by the way - I just went through much the 
same thing in getting my preliminary genealogy effort on my web page 
<http://www.eaglenet.com/fteachou/homepage.html>.  I am curious about 
this 'mymove' ?command? that was mentioned.  I used the SemWare editor to 
change all .htm references in the files to .html - then piped the 
directory of .htm files into a text file, removed the byte/creation date 
info, copied the remaining column of files to a column next to the first 
column, modified the last column extension to read .html, then stuffed an 
'mv' in front of each line.  My text file then looked something like:
 
mv S00001.htm S00001.html
mv S00002.htm S00002.html, etc.
 
Then I uploaded this text file into the Unix directory, made it 
executable, and ran the resulting shell script against the directory 
containing all the .htm files.  Worked like a champ.  It's a pain; but it 
beats typing 'mv file file' 30 times.
 
Regards,
 
				- fleet -




          Tuesday, July 4, 1995 8:48:48 PM
          GenWeb Item
  From:           Gene Stark,gene@starkhome.cs.sunysb.edu,Internet
  Subject:        A GenWeb Index Experiment (you can participate!)
  To:             GenWeb
SUMMARY:  An updated version of GED2HTML, available now, will produce a
new textual index file "GENDEX.txt" in addition to the HTML files.
I believe these textual index files can easily be collected automatically
and merged into a master GenWeb index, and I will produce software to do it.
You can participate by creating a GENDEX.txt file and sending me its URL.

DETAILS: I wasn't sure that I heard all that much agreement with my thoughts
on GenWeb indexes that I posted here a few days ago.  However, I am very
interested in moving GenWeb forward in this respect, so I decided to try
an experiment.  Here it is:

I had proposed the use of a limited-information GEDCOM as an index file
for use by automatic indexing programs.  I added code to my GED2HTML
program to generate such GEDCOMs, but I was disappointed by the size of
these files as compared with the size of the original input GEDCOM.
Typically, they seemed to be 30% to 50% of the size of the original,
which I felt was unacceptably large.  So, I commented out that code and
rewrote it to produce an index file with one line per individual,
in a format suitable for processing by the Unix tools "awk", "sort", etc.
Here is a sample of a few lines from some of my data:

D0001/I143.html|ALLES|Mary Ann /ALLES/|
D0001/I332.html|APPLETON|Ann  /APPLETON/|29 JUL 1721|New London,,Connecticut|22 
AUG 1806||
D0001/I387.html|APPLETON|Benjamin  /APPLETON/|18 JUL 1723||
D0001/I390.html|APPLETON|David  /APPLETON/|10 MAR 1730/1731||

Each line consists of several fields, separated by a "|" character.
Some of the fields may be empty.  The first field is a partial URL for the
actual data page, relative to the base directory of the database.
The second field is the surname of the individual.  The third field
is the full name of the individual as it would appear in a GEDCOM "INDI"
record.  The fourth field is the birth date of the individual.
The fifth field is the birth place of the individual.  The sixth and seventh
fields are the death date and death place, respectively.
A file in this format generally seems to be about half the size of the
GEDCOM version I originally tried.

I have updated the release of GED2HTML to version 2.3.  This version
will automatically produce a file "GENDEX.txt" in the above format along
with the other index files.  As a significant fraction of people with GenWeb
data are using my program to generate their HTML pages, after awhile there
should accumulate a number of databases with "GENDEX.txt" files.  I will
produce a program to automaticatically collect and merge these files into
a master index that can be accessed over the Web.  What I will need to
accomplish this is to obtain the  URL's people's "GENDEX.txt" files.
For the moment, I can get these manually, either by periodically scanning
through the databases in Tim Doyle's home page, or by E-mail, if those
people who have created databases with a GENDEX.txt file will send me its URL.

NOTE: Ideally, in the future I would like to be able to obtain these URL's
automatically, by doing a DNS zone transfer from some subdomain of GenWeb.ORG
to obtain information about all the current GenWeb hosts and databases.
Unfortunately, this will have to wait until we get some actual entries in
GenWeb.ORG, which I hope will be soon.

The purposes of the experiment are: (1) to get some sort of unified index
of GenWeb data up and running ASAP; (2) to try to get a sense of how useful
a simple index like the one I am proposing is and what other information
or organization might make it more useful; (3) to find out how long it will
take before a master index of this type exceeds a manageble size and a more
sophisticated scheme is required.

							- Gene Stark



          Tuesday, July 4, 1995 10:57:10 PM
          GenWeb Item
  From:           mavrogeorge@genealogysf.com,Internet
  Subject:        Re: A GenWeb Index Experiment (you can participate!)
  To:             GenWeb
Correct handling of SOUR is more important to me than centralized  

individual indexes <grin>.

Am glad you are doing it but am not convinced of the approach - 

individual indexing rather than a hierarchical one - and reliance on the 

central site harvesting the indexes.

and to once more put in a pitch... GenWeb's most often criticized aspect 

is the proliferation of bad data. The inclusion of source info in GenWeb 

is -very- important!

Having said that -- I will generate the index this week.


          Wednesday, July 5, 1995 5:43:24 AM
          GenWeb Item
  From:           Anders Andersson,andersa@Mizar.DoCS.UU.SE,Internet
  Subject:        Re: Another GenWeb database
  To:             GenWeb
Phlete Teachout writes:
>Brian, I don't believe it is your browser that is forcing the .html 
>suffix - it is the client you are using at your web site.  Most UNIX 
>based versions of the .http (and related) clients require the .html 
>extension.  When the software got ported to DOS machines, the fourth 
>letter needed to be dropped to accommodate the DOS operating system.  

[GenWeb project groups: User Interfaces, Resource Identifiers]

Phlete, your point is somewhat blurred by the fact that you seem to
suggest that there is a difference between Brian's "browser" and his
"client".  To me they are the same thing.  Maybe you mean "server" in
either of these cases?

According to my collection of GenWeb list messages, both Irwin Greenwald
and Brian Mavrogeorge have mentioned having to rename their files.
As you say, this may be needed due to restrictions on file name syntax
imposed by the operating system on the HTTP server side.  However, Unix
doesn't require that three-letter suffixes be converted to four-letter
ones.

It may be true that for very old clients, the filename suffix was the
only means of determining the type of a document.  However, the HTTP
protocol standard has since long supported a way of communicating the
document type via the Content-Type: parameter, independently of any
filename syntax conventions.  Unless your client is completely archaic,
it should trust this parameter for documents retrieved via HTTP, and
resort to filename heuristics only if that parameter is not available.

Since it's not clear to me how Brian accesses his documents (i.e. which
URL he uses), I can't determine where the fault is.  If he accesses
them locally via a "file:" URL, then no HTTP server is involved, and
the client must rely on filename heuristics.  However, I believe that
it's usually easier to reconfigure the client to understand a particular
filename suffix than to rename every file in the database.

It could also be that the server isn't configured to translate Brian's
".htm" suffix to the corresponding text/html document type, in which
case everybody trying to look at his pages would be affected.  Since
Brian's database at <http://www.sfo.com/%7Egenealogysf/genweb/> now
uses the ".html" suffix, I don't know how to test it.  Whether it's
easier to reconfigure the server than to rename the files depends on
how keen Brian's server maintainer is to support such requests.  Of the
two servers I have access to, CERN httpd and NCSA httpd (both running
under Unix), only the CERN implementation appears to understand ".htm"
in the default configuration (both can be configured to understand any
filename suffix you like).  Brian's server is NCSA...

In short, a database maintainer should be able to choose any filename
convention suitable at the server side, and if the HTTP server is
properly configured for that, the choice shouldn't matter to the
client users on the Internet.
--
Anders Andersson, Dept. of Computer Systems, Uppsala University
Paper Mail: Box 325, S-751 05 UPPSALA, Sweden
Phone: +46 18 183170   EMail: andersa@DoCS.UU.SE



          Wednesday, July 5, 1995 6:04:50 AM
          GenWeb Item
  From:           Phlete Teachout,fteachou@eagle1.eaglenet.com,Internet
  Subject:        Re: Another GenWeb database
  To:             GenWeb
I boo-boo'ed.  I meant to say the client httd software (the software 
installed on the service providers) that allows the web pages to be read 
by the http browsers.  Please note the final letters in httd and http.  
As I said, I'm not an expert - so I will now go looking for help and will 
reply later at more length (and, hopefully, with more (and more accurate) 
information).
 
				- fleet -



          Wednesday, July 5, 1995 2:41:36 PM
          GenWeb Item
  From:           Irwin Greenwald,irwin.greenwald@mogur.com,Internet
  Subject:        Re: Another GenWeb databa
  To:             GenWeb

On 07-05-95, ANDERSA@MIZAR.DOCS.UU.SE  wrote:


A>According to my collection of GenWeb list messages, both Irwin
A>Greenwald and Brian Mavrogeorge have mentioned having to rename their
A>files.

In my case, it was because version 2.0 of GED2HTML generated ".html" 
references.  With version 2.2 generating ".htm" references, the rename 
is not necessary. (Due to an oversight, references to index and surname 
files still have the "html" reference and the reference is to "SURNAMES" 
rather than "surnames", requiring that a minor edit be performed.)


Irwin

               Woodland Hills, CA   07/05/95 @ PM

               Internet: irwin.greenwald@mogur.com

---
 *CMPQwk 1.42-R2 #933



          Wednesday, July 5, 1995 7:02:57 PM
          GenWeb Item
  From:           Beau Sharbrough,beau@connect.net,Internet
  Subject:        FW: Tech Sessions Note
  To:             GenWeb

To interested persons:

I have collected a variety of email addresses of persons with 
an interest in genealogy.  It is not intended to be a complete
list.  However, in publicizing the Technical Sessions for GENTECH96,
all of you were included in the list of persons who I thought might
benefit from the information attached.

If, after reading the announcement, you still have questions about the
Technical Sessions, or you wish to volunteer to participate, or you'd
like a faxed application (they creatively use fonts that look almost 
as bad as my handwriting), please contact me.  Hell, contact me if you
don't, I like the sense of community it engenders.

Thanks in advance for your patience, the announcement follows this
non-commercial .sig:

--------------------------------------------------------
 Beau Sharbrough         The Rockets are back.
<beau@connect.net>       Break up the Rockets.
--------------------------------------------------------






GENTECH96
                                       4th Annual National Conference
                                  
             CALL FOR PRESENTATION IN TECHNICAL SESSIONS
                                  
GENTECH96 is proud to invite you to attend the Technical Sessions at our 4th
Annual Conference.  GENTECH is the Technology Conference for Genealogists.  The
Technical Sessions are the Genealogy Meetings for Technologists.  GENTECH96
Technical Sessions will be held in Plano, just north of Dallas, Texas, on
January 25 1996.

            THE GENEALOGY TECHNOLOGY SPEAKING ENGAGEMENT

As a lecturer at the GENTECH96 Technical Sessions, you will establish yourself
and your organization as leaders and innovators among the genealogy community.  
You'll share your ideas about open problems and current issues facing
technologists in this area.  You'll present your peers with your creative
approach to relevant issues.  

Another reason to participate is that addressing Technical Sessions entitles you 
to a reduced registration rate for the conference, as well as waiving the fee 
for attending the Technical Sessions.

The Selection Committee is composed of senior genealogy and technology
volunteers.  Presenters will be selected based on the technical merits of their
proposals and on their contributions to the knowledge base of genealogists
worldwide.  The deadline for informing us of your interest in speaking at these
sessions is August 15, 1995.

            THE REQUIRED BACKGROUND FOR YOUR PRESENTATION

Presenters at the Technical Sessions are asked to submit a short abstract
describing their presentation and its relevance to the genealogy and technology
community.  A suggested guideline is:  

A brief biographical sketch.
The title of your presentation.
Two to three sentence summary for the brochure.
Brief description of your organization, and problems that led you to this
solution.

Sessions are 20 minutes, followed by 10 minute question and answer periods. 
GENTECH will provide an overhead projector, LCD projection panel and screen.  If
you have other requirements, please contact us in advance.  The focus of GENTECH
Technical Sessions is the free exchange of information.  Presentations and
supporting documents must be free of commercialism.

Attach a copy of this form to your proposal.  Address it to:

General Chair:                          Program Chair:
John Wylie (Acting)                     Beau Sharbrough (Acting)
2818 Landershire Lane                   600 Six Flags Dr Ste 456
Garland TX 75044-5974                   Arlington TX  76011
(214) 495-4410                          (817) 633-4522
(FAX) 495-1569                          (FAX) 640-7533
Internet: jwylie@metronet.com           Cserve:     76057,2127
                                        Inet:       beau@connect.net

GENTECH96 Technical Sessions Call for Presentations Form

Name_____________________________________________________________________       
 
                                 
Title ______________________ Company ____________________________________

Street __________________________ City ___________________ ST/Zip _______

Day phone __________________ FAX ___________________ EMAIL ______________

Other information you think we need _____________________________________

_________________________________________________________________________

I acknowledge that I have my organization's approval to make the presentation
described, if applicable.  I have full legal rights to the materials I will
present, and agree that GENTECH INC may publish, audiotape, or videotape all or
part of my presentation.  I also agree to respect the commercial prohibition of
these sessions.

_____________________         _________________
Signature                     Date                            


                                  
August 15, 1995:         Abstracts due
November 15, 1995:       Presenter registration deadline
January 25-27:           GENTECH96 - BE THERE!





          Wednesday, July 5, 1995 7:09:33 PM
          GenWeb Item
  From:           Beau Sharbrough,beau@connect.net,Internet
  Subject:        FW: Press Release for Tech Comte
  To:             GenWeb

Interested parties will appreciate the enclosed press release.

Please replace all references to "Mr Sharbrough" with "that
hippie from GenTech" and you'll get a more accurate picture
of what's going on in the world.

Any reports you wish to make should go to Lee Hoffman or 
myself.  I'll just forward them to Lee.

--------------------------------------------------------
 Beau Sharbrough         The Rockets are back.
<beau@connect.net>       Break up the Rockets.
--------------------------------------------------------

˙PRESS RELEASE
June 15 1995 Dallas TX

GENTECH Inc Announces Data Exchange Project

At genealogy conferences such as GENTECH in Dallas, on electronic
bulletin boards, in the regular mail, and everywhere else that
genealogy computer users gather, one of the most widely heard
complaints is about the problem of data exchange.  As one genealogist
attempts to share information with another, the exchange of data,
called importing and exporting, is proving to be more difficult to do
than most people expect.

Most data is exchanged by converting it into a format called GEDCOM,
which is an acronym for Genealogy Data Communication.  This
specification is owned by the Church of Jesus Christ of the Latter Day
Saints, more commonly known as the LDS.  One of the difficulties of
GEDCOM file exchange is that there is some confusion among programmers
and users about how it works, and about which version is current.

The Technology Committee of GENTECH Inc has been authorized by the
board to pursue a data exchange project.  GENTECH is a non-profit
corporation devoted to bridging the gap between genealogy and
technology.  The Technology Committee chairman, Beau Sharbrough, has
been authorized to begin the first phase of a four phase project to
improve the quality of data exchange between genealogists.  The
original schedule for the project is 18 months.  The Project Leader is
Mr Lee Hoffman.

The first phase is a simple Problem Determination phase.  Between June
15 and August 31, Gentech is soliciting complaints.  Users and
developers are encouraged to write or email their concerns about
genealogy data exchange to the Technology Committee.  The goal of this
phase is to focus the attention of the participants on the issues
involved, and to articulate those issues for further discussion.  The
Problem List will be maintained by the committee.  On August 31, it
will be made available to the public.

The second phase is the Blue Sky phase.  Between September 1 and
November 30, the committee will solicit user and developer suggestions
for ways to deal with the issues identified in the first phase.  The
Suggestion List will be maintained by the committee, and will be
released on November 30.

The third phase is the Recommended Solution phase.  Between December
1, 1995 and April 30, 1996, the committee will evaluate the solutions
discussed in the second phase.  Discussion with developers, the LDS,
and interested users will focus on building a consensus about what
might be done to address the problems identified in the first phase. 
By April 30, 1996, a set of recommendations will be published by the
committee.

The fourth phase is the Implementation phase.  During this time, the
committee will produce a set of data to test compliance with the
recommendations developed in the third phase.  The group will also
communicate with users and developers to assist in compliance.

Mr Sharbrough is sure that the project will be a matter of discussion
at GENTECH96, the Technology Conference for Genealogists, to be held
January 26 & 27, 1996 at the Plano Convention Center.  The theme of
this, the 4th annual conference, will be the Genealogy Technology
Triangle: Modern Tools for Researchers, Societies, CIGS, Associations,
and Developers.  It will also likely be a topic of importance at the
Technical Sessions, to be held on January 25.  The Technical Sessions
are a series of presentations on open problems and current issues
facing technologists in the area of genealogy.

If you wish to participate in the project by email, you can contact Mr
Hoffman on CompuServe at 71147,1736.  You can also mail your inquiries
to Gentech at PO Box 28021, Dallas, TX 75228.  Send them to the
attention of the Technology Committee.


---------------------------------------------------------
 Beau Sharbrough         The Rockets are back.
<beau@connect.net>       Break up the Rockets.
---------------------------------------------------------




          Wednesday, July 5, 1995 8:07:34 PM
          GenWeb Item
  From:           mavrogeorge@genealogysf.com,Internet
  Subject:        html or htm
  To:             GenWeb
Thanks for the clue as to what to do. I passed Anders message on to the 

webmaster and he has fixed the problem. I can now use htm or html.


          Friday, July 7, 1995 12:06:55 AM
          GenWeb Item
  From:           jsanborn,jsanborn@gn2.getnet.com,Internet
  Subject:        Everton Publishers Online
  To:             GenWeb
Howdy,
        Found this message on CompuServe and thought it of interest to everyone.

> -------------------------------------------------
> FORWARDED MESSAGE - Orig: 06-Jul-95  15:09:57
> Subject: Internet is on-line - Msg Number #11095
>    From: Lee Everton 75162,220
>      To: All
>   Forum: GENSUP  Section: 07 - Everton Publishers
> -------------------------------------------------
>
> Everton Publishers, Inc. has completed the installation of a T1 line to run
>internet services.
>
> These services include our WWW pages that will point you to many locations
>around the world that deal with genealogy.  These include the National
>Archives, the Library of Congress, "Everton's Genealogical Helper: on-line
>edition", and personal web pages.
>
> The address for WWW is www.everton.com.
>
> You can also connect to On-Line Search via the internet.  Use the telnet
>capabilities of any internet provider to access our databases.  You can have
>limited access before you subscribe, but it should give you a good idea of
>what is available.
>
> Presently, there are over 67 million ancestor names on-line.
>
> The telnet address is: telnet bbs.everton.com
>
> Thanks - Lee



          Friday, July 7, 1995 10:23:34 AM
          GenWeb Item
  From:           Isaac E. King,iking@remus.ultranet.com,Internet
  Subject:        Re: Everton Publishers Online
  To:             GenWeb
Quick hop to everton.com for test run yielded naught! All links
report "no DSN".

Ike


          Friday, July 7, 1995 10:31:38 AM
          GenWeb Item
  From:           Scott McGee,smcgee@microware.com,Internet
  Subject:        new stuff in my genweb site
  To:             GenWeb

Ok, I bit the bullet and loaded a copy of my database into lifelines, and 
started unlinking people. I kept my grandparents. I kept my aunts and uncles
(only aunts by birth - kept the uncles by marriage as they effect the aunts
surname) but deleted all my cousins (except for one family as that cousin
is on the net and likely to find the data usefull too). I also kept my wife's
parents and siblings (but no spouses or children for the siblings) and also
my own siblings (I kept their spouses and children).

I ended up with 63 people in the database. Then wrote it out as a gedcom. I
used Gene Starks ged2html to produce html files with five people per file.
This is what is now availble in my genweb stuff. I also added a genweb.html
page that acts as an anchor to my genweb stuff, seperate from my home page.
This page is at the URL: http://www.cc.utah.edu/~sam8644/genweb/genweb.html
and will remain there as the base of my genweb stuff. I encourage anyone
interested to stop by and have a look.

Gene, a few things about the process I went through. First, when writting 
multiple persons to a single html file, your program does some nasty things.
for each person in the file, the program writes a <HTML><HEAD><TITLE> and
<BODY> tag (with apropriate data and closing tags). Some browsers don't like
the multiple tags at all, and the multiple <title> tags meant that the last
was normally used and I would have things like text for John Doe, but a title
for Jane Smith and silly things like that.

Another thing it did was to make empty links in the pedigree charts whereever
there was an empty entry. I edited these out, and removed the empty entries
too. In fact, if there are no ancestors in the gedcom, I deleted the pedigree
chart all together.

When the SURNAME file is created, why not create a link to the actual first
corresonding surname in the index and add anchors at the first entry for each
surname in the index page. I did this by hand and now if I select McGee in
the surname page, I go right to the start of the McGees in the index.

Also, I don't know about other people, but I prefer the NOTES: to be just
below the other personal data, and before the pedigree chart. I changed this
in the generated files.

I suspect that had I taken the time, I could have built a template that did
all these things for me, but didn't want to try learning your macro stuff
right now.

Anyway, thanks Gene, for the program. I have a rather strict limit on my
file count at my universty account and that had previously limited how many
people I dared to put online this way. With your program, I easly put up
five times as many people in roughly the same number of files. I may play
with trying even more people per file. I don't think I could ever fit the
full 2000 names in my database into a workable number of files without making
the files too large to be workable, but I may see how many of my direct 
ancestors I can get into the system.

Scott

  If at first, you don't succeed, |    smcgee@microware.com (Scott McGee)
  go fry a hen. After all, fried  | -----------------------------------------
  chicken beats failure any time. |  I was paid $5.00 to express these views!
  -------------> http://www.cc.utah.edu/~sam8644/homepage.html <-------------


          Saturday, July 8, 1995 8:00:02 PM
          GenWeb Item
  From:           T.T.Wetmore,ttw@beltway.att.com,Internet
  Subject:        Re:  new stuff in my genweb site
  To:             GenWeb
Scott writes:

>Ok, I bit the bullet and loaded a copy of my database into LifeLines, and 
>started unlinking people. I kept my grandparents. I kept my aunts and uncles
>(only aunts by birth - kept the uncles by marriage as they effect the aunts
>surname) but deleted all my cousins (except for one family as that cousin
>is on the net and likely to find the data usefull too). I also kept my wife's
>parents and siblings (but no spouses or children for the siblings) and also
>my own siblings (I kept their spouses and children).
>
>I ended up with 63 people in the database. Then wrote it out as a gedcom. I
>used Gene Starks ged2html to produce html files with five people per file.
>This is what is now availble in my genweb stuff. I also added a genweb.html
>page that acts as an anchor to my genweb stuff, seperate from my home page.
>This page is at the URL: http://www.cc.utah.edu/~sam8644/genweb/genweb.html
>and will remain there as the base of my genweb stuff. I encourage anyone
>interested to stop by and have a look.

For anyone's interest who's using LifeLines.  You can write LifeLines
programs to generate small GEDCOM files for small sets of persons, without
messing around with your database.  Just write a program that creates
whatever set of persons you want in the GEDCOM file and then run the
gengedcom() function on them.  You do not have to go into your database and
remove links in order to create short GEDCOM files with LifeLines.

Scott,
if you would send me the criteria you are using to select small sets of
persons, I will show you how to write a LifeLines program to generate
GEDCOM files for them.

In addition, with LifeLines available I wonder why use GED2HTML.  It is
straightforward to use LifeLines to build a small set of persons and then
generate an HTML file for them.  The major advantage that LifeLines has
over GED2HTML for this purpose is you have full control over the HTML.  I'm
sure Gene has done an excellent job with GED2HTML, but you still have to
live with his specific idea about what an HTML file for a group of people
should look like.  With LifeLines you can generate HTML that looks like what
GED2HTML generates, or like anything else of your own you might perfer.  If
you use special tags in your GEDCOM that you want to have handled specially
in the HTML form, you can get exactly what you want.  There is quite a wide
difference implicit here.  For LifeLines, GED2HTML is one, quite short,
report program.  GED2TEX, GED2MIF, GED2TROFF, GED2POSTSCRIPT,
GED2NEHGS-REGISTER, GED2PAF-GROUP-SHEET,
GED2REUNION-TWO-DIMENSIONAL-DROP-CHART, etc, etc, are just more LifeLines
programs.

Scott,
If you would send me an HTML file the way you would like it to look from a
small GEDCOM file, whether created directly by GED2HTML, or after you have
modified it to look the way you want, I will write you a short LifeLines
program to generate the same or similar HTML output.  You can then use that
program as the basis for whatever other changes you may want to apply
later.  There already are a number of LifeLines programs for generating
HTML files; if you put a request out on the LINES-L list you may be some of
their authors to respond with examples for you.

Tom Wetmore



          Saturday, July 8, 1995 8:35:05 PM
          GenWeb Item
  From:           Irwin Greenwald,irwin.greenwald@mogur.com,Internet
  Subject:        Re: Comments from Irwin G
  To:             GenWeb
CC: genweb@ucsd.edu          

CC: genweb@ucsd.edu          
    gene@StarkHome.cs.sunysb.edu

TO: genweb@ucsd.edu

* * * This message was from IRWIN GREENWALD to TDOYLE@JUST.DOIT.COM * * 
      * * * and has been forwarded to you by IRWIN GREENWALD * * *      
            -----------------------------------------------             
 
T>I can access these files, but my viewer (Netscape) does not recognize
T>the  .htm files as being HTML.  Have you had other such reports?

I just tried it using Netscape (rather than Netcruiser, my normal 
viewer) and you are correct!  This is weird because Netscape works fine 
when I access the same stuff on my localhost (DOS) system.  Why would 
they go to all the trouble of recognizing the system they're on (Unix vs
DOS) and insist on these naming conventions?

I just tried the GED2HTML option to produce html references (-f %s.html)
and this allowed Netscape to access the database OK.  However GED2HTML, 
even with this option, produces index references with an ".htm" 
extension, so we've still got problems.

I'll report the problem to Gene Stark, as a warning.

Irwin

               Internet: irwin.greenwald@mogur.com

---
 *CMPQwk 1.42-R2 #933



          Saturday, July 8, 1995 10:57:24 PM
          GenWeb Item
  From:           Scott McGee,smcgee@microware.com,Internet
  Subject:        Re:  new stuff in my genweb site
  To:             GenWeb
Tom, you caught me!

Actually, the reason I made a copy of my LL database, unlinked a bunch
of people, and then used Gene's ged2html on it was that it was an easier
and more brainless aproach than writting the LL program I really wanted.
(Gene's program was already there, I just had to type make)

What I really needed was a LL program that would ask for a person and put
them in a list, then, for each person in the list, ask if you want that 
person in the final results, if you do, it should add their parents and 
children to the list, and then put them in the output list. Continue this
untill the input list is empty. Then, ask for the number of persons per
html file, and start dumping these people into html files that many at a
time. It should also be able to generate an index files (hierarchical would
be a nice option) and a GENDEX.txt files for Gene's genweb indexing proposal.

If the program were able to prompt for the input set as described above, 
generate a flat index, and output html files like ll2html, but with a given
number of individuals per file, I would consider it a heaven sent gift.
I could hack that to do the GENDEX.txt or write a program that do it from 
the INDEX file.

Basically, I was too lazy to write the above (I could do it, though it 
would push my LL programing a bit).

Scott

  When in danger,    |   If it has my name on it, it must be MY opinion!
  or in doubt,       |______________________________________________________
  run in circles,    | Email: smcgee@microware.com (Scott McGee)
  scream and shout!  | Web:   http://www.cc.utah.edu/~sam8644/homepage.html



          Wednesday, July 12, 1995 8:44:35 AM
          GenWeb Item
  From:           Gary Hoffman,ghoffman@ucsd.edu,Internet
  Subject:        GenWeb Parking Places
  To:             GenWeb
The GenWeb project has accelerated an interest in electronic publishing
among genealogists who are net-connected. Besides the GenWeb-type data
discussed in my proposal documents
(http://demo.genweb.org/gene/genedemo.html), there appears to be a need to
publish family histories, regional histories, and even electronic versions
of source documents.

Recently several people have contacted me to inquire where they can publish
their documents on the Net. Such documents are usually in HTML, PostScript
or word processor format, or even in a portable document format. They can
be published using either ftp or http protocols.

I don't have an answer for them. I myself am borrowing space from my
employer's computers to publish the GenWeb proposal. Does anyone have space
available to publish such documents? 

If so, please post a notice to me or to the mailing list genweb@ucsd.edu.

Thanks,
Gary

***************************************************************************
*Gary B. Hoffman, Computer/Language Lab Director e-mail: ghoffman@ucsd.edu*
*Graduate School of International Relations and Pacific Studies    (IR/PS)*
*University of California, San Diego (UCSD)          voice: (619) 534-7733*
*9500 Gilman Dr., La Jolla, CA 92093-0519 USA          fax: (619) 534-5727*
***************************************************************************



-
          Wednesday, July 12, 1995 10:48:18 AM
          GenWeb Item
  From:           Scott McGee,smcgee@microware.com,Internet
  Subject:        Re:  GenWeb Parking Places
  To:             GenWeb
Gary,

There are a nujber of freenets around that provide free accounts with web
access. It is quite likely that someone wishing to publish material on the
web (of a non-comercial nature) could use such accounts to make it availible.

Another possibility is to contact the people who maintain sites and ask them
about the possiblity of donating file space for such things.

Scott

  If at first, you don't succeed, |    smcgee@microware.com (Scott McGee)
  go fry a hen. After all, fried  | -----------------------------------------
  chicken beats failure any time. |  I was paid $5.00 to express these views!
  -------------> http://www.cc.utah.edu/~sam8644/homepage.html <-------------



          Wednesday, July 12, 1995 2:01:56 PM
          GenWeb Item
  From:           peterson@warp.msoe.edu,Internet
  Subject:        Re:  GenWeb Parking Places
  To:             GenWeb
Gary-

Another possibility (not free, unfortunately) is to locate an 
Internet Service Provider who includes storage space for your web 
page along with your access account.  For example, there are at 
least two providers here in Milwaukee who offer 2 meg of space 
for your web page with a $12/month Internet access fee, and 4 
meg with a $20/month unlimited access account.

Kent Peterson

 


***************************************************************
Kent A. Peterson          Director of University Media Services       
PROFNET Telecourse Network      Milwaukee School of Engineering          
eMail: peterson@warp.msoe.edu            Voice:  (414) 277-7176           
WWW Page:  http://www.msoe.edu/            Fax:  (414) 277-7440
***************************************************************
  


          Wednesday, July 12, 1995 5:05:26 PM
          GenWeb Item
  From:           Anders Andersson,andersa@Mizar.DoCS.UU.SE,Internet
  Subject:        HTTP Content-type information
  To:             GenWeb
[GenWeb project groups: User Interfaces, Resource Identifiers]

This may be of use to the rest of the list.  Irwin Greenwald gave
me the following URL to check:

>        http://genealogy.emcee.com/pub/users/irwin/geilich/index.htm

Well, when requesting that particular document, the HTTP header from
the server at emcee.com looks as follows:

HTTP/1.0 200 OK
Date: Wednesday, 12-Jul-95 23:40:08 GMT
Server: NCSA/1.3
MIME-version: 1.0
Content-type: text/plain
Last-modified: Monday, 10-Jul-95 01:17:54 GMT
Content-length: 21527

... after which comes the HTML code.  Adding an "l" to the URL (I guessed
it existed), I received the same document but with "text/html" in the
HTTP header instead of "text/plain".  This is the key to the problem;
the server (incorrectly, I assume) interprets the "htm" extension as
meaning plain text, and normal clients then have no reason to ignore
the server's opinion.

Since your name is in the URL, I assume you are responsible for these
pages, right?  I suggest that you talk to your server manager and ask
if he would like to modify the configuration to have the server accept
"htm" as well as "html" extensions as indications of HTML documents.
If he doesn't want to do that, your remaining alternative will be to
consistently name all your files to suit the server configuration.
Asking clients `at the other end of the wire' to disregard HTTP header
info and start employing filename syntax heuristics is not a reasonable
long-term solution, now that there is a fully functional HTTP standard.

If your server manager allows you to provide your documents via a CGI
program or similar, then you have almost full control over the HTTP
header and can supply any document type you like yourself, but I guess
from your URL that this is currently not an option for you.
--
Anders Andersson, Dept. of Computer Systems, Uppsala University
Paper Mail: Box 325, S-751 05 UPPSALA, Sweden
Phone: +46 18 183170   EMail: andersa@DoCS.UU.SE



          Wednesday, July 12, 1995 5:34:13 PM
          GenWeb Item
  From:           Irwin Greenwald,irwin.g@ix.netcom.com,Internet
  Subject:        Fwd: Re: Problem
  To:             GenWeb
---- Begin Forwarded Message
Return-Path: <emcee@genealogy.emcee.com>
Received: from genealogy.emcee.com by ix3.ix.netcom.com 
(8.6.12/SMI-4.1/Netcom)
	id PAA27235; Wed, 12 Jul 1995 15:33:38 -0700
Date: Wed, 12 Jul 1995 15:53:21 -0700 (PDT)
From: Genealogy Online <emcee@genealogy.org>
X-Sender: emcee@emcee.com
To: Irwin Greenwald <irwin.g@ix.netcom.com>
Subject: Re: Problem
In-Reply-To: <199507121737.KAA10789@ix3.ix.netcom.com>
Message-ID: <Pine.LNX.3.91.950712154842.22837B-100000@emcee.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII


Hi Irwin,

	I've just started a new job and I'm still in the process of 
moving into a new home.  I've been too busy to take a look at this, but 

will soon. I don't know that Gene is talking about any one server.  
That 
could be the situation with any server.

	I knew that Lynx wouldn't read .htm files so have been advising 
everyone to change their file names to .html.  I didn't know, but will 
trust Gene's comments, that it could be configured in the server. I'll 
take a look at it.

Thanks,
Michael

On Wed, 12 Jul 1995, Irwin Greenwald wrote:

> Because I have received no reply to the following, I'm guessing that 
> you never received it, so I'm sending it over another route.  Please 
> let me know when you receive it.
> 
> Irwin
> 
> 
> Date: 07-10-95                         Msg # 261
>   To: emcee@genealogy.emcee.com        Conf: (261) 
> From: IRWIN GREENWALD                  Stat: Private
> Subj: Re: Problems                     
> 
------------------------------------------------------------------------

> 
> TO: emcee@genealogy.emcee.com
> 
> * * * This message was from GENE@STARKHOME.CS.SUNYSB. to IRWIN 
> GREENWALD
>       * * * and has been forwarded to you by IRWIN GREENWALD * * *    
  
> 
>             -----------------------------------------------           
  
> 
> Michael:
> 
> The problem being discussed is that Netscape doesn't recognize ".htm" 

> files on emcee.com.  Do you agree with Gene Stark's analysis? Is 
emcee 
> the server he is talking about or is it Netcom (my service provider0?
> 
> Irwin
> 
> From: Gene Stark <gene@starkhome.cs.sunysb.edu>
> To: mogur.com!irwin.greenwald@sbstark.cs.sunysb.edu
> Date: Sun, 9 Jul 1995 17:51:42 -0400
> Subject: Re: Problems
>  
> >M>T>I can access these files, but my viewer (Netscape) does not
> >M>T>recognize the  .htm files as being HTML.  Have you had other 
such
> >M>T>reports? 
>  
> The problem you report is basically a misconfiguration of the Web 
> server
> by your service provider.  The HTTP daemon that is running to deliver 

> up
> the Web files provides file type information deduced from a 
collection 
> o
> mappings of filename suffixes to MIME content types.  There should be 
a
> mapping taking ".htm" to "HTML file".  The CERN HTTPD program has 
this 
> b
> default.  The other major HTTP server, by NCSA, requires that the 
> servic
> provider add the proper mapping to the "mime.types" file in the 
server
> configuration directory.
>  
> The problem can also be worked around in your browser, which also has
> its own mappings.  Changing these helps you view files delivered by a
> misconfigured server, but does not help other people in viewing the 
> same
> files.
>  
> In summary, you need to ask your service provider to kindly add the 
> prop
> mapping.
>  
> 


--
info@genealogy.emcee.com	    General info about Genealogy Online
1880@genealogy.emcee.com	    Info about the 1% 1880 census
autoreply-info@genealogy.emcee.com  Autoreply account information
paf-review@genealogy.emcee.com      Info about the PAF Review and 
archives
rsl-info@genealogy.emcee.com        Info about the Roots Surname List
tt-info@genealogy.emcee.com	    Info about the Tiny Tafel server
geonames@genealogy.emcee.com        Info about the GeoNames server
server-info@genealogy.emcee.com     Email server instructions and 
commands
calendar@genealogy.emcee.com        Upcoming genealogy events

------------------------------------------------------------------------
--
 Michael Cooley, System Administrator                     
emcee@emcee.com
 Genealogy Online   *   http://emcee.com/   *    
info@genealogy.emcee.com
------------------------------------------------------------------------
--





------ Message Header Follows ------
Received: from ix4.ix.netcom.com (ix4.ix.netcom.com [199.182.120.7]) by UCSD.EDU
(8.6.12/8.6.9) with ESMTP id RAA11573 for <genweb@ucsd.edu>; Wed, 12 Jul 1995
17:57:03 -0700
Received: from  by ix4.ix.netcom.com (8.6.12/SMI-4.1/Netcom)
	id RAA11370; Wed, 12 Jul 1995 17:29:27 -0700
Date: Wed, 12 Jul 1995 17:29:27 -0700
Message-Id: <199507130029.RAA11370@ix4.ix.netcom.com>
From: irwin.g@ix.netcom.com (Irwin Greenwald)
Subject: Fwd: Re: Problem
To: genweb@ucsd.edu