2010-01-22 Some GEDCOM Torture Test Results

Three Torture Tests

I recently published Three Torture Tests, three extreme GEDCOM files inspired by genealogical records such as most married, most children and longest name.

test file	brief description
Married1200.ged	Much Married Man
Children1200.ged	Many Children
Long26CC.ged	Long Names 26 - ConCatenated Lines
Long26LL.ged	Long Names 26 - Long Lines

The above table summarises the three test. The fourth entry in the table is a bonus test, a Long Lines variant of Long26; the longest lines are so long that it is not a valid GEDCOM file.

character encodings

The tests below ignore character encoding issues. Several applications write nothing but so-called ANSI GEDCOM, which are not valid GEDCOM files. In the tests below this aspect is ignored.
So, this text may conclude that an application imports and then exports a GEDCOM file correctly, when in actual fact the application does not support real GEDCOM at all. Caveat emptor.

Tests

Daub Ages! 1.53

test file	result
Married1200.ged	Imports fine, navigates fine, exports fine.
Children1200.ged	Imports fine, navigates fine, exports fine.
Long26CC.ged	Ages! does not recognise the `CONC` lines, but truncates the names instead.
Long26LL.ged	Ages! imports most names correctly, but without warning about the long lines, and on export of the long names, Ages! fails to use `CONC` and thus produces invalid GEDCOM files.

The maximum name length supported by Daub Ages! appears to be 243 characters; the names that are longer than that (starting with Y and Z) are truncated. There is no error or warning, and Daub Ages! does not produce an import log.

Daub Ages! fails the long name tests.

Ahnenblatt 2.62

test file	result
Married1200.ged	Imports fine, navigates so-so, exports fine (export individuals in alphabetical order, but preserves marriage order).
Children1200.ged	Appears to hang on import. Terminated after five minutes.
Long26CC.ged	Ahnenblatt complains about the `CONC` lines, complains and truncates the names.
Long26LL.ged	Ahnenblatt imports most names correctly, but without warning about the long lines, and on export of the long names, Ahnenblatt fails to use `CONC` and thus produces invalid GEDCOM files.

After importing Married1200.ged Ahnenblatt defaults to showing wife Eighty Four. The Next Spouse button below the couple is woefully inadequate for navigating to say wife Nine Hundred. Ahnenblatt has a Go to... menu item that presents a dialog box with a list to choose from, which is good enough if you know the name of the wife you want to navigate to. A problem with that dialog is that it is automatically sized to be just too small for the longest name.

Ahnenblatt completes the Married1200.ged test successfully, but fails all the other tests.

Aldfaer 4 (4.2.0.7)

test file	result
Married1200.ged	Import messes up database by inserting comments, navigation is very slow, export includes the comments.
Children1200.ged	Import messes up database by inserting comments, navigation is okay, export includes the comments.
Long26CC.ged	Aldfaer imports the `CONC` lines correctly, and does use `CONC` upon export to avoid long lines. The import messes up the database by inserting comments, and the export includes these comments.
Long26LL.ged	Aldfaer imports the long lines without warning, but does use `CONC` upon export to avoid long lines. The import messes up the database by inserting comments, and the export includes these comments.

Although the SURN tag for the husband in Married1200.ged is an INDI.NAME.SURN tag to be precise, Aldfaer reports 22 GEDCOM-tag INDI::SURN wordt niet ondersteund (is not supported).
This reflects a defect in Aldfaer’s GEDCOM import: Aldfaer still does not support the GIVN and SURN tags on export, and still does not recognise the SURN tag on import. Instead, the Aldfaer GEDCOM import does the worst possible thing: it messes up your database by inserting the SURN line as a comment to the individual record.

Aldfaer does a few things right. Although Aldfaer is a dated design still based on an 8-bit code page, Aldfaer does support names longer than 256 characters. Aldfaer recognises the use of CONC for long names on import, and correctly uses it upon export.

However, Aldfaer also fails to issue warnings for long lines upon import, and messes up the database by inserting comments whenever it encounters a SURN tag.

Aldfaer fails all the tests.

Brother’s Keeper 6 (6.3.37)

test file	result
Married1200.ged	Error: 6 Overflow during import. App seems to hang and needs to be terminated.
Children1200.ged	Brother’s Keeper shows a messagebox: Gedcom line: 1 CHIL @I76@@ Family has over the limit of 75 children.
Long26CC.ged	Family Tree Builder does not process the `CONC` lines, does not issue any error or warning, but truncates the names instead.
Long26LL.ged	Brother’s Keeper fails to complain about the long lines, but does report that it truncates names, after which none are more than 99 characters long.

Brother’s Keeper seems to have a hard-coded limit of 999 marriages that it does not test for, a hard-coded limit of 75 children that it does test for and a hard-code limit of 99 characters per name that it enforces by truncating names that are longer.

Brother’s Keeper fails all the tests.

Behold Beta (0.99.4)

test file	result
Married1200.ged	Imports and mostly displays correctly; the table of contents is incomplete and lists only 92 marriages.
Children1200.ged	Imports and display correctly.
Long26CC.ged	Behold does not process the `CONC` lines correctly and does not issue any error or warning either, but truncates the names.
Long26LL.ged	Behold imports and displays the data correctly, but fails to complain about the long lines.

Behold passes the Children1200.ged tests, but fails all other tests.

Cognatio 1.4.3

test file	result
Married1200.ged	Cognatio fails to import the marriages.
Children1200.ged	Imports fine, navigates fine, exports fine.
Long26CC.ged	Cognatio does not recognise the `CONC` lines, issues erroneous Misplaced tag error, and truncates the names.
Long26LL.ged	Windows reports that Cognatio has stopped working.

Cognatio complains about the perfectly valid GEDCOM header by highlighting several tags as Misplaced tag. That does not build confidence.

Cognatio passes the Children1200.ged tests, but fails all the other tests.

Deudos 5 (5.1.274)

test file	result
Married1200.ged	Deudos imports and exports fine, but the navigation through the 1200 wives using but a forward and backward button is impractical.
Children1200.ged	Again, imports and exports fine, but awkwardly impractical navigation with nothing more than a see Other children button that shows the next few children.
Long26CC.ged	Deudos does not process the GEDCOM correctly, and shows truncated names.
Long26LL.ged	Deudos imports the names correctly, but without warning about the long lines, and on export of the long names, Deudos fails to use `CONC` and thus produces invalid GEDCOM files.

The GEDCOM files that Deudos writes do not have a Deudos GEDCOM header, but the same GEDCOM header as the test file Deudos imported. That is wrong.

Deudos does not really pass the Married1200.ged and Children1200.ged tests, and fails the long name tests. Because its GEDCOM headers are not correct, it fails all the tests.

Embla Family Treasures 8 (8.0.115)

test file	result
Married1200.ged	Family Treasures imports and exports the data, but its navigation is not up to the task; it only shows wife One through Two Hundred and One.
Children1200.ged	Family Treasures imports and exports the data, but fails to show the children on the mother’s person view. The family view does show them all.
Long26CC.ged	Family Treasures does not process the `CONC` lines, does not issue any error or warning, but truncates the names instead.
Long26LL.ged	Family Treasures imports the names correctly, but without warning about the long lines, and on export of the long names, Family Treasures fails to use `CONC` and thus produces invalid GEDCOM files.

Embla Family Treasures GEDCOM import is remarkably slow. It took minutes for each file.

Embla Family Treasures fails every test already, but there is more. In September of last year, Embla Family Treasures 8 GEDCOM noted that Family Treasures does not even write a correct GEDCOM header. This test was done with the latest version, but Embla still has not fixed that major defect.

Embla Family Treasures doubly fails all the tests.

Family Tree Builder 4 (4.0.0.893)

test file	result
Married1200.ged	Imports fine, navigates fine, exports fine. Charts are less than perfect, with lines missing, but does not crash.
Children1200.ged	Imports fine, navigates fine, exports fine. Charts are less than perfect, with overlapping children, but application does not crash.
Long26CC.ged	Family Tree Builder does not recognise the `CONC` lines, but truncates the names instead.
Long26LL.ged	Family Tree Builder imports the names correctly, but without warning about the long lines, and on export of the long names, FTB fails to use `CONC` and thus produces invalid GEDCOM files.

Family Tree Builder fails the long name tests.

Family Historian 4 (4.0.2)

test file	result
Married1200.ged	Imports and exports fine. Navigation is a bit odd, but works fine. The All Relatives diagram is displayed quickly and works fine.
Children1200.ged	Imports fine, navigates fine, and exports fine. Diagram display quickly and works fine.
Long26CC.ged	Family Historian does not understand the `CONC` lines, logs erroneous complaints, and truncates the names. Because the exported GEDCOM file does include the `CONC` lines, it is accidentally correct.
Long26LL.ged	Family Historian imports the names correctly, but without warning about the long lines, and on export of the long names, Family Historian fails to use `CONC` and thus produces invalid GEDCOM files.

Family Historian fails the long name tests.

Family Tree Maker 16 (16.0.350)

test file	result
Married1200.ged	Maximum is 99 marriages. Additional marriages are ignored.
Children1200.ged	Maximum is 99 children.
Long26CC.ged	Does not understand `CONC` used with `NAME`.
Long26LL.ged	Maximum is 40 characters; names truncated. Fails to complain about line length.

Family Tree Maker 16 does not crash and does produce clear import reports, but does not succeed importing any of the files.

That Family Tree Maker does not support names longer than 40 character did not hinder the Married1200.ged and Children1200.ged tests, as all the names in those files are shorter than that.

Family Tree Maker 16 fails all the tests.

Family Tree Maker 2010 (19.0.0.180)

test file	result
Married1200.ged	Remains Not Responding after import. Import log is empty. FTM 2010 can export to GEDCOM again, but it is practically impossible to edit the file. Trying to switch to the People view is enough to make FTM Not Responding again.
Children1200.ged	Import proceeds without problems. Try to switch from Family tab to Person tab and FTM becomes Not Responding again. FTM uses more than 100 MB of RAM. FTM 2010 can export the data to GEDCOM again, but it is practically impossible to work with the file.
Long26CC.ged	FTM imports the file without problems. It is possible to navigate and edit. FTM exports to GEDCOM, but exports long lines instead of using CONC.
Long26LL.ged	FTM imports the file without problems. The import log does not contain errors or warnings for the long lines. It is possible to navigate and edit. FTM exports to GEDCOM, but exports long lines instead of using CONC.

Family Tree Maker starts slow, responds slow. Often becomes and remains Not Responding for minutes on end, necessitating termination from Windows Task Manager.

Family Tree Maker already fails all the tests, but on top of that, all the exported GEDCOM files have an invalid GEDCOM header.

Family Tree Maker 2010 doubly fails all the tests.

Genbox Family History 3 (3.7.1)

test file	result
Married1200.ged	Imports fine, exports fine, navigates fine.
Children1200.ged	Imports fine, exports fine, navigates fine.
Long26CC.ged	Upon starting the import, Windows reports that Genbox Family History has stopped working.
Long26LL.ged	Upon starting the import, Windows reports that Genbox Family History has stopped working.

Genbox Family History fails the long name tests.

GENViewer Lite 1.15

test file	result
Married1200.ged	GENViewer Lite fails immediately with an Access Violation.
Children1200.ged	Windows immediately reports that GENViewer Lite has stopped working.
Long26CC.ged	GENViewer Lite import the file, but does not process the `CONC` lines, and instead truncates names without so much as a warning
Long26LL.ged	GENViewer Lite fails to import the 26th name, the one that starts with Z.

GENViewer Lite does not produce errors, warnings or an import log.

The Reload last file viewed option has to be turned off to keep GENViewer Lite from continually producing exceptions.

GENViewer Lite fails all the tests.

Genealogica Grafica 1.12

test file	result
Married1200.ged	Loads fast, displays fine.
Children1200.ged	Loads fast, displays fine.
Long26CC.ged	Genealogica Grafica imports the file, but does not process the `CONC` lines, and instead truncates names without so much as a warning.
Long26LL.ged	Genealogica Grafica fails to warn about the long lines and truncates the last name, the one that start with Z, to 255 characters without warning.

Genealogica Grafica does not handle either CONC lines or long lines correctly, and cannot handle names longer than 255 characters, but it does warn that the 26 individuals in the long name tests are not connected.

Genealogica Grafica fails the long name tests.

Geves 1.3.28

test file	result
Married1200.ged	Imports the individuals, but not their marriages.
Children1200.ged	Seems to hang when finishing up the GEDCOM import, but does finish. Becomes Not Responding when switching to Person tab. Navigation too slow for practical use.
Long26CC.ged	Geves imports the file, but does not process the `CONC` lines, and instead truncates names without so much as a warning.
Long26LL.ged	Geves imports the names correctly, but without warning about the long lines, and on export of the long names, Geves fails to use `CONC` and thus produces invalid GEDCOM files.

Geves already fails all the tests, but on top of that creates GEDCOM files that do not even have a valid GEDCOM header.

Geves doubly fails all the tests.

GRAMPS 3 (3.1.1-1)

test file	result
Married1200.ged	Imports the data, but hangs on attempt to export it. Even after ten minutes, it remained stuck at 50%. GRAMPS had to be terminated from the Windows Task Manager.
Children1200.ged	Imports the data, but hangs on attempt to export it. Even after ten minutes, it remained stuck at 50%. GRAMPS had to be terminated from the Windows Task Manager.
Long26CC.ged	GRAMPS imports and exports the data correctly.
Long26LL.ged	GRAMP imports the names correctly, but fails to warn about the long lines. On export GRAMPS uses `CONC` to keep line length reasonable.

All tests where done using Portable GRAMPS, the only variant of GRAMPS with an easy Windows install.

GRAMPS passes the Long26CC.ged test, but fails all other tests.

Legacy Family Tree 7.0 (7.0.0.111)

test file	result
Married1200.ged	Import seems to hang, takes more than twelve minutes. The result seems fine and exports takes just a few seconds.
Children1200.ged	Legacy’s maximum is 60 children. Legacy refuses to import the rest.
Long26CC.ged	Legacy does not recognise the `CONC` lines, truncates the names and adds the `CONC` lines to notes.
Long26LL.ged	Legacy imports the names correctly, but without warning about the long lines, and on export of the long names, Legacy fails to use `CONC` and thus produces invalid GEDCOM files.

Legacy 7 refuses to properly import the Children1200.ged files, and pauses the import while showing a messagebox saying You have reached the maximum of 60 children per family. All additional children will be ignored..

Legacy 7 passes the Married1200.ged test, but that success is overshadowed by the embarrassingly slow import speed. Legacy fails all the other tests.

Legacy Charting 7.0 (7.0.120.951)

test file	result
Married1200.ged	Legacy Charting fails immediately with an Access Violation.
Children1200.ged	Legacy Charting imports the file just fine, but as soon as you try to draw a descendants chart, Windows reports that Legacy Charting has stopped working.
Long26CC.ged	Legacy Charting does not recognise the `CONC` lines, but truncates the names instead.
Long26LL.ged	Legacy Charting fails to import the 26th name, the one that starts with Z.

Legacy Charting does not produce errors, warnings or an import log.

Legacy Charting imports Long26CC.ged correctly, but fails all the other tests.

MyBlood 1 (1.1.0)

test file	result
Married1200.ged	Import and exports fine. Navigation to partners uses a large list, but it is not practical because it allows slow scroll only.
Children1200.ged	Imports fine, exports fine, navigates fine.
Long26CC.ged	MyBlood does not recognise the `CONC` lines, but truncates the names instead.
Long26LL.ged	MyBlood imports the names correctly, but without warning about the long lines, and on export of the long names, MyBlood uses `CONC` but still produces invalid GEDCOM files.

What MyBlood calls the People tab is really a Family tab. The lack of a real People tab and the absence of a Go To command make it hard to navigate between the disconnected individuals in the long names tests, until you remember to use Find dialog; the People search in its General Find is closest MyBlood comes to a real People tab.

MyBlood uses CONC lines on export, but does not recognise the CONC lines on import. Although MyBlood uses CONC lines on export to avoid long lines, the produced GEDCOM files are still invalid, because the lines are more than 255 characters long; 259 characters to be precise (257 plus Carriage Return and Line Feed).
It seems that Vertical Horizon tried to do the right thing, but forget about the line numbers and the line delimiters. The GEDCOM specification is clear that the total length of the line may not exceed 255 characters.
Most surprising is the lack of symmetry of surprising; MyBlood uses CONC on output, but still does not recognise it upon input. Thus, MyBlood cannot be relied upon to read GEDCOM that MyBlood itself wrote.

MyBlood barely passes the Married1200.ged test, and fails all the other tests.

Personal Ancestral File 5 (5.2.18.0)

test file	result
Married1200.ged	Imports fine, navigates fine, exports a tad slow.
Children1200.ged	Imports fine, navigates fine, exports fine.
Long26CC.ged	Windows reports that Personal Ancestral File has stopped working.
Long26LL.ged	Windows reports that Personal Ancestral File has stopped working.

Personal Ancestral File has no problem dealing with many partners or many children, but fails to handle long names.

PAF fails the long name tests.

RootsMagic 4 (4.0.7.0)

test file	result
Married1200.ged	The RootsMagic import crashes, and shows an Access Violation dialog box.
Children1200.ged	The RootsMagic import crashes, and shows an Access Violation dialog box.
Long26CC.ged	RootsMagic does not recognise the `CONC` lines, but truncates the names instead.
Long26LL.ged	RootsMagic imports the names correctly, but without warning about the long lines, and on export of the long names, RootsMagic fails to use `CONC` and thus produces invalid GEDCOM files.

RootsMagic 4 fails all the tests.

The Master Genealogist 7 (7.04.0000)

test file	result
Married1200.ged	TMG does not produce errors and does import all persons, but not their marriages.
Children1200.ged	Import problems as detailed below, but TMG does import the file. Apart from general TMG GEDCOM quality issues, the export is fine.
Long26CC.ged	TMG does not process the `CONC` lines correctly and truncates the names instead.
Long26LL.ged	TMG does not warn about the long lines, but does fail to import the names correctly, truncating them to a maximum length of 246 characters. On export, TMG fails to use `CONC` and despite the truncation of the names still produces a longest line of 259 characters, which is invalid GEDCOM.

After import of Children1200.ged, TMG shows a messagebox with the text Menu has not been defined with DEFINE POPUP. 17 FRMMEMO.OLETER1.GOTFOCUS, and asks whether you want to Abort, Retry or Ignore. Retrying does not help, and you need to choose Ignore multiple times before TMG proceeds. TMG then shows an Abort-Retry-Ignore messagebox with the text Menu has not been defined with DEFINE POPUP. 169 TMGAPPOBJ.MREDEFINEMENUEDIT and you once again need to choose Ignore many times. After that, TMG repeats the same behaviour with the same two messageboxes again.
Well, I would not be surprised if the actual error messages varied, but TMG is infamous for its frequent FoxPro run-time errors like these.

The GEDCOM files TMG produces do not include GIVN and SURN tags, but do include illegal trailing white space on DATE and NAME lines.
On import of Long26CC.ged, TMG does not complain about the INDI.NAME.CONC tag, but does complain about every INDI.NAME.SURN.CONC tag. That seems inconsistent and is perhaps best understood as limited support for the SURN tag.

On export, TMG defaults to a maximum GEDCOM line length of 80 characters. That is not wrong, but its tip message that the value should be between 80 and 246 is; there is no official minimum and the maximum is 255, not 246. More worrying than the misinformation is that TMG does not respect the value you choose; even if you leave the default of 80 characters untouched, TMG will still write lines longer than 255 characters.

The Master Genealogist 7 barely passes the Children1200.ged tests and fails all other tests.

conclusions

failure

Of all the applications tested, not one passes all the tests. Many fail all the tests. Embla Family Treasures and Family Tree Maker 2010 and doubly failed because the ostensible GEDCOM files they write do not even have a valid GEDCOM header. If character encoding issues were taken into account, Family Tree Maker 2010 would triple fail all the tests.

name length

Several applications fail to import the 26th name, because they have an name length limit of 255 or 256 characters and that last name, the one that starts with a Z, because it is the only name in the file that is longer than 256 characters.

The Master Genealogist truncates names at 246 characters, Daub Ages! supports a maximum name length of 243 characters and Family Tree Maker 16 is infamous for supporting no more than 40characters.

fixed limits

Various application have arbitrary fixed limits. Family Tree Maker 16 allows only 99 marriages and only 99 children. Brother’s Keeper allows only 75 children.

crashes

Several applications became slow, Not Responding or awkward to navigate. Several appeared to hang on import, and GRAMPS hang on export. A few applications crashed.

The applications that hang or crash may well do so because the vendors decided to use fixed limits but forgot to test against those limits.

long lines

Many applications write illegally long lines and do not bother to alert the user to erroneously long input either.

Few applications use the CONC tag when writing a GEDCOM file as they should and even fewer do support the CONC tag when reading a GEDCOM file. Not one application processed the Long26CC.ged file correctly.

updates

2010-02-07: Behold Beta (0.99.7)

In his blog post Version 0.99.7 Beta released, Louis Kessler claims that Behold version 0.99.7 passes all the tests.

test file	result
Married1200.ged	Imports and displays correctly. All tables are complete now.
Children1200.ged	Imports and display correctly.
Long26CC.ged	Behold imports and displays the data correctly, without error or warning.
Long26LL.ged	Behold imports and displays the data correctly, and issues a message about the long lines. There is no log file.

Except for the fact that this version fails to produce a log file, Behold Beta 0.99.7 is the first genealogy viewer to pass all the tests. No genealogy editor has passed all the tests yet.

2010-02-26: Family ChArtist 1.0

test file	result
Married1200.ged	Imports correctly (but no chart types shows wives).
Children1200.ged	Imports correctly (but no chart type shows children).
Long26CC.ged	Family ChArtist does not process the `CONC` lines correctly and truncates the names instead.
Long26LL.ged	Family ChArtist imports the names correctly, but without warning about the long lines.

Family ChArtist did not fail the first two tests, but did not pass them either; to pass, it must show the result on screen. Family Chartist failed the last two test.

2010-12-23: Family Tree Maker 2011

Family Tree Maker 2011 GEDCOM Torture Test Results provides tests results for Family Tree Maker 2011.

2010-12-31: Sherwood Branches

test file	result
Married1200.ged	Imports correctly, displays correctly.
Children1200.ged	Imports correctly, displays correctly.
Long26CC.ged	Branches does not process the `CONC` lines correctly and truncates the names instead.
Long26LL.ged	Branches imports the names correctly, but without warning about the long lines.

Branches passed the first two test. Branches failed both the concatenation and the long lines test.