2017-12-07 Ahnenblatt is Large Address Aware

Microsoft Memory Magic

Ahnenblatt

Ahnenblatt is a free genealogy application for Windows. Ahnenblatt isn't a Java or Microsoft .NET application, but a true Windows application, a 32-bit Windows application to be more precise.

A 32-bit application can address 32 bits of memory, and that is 4 GB. However, for several technical reasons, 32-bit Windows limits the address space of each application to 2 GB; if a 32-bit application tries to use more than 2 GB of memory, Windows refuses the memory request, and, depending on how badly the application needs it, and how well it is written, the application may terminate or crash.

Large Address Aware

Have a look at the screenshot of Windows Task Manager. I've sorted the applications I'm running by memory usage. At the top of the list is Ahnenblatt, using more than 2GB of memory.
That isn't a Beta of some future 64-bit edition of Ahnenblatt, that is the regular 32-bit Ahnenblatt you can download today. That is a 32-bit Windows application using more than 2GB of memory. Ahnenblatt is allowed to use more than 2 GB because Ahnenblatt is Large Address Aware.

Out of Memory

Ahnenblatt is one of several applications I use regularly for its consistency checks. Early in 2016, it took Ahnenblatt somewhere between 6 and 8 hours to import my PAF GEDCOM file. That is a remarkably slow GEDCOM import. I never bothered to time the GEDCOM import exactly, I would just run it overnight. When I performed the consistency checks, memory usage would grow dangerously close to 2 GB. Mid 2016, my database had grown to big for Ahnenblatt 2.73 to handle, Ahnenblatt now needed more than 2GB and crashed.

I downloaded the latest version of Ahnenblatt, version 2.95, to see how well that version would handle it. When I tried version 2.95, it was immediately clear to me that some things had changed drastically. Ahnenblatt 2.95 did no longer take hours but mere minutes to import my GEDCOM file. Well, it showed Out of memory messages during import, making me wonder whether the import was truly successful.

I contacted Dirk Bötcher, informed him about my experiences, and asked whether he could create a 64-bit build of Ahnenblatt. He confirmed that, starting with Ahnenblatt version 2.8, he had been making changes to the Ahnenblatt's internal format and thus the GEDCOM import. The goal of the changes he was making is to offer even better compatibility with more GEDCOM dialects than Ahnenblatt already did.
Now, it's not uncommon for sweeping changes to make things worse before they get better; the initial goal is to have things workings again, import speed and memory usage considerations come later. The current version, Ahnenblatt 2.97a, has very reasonable performance. GEDCOM import is much faster now than it used be before the changes.

no 64-bit build

Dirk Bötcher told me that he had no immediate plans to make a 64-bit build. I then asked him if he would be willing to change just one bit in the program, by using one particular build instruction, a so-called linker flag, when building the software. He did this, and about a week later I received a private test build, and confirmed that it worked as expected; the new version of Ahnenblatt was allowed to use more than 2GB.

When you run a 32-bit Windows application on 64-bit Windows, it is still limited to 2 GB of memory.

32-bit & 64-bit Windows

32-bit Windows can address 4 GB of memory. It needs to reserve some of that memory for itself and some drivers, and then try to fairly divvy up the rest between the various applications the user is running. That is, in a nutshell, why 32-bit Windows limits 32-bit applications to using 2 GB.

To take advantage of more than 4 GB of RAM, you must be running a 64-bit edition of Windows. You don't want to leave all your 32-bit applications behind just because you are running 64-bit Windows, so 64-bit Windows has a feature called Windows on Windows to run 32-bit Windows applications. This subsystem of Windows is essentially a compatibility layer, and it tries to mimic 32-bit Windows complete with some undocumented features and known limitations.

When you run a 32-bit Windows application on 64-bit Windows, it is still limited to 2 GB of memory. That makes sense because it is a 32-bit application, and a 32-bit Windows application does not expect to get more than 2 GB. Then again, that 32-bit application is really running on 64-bit Windows, and 64-bit Windows could allow that 32-bit application to use 4 GB, but there is a good reason why 64-bit Windows still limits 32-bit applications to 2 GB; compatibility.
The thing is that 32-bit Windows literally limits applications to 31-bit memory, 32-bit Windows never uses the 32nd bit. Applications can use that 32nd bit for their own purposes, but when they do, they should definitely continue to be limited to 31-bit memory. Most 32-bit applications do not use that 32nd bit at all, and need not be limited to 2GB, but 64-bit Windows errs on the safe side of compatibility.

The Large Address Aware flag is only understood by 64-bit Windows.

`LARGEADDRESSAWARE`

64-bit Windows can allow a 32-bit application to use the full 32-bit address space, all 4 GB instead of just 2GB, but will only do so if the application explicitly informs Windows that there is no compatibility issue. A 32-bit application can do so by setting a so-called flag in the header.

A windows application has a particular file structure, which starts with a header. That header provides all kinds of information to Windows, for example the minimum Windows versions need. One bit within that header is the Large Address Aware flag. If an application sets that flag, Windows knows that it is safe to let that application use 4 GB of memory.

Setting that Large Address Aware header flag is ridiculously easy. It does not require any change to the application design, it does not require any code, or even some configuration file. It just requires the developer to specify a so-called linker flag /LARGEADDRESSAWARE. Doing so takes less than a minute; that's less than a minute to double the application's memory space from 2 GB to 4GB!

The Large Address Aware flag is only understood by 64-bit Windows. It gives 32-bit applications a 4 GB instead of a 2 GB address when running on 64-bit Windows. When you run the same application on 32-bit Windows, it remains limited to 2 GB of memory.

Ahnenblatt is Large Address Aware

After I confirmed that it works just fine, and Ahnenblatt could handle my database again, using about 2.1 GB to do so, Dirk Bötcher decided to keep this change. Ahnenblatt currently needs about 2½ GB to import my database database and perform consistency checks. For me, this small change makes the difference between be able to use Ahnenblatt and not being able to use it.
As far as I know, Ahnenblatt is the first and still only 32-bit Windows application to be Large Address Aware.