Modern Software Experience

2011-05-27

genealogy technology

AncestorSync Logo

AncestorSync

This text briefly explains what AncestorSync is and what it does, and then explains in some detail how AncestorSync works. The text and diagrams are mine, but have been reviewed for accuracy by Dovy Paukstys, Real-Time Collaboration's Chief Technologist.

What is it?

freemium service

AncestorSync is a commercial product and service of Real-Time Collaboration, available for both Windows and MacOS users. AncestorSync is a freemium service; limited functionality is free, full functionality requires a subscription.

AncestorSync is not a single product, but a technology, and a product family based on that technology.

product family

AncestorSync is not a single product, but a technology, and a product family based on that technology. The first AncestorSync product is AncestorSync for Geni, to be followed by AncestorSync for FamilySearch, AncestorSync for ourFamily•ology and several other products. As the names for these first few products suggests, each one works with one particular genealogical web app. Other kinds of products are planned.

What does it do?

genealogical utility

AncestorSync is a genealogical utility aimed at users of multiple genealogy applications. AncestorSync is a tool conceived to synchronise genealogy data between desktop applications and web apps. AncestorSync isn't a data transfer utility meant to be used to transfer your data once, but a synchronisation tool meant to be used continually. The idea is that you can switch between your desktop application and web app as you like, using AncestorSync to keep the databases synchronised with each other.

How does AncestorSync work?

the concept

Here's is a conceptual diagram.

AncestorSync Concept

Using AncestorSync involves three products; your desktop application, AncestorSync and your web app. You use both your desktop and your web app, and use AncestorSync to synchronise your data between them. AncestorSync offers two-way synchronisation, which means that it can sync your data in both directions; from the desktop application to web app, and from the web app to the desktop application.

existing integration

Some desktop applications already integrate with the vendor's own web app. For example, Family Tree Maker integrates with Ancestry Family Trees, and Family Tree Builder integrates with MyHeritage. If you use such a combination, and the vendor already offers full two-way sync, then you do not need AncestorSync. If the vendor offers only one-way sync or less than complete two-way sync, then you should look into AncestorSync to see whether it can provide full two-way sync for that combination, but keep an eye out for vendor updates as well.

two parts

This is a slightly more detailed diagram, that reveals a key design decision of the AncestorSync architecture; AncestorSync consists of two different, communicating parts.

AncestorSync: Two Parts

AncestorSync consists of an application on the desktop and a web server on the web; AncestorSync Desktop and AncestorSync Web Server. AncestorSync Desktop does desktop stuff. AncestorSync Web Server does web stuff, and they communicate with each other to make the sync happen.

AncestorSync Desktop reads and writes desktop genealogy databases so that AncestorSync Web Server does not have to know how to do that; AncestorSync Web Server simply communicates with AncestorSync Desktop. AncestorSync Web Server reads and writes the web genealogy databases, so that AncestorSync Desktop does not have to how to do that; AncestorSync Desktop simply communicates with AncestorSync Web Server.

AncestorSync Desktop

The AncestorSync Desktop service is loaded upon system startup, and a TaskBar icon shows that the AncestorSync service is resident.
This is not necessary for manually initiated synchronisation, but it allows AncestorSync to automatically start scheduled synchronisation, and to automatically resume a synchronisation interrupted by a system reset.

AncestorSync Desktop reads and writes desktop databases directly, without any assistance from the desktop genealogy application. That should work just fine, but just in case AncestorSync Desktop accidentally messes up your database when writing to it, it makes a backup copy first. You do not need to import or export a GEDCOM file to perform a sync, but you do need to exit your genealogy application or at least close the database, so that AncestorSync Desktop can access it.
AncestorSync Web Server does not read and write web genealogy databases directly, the servers that the genealogy web apps run on simply do not allow third parties to mess with their database. AncestorSync Web Server read and writes the web genealogy database through the interface that the server offers for just that purpose, their genealogy API.
AncestorSync Desktop and AncestorSync Web Server communicate with each through a private protocol. That only works when your Internet connection is live, but everything else continues to work when it it not.

convoluted up- and download

If you held off using a web app because you could not sync with it, then your first-time use would amount to a convoluted file upload; instead of uploading your database yourself, you let AncestorSync do it, and AncestorSync does so in three steps. AncestorSync Desktop read your desktop database, sends the data AncestorSync Web Server, and AncestorSync Web Server sends it to the web app.

If you are only using a web app, and decide that you would like to have your data in a desktop app as well, you would not do it yourself either, but again let AncestorSync do it in what amounts to a convoluted download; AncestorSync Web Server gets your data from the web app, sends it to AncestorSync Desktop and AncestorSync Desktop writes the desktop database.

filtering

One reason for this more complex procedure is that it allows AncestorSync to filter your data. You often filter your data when you create a GEDCOM to upload, for example by excluding living individuals and private notes. AncestorSync offers similar filtering options.
Some of the AncestorSync filters affects what gets uploaded, none of the AncestorSync filters affect what gets downloaded. AncestorSync Web Server always sends all the changes it find to AncestorSync Desktop. Once all changes have been collected on the desktop, AncestorSync Desktop applies all the filters.

setup

The more important reason to do it this ways is that, like with any sync technology, the first-time sync is really part of the set up. The first time you sync, you have to provide AncestorSync some details about your desktop database and your web database. When the first-time sync is done, AncestorSync knows where to find your desktop database, knows which web app you use, knows which filters you want it to use, and remembers your login name.
More importantly, it also remembers what data you have each in system.

sync

AncestorSync isn't an upload or download utility, it is a synchronisation tool. When you sync, AncestorSync does not upload or download your entire database, but only the changes since the last time you synced.
When you've made changes to your desktop application, and decide to sync those changes to the web app, Ancestor Desktop detects what has changed, tells AncestorSync Web Server about, and AncestorSync Web Server then makes the same changes to the web database. Conversely, when you've made changes in your web app and decide to sync those changes to the desktop database, AncestorSync Web Server detects what has changed, tells AncestorSync Desktop about it, and AncestorSync Desktop makes the same changes to your desktop database.

AncestorSync: Databases

detection

AncestorSync tries to be smart about detecting changes, but the basic idea is quite simple; AncestorSync detects changes by comparing your current database to the database you had the last time you used AncestorSync. To do so, AncestorSync remembers what's in your database.
To be more precise; AncestorSync remembers what's in your databases; AncestorSync Desktop keeps a copy of your desktop database and AncestorSync Web Server keeps a copy of your web database.
The desktop and web database need not be identical, not even immediately after a successful sync. In fact, your AncestorSync filter settings probably prevent them from ever becoming the same. With typical filter settings, your desktop database will contain all your private data, and your web database will contain none or very little of it.

AncestorSync may take advantage of features, such as timestamps, that your desktop and web database have, to discover changes as quickly as possible, but that does not otherwise affect how the synchronisation process.

AncestorSync Web Server

The AncestorSync Web Server keeps a copy of your web app data, but you do not need a separate login credentials, you only need the user name and password you already have for the web app.

AncestorSync does not know or store your password, but, like many twitter clients, uses OAuth to ask the web app to authenticate you; AncestorSync Web Server gets access to your web database, and then grants you access to your data on the AncestorSync Web Server iff the web app confirms that you are who you say you are.

changes

So, when you've made changes to your desktop database, AncestorSync Desktop reads your desktop database and compares it to the previous copy it made to detect changes. AncestorSync Desktop applies the filters, and sends the remaining changes to AncestorSync Web Server, which then applies those changes to your web database.

When you've made changes to your web database, AncestorSync Web Server reads the current web database, and compares it to the previous copy it made to detect what has changed. Well, conceptually. If the web database used supports that functionality, AncestorSync Web Server may actually get the changes directly from the web database itself. AncestorSync Web Server then sends the changes to AncestorSync Desktop. AncestorSync Desktop applies the filters, lets AncestorSync Web Server know about its decisions, and applies the remaining changes to your desktop database.

conflict resolution

Well, that's the basic idea. Things are actually a bit more complex. AncestorSync normally performs a two-way sync. You don't have to tell it in which direction to sync, because it automatically detects your changes, and then uploads or downloads these changes. If there are changes to both the desktop and the web database, it will both upload and download changes; it will upload the desktop changes to the web, and download the web changes to the desktop.
This is where things get interesting; if a record has been changed on both ends, those changes may conflict with each other. AncestorSync detects such conflicts, and then asks the user what to do. Typically, you'll approve one change and reject the other, and AncestorSync will act accordingly.

change approval

When you make changes in either database, AncestorSync will sync those changes to the other database; well, at least when your filters permit it and there's no conflict to deal with. But when you are sync with a shared database like Geni.com, AncestorSync also has to deal with changes made by other users. So, AncestorSync will not only ask you to decide on conflicting changes, but may also ask to you decide on changes made by others, and will definitely ask you to decide on conflicting changes made by others.

So, AncestorSync does not simply send (filtered) changes from the desktop to the web, or from the web to the desktop, and then apply them. AncestorSync does not apply any change until it has collected all changes together. Once AncestorSync Desktop has collected all pending changes together, it looks for conflicts and other issues, such as changes made by others, that need to be resolved or approved. Only after doing that does AncestorSync Desktop approve and apply all the non-problematic changes. AncestorSync Desktop then presents all problematic changes to you, for you to make a decision on; only after you've made your decisions are the appropriate changes applied.

speed

Synchronisation speed is hardly affected by the speed of your PC. Synchronisation speed is noticeably affected by size of your database, the bandwidth of your network connection and your filter settings, and the amount of RAM available to the application is not unimportant.

The larger your database, the longer the first-time sync takes. Your filter settings determine how much data gets synced, and your Internet bandwidth limits how fast that data gets transferred. AncestorSync works over dial-up connections, but a broadband connection is recommended, particularly for the first-time sync. After the first-time sync, network traffic will be modest, even if your filter settings allow syncing everything, as AncestorSync only communicates changes.
Handling a larger database requires a bit more more RAM, but if you have a fairly modern system with a few gigabytes of RAM, the amount of RAM should not be a limiting factor.

Synchronisation speed is also affected by both the AncestorSync Server and the web app server. Either one might be slow because it is overloaded, and the web app might very well limit the speed at which it allows third parties to make changes through its genealogy API.

Several things might slow a sync down, but after the first sync, no sync should take very long. After the first sync, sync speed mostly depends on the number of changes since the last sync.

large databases

This makes AncestorSync of particular interest to users with large databases. The larger your database, the longer import and export take. The larger your database, the longer upload and download take. When you use AncestorSync, you do not import or export your database to upload or download it, you merely synchronise the changes. Detection of the changes is a bit slower for larger databases, but the synchronisation speed mostly depends on the number of changes.

system failure

Synchronisation systems are complex, but they are not brittle. On the contrary, synchronisation systems are quite resilient in the face of system failures.

If you reset your system during a synchronisation, synchronisation will simply continue where it left off when you restart it.
If you are temporarily without Internet connection, synchronisation will simply wait until the connection is live again.
If AncestorSync Desktop gets so corrupted that you need to remove it and do a fresh install, synchronisations that already happened need not be done again.
If the AncestorSync Web Server is wiped, you will be unable to sync while it is brought back online and filled with your web app data again, but after that you will be able to sync again as if it was never gone.

The fact that AncestorSync keeps copies may come in handy when the two systems it connects together fail. If the web app fails and the vendor is unable to restore your data, it could be restored from the AncestorSync Web Server. If your desktop database gets corrupted, you could restore from AncestorSync Desktop instead of your backup. If even your backups are unusable, you'll still be able to recreate your database from the data on the AncestorSync Web Server; how much you'll be able to recreate will depend on the filter settings you used; if you did not allow upload of living individuals and private data to the web, then you cannot restore that from the AncestorSync Web Server either.

updates

2012-05-29 AncestorSync Beta

AncestorSync Beta is an exclusive technology preview of the AncestorSync Beta; how to sign up, what to expect.

links

AncestorSync

genealogy web apps