Maintenance — Duplicate Handling¶
PastPaths can find and merge records that were imported multiple times or registered as separate entries even though they refer to the same person, place, media file, or event. The feature is accessed via Administration → Maintenance or via dedicated buttons on the respective list pages.
Overview¶
Each entity type has its own points-based system that weighs several signals together into a probability score (0–100). The score determines whether a pair is shown as a possible duplicate. You then manually choose which fields to keep, and the system handles all database links automatically.
| Entity | Primary signal | Default threshold |
|---|---|---|
| Person | Phonetic name + date + place of birth | 50 |
| Place | Source text (GEDCOM PLAC) + coordinates | 30 |
| Media | SHA-256 hash | 35 |
| Event | Type + date + place | 60 |
Sources, categories, and tags¶
Unlike persons, places, media, and events, tags, categories, and sources have no automatic scored detection. Tag and category names are already deduplicated technically (the same name can't be created twice), so "duplicates" here are in practice spelling variants or synonyms — Probate and Probate record, for example — which the system can't determine automatically.
Instead, you select what to merge yourself:
- Go to Administration → Taxonomy (for tags/categories) or Administration → Sources
- Check at least two rows in the table
- The Merge selected button appears — click it
- Choose which of the selected items to keep (default: the one with the most links)
- Confirm
All links (persons, places, events, media, blog posts for tags/categories; citations for sources) are automatically moved to the kept record. For categories, any subcategories are moved to the kept category instead of being deleted.
The same shortcut is available on the Maintenance page: the Sources and Taxonomy buttons take you directly to the respective tab, with a short description of how merging works there.
Searching for duplicates¶
Automatic search¶
Open the search via Maintenance → Find duplicates or the button on the respective page (Persons, Places, Media, Events). Adjust the threshold with the slider and click Search. The system returns pairs sorted by score, with a color-coded confidence indicator:
- Green (high) — strong match, likely the same record
- Orange (medium) — possible duplicate, review manually
- Gray (low) — weak signal, use your judgment
Under each pair, the score breakdown per signal is shown (e.g. "Exact name: 25/25," "Coordinates: 30/30") to help you understand why the pair was flagged.
Manual search¶
The Manual search tab lets you search freely and select any records for merging — regardless of how similar they are score-wise. Useful when you know two records are the same but the system hasn't found them automatically (e.g. a name typo that the phonetic matching doesn't catch).
Select at least two records and click Merge selected. With more than two, the merge dialog opens sequentially: you merge one pair at a time until all are done.
The merge tab¶
When you click Review on a duplicate pair, the merge dialog opens.
Choosing a keeper¶
At the top, you choose which of the two records will be the keeper — the one that's kept. Click the name chips to switch. This choice affects the default setting for all fields below (the keeper's field is preselected).
Field selection¶
Each data field is shown as a row with the keeper's value on the left and the other record's value on the right. Click the side you want to select which value is kept.
For text fields (description, notes/biography) and taxonomy (categories, tags), there's a third option:
Combine both — the texts are concatenated with a separator, or the taxonomy is merged as a union. This option is selected automatically when both records have content, to minimize data loss.
The confirmation dialog¶
Before the merge is carried out, a summary is shown:
- Which record will be deleted and which will be kept
- How many links will be moved (relationships, events, citations, etc.)
- A warning if any text field or taxonomy field will be discarded (i.e. you chose only one side's data)
The merge cannot be undone. Check that you've chosen the right keeper before confirming.
What happens in the background¶
Upon confirmation:
- Selected field values are written to the keeper
- All foreign-key links are updated (see table below)
- Taxonomy (categories and tags) is merged according to your choice
- Any "Not duplicates" markings for the removed record are cleared
- The removed record is permanently deleted from the database
| Entity | Links that are moved |
|---|---|
| Person | Relationships, event links, media, places, citations |
| Place | Birthplace references, place-of-death references, person-place links, events, citations |
| Media | Person links, place links, event links, citations |
| Event | Person roles, media links, citations |
Media files on disk
When merging media records, the database row for the removed record is deleted, but the physical file is left on disk as an orphan. This is a deliberate choice to avoid data loss. Orphan files can be cleaned up manually if needed, via Maintenance → Orphaned files on the media card. The scan goes through the entire media folder and lists files no longer referenced by any media record, with file size and last modified date — check the ones you want to delete.
Not duplicates¶
If the system suggests a pair that you know aren't duplicates, click Not duplicates. The pair is hidden from future searches, and an Undo button is shown for 5 seconds if you change your mind.
The marking is stored in the database and applies until you undo it via the Undo action.
Persons without connections¶
Open via Maintenance → No connections on the persons card. Finds persons with no family relationships at all — regardless of whether they have events, source citations, or known birth/death dates. Common after large GEDCOM imports where individual records lose their connections.
Click Search to list them. Per row, you can:
- Ignore — marks the person as reviewed, hidden from future searches
- Tag "Missing relations" — same principle as the import tag, for deliberate categorization
- Open — opens the person's edit dialog directly, e.g. to write a note about why the connection is missing
- Delete
The list doesn't refresh automatically after an action — press Search again if you want an updated list.
Scoring system per entity¶
Person¶
| Signal | Points |
|---|---|
| Exact name match | 40 |
| Phonetic match (Carlsson/Karlsson, Wahlström/Vahlström, etc.) | 20 |
| Birth year matches (precision: year) | 22 |
| Birth year + month matches | 25 |
| Exact date of birth matches | 30 |
| Birthplace: coordinates < 500 m | 20 |
| Birthplace: parish + country matches | 10 |
| Date of death matches | 15 |
| Gender conflict (both set, different genders) | −30 |
Default threshold: 50. Recommended range: 40–80.
Place¶
| Signal | Points |
|---|---|
| Source text (GEDCOM PLAC string) exact match | 50 |
| Coordinates < 100 m | 30 |
| Exact name match | 25 |
| Normalized name matches (slug) | 20 |
| Parish + country matches | 15 |
| Similar name (fuzzy) | up to 15 |
Default threshold: 30. A lower threshold catches more possible pairs — useful for large GEDCOM imports.
Media¶
| Signal | Points |
|---|---|
| Identical file (SHA-256 hash) | 80 |
| Same file size | 10 |
| Similar filename | up to 20 |
| Same media type | 5 |
Hash calculation
The SHA-256 hash is calculated the first time a duplicate search runs and is saved to the database. Searching a large media collection may take a few minutes the first time. Subsequent searches are fast.
Default threshold: 35. A hash match (80 pts) is always "high" confidence.
Event¶
Only searches among non-trivial event types: marriage, divorce, baptism, confirmation, graduation, military service, emigration, immigration, occupation, retirement, burial, census, residence, education, and other. Birth and death are handled automatically when merging persons.
| Signal | Points |
|---|---|
| Same type + date + place | 80 |
| Same type + date (no place) | 40 |
| Same type + place (no date) | 25 |
| 2 or more shared participants | +15 |
| 1 shared participant | +5 |
| Identical source string | +20 |
| Shared source citations | +15 |
Default threshold: 60. Raise to 80 if you only want near-certain matches.
Sequential merging¶
During a manual search, you can select more than two records. The dialog then opens for the first two, and once you've confirmed that merge, the next pair opens automatically (the keeper from the previous step against the next record in the queue). You can cancel the sequence at any time with Cancel in the dialog.
Event type during sequential merge
For events, the system checks that all records in the queue have the same event type. If a record in the queue has a different type, it's automatically skipped with a message.
Tips¶
- Start with a low threshold, raise it gradually. Run a search at threshold 30 for places to see the volume. Then raise it if the list is overwhelming.
- Taxonomy defaults to "Combine both." If you want to remove duplicated import tags (e.g.
tng-import-2025-01-01on both), choose "Keep keeper" instead. - The Maintenance page aggregates all entities but is just a shortcut to the same dialogs found on the respective list pages. The results are identical.