WikiTree AGC Release Notes

Profile manager: Rob Pavey [send private message]

Release Notes for the WikiTree AGC browser extension

The dates are when they were submitted to the Chrome Web Store. There is sometimes a short delay before they are available there because Google does a review.

There may be a delay between the version appearing on the Chrome Web Store and it appearing on the Firefox Add-on page.

Version 1.2.0 (2 August 2022)

• Update so that the Chrome version uses Manifest Version 2
• Incorporate all the fix fixes below from the Firefox only versions

Version 1.1.3 (2 August 2022) Firefox only

• Keep lines starting with : after a source line with no span ID
• Improve the Note flattening for a case using the @Nxxx@ format
• Ignore invalid dates that are just symbols e.g. “__”
• Move the “There are additional sources with no facts or date.” section into the Research Notes section
• Ensure there are always 2 newlines before == Research Notes ==
• Improve parsing of GEDCOM file name
• Do not report an error for “:Date: DEAD”
• Handle AFN in format :Ancestral File Number: 5134-S7
• Recognize this string when checking for a GEDCOM import: This biography was auto-generated by a GEDCOM import.^[1] It's a rough draft and needs to be edited.
• Handle the line “An explicit Surname and Married Name were both found.” in a name fact

Version 1.1.3 (1 August 2022) Firefox only

• Ignore lines like “Couldn't find any valid last name at birth." “Couldn't find any valid first name."
• Convert links to templates even if in “other source lines”
• Recognize GEDCOM verbiage of the form “WikiTree profile Di Rocco-2 created through the import of “
• Preserve text of the form “Record ID Number: MH…”
• Do not report an error for “:Date: DECEASED”
• Handle standard GEDCOM verbiage at the end of a junk section - can happen due to merge
• Handle old sections with the heading === Source ===

Version 1.1.2 (31 July 2022) Firefox only

• Handle WikiIDs with characters outside the basic Latin alphabet (e.g. Solé-666). This was preventing some of the standard GEDCOM sentences from being matched.
• Sometimes the junk sections (like “Data Changed”) have extra stuff at the end during a merge. This prevented AGC from being run. Change to put a note with the extra text in the Research notes.
• Handle profiles that have no sub-headings in the biography section - just create an empty biography.
• Turn more Ancestry family tree links into templates in older imports
• Handle the old === Source === section that has a different format
• Ignore lines like “:: Type: 3” in old GEDCOM imports

Version 1.1.1 (5 January 2022)

• Fix an issue where the generated ref name can be longer than 80 characters

Version 1.1.0 (4 January 2022) Safari Only

• Add Safari support and published in Apple App Store for Mac and iOS.

Version 1.0.0 (29 October 2021)

• Add a warning/confirmation if you press undo and you have made edits since running AGC
• Added new option: In narrative text, if a fact has no location, add the text "at an unknown location". This used to always be true.
• Add new option: For old GEDCOM imports move additional names from the Proper First Name field to the Middle Name field. This used to always ben done for UK and USA profiles, now it is optional.
• Handle different spellings of Acknowledgements/Acknowledgments
• Preserve more information from GEDCOM verbiage, User ID and profile ID
• Fix bug when there are two equivalent refs on the same fact
• Handle the case where, in new format, there is no space between * and Source: <span id
• Allow some date ranges (e.g. "BET ???? AND ????")
• Fix meaningful ref title for census facts embedded in a marriage section
• Add custom narrative for probate fact

Version 0.3.3 (24 September 2021)

• Improvements to handling of Notes sections
• Improve handling of census fact with no date in older format

Version 0.3.2 (15 September 2021)

• Updated to for the changes to WikiTree.com for Draft Saves

Version 0.3.1 (8 September 2021)

• Updated to handle the new GEDCompare format that is used for GEDCOMS uploaded after 1 Aug 2021 or thereabouts. See this G2G post. In the case of a modern format profile with no span ids it will now look for the Source: S* format and should produce the same results as if span ids were used.

Version 0.3.0 (25 August 2021)

• Added test for edited Sources section in older format. If it appears edited then AGC not allowed to avoid data being lost.
• Added new option to standardize/abbreviate country names in the narrative.

Version 0.2.9 (29 July 2021)

• Added various user options:
  • Option to omit the country name from locations in narrative events, either in all cases or when it is the same as in the previous narrative event
  • Make this existing behavior optional: If an employment, residence or census fact has a known record type start the narrative event with that. E.g. "In the 1911 census John was living in ..."
  • Make this existing behavior optional: If a census style narrative event starts with just the date then use the full date if known. E.g. "On 2 April 1911 John was living in ..."
  • Option to suppress certain sources with no date or fact associated with them
  • Option to suppress certain Issues to be checked

Version 0.2.7 (7 Jun 2021)

• Fix bug where it would report that there was no sources line if there were no spaces in it (e.g. "==Sources==")
• Add new option for the accessed date in the sources. Linda Petersen pointed out that just saying "(accessed [today's date])" could be misleading, especially on old GEDCOM imports. I do like adding this though - to encourage people to follow the recommendation of including an accessed date in source citations. There are now the following options for "Add an accessed date to citation:" and the default is "Before":
  • None: Don't add it
  • Unknown: Add "(accessed unknown date)"
  • Before: Add "(accessed before [date])". For recent GEDCOMs the date is today. For older format ones it is the date it was imported if known.
  • Today: Add "(accessed [today's date])"". The assumption is that you will access each source after running AGC.

Version 0.2.6 (3 Jun 2021)

• Some minor tweaks made after working on Barron GEDCOM challenge
  • If a line like this is found it is ignored since the appropriate action was taken on import: "Removed ABT from Birth Date and marked as uncertain"
  • If a line like this is found then it is removed and an alert is added to the "Issues to be checked" section: "Could not parse date out of ABT JUN 1894."
  • A blank line is always inserted after the "Research Notes" section and before the Sources. I changed this after accidentally deleting the Sources when deleting the Research Notes after checking the "Issues to be checked"

Version 0.2.5 (21 May 2021)

• Fix a bug where a line like the below was between the Sources and references lines was removed even when the option "Remove the text that states which gedcom the profile was created from and other standard GEDCOM import verbiage." is not checked.
  • WikiTree profile Howell-887 created through the import of family_book_EN.ged on Sep 5, 2011 by Katherine Picardy. See the Changes page for the details of edits by Katherine and others.
• Rename the option for removing the lines that contain the GEDCOM name and add a note to say this should only be done if fully cleaned up and sourced.
• Changed the removal of the standard GEDCOM verbiage. Now all the standard verbiage is removed, and if the option to remove the text containing the GEDCOM name is NOT checked, then lines are added at the very bottom of the profile (in Acknowledgements) of the form:
  • This person was created through the import of Ronalda-four.ged on Nov 10, 2015.

Version 0.2.4 (10 May 2021)

• Improvements to GEDCOM Junk removal
  • In LDS Endowment/Baptism allow the "-" character in temple name
  • Handle removal of LDS Sealing Child
• Improvements to reformatting in old format GEDCOM profiles
  • If there is a lot of extra data add it after the refs as indented lines (example Leaker-4)
  • If two facts get combined combine the descriptions and extraData
  • If there are multiple birth or death facts preserve the description and extra data in research note alert
  • Handle the :: Source: line better - it now captures all the following indented lines
  • Add code to add == Biography == after categories and the Unsourced project box
  • Add a == Source == line at the end if there is none (only for old profiles)
  • Remove a single character description of "Y" after Death: for example.

Version 0.2.3 (5 May 2021)

• Removing of standard GEDCOM import verbiage:
  • Example line: "This person was created through the import of WikiTree.ged on 20 February 2011. The following data was included in the gedcom. You may wish to edit it for readability."
  • Previous versions always removed most (but not all) of these lines.
  • There is now an option on the Options page to control this (defaults to removing)
  • If the option is set it now removes all known cases of this text.
• The extension now reports an error if an unsupported date format found on a ":: Date:" line. The date format must be changed before the extension can be run. This is to avoid possible data loss (of the date).
• Missing Biography line. Previous versions would not run AGC if the line was missing. But it turns out that early (2010/11) imports did not have this line. So for old profiles it now creates the Biography line if it is missing.
• Firefox support. This is the first version that was ported to be a Firefox Add-on.

Version 0.2.2 (29 April 2021)

• Fix bug where some occupation information was lost in old format
• Add code to remove the junk section “Ancestors of Interest”
• Add extra code to remove duplicate refs

Version 0.2.1 (28 April 2021)

• Add code to clean up and simplify MyHeritage sources
• Restrict middle name changes to old format profiles with locations in England and US
• Make detection of GEDCOM junk safer and add more cases of GEDCOM junk
• Added handling of Objects linked to facts (ignore them since they are also in the Objects section) (example: Coombe-146)
• Cleanup another format of Ancestry sources (seen in Coombe-146)
• Sometimes there is Census data under a marriage section (see Streeter-87). Added code to handle that.
• Handle Data Changed lines with a section (see Chown-104)
• Turn latitude and longitude into openstreetmap links
• Add checks to better detect edited or merged profiles and bail out
• Improve parsing of older formats
• Separate out the facts with no date into a Research Note section
• Improve date parsing
• Fix bug with combining facts where refs could get lost

Version 0.2.0 (19 April 2021)

• Changed the way that sources are formatted and cleaned up (big change)
  • For Ancestry sources it tries to remove a lot of Ancestry specific verbiage
  • For certain specific Ancestry sources (mostly England ones so far) it identifies them and formats them appropriately
    • There is a new option (defaulting to on) :
      For subscription sources, attempt to add links to free sources as well (usually links that do a search).
      For these specific Ancestry sources that it identifies it will also add links to search free sources for the same records (such as GRO, FreeBMD or FamilySearch)
• Improved the combining of Residence/Census/Occupation facts
• Fix bug where an error was reported when a census fact had a description line at the start beginning with a single ':'

Version 0.1.26 (2 April 2021)

• Added a safety check for the Could not interpret date in .... message in the profile text. If this occurs then WikiTree AGC will do nothing and display a message saying that the dates should be checked and this message removed before running AGC.

Version 0.1.25 (30 March 2021)

• Fixed a bug in latest format GEDCOMpare output where extra </ref> tags were being added and the citations were lost. It was caused by input cases where there is a \n before the "<ref> but not after it.
• Added handling for a Note: line after the location line in latest format
• Added handling for FSFTID section in latest format

Version 0.1.24 (27 February 2021)

• Follow-on to previous change. Now will also find the ::: Data and :::: Text lines following :: Source and put them in the ref.

Version 0.1.23 (27 February 2021)

• Added handling for 2010/11 format GEDCOM created profiles that don't use <ref> at all and just use span ID directly. It now converts them to <ref>
• Added handling for when the references line is "<references/>" instead of "<references />"

Version 0.1.22 (17 February 2021)

• Added handling for badly formed Ancestry links that GEDCOMpare did not convert to Ancestry templates. Often these are broken links to Ancestry trees which are no use but occasionally when I convert them to templates they work.
• Add improvements for unusual older versions:
  • Handle ==Biography== with no spaces
  • Allow a 2013 variant of the 2011 format where Places and Dates do not have "Place" and "Date" preceding them

Version 0.1.21 (23 Dec 2020)

• Fixed bug when there are multiple first names that are moved to the middle name. The order of them was reversed.

Version 0.1.20 (3 Dec 2020)

• Added an error reporting system. If pressing the AGC button fails and does nothing then an orange error dialog is displayed with a description of the error.
• Added some new error checks
  • If there is more than one Biography line it will error out
  • If unexpected data is found in a fact section it will error our rather than discarding that data
• Improved some of the parsing of 2011 format fact sections

Version 0.1.19 (15 Nov 2020)

• Minor fix to the previous change to make first, preferred and middle names work correctly with the undo feature.

Version 0.1.18 (15 Nov 2020)

• GEDCOMs imported in 2014 put first and middle names in the first name (and preferred name) field. This change moves the additional names to the middle name field.

Version 0.1.17 (5 Nov 2020)

• Improve handling for the 2018 Gedcom format. In 2018 it seems that GEDCOMpare went though a bunch of changes in format that are inconsistent and hard to parse.
  • Support refs where the name does not have quotes around it
  • Support date, location and description on lines that have -, : or :: on the start

Version 0.1.16 (28 Oct 2020)

• Add support for "Record File Number" sections that contain Geni person IDs
• Fix a bug where fixing up the current last name no longer worked. This was because recent WikiTree web site changes meant I wasn't able to get the Wiki ID from the document title.

Version 0.1.15 (22 Sep 2020)

• Update the extension for the WikiTree website update that changed the URLs
  • That prevented the AGC button showing up because the URL was not recognized as WikiTree edit mode

Version 0.1.14 (13 Sep 2020)

• Fix the title of the window/tab for the options page
• Handle the case in 2011 format where there is a heading (typically === Notes ===) between the == Sources == line and the references line.
• Added "Note flattening" for the 2011 format. This supports both the [[#N0499]] style notes used in 2020 format and the @HI1085@ style which only appear occur in 2011 format
• Also, add support for multi-line notes in note flattening
• In old format, if there is a fact with no date, location or description still add a narrative event for it if there is any usable info (like husband etc).
• Fixed an extra space in output before "one child".
• Add support for "Ancestral File Number" section
• Modified support for FSID and FSFTID to create a link to familysearch
• When the text " CONT " appears in a citation add it is a list item (bulleted)

Version 0.1.13 (3 Sep 2020)

• Fix some problems shown by Carlsdotter-1116. This is a newly created profile from a GEDCOM that has note span id's.
  • Main change was to pre-parse new format profiles and flatten remove lines like "Note <span id='N0068'>N0068<span> contents " and replace any references to this id like "Note: [[#N0499]]" with contents.
  • Also remove any empty "=== Notes ===" section that is remaining.
  • Ignored "married" line in name section
  • Ignore "Marriage" line in Marriage section
  • Ignore sections like MH:I500058 that start with "MH:"
  • Ignore lines in facts that start with "MH:" and the following line if it looks like "RIN".
  • Fix some specific links to websites that were broken by the GEDCOM import. For example "https://app.arkivdigital.se/volumev72425?image=24" should be "https://app.arkivdigital.se/volume/v72425?image=24". The GEDCOM import process removes some "/" characters from URLs. I don't see anyway to automatically correct for this bug. All I can do is look for specific malformed URLs on sites that I hard-code in.

Version 0.1.12 (1 Sep 2020)

• Fix bug in parsing when there are no Name, Born or Died sections (Arseneau-156 and Mountz-21)
• Ignore the TAG section in old format
• Handle an === External Files === section in new format by just removing it, it doesn't contain any usable information (Horner-3289)
• Change all the icon and image files to be lower case files name to solve case sensitivity issues on some platforms
• Fix bug where the extension disabled the Enhanced Editor and did not reenable it if there was an error.

Version 0.1.11 (31 Aug 2020)

• Preserve the Family Search ID (in old format FSFTID or FID sections) as a source

Version 0.1.10 (30 Aug 2020)

• Improvements to parsing the older (2011) format
  • More cleanup of "GEDCOM Junk"
    • Ignore the === FID ===, === FSD === and === CREA === sections
  • Handle case where there is no <references /> line in old format
  • Handle case where there is a === heading after the sources.
  • No SOUR line is ignored in sources
  • Handle old format gedcoms with no Biography line.
  • Handle old format GEDCOMs with no :: Date and :: Place but the date and place just after ::
• Fix issue where a fact with no date that shared a recordId with a birth could get moved to birth date incorrectly
• More robust parsing for case where the first name is a bold title (2018 format)
• The extension was generating quarter dates like “was born in the July quarter of 1863” for UK birth and death registrations. These are not really accurate since they could have been born a couple of months before the quarter. Also it is not clear to some what “July quarter" means. So the extension now uses something like "was born between May and September 1863" instead.
• Add options to add newlines to refs
  • before the first <ref> (this does cause a slight space in public view)
  • after the <ref> and before the </ref>. This makes it easier to edit but has no effect on the public view.
• For facts with no date, add code to try to extract date from the citation in some situations.
• If an age for an event is less than 2 years and day of birth or event is not known then treat it as “about”

Version 0.1.9 (28 Aug 2020)

• More cleanup of older formats and "GEDCOM Junk"
  • Remove rough draft verbiages with a ref. E.g. This biography was auto-generated by a GEDCOM import.<ref> ... </ref> It's a rough draft and needs to be edited.
  • Allow any title to start old format bio if no name, birth or death.
  • Remove the source in old format that references the gedcom - all this info is in the changes list.
  • Ignore any “GEDCOM junk” headings : "=== LDS Endowment ===", "=== LDS Baptism===", “=== Record File Number ===”, “=== Submitter ===”, “=== COLOR ===”
  • Ignore items in the Object section if the file path doesn’t include http.
  • Ignore lines of the form “No REPO record found with id R-593030340.”
  • Ignore “: User ID:” line in a marriage.
  • Add Unsourced sticker if there are no sources
  • Remove all repository information
• Add “See also:” if there are additional sources
• Fix bug: If there was more than one source without a source ID it was losing all but one of them.

Version 0.1.8 (26 Aug 2020)

• Fixes to parsing the older format:
  • Fixed identification of the "=== Burial ===" section
  • Fix parsing so that it doesn’t lose the last event of a section in some cases
  • Process the "=== Object ===" section and create external media links
  • Ignore the “User ID”, “FSFTID” and “Data Changed” sections.
• Combine the death and burial facts when there is no burial date
• Fixed a double space before the accessed date when there is no citation for a ref.

Version 0.1.7 (24 Aug 2020)

• Changed research notes sections that mentioned “WikiTreeAGC” to say “WikiTree AGC” instead.
• Lots of small changes to cleanup source, citation and description fields:
  • Remove text of the form: "Data Changed: Date: 15 JUL 2011 Time: 12:02:33." from sources
  • Remove text like “APID: 1,8767::0” from sources when the citation also has a reference to that database
  • Remove duplicate references to an ancestry record in a source or citation. These could be an APID, a link and a template for example.
  • Remove multiple spaces from citations and sources
  • Remove dangling “Note:” on end of sources
  • If the citation has the text “Page: Class:” then remove the “Page: “
  • For sources from MyHeritage that include a lot of data remove some html tags and add some spaces.
  • Add some punctuation in census descriptions and citations from Ancestry
  • In source add punctuation by changing “.Original data” to “. Original data”

Version 0.1.6 (24 Aug 2020)

• Fix bug in finding existing biography preamble when there is a space after “== Biography ==“
• Fix bug where a blank line between sources and references caused it to fail to parse the sources
• Enhanced the date parsing to handle dates in the format 10-1864

Version 0.1.5 (23 Aug 2020)

• Remove find-a-grave links in sources that have no memorial number to avoid profile suggestion 571
• Add more string matching to reduce the amount of “Unclassified” sources
• Improve reporting of multiple birth facts (avoid listing multiple variations of same location)
• Add consolidation and reporting of multiple death facts
• Other Marriages
  • Improve the reporting of “Other Marriages”. When there is a missing husband or wife name.
  • Change the title displayed on the source when it is an “Other marriage”.
• clean up locations in old format that have multiple commas etc
• Set “Current last name” for married women to the last husband’s last name if is known. Otherwise report if it looks like the Current Last Name is incorrect
• Use “Family Tree” as a title for sources
• Improve removal of duplicate refs
• Parse the Map data that comes from FamilyTree Maker
• Handle “Event” sections that can contain arrival etc - in both old and new formats
• Avoid duplicate alerts messages in the "Issues to be Checked" section
• Sort facts with no date to be after the birth date
• Improve parsing to work for profiles where the Name section has no ‘’’Name:”’’ title but has the first name in bold instead
• Remove comments of the form "Please edit, add, or delete anything in this text, including this note. Be bold and experiment! If you make a mistake you can always see the previous version of the text on the Changes page."
• Improve parsing so that it handles source between the Source header and the <references/> line
• If unnamed refs are the same then created named refs for them. This change has ripple effects and can cause facts to be merged etc.
• Fix bug in how unnamed refs on marriage are compared with refs on nameFact - it was too lenient

Version 0.1.4 (17 Aug 2020)

• The AGC button now works correctly if the Enhanced Editor is being used
• Make the AGC button show up on more old format GEDCOM profiles. Now it will show up if the profile meets any one of these conditions:
  • Contains the string This biography is a rough draft. It was auto-generated by a GEDCOM import and needs to be edited.
  • Contains the string This biography was auto-generated by a GEDCOM import. It's a rough draft and needs to be edited.
  • Contains at least one of these three strings: "=== Name ===", "=== Birth ===", "=== Death ===" AND also contains the strings ":: Date:" AND ":: Place:".
• Handle a secondary level acknowledgements heading after the sources. E.g. "=== Acknowledgments ===" and change it to a primary level heading. E.g. "== Acknowledgements =="
• Remove the "Please edit, add, or delete anything in this text. Thank you!" comment that is sometimes at end of old format
• Fixup the “Missing source ID” errors automatically in the case of database IDs that are recognized by the extension.
• Added the extension version number to the change summary

Version 0.1.3 (15 Aug 2020)

• Extended checks for birth “before” baptism to within a year (same for death/burial)
• Change the way the age is output for infants with approximate dates
• Handle the marriage data for old format profiles better
• Include cause of death if it is mentioned in profile
• Handle marriages that are before birth or appear to be other people. These happen in the old format that often included the parents' marriage. They are now moved to a section in the research notes.
• Handle the notes section in the old 2011 format properly
• If the profile includes links to Ancestry records rather than the Ancestry templates then switch them to templates. Also do this for APID fields. This can make links to ancestry.com.au for example work rather than saying you don't have an account there (so long as you have an Ancestry account somewhere).

Version 0.1.1 (12 Aug 2020)

• Made sure a year-only or approximate death comes before an exact burial in same year
• Allow parentheses in section names
• Handle a section named “Census:” in 2020 format
• Add an alert message if a burial with no date has its date set to after the death date
• If there is a burial fact with no date, and there is no death date, set its date to after the latest fact with a date,
• Fix the lack of a newline between name refs and external media
• Improve narrative output for events with no date
• Handle extra top-level sections and preserve them
• Handle Source: lines within facts that have info from the census like Age.
• Combine death and burial into one fact when the approximate death date comes from only burial source and there is no death location.

Version 0.1.0 (11 Aug 2020)

• Continuation of better handling where a GEDCOM was imported twice, causing duplicate facts:
  • Handle duplicate notes
  • Handle duplicate file facts
• Improved handling of facts with no date and/or no location
  • Improved case where there is no date on fact but location was used as description.
  • If a fact with no date shares a ref with a fact that has a date, then use that date
• Fixed an extra space that was added after “On an unknown date” for residence facts.

Version 0.0.7 (9 Aug 2020)

• Handle 2011 format bios. These are bios created by GEDCOMpare before the latest format (the latest format started sometime around 2017 or earlier).
• Recognize and preserve stickers and categories before the "== Biography ==" line that were added after the import
• Remove duplicate facts. These can happen if the GEDCOM was imported twice. Surprisingly this seems to be quite a common occurrence. Note: It only removes them if everything (including the refs) is the same.
• Handle occupations better
• Handle edited refs where newlines have been inserted in the refs since GEDCOMpare created the profile.
• Handled the line that says : Found multiple copies of death date
• Changed default for the Age option from “none” to “most”

Version 0.0.6 (3 Aug 2020)

• Sometimes there are lines in Residence facts that look like:

Source:

APID 1,Number::Number

These are now recognized and turned into a reference to the numbered Ancestry Record

• Added a custom format for the Employment/EMPLOY/Occupation facts
• Recognize some child marriages so that they are no longer named "Unclassified" in the source.
• Recognize broken Google book links in fact descriptions (GEDCOMPare has a bug that breaks URLs by removing / characters)
• Remove duplicate Alternate Names (and also trailing periods)
• If a location starts with a number use "at" rather than "in"
• Clean up the Options page and add more options:
  • Whether to output the meaningful names on references/sources
  • Whether to use baptism date for birth date in certain situations
  • Whether to use burial date for death date in certain situations
• If a profile contains no Name: or Born sections it will look for the Died section as the start rather than disabling the AGC button.