Computer macros and genetic genealogy?

+9 votes
310 views
I'm thinking back to the days of DOS macros when I could automate just about anything I could do with a keyboard.

Why is it not possible to launch a macro to login one's self to GEDmatch and do a one to one compare between two GEDmatch IDs?  Then save that page (showing the shared segment(s)) to one's computer?
in WikiTree Tech by Peter Roberts G2G6 Pilot (531k points)

4 Answers

+6 votes
Peter, Macros are simply a file of a set of keystrokes. When you run the macro (which requires a manual command to do) it executes the keystrokes as if you had typed them.

A macro is something that can only run on the computer on which it is activated.  You can't sit at your computer and run a macro on a web server.  Could you imagine the havoc if anyone could create a macro on their computer and then run it on any web server?  It would eliminate any motivation for hacking, since anyone could do anything they wanted to any website!!!

On the other hand, the developer of a website can (and does) create functions that can be performed ... such as searches ... and would be able to create one that does the sort of thing you are asking here.  That would be activated on the server when you click a button on one of its webpages.
by Gaile Connolly G2G6 Pilot (788k points)
Thanks Gaile,

What you say is part of what I don't understand.  I can sit at my computer and login to GEDmatch as me.  Why can't I have a macro to do that? Or can I already?  I could do that with a DOS macro.  Then a macro could mimic my keyboard strokes and do a one to one compare between my GEDmatch ID and another user.  Then save that result page to my computer.

Most (All?) mouse controls can also be done with keyboard commands.  Why can't I automate those keyboard commands?

Thanks for your patience.
Can the Chrome browser do this type of automation?
Peter, you can create a batch file to run a set of keyboard (or mouse) commands, but when those commands need to be executed by a program - like your browser, for example - then it becomes much more complex than when the commands are being applied to the operating system.

Some programs have their own macro languages built in and are set up to enable users to create macros that the program can run.  Microsoft Word and Excel are examplse of some of these.  To my knowledge, no browser has a macro language built into it,  Browsers do have the ability to save some of the things you type, though.  In most browsers, when you type a password, you can set the browser to "remember" that password.  After that, every time you go to the same site and enter the same user name, the password will automatically be entered.  The browser can't do more than automatically fill in some fields on a web page, though, by "remembering" what you last used in that field.  It cannot automatically submit a form that you fill out and it certainly can't control the server's response when you do something.

The server's actions are controlled by the programming behind the website that interprets forms that you submit and does processing to produce whatever result is intended to be displayed for you.  When a website uses cookies, that provides the means to automate actions on the server when you do certain things, but that can only be done by the server's programming.  If there is some processing that you'd like to be able to have automated then you can write to the appropriate person at the website where you want to do it and ask if they would consider providing that capability.  Perhaps, if they think it's a good idea that many users would like to be able to do, then they'll set up the programming to make it happen.  Good luck!

It seems that iMacros for FireFox or Chrome can do the following.  Do any of you use iMacros?

Go to http://v2.gedmatch.com/login1.php

[my browser already knows to how to fill in my Email Address and Password]

Click on Log in

Click on ‘One-to-one’ compare

Enter text string variable in field labeled Kit Number 1: [F008867]

Enter text string variable in field labeled Kit Number 2: [M191333]

Click on Submit

+5 votes

It's difficult to automate browsers from the outside, and this is on purpose for security reasons. You could automate within a browser, but still it's clunky for what you're trying to do, which should be straightforward.

The number of steps it takes to get to the tools in GEDMatch has been bugging me, along with the way each page checks the referer header, which often gets you lost when you're trying to work in multiple tabs.

The script below goes through steps that GEDMatch likes to see in order, then gets your comparison.

If you install cygwin (https://cygwin.com/install.html) and then in cygwin install the cURL package (http://stackoverflow.com/questions/3647569/how-do-i-install-curl-on-cygwin), then the shell script below should do what you want without using any browsers.

Save it as an executable .sh file where your cygwin environment can access it.

(assuming if you've modified the script to use your credentials)... it will save the results of the kit comparison to myFile.html, and will open myFile.html in your default browser.

Long story short: the script below is a bash script to do a one-to-one compare on GEDMatch, save the file locally, then open the file in your default browser. It should work out-of-the-box in MacOS or Linux, but on Windows, you need cygwin and curl to do it. 

Save it as "one2one.sh" or whatever, make it executable, then call it from the cygwin prompt like

$ ./one2one.sh f123456 f654321 ./myFile.html

 


#!/bin/bash

uname="youremail@example.com" #replace me
pword="yourpassword" #replace me

#alternative is 0 if you don't want the charts
chart=1

resolution=1000
threshold=500
shared=6
win_size=51
bunch_limit=25

credentials="email=$uname&password=$pword"

initialPage="http://v2.gedmatch.com/login1.php"
loginPage="http://v2.gedmatch.com/login2.php"
selectPage="http://v2.gedmatch.com/select.php"
compare1Page="http://v2.gedmatch.com/u_compare1.php"
compare2Page="http://v2.gedmatch.com/cgi-bin/u_compare2.cgi?kit1=$1&kit2=$2&chart=$chart&resolution=$resolution&threshold=$threshold&shared=$shared&win_size=$win_size&bunch_limit=$bunch_limit&xsubmit=Submit"

# these requests go through a sequence that logs you in and then travels
# to the comparison page in a way makes GEDMatch happy
res=$(curl -i -c ./cookies.txt -s "$initialPage")
res=$(curl -i -b ./cookies.txt -s --referer "$initialPage" --data "$credentials" "$loginPage")
res=$(curl -i -b ./cookies.txt -s --referer "$loginPage" "$selectPage")
res=$(curl -i -b ./cookies.txt -s --referer "$selectPage" "$compare1Page")
res=$(curl -b ./cookies.txt -s --referer "$compare1Page" "$compare2Page")
# res now is the response from the comparison
# echo it into a file
# customize this to taste.

# replaces "base" so that the images are still loaded from the server
res=$(echo $res | sed -e 's|<head>|<head><base href="http://v2.gedmatch.com/cgi-bin/"/>|g')
echo $res > $3
# open the file with the default opener
open $3
by Daphne Maddox G2G6 Mach 1 (14.5k points)

Cool

Has no one tested doing something with Javascript to do the same?!?!?

I think I will try.... A Swedish developer has done magic things with Javascript and extracting the result list at FTDNA
see https://dnagen.net/

Video how to use his Javascript tool
and it also works on an Android phone (video)

+2 votes

AutoHotkey is a free program for Windows that can be used to automate tasks like you are suggesting.

by Stephen Reeves G2G5 (5.5k points)
+3 votes

Hi Peter,

since you ask it's about time to have my "coming out" I think.

I'll be launching a new web service that will help you with GedMatch and other DNA Genealogy websites in January. It's still in development and as I indicated in an earlier forum post (see here: http://www.wikitree.com/g2g/172100/dna-genealogy-why-and-which-problems-are-you-facing?show=172100#q172100) a couple of months ago I want to build something to solve the biggest problems that people have with DNA genealogy.

I've got very valuable feedback from the questionnaire with close to 200 people answering.

Working with Gedmatch is high on the list and as it's the only pool of DNA tests from all three major testing companies this will be my first feature that is launched.

However developing a solution that is appealing to pros (think people that use Gedmatch or Genome Mate Pro on a daily basis) and to people that are new to DNA genealogy (those that join the Facebook group of DNA-Newbies for example) is a difficult tasks.

While it should help the pros do more in less time my real aim is to get more people take DNA test and work with others on finding their common ancestors (the ultimate goal IMO).

Triangulation is at the basis to ensure we're not wasting time going through family trees and information from people that aren't closely related to us in the last 200 years (can be more sometimes).

As I have to concentrate on very important things like security, usability, developing & maintaining such service (there is a huge demand for computation and data storage required) the good old saying "Rome wasn't build in a day" applies.

 

So please bare with me a little bit longer. If you're interested to help on developing the best features & solution please send me a PM (or email me directly at: ahnen "at" awest "dot" de with the subject of "beta test") to get on the list of extended beta testers (if you haven't done so in the original questionnaire).

Thank you all for your support,

Andreas

PS: Forget to mention that WikiTree will be part of the solution as well

by Andreas West G2G6 Mach 5 (54.3k points)
edited by Andreas West
Hello Andreas,

I am ever so hopeful that you are successful.  I hope this will encourage more people who have auDNA matches to add their ancestry to WikiTree and help them see their shared ancestry in WikiTree.

My fingers are crossed.

Best regards, Peter
Hi Andreas,

I'd be interested in volunteering to help if there's something I can do, but I can't beta test because I know nothing about DNA and have not taken any tests.  My offer is for another pair of hands to help with development.  I have a lot of experience designing and implementing security controls, user interfaces, and documentation for help files.  Unfortunately, since I officially retired 12 years ago, I have not kept up with currently popular languages.  My preference is to use asp classic, but anything I do could be ported to whatever you're using with far less effort than to develop it from scratch.

Shoot me a PM if you want to discuss any ways I could be of help.
Thanks Peter, that's the big plan. More people taking a test and then continuing to work with DNA genealogy is one of the main goals that I aim for. Right now unfortunately a lot of people have either a negative perception after taking a DNA test (based on remarks like "I don't understand anything", "What shall I do now") or are quickly disappointed by the steep learning curve that is required.

I hope that by lowering those barriers to entry I can get people with little computer/DNA skills to communicate and work together on:

- find their common ancestor

- proof and extend their family tree

- finding new relatives

- overcome bottlenecks in their research
Thanks Gaile for your generous offer. Will send a PM
hi Andreas

Do you have an User story?

The scenario I see:

1) login to Gedmatch

2) run a javascript

3) input the  ID of the wikitree profile you would like to check

4) the script Access The Wikitree profile and extracts auDNA users on the profile with gedmatch Id

5) a report is created....

Related questions

+11 votes
3 answers
+15 votes
4 answers
465 views asked Apr 27, 2018 in The Tree House by anonymous G2G6 Pilot (127k points)
+8 votes
7 answers
272 views asked Jan 22, 2018 in The Tree House by Barry Smith G2G6 Pilot (148k points)
+17 votes
3 answers
+4 votes
1 answer
+10 votes
0 answers
+7 votes
4 answers
97 views asked Aug 17, 2014 in WikiTree Tech by Michael Stills G2G6 Pilot (394k points)
+3 votes
1 answer
106 views asked Dec 9, 2013 in WikiTree Tech by Mary Hammond G2G6 Mach 7 (79.1k points)
+5 votes
1 answer
+7 votes
3 answers
244 views asked May 16 in The Tree House by Barry Smith G2G6 Pilot (148k points)

WikiTree  ~  About  ~  Help Help  ~  Search Person Search  ~  Surname:

disclaimer - terms - copyright

...