Towards Easier Cross Matching with Astrogrid: AstroFederator

Classes of Cross match

there are a number of possibilities where we'll be cross matching tables of various sizes.

Very Small

one or a few objects against many tables

• use AstroScope

Small to Small

a few thousand objects against a few thousand objects

• use STILTS
• STILTSXMatchTool

Small to Medium

a few thousand objects against tables of upto 1M objects

• can use STILTS, but note that in a CEA-like context moving the larger catalogue to wherever STILTS is running may be expensive

Small to Large

a few hundred objects against tables of over 1M objects

• if the larger table has some cone-search like interface, fake the crossmatch by doing one cone search for each row of the smaller one. STILTS multicone automates this at the client side for actual Cone Search services, and I think Eduardo has done something similar using python scripts. Mark and Kona are thinking about/working on server-side functionality that does the same kind of thing, which should be more convenient and faster for users.

Medium to Medium

up to 1M objects against up to 1M objects

• can use STILTS, but note that in a CEA-like context moving the large catalogues to wherever STILTS is running may be expensive

Quite Large to Quite Large

several million objects in either or both tables

• The algorithms in STILTS could be souped up to cope with this on request, I think -- MarkTaylor

Really Large to Medium

hundreds of millions against hundreds of thousands

• This could maybe be done by ingesting the medium table into a database holding the large one and doing a match in the database.
• It could be done outside a database using custom crossmatching code, if the large table was available in non-database form.

Really Large to Really Large

hundreds of millions against hundreds of millions

• Since it is not feasible to move either table from one place to another, this can only be done by a data centre holding both catalogues. It would presumably be done once for each pair [that may be] of interest, rather than on user demand.
  • ROE have some pre-calculated neighbour catalogues generated along these lines: see WSA crossmatching.
  • CDS have for several years been threatening to prepare pre-calculated indices and hence a service which will allow spatial crossmatches between any of their catalogues, even the large ones. This has so far failed to appear however.
• The data centres will have their own run-once way of doing this match. One good algorithm for sky matches is using the sort-sweep algorithm described by Devereux, Abel, Power and Lamb.

Techniques using AstroGrid

• SimpleCrossMatchTool for basic positional matching of catalogues
• SimpleCrossMatchTaverna - Via a Taverna workflow

Other Approaches

Openskyquery

• http://www.openskyquery.net
• 'Probabilistic Cross-Identification of Astronomical Sources': Budavari & Szalay - http://www.arxiv.org/abs/0707.1611

Starglobe

• http://www3.in.tum.de/research/projects/StarGlobe/

Multi-Catalogue Multi-Cone Search

• http://www.g-vo.org/mcmcs/