Newly Reduced Hipparcos Astrometric Catalogue

People Involved

The following people are involved in this tools call:

• Dugan Witherick (DKW) - will be setting up the service
• Floor van Leeuwen (FVL) - the data provider
• Guy Rixon (GR) - for technical support
• Kona Andrews (KEA) - DSA Help

General Aim of the Tools Call

The main aim of this tools call is to make the Newly Reduced Hipparcos astrometric catalogue available through the AstroGrid virtual observatory. This will be one of only two ways to access this data, the other being via the DVD with the book. Following the successful installation in a DSA and registration of the service, Python and Taverna workflows will be written to demonstrate querying the catalogues and potentially cross-correlating them with other big surveys such as SDSS and UKIDSS.

Deadlines

• There is an astrometric conference at the end of October 2007 and it is desirable to be able to query the catalogue through AstroGrid by then.
• AstroGridProjectMeeting13 on the 13-14 December. Ideally the workflows should be ready to demonstrate by this stage.

More details Plan (by DKW)

My plan for this tool call is as follows:

• Install a DSA on the UCL community (uk.ac.ucl.star) to supply the hipparcos catalogue to the VO
• Register the DSA onto the UCL publishing registry
• This UCL registered service will temporarily act as the main service but I would like to move the service to Cambridge (with GRs help) and use UCL as the backup service.
• Write python and Taverna workflows to query the service and cross-match with SDSS

Update by DKW as of 6th February 2008

In the previous month the Hipparcos service had a number of additional tables added to it. Some of these tables are copies of those provided by the original Hipparcos and Tycho catalogue set and some were provided by Floor as identifier translations. These new tables were imported into the mysql database running on astro.phys and the metadata file on astro2.phys was modified to provide details about these catalogues. The multi-cone service was additionally switched on. The updated service is now visible in the galahad registry.

It was discovered just before Christmas that at high RA the cone search of the hipparcos data set was returning an incorrect region. This has now been corrected by installing an updated DSA (2007.2 I think).

Update by DKW as of 14th November 2007

The hipparcos DSA is now running again and the cone search and ADQL parsers appear to be working correctly.

Following the update of the 25th October, it was discovered that although the cone searches were working correctly, the ADQL parser was not accepting namespace formatted ADQL/x queries. KEA worked with Jeff Lusted to fix this problem but on installing the fix the problem with the failing Installation Test returned. It is now believed that the problem may originate from running the DSA on a machine with multiple CPUs running an SMP kernel. Since there are DSAs installed on servers with multiple processors it may be that the problem is also related to running the Scientific Linux operating system although this is not to say that the problem will not occur on multi-CPU machines running other linux distributions.

The simplest and quickest way to work around this problem was to install a virtual machine on the astro.phys server which would just supply the hipparcos-dsa. Ubuntu 6.06LTS was installed in this virtual machine, along with apache2 and tomcat 5.0.28 (on Kona's advice) - the hostname of this virtual machine is astro2.phys.ucl.ac.uk and so the new DSA can be found at http://astro2.phys.ucl.ac.uk:8080/hipparcos-dsa/ . I have made very little changes to the DSA configuration, only to reflect the change in its location. The mysql database remains on astro.phys and the new DSA connects to it remotely. So far I have not observed any stability issues with this new DSA. The change in the location of the DSA has meant that the service needed to be re-registered and the uk.ac.ucl.star publishing registry needed to be reharvested (this has been done).

Update by DKW as of 25th October 2007

KEA has managed to find and fix the bug that was causing the Installation Test to fail. The server is now up and running and upon confirmation from FVL that the metadoc is okay, will be registered with the uk.ac.ucl.star.registry and harvested by Galahad so as to become visible to the rest of the VO.

Update by DKW as of 22 October 2007

• A DSA has been installed on the UCL community to provide the hipparcos database. http://astro.phys.ucl.ac.uk:8080/hipparcos-dsa/
• 5 tables from the catalogue have been added to a MySql database on the server. The tables are:
  • MainCat - the main astrometric catalogue
  • MainCatInd - index table for the astrometric catalogue
  • SevenPCat - supplementary data provided for the 7-parameter solutions
  • NinePCat - supplementary data provided for the 9-parameter solutions
  • VIMCat - supplementrary data provided for the Variability Induced Motion solutions
• A meta-doc has been edited to describe the data - the doc provides units and UCDs for most columns in the tables (click hipparcosnr-metadoc-v5.xml to download the most recent version)
• The DSA has been configured to use mysql and the metadoc.
• FVL is checking the meta-docs (http://astro.phys.ucl.ac.uk:8080/hipparcos-dsa/viewMetadata.jsp) to ensure that he is happy with the names of columns and the descriptions provided.

Current problems (by DKW)

Essentially the service is configured and is ready to be registered on the UCL publishing registry so that it becomes visible to the rest of AstroGrid. However, although most of the DSA test completes successfully without complaint, the Installation Self-Test is intermittently failing, complaining of a problem verifying the meta-doc. The error message that it returns is not consistent and sometimes the test completes without any error and without any modification to the configuration. I contacted KEA about this problem and it is possible that it is a more frequent version of the bug http://www.astrogrid.org/bugzilla/show_bug.cgi?id=2225. KEA has been given full access to the server and is investigating (especially since it is similar to the problem LEDAS have).

-- DuganWitherick

Moving the service to Cambridge

"I would like to move the service to Cambridge (with GRs help) and use UCL as the backup service." -- DuganWitherick

This new copy of the service would presumably be on a CASU machine. If a Solaris host is used, the MySQL that comes with the OS would be appropriate. cass123.ast.cam.ac.uk already has this enabled.

If we want a live, back-up copy at UCL, then there are three ways to go.

1. Install at Cambridge, and register in the CASU registry, a separate DSA that happens to look at the same data.
2. Install a DSA at Cambridge and make it mirror the CEA application from the the UCL installation.
3. As option 2, but have the UCL service mirror an application at Cambridge.

The differences between the options affect the "branding" of the service. With the first option, users see in VOExplorer two separate data resources. In the second option they see one resource, published by UCL, but Cambridge does some of the work. In the third option, the resource is republished by CASU.

In fact, it's not certain what would happen with two DSA installations serving the same resource. It may be that the AR will always pick the first one in the registry (for some subtle value of first that we'd need to find by experiment); or it may distribute calls between the two. Therefore, it might be best to have two, separately-branded services running at once.