Catalogue publishing

Participants

• Tim Garn
• Anton Smit
• Stewart Williams

FLSGRT database

I worked on creating a registry and publishing a database that could be found with AstroScope.

Step 1)

Install Java, Tomcat

• http://wiki.astrogrid.org/pub/Astrogrid/RadioAgenda/tomcat_radionet.txt

Step 2)

Create the registry

• http://wiki.astrogrid.org/pub/Astrogrid/RadioAgenda/registry_radionet.txt
• Install the WebApp^?
• Set up the database location
• Check that it loaded correctly
• Register it with the main AstroGrid external registry
• Once it has been harvested, your registry should show up on the main list

Step 3)

Set up the DSA / catalogue service

• http://wiki.astrogrid.org/bin/view/Astrogrid/RadioAgendaDsaInstallNotes
• Set up WebApp^?
• Get default files
• Fill in relevant details
• Restart Tomcat, check that the catalogue page shows up ok
• Configure it
  • http://beacon2.star.le.ac.uk:8080/astrogrid-dsa-catalog/configure.jsp
  • Check Java, Tomcat are working
  • Test the default configuration
  • Install JDBC driver, configure the connection
  • Now import your database into MySQL
    • See Step 3a!
  • Customise the properties file
  • Write a detailed .xml metadoc file
    • http://beacon2.star.le.ac.uk:8080/astrogrid-dsa-catalog/admin/GenerateMetaDoc
    • Customise file, adding UCDS (http://beacon2.star.le.ac.uk:8080/astrogrid-dsa-catalog/docs/the%20UCD1+%20controlled%20vocabulary), units, descriptions
  • Test the configuration again
    • Everything should work now
• Register your datacenter
  • http://beacon2.star.le.ac.uk:8080/astrogrid-dsa-catalog/admin/register.jsp

Step 3a)

Importing a FITS database into MySQL

• This allows a cone search or other adql query to be run
• http://wiki.astrogrid.org/bin/view/Astrogrid/DataAccess20061206
• Install MySQL
• Log in, create database
• Set permissions
• Install STILTS
• Import the FITS file into STILTS
• Connect to the database, check the columns / rows are as expected

Step 4)

Check your data works

• Nothing will show up on AstroScope until your registry name has been harvested
• Search your catalogues position - hope something appears!
• Correct any xml errors found

Problems you may encounter:

• All sorts of installation problems - find a friendly SysAdmin^?
• You need to shut down and restart Tomcat on a regular basis

Splatalogue

Splatalogue is a transition-resolved compilation of the JPL, CDMS and Lovas/NIST lists (http://www.splatalogue.net). From the splatalogue sales brochure:

The splatalogue in particular is an attempt to collate, rationalize and extend existing resources. The JPL, CDMS and Lovas/NIST line lists provide an enormous amount of data – just throwing them together generates over 3.5 million transition data entries across almost 700 molecular species. The JPL and CDMS databases do not describe the transitions in a user-friendly way, and where the catalogues overlap, the descriptions have to be compared and resolved to be consistent. The Lovas/NIST list tabulates observed interstellar transitions, but it somewhat out-of-date. One of this project’s goals is to update it - the splatalogue will contain at least one example of every detected line.

Results

Publishing a catalogue really is just a matter of following the instructions. There were no steps where I thought 'I don't know why I'm doing this'; I think publishing catalogues is not far from being as simple as it can be. Impressive! The only problems now lie outside Astrogrid, in DB design, metadata assignments etc.

The transition-resolved entries in the Splatalogue are now published on the VO. For example, here are the results of a search for 13C transition lines:

And here are the results of a search for lines between 100 and 105 MHz.

Problems encountered

• The Splatalogue database is normalized into a number of tables. This was a problem, as we want ADQL queries to be as simple as possible; astronomers shouldn't have to select across multiple tables to get the data they need. The solution was to create a view that consolidates the scientifically useful information into one table; this is the only view that is exposed to the VO.
• MySQL 4.x doesn't have view support, so I had to upgrade the Splatalogue database to MySQL 5 to get a scientifically useful perspective of the data.

Next steps

• More fields (degeneracy, degrees of freedom etc.)
• Better metadata! Talk to some experts..
• Try creating a Simple Line Access Protocol (SLAP) server by HTTP POSTing ADQL queries to the DSA

Wishlist

• The column names published by the DSA are the same as the database field names, which is a problem if the column names are shorthand or self-explanatory. It'd be good if we could redefine the published column names in the metadoc definition.

Goals

To recreate all of this when we get home

Access to the WSRT catalogue

• First we installed and configured a Registry Server at the internal test webserver at the ASTRON institute.
• Installed and configured a DSA application to get access to the WSRT database.
• Created some glue to select the correct information from the WSRT database.
• Tested the created link by doing a cone search on the WSRT catalog and getting the correct results back.
• Future: Implement correct validity checks and user access and an indepent database.
• Future: Make all of this accessible through the VO interface to tools like VOScope

Some small notes from the WSRT tream:

1. Good to see that the applications work with our system (tomcat 5.0 and MySQL 5.0)
2. We found a bug in the DSA using multiple virtual hosts in Tomcat 5.0. But we found a workaround
3. We have had very good support by the AstroGrid team.