AstroGrid Caltech SDSC Technical Meeting

Dates and Location

• 12-14 Feb 2007: CACR, Caltech (start 9am on Mon 12 Feb)
  • for directions to the building, see http://www.cacr.caltech.edu/visit_contact/index.cfm (Room 100 is just up the steps in the picture and first on the right.)
• 15-16 Feb 2007: SDSC, San Diego (end 4pm Fri 16 Feb)

Agenda and Discussion Topics

AstroGrid Aims

1. To look at a number of IVOA Interoperability issues, focussing on the IVOA GWS-WG activities. In particular we hope to look at:
   • interoperability between the AstroGrid MySpace and the Caltech and SDSC VOSpace implementations. This would include VOSpace as implemented in front of SDSC's SRB
   • single sign on interoperability - this would build on the earlier demonstrations of authorised transfers between UK/USA VOSpace's
2. On demand application implementation - utilising the IVOA UWS standard for application presentation. Montage is a toolkit develped at IPAC to create mosaics from FITS images. It is already been used at IRSA to produce postage stamps from datasets hosted at IPAC, e.g. 2MASS, SDSS, IRAS, Spitzer and COSMOS data (see here). The aim of AstroGrid is to deploy Montage (v3.0) at SDCS (and possibly Caltech) to be accessible through a UWS interface (in this initial stage this would make use of the AstroGrid CEA component). This would then allow for user submission of jobs to Montage through the IVOA AstroRuntime (to be tested in this first instance using the AstroGrid Workbench). The user would then be able to run the application on demand with his/her own FITS images. Key data as input to Montage for initial tests would be:
   • Spitzer Images. These are hosted at Caltech and Montage has already been used to mosaic these images in a number of Legacy Surveys. This could be a initial test to check that the service works.
   • INT IPHAS (see http://www.iphas.org). Montage has also been used to mosaic images coming from the INT Wide Field Survey (WFS; mosaiced images released by SWIRE collaboration). IPHAS data has been processed using the same pipeline as WFS.
   • SDSS. Particularly the strip with repeated observations.
   • WFCAM. There is public data available here

Also investigate similar deployment of the Argonne 'image stacker' application - Ioin to advise.

Caltech Aims

• IPAC mosaic service; ROME, Pegasus, future
• Nesssi services for mosaic, cutout, coadd; certificates, usage models, future
• Raicu/Foster service for coadd; future
• How to integrate these three approaches to do science and bring astronomers to the VO
• How to use UK certs in US services and vice versa
• Using VOSpace to keep intermediate image products
• Towards an Astronomical Image Analysis Workbench (that astronomers will actually use!)
• Integrating asynchronous secure services with Plastic?

SDSC Aims

• VOSpace interface to the SRB data grid
• Demonstrate the new iRODS environment for managing distributed data

Pre Visit Activities

AstroGrid

• CEA wrap Montage

At SDSC

• Requirements for s/w install at SDSC - see email from GTR to Reagan Moore 2/2/2007

Particpants

Argonne

• Ian Foster (email)
• Ioan Raicu

AstroGrid

• EduardoGonzalezSolares
• DaveMorris
• KeithNoddle
• GuyRixon
• NicholasWalton

Caltech/ CACR

• Matthew Graham (email)
• Roy Williams
• Sarah Emery Bunn

Caltech/ IPAC

• Bruce Berriman
• John Good
• Joe Jacobs (Mon-Tues)

SDSC

• Arun Jagatheesan
• George Kremenek
• Reagan Moore
• Krishna Muriki

Meeting Notes

CACR: recap as of 15.00 PST 14 Feb 2007

Achievements

• Montage 3.0
  • Aim - interface mosaic capability through AG CEA frontend
  • Achieved: CEA application which actions a montage 2mass mosaicing service (RA, Dec, Size, Band - choice of J, H, K). * Tested for sizes upto 1 degree: gives a ~100MB o/p file
    • 2MASS mosaicking, ivo://astrogrid.cam/Montage/prototype/2MASS-mosaic
    • The actual programme polls for status of the run, this gets viewed by the CEA
    • The actual mosaic app runs at IPAC, the CEA component is located at Cambridge (could be easily migrated to alternative locations)
    • The IPAC service will be migrated soon to an operational server (upgrade route: to a cluster)
  • Time taken to develop - ~4 hours fro John Good (on the IPAC side) and ~1 hour for Guy Rixon (for the CEA interface)
  • S/w installs - Cambridge - one command line adaptor programme and one webserver app in an existing Tomcat. At IPAC - one command line app wraped as a CGI service and one 'agent' programme (this running in Cambridge)

• NESSSI mosaic service
  • Aim - interface mosaic capability through AG CEA frontend
  • Achieved: security components identified, and what does what

Next Steps

• Montage
  • Modify service so that it takes two std inputs - header file and list of images (e.g. from a SIAP o/p, thus generalise somewhat to other survey's, image sets) and users identity (to allow for service usage tracking)
  • Move service end point to a larger scale cluster - s/w from cambridge could be set up on a server at IPAC. VOSpace maybe defered - in the meantime mosiac o/ps would be pushed out to an external VOSpace
  • Science focused user help pages
  • AG to provide John Good - example table of IPHAS optical images

• NESSSI (see http://us-vo.org/nesssi)
  • Location of service compute location - SDSC identified
  • i/p table giving list of files - need to look at the o/p from a SIAP call - probably need to do some manual editing
  • install AG server on a CACR machine - application description file -

• RW: CACR montage service will be made available as a AG CEA app (to be implemented by Matthew Graham)
  • access to the AG account on web8 at SDSC?
  • where to install the application source at SDSC?

Issues

• Montage
  • asynchronous messaging
  • more work is required to define the standard interface (e.g. what goes into the application - is the header/ list of images correct)

SDSC

Issues

• set up an agent to handle authentication - for access to SDSC - a gateway - discuss with Nancy Wilkins-Diehr
• how to get usage rights on SDSC - in collaboration with a USA PI

• Interface to iRODS (see http://irods.sdsc.edu/index.php/Main_Page)

Next Steps

• AG VOSpace 1.0 will investigate the interface of VOSPACE (at the moment AG's MySpace) to SRB utilising Jaragon (Java API library for SRB/iRODS)
  • contacts: Dave Morris and Arun Jagatheesan/ Lucas Gilbert
    • after dave/ arun discussed the issue, this was indeed thought a sensible route forward
    • dave: there are concepts in iRODS/SRB which are not in VOSPACE

• • SRB/ iRODS, principle developer - Raja (Arcot Rajaseka)

• Astroportal - see http://people.cs.uchicago.edu/~iraicu/research/AstroPortal/ - Ioan Raicu
  • downloading SDSS DR5 - see http://sdss.ncdm.uic.edu/

• Image extraction service at SDSC: aim to provide service accessible via USA or UK (Europe) access mechanisms (e.g. NESSSI or Workbench/Taverna)
  • Science aim: astronomer has a list of positions (RA, Dec, size, band, survey) - this goes then to the service which generates the mosaic (cutouts) of SDSS, 2MASS, and then runs Sextractor on those. returns are the cutouts and catalogues of those cutouts.
  • Upgrade plan: UK based WFS, IPHAS, UKIDSS image colllections and indexes
    • copy images ot NGS
  • Implementation plan:
    1. TeraGrid^? account - for gtr - Action: email roy to authorise under NVO account in the shortterm
    2. Mosaic web service (this as prototyped at Caltech, 14 Feb 2007 - see above) - Action: Contact John Good
    3. Image index service - 2MASS (john Good), SDSS (Ioan) - these are available
    4. SIAP cutout service (being developed by George Kremenk and Ioan)
    5. Application binaries - Montage and Sextractor to run at SDSC
    6. Access via AG
       • authenticate to teragrid
       • invoke on teragrid (nvo account for example)
         • image index - port from webservice / query to SRB
         • cutout
         • mosaic
         • source extraction

Timeline for future activity

• OGF20 (Manchester, UK): 7-11 May 2007
• IVOA Interop May 2007 (Beijing, China) - 14-18 May 2007
  • Astro-RG session - update of activities underway
• ADASS XVII (London, UK) - 23-26 Sept
  • Science paper describing worth and use of the services being developed here
  • demo of anything that works by then
• Meet in Cambridge - week 1 Oct 2007
  • Caltech + SDSC + Ioan to IoA ?
• OGF21 (West Coast, USA) - 15-19 Oct 2007
• SC07 (Reno, USA): 10-16 November 2007
  • Tech paper submission deadline 9 Apr 2007: Yes - USA/UK joint paper describing AG/Caltech/SDSC mosaic/image catalogue service
• AAS - Jan 2008

Logistics:

CACR, Caltech

• Address: Center for Advanced Computing Research, California Institute of Technology, MC 158-79. 1200 E. California Boulevard, Pasadena, CA 91125
  • Directions etc at http://www.cacr.caltech.edu/visit_contact/index.cfm
  • Local contact on arrival - Sarah Emery Bunn on 626-395-4622
• Meeting room 100 of the Powell-Booth building where CACR is housed

• Wireless: open network - MAC addresses not needed beforehand

• Hotel info: http://admissions.caltech.edu/visiting/accommodations/ (try the Sheraton Pasadena)

SDSC, San Diego

• Address: University of California, San Diego, San Diego Supercomputer Center. 9500 Gilman Drive, MC 0505, La Jolla, CA 92093-0505
• Meeting room 233

• Hotel info: http://www.sdsc.edu/about/Visitorinfo.html (try the The Residence Inn)