AG-SAG - Full Meeting #3

Imperial College London, 12 Feb 2004, 10.00-16.00

Supporting Paperwork:

• Current resources to test are linked from the BetaTesting pages
• The key science problems are at ScienceProblems
• The milestones are at ScienceMilestonesR01

Agenda:

Location: 737 Blackett Laboratory, Imperial College London

For travel directions check - http://www.imperial.ac.uk/physics/about/maps.htm

Accommodation information at accommodationlink@imperialNOSPAM.ac.uk or 020 7594 9507/11

There are a limited number of internet connections in the room, and pre-approval is necessary for their use. Send a.jaffe@imperialNOSPAM.ac.uk an email with your MAC address if you'd like to connect (no guarantees).

Meeting minutes prepared by NicholasWalton

Status: Draft 10 Sep 2003

Present:

• Carlton Baugh (CB)
• Bill Chaplin (BC)
• Lyndsay Fletcher (LF)
• Andrew Jaffe (AJ)
• Tony Linde (In attendence) (TL)
• Tom Marsh (TM)
• Nial Tanvir (NT)
• Anita Richards (In attendance) (AR)
• Nicholas Walton (NW)
• Jeremy Yates (JY)

Apologies:

• Shadia Habbal
• Daniel Lennon
• Steve Maddox

Meeting Minutes: DRAFT ...

0. Summary

The meeting assessed required tools and data sets needed in Iteration 6

1. Review of Last Minutes and Actions

The minutes from AGSAG-FM2 were approved.

1.1 Actions

ACTION FM2-3 KN: to clarify the names of machines used in myspace creation.

Closed This process is being simplified in the AstroGrid Iteration 5 release.

ACTION FM2-5 NW: contact Lyndsay, Shadia, Bill re upcoming priority data sets for Itn06 and assess technical issues.

Closed Discussion taking place at this meeting

ACTION FM2-6A NW: detailed workflow for these steps.

Closed These workflows being developed for AstroGrid Itn06 - see TspAgenda06

ACTION FM2-6B BC: look at future Itn06 Solar/STP science priorities, requirements on data access and tools.

Ongoing Initial assessment for the first solar case provided at Itn06SolarCoronalWaves

ACTION FM2-6C KN: investigate licensing issues for 'grid' use of IDL.

Ongoing Astrogrid are now in contact with IDL and are exploring possible solutions involving the use of the ION product.

ACTION FM2-8 KN: server side beta testing (technical) from Itn05

Closed AstroGrid is beginning formal contacts with data centre staff in the UK to organise deployment and testing of AstroGrid components in an operational environment. Further, interaction with the UK astronomical content providers will occur through the VOProcPlus initiative - see e.g. agenda for upcoming meeting (was to have been held 25 Feb 2004 but unfortunately cancelled then).

ACTION FM2-12 NW: poll date with AG-SAG members.

Closed this meeting took place.

2. Report on Demos

2.1 Astrogrid Iteration 3

• Presented at 'Supercomputing 2003' in Phoenix Nov 2003. Interest was shown especially from the more technical audience at this meeting.

NT: how was Astrogrid planning to allow user estimation of job duration. This was considered a useful capability to implement.

SAG: agreed that the Astrogrid demo should be enabled on machines with enough resource to allow a reasonable user experience of the system in a real life context. This would have implications for the sort of hardware, storgae, and data set access provided through the demo. It would be important that the system ran with reasonable response times.

2.2. AVO '1st Science'

• A development of the Jan 2003 aVO demo prototype.
• New capabilities, including access to spectral data enable scientific use of the product
• Jan 2004 '1st Science' release

NW reported that the AVO demo tool is useful for certain science applications now. In particular its use has been demonstrated in the discovery of hi-z qso candidates.

2.3 Demo Feedback

2.4 Realtime demonstration.

NW ran through the AVO and upcoming Astrogrid Iteration 4 demos. The focus of iteration 4 would be on:

• Distributed computing and workflows
• Photometric redshift determinations

AR demonstrated the use of the AVO demo to address a star forming region science problem. This showed the link to ISO IR data employing the IVOA SSA access protocol and the ability to upload user provided data sets.

3. Summary of Iteration 4 Deliverables

see TL slides

4. Iteration 6 Technical Issues

see TL slides

SAG: worries over balance between waiting for stds to allow general features, and delivery of capability now.

SAG: need a user friendly way to access data/resource through the registry.

SAG: Myspace is a key concept of Astrogrid, it would be essential to demonstarte this in a useable fashion as soon as feasible. AJ: need to be able to make user disk space available as MySpace SAG: Astrogrid needs to make available for demo use a few TB of space (located at RAL?).

TL: AstroGrid will see if this is possible

ACTION FM3-4 TL: Ensure adequate Myspace provision in Iteration 6

5. Science Cases for Iteration 06

NW reported on the projected science focus for iteration 6:

• Solar Coronal Waves
  • Feature evolution movies, from discovered data sets
• Magnetic Storm Onset
  • Data source, position through magnetic storm
• HiZ Quasars
  • Advanced sample selection
• Supernova Galaxy Environments
  • Target field properties
  • Multiple Redshift determinations

5.1 Tools and Data in Iteration 06

• Solar Coronal Waves
  • TRACE
  • SOHO XRT
  • SOHO EIT
  • Yohkoh SXT
  • Map software: http://orpheus.nascom.nasa.gov/~zarro/idl/maps.html

• Magnetic Storm Onset
• Deep Field Surveys
• HiZ Quasars
• Supernova Galaxy Environments
  • Redshift tool: Hyperz, Bpz, ANNz,

5.2 Solar Coronal Waves Case

LF noted that there will be a need to worry about epoch of observations.

AJ nted that mutliple data sources would be required.

LF indicated that the minimum data sets would inlcude the use of Yohkoh SXT (held at mssl) amd TRACE

The workflow for this example is located at Itn06SolarCoronalWaves. The idea is to find relevant TRACE data for any coronal ejection. and then use IDL tools to generate image movies.

CB noted that IDL tools were used to produce movies of comological model data sets as well.

NW: summarized that a generic movie tool would be useful for Astrogrid to provide. CB, TM, JY, LF indicated that this woul dbe of direct relevance to their research. The moie tool would enable a user to define an area of interest and the service would get all time data, and put it into a movie (so handles data grabbing, registration, etc).

LF noted that this could be a useful aid for the STP community. A key STP key driver is space weather - how can they predict that there is a major episode of space weather which could impact on e.g. satellites.

ACTION FM3-5.2 WC, LF, NW, CB: to look at movie service for solar data (but impacting on theory data as well).

5.3 Redshift Estimators

The SAG recommended the development of an Astrogrid 'suite of redshift estimators' service.

AJ: hi z gals - dropounts - andrew, nial, nic

measure via colour cuts - but compare against say results of multiple photoredshift runs.

LF: need to make sure it is clear what the code is in any app that AG provides - ie link to the source, or version of the code.

SAG: need to worry about errors, ability to determine limiting mags of the survey etc.

NRT: tool to go an image and extract psf across images. Use of DAOPHOT type thing over a wide area. look at workflow of say sextractor to find areas which are crowded then run say daophot.

6. Science Priorities for Iteration 6

SAG: urgency of accessing Vizier data (catalogues) through the Astrogrid portal.

SAG: agreed that the key issue is good transparent access to data - then as a secondary aim, added value areas such as vizualisation

7. Beta Testing Update

NW reported that the Beta Testing pages were being revamped to allow access to testing notes for the upcoming iteration releases. The main page is located at BetaTesting, with the page BetaTestCurrent pointing to the full details for the current iteration (at the time of writing preparing for Iteration 5, end Mar 2004).

The current release would always be linked from http://www.astrogrid.org/release

NW reported that:

• Limited testing to date as Itn03 had insufficient baseline functionality
• Programme begun in earnest Dec 2003 (Itn03) with a mailshot to the interested group (~25 astronomers, including the SAG)
• Beta test page has links to s/w plus links to Bugzilla for fault reporting. Input would be welcome on what additional info is useful
• Beta test pages will be updated for future iteration releases
• The beta test pages include links to relevant external VO project systems - e.g. NVO, AVO.

8. Future Funding Update

8.1 Astrogrid 2

NW reported on the outcome of the Astrogrid 2 funding proposal.

• Recent feedback from PPARC indicates funding at ~3.7M level till end 2007
  • Focus on delivery of a robust infrastructure
  • Limited funds provided for data mining
  • No funding for data centre component
• Additional SSVO effort (£300K) merged in to AG2
• Full programme details currently being scoped
• Content delivery forum: VoProcPlus
  • Enables interface to processing projects
  • Vista, GAIA, Alma, SDO, GammaGrid , etc

The Astrogrid 2 partners include the current Astrogrid consortium members plus Bristol, Exeter, Leeds and Portsmouth. Additionally RAL joins representing the SSVO activity.

8.2 Euro-VO

• VO-TECH: to be submitted Mar 2004
  • Asking for 2-3M euros
  • AstroGrid led
• VO-EURO to be initiated summer 2004
  • umbrella of funded national programmes: AstroGrid, CDS/F-VO, ESO, etc
  • ESO to provide positions to start the VO-FC

9. Data Mining and Visualisation in Astrogrid 2

NW reported on possible areas of work in this area in the Astrogrid 2 programme:

• Visualisation of Multivariate data
  • e.g. Xmdvtool, gnuplot, Mirage ...
• Data mining
  • e.g. Weka, Principal component analysis, stats
• Data Exploration services
  • Workflow of data mining and visualisation services
• Key issue for SAG
  • Priority uses now, future uses
  • Requirements: cluster analysis, correlations, etc

AJ: Planck have hired a company to write a vizualiser for data on a sphere - so hardware specific - hard to see this running through Astrogrid. But - perhaps if we had more experience of say higher order viz that might be good.

TM: find all objects that have varied by more than 5 mags over the whole sky?

TM: fourier analysis - for power in a spectrum.

AJ: Gobi - 3 d data sets

CB: Tipsy - n-body viz tool

SAG: agreed that the issue is for Astrogrid to provide easy interfaces which allow the interface of externally provided dataa minining applications, making full use fo the access to data and compute resources provided through Astrogrid.

In discussion the SAG considered science priorities more generally for Astrogrid. Their current opinion is that Astrogrid should focus on the delivery of a robust infrastructure, providing wide access to data resources, with suitable interfaces to allow provision of advanced tools with which to manipulate these data resources. These 'tool' services could be internally provided by AstroGrid, although the SAG hope that they will be increasingly made avilable by external providers.

10. AOB

The next meeting, AGSAG-FM4, would be scheduled for July/August 2004.

ACTION FM3-10 NW: poll date with AG-SAG members.