PhaseAReport

(0) Executive Summary

(0.1) Introduction

AstroGrid is one of three major world-wide projects (along with European AVO and US-VO projects) which aim to create an astronomical Virtual Observatory. The Virtual Observatory will be a set of co-operating and interoperable software systems that:

• allow users to interrogate multiple data centres in a seamless and transparent way;
• provide powerful new analysis and visualisation tools;
• give data centres a standard framework for publishing and delivering services using their data.

The long term vision of the Virtual Observatory is not one of a single software package, but rather of a framework which enables data centres to provide competing and co-operating data services, and software providers to offer compatible analysis and visualisation tools. The first priority of AstroGrid, along with the other VO projects worldwide, is to develop this standardised framework to allow such creative diversity.

However, our intentions are to go beyond this framework. We will develop a working implementation of immediate use to astronomers. As a consortium of data centres and software providers, we will pool resources, including key UK databases, storage, and compute facilities. As a UK e-Science project, our architecture will be firmly based on a data-grid approach: we will make use of grid components produced by other projects and will make our own components freely available to the e-Science community.

This document is the formal report of the AstroGrid Phase A study to PPARC's Grid Steering Committee (GSC). It includes a summary of the project vision, an overview of our intended architecture, a discussion of science requirements, a series of study reports, a summary report on project management and finances, and a description of our Phase B plan.

Following a formal proposal to PPARC in April 2001, AstroGrid began in September 2001 with a one-year Phase A study, with final project funding to be reviewed at the end of Phase A. Following this review, we expect to begin the software construction phase (Phase B) at the beginning of 2003. Although this document is in the first instance intended for this formal review, we intend in due course to circulate it widely. It is available as a single pdf file or as a connected set of pages on the AstroGrid Wiki web site.

(0.2) Project Progress

Project progress has been significant, and the greater part of this Phase A Report documents the project's achievements. Some of the highlights are:

• science requirements
  Through internal meetings and contact with external scientists, we developed an extensive list of requirements, documented as science problems. These were the problems that any VO would be expected to enable scientists to tackle. From these we selected ten key science cases, which the AstroGrid project would take as its drivers.
• architecture
  From the key science drivers, we derived use cases: formal definitions of the steps that a scientist would follow with the VO to tackle each problem. The use cases were then used to derive an architecture for the AstroGrid software framework. This architecture is still under development and is being created fully using the standard Unified Modelling Language (UML).
• external relations
  Our team has had extensive contact with people and projects worldwide, in the VO, grid and e-Science arenas:
  • VO: AstroGrid is a founding member (along with the European AVO and US-VO projects) of the International Virtual Observatory Alliance (IVOA). We work closely with our IVOA colleagues in the definition of interoperability standards. The VOTable data exchange format has been a great success. We are currently engaged in discussions about an image access protocol.
  • grid: We have documented our experiences of implementing of a working data grid and have provided feedback to the developers of Globus' CAS component for OGSA based on our use of early beta software
  • e-Science: Following on from our involvement in the e-Science Database Task Force, we have been chosen (along with myGrid) as early adopters of the OGSA-DAI technologies. We are in discussion with several other e-Science projects (eg DataGrid, GridLab) about re-using some of their components in the AstroGrid VO.
• pilot studies
  Our original pilot studies were quite extensive. The optical/near-IR and x-ray ones were delayed due to late data arrival but the solar, STP and radio studies have achieved their goals, both in terms of providing input to the architecture development and demonstrating the feasibility of certain technology approaches.
• technology demonstrations
  On the technical side, we have established a working grid over several institutions (currently using Globus V2 toolkit and soon to implement beta versions of OGSA) and have begun demonstration projects in:
  • grid authentication and authorization,
  • ontology usage in registry and workflow development, and
  • database access via the grid.
• online collaboration tools
  We have deployed a set of ground-breaking collaboration tools which have enabled team members to share experiences, seek opinions and expert advice, and create an extensive library of documents pertaining to all aspects of the project. These tools are now being deployed on other project web sites.

(0.3) Financial Overview

The total Phase A expenditure up to the end of August 2002 has been approximately £661K. (This is the PPARC expenditure only). Staff costs have been expended through a variety of grants to the consortium institutes. For University based staff, on top of salary, these costs include standard fractions of secretarial and system management support staff, and the standard PPARC grant overhead of 46%. Some of the grants concerned had other minor costs attached. For RAL based staff, time is charged at a uniform staff rate agreed by negotiation between PPARC and CLRC.

The total funded staff effort over the year has been 8.25sy, some of which, the AVO funded staff effort, is at zero cost to PPARC. Grant claims are not all made yet so the accurate final cost is not known, but our out-turn forecast for the one year staff-related costs to PPARC, including all the above related costs and overheads, is £492K.

Non-personnel expenses were provided by PPARC through three grants, two for capital equipment and one central budget grant. These were seen however as representing a single budget controlled by the PM, who set out an overall budget plan in November 2001. Procedures were then put in place for managing these finances. Actual expenditure is very close to that budgeted.

(0.4) Phase B Proposal

Our assumption is that Phase B will commence on 1st January 2003 (ie that Phase A is extended to 31st December 2002 to enable the completion of the system architecture and the demonstration projects). The end date will be approximately December 2004, but in fact the effort profile will not be flat, and a small amount of staff effort will extend into 2005. The end goal of the project is to produce software which will enable the creation of a working, grid-enabled Virtual Observatory (VO) based around key UK astronomical data centres.

Our approach to Phase B will be based upon the Unified Software Development Process (UP, or our own variant of it, UPeSc). The UP is a software development methodology which is both iterative and incremental: each iteration contains analysis, design, code, test and deployment activities and each iteration incrementally adds to the functionality of the system components. This approach will replace the work-package approach of Phase A.

The work we anticipate breaks into a few major strands:

• Continuing research and development;
• Developing the software infrastructure that will make a VO possible;
• Developing user tools to make it possible to do science with AstroGrid (portals, visualisation tools, analysis tools, datamining algorithms, workflow editors, and so on).

The iterative/incremental approach has allowed us to specify milestones in terms of delivered functionality, though these may be switched around or altered depending on the evolving needs of the astronomers who test the early releases of the VO.

We have developed a detailed, component-level breakdown of the work required and, from that, estimates for completing each of those components. Our estimates show a total effort required of 48 staff years. Assuming a two year Phase B, this equates to 24 staff. Note that these estimates do not include management, co-ordination, and support tasks.

At the moment, the project employs 13.1 FTEs spread across 18 individuals who are actively involved in the project. (This does not include the AGLI but does include 3.0 FTEs funded by AVO at no cost to PPARC). Of these, 2.7 FTES across 4 people are primarily employed in non-development tasks - management and co-ordination, science leadership, support tasks (Project Manager, Project Scientist, Web Developer (0.5 FTE), and RAL co-ordination (0.2 FTE)). This leaves 10.4 FTES spread across 14 indviduals available for development and related research tasks.

The project therefore needs an additional 14 development staff. In addition to this, to deliver such a complex programme of software development, we need a new senior position - the Technical Lead who will directly co-ordinate developer tasks. In total then, we are asking PPARC for 15 additional staff.

(0.5) Conclusion

In summary, these are our SCIENTIFIC AIMS :

• to improve the quality, efficiency, ease, speed, and cost-effectiveness of on-line astronomical research
• to make comparison and integration of data from diverse sources seamless and transparent
• to remove data analysis barriers to interdisciplinary research
• to make science involving manipulation of large datasets as easy and as powerful as possible.

And these are are our top-level PRACTICAL GOALS :

• to develop, with our IVOA partners, internationally agreed standards for data, metadata, data exchange and provenance
• to develop a software infrastructure for data services
• to establish a physical grid of resources shared by AstroGrid and key data centres
• to construct and maintain an AstroGrid Service and Resource Registry
• to implement a working Virtual Observatory system based around key UK databases and of real scientific use to astronomers
• to provide a user interface to that VO system
• to provide, either by construction or by adaptation, a set of science user tools to work with that VO system
• to establish a leading position for the UK in VO work

-- TonyLinde - 19 Sep 2002