PhaseAReport

(8) Pilot Programme Report

(8.1) The AstroGrid Pilot Programme

The AstroGrid Pilot Programme was designed to complement the Phase A technology assessment workpackages. While the latter were intended to evaluate some of the technologies likely to be deployed to help meet AstroGrid's science requirements, the Pilot Programme was designed to help elucidate those requirements in more detail, by developing prototype systems to deliver small portions of AstroGrid's desired functionality. These prototype systems were intended to be constructed using technologies that staff were familiar with; the accent here was on learning from the experience of producing pilot software to meet aspects of AstroGrid's needs, without necessarily doing so in the way that AstroGrid would ultimately meet them. An important goal from the outset was that all pilots should proceed to the point of delivering software that could be used by test users to do real science, but, equally, it was stressed that no undue effort should be expended in producing pretty user interfaces, etc, as the lessons to be learnt concerned the delivery of new functionality, not its presentation to users.

(8.2) Pilot Selection

A set of five pilot topics was selected, one each for the five broad fields covered by AstroGrid, namely

• optical/near-IR astronomy: Large object catalogues
• X-ray astronomy: Association methods
• radio astronomy: Fourier data
• solar physics: Data selection from summary information
• solar/terrestrial physics (STP): Time-series data

The reason for this selection was two-fold. Firstly, each of these areas has specific requirements, and by selecting a pilot topic of particular importance to each of the five areas we could help ensure that the final AstroGrid system meets the needs of its full user community. Secondly, it was intended that the test users for the pilots would be selected from the likely "early adopters" of the final AstroGrid system, so, by covering all disciplines, it was hoped that the Pilot Programme would start to engage appropriate members of all communities in the work of AstroGrid.

The five pilots are described in turn below, together with an Example Science Case for each, illustrating the type of research problem each is intended to address.

(8.2.1) The Optical/Near-IR pilot - Large Object Catalogues

Example Science Cases:
A scientist wishes to search for halo white dwarf stars, which requires selection criteria making use of both colour and proper motion information for a large sample of stars (for these are rare objects). This can be achieved by querying the multi-epoch/multi-colour dataset produced by federating the Sloan EDR dataset with the SuperCOSMOS Sky Survey coverage of the same region.
A scientist wants to determine the optical and infrared colours of an object, e.g. an X-ray (XMM) or radio (FIRST) or far infrared (ISO) source. This can be achieved by querying the multi-colour dataset produced by federating the INT WFC five colour optical dataset with the 2 colour INT CIRSI dataset of the same region.

The optical/near-IR object catalogues to be included in the AstroGrid system greatly exceed all its other databases in size, so there was a clear need for a pilot that addressed the practical problems of federating large object catalogues. Initially, it was intended to study this using two methods. Firstly, data from the SuperCOSMOS Sky Survey (1) covering the fields of the Sloan Digital Sky Survey (2) Early Data Release (EDR) would be federated with the EDR data themselves, using a version of the SDSS science archive software (called SX, (3)), modified for use with the SSS data model. Secondly, catalogues of objects derived from INT imaging with the Wide Field Camera (optical) and CIRSI (near-infrared) would be federated by making them both accessible via the VizieR (4) system developed by the Centre de Données astronomiques de Strasbourg (CDS (5)). The functionality provided by the two approaches could then be compared by providing access to both federations to test users, who can assess how well they each match the needs of scientists using future federations of large object catalogues in the VO. In the event, the evaluation of VizieR in this regard was performed by AVO (6)Work Area 2 (Interoperability), not as part of this pilot: the results of this work will be reported elsewhere and we shall only consider the SSS-SDSS federation in what follows.

(8.2.2) The X-ray pilot - Association Methods

Example Science Case:
A scientist wishes to determine the variation of some X-ray hardness ratio as a function of X-ray flux and optical/near-infrared colour, to constrain models for the properties of the population of obscured AGN. To do this requires the association of objects in X-ray and optical/near-infrared catalogues, followed by the selection of subsamples of associated sources on the basis of X-ray properties.

The association of entries in different databases identified as being observations of the same astronomical object lies at the heart of the Virtual Observatory (VO) concept, but it appears to have received little attention from the VO community to date. Astronomical data has a natural indexing - spatial location in the sky - which aids the making of such identifications, but, in many cases, association by spatial proximity alone is not adequate. This is most clearly the case in situations, such as the determination of optical counterparts for infrared sources from ISO or submillimetre sources from SCUBA, where the angular resolution of one catalogue is so poor, and the surface density of objects in the other so high, that there can be many objects from one catalogue located within the positional error ellipse of each source from the other. In this case, the easy identification of the true optical counterpart is not possible, and a probabilistic method must be used, to assess which of the candidate counterparts is the most likely to be the true match. Astronomers already use several such probabilistic prescriptions, but they are frequently used in situations in which each possible association can be checked manually for plausibility. The VO provides a much greater challenge. Not only does the size of the datasets to be made available by the VO mean that associations could be sought between databases containing millions of objects each, raising concerns about the scalability of the assocation techniques commonly used, but there are also issues relating to how the method employed to make a set of associations can be recorded so that a later user can judge whether s/he can employ them with confidence, rather that recomputing them all anew. This pilot was designed to start addressing these important issues, through seeking optical counterparts for sources in the first XMM-Newton catalogue, produced by the XMM-Newton Survey Science Centre. This catalogue would be large enough (several tens of thousands of sources), and X-ray and optical data are disimilar enough, that this pilot can address several aspects of the general association problem (discussed in (29)) at once.

(8.2.3) The radio pilot - Fourier data

Example Science Case:
The incidence of AGN in star-forming galaxies is an important test of theories of galaxy evolution. An astronomer addresses this issue by taking X-ray (e.g. Chandra) and optical/near-IR (e.g. CFHT or Subaru) catalogues, selecting a sample of candidate AGN and generating a radio image around the position of each from archival visibility data. The radio structure on various scales (including any evidence of mergers) and the radio spectral index can then be used to reveal starburst regions and obscured AGN.

High resolution radio interferometer arrays produce data sets which are samples of the Fourier transform of the radio sky. These 'visibility data' can be processed in different ways depending on the astronomical requirements. Since sampling in the Fourier plane can be sparse, non-linear deconvolution is a necessary and critical step in the production of images which can be easily interpreted. Although the field-of-view of high resolution interferometer arrays is often large, the information content can be significantly smaller, due to the limited Fourier sampling. It is therefore more efficient and more productive to maintain access to the data in the Fourier plane and produce images or data products on demand, with options as to whether to carry out a particular deconvolution, fit models to the data in the Fourier plane, combine with data from another interferometer, or to select a particular region on the sky. This pilot, taken together with work from AVO WA3.3 (Scalable computing and storage), was designed to produce a test-bed system that allows users to access visibility data remotely and then launch image production on-the-fly, tailored to the requirements of the specific science goal, and implemented in a parallel computing environment to enhance speed.

(8.2.4) The solar pilot - Data selection from summary information

Example science case:
A scientist is studying a particular X-class solar flare. S/he wishes to identify and explore the data available which cover its site during the 24 hours leading up to the event. GOES X-ray flux data can be used to identify the timing of the event but the location is often very approximate and will be taken from the reported position of the associated active region. Catalogue data for all observations taken during the required time-period can be used to refine the flare location. This stage may also require access to original data (possibly in the form of quick-look images made on-the-fly) rather than relying on metadata. Due account will need to be taken of solar rotation during the 24 hrs. With accurate time and location parameters, the search for supporting data will be refined. The morphological and photometric history of the region will be investigated using imaging data and its plasma properties can be characterized from spectroscopic data found to match the event. Magnetogram data will be needed in a form suitable for automatic spatial- and temporal-registration with any monochromatic images available from a variety of space and ground-based sources.

This pilot is principally a test bed for improving methods of data selection in solar physics. This is important because it is not typical in solar physics for all data from a particular experiment to be automatically pipeline-reduced all the way to science-quality products. What is more common is that summary data products are made available, together with catalogues listing observational parameters, which the user may then interrogate, to select datasets containing observations of solar features of the desired sort, and s/he can then request a copy of the processed data from the primary archive for that experiment. The main goal here, therefore, is to develop mechanisms for making it easier for the user to select which datasets are of interest, since this is the stumbling block to doing science with the data. This pilot involves much more interactive work than the others, so it does more to address on-the-fly federation of datasets than the others, which are principally implementing static federations, and much of its work is undertaken in collaboration with the European Grid of Solar Observations (EGSO,(7)).

(8.2.5) The STP pilot - Time-series data

Example science case:
A scientist wishes to study the propagation and effect of a coronal mass ejection. This requires use of: (i) the coronagraph on SOHO(8); (ii) upstream solar wind measurements from ACE; (iii) Cluster(9) plasma and field measurements near the magnetopause; (iv) plasma composition measurements in the mid altitude cusp; (v) ring current enhancements, in situ, remote sampling and ground-based geomagnetic indices; (vi) position and timing information. These data sets range from simple scalar time series data, to sequences of images and higher dimensionality arrays. They currently have different locations, query specifications and are returned in different formats. The data may need to be transformed into a consistent co-ordinate frame or combined to produce ancillary products. A uniform, and flexible, metadata specification is therefore crucial to ensure that manipulation of data from different archives can be done in a consistent and correct way.

This test federation was designed to investigate the federation of heterogeneous time-series data. This is of particular relevance to Solar Terrestrial Physics (STP) data sets due to the large number of in situ, multi-point and remote sensing measurements made across a wide range of scales in both time and space. STP data sets are relatively small compared to the other AstroGrid domains. The main issues to address come from the complexity of the analysis and in particular the need to locate, search, extract, manipulate and combine multiple data sets. It is also important to consider the international perspective since many of the key datasets that will be required by UK STP scientists originate from non-UK instruments and facilities.

(8.3) Highlights from the Pilot Programme

(8.3.1) The STP pilot - Time-series data

The teams running the STP and solar pilot conducted a wide-ranging questionnaire in conjunction with ESA's SpaceGRID(10) initiative, addressed to a wide cross-section of the international solar system research community and elicting more than one hundred responses. The details of these responses are beyond the scope of this document - a summary is available (11) - but a few points from it are worth noting here. This exercise produced some quite explicit performance requirements, not specified so concretely yet elsewhere within AstroGrid, for example: the system should provide feedback on an action with 30s; simple, online tasks should be completed within an average time of a minute; and complex, offline tasks should be completed within an average time of 24 hours. Interestingly, this survey also identified some Intellectual Property Rights issues not discussed much within AstroGrid: some respondents thought that there should be a possibility of keeping workflows and query results within AstroGrid's MySpace.

The results of this requirements survey influenced the design of the pilot's top-level architecture, which is sketched below:

where the rectangles are programs, the ellipses data stores, the dotted lines demarcate abstract entities (i.e. the Resource catalogue, an arbitrary Data resource, the Query handler and the User interface) and the arrows indicate major data flows. Where possible, this architecture was implemented using existing software, such as the Solar Terrestrial Physics Data Facility (STPDF (12)) system, and using data sources selected from those of the UK Cluster Data Centre (UKCDC (13)) and the World Data Centre for Solar-Terrestrial Physics (WDC (14)), already on line at RAL. These comprise about 35 million time series records in total, of several types: from UKCDC would come data from ACE, as well as GOES Key Parameters, while the WDC would provide geomagnetic indices, Dst and aa data. The UKCDC data are held as CDF files (one per day), while the WDC data are stored as ASCII or binary tables; onestrength of STPDF is that it can provide a uniform view across this heteregeneous set of data resources.

The full pilot system looks like this:

and involves three servers at RAL:

• The AstroGrid Server: is a Sun workstation, with STPDF installed on it. The local reference directory and data catalogues provide information stored on the local server. The master reference directory provides information about the location of network-accessible resources. In the case of the pilot work, this contains entries for the UKCDC and WDC servers. A web-based query is dynamically generated from information extracted from the STPDF system, plus additional metadata not available from it. The query builder generates an XML query file that is passed to the query translator, which handles the interface to the STPDF system. STPDF then handles the requests for data from each of the required archives and returns a result. That is formatted and returned to the user through the web server.
• The WDC server: provides access to its data holdings via the STPDF server. The STPDF system handles the translation from the underlying data format and the sub-setting of the data.
• The UKCDC server provides access to its holdings in the same way as those of the WDC server, described above.

The development of the pilot's metadata translation layer started with an assessment of a number of existing XML formats that might be used. A report on this (15) may be found on AstroGrid Wiki page, and it may be summarised as follows: XSIL(16) is too simple; XDF(17) is too complicated; CDFML(18) is too STP-specific; and VOTable(19) looks usable. Despite the fact that VOTable was considered usable for STP work, it was decided to use an ad hoc XML format for the pilot work, for several reasons. Firstly, as with the consideration of use of VOTable in the solar pilot, its use in a new area would require the definition of a new set of Unfied Column Descriptors (UCDs(20)), for which there was insufficient time in the pilot. Secondly, it would be easier to define the restricted DTD or schema in its own namespace, without having to implement the whole of VOTable. Whilst the XML format used here was ad hoc, in some sense, it was decided to make use of International Solar-Terrestrial Physics (ISTP(21)) guidelines for defining its terms. For the STPDF software to be able to pick up the metadata for the WDC data, it was necessary to hand-craft text files, while the UKCDC metadata could be read straight from their CDF files. (N.B. SpaceGRID (in collaboration with GSFC, CDPP, Southwest Research Institute and PDS) are defining a space physics query language, which will yield a data dictionary likely to become the default for use in STP: this effort is starting from low level terms (i.e. Dublin Core) and then building up domain-specific metadata within a namespace. The AstroGrid STP pilot is constructing domain-specific descriptors based on the SpaceGRID work, and it is suggested that these would have to be included in any more general AstroGrid ontology via an STP namespace.)

Queries to the pilot system are constructed using a simple UI

which allows selection of system or user datasets (i.e. the results of previous operations, thereby allowing compound queries to be built). The UI is dynamically updated to show available fields, and the number of records for each selected dataset. Four basic operations are supported: select/output fields; query data set; find/select time interval; and time series join (nearest neighbour). Finally, a Quicklook UI (based on the UKCDC UI) was produced to view the selected dataset(s). The user selects a time range subset of the dataset and choose which parameters to plot. The example plot below shows a joined dataset comprising Dst data from the WDC and ACE magnetic field data from the UKCDC.

The user can then zoom in on a portion of the plot and can also obtain an ASCII dump of the data plotted by the Quicklook UI.

(8.3.2) The solar pilot - Data selection from summary information

The central design issue for this pilot was the definition of the minimal set of parameters required for solar observing catalogues. This resulted in a document issued under the aegis of EGSO and entitled "EGSO Unified Observing Catalogues" (22). This was constructed from the perspective of catalogue searches, collating the set of parameters that a user might need to query in order to select a particular dataset, but bearing in mind that the set defined must work within the context of the standard SolarSoft(23) software package ubiquitous within the solar physics community. In addition to this minimal parameter set, general information about the observatory (instrument description, contact info, etc) should be available to the user: it was decided that the definition of the requirements for such ancillary data should be left to EGSO, since this information in unlikely to be interrogated via the kind of catalogue queries being prototyped in this pilot.

Another design issue was the format for storing the solar observing catalogue data. Currently, this is usually held with an IDL(24) database, for ease of manipulation using SolarSoft, but there is a desire to remove the reliance on IDL, to ease the integration of solar physics software and Grid middleware. Another possibility would be to store these data in an XML repository, such as Xindice(25), possibly using VOTable documents for each entry. This would require the addition of further UCDs relevant to solar physics: this avenue is being explored within the context of EGSO, and it was decided that the time constraints on the delivery of working prototype software within this pilot would necessitate the use of an IDL-based system for this pilot.

The series of screenshots below follows the course of a query using this IDL-based system.

Step 1: The image below shows the start of the selection process. The right hand pane shows a SOHO-EIT EUV image, marking known features. The left hand pane displays information from three different satellites. At the bottom is the time series of GOES X-ray flux measurements in a number of bands. In the middle, orange crosses mark Yohkoh-SXT observations made in various modes during the same time period, but white crosses in the top portion mark TRACE EUV observations in various filters during that interval.

Step 2: The user notes that the GOES X-ray data are exhibiting a solar flare at about 14:00. Using a mouse the user can the define a time interval in the left hand panel (shown by the dashed lines) and then, only the image on the right hand panel are plotted the fields of view (FOVs) of all observations (TRACE in white, SXT in orange) taken during that interval.

Step 3: The user can then zoom in on the region of interest, by selecting an area using the mouse: the selected region is indicated by the dashed square on the EIT image in the right hand pane.

Step 4: Once that region has been defined, the right hand pane is replotted, showing the FOVs of only those observations whose FOVs intersect with the selected region. The lower portion of the right hand window then reports the temporal and spatial selection criteria the user defined, as well as the number of images from each instrument that these have selected.

At the moment, the user then has to go away to the normal UI for the respective databases and extract the desired data via inputting these selection criteria, but it is intended that this pilot will be extended so that the selection can be made directly from this UI. One slight complication is that this current selection procedure can return a large amount of data - 285 TRACE images in the example above - and it would be desirable to add some additional cadence criteria to the selection (e.g. only extract an image every five minutes, say).

(8.3.3) The radio pilot - Fourier data

It was decided to undertake this pilot using data from a region of sky which has been extensively observed right across the electromagnetic spectrum, namely the Hubble Deep Field. The unprecedented sensitivity of the observations required and produced very large data sets, from which there is still significant scientific information to be extracted, making it an ideal candidate for the AstroGrid pilot.

The development of a prototype environment for access to visibility data centred on running AIPS within a cgi wrapper. By this route, the MERLIN Archive(26) now supports simple queries (returning text and ready-made plots and FITS images) either via its web page or via CDS. A prototype interface for on-the-fly imaging of visibility data was produced and has been tested locally and remotely. This enables users to extract maps from calibrated visibility data at any position within the field of view. Typically, only the central few arcsec of archive data have ever been imaged: in the case of the HDF data, only about 1/7 of the total field of view has ever been imaged, but the sensitivity means that more science can undoubtedly be done (e.g. the background radio flux or barely detectable sources at the position of Chandra sources). The calibrated visibility data are several GB, making it impractical to supply it to off-site astronomers even if they could use AIPS locally. Thus a Virtual Observatory service like the one developed here is not merely a convenience but a necessity for such data sets.

The user has only to specify the size, position and resolution required to obtain an image. In the example shown below, the astronomer has interrogated the MERLIN archive and learnt of the Muxlow et al. observations in the Hubble Deep Field. The user then enters information about the field ("Offset field 1") for which s/he wants to generate an image from the visibility data.

The AIPS script then generates the image specified by the user on-the-fly and displays a postage stamp showing the resulting image,

and the data may then be downloaded in FITS format. Astronomers using this facility do not need any knowledge of AIPS or of radio data, but behind the scenes the archive uses two alternative routes depending on the size and complexity of the input data sets. MERLIN-only visibility data around a chosen region can be imaged on demand. A small amount of further development is needed to provide informative feedback if the user requests images at a position or resolution which the data cannot provide, based on individual data set properties. For very large data sets taken from more than one array (e.g. the HDF MERLIN+VLA data) two-stage data processing is required and only one route is currently implimented (combining images in the map plane); other possibilities will be investigated, in due course.

User feedback on this prototype system was positive, confirming that it produces (in an automated fashion) data of the same quality as that produced by an expert reducing the same data, although it was noted that the system ran slower than might be expected. Several performance bottlenecks have been identified, and possible enhancements are currently being investigated. The test user did note that the lack of defaults and restrictions in the current system means that the inexperienced user could be led to a poor specification of the image to be produced.

Work on deriving interferometry metadata standards has proceeded in conjunction with AVO Interoperability work. A library of terms to describe radio interferometry observations has been drawn up and these have been translated into CDS UCDs and work is ongoing to expand these where they are not at present sufficient. A fuller report on this activity can be found in (27) and consultations are underway with experts on higher frequency Fourier data to expand the scope of these metadata to ensure that it covers the requirements of interferometry in general, not just those of radio astronomy.

(8.3.4) Progress with the optical/near-IR and X-ray pilots

In common with the other three, the optical/near-IR and X-ray pilots were designed to be scientifically interesting, as well as functionally instructive, so both planned to use very topical data sets. Problems with the availability of these data sets - i.e. delays with the installation of a copy of the Sloan EDR database at Edinburgh, and with the production of the first XMM source catalogue by the Survey Science Centre, respectively - meant that neither of these pilots was able to proceed all the way to the delivery of prototype software to test users in August 2002. Indeed, work on them both was suspended midway through Phase A, as a result of delays with the test datasets, and the effort associated with them allocated to other parts of the Phase A programme.

However, both pilots did yield interesting results. For example, as part of his work for the X-ray pilot, Clive Page derived the PCODE method for performing spatial cross-matches in relational databases (28, 29). This was used to find optical associations of sources in a ROSAT catalogue, but the surface density of objects in this situation was not sufficient to exercise the algorithm thoroughly. More generally, the X-ray pilot provided the motivation for some thought of how to perform associations in the VO (29), although this remains a problem in need of a proper solution. The results of the optical/near-IR pilot are less obvious, since most of the work undertaken for it centred on gaining an understanding of the SX(3) archive system, which has since been dropped by the SDSS (2) consortium. Nevertheless, the effort expended on the optical/near-IR pilot before its suspension was not all wasted, since much of the design work for a SX-like schema for the SuperCOSMOS Sky Survey could be re-used once it was decided to produce the new SSS archive using SQL Server and not Objectivity/DB.

(8.4) Summary and Conclusions

The Pilot Programme was designed to complement the AstroGrid Science Requirements exercise and the Phase A technology evaluation workpackages, by trying to implement some small parts of AstroGrid's desired functionality using existing technologies. In this way, their limitations could be assessed, at the same time as offering the opportunity to supplement the AstroGrid Science Requirements with feedback from test users from each of AstroGrid's user communities exposed to prototype software from the pilots, which would give them an indication of what AstroGrid would eventually deliver.

The set of five pilots was chosen to cover different aspects of the general database federation problem of particular interest to the different parts of AstroGrid's user community. In each case, the pilot was intended to be performed using scientifically-interesting datasets, to motivate the use of the prototype software by test users. This led to difficulties with both the optical/near-IR and X-ray pilots, where delays with the delivery of data led to the suspension of work on the pilot mid-way through Phase A. This meant that neither of these pilots delivered prototype software to test users in August 2002, as originally intended. However, the data required for both pilots are now becoming available and it is intended that a sigificant fraction of the originally-intended work of both pilots will be completed by the end of December 2002, when the extended Phase A comes to a close.

The remaining three pilots succeeded in producing prototype software for test users to evaluate by the end of August 2002, which marks the originally-planned close of Phase A. User response to all three was positive, reassuringly confirming that initial ideas of desirable functionality were correct. One thing noted by test users of several pilots was the ease with the prototype systems could generate undesirably large volumes of data. This leads to one of the most important lessons from the Pilot Programme, which is the importance of having a resource estimation capability, both to indicate how long a job will take to run and how much data it is likely to generate.

Many useful lessons have been learnt from the AstroGrid Pilot Programme. At the most general level, test users responded positively to the additional functionality they were offered, but quickly wanted the ability to do more. This reassuringly confirms that the VO enterprise is worthwhile, but also that it will be difficult to meet expectations, and that considerable flexibility will have to be designed into VO systems to help them meet the range of user requirements. A number of more specific results emerged, too, for example that the VOTable prescription for presenting tabular data in XML may well be useful well beyond its originally intended domain, if appropriate UCDs (e.g. for interferometric, solar physics and STP data) can be defined. It was also clear that the VO must provide the means of estimating the resource implications (e.g. result dataset volume, time taken to run, etc) of proposed operations, lest users grind the system to a halt and/or generate much more data than they can handle.

A questionnaire circulated by the teams running the STP and solar pilot, in conjunction with ESA's SpaceGRID initiative, and addressed to a wide cross-section of the international solar system research community, produced some quite explicit performance requirements, for example: the system should provide feedback on an action with 30s; simple, online tasks should be completed within an average time of a minute; and complex, offline tasks should be completed within an average time of 24 hours. Interestingly, this survey also identified some Intellectual Property Rights issues not discussed much within the VO community to date, as some respondents thought that there should be a possibility of keeping workflows and query results private within the VO.

Perhaps the most valuable lesson learnt from the AstroGrid Pilot Programme is the importance of keeping (at least some) users closely engaged in the VO development process. New technologies, and the vast wealth of astronomical data now available, mean that the possible directions that the VO can take greatly exceed what can possibly be delivered, given the finite funding for the various VO projects, and it is essential that the course of VO development is decided by what users most need to do their science and not what the technology can deliver most readily.

A fuller account of the Pilot Programme can be found in (28).

References

(1) SuperCOSMOS Sky Survey: http://www-wfau.roe.ac.uk/sss
(2) Sloan Digital Sky Survey: http://www.sdss.org
(3) SX archive system: http://www.sdss.jhu.edu/ScienceArchive/home.html
(4) VizieR: http://vizier.u-strasbg.fr/viz-bin/VizieR
(5) Centre de Données astronomiques de Strasbourg (CDS): http://cdsweb.u-strasbg.fr
(6) Astrophysical Virtual Observatory (AVO): http://www.eso.org/avo/
(7) European Grid of Solar Observations (EGSO):http://www.mssl.ucl.ac.uk/grid/egso
(8) SOHO: http://sohowww.nascom.nasa.gov/
(9) Cluster: http://www.cluster.rl.ac.uk/
(10) SpaceGRID: http://spacegrid.esa.int
(11) Summary of solar system research requirements: http://wiki.astrogrid.org/bin/view/Astrogrid/SpaceGRIDRequirements
(12) Solar Terrestrial Physics Data Facility: http://www.ssd.rl.ac.uk/stpdf/
(13) Cluster Coordinated Data Handling Facility: http://www.cluster.rl.ac.uk/cdhf.htm
(14) World Data Centre for Solar-Terrestrial Physics: http://wdcc1.bnsc.rl.ac.uk/
(15) Report on "XML for STP data": http://wiki.astrogrid.org/pub/Astrogrid/PilotDocs/agstp-0002.pdf
(16) Extensible Scientific Interchange Language(XSIL): http://www.cacr.caltech.edu/SDA/xsil/
(17) eXtensible Data Format (XDF): http://xml.gsfc.nasa.gov/XDF/XDF_home.html
(18) CDF Markup Langugage (CDFML):http://nssdc.gsfc.nasa.gov/cdf/html/cdf_xml.html
(19) VOTable: http://cdsweb.u-strasbg.fr/doc/VOTable/
(20) Unified Column Descriptors: http://cdsweb.u-strasbg.fr/doc/UCD.htx
(21) International Solar-Terrestrial Physics (ISTP): http://www-istp.gsfc.nasa.gov
(22) EGSO Unified Observing Catalogues: http://www.mssl.ucl.ac.uk/grid/egso/public/documents/EGSO_UOC_Contents.doc
(23) SolarSoft: http://surfwww.mssl.ucl.ac.uk/surf/surf_software.html
(24) Interactive Data Language (IDL): http://www.rsinc.com/idl
(25) Xindice: http://xml.apache.org/xindice/
(26) Merlin Archive: http://www.merlin.ac.uk/archive
(27) Interferometry metadata report: http://www.jb.man.ac.uk/~amsr/WP5.3/radiometadata.html
(28) Pilot Programme Final Report: http://wiki.astrogrid.org/bin/view/Astrogrid/WPA5FinalReport
(29) Association Methods report from X-ray pilot: http://wiki.astrogrid.org/bin/view/Astrogrid/WPA5AssociationMethods

-- BobMann - 30 Sep 2002