AstroGrid Architecture: Detail

Introduction

This is the last of three documents which describe the AstroGrid Architecture. The audience for this document is those who will design and/or build components of the Virtual Observatory (VO), whether as part of the AstroGrid (AG) project or another VO project.

The other two documents in the Architecture Documentation Set are:

• AstroGrid Architecture Vision^?
• AstroGrid Architecture Overview^?

Despite the different names, these documents all present overview-type information. The actual description of the AstroGrid architecture will be developed as part of the project documentation and the details of the components themselves will be derived from the JavaDoc documentation.

Domain model

AG started with a domain model (see rbAgDomain.gif) but this picture ended up focussing on the domain of astronomy rather than that of the virtual observatory (VO). The VO is largely an information system which just happens to deal with astronomical information. It will be the tools that work on top of the VO infrastructure which enable astronomy to be done, not the infrastructure itself.

We've been working with the original component diagram, as represented in the Phase A Report for some time now.

I'd like to simplify that diagram a little into one which represents the core domain entities of any VO (as envisaged by AstroGrid).

Entities

As is obvious, this diagram is based on the component one, drawing multiple entities into one. A brief description of each entity:

• Registry : maintains a list of resources known to the VO
• Community : manages aspects of VO to do with people, organisations, projects etc
• Portal : the user interface to the VO, whether server/web-based or client-based
• Security : to do with authenticating and authorising users and permissioning access to resources
• Workflow : doing work on the VO, either immediately or by constructing and running jobs
• Messaging : passing messages between components
• Data Access : how access to data is enabled
• MySpace : a virtual directory of user owned information space
• Tools : any service which can be activated (or plugged into a workflow)

Links

It is in the links between the entities where VO interoperability will be enabled. Document and message exchange formats and protocols must be standardised if the components created by the different VO projects can work together and, for any user, replace one another.

Core entities

There are many ways to picture the domain space of a VO system. The one above is very simplistic and is component focussed, ie based on the likely major components of such a system. There may be over-simplification in this model: the Workflow component includes workflow construction and job control which are quite distinct; Tools covers a wide variety of different types of software component; Security encompasses both user authentication and the access policies for resources. At least this approach is a start and will enable people to picture both the way AstroGrid is structured and one possible generic architecture for the VO.

Portal

The Portal is the only user-facing component. Any other component which requires input from the user or which will present information to the user must do so through the Portal. It is designed to be accessible from any of the modern web browsers: (Internet Explorer 5.5+, Mozilla 1.4+. We have adopted Apache Cocoon as the technology for the Portal: this allows us to strictly separate content from both styling and delivery mechanism and, as an xml-based technology, ensures easy interoperability with other projects and components.

In considering the Portal, we need to look at two aspects:

• the server-based documents which are used by Cocoon to construct web pages along with applications which provide the functionality behind those pages;
• the browser-based pages which give the feel of an integrated software application to any set of pages.

Unless strictly necessary, we will not distinguish these aspects in this and following sections and so will talk of, say, the Registry application as a whole although it consists of both registry components installed on one server and support Cocoon documents installed with the Portal.

Page Structure

Any Portal page will have a structure like:

  ________________________________
  |                              |
  |             A                |
  |______________________________|
  |      |                       |
  |      |          C            |
  |      |_______________________|
  |      |                       |
  |      |                       |
  |   B  |                       |
  |      |          D            |
  |      |                       |
  |      |                       |
  |      |                       |
  |______|_______________________|
  |             E                |
  |______________________________|

The areas of the typical portal page are:

• A) Organisation Header: contains name & logo of organisation running portal; supplied via Portal config file;
• B) Portal/Application Menu: lists available applications (set up in config file); applications can contribute sub-menus;
• Astrogrid.C) Page Menu: whatever menu the application wants to present as appropriate to the current page; can be altered directly by the application;
• D) Application Page: whatever content served by application; must be Cocoon-type page;
• E) Portal Footer: copyright & license of Portal developer.

Component UI

Any component which is available by a Portal must, itself, supply the UI pages. When a Portal first runs, it checks its config file to get the list of available applications

Registry

The registry is the source of metadata for any resource in the VO. Three terms in that statement deserve further explanation:

Resource:

A resource is anything that might be used or referenced by any user, agent or component of a VO system. Resources include, as expected, services such as those which provide access to datasets or images, those which will analyse or reduce data, those which provide visualisation of data etc. Other resources will include people, organisations, projects, groups of people etc. AstroGrid will also want to register items such as files and tables which are managed under the MySpace virtual storage system (referred to in this document by the generic name of MySpace).

Metadata:

Metadata are items of information used to describe a resource. So all resources will include some identity information, a service will be described by curation information (who created and publishes the service, a URL for more information etc.) and means of invocation, a specific data access service will include coverage information and information about data itself.

Source:

That a registry can provide all this metadata does not mean that every registry must store every piece of metadata about every resource. A registry may choose to only store certain classes of resource or to only store the metadata for a select number of resources (the difference being that the list of resources in a class will grow as such resources are added to any registry in the VO while the select list will only grow as teh curator of that registry adds to it).

Resource Identity

A resource will be identified by a ResourceID, consisting of an AuthorityID (which looks like a domain URI) and a ResourceKey (which looks like a path). In xml form this looks like:

   <ResourceID>
      <AuthorityID>...</AuthorityID>
      <ResourceKey>...</ResourceKey>
   </AuthorityID>

and in string form:

   ivo://AuthorityID/ResourceKey

where the string form might be terminated by the end of the string or either of '?' or '#' characters, so:

   ivo://AuthorityID/ResourceKey
   ivo://AuthorityID/ResourceKey?keyword=value
   ivo://AuthorityID/ResourceKey#pointer
   ivo://AuthorityID/ResourceKey#pointer?keyword=value

are all recognised by a Registry as representations of the same resource: ivo://AuthorityID/ResourceKey.

An authority is owned by some person or organisation. They will register it with a Registry and from then on, only that registry will be permitted to register a resource on the VO with that authority (to ensure that this happens, every registry will be required to 'know' which authorities have been registered at all other registries). It will be up to the registry implementation to ensure that only the authority owner (ie the person who registered it) can add resources to the authority.

Registry Structure

The structure of the registry is determined in two documents issued by the IVOA:

• Resource Metadata
• Registry Schema

and available from the IVOA Registry Working Group and the Document Repository.

AstroGrid will implement three separate registries:

• Resources
• Community
• MySpace

Only the Resource registry must conform to the IVOA standards but we expect that the Community registry will follow much the same structure. The MySpace registry is likely to be different.

Community

The Community entity encompasses everything to do with people, groups, organisations and projects. The main goal of creating a community is to allow access to resources to be controlled by groups, projects or organisations and for those groups to decide which people will belong to the group.

In general, a community will consist of:

registry

this will conform to the IVOA standards for all resource registries

software

which, in addition to normal registry functions, will allow the management of groups, organisations and projects: creation and deletion, adding and removing members. assigning privileges etc.

space

access for community members to personal MySpace

Community Registry

The community registry

Community Structure

Community and local MySpace

Security

There are two key aspects to Security:

authentication

user is who he/she says they are

authorisation

user is allowed to perform the selected action

Authentication

Authorisation

The process of authorising a user to perform an action is in three steps:

• finding the policy document that covers that action
• determining the identity of those entities authorised to perform the action
• checking if the user belongs to one of those authorised entities (usually a group)

The above process might be made quicker if the agent through which the user operates (or which is operating on the user's behalf) 'remembers' or 'is told' which authorised entity the user belongs to so that it does not have to check all of them looking for the user as a member. There must be algorithms in existence which deal with this sort of circumstance.

We imagine that policy documents will be managed by a policy server in much the same way that registry entries are managed by a registry server. So some areas of functionality may be similar. The key difference is that one will search a registry for entries whose metadata match some criteria but one will go directly to a policy server looking for the access policies for a specific item.

Workflow

Messaging

Data Access

MySpace

Tools

Sample Sequences

In this section, a few sequence diagrams are constructed to demonstrate how components are expected to interoperate.

Data Centre registers dataset

Demonstrates:

• how resource is added to local registry

Sequence:

Harvest local registry

Demonstrates:

• how full registry harvests information from local registry

Sequence:

Set access policy for resource

Demonstrates:

• how access restrictions are added to resource

Sequence:

Tool provider registers new tool

Demonstrates:

• how non-data resource is registered in full registry

Sequence:

User registers with community

Demonstrates:

• how user is added to community

Sequence:

Create group for resource access

Demonstrates:

• how a group (eg those with access to restricted resource) is created
• how additional users (local & remote) are added to a group

Sequence:

User logs in

Demonstrates:

• how user logs into portal
• creation of user identity package at portal

Sequence:

User uploads file to MySpace

Demonstrates:

• how new file is added into personal MySpace

Sequence:

Construct workflow

Demonstrates:

• how user creates workflow
• how registry is searched for data & service resources
• how steps are added to workflow
• how workflow engine validates step linking (output to input)
• how workflow branching and merging is indicated
• how workflow is stored as template

Sequence:

Submit job

Demonstrates:

• how workflow is retrieved
• how step specifics are set
• how job is submitted
• how job tasks are invoked
• how user tracks progress

Sequence:

Job: Query + Data analysis + Visualisation

Demonstrates:

• how batch and interactive tasks can be undertaken

Sequence:

---

Note: the previous version of this document is retained in ArchDetail_Old01 just in case this changed approach does not work.