AstroGrid Architecture Discussion Document

This is a short position paper on what form any Architecture document should take, and is to be (or was) presented to the AGOC06 meeting. If you are reading a pdf or hard copy version of this document, the latest version is at http://wiki.astrogrid.org/bin/view/Main/TonyOnArchitecture where you can follow the links to find the associated documents.

Introduction

The AstroGrid Oversight Committee (AGOC), at its Feb'05 meeting, asked for an Architecture document to be produced. This was after a fairly comprehensive architecture document was produced by Guy Rixon and made available. This document ran to many pages and was felt to be too detailed for what was wanted.

I wasn't sure what format this new document should take and so engaged in an email exchange with Paul Watson (AGOC chair) to try and clarify the issue. As this went on, I felt that what was expected was inappropriate for the type of project that AstroGrid was, while agreeing that certain documents were necessary. In the end, Paul and I agreed that this ought to be discussed at the AGOC meeting. This short paper will introduce the issue and hopefully aid in that discussion.

What is an architecture document?

Paul Watson stated what he expected of an architecture document as: 'The idea of the architecture document is to provide a succinct description of the system, its major components and their functionality. It would give us a high-level desription of astrogrid so that we know what is being designed and built. It should therefore not become out of date.' (my italics).

I completely agree with that description of an architecture document: it is exactly what I have produced for many past projects as part of the initial design work. But the section I highlighted in italics demonstrates why it might not be appropriate for an AG-type project.

Another common definition (this one from the Open Group, see TOGAF FAQ) includes:

Both the definitions from Paul and from the Open Group make it clear that an architecture is something that is produced as a plan from which ... systems [can be] developed. It is produced up front and states what the system will look like when it is complete.

But that is not appropriate to the AstroGrid project.

AstroGrid architecture

Since the beginning of AstroGrid, I've produced a number of high level diagrams to convey, in simple form, the end-goal that the project was working towards. These were quite openly more 'vision' documents rather than 'architecture' documents. As an example, one can look at the ArchitectureThesis (AppA in pdf) which was presented to the AGOC on 30-Jan-2003 and TechnicalScope (AppB in pdf) which was one of the preparatory documents for the AG2 proposal.

The latter still fits the overall 'structure' of the AG product. At first glance it might seem similar to something like the DSpace architecture diagram (AppC in pdf). But there is a key difference. The DSpace layers are separate and accessed from one to the other via a fixed API. In the case of AstroGrid though, each layer is conceptual only; in effect, every single component is a layer unto itself with its own API. Each component can and does call pretty much any other component to achieve its ends, and which ones it calls depends on the job it is asked to do, and that often depends on the workflow which the user has constructed.

Things are not quite as chaotic as that at the infrastructure level, but it certainly is the case that, as each Cycle of the project develops, new links between components will be forged as new functionality is required. One might argue from this that an architecture and upfront design documents might be constructed at the beginning of each Cycle. But this is also valueless, since each iteration within any Cycle can alter the plans and designs of later iterations (and previous ones!). The engineers only produce designs at the very last moment before implementing changes or new developments: that is the only way a project like this can proceed. It might seem chaotic to those used to traditional systems development practices (like me) but it works and has been proven to be successful.

All that said, it is clear that the above IEEE definition of an architecture does highlight areas which external and internal developers need filling in:

• components: easily accessible list of components
• interfaces: interfaces to each component documented
• links: list of other components accessed by each component and which interface(s) used

I therefore propose the following changes to our document set.

Architecture proposal

The AstroGrid software site is the starting point for all technical documentation relating to the project and this is where any further documentation must be located. From this one can get to the Developer documentation page.

Technical introduction

That first developer page ought to include more of a technical introduction. This might be at the level of the AG2 TechnicalScope, showing the layer diagram and listing the function (in one paragraph) of each component. It also ought to include a brief list of technologies used and how one might go about interacting with the AG components.

Changes to developer docs

The root http://software.astrogrid.org/developerdocs page needs to include a component section as has been provided for the installation root page (see list of components in left-hand sidebar).

For each component, we require a submenu (eg see installation submenu for registry component) which includes:

• interfaces
  • with a textual description of each method/property accessible via the interface,
  • a link to the appropriate javadocs,
  • and a piece of sample code showing how to call the component
• links
  • a list of components (AG ones) accessed by the component,
  • with a brief description of what each call performs
• change list, per version released (example)

All of these need to be fully updated before new versions are released so that developers can see changes immediately.

Test run

Any documentation has to be tried out on its putative audience before it can be said to be properly effective or complete.

The VOTech project has asked that AstroGrid run some developer workshops in the near future. I would propose that one of the more simple components be documented according to the above scheme and feedback from the developers at the workshop be solicited. In addition, Russian Virtual Observatory developers will be using AG to develop their own systems so we ought to keep track of problems they encounter which indicate where developer documentation could be improved.

End-of-project architecture document

When the project completes at the end of 2007, we ought to produce some overall description of the AG architecture as it stands at that time. This will be a compilation of documents from the Developer documentation site but in a single document form. In particular, it ought to be produced to whatever at the time are commonly agreed standards (or common practices) for documenting Service Oriented Architectures. It might also be worth looking at the W3C Architecture of the World Wide Web document for ideas. This document ought to be the focus of a concerted effort by all engineers in the final Cycle (C08) of the project.

In advance of that effort, Cycle07 ought to include a task in which a definition of the document standard to be followed for the AG SOA description is created and another to identify the source of all the contents of that description from existing documents on the software site.

-- TonyLinde - 08 Oct 2005