AstroGrid Distributed Development

Background

Distributed development is both a blessing and a curse. By being able to utilise disparate resources, expertise otherwise not available in just one place can be brought to bear upon a project with considerable positive impact. Set against this are the substantial problems of successfully managing a distributed development effort and steering it to a successful conclusion. This document considers the problems of distributed development and provides the infrastructure upon which to base the project controls.

Keys to Success

The following list is pretty much the accepted industry standard for a development project:

• Level of detail in which team activities are recorded
• Quality of the software specification
• Integrity of the software specification
• Accuracy of the software specification
• Established development processes
• Rigour with which the established process is followed
• Quality and number of reviews and audits
• Accuracy of the models that have been used to estimate software attributes
• Effectiveness of the test and integration plan, specification, and test data
• Level of preparation for system maintenance

The AstroGrid project must bear these in mind whilst striving toward a successful conclusion!

Structure

Open Source developments by their very nature are not well suited to a highly structured management hierarchy. However, controls must be in place where they are needed. Typically this will include planning (short and long term), iteration definition, release management and source control. Clearly, whilst everyone has the right to be heard (indeed duty to ensure they are heard), there comes a point where decisions have to be made if anarchy is to be avoided. Provided these instruments are used with a light touch, genuine community development is possible. The following sections describe the breakdown and purpose of these components.

Roles and Responsibilities

Each active contributor to the AstroGrid project will be a member of one or more of the following groups:

Technical Support Panel

To ensure the project stays on track, the Technical Support Panel (TSP) will determine the overall direction of the project. It will create short and long term plans, decide the content and schedule for each development iteration and provide a release plan for those iterations, ultimately leading to the release of the first full version of AstroGrid.

The members of the TSP will include:

Project Manager	Tony Linde
Project Scientist	Nic Walton
Technical Lead	Keith Noddle
Leicester	Clive Page
Cambridge	Guy Rixon
Edinburgh	Bob Mann
JBO	Anita Richards
MSSL	Bob Bentley
QUB	tba
RAL	Peter Allen

The group will meet regularly to ensure the project is on track and to provide guidance as well as discharge its own responsibilities.

Development Teams

Every member of the development teams will be a direct contributor to the substance of the project. "Development" covers a wide range of activities from building and maintaining the development environments through coding and testing and deployment to recommended target systems configuration and tuning. All plans, designs, document and code will be held within a central archive to allow full control over revision and release planning as well as recovery and back-tracking. This archive will be a CVS repository held on uluru.star.el.ac.uk in Leicester and will be the "Release" repository. Further details can be found on the [Coding Standards and Processes] Wiki page. Note that teams may be geographically dispersed and that “committers” (see below) may change between iterations.

Single Committer Development

Successful open source projects have almost universally adopted the principle of “single committer”. Each development team has a designated committer responsible for uploading solid code into the central CVS repository. The system works within teams as follows:

1. Developer develops the code
2. Developer subjects the newly developed code to unit and/or package tests
3. New code is run from a user perspective (so-called Black Box testing)
4. Developer sends the code to all team members (or a code review meeting takes place as locally defined)
5. Team members comment on code providing feedback on quality and bugs
6. Developer resolves any issues raised and publishes a revision
7. Cycle repeats until all issues resolved
8. Designated committer merges code into the Release CVS repository

(N.B. This same process will be used to manage changes and bug fixes.) Thus each contributor will be either a "Committer" or a "Developer".

Developers develop code, plans, designs, documents etc and are responsible for the integrity of those items. Developers have read-only access to the Release CVS repository.

Committers are also developers but with the added responsibility of uploading items into the Release CVS repository, following review and successful unit/package testing.

Roles and responsibilities within the developments teams are:

Developers	Write plans, designs, documents, code etc
	Use local archives (CVS repository)
	Code developers build unit/package test suites complete with usage documentation
	Submit tested code for review
	Participate in code reviews
Team Leaders	As developers plus:
	Create and maintain test plan for all local code development
	Schedule and manage code reviews (these may take the form of meetings, 1 to 1 reviews or private review as locally decided)
	Arrange code walk-through presentations as necessary
	Enter clean code into central repository at least twice per week (Tuesday and Friday evenings)
Technical Lead	Develop and publish Standards and Procedures
	Provide overall technical direction
	Advise on short and long term planning requirements
	Assist with project planning and management
	Plan system and integration testing
	Manage releases including feedback from Alpha and Beta test programmes
	Implement and manage changes and bugs
	Plan and manage patches
	Back-fill any technical gaps

Sub-projects

As the project progresses ideas for additions to functionality or scope will appear. It is vitally important that a project as complex and diverse as AstroGrid at it's outset should not expand without strict control. So called “scope-creep” is the projects biggest enemy. However, many of the suggested changes will be genuinely of benefit. Therefore the TSP (initially) will manage the formulation of sub project proposals. In this way changes outside the remit of AstroGrid but which have significant merit will not be lost.

Communications

AstroGrid already has good communication tools in the form of the Forum and the Wiki. It is vital these continue to be used. Formal reporting is probably unnecessary provided lines of communication are kept open and used. The only additional communication will be the publishing of plans by the TSP which detail the short and long term direction of the project. These will be made available via the Wiki. Iteration plans (short term) will form the basis of the current “work package” as allocated to the development teams by the TSP (or subset as appropriate). Team Leaders will be expected to monitor progress against plan and report any problems or issues early so they may be addressed in good order.

-- KeithNoddle - 04 Dec 2002