Using and Use Of CVS within Astrogrid

This page discusses the use CVS within the context of AstroGrid development. It contains information about how to use CVS as well as why we use it in the way we do. In consolidating this page I would like to thank Tim, Martin, Dave and Kona for their very significant contributions over the last weeks and months.






Development Processes

We use CVS to manage all code associated with AstroGrid. There is a central repository (see below) which is backed up daily and is accessible to all developers. In the spirit of Open Source development, it is also read-only enabled to the world via pserver access.

To recap the process fundamentals, given the development goals for an iteration, Work Group Leaders are responsible for breaking these goals down into tasks and assigning those tasks to individual developers. Such tasks should be as atomic as possible and certainly not be designed to last more than 1 week. Tasks will have an associated BugZilla entry so they can be easily tracked. The BugZilla ID will be used in code comments and for use in development branch names (see below). Code developed as part of the current iteration is maintained on the HEAD branch of CVS. At all times, HEAD code should be as clean as possible. This means that before a task is added to HEAD it should compile, pass unit and component tests and have been subject to code review. Thus trivial error should not derail the daily build processes and a clear picture of the state of development can be seen.

Definitions

Two aspects of CVS can be very confusing: Tags and Branches. At first glance they appear very similar; indeed the CVS commands for both are very similar too. However, tags and branches are fundamentally different and each has a specific use and value in the development cycles.

  • Tags are used to mark the code to provide a snapshot in time. We use tags to mark a specific revision of the code, mostly relating to completion of a task group or the resolution of a complex bug (perhaps with many BugZilla entries). Although tags are shortcuts to dates, the meaning in the tag name makes comparing versions of modules and following an audit trail of significant changes over time much clearer than just using timestamps.

  • Branches are in effect a copy of the code at a given point in time (OK, CVS doesn't actually copy files, it works with version numbering, but the effect is the same). It is possible to branch the entire CVS tree or just portions of it. The purpose of a branch is to allow different development paths to be followed without impacting other branches or the HEAD code.

Use of Branches

So, we tag significant milestones, but when do we branch? There are two events which call for a new branch to be taken:

  1. The conclusion of an Iteration. Once an iteration is released the code becomes available for installation by anyone and we assume a responsibility to support that code - albeit only very minimally. It is possible that it will be necessary to issues patches to this code if significant bugs are found. Such fixes may not be appropriate for inclusion in HEAD code: this is the reason for taking a branch. However, if these fixes are required in the latest code, we can include them in HEAD. Patches can thus be developed in isolation and the release rebuilt with minimal dependency issues. Note that this implies the entire CVS tree was branched. It is also possible that an iteration may finish without fully stable code being available for release. In order that development commence as seamlessly as possible on the next iteration, taking a branch allows the stability issues to be resolved whilst at the same time new developments can start without there being a conflict between the two.
  2. The commencement of a task. As stated above, we try at all times to keep the code in HEAD as clean as possible. Where a task is non-trivial (i.e. likely to take more than 1 day to complete), developers can take a temporary branch to complete the task and then merge the completed code into HEAD. This temporary branch is then deleted. This prevents the automated build procedures from failing due to incomplete code and provides a better picture of the overall state of development. Note that such additions should not adversly affect other components unless the delegate interfaces change (either syntactically or semantically) and any such changes must be notified to and cleared through the Integration & Release team.

Repositories

The AstroGrid project has a central CVS repository which can be found at:

    cvs.astrogrid.org:/devel

Local repositories can be set up at the discretion of individual developers, but code must be committed to the central CVS as often as possible; certainly no less frequently than weekly and mostly at much shorter intervals

Read-only Access

The central repository does not support read only ("anonymous") access. However, there is a daily mirror of the repository which does allow this, using the pserver protocol.

First, you must login to the server (you only need to do this once):

    cvs -d :pserver:anonymous@cvs2.astrogrid.org:/astrogrid login

You can use any password: It is recommended that you use the empty string. smile

Then, check out the sources with this command.

    cvs -d :pserver:anonymous@cvs2.astrogrid.org:/astrogrid checkout -P devel

NB, note -P flag, and use it on update, etc too. There are a fair number of empty directories now from package moves.

Commit Access

For commit access to the repository, you will need to be authenticated. Currently, we only support SSH2 user key pairs. If you really can't use that, talk to Gary.

See below for setting up ssh keypairs under Unix and Windows.

Then check out the sources with

    cvs -d :ext:bob@cvs.astrogrid.org:/devel checkout -P test-proj

where bob is your login name on cvs.astrogrid.org.

Unix

First, you need to create a key pair, using the following command.

    ssh-keygen -t dsa

The program will offer to store the key in ~/.ssh/id_dsa; accept this default. Then it will ask for a pass phrase. It is absolutely vital that you use a pass phrase. Unlike a traditional Unix pass word, your pass phrase can be as long as you like. It is strongly recommended that you make it considerably longer than 8 characters.

You can use ssh-agent to arrange that you'll only need to type the pass phrase once per login session, so a long pass phrase is not too much of a burden. More details at UsingSshAgent.

ssh-keygen has now created two files in your .ssh directory. Please send Gary id_dsa.pub; this is the public half of the key, and is an ASCII file. Also, let Gary know what login name you'd like.

Once you hear back from Gary, you can proceed. First you need to tell CVS to use ssh. If you use a Bourne shell, or similar, say

    CVS_RSH=ssh 
    export CVS_RSH

If you use csh or tcsh, say

    setenv CVS_RSH ssh

Setting CVSROOT

If you use a Bourne shell, or similar, then you can set the default CVSROOT with the following command :

export CVSROOT=:ext:${username}@cvs.astrogrid.org:/devel

This simplifies things a bit, because you can leave out the -d bit in the commands.

  cvs -d :ext:{$username}@cvs.astrogrid.org:/devel checkout -P astrogrid
becomes
  cvs checkout -P astrogrid

Windows

See http://www.bgw.org/tutorials/operating_systems/win2k/cvs_over_ssh.php

Tim's version of this is (using \cli\ as command-line interface programs):

  • Run puttygen.exe to generate a keypair. Set a passphrase and save the private half to c:\cli\cvs\ssh\ (ie a new ssh subdirectory to keep CVS stuff). email the public key to Gary.

  • Set the following environment variable via Control Panel -> System -> Advanced tab -> Environment Variables:
         CVS_RSH = c:\cli\putty\pssh.bat
  • Create (or copy/paste) the following batch file, and put in c:\cli\putty\. Modify it so that the path to the 'plink' application is absolute for your platform.
@ECHO OFF
REM pssh.bat - run plink with ssh enabled, force level 2, give private key location
c:\cli\putty\plink -ssh -2 -i \cli\cvs\ssh\astrogrid.ppk %*

Every time you logon, and want to access the CVS, you will need to run pageant to set the keys in memory. Run it (or put it in your startup folder). Right click on the icon in the system tray, and select 'add key'. Select the key you saved above, and enter the pass phrase.

Branch names

Whether taking an Iteration branch or a development branch, the process is the same and is detailed below. However, without a sensible convention, we risk cluttering the tag "namespace", thus release branch names will reflect the iteration:

  Itn{$ITERATION_MAJOR_NUMBER}_{$ITERATION_MINOR_NUMBER}

whilst developmnent branch names should have the following form:

  {$COMPONENT}_{$USER}_{$BUGZILLA-ID}

with the intial part representing the component (short acronym/abbreviation), the user (initials) being those of whoever is creating/using the branch and the bugzilla-id being the ID number of the task in the bugzilla entry for that task, for example

  GDW_KEA_123
  REG_ECA_221

Creating a Branch

For example:

  cvs -d :ext:{$username}@cvs.astrogrid.org:/devel rtag -b {$branchname} astrogrid

This will create a branch in the CVS repository, tagged with name $branchname. This new branch will initially be identical to the current state of astrogrid and below in HEAD.

Working on tasks in a Branch

Find a nice fresh clean directory and check out the branch with:

  cvs -d :ext:{$username}@cvs.astrogrid.org:/devel checkout -P -r {$branchname} astrogrid
This will check out the branched version. Now you work on that just as you would normally - add, diff, etc will all work on the branched version. The HEAD will be unaffected by changes you make and commit in this branch. Use comments in the source which include the BugZilla ID (e.g. BZ123) to identify code relating to the task.

Setting the local directory name

If you add '-d {$branchname}' after the 'checkout' command, then it will put the branched verson in a local directory called {$branchname}
  cvs -d :ext:{$username}@cvs.astrogrid.org:/devel checkout -P -d {$branchname} -r {$branchname} astrogrid
Note, this is using same option, '-d', to do different things. I know this may be confusing, CVS re-uses option flags to mean different things depending on where they are in the command line.
  cvs -d {remote location} checkout -d {local directory} ....
However, it is very useful, as it makes it much easier to figure out which local copy is a branch and which is the main HEAD.

Merging the task code back into HEAD

Merging involves three (possibly four or five) steps:

1. Check out a clean version of the current HEAD:

  cvs -d :ext:{$username}@cvs.astrogrid.org:/devel checkout -P astrogrid 

2. Merge your branch into this clean HEAD using the -j flag

  cvs -d :ext:{$username}@cvs.astrogrid.org:/devel update -j {$branchname}

(2a. If conflicts are reported, rectify them by editing the conflicting files)

3. Finally, commit the new HEAD (into which your branch has been merged):

  cvs -d :ext:{$username}@cvs.astrogrid.org:/devel commit

4. Delete local modules associated with the branch

Although it's possible to merge a branch more than once, it's not recommended; once a branch has been created, used and re-merged into head, it should not be used again and the recommendation is to delete the local development modules. (The branch tags will remain in CVS however, hence the tag naming suggestion above).

Working with Iteration branches

Much of the above applies to developing fixes to released iterations, with the exception that the branch is not deleted upon completion of the code! If it is necessary to include the fix in HEAD code, very careful merging will be required. If the change is modest in size, manual merging into HEAD may be the easiest option.

Note that we may tag modules within a branch if the changes to them are considered significant...

More documentation

A good starting point is the CVS info documentation. If you're using emacs, type C-h i m cvs. From the command line, you can start the info browser with info cvs.

Below are a couple of reasonable reference docs for CVS. One is aimed at CVS client use, one at setting up a server and the last is the CVS "cederqvist", really the CVS bible. A bit of light background reading...

Update on Windows CVS / Eclipse procedure - 29 November 2005

  1. In puttygen.exe v 0.58 - after generating a public key, open key file and add "ssh-dss=" before the first capital A.
  2. Once you have successful ssh connection to cvs.astrogrid.org, before adding CVS repository location to Eclipse, open Eclipse -> Preferences -> Team -> Ext Connection Method -> CVS_RSH: (enter the full path to your pssh.bat file). Apply and OK. -- ElizabethAuden
Topic revision: r32 - 2007-07-03 - 10:03:48 - GaryGilchrist
 
AstroGrid Service Click here for the
AstroGrid Service Web
This is the AstroGrid
Development Wiki

This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback