/var/local/projects/vospace.
You can put the source tree wherever you want, but be aware parts of the tree may be shared with other Unix accounts, so things get complicated if you place it within your home directory.
Set an environment variable for the project base.
VOSPACE_BASE=/var/local/projects/vospace
The rest of these instructions will use the VOSPACE_BASE environment variable to indicate the top level of the project.
pushd ${VOSPACE_BASE}
svn checkout http://esavo02.esac.esa.int/svnvo/VOSpace/trunk vospace.trunk
....
chgrp -R users vospace.trunk
popd
This will place the source tree in a sub-directory called vospace.trunk.
Naming the directory like this makes it easier to manage things when you have multiple branches checked out at the same time.
If you want to handle multiple branches it is useful to add a symbolic link to the 'current' version.
pushd ${VOSPACE_BASE}
ln -s vospace.trunk current
popd
This allows you to swap to a different branch by just changing the symbolic link.
The rest of these instructions will use ${VOSPACE_BASE}/current to indicate the current version of the source tree.
vospacejunit_2008_00.
The live server uses a PostgreSQL? database called vospace_2008_00.
This separation means we can run tests that delete or modify the database tables or contents without
having to worry about accidentally damaging data in a live service.
The build tree includes a psql command line instructions for creating the required databases.
#
# To create the JUnit test database 1st time.
psql template1 postgres
\c template1
CREATE USER junit ;
CREATE DATABASE vospacejunit_2008_00 ;
GRANT ALL ON DATABASE vospacejunit_2008_00 TO junit ;
\q
If you already have a vospacejunit_2008_00 database setup, then use the commands in the drop and create sequence.
#
# To drop and re-create the JUnit test database (after table schema changes).
psql template1 postgres
\c template1
DROP DATABASE vospacejunit_2008_00 ;
CREATE DATABASE vospacejunit_2008_00 ;
GRANT ALL ON DATABASE vospacejunit_2008_00 TO junit ;
\q
Note that the database name now includes a version number.
If a new release changes the table structure, we will increment the version number in the database name.
This will allow us to install a new database schema without deleting the existing tables,
and provide migration tools to copy the data from old tonew databases.
buildnew.sh and buildall.sh is that buildall.sh adds the -o option when calling Maven.
This tells Maven not to call out to external repositories to look for new versions of the jar files.
The first time you run the build, you will need to run buildnew.sh, which gets all the required 3rd party jars
from our Maven repository.
After that you should be able to use buildall.sh, and it will speed up the build by reduceing the number of calls to the external webserver.
pushd ${VOSPACE_BASE}
./buildnew.sh
....
popd
agtomcat.
su -
useradd --comment "AstroGrid test tomcat" --create-home agtomcat
passwd agtomcat
exit
Login to the agtomcat account.
su - agtomcat
To make it esier to find the vospace source tree, create a symbolic link to the current version.
VOSPACE_BASE=/var/local/projects/vospace
ln -s ${VOSPACE_BASE}/current current
Run the AstroGrid install script in the testing section of the source tree.
./current/webapp/myspace/testing/services/install/install.sh
The initial stage of the install script will show default settings and prompt you t accept or change them.
If you are installing a test system on a laptop, then you should change the host name to localhost.
[agtomcat@hostname ~]$ ./current/webapp/myspace/testing/services/install/install.sh
JAVA_HOME setting: /usr/java/default - return for OK or enter new value
setting unchanged
INSTALL_USER setting: agtomcat - return for OK or enter new value
setting unchanged
.....
ASTROGRID_HOST setting: leucine.metagrid.co.uk - return for OK or enter new value
localhost
.....
Installing AstroGrid services
JAVA_HOME : /usr/java/default
INSTALL_USER : agtomcat
INSTALL_HOME : /home/agtomcat
INSTALL_TEMP : /home/agtomcat/temp
ASTROGRID_USER : astrogrid
ASTROGRID_PASS : W6G117
ASTROGRID_HOST : leucine.metagrid.co.uk
ASTROGRID_PORT : 8080
ASTROGRID_AUTH : org.astrogrid.demo
ASTROGRID_EMAIL : demo@astrogrid.org
ASTROGRID_ADMIN : Demo admin
INTERNAL_URL : http://localhost:8080
EXTERNAL_URL : http://localhost:8080
Are these settings OK (Y/n)?
y
Install Tomcat (Y/n) ?
y
Installing AstroGrid Tomcat
TOMCAT_VERSION : 5.5.20
TOMCAT_COREZIP : apache-tomcat-5.5.20.zip
TOMCAT_MIRROR : http://www.astrogrid.org/maven/tomcat/zips
JAVA_HOME : /usr/java/default
CATALINA_HOME : /home/agtomcat/apache-tomcat-5.5.20
INSTALL_USER : agtomcat
INSTALL_HOME : /home/agtomcat
ASTROGRID_USER : astrogrid
ASTROGRID_PASS : W6G117
ASTROGRID_HOST : localhost
ASTROGRID_PORT : 8080
ASTROGRID_AUTH : org.astrogrid.demo
ASTROGRID_EMAIL : demo@astrogrid.org
ASTROGRID_ADMIN : Demo admin
INTERNAL_URL : http://localhost:8080/
EXTERNAL_URL : http://localhost:8080/
....
....
Checking Tomcat home page
URL : http://localhost:8080/
PASS
Checking Tomcat admin
URL : http://localhost:8080/admin/
name : astrogrid
pass : W6G117
PASS
Checking Tomcat manager
URL : http://localhost:8080/manager/html
name : astrogrid
pass : W6G117
PASS
Install Registry (Y/n) ?
y
Installing AstroGrid registry
REGISTRY_VERSION : 2007.1r
REGISTRY_CONTEXT : astrogrid-registry
JAVA_HOME : /usr/java/default
CATALINA_HOME : /home/agtomcat/apache-tomcat-5.5.20
INSTALL_USER : agtomcat
INSTALL_HOME : /home/agtomcat
ASTROGRID_USER : astrogrid
ASTROGRID_PASS : W6G117
ASTROGRID_HOST : localhost
ASTROGRID_PORT : 8080
ASTROGRID_AUTH : org.astrogrid.demo
ASTROGRID_EMAIL : demo@astrogrid.org
ASTROGRID_ADMIN : Demo admin
INTERNAL_URL : http://localhost:8080/astrogrid-registry/
EXTERNAL_URL : http://localhost:8080/astrogrid-registry/
The Registry install script will prompt for the external registry endpoint.
Please select the external registry endpoint :
1) Local registry (internal) :
http://localhost:8080/astrogrid-registry/services/RegistryQuery
2) Local registry (external) :
http://localhost:8080/astrogrid-registry/services/RegistryQuery
3) AstroGrid dev registry :
http://katatjuta.star.le.ac.uk:8080/astrogrid-registry/services/RegistryQuery
4) AstroGrid live registry :
http://galahad.star.le.ac.uk:8080/astrogrid-registry/services/RegistryQuery
*) Current setting :
http://localhost:8080/astrogrid-registry/services/RegistryQuery
For a test system, this should be set to point back to the local test Registry on localhost,
so select the first option (1).
Please select the external registry endpoint :
1) Local registry (internal) :
http://localhost:8080/astrogrid-registry/services/RegistryQuery
....
*) Current setting :
http://localhost:8080/astrogrid-registry/services/RegistryQuery
1
Registry endpoints
REGISTRY_ADMIN : http://localhost:8080/astrogrid-registry/services/RegistryUpdate
REGISTRY_QUERY : http://localhost:8080/astrogrid-registry/services/RegistryQuery
REGISTRY_ENTRY : http://localhost:8080/astrogrid-registry/admin/addResourceEntry.jsp
Install FileStore (Y/n) ?
y
Installing AstroGrid FileStore
FILESTORE_VERSION : 2007.2fs
FILESTORE_CONTEXT : astrogrid-filestore
.....
Install FileManager (Y/n) ?
y
Installing AstroGrid FileManager
FILEMANAGER_VERSION : 2007.2fm
FILEMANAGER_CONTEXT : astrogrid-filemanager
.....
Install Community (Y/n) ?
y
Installing AstroGrid Community
COMMUNITY_VERSION : 2007.2ct
COMMUNITY_CONTEXT : astrogrid-community
.....
Part of the Community install scripts adds a test user accout to the Community.
We will replace this account later, but at this stage it is useful to check that the system is configured correctly.
Add test user (Y/n) ?
y
Adding test account
URL : http://localhost:8080/astrogrid-community/admin/AccountAdmin.jsp
Name : frog
Pass : qwerty
PASS
PASS
.....
Install FIRST DSA (Y/n) ?
n
agtester.
su -
useradd --comment "AstroGrid tester" --create-home agtester
passwd agtester
exit
agtester account.
su - agtester
To make it esier to find the vospace source tree, create a symbolic link to the current version.
VOSPACE_BASE=/var/local/projects/vospace
ln -s ${VOSPACE_BASE}/current current
To make it esier to run VOExplorer, create a create a symbolic link to the launching shell script.
ln -s current/webapp/myspace/testing/client/runtime/voexplorer.sh voexplorer.sh
You should then be able to launch VOExplorer with the following command :
~/voexplorer.sh
agtester account.
su - agtester
To make it esier to run the Python tests scripts, create a create a symbolic link to the script directory.
ln -s current/webapp/myspace/testing/client/python python
You should then be able to run the test scripts using the following command :
~/python/test-002.py -u frog -p qwerty -l 5 -t
You can shutdown the VOExplorer from the command line using the exit.py Python script.
~/python/exit.py
su - agtomcat
~/shutdown.sh
VOSPACE_BASE=/var/local/projects/vospace
cat >> ~/tomcat/conf/Catalina/localhost/astrogrid-vospace.xml << EOF
<?xml version='1.0' encoding='utf-8'?>
<Context
displayName="AstroGrid VOSpace service"
docBase="${VOSPACE_BASE}/current/webapp/webapp/target/astrogrid-vospace-webapp"
path="/astrogrid-vospace"
>
</Context>
EOF
~/start.sh
Note if you have used a different location for the project, then change the value of VOSPACE_BASE before you run the command.
You should now have a deployed vospace service here
2008.0.0a service.
Please note - we know that the current configuration JSP pages are difficult to use,
and a new design for these pages is high on our todo list.
However, that said, I'll do the best I can.
su - agtomcat
tail -f ~/tomcat/logs/vospace-debug.log
vospace_2008_00.
The build tree includes a psql command line instructions for creating the required databases.
#
# To create the live service database.
psql template1 postgres
\c template1
CREATE USER vospace ;
CREATE DATABASE vospace_2008_00 ;
GRANT ALL ON DATABASE vospace_2008_00 TO vospace ;
\q
If you already have a vospacejunit_2008_00 database setup, then use the commands in the drop and create sequence.
#
# To drop and re-create the live database.
# Use with caution - this will invalidate all the service registrations and endpoints.
psql template1 postgres
\c template1
DROP DATABASE vospace_2008_00 ;
CREATE DATABASE vospace_2008_00 ;
GRANT ALL ON DATABASE vospace_2008_00 TO vospace ;
\q
Note that the database name now includes a version number.
If a new release changes the table structure, we will increment the version number in the database name.
This will allow us to install a new database schema without deleting the existing tables,
and provide migration tools to copy the data from old to new databases.
Once you have created the empty database, you can run the reset database JSP page.
This JSP page will connect to the database, drop any existing tables and create a new set of empty tables.
Note - The first time this is used, the debug logs will show a list of warnings :
2008-02-18 20:16:19,809 VOSPACE INFO Resetting database tables
2008-02-18 20:16:20,504 VOSPACE WARN java.sql.SQLException: ERROR: relation "basenode" does not exist
2008-02-18 20:16:20,504 VOSPACE WARN java.sql.SQLException: ERROR: relation "filenode" does not exist
2008-02-18 20:16:20,505 VOSPACE WARN java.sql.SQLException: ERROR: relation "filenode" does not exist
2008-02-18 20:16:20,505 VOSPACE WARN java.sql.SQLException: ERROR: relation "handlerstatus" does not exist
2008-02-18 20:16:20,505 VOSPACE WARN java.sql.SQLException: ERROR: relation "handlerstatus" does not exist
2008-02-18 20:16:20,505 VOSPACE WARN java.sql.SQLException: ERROR: relation "handlerstatus" does not exist
2008-02-18 20:16:20,505 VOSPACE WARN java.sql.SQLException: ERROR: relation "linknode" does not exist
2008-02-18 20:16:20,505 VOSPACE WARN java.sql.SQLException: ERROR: relation "nodelistparams" does not exist
2008-02-18 20:16:20,505 VOSPACE WARN java.sql.SQLException: ERROR: relation "nodeservicecapability" does not exist
2008-02-18 20:16:20,505 VOSPACE WARN java.sql.SQLException: ERROR: relation "nodeservicecapability" does not exist
2008-02-18 20:16:20,506 VOSPACE WARN java.sql.SQLException: ERROR: relation "rootnode" does not exist
2008-02-18 20:16:20,506 VOSPACE WARN java.sql.SQLException: ERROR: relation "serviceendpoint" does not exist
2008-02-18 20:16:20,506 VOSPACE WARN java.sql.SQLException: ERROR: relation "storefile" does not exist
2008-02-18 20:16:20,506 VOSPACE WARN java.sql.SQLException: ERROR: relation "treenode" does not exist
2008-02-18 20:16:20,506 VOSPACE WARN java.sql.SQLException: ERROR: table "basenode" does not exist
2008-02-18 20:16:20,506 VOSPACE WARN java.sql.SQLException: ERROR: table "filenode" does not exist
2008-02-18 20:16:20,506 VOSPACE WARN java.sql.SQLException: ERROR: table "filestore" does not exist
2008-02-18 20:16:20,506 VOSPACE WARN java.sql.SQLException: ERROR: table "handlerconfig" does not exist
2008-02-18 20:16:20,506 VOSPACE WARN java.sql.SQLException: ERROR: table "handlerstatus" does not exist
2008-02-18 20:16:20,507 VOSPACE WARN java.sql.SQLException: ERROR: table "linknode" does not exist
2008-02-18 20:16:20,507 VOSPACE WARN java.sql.SQLException: ERROR: table "nodelist" does not exist
2008-02-18 20:16:20,507 VOSPACE WARN java.sql.SQLException: ERROR: table "nodelistparams" does not exist
2008-02-18 20:16:20,507 VOSPACE WARN java.sql.SQLException: ERROR: table "nodeservice" does not exist
2008-02-18 20:16:20,507 VOSPACE WARN java.sql.SQLException: ERROR: table "nodeservicecapability" does not exist
2008-02-18 20:16:20,507 VOSPACE WARN java.sql.SQLException: ERROR: table "property" does not exist
2008-02-18 20:16:20,507 VOSPACE WARN java.sql.SQLException: ERROR: table "rootnode" does not exist
2008-02-18 20:16:20,507 VOSPACE WARN java.sql.SQLException: ERROR: table "servicecapability" does not exist
2008-02-18 20:16:20,507 VOSPACE WARN java.sql.SQLException: ERROR: table "serviceendpoint" does not exist
2008-02-18 20:16:20,507 VOSPACE WARN java.sql.SQLException: ERROR: table "storefile" does not exist
2008-02-18 20:16:20,508 VOSPACE WARN java.sql.SQLException: ERROR: table "transferstatus" does not exist
2008-02-18 20:16:20,508 VOSPACE WARN java.sql.SQLException: ERROR: table "treenode" does not exist
2008-02-18 20:16:20,508 VOSPACE DEBUG JSP reset done
This is expected behaviour, the database resest attempts to drop any existing tables before creating new ones.
The warnings are because the first time this is run, there are no tables to drop.
.auto, which will create a new tree using the unique identifier as the name.
For a test deployment you can accept the default file repository location, /tmp/vospace.
However, you will need to make sure that the directory exists, and is writable by the VOSpace service.
mkdir /tmp/vospace
chgrp users /tmp/vospace
chmod g+rwx /tmp/vospace
Note - at the moment the JSP pages do not check that the directory exists.
So if you miss this step, the only clue will be an IOException in the debug log when you do a data import.
org.astrogrid.*test* to org.astrogrid.*demo*,
to match the local registry, and add the new service.
This should create a new entry in the 'Services' list.
http://localhost:8080/astrogrid-vospace/registration-0.1/8a808088182e4e7501182ee44a2d0006
Copy the resource URL, open the Edit entry (name:astrogrid pass:W6G117)
page on your local registry, paste the URL into the 'Upload from URL' box and click the 'Upload' button.
If all goes well, then you should see a new entry for the MySpace 1.0 service
in your local registry.
default.vospace property is set to point to the local filemanager service.
org.astrogrid.community.default.vospace: java.lang.String = ivo://org.astrogrid.demo/filemanager
Login as the agtomcat user and shutdown the tomcat service
su - agtomcat
~/shutdown.sh
You need to edit the Community service context file and change the default.vospace property
to point to the new VOSpace service ivo://org.astrogrid.demo/*vospace-service*.
Open the context file in a test editor
vi ~/tomcat/conf/Catalina/localhost/astrogrid-community.xml
and change the property to point to the new VOSpace service
....
<!-- Configure the default FileManager -->
<Environment
description="The default FileManager"
name="org.astrogrid.community.default.vospace"
type="java.lang.String"
value="ivo://org.astrogrid.demo/filemanager" <-- Change this from 'filemanager' to 'vospace-service'.
/>
....
Alternatively, you can run the following command to use sed to edit the file in place.
sed -i ' s/org.astrogrid.demo\/filemanager/org.astrogrid.demo\/vospace-service/ ' ~/tomcat/conf/Catalina/localhost/astrogrid-community.xmlRestart Tomcat
~/start.sh
and reload the fingerprint page
to check that the default.vospace property is set to point to the local vospace-service service.
org.astrogrid.community.default.vospace: java.lang.String = ivo://org.astrogrid.demo/vospace-service
2008-02-19 00:28:38,720 VOSPACE INFO myspace [add account][8a808088182e4e7501182eb7e0dc0005][ivo://org.astrogrid.demo/alpha]and the new account will be listed in the Community Account admin page with the 'Home space' ivorn set to something like this :
ivo://org.astrogrid.demo/vospace-service#8a808088182f17a001182f17ac900001
exit.py Python script.
su - agtester
~/python/exit.py
exit
Start a separate command shell, login to the agtester account, and run the voexplorer.sh script.
su - agtester
~/voexplorer.sh
Start a separate command shell, login to the agtester account, and run the test script again,
using the new alpha account :
su - agtester
~/python/test-002.py -u alpha -p qwerty -l 10 -t
You should now see the operations logged in the vospace debug log.
su - agtomcat
tail -f ~/tomcat/logs/vospace-debug.log
2008-02-19 02:10:23,607 VOSPACE INFO myspace [read][8a808088182e4e7501182eb7e0dc0005][8a808088182f6eb701182f748a7e0574] 2008-02-19 02:10:23,622 VOSPACE INFO Interceptor [15][13][2][2][0] 2008-02-19 02:10:23,622 VOSPACE INFO ServiceImpl [15] 2008-02-19 02:10:23,623 VOSPACE INFO AxisServlet [18] 2008-02-19 02:10:23,627 VOSPACE INFO HttpExportServlet [GET][DATA][8a808088182f6eb701182f74c101061e] 2008-02-19 02:10:23,638 VOSPACE INFO Interceptor [11][11][2][0][0] 2008-02-19 02:10:23,640 VOSPACE INFO Interceptor [2][0][0][0][0] 2008-02-19 02:10:23,643 VOSPACE INFO Interceptor [3][0][1][0][0] 2008-02-19 02:10:23,651 VOSPACE INFO Interceptor [7][0][2][0][0]
webapp/target/ directory.
${VOSPACE_BASE}/current/webapp/webapp/target/astrogrid-vospace-webapp
Tomcat should re-compile the JSP pages automatically, and you don't need to re-deploy the webapp.
Note - If you want the changes to be preserved, then you need to copy the changed pages from the target directory into the src directory.
Otherwise, the JSP pages will revert back to their original form next time the webapp is built.
cp ${VOSPACE_BASE}/current/webapp/webapp/target/astrogrid-vospace-webapp/*.jsp \
${VOSPACE_BASE}/current/webapp/webapp/src/webapp \
If you make changes to the Java source code and need to need to re-build the webapp, you don't need to re-deploy the
The deployment technique used for the tests system adds a webapp context file astrogrid-vospace.xml to the Tomcat configuration, that references the
webapp in the project build target directory. This means that you can re-build the project using the Maven scripts, and Tomcat will get the new version immediately.
One warning - you need to stop Tomcat while the build is in progress.
The Maven build scripts are setup to delete the target directory each time the build is run.
If Tomcat is running and sees the webapp docbase directory dissapear, Tomcat will undeploy the webapp
and delete the context file.
So to re-build the Java source code, all you need to do is stop Tomcat, run the build, and then restart Tomcat.
su - agtomcat
~/shutdown.sh
exit
... build the webapp ....
su - agtomcat
~/start.sh
exit
If Tomcat does delete the context file, just login as the agtomcat user and run the following commands again to re-create it :
su - agtomcat
~/shutdown.sh
VOSPACE_BASE=/var/local/projects/vospace
cat >> ~/tomcat/conf/Catalina/localhost/astrogrid-vospace.xml << EOF
<?xml version='1.0' encoding='utf-8'?>
<Context
displayName="AstroGrid VOSpace service"
docBase="${VOSPACE_BASE}/current/webapp/webapp/target/astrogrid-vospace-webapp"
path="/astrogrid-vospace"
>
</Context>
EOF
~/start.sh
-- DaveMorris - 21 Feb 2008![]() |
Click here for the AstroGrid Service Web |
This is the AstroGrid Development Wiki |
|