Implementationcase: resolve the name of a VOSpace node to a URL

Motivation

• Get at the data indexed by VOSpace.

Story 1

A Service has been given the name of a node in MySpace from where it is to read a file. Service needs a URL from which it can stream the file contents. The file contains public data.

Service engages with the IVO Registry and locates the a VOSpace Directory service. Any directory service will do; VOSpace delegates requests as needed.

Service invokes the resolveNameToUrl operation on Directory. Service passes as parameters the name of the node and the access mode: "pull".

Directory executes IcDelegateOperationWithinVOSpace to pass on the request to the appropriate VOSpace service, Directory' say. Directory' extracts and returns the URL metadata (see discussion, below) for the node and returns them to Directory. Directory returns these metadata to Service.

The metadata include these elements.

   <dataStreamURL>http://some.server.somewhere/some/file/path/file.dat</dataStreamURL>
   <dataStreamURL>gsiftp://another.server.somewhere/some/file/path/file.dat</dataStreamURL>
   <dataStreamURL>http://yet.another.server/some/path/file-streamer.cgi?file=file42.dat</dataStreamURL>

The element type dataStreamURL means an element of which the value is a URL, and for which the URL points to the data directly, or the data via some CGI process that is entirely qualified by the URL (i.e. an HTTP-get CGI-service with all arguments included in the URL). These elements never contain a URL for a SOAP service.

Service picks the first HTTP URL because the file is too small to both with GridFTP and is public (so doesn't need a data-transport with an authenticated connection).

Story 2

As story 1, but the data are private and therefore do not appear on an unprotected file-server. There are no http or ftp URLs listed for the logical file; there is a gsiftp URL.

If service does not "speak" GridFTP , then service cannot get the data.

Story 3

As story 2, but Directory' can generate HTTP URLs including single-use passwords. These URLs point to CGI programmes that check the password and stream the desired data.

In general, this syetem only works if Directory' can talk to an intelligent storage service to arrange the single-use password. AstroGrid MySpace and Storage Resource Broker are examples of this kind of storage layer. If Directory' can only talk to basic web and FTP servers, then this story does not work and the system reverts to story 2.

See also IcAppLoadsDataFromNet, story 2.

Story 4

As story 2, but Service needs to write to the file. It sends push as the mode parameter. Directory' allows only file and gsiftp schemes, in order to control access to the file, and fauls for other schemes. When a mirror set of gsiftp URLs is available, Directory' selects the master copy of the file to be written to.

-- GuyRixon - 28 Jun 2004, updated by GuyRixon - 14 Jul 2004