Resolving myspace and vospace identifiers

Myspace ivorns

In order to look at the options available for handling vospace identifiers within AstroGrid, we need to look at how the current system deals with myspace identifiers.

The current system uses two forms of ivo:// identifiers to refer to objects in myspace.

Concrete ivorns

The first form, which I'll call the concrete form, combine the registry identifier for a FileManager^? service with the identifier for a specific node within the service.

ivo://authority/filemanager#node

The concrete form may also include an optional path, pointing to a child node within the FileManager^? service.

ivo://authority/filemanager#node/path/file

The part before the # symbol is a conventional IVOA identifier for a registered FileManager^? service, resolvable in the IVOA registry.

The fragment part of the identifier points to a specific node within the FileManager^? service, and the optional path points to a location relative to the initial node.

The fragment identifier may point to the specific node itself, or the fragment may identify a parent or grandparent node and a path to the target node. This means that the following identifiers both refer to the same node.

The first case points directly to the target node (node-102).

The second case points to a grandparent node (node-100), and then uses a path to identify the target node (path/file) relative to the initial node.

Abstract ivorns

The second form of ivo:// identifiers used by the current system relies on an AstroGrid naming convention to create an identifier which refers to an object in a users myspace account.

I'll call this form an abstract identifier, because in the conventional IVOA scheme they don't actually refer to anything in the registry.

An abstract identifier looks like this :

ivo://authority/user#path/file

The initial part of the identifier, the part before the # symbol, looks like it refers to a user account. Which it does, but only indirectly. If you try to resolve the IVOA part of an abstract myspace identifier in the registry you will get a null result, because the user accout it refers to is not registered in the registry.

Resolving the abstract identifier relies on a naming convention that was added early on in the project.

The first step is to locate the corresponding community service for the user account. This involves taking the registry authority ID from the identifier (everything upto the first '/') :

ivo://authority

and appending the 'standard service name' (the Java class name) for the community PolicyManager^? service :

ivo://authority/org.astrogrid.community.common.policy.manager.PolicyManager

This can then be resolved into the community PolicyManager^? service using the registry.

The user name part of the abstract identifier (from the first '/' to the '#') can be used to request the account details from the community PolicyManager^? service. The information returned by the community service includes the concrete identifier for the root node of the users home space.

This can be combined with the path fragment from the original abstract identifier to create a concrete identifier for the target node in a FileManager^? service.

The resulting concrete identifier can then be processed via the registry to resolve the FileManager^? service and target node.

Client interaction

Although internally parts the AstroGrid client AR uses concrete identifiers to build the myspace explorer tree, it always displays abstract identifiers to the user. All the user interaction, including the GUI selection screens and the Python script API, use abstract identifiers.

Note : It is possible to use concrete FileManager^? identifiers in some parts of the Python scripting API, but this tends to leave the GUI display in an unstable state.

Service interaction

Abstract identifiers

When the client AR sends a myspace reference to a CEA or DSA service it currently passes the unprocessed abstract identifier in the message. This means that the service has to resolve the abstract identifier into a concrete FileManager^? identifier before it attempts to access the myspace location.

---

This represents the stage we are at now.

---

Vospace identifiers

Although the current plan for the next release is to use the FileManager^? interface on the vospace service, we also need to begin to look at how we want to handle vospace vos:// identifiers in the future.

Abstract identifiers

At the moment, all vospace identifiers are concrete, there is no equivalent to the abstract myspace identifier used within AstroGrid.

One proposal that would enable us to create abstract vospace identifiers is to use the vos:// naming scheme, but add an AstroGrid specific syntax that identifies the user name and community within the vos:// identifier to create an abstract vospace identifier.

For example :

vos://authority!vospace/~user@community/path/file

or

vos://~user@community/path/file

Adopting this scheme may cause a number of problems, the primary one being that at the moment there is no concept of a community service within the IVOA. This means that any extensions or special meaning that we add to the IVOA vos:// naming scheme would be AstroGrid specific.

If we wanted to be able to use the new abstract vospace identifiers with 3rd party SIAP, SSAP, UWS and TAP services we would need to get IVOA approval for the new identifier syntax, and by implication, would require IVOA agreement on the concept and behaviour of what a community service is and does.

This could take a while.

Without IVOA approval, only AstroGrid services would be able to process the abstract vospace identifiers, making it difficult to use AstroGrid and 3rd party IVOA services together.

Concrete identifiers

A different approach would be to gradually phase out the use of the abstract identifiers by the services, and move to a system that only used concrete identifiers for everything apart from client side user interaction.

Both the existing CEA and DSA services should already be equipped to handle concrete FileManager^? identifiers.

In fact, when the current resolver library receives a myspace reference, the first thing it tries to do is resolve it as concrete identifier, by passing the resource name to the registry.

It is only after the initial registry lookup fails that the resolver treats the myspace reference as an abstract identifier and tries to resolve it via the community service.

So at the moment, the sequence of steps involved in resolving a myspace reference by a CEA or DSA service includes an initial failed attempt to treat the reference as a concrete FileManager^? identifier, followed by the steps required to resolve an abstract identifier.

Note that the failed lookup adds an additional delay to the processing because the result can't be cached easily. A registry client can cache a positive result, but a negative 'resource not found' response is more difficult to cache, it might be 'not found' because the identifier really is invalid, or because a newly updated resource hasn't been harvested yet. In the latter case, any cached 'not found' results would prevent newly updated resources from being discovered and used until all the cached entries in all the various services had expied (this doesn't make it impossible to cache, just more complicated).

In theory, if we modified the client AR to pass myspace references as resolved concrete identifiers then this could simplify the number of steps a CEA or DSA service would need to take to access a myspace location.

Just making this change on its own would decrease the number of web service calls and increase the reliability of passing myspace references to a CEA or DSA service.

Note that the registry used by the CEA or DSA service may not be the same registry service used by the client AR to generate the abstract identifier in the first place.

If a user registered with the Leicester community sends a query to a DSA service at ROE, then the client AR will have used the default registry service at Leicester, but the DSA service at ROE will be configured to use the local ROE registry to resolve the abstract myspace reference.

If the ROE registry contained different data to the Leicester registry (perhaps ROE registry has not harvested the most recent changes, or the ROE registry is temporarily offline), then the DSA will be unable to transfer data into the users myspace.

However, as far as the user is concerned there does not appear to be any problem with accessing data in their myspace account from the client in Leicester.

Vospace identifiers

Serverside delegate

The next stage would be to add a vospace delegate to the library installed in the DSA and CEA services. This would be capable of resolving concrete vos:// identifiers, enabling our DSA and CEA services to access data in vospace 1.1 services.

Note : Even if we were planning on keeping some form of abstract identifier in the AstroGrid services, we would need to provide a mechanism for resolving concrete vospace identifiers in order to update our CEa and DSA services to meet the new UWS and TAP specifications.

Clientside delegate

The next stage in the process would be to add an Apache VFS interface to the vospace client, enabling the client AR to access and display data in a vospace 1.1 service.

At this point, the client AR would now be using concrete vos://vospace identifiers internally, but the user interface could still display some form of abstract myspace identifiers. When the AR client tries to resolve an abstract identifier into a concrete one, the identifier returned by the community service would now return a vos:// identifier pointing to a location within a vospace service rather than an ivo:// FileManager^? identifier.

The final stage would be to update the GUI to enable it to display data in other vospace services, in addition to the users home space. Any external vospace services would have to be displayed using their full concrete vos:// identifiers, as they would not be associated with a specific user account or a community service.

Abstract home

A separate change that could be introduced early on in the process would be to use a separate URI scheme to represent an AstroGrid users myspace home, to replace the non-standard ivo://authority/user#path/file identifiers used at the moment.

There have been a number of alternatives proposed for this :

• Using the Unix standard '~'
• Using workspace://path/file (there may already some code in the client AR that supports this)
• Using myspace://path/file

Out of these my own favorite would be myspace://path/file.

Using the Unix standard '~' is tempting, but if this was used in Python scripts it would not be clear if this referred to the users 'myspace' home, or their 'home' directory on the local disk.

Equally, workspace://path/file is not that clearly identified with AstroGrid, and there are already several other tools that use the word 'workspace' to denote the current work directory.

Whichever prefix we decide use, the resolving process would be the same.

Once we start to use concrete identifiers in the DSA and CEA messages, then only the AR client needs to know the users identity and home space.

The client AR, GUI and Python scripts can use an abstract myspace://path/file identifier to refer to data within the current users myspace account. Any abstract identifiers would be resolved by the client AR into their corresponding concrete ivo://filemanager or vos://vospace identifiers before being passed to an external service.

So, initially the process would be

then once we move to using vos://vospace identifiers

The GUI and Python script API would use abstract myspace://path/file identifiers to refer to locations relative to the current users myspace and concrete vos:// vospace identifiers to refer to specific locations within the global vospace namespace.

In effect, a myspace:// identifier becomes a synonym for 'something in my (the current users) part of the global vospace'.

The advantage of doing this is that it moves us away from relying on the current AstroGrid specific naming conventions to decode the abstract myspace identifiers. It also makes it clear that the abstract myspace identifiers do not point to IVOA registry resources.

-- DaveMorris - 28 Dec 2007

Corrections

Use workspace for users data

When I wrote the article I didn't know that the GUI design team had already decided that the home space identifiers should be workspace:/path/file. So replace myspace://path/file with workspace:/path/file in the section on the abstract home URIs.

Alternate form of concrete ivorns

The system can cope with two forms of the concrete identifiers.

ivo://authority/filemanager#node

and

ivo://authority/filemanager#node/path/file

Although the existing FileManager^? normally uses the short form with just the target node identifier, it may be better to use the long from containing the root node identifier and path during the transition. Using this form works better in the new server implementation, but we would need to check that the existing FileManager^? service can handle this form too.

-- DaveMorris - 08 Jan 2008