Further discussions about VoStore^? and VoSpace^?.

Notes from discussions about VoStore^? and VoSpace^?.

Cambridge 25th October 2005

Participants

• Matthew Graham
• GuyRixon
• DaveMorris

Latest draft of registering VoStore^? capabilities VoStoreCapabilities20051027

Registry resources for URIs

Using registry resources as placeholders for identifiers.

For example : registry entry for JPEG file format

    <vo:Resource>
        <title>JPEG file format</title>
        <identifier>ivo://net.ivoa/formats/jpeg</identifier>
        ....
        <content>
            <subject>Registry entry for JPEG file format</subject>
            <description>
                Some text describing the JPEG format ....
            </description>
        </content>
    </vo:Resource>

This associates the URI [ivo://net.ivoa/formats/jpeg] with the JPEG format.

This avoids the restrictions inherent in defining enumerations in the service WSDL. If we add a new format to the existing enumeration, it means the service WSDL has changed, so it is no longer compatible with existing clients.

If someone invents a new file format, all they need to do is create a corresponding registry resource.

For example : registry entry for new file format

    <vo:Resource>
        <title>New file format</title>
        <identifier>ivo://net.ivoa/formats/kpeg</identifier>
        ....
        <content>
            <subject>Registry entry for KPEG file format</subject>
            <description>
                Some text describing the new KPEG format ....
            </description>
        </content>
    </vo:Resource>

VoStore^? services can then use the URI [ivo://net.ivoa/formats/kpeg] to refer the new format.

Same applies for

• data formats
  • xml
  • votable
  • jpeg
  • fits
  • fits (nested)

• transport protocols
  • http
  • https
  • webdav
  • ftp
  • gftp
  • srb

• security protocols
  • SSL + pass
  • SSL + cert
  • WSS + cert
  • anon

In each case, the primary reason is to associate a unique URI with a specific format or protocol. It also provides a well understood process for registering a new data format or protocol.

These URI identifiers could be used to list the formats and protocols that a particular VoStore^? service supported.

For example : formats response from a VoStore^? service

    <formats>
        <format>ivo://net.ivoa/formats/jpeg</format>
        <format>ivo://net.ivoa/formats/fits</format>
        ....
    </formats>

Registry URIs for service capabilities

We can also use the same technique to register service contracts. This is already supported by the Capability element in the current registry schema.

   <xs:complexType name="Capability">
      <xs:annotation>
         <xs:documentation>
            To provide a metadata specific to a particular Service type, 
            this type is usually extended and added as a child of the 
            specific Service subclass.  
         </xs:documentation>
      </xs:annotation>
     ....
      <xs:attribute name="standardID" type="vr:IdentifierURI">
         <xs:annotation>
            <xs:documentation>
               An IVOA identifier for a standard service. 
            </xs:documentation>
            <xs:documentation>
               This provides a unique way to refer to a service
               specification standard, such as a Simple Image Access service.
               It assumes that such standard is registered and accessible.  
            </xs:documentation>
         </xs:annotation>
      </xs:attribute>
      <xs:attribute name="standardURL" type="xs:anyURI">
         <xs:annotation>
            <xs:documentation>
               A URL that points to a human-readable document that
               describes the standard upon which a service is based. 
            </xs:documentation>
         </xs:annotation>
      </xs:attribute>
   </xs:complexType>

For example : registry entry for VoStore^? version 1.0

    <vo:Resource>
        <title>VoStore 1.0</title>
        <identifier>ivo://net.ivoa/services/vostore-1.0</identifier>
        ....
        <content>
            <subject>Registry entry for VoStore version 1.0</subject>
            <description>
                Some text describing the VoStore version 1.0 ....
            </description>
        </content>
    </vo:Resource>

This associates the URI [ivo://net.ivoa/services/vostore-1.0] with a specific version of the VoStore^? service API. The URI can then be used to register an instance of this service, by referring to the specific URI for the API the service implements.

For example : registry entry for a VoStore^? 1.0 service

    <vo:Resource xsi:type="vo:Service">
        ....
        <Capability standardID="ivo://net.ivoa/services/vostore-1.0"/>
    </vo:Resource>

Service APIs that include a core API and optional extensions could create resource documents for each extension.

For example : registry entries for core SIAP service and extensions "A" and "B"

    <vo:Resource>
        <title>SIAP 1.0 core</title>
        <identifier>ivo://net.ivoa/services/siap-1.0-core</identifier>
        ....
        <content>
            <subject>Registry entry for SIAP 1.0 core</subject>
            <description>
                Some text describing SIAP 1.0 core service API ...
            </description>
        </content>
    </vo:Resource>

    <vo:Resource>
        <title>SIAP 1.0 ext A</title>
        <identifier>ivo://net.ivoa/services/siap-1.0-a</identifier>
        ....
        <content>
            <subject>Registry entry for SIAP version 1.0, extension A</subject>
            <description>
                Some text describing SIAP extension A ...
            </description>
        </content>
    </vo:Resource>

    <vo:Resource>
        <title>SIAP 1.0 ext B</title>
        <identifier>ivo://net.ivoa/services/siap-1.0-b</identifier>
        <curation>
            ....
        </curation>
        <content>
            <subject>Registry entry for SIAP version 1.0, extension B</subject>
            <description>
                Some text describing SIAP extension B ...
            </description>
        </content>
    </vo:Resource>

This would enable us to declare the service API version and supported extensions in the registry entry for a service.

For example : registry entry for a service that supports SIAP 1.0 and extensions "A" and "B"

    <vo:Resource xsi:type="vo:Service">
        ....
        <!--+
            | Service implements SIAP 1.0 and extensions "A" and "B".
            +-->
        <Capability standardID="ivo://net.ivoa/services/siap-1.0-core"/>
        <Capability standardID="ivo://net.ivoa/services/siap-1.0-a"/>
        <Capability standardID="ivo://net.ivoa/services/siap-1.0-b"/>
    </vo:Resource>

This may need a change to the registry schema to enable multiple Capability elements for a Service

Registry URI for interface capabilities

If we extend the registry schema to add zero or more Capability elements to a Service Interface, then this enables us to describe specific capabilities for each service interface.

This would enable us to describe a service that supported different versions of an API on diferent interfaces.

For example : registry entry for a VoStore^? service that supports VoStore^?-1.0 and VoStore^?-1.2

    <vo:Resource xsi:type="vo:Service">
        ....
        <!--+
            | Service implements VoStore-1.0 and VoStore-1.2.
            +-->
        <Capability standardID="ivo://net.ivoa/services/vostore-1.0"/>
        <Capability standardID="ivo://net.ivoa/services/vostore-1.2"/>
        ....
        <!--+
            | This interface implements VoStore-1.0.
            +-->
        <Interface>
            <Capability standardID="ivo://net.ivoa/services/vostore-1.0"/>
            <AccessURL>http://....</AccessURL>
        </Interface>
        ....
        <!--+
            | This interface implements VoStore-1.2.
            +-->
        <Interface>
            <Capability standardID="ivo://net.ivoa/services/vostore-1.2"/>
            <AccessURL>http://....</AccessURL>
        </Interface>
    </vo:Resource>

This will need a change to the schema to enable multiple Capability elements for a Service Interface

Service capabilities for other capabilities

This technique could be extended to include details about the transport protocols, security protocols and data formats that a service or interface can handle.

For example : full registry entry for a VoStore^? service that supports VoStore^?-1.0 and VoStore^?-1.2, with details of the protocols supported by each interface

    <vo:Resource xsi:type="vo:Service">
        ....
        <!--+
            | Service implements VoStore-1.0 and VoStore-1.2.
            +-->
        <Capability standardID="ivo://net.ivoa/services/vostore-1.0"/>
        <Capability standardID="ivo://net.ivoa/services/vostore-1.2"/>
        ....
        <!--+
            | Service supports [WSS + cert] and [SSL + cert] security protocols.
            +-->
        <Capability standardID="ivo://net.ivoa/security/wss.cert-1.0"/>
        <Capability standardID="ivo://net.ivoa/security/ssl.cert-1.1"/>
        ....
        <!--+
            | Service supports [http], [https], and [gridsite] transfer protocols.
            +-->
        <Capability standardID="ivo://net.ivoa/transfer/http-1.1"/>
        <Capability standardID="ivo://net.ivoa/transfer/https-1.1"/>
        <Capability standardID="ivo://net.ivoa/transfer/gridsite-1.0"/>
        ....
        <!--+
            | This interface implements VoStore-1.0.
            | It can accept [WSS + cert] authentication.
            | It can handle [fits] and [votable] formats.
            | It can handle [http] and [https] transfer protocols.
            +-->
        <Interface>
            <Capability standardID="ivo://net.ivoa/services/vostore-1.0"/>
            <Capability standardID="ivo://net.ivoa/security/wss.cert-1.0"/>
            <Capability standardID="ivo://net.ivoa/formats/votable-1.0"/>
            <Capability standardID="ivo://net.ivoa/formats/fits-1.0"/>
            <Capability standardID="ivo://net.ivoa/transfer/http-1.1"/>
            <Capability standardID="ivo://net.ivoa/transfer/https-1.1"/>
            <AccessURL>http://....</AccessURL>
        </Interface>
        ....
        <!--+
            | This interface implements VoStore-1.2.
            | It can accept [WSS + cert] and [SSL + cert] authentication.
            | It can handle [fits], [votable] and [binary] formats.
            | It can handle [http], [https] and [gridsite] transfer protocols.
            +-->
        <Interface>
            <Capability standardID="ivo://net.ivoa/services/vostore-1.0"/>
            <Capability standardID="ivo://net.ivoa/security/wss.cert-1.0"/>
            <Capability standardID="ivo://net.ivoa/formats/votable-1.0"/>
            <Capability standardID="ivo://net.ivoa/formats/fits-1.0"/>
            <Capability standardID="ivo://net.ivoa/transfer/http-1.1"/>
            <Capability standardID="ivo://net.ivoa/transfer/https-1.1"/>
            <Capability standardID="ivo://net.ivoa/transfer/gridsite-1.0"/>
            <AccessURL>http://....</AccessURL>
        </Interface>
    </vo:Resource>

Possible to use UCD for data formats.

Example :

    <!--+
        | An interface that returns images in FITS and JPEG formats.
        +-->
    <Interface>
        <Capability ucd="meta.format.fits"/>
        <Capability ucd="meta.format.jpeg"/>
    </Interface>

Possible to use both UCD and URI for data formats.

Example :

    <!--+
        | An interface that returns images in FITS and JPEG formats.
        +-->
    <Interface>
        <Capability uri="ivo://net.ivoa/formats/fits-1.0" ucd="meta.format.fits"/>
        <Capability uri="ivo://net.ivoa/formats/jpeg-1.0" ucd="meta.format.jpeg"/>
    </Interface>

Using lists of capabilities enables us to describe services that provide more than one service e.g. SIAP and VoStore^?.

Example : registration for a service that implements both SIAP and VoStore^?

    <!--+
        | Registration document for a service that implements both SIAP and VoStore.
        +-->
    <vo:Resource xsi:type="vo:Service">
        ....
        <!--+
            | Registered as a VoStore service, enabling UI tools to look for 'any vostore'.
            | vostore:Capability extends vo:Capability to add VoStore specific data.
            +-->
        <Capability xsi:type="vostore:Capability" uri="ivo://net.ivoa/services/vostore">
            <!--+
                | Additional elements specific to VoStore, e.g. store-size.
                +-->
        </Capability>

        <!--+
            | Registered as a SIAP service, enabling UI tools to look for 'any SIAP'.
            | siap:Capability extends vo:Capability to add SIAP specific data.
            +-->
        <Capability xsi:type="siap:Capability" standardID="ivo://net.ivoa/services/siap">
            <!--+
                | Additional elements specific to SIAP.
                +-->
        </Capability>

        <!--+
            | This interface implements VoStore-1.0.
            | It can accept [WSS + cert] authentication.
            | It can handle [http] and [https] transfer protocols.
            | It can handle [fits], [jpeg] and [votable] formats.
            +-->
        <Interface>
            <Capability uri="ivo://net.ivoa/services/vostore-1.0"/>
            <Capability uri="ivo://net.ivoa/security/wss.cert-1.0"/>
            <Capability uri="ivo://net.ivoa/formats/votable-1.0"/>
            <Capability uri="ivo://net.ivoa/formats/fits-1.0" ucd="meta.format.fits"/>
            <Capability uri="ivo://net.ivoa/formats/jpeg-1.0" ucd="meta.format.jpeg"/>
            <Capability uri="ivo://net.ivoa/transfer/http-1.1"/>
            <Capability uri="ivo://net.ivoa/transfer/https-1.1"/>
            <AccessURL>http://....</AccessURL>
        </Interface>
        ....
        <!--+
            | This interface implements the core SIAP-1.0, plus extension 'A'.
            | It can accept [WSS + cert] and [SSL + cert] authentication.
            | It returns images in FITS and JPEG formats.
            +-->
        <Interface>
            <Capability uri="ivo://net.ivoa/services/siap-1.0-core"/>
            <Capability uri="ivo://net.ivoa/services/siap-1.0-ext.a"/>
            <Capability uri="ivo://net.ivoa/security/wss.cert-1.0"/>
            <Capability uri="ivo://net.ivoa/formats/fits-1.0" ucd="meta.format.fits"/>
            <Capability uri="ivo://net.ivoa/formats/jpeg-1.0" ucd="meta.format.jpeg"/>
            <AccessURL>http://....</AccessURL>
        </Interface>

    </vo:Resource>

Changes to existing VoStore^? doc.

• REST style service - v 2.0 or later (if required).

• VoStore^? assigns ident internally
  • Remove name from import and put methods.

• DIME moved to just another protocol
  • importInit with 'protocol=DIME'
  • => endpoint url of DIME webservice to put to (not necessarily this one)
  • => identifier to put as (not necessarily same as object ident)

• MTOM to replace DIME as soon as toolkits available

• Investigate GridFtp^? ?
• Investigate GridSite^? ?
• Investigate ApacheModule^? ?

• Default protocols (we have at the moment).
  • http get (ok for public data)
  • http put (no auth, size problems - fixed in JDK 1.5)
  • dime get
  • dime put (size problems - might be fixed in Axis 1.3 & JDK 1.5)
  • https get (unknown)
  • https put (unknown)

• Is this enough to get standard agreed ?

• Remove listAll from API - no use case apart from local admin (which is impl specific).
• list returns all objects where owner=[DN=....]
  • ident : string
  • owner : string
  • created : date
  • properties ....

• Does modified date make sense for a VoStore^? ?
• Are there issues with comparing [DN=....] as strings ?

• Add get/set property to API.

• release(ident) sets current owner to 'null'
• Individual service may throw 'NotSupported'

• adoptInit(ident, whoto) - allows 'whoto' to adopt a file.
• equivalent to grant([adopt], ident, who) but want to keep 'grant' for later.
• adopt(ident) - adopt a file to my ownership - either root, or have been granted adopt rights.

• Service specific behaviour
  • release allowed (may still track original owner)
  • public read to all files ?
  • public delete to released files ?

• Investigate WSRF for v 2.0 ?

• Exceptions
  • ActionNotSupported^? * ObjectNotFound^? * IdentSyntaxError^? * PermissionDenied^? * AuthenticationFailed^? * InternalServerError^?
• Need to decide which are server and which are client.