This document provides suggested templates for REST-based APIs, for potential adoption by many of the services in CollectionSpace's services layer. This implements part of the work described in the "REST Service Contracts" section of Web Services - Paradigms and Contracts.

Some generic message payloads and envelopes that are to be used with the proposed APIs below, such as the message payloads to be returned in a response when an error occurs, are detailed in Common System Specific Elements.

REST-based APIs, and their associated client-visible resource models, are suggested below for the three major types of CollectionSpace services: entity services, relation (association) services, and task services:

REST-based APIs for Entity Services

• Service Discovery
  • Description
  • Schema
• CRUD Operations (Create, Read, Update, Delete)
  • Create
  • Read
  • Read (Multiple)
  • Update
  • Delete
• Search rough, in-process notes

REST-based APIs for Relation (Association) Services

REST-based APIs for Task Services

Conventions and Practices

References

REST-based APIs for Entity Services

Service Discovery

Description

Purpose
Retrieves a human-readable description of the service. This also verifies that the service exists, without interacting with the resources that it provides.

Request

GET /{resources_as_a_plural_noun}/description

Request: Entity Body
The body of this request SHOULD be empty. If it is non-empty, the body will be ignored.

Response: Status Codes

Response: Content-Type

• text/plain (for plain text)
• application/xml (for generic XML content)
  See Questions or Issues, below.

Response: Entity Body (on Success)
Returns a human-readable description of the service: in plain text, in an XML format, or both, TBA. See Questions or Issues, below.

Response: Entity Body (on Error)
Returns an XML document containing a description of the error. See Common System Specific Elements for details.

As RESTful Service - Ajax Patterns notes, "Error responses should ... be in human-friendly language and with examples and hyperlinks."

Questions or Issues

• We need to decide if this description is made available in plain text, in a particular XML format, or both. If both, the default format can be retrieved using the pattern above, and specific formats can be retrieved by adding filename extensions; e.g.:
  • /{resources_as_a_plural_noun}/description.txt (for a plain text description)
  • /{resources_as_a_plural_noun}/description.xml (for an XML-based description)

Schema

Purpose
Retrieves the schema for the XML-based message payload that is exchanged in several of this service's operations.

Instance documents conforming to this schema may be exchanged in a variety of contexts. For instance, they may be:

• Sent by a client in the body of a request, to perform a Create operation.
• Sent by a client in the body of a request, to perform an Update operation.
• Returned by the service in a response, when a specific object (entity) is requested.
• Returned by the service in a response, when a list of objects (entities) is requested, in a format that returns full object records, (rather than just hyperlinks, or just hyperlinks accompanied by abbreviated object records).

Request

GET /{resources_as_a_plural_noun}/schema

There will be only one schema returned per resource. If there may be typed resources, or child resources, available anywhere within the URL path, schemas for these resources can be retrieved via variations on this generic pattern, as in this example:

GET /{resources_as_a_plural_noun}/{typed_or_child_resources_as_a_plural_noun}/schema

Request: Entity Body
The body of this request SHOULD be empty. If it is non-empty, the body will be ignored.

Response: Status Codes

Response: Content-Type
application/xml (for schemas expressed in XML-based schema languages)

Response: Entity Body (on Success)
Returns the schema for the XML-based message payload that is exchanged in several of this service's operations.

By default, the schema will be declared in the W3C XML Schema language.

Response: Entity Body (on Error)
Returns an XML document containing a description of the error. See Common System Specific Elements for details.

Questions or Issues

• If there may be a requirement to return schemas in additional schema languages, such as RELAX NG, the default format can be retrieved using the pattern above, and specific formats can be retrieved by adding filename extensions; e.g.
  • /{resources_as_a_plural_noun}/schema.xsd (for a W3C XML Schema document)
  • /{resources_as_a_plural_noun}/schema.rng (for a RELAX NG schema document)

CRUD Operations (Create, Read, Update, Delete)

Create

Purpose
Creates a new resource of the specified type.

Request

POST /{resources_as_a_plural_noun}

The RESTful metaphor is that of creating a new instance of the resource within a container of, or bucket of, resources of this type.

If there may be typed resources, or child resources, available anywhere within the URL path, new resources of that type can be created via variations on this generic pattern, as in this example:

POST /{resources_as_a_plural_noun}/{typed_or_child_resources_as_a_plural_noun}

Request: Entity Body
If the service allows the client to supply the resource to be created:
The body of this request MUST contain a valid instance of an XML document, conforming to the schema used by the service.

If the service generates an instance of the resource algorithmically:
The body of this request SHOULD be empty. If it is non-empty, the body will be ignored.

Response: Status Codes

Response: Content-Type
application/xml

Response: Entity Body (on Success)
Returns a representation of the newly-created resource, as an XML document conforming to the schema used by the service.

Response: Entity Body (on Error)
Returns an XML document containing a description of the error. See Common System Specific Elements for details.

Questions or Issues

• We need to decide when a POST of a new resource to a container or bucket results in a 409 Conflict error. One example may be when the value of an information unit in the POSTed resource conflicts with a value of that same information unit in an existing resource. This might occur, for instance, in the case of a RDBMS column (field) on which there is a PRIMARY KEY or UNIQUE constraint.

Read

Purpose
Reads (aka retrieves or returns) a representation of a specific instance of this resource.

Request

GET /{resources_as_a_plural_noun}/{resource_identifier}

The {resource_identifier} is a value that uniquely identifies a particular resource. It may be an ID or some other type of identifier.

If there may be typed resources, or child resources, available anywhere within the URL path, these resources can be retrieved via variations on this generic pattern, as in this example:

GET /{resources_as_a_plural_noun}/{typed_or_child_resources_as_a_plural_noun}/{resource_identifier}

Request: Entity Body
The body of this request SHOULD be empty. If it is non-empty, the body will be ignored.

Response: Status Codes

Response: Content-Type
application/xml

Response: Entity Body (on Success)
Returns a representation of the requested resource, as an XML document conforming to the schema used by the service.

Response: Entity Body (on Error)
Returns an XML document containing a description of the error. See Common System Specific Elements for details.

Read (Multiple)

Purpose
Reads (aka retrieves or returns) a list of 0 to n instances of this resource.

Depending on the query parameters submitted, the list may include either:

• 0 to n hyperlinks, each accompanied by the minimum information required to identify, in a human-readable manner, their linked-to resources.
  • Hyperlinks are fully-qualified (i.e. not relative) URLs that may subsequently be used to retrieve any of the individual records in the list, via the Read operation, above.
  • Hyperlinks are accompanied by human-readable information, which provides a brief preview of what may be expected to be retrieved by following those links. This information is typically obtained from one or more of the information units (aka fields or data elements) in each record, and may represent an abbreviation, combination or adaptation of that information. The content of this human-readable identification is service-specific.

or

• 0 to n full records for matching resources.

The default behavior is to return hyperlinks, each accompanied by the minimum information required to identify, in a human-readable format, their linked-to resources.

Request

GET /{resources_as_a_plural_noun}

GET /{resources_as_a_plural_noun}?{optional_query_parameters ...}

If there may be typed resources, or child resources, available anywhere within the URL path, these resources can be retrieved via variations on this generic pattern, as in these examples:

GET /{resources_as_a_plural_noun}/{typed_or_child_resources_as_a_plural_noun}

GET /{resources_as_a_plural_noun}/{typed_or_child_resources_as_a_plural_noun}?{optional_query_parameters ...}

A highly recommended child resource, for read (multiple) operations, will return a count of the numbers of records to be returned, as in this example:

GET /{resources_as_a_plural_noun}/count

GET /{resources_as_a_plural_noun}/count?{optional_query_parameters ...}

Query parameters

Some possible examples of query parameters to modify read (multiple) requests. These in part come from Common System Specific Elements:

?orderby={information_unit}
Requests that hyperlinks or full records be returned in the natural sort order of a particular information unit (field). Defaults to ascending order if the sortorder query parameter isn't present.

sortorder={ascending|descending}
Specifies the sort order. Ignored if the orderby query parameter isn't also present.

?groupby={}
Specifies grouping of results. (Additional details TBA.)

?pagesize={nn} (or maxrecords=nn, etc.)
Return results with a maximum of nn items (hyperlinks or full records) returned per query.

?format={hyperlinks|full_records}
Requests that either hyperlinks or full records be returned. Defaults to hyperlinks.

Note that these query parameters may be combined; e.g. ?orderBy={information_unit}&pageSize={nn}

Request: Entity Body
The body of this request SHOULD be empty. If it is non-empty, the body will be ignored.

Response: Status Codes

Response: Content-Type
application/xml

Response: Entity Body (on Success)
Returns a representation of the requested resource, as an XML document conforming to the schema used by the service.

Response: Entity Body (on Error)
Returns an XML document containing a description of the error. See Common System Specific Elements for details.

Questions or Issues

• We need to decide on XML-based representations for returning lists of entities:
  • With hyperlinks and abbreviated identification
  • With full records
• We need to fully discuss the types of query parameters required.
• We need to fully discuss how paging of the results returned (via a query parameter like pagesize or maxrecords), in responses from CollectionSpace service, will work. One way of facilitating this might be to return, in an envelope or preface in the entity body, values indicating a) the total number of records retrieved, b) the start position (index) of the first record in the current result set, and c) the numbers of records returned in that set. Note that this is not currently expected to use HTTP's Chunked Transfer Encoding mechanism. Rather, each response is expected to be a complete HTTP message, with some indication (such as the envelope or preface mentioned above) that the results returned constitute a subset of the total records available.
• We need to decide and standardize on a convention for query parameter names that combine multiple words, including case and/or punctuation (e.g. "queryparameter", "query-parameter", "queryParameter" ...)

Update

Purpose
Completely updates in place (i.e. replaces) an existing resource of the specified type.

Request

PUT /{resources_as_a_plural_noun}/{resource_identifier}

The {resource_identifier} is a value that uniquely identifies a particular resource. It may be an ID or some other type of identifier.

If there may be typed resources, or child resources, available anywhere within the URL path, resources of that type can be updated via variations on this generic pattern, as in this example:

PUT /{resources_as_a_plural_noun}/{typed_or_child_resources_as_a_plural_noun}/{resource_identifier}

Request: Entity Body
The body of this request MUST contain a valid instance of an XML document, conforming to the schema used by the service.

Response: Status Codes

Response: Content-Type
application/xml

Response: Entity Body (on Success)
Returns a representation of the updated resource, as an XML document conforming to the schema used by the service.

Response: Entity Body (on Error)
Returns an XML document containing a description of the error. See Common System Specific Elements for details.

Questions or Issues

• There doesn't appear to be well-defined support in the REST architecture for partial (sparse) updates of a resource, equivalent to the UPDATE table_name SET column=value statement in SQL. Some discussion:
  • Bill de hÓra, "'Just' use POST"
  • Roy T. Fielding, "It is okay to use POST"

Two of several mechanisms through which we might facilitate partial (sparse) updates, noting that these mechanisms are not currently suggested here as patterns, include:

• Providing direct access to any interesting information units (fields or data elements) via URL-addressable resources; e.g. /{resources_as_a_plural_noun}/{resource_identifier}/{information_unit_name}, and then using PUT to completely update the contents of any of these individual information units.
• POSTing (as opposed to PUTting) a partial update to the resource at /{resources_as_a_plural_noun}/{resource_identifier}. This sidesteps the lack of support for partial (sparse) updates in REST, since the semantics of POST are entirely up to the application. The payload sent in the body of the update request might consist of form data in the 'application/x-www-form-urlencoded' content-type, or might potentially be in an XML format (TBA). There may well need to be some distinction, in any partial (sparse) update submitted in those or similar formats, between fields that are simply to be ignored - not processed - in the update request, and fields whose values are to be removed (made empty, or blank).

Delete

Purpose
Deletes a specific instance of this resource.

Request

DELETE /{resources_as_a_plural_noun}/{resource_identifier}

The {resource_identifier} is a value that uniquely identifies a particular resource. It may be an ID or some other type of identifier.

If there may be typed resources, or child resources, available anywhere within the URL path, resources of that type can be deleted via variations on this generic pattern, as in this example:

DELETE /{resources_as_a_plural_noun}/{typed_or_child_resources_as_a_plural_noun}/{resource_identifier}

Request: Entity Body
The body of this request SHOULD be empty. If it is non-empty, the body will be ignored.

Response: Status Codes

Response: Content-Type
application/xml

Response: Entity Body (on Success)
Returns a representation of the just-deleted resource, as an XML document conforming to the schema used by the service.
If the deletion occurred in error, or an 'undo' operation is desired, the client can use this representation to once again create the resource.

Response: Entity Body (on Error)
Returns an XML document containing a description of the error. See Common System Specific Elements for details.

Search

Approaches to Search

As RESTful Service - Ajax Patterns notes:

... REST will always have different interpretations. The ambiguity is exacerbated by the fact that there aren't nearly enough HTTP methods to support common operations. The most common example is the lack of a search method, meaning that different people will design search in different ways.

The two most widely used approaches to REST-based search that we have observed to date involve a) submitting a request containing query parameters, via GET and b) constructing and submitting a structure, via POST. Some variations on these approaches, both observed and hypothesized, include:

1. Accepting GET requests with a single query parameter. (That query parameter may often be named 'q=', 'query=', or 'search='.)
2. Accepting GET requests with a multiple query parameters.
3. Using a GET request to retrieve an XHTML advanced search form, which the client can then submit to the service via a POST request. See Mark Baker's November 2006 comments on the atom-syntax list.
4. Creating a new resource within a collection of searches, populating that resource, and activating it, perhaps by POSTing it to an active searches collection, or by retrieving its "./results" child resource.

A variation on the last approach is suggested in Rich Apodaca's comment on the question "RESTful URL design for search" on the Stack Overflow Q&A site:

... you could make Search a first-class resource:

POST /searches # create a new search
GET /searches # list all searches (admin)
GET /searches/{id} # show the results of a previously-run search
DELETE /searches/{id} # delete a search (admin)

The Search resource would have fields for color, make model, garaged status, etc and could be specified in XML, JSON, or any other format. Like the Car and Garage resource, you could restrict access to Searches based on authentication. Users who frequently run the same Searches can store them in their profiles so that they don't need to be re-created. The URLs will be short enough that in many cases they can be easily traded via email. These stored Searches can be the basis of custom RSS feeds, and so on. There are many possibilities for using Searches when you think of them as resources. The idea is explained in more detail in this Railscast.

Also of interest:

• This proposal for a WebDAV SEARCH method.

Advanced Search

Notes: Look at Lucene search syntax, as implemented in RESTful services.

Tim Bray wrote in 2003:

... there is rough consensus among search engines on what a query should look like: quotes mark "phrase searches", you use "+" to say a word must +occur and "-" to -exclude words. You URI-restrict searches like so: tbray.org:kirkby. Down the road, you might want to support something like XPath or even XQuery, but that would work fine through a Web interface too.

So a search engine would publicize a URI that you could use HTTP GET on to run searches; arguments would be q for the query, mr for maximum results, and ie for input-encoding. So a complete search request might look like:

http://www.tbray.org/search?q=electric%20sheep&mr=20

q=%2busername:miso%20%2bhostname:miso.demo.sun.com%20%2bsubject:test

"Simple" Search

To conduct a "simple" search using default assumptions ...

Query parameters:
q= or query=

REST-based APIs for Relation (Association) Services

APIs for relation (or association) Services are currently expected to follow the API model for entity services, at least for the core operations of CRUD and Search. One notable difference to the APIs for relation services:

• The payloads for relation services will differ from those for entity services. Generally, they may be expected to:
  • Contain unique identifiers for two or more entities, which are used to associate those entities.
  • Contain other information units that are unique to relation services. These may include:
    • A Reason why the entities are being associated. In some instances, the reason may be an identifier for another entity. Example: a CollectionObject and Location may be associated, when the location of that object changes as a result of being assigned to an Exhibit or a Loan. In that case, the identifier for the relevant Exhibit or Loan would be the value of the Reason.
    • Temporal values, such as a timestamp indicating when the entities were associated and a second timestamp indicating when, if any, they are scheduled to no longer be associated. (Since that second timestamp may become out of sync with the value of the activity that might end the association, such as a timestamp for the end of an exhibit, that might become out of date if the end of that exhibit is subsequently changed, we might instead consider a different method for modeling the termination of associations.)

As one notable "behind the scenes" difference:

• There may be Workflow associated with associations.

REST-based APIs for Task Services

There are no standard REST-based APIs associated with task services. Most frequently, these are implemented in the REST RPC paradigm, where the name of a task service is included in the request URL, followed by query parameters that pass arguments to the service; e.g. /nameOfTaskService?param1=value&param2=value...

However, for consistency with the resource-oriented, REST-based APIs above, we might instead model task services as resources. For some potential patterns that we might use for doing so, see "Handling arbitrary actions" in RESTful Service - Ajax Patterns:

• Where you're trying to transform state, have the client compute the new state and PUT a specification of the desired state - a standard update operation. For updates that are complex or entail business logic, you can provide a read-only service - accessed with GET queries - to help the client compute the new state.
• You can POST a message to the resource in question, for example, POST a "pause" message to a URL representing the printer.
• The server can expose a stateless "processor" resource to perform the action, e.g. a printer controller, and have clients POST command-like messages there, with URLs referencing the resources to be acted upon.

Another resource-oriented pattern for implementing task services and handling the equivalent of statefulness has been used successfully in real-world applications by Rob Heittman, and may actually be a concrete example of the first bullet point, above:

The classic example of something very hard to implement without some server side state is an e-commerce shopping cart. There are a handful of examples on the web of a "purely RESTful shopping cart," but they tend to be a bit odd ...

But one way to RESTify this model, without changing it completely, is to deconstruct the idea of the "session" a bit. Instead of having one generalized server-side bag of state (the JSP/Servlet style HttpSession), which can only be accessed using server-side code, the client can use RESTful operations to create a needed resource on the server (e.g. a ShoppingCartResource); then these resources may be exposed back to the creator using RESTful patterns.

This hypothetical ShoppingCartResource would have a URI and ... representations that can be PUT, GET, etc. ... Now you have your shopping cart abstracted in a way that you can work with it on the server side using internal requests ... OR work with it on the client side using an Applet, Flex/Flash RIA, GWT, or AJAX application.

Conventions and Practices

Some conventions and practices followed in the proposed APIs above are discussed here.

Naming Conventions

• The names of path elements and query parameters are:
  • Written in the English language. Any cases where variant spellings or forms exist in different English dialects (e.g. USA and British English) will be resolved by consensus among the project team.
  • Internationalization capability for path elements and query parameters in the REST-based API to CollectionSpace services will be left open as a potential future enhancement, following release 1.0, and care will be taken to not preclude Internationalization.
  • Always written in all lowercase letters.
• The names of path elements are:
  • Always nouns.
  • Pluralized using the simplest possible pluralization rules (e.g. the plural of "person" is "persons"), where the plural form varies as little as possible in spelling and number of characters from the singular. With English language path elements, whenever possible, the plural form will append "s" or "es" to the singular form. More complex pluralization rules (e.g. in which the plural of "person" is "people," akin to Ruby on Rails' pluralization) are not used.
• When the name of a path element or query parameter would, in the English language, include a combination of multiple, space-separated word tokens, these word tokens are combined (run together) without any URL encoded characters, separators, or the use of mixed case. Examples:
  • A path element that serves as a top-level REST-based container or "bucket" for "collection objects" is written as /collectionobjects (and not, for example, as /collection%20objects, /collection_objects, /collection-objects, /collection.objects, or /collectionObjects. (The latter would also violate the "written in all lowercase letters" convention above.)
  • A query parameter named "page size" is similarly written as ?pagesize={value} or &pagesize={value}, depending on its position within a set of query parameter(s).

Where possible, naming conventions should be consistent - after reflecting software-specific requirements or idioms - across all of the artifacts that pertain to a path element, including source code files, schemas, and the like.

References

Normative Guides to Developing RESTful Services

Leonard Richardson and Sam Ruby, RESTful Web Services

Michael Mahemoff, et al., RESTful Service - Ajax Patterns

Brief Introductions to Designing RESTful Services

Joe Gregorio, How to Create a REST Protocol

Joe Gregorio, Show Me the Code

Constructing "Meaningful" URLs for Resources

Roy T. Fielding noted in his September 3, 2008 blog post, "Paper tigers and hidden dragons":

Web architects must understand that resources are just consistent mappings from an identifier to some set of views on server-side state. If one view doesn't suit your needs, then feel free to create a different resource that provides a better view (for any definition of "better"). These views need not have anything to do with how the information is stored on the server, or even what kind of state it ultimately reflects. It just needs to be understandable (and actionable) by the recipient.

Aaron Johnson, Clearspace and Meaningful URLs