Creating an efficient REST API with HTTP

Many API’s produced call themselves RESTful, but barely cover any of the core principles of REST. REST is a standard you can choose to follow, and many (but not all) will help you to build an API that is efficient, easy to use and highly understandable. Here I highlight all the core principles, and how using them can help you to build an excellent API, which is a rare thing indeed.

image_print

Why bother

A REST API follows standards, and these standards take into account many of the gotchas you would encounter if you just made up your own rules, as many choose to.

It is likely you will also be creating the clients (for example, you may be making mobile apps that the API supports), and the better the API, the easier it is to create those clients.

Following these standards means:

Create a uniform interface by mapping HTTP methods to CRUD

CRUD = Create, Read, Update, Delete

HTTP methods = POST, GET, PUT, DELETE

So instead of API calls such as:

Simply:

See IBM – RESTful Web Services: The Basics for more justifications on this.

Following this means everyone can understand how to use your API in a common way and keeps the API simple.

Understand the difference between a URI and a Resource

Consider the following:

POST, GET, PUT, DELETE are the methods

http:/test.com/user, http:/test.com/user/1 are URI’s

user/1 is a user, which is a resource

Understand the difference between a resource and a representation

From the above example calling GET: http:/test.com/user/1 will return a representation of the resource user:1.

A representation is a set of attributes related to the resource, e.g.

A representation will have a format (XML, JSON , XHTML etc), and a language.

Keep the number of resources low

The resource names should often map conceptual domain entities. They should map to objects, not actions on the objects.

If you want to declare a user as a subscriber, you don’t need to add a resource called subscribe. Just pass the following representation:

This keeps documentation and coding simpler.

Create addressability by mapping a URI to each resource, possibly each representation

A URI should map to only one resource.

There should be URI mapping to each representation of the resource

This enables URI’s to be shared in a consistent way, which keeps things simple.

Name resources and representations carefully

Follow simple rules for naming resources. I recommend:

Use forward slashes to show child objects, with a resource for each item.

Use semi-colons where there is no obvious sequence:

Use comma’s when the order matters but they are not child items:

This keeps API’s consistent, making it easier to interact with them.

Use POST for creating, not editing

When you POST you POST to an unknown resource.

How should you handle the following situation:

You could update it with the new data, or you could respond with a already created code, and force the use of PUT.

I recommend keeping things simple, and refusing to update.

Consider the use of overloaded POST only to handle situations where PUT and DELETE are unavailable

If you want to allow your API to support clients without PUT and DELETE (in reality a rare situation these days), use overloaded POST. A true REST API wouldn’t do this, but sometimes you need to be flexible.

One way of doing this is to define in the API the actual call you want to use to replace POST:

The API can then convert the POST call into the appropriate method, and funnel it into the same route.

GET and HEAD should only ever fetch data, never alter or delete it

If you manipulate or delete data using GET, you put your data at risk.

It is possible a web crawler or browser call could delete your data.

GET should only ever fetch data.

Allow for multiple identical PUT or DELETE requests, without side effects

If you are updating the price of a widget in a PUT request, don’t allow anything like:

Instead, only accept a new price:

This is because an app could submit a PUT multiple times, without realising the negative effects. In the first example this would result in the price going up by 1 (or down by 2) on each submit.

In the second, the price would always remain at 3.99, there would be no ill effects.

Use HTTP Response Codes

HTTP provides response codes to inform clients of the status of their request. Use them! Don’t just return a 200 response with an error description if something is wrong.

These are the main ones used:

2xx – Success codes

3xx – Redirection codes

4xx – Client side error

5xx – Server side error

More codes can be found in the HTTP/1.1 Status Code Definitions guide

Handle and use meta-data in HTTP Headers

Implementing these will make your API more flexible and keep information in standard locations rather than URI’s.

These are common request headers (from client to server)

These are common response headers (from server to client)

These are common request and response headers (bi-directional)

REST is stateless – so don’t handle state in your API

REST API’s must be stateless. Every call requiring authentication should provide the authentication and never a session key.

When you attempt to handle state in your API you create a nightmare of sessions and session handling on the API side, and create lots of repetitive work for each service connecting to your API who has to potentially always handle the case where the session has expired. Handling state will slow down your API and all clients using it.

Stateless API’s are easier to scale, more reliable and simpler to use.

REST is cacheable 1 – allow clients to GET only updated resources via HTTP

If your clients are repeatedly requesting the same data you are unnecessarily slowing all applications down, increasing bandwidth from the server to the clients and wasting computation power. Instead, use HTTP to handle this.

The easy way with Last-Modified

Get more precise with ETags

There is a risk here where more than one person can update data, that the data updated again within the same second. This is why you can also use ETags.

However, ETags are more costly to generate than Last-Modified, and should be considered carefully.

REST is cacheable 2 – prevent accidental simultaneous updates of the same resource via HTTP

A common issue when more than one client can access the same resource is having one client do an update, and then another client overwrite that update because they didn’t know it had occurred.

The basic way with Last-Modified

Get more precise with ETags

Read more about preventing race conditions.

Use the HEAD method to allow the client to manage its bandwidth better

The HEAD method for an resource should return a meta-data only representation of the resource.

HEAD: /user should return everything GET: /user would except the entity-body.

This could be useful for a number of reasons, e.g. the Content-Length header might tell the client it doesn’t want to fetch this particular entity-body if its too large.

Version your API, and never change released features

Each URI should begin with a version number.

Returns:

If you remove anything from the representation being returned for user, or change the rules about what you accept, you need to increase the version number – don’t just change the current representation.

Returns

This prevents your clients from breaking!

 

Understand Hypermedia application state and why you probably shouldn’t use it

Hypermedia Application State (also known as HATEOAS) is a concept wherein the client uses just one URI, and from that URI is able to deduce how to interact with the API by following links, as one would when using a website. What this means in theory is that the client need not know how to use the API in a certain way, which enables the API to evolve without updating all the clients. This talk explains it well.

This is one part of REST I don’t agree with for most situations, although it is technically part of having a true REST API.

The concept is great in theory, but it poses the following issues:

Ultimately what this does is put the needs of the developers over that of the users. It places theory over practicality. No other aspect of REST does this, and that is why this is the one thing I don’t recommend, and neither does the creator of Ruby on Rails, nor Michael Bleigh. Others do, and its a big debate area. Understand it and make your own choices.

Next steps

Learn more about API’s

The best book you can read on this subject is the REST bible: RESTful Web Services, by Leonard Richards, a Kindle edition is also available.

Get outside help with your API’s

If you’ve enjoyed this article and have an API project you’d like help with, I may be available as a consultant on your project. I’ve built REST API’s for mobile apps, and worked with Tesco on the early versions of their (non REST) API as a mobile developer. I learned much from this experience, and have both made, and seen, many of the issues that can occur from not following the above guidelines. My business partner and our CTO James McCarthy has over 20 years experience building back-end systems and API’s for organisations such as AMEX and may also be available.

To hire one of us, check out the contact page.

14 Comments

Andy Kaminski

Wow!

This is a great article Mark – highly informative and well considered. You’ve nailed pretty much everything that’s relevant in relatively few words. I learnt a few new things along the way too.

Good stuff – keep ‘em coming.

kirby.mark

Thanks Andy! I’ve got a couple more coming over the next couple of days on technologies to create and host API’s…

Chris Ismael

Awesome. I’ll add this to our blog at http://blog.mashape.com/ For those who have created RESTful APIs, you can add it to http://mashape.com so developers can discover them

w5m

A very useful article Mark – thank you.

You mentioned that “There should be URI mapping to each representation of the resource” e.g. “test.com/user.xml maps to the xml representation of the user”. What’s your view on omitting the file extension and using the Accept request-header field to indicate the desired format instead?

Tito

Very useful article Mark.

One comment. The phrase “HEAD: /user should return everything GET: /user would except the entity-body.” should read the opposite. GET returns everything, HEAD just the headers.

Deepak Mittal

Loved this article. Short and crisp. Useful for anyone who is starting using REST

amowu

Thanks! great article.

Lukasz Gintowt

“Each URI should begin with a version number.”
This is considered very bad practice. Unfortunatly its quite common bad practice.
It forces you to handle deprecated routes like /v1/… forever.

ex: After deprecating API v1, each Client needs to update their connectors,
Imagine having 50 Clients, on API that handles 50 Endpoints => 2500! changes required.

Better to use Accept request-header (AKA Content Negotiation), like @w5m:disqus mentioned.
ex2: One have 50 Clients, API handles 50 Endpoints => 50 Changes required;

Kilroy

Shouldn’t the following sentence

“HEAD: /user should return everything GET: /user would except the entity-body.”

be

“GET: /user should return everything HEAD: /user would except the entity-body.”

instead?

Paul

Great article, concise whilst comprehensive. It pains me to see so many people doing API’s that just don’t get REST. I’ll certainly point people to this post in future. Which makes me feel like such an arse to highlight the section on “overloaded POST”. In this scenario I’d definitely recommend the use of the X-HTTP-Method-Override header as detailed quite nicely by Hanselman:
http://www.hanselman.com/blog/HTTPPUTOrDELETENotAllowedUseXHTTPMethodOverrideForYourRESTServiceWithASPNETWebAPI.aspx

Manoj

Very Good Article.

Robert

Great article, very good summary. I just have a minor factual correction:

You don’t really need 2 calls for every function in case your are taking HATEOAS seriously. You can easily bookmark some url you discovered. Our REST-ish client for example has a map of bookmarks of already discovered functions and important resources. You can build these bookmarks up always on startup, or even make them persistent if you want (changing only in response to 301 or so).

Also, I think it is especially important for Mobile, since you don’t need to change the clients (which you can’t really) every time you change something structural on the server.

On that note, if you follow HATEOAS it is really not important how you name your resources, since it will be discovered anyhow. You are completely correct we are trading “developer convenience” for “user convenience”. As in: it will be more difficult to implement a client, for the gain of not having to modify the client when the server changes.

Anurag Jain

Great article. How is data expected to be sent let us say for PUT in a REST api. So the API will expect:
Method: PUT
URI as http://www.example.com/user/1
where user/1 is the resource
How to send the user data in REST. Say update the name and age of the user/1
Is the data to be sent in the URL or some other way.

Leave a comment: