Creating an API

A few weeks back, we publicly released the Gauges API. Despite building Gauges from the ground up as an API, it was a lot of work. You really have to cross your t’s and dot your i’s when releasing an API.

1. Document as You Build

We made the mistake of documenting after most of the build was done. The problem is documenting sucks. Leaving that pain until the end, when you are excited to release it, makes doing the work twice as hard. Thankfully, we have a closer on our team who powered through it.

2. Be Consistent

As we documented the API, we noticed a lot of inconsistencies. For example, in some places we return a hash and in others we returned an array. Upon realizing these issues, we started making some rules.

To solve the array/hash issue, we elected that every response should return a hash. This is the most flexible solution going forward. It allows us to inject new keys without having to convert the response or release a whole new version of the API.

Changing from an array to a hash meant that we needed to namespace the array with a key. We then noticed that some places were name-spaced and others weren’t. Again, we decided on a rule. In this case, all top level objects should be name-spaced, but objects referenced from a top level object or a collection of several objects did not require name-spacing.

{users:[{user:{...}}, {user:{...}}]} // nope
{users:[{...}, {...}]} // yep
{username: 'jnunemaker'} // nope
{user: {username:'jnunemaker'}} // yep

You get the idea. Consistency is important. It is not so much how you do it as that you always do it the same.

3. Provide the URLs

Most of my initial open source work was wrapping APIs. The one thing that always annoyed me was having to generate urls. Each resource should know the URLs that matter. For example, a user resource in Gauges has a few URLs that can be called to get various data:

{
  "user": {
    "name": "John Doe",
    "urls": {
      "self": "https://secure.gaug.es/me",
      "gauges": "https://secure.gaug.es/gauges",
      "clients": "https://secure.gaug.es/clients"
    },
    "id": "4e206261e5947c1d38000001",
    "last_name": "Doe",
    "email": "john@doe.com",
    "first_name": "John"
  }
}

The previous JSON is the response of the resource /me. /me returns data about the authenticated user and the URLs to update itself (self), get all gauges (/gauges), and get all API clients (/clients). Let’s say next you request /gauges. Each gauge returned has the URLs to get more data about the gauge.

{
  "gauges": [
    {
      // various attributes
      "urls": {
        "self":"https://secure.gaug.es/gauges/4ea97a8be5947ccda1000001",
        "referrers":"https://secure.gaug.es/gauges/4ea97a8be5947ccda1000001/referrers",
        "technology":"https://secure.gaug.es/gauges/4ea97a8be5947ccda1000001/technology",
        // ... etc
      },
    }
  ]
}

We thought this would prove helpful. We’ll see in the long run if it turns out to work well.

4. Present the Data

Finally, never ever use to_json and friends from a controller or sinatra get/post/put block. At least as a bare minimum rule, the second you start calling to_json with :methods, :except, :only, or any of the other options, you probably want to move it to a separate class.

For Gauges, we call these classes presenters. For example, here is a simplified version of the UserPresenter.

class UserPresenter
  def initialize(user)
    @user = user
  end

  def as_json(*)
    {
      'id'          => @user.id,
      'email'       => @user.email,
      'name'        => @user.name,
      'first_name'  => @user.first_name,
      'last_name'   => @user.last_name,
      'urls'        => {
        'self'    => "#{Gauges.api_url}/me",
        'gauges'  => "#{Gauges.api_url}/gauges",
        'clients' => "#{Gauges.api_url}/clients",
      }
    }
  end
end

Nothing fancy. Just a simple ruby class that sits in app/presenters. Here is an example of the the /me route looks like in our Sinatra app.

get('/me') do
  content_type(:json)
  sign_in_required
  {:user => UserPresenter.new(current_user)}.to_json
end

This simple presentation layer makes it really easy to test the responses in detail using unit tests and then just have a single integration test that makes sure overall things look good. I’ve found this tiny layer a breath of fresh air.

I am sure that nothing above was shocking or awe-inspiring, but I hope that it saves you some time on your next public API.

24 Comments

Fredrik Björk
Dec 01, 2011

Great article.

I’ve seen great value in presenting the JSON with tilt (https://github.com/rtomayko/tilt) and Yajl JSON templates. Makes it more readable and sticks to the MVC pattern.
John Topley
Dec 01, 2011

Did you think about versioning your API in case you have to introduce breaking changes in the future?
John Nunemaker
Dec 01, 2011

@John: Yep, versioning will be done through a header. I don’t really believe in versioning through the URL.
Fredrik Björk
Dec 01, 2011

@John Nunemaker: I tend to agree that versioning should be done in a header, but sometimes clients can’t pass headers, i.e JSONP
John Nunemaker
Dec 01, 2011

@Fredrik Björk: That is a good point. Though I’m sure we could also allow for a query param in that instance if necessary.
Duff OMelia
Dec 01, 2011

The other thing I like about the Presenter approach for public api’s is that it’s makes API versioning much easier to implement.
John Nunemaker
Dec 01, 2011

@Duff OMelia: Yep, inside your app you can certainly have directories of version numbers and presenters. You can also inherit from v1 for v2 or whatever to share responses that are the same. Moving to presenter classes opens up a lot of possibilities.
bai
Dec 01, 2011

Any reason for not using Link headers (http://tools.ietf.org/html/rfc5988) for URLs?
John Nunemaker
Dec 01, 2011

@bai: interesting idea. Had forgot about those.
Nate Klaiber
Dec 01, 2011

Many of the API’s I have worked with allow you to specify certain requirements in either the URL query string or the HTTP headers. This makes it easy to construct with a URL or to put the HTTP request together behind the scenes.

The presenter approach is what we used a few years back when building an API. They made the most sense, and I still think they make the most sense.
Ben Smith
Dec 01, 2011
Not sure if you’ve already seen these articles, but Tribesports wrote a couple of informative posts on versioning their API using a custom MIME type in Rails and the draper gem to implement the decorator pattern.
- Versioning the Tribesports API
- Separating API Logic With Decorators
John Nunemaker
Dec 01, 2011

@Ben: Yep, I think I can see a custom MIME making sense. Thanks for linking.
ivan
Dec 01, 2011

very interesting, do you have any other good resource I can read about the best ways on creating an API? in few months I’ll have to add them to an existing platform (my bad, next time I’ll start with APIs from the beginning)
Noah Hendrix
Dec 01, 2011

@ivan you might look into https://github.com/intridea/grape

It’s a DSL for APIs. You can see Michael Bleigh discuss it in a RubyConf talk here http://confreaks.net/videos/475-rubyconf2010-the-grapes-of-rapid
Keith DeLong
Dec 01, 2011

Consistency IS hard – but I’d say it’s more than important – it’s essential. Sometimes new rules present themselves along the way. Refactoring those rules into completed API work takes courage and perseverance. But the time up front really pays dividends for both users and future development.
Ivan K.
Dec 02, 2011

translated to russian http://habrahabr.ru/blogs/webdev/133821/
ivan
Dec 02, 2011

Thanks Noah, why use an extra library like Grape when Rails 3 provides already a way to respond based on the requested format? With Grape all the controller logic (get all the records based on the current user, and things like that) would be duplicated in the Grape dsl too
zimbatm
Dec 02, 2011

@bai: if you start adding header informations like that it means that http header and body can’t be disassociated. Usually I would try to keep header information strictly for the clientserver communication.
Cameron
Dec 02, 2011

How does as_json(*) work. I have been playing around with the json gem in irb, but maybe I am missing something here. Does calling to_json on the has invite as_json on the presenter?
John Nunemaker
Dec 02, 2011

@Cameron: Pretty sure that is part of Active Support or something. to_json calls as_json.
Bernd Ahlers
Dec 02, 2011

John, thanks for the article!

I have a somewhat unrelated question. How do you guys handle the sharing of stuff like database models for separate API apps?

Thanks!
John Nunemaker
Dec 02, 2011

@Bernd: Your API should provide all the functionality other apps need. If it doesn’t, then add it to the API.
Tyler Stromberg
Jan 18, 2013
I know this post is a bit old, but I was wondering how you handle presenters where you need to return a collection of objects. For example, does the ‘/gauges’ route look something like this?
```
get('/gauges') do
  content_type(:json)
  sign_in_required
  {:gauges =&gt; current_user.gauges.collect {|gauge| GaugePresenter.new(gauge)}.to_json
end
```
Or is there a cleaner way to handle this (e.g. the presenter itself knows how to render a collection)?
John Nunemaker
Jan 21, 2013

@Tyler: Yep, either that or make a presenter for the collection (ie: GaugesPresenter, note the pluralization).