How not to do APIs

The company involved here will remain anonymous, as they offer a good (and relatively unique) service and I don't want to damage any relationship with them, nor discourage others from using them just for this reason! Additionally, their support has been excellent.

I hope others find this though, and take it into account when considering their own API design and strategy decisions.

As you may know I have some experience developing complex APIs for large service providers. The approach to this has changed over the years, but some of the basic principles remain the same.

The Company

The company will remain nameless, but they are a specialist SMS provider that caters for a niché market. They caters for some use cases that most other providers do not.

The API is not currently the centrepiece of their offering, and they have a flexible web-based interface that suits most of their customers.

The first rule of our API, don't talk about our API?

My initial eMail asked if they could deliver inbound messages by HTTP POST and if they had an API.

The first person I spoke to was aware of the existence of "An API" but knew nothing of it, deferring to their technical team (which is fair enough) but yet seemed confident they could not do HTTP POST - but that I could download messages from their web site.

Why not do a brief staff training thing on what an API is, how it can be used, and why bigger customers may like it?

The API is "hidden"

The website mentions an API and, after logging into the portal, I find there is an option to enable API access for individual users – great.

However, nowhere could I find any documentation for this mysterious API.

Two eMails and a day later, I have the API documentation.

The API is arguably the most flexible part of the whole offering, details should be easy to find in the documentation or linked to from the portal

API Design

The API is functional, but feels like it's designed as a 'hack' on top of an existing system.

In contrast my own experience has been to build an API, and then build the UI atop this.

URL and Parameter structure

There is none, the whole API is accessed via a single URL e.g.; www.example.com/api/api_server.php

The documentation says HTTP POST requests should be made to this and the basic required parameters for every request are;

• Username
• Password
• Function Number

Then gives an example of
www.example.com/api/api_server.php?user=USERNAME&pass=PASSWORD&func=2

Firstly, the example looks like a GET request (the URI arguments after the ?) - but it must be accessed using POST.

Also, the error returned when using GET is not "Invalid Method" but rather "Username not recognised", despite it being provided as shown in their documentation.

Secondly, the parameter names aren't shown anywhere other than in the example (so no explanation that "Function Number" becomes "func", for example)

Function Numbers

On the subject of the URLs... the API uses almost 20 numeric function numbers passed as a POST parameter - e.g.

• 1 - Get List
• 2 - Add to List
• 4 - Send Message

To me, the obvious way to design these would be to use the HTTP method to define the action alongside URLs such as;

• GET /lists/<list id>
• POST /lists/<list id>
• POST /message/send

Or, if not using the HTTP method to determine the action, you could instead use endpoints like;

• /lists/get
• /lists/add
• /message/send

At the very least if you're going to use a func parameter in the request, use a readable function name e.g. get_list, add_list, send_msg

API designers should use meaningful names and URLs wherever possible.

Responses

Depending on the function responses are either in the form of;

1\r\n\<Message> or  JSON content.

Errors are always in the format

0\r\n\<Message>

So; some successful responses are JSON, some are human readable plain text across multiple lines (the first line is always 1). All errors are human readable (but the first line is a 0)

Indeed, the APIs I built around 2002 used this sort of CRLF-separated format, but JSON didn't exist then!

Getting messages is always a JSON list, but sending one results in the following;

1\r\nSMS sent successfully to 07700900123 at 12:34 27/02/16\r\n942842NNE

Why not {"success": true, "destination": "07700900123", "time": "2016-02-27 12:34", "ref": "942842NNE"} ?

I know which is easier to parse! Worse still, as you can't rely on the response being JSON (errors will always be plain text) you can't just throw it into a JSON parser as it'll just throw an exception.

Please, standardise your APIs, use JSON for everything (or whatever format you have chosen to use but - be consistent)

HTTP Post ("Webhooks")

Firstly, this has to be manually configured by raising a ticket - the portal makes no mention of this facility - which is the best thing about SMS services if you're integrating them with anything else!

With most SMS providers, if you have an inbound number you can expect to receive the SMS message as an HTTP POST to your own platform - this takes around 2-3 seconds, maybe 10 in some outlying cases.

This service delivers them in around 2-3 minutes in some cases. Fine for some applications, but not too useful for real-time voting, live interaction, contests etc.

This is "due to a queuing mechanism designed to ensure server stability during times of very heavy traffic" – I don't know what sort of traffic volumes they have, but - speaking as a service provider - the heavier they are the quicker i'd want those requests to be made so that they're out of my queue!

However, if you configure an auto-response it's received almost instantly. And, likewise, if you query inbound messages from the API it's also present almost immediately.

Make your APIs as close to real-time as possible, technologies such as REDIS and queuing platforms like RabbitMQ, Beanstalkd etc make this relatively easy and scalable

Variable Names (again)

Also, the variable names in the HTTP request differ entirely from those in other API calls.

Again, consistency is needed here, if a customer can retrieve a message by an API request - the data should be the same as in the HTTP POST if they're using this method.

My workaround

Since the HTTP POST is slow, I put together a very basic script which can be summed up in the following (pseudo code with a hint of Node)

function getMessages() {
   arrMessages = getNewMessages();
   for each (arrMessages as message) {
      postMessage(message);
   }
}

setTimeout(getMessages, 700);

Basically, poll their API every 0.7s, get any new messages and POST it to the endpoint on my own server designed to receive their HTTP POST.

Brilliant, it works as expected! :)

Then I got this eMail three days later (which ultimately is what prompted this blog post!)

Due to server load requirements we are having to reduce the limit on API requests that can be made to a maximum of 20 per minute.

Just giving you a personal heads up in-case this affects your logic. Requests made above this timing threshold will start to receive a ‘Throttle Limit’ error.

So my workaround (which only exists because HTTP POST is slow or unreliable) appears to have broken their API (else it was a remarkable co-incidence!)

I give up.

Some nice touches

Despite my ranting above, the API has some nice touches that others could learn from.

'New' Messages

Their server keeps track of messages you have already downloaded as the result of an API request, therefore there is a simple API call to get any new messages that haven't yet been retrieved.

Most APIs allow searching by date, but then you need to keep track of the time of the last received message and it's usually advisable to have a bit of overlap between queries, this is much simpler!

They also provide a function to easily reset this for a particular campaign, so you can get all the messages again.

Authentication

You can grant individual users access to the API, and they can use the same credentials as the portal. This has advantages and disadvantages (Personally, I like API keys normally!)

However, for this application it's ideal as larger organisations who have their own tools built around the API can have their staff login with the same credentials as the portal, and the service provider's own audit trail of actions is still useful (as opposed to showing every action by "API User")

Error Handling

Whilst it'd be nice if the messages were in JSON format, they are all human-readable and quite clear (no random error numbers to look up!)

Documentation

The actual design (function numbers, inconsistent parameter names, inconsistent response formats) leaves a lot to be desired but - credit where it's due - it's actually pretty well documented.

• Each function is documented on its own page.
• Parameters (other than the user/pass/func ones detailed at the start) are generally shown clearly together with the type of variable (e.g. String, Integer, Boolean etc) and a description.
• Any parameters with discrete values have a table with all possible values shown, alongside an explanation.
• Optional parameters are clearly noted
• The results are colour-coded (green is used for JSON values, orange for plain text) however this convention isn't actually explained.
• All possible errors that can be thrown are listed

---

Ultimately, this company offers a great service - and is fairly unique in their space - modernise their API a bit and they'd really have an unbeatable offering.

If you're reading this and you know who the company is, please don't share it!, this was intended to provide others with some guidance when thinking about their API - not to complain about them!

If you're reading this and you work for the company, expect an invoice in the post ;-) ... no, but seriously, make v2 of your API awesome :)