Here’s a possibly half-baked post on software design. In the end, I’m not sure I have anything more to say than “don’t turn off your brain.”
Recently while profiling a large C++ application I develop, a thought struck me. ? The graphical profile lead me to a hypothesis: API mis-use significantly contributes to poor performance. Put another way, uses of APIs should be some of the first places you look for performance issues. And unlike some performance problems, fixing API related performance bottlenecks can be relatively high-value, especially if more than just your current application uses the API in question.
Developers should look carefully at their use of APIs as the interface between the consumer application and the API may be an unexamined performance bottleneck. Mindlessly designing around an API is asking for trouble. The first step to avoiding API bottlenecks would be to ensure you understand the entire API and understand what use cases it was intended to support.
Performance profiling is the first step to identifying problem areas. Don’t waste too much time on guesswork. A “Flame Graph” will illuminate the situation. See my recent post for more detail and a real-world example of a slow-down introduced at the API layer. In the flame graph in that post the functions in the API are easily found as they occupy the layer above the “EditingAPI” namespace.
The usual story goes that software re-use is good (of course) and APIs give the API creator multiple benefits. I wouldn’t dispute any of this. However, benefits of (public) APIs in particular are only one side of the coin; in isolation they benefit the provider, and users too, but the API externalizes some costs. When taken as a whole APIs have some negative effects on systems using them. Of course, the trade-off may be worth the cost as lots of applications could not exist without use of public web APIs, and any decent library has a well thought-out API. In this post I’m not considering problems arising from changes in APIs you don’t control or shutting down of a service you rely on, which are additional negative externalities of a sort that APIs – or reliance on APIs – can impose.
An API (Application Programming Interface) is the layer between a library or software service and third-party developers. It provides a layer of stability on top of a sometimes rapidly changing library and a set of well defined, documented set of entry points into the library or service.
While recently the term API has come to mean a “web API” – services delivered over HTTP – the way I’m using the term here applies equally well to traditional software library APIs.
An API’s design will suggest how to use a product and discourage what the providers may consider “pathological” uses of their software, whether those discouraged uses are for purely technical or business reasons. Sometimes an API discourages types of uses simply because it’s early on in development or the creators haven’t considered a particular use.
In any case you will find yourself as an API consumer sometimes going against the grain of an API either knowingly or (at first) unknowingly.
Bad App Behavior : Did the API Make me do it?
The (not great) metaphor of “impedance mis-match” gets over-used in software discussions, but I think it’s fairly apt in this next example.
Let’s say you had access to a movie database via a web API. You can get a single movie by title and year, getting back its cast, writer and director, along with short blurbs from reviews. You can also search by actor, getting back lists of films and years. Now, using this API, you want to make a “six degrees of Kevin Bacan” type app, where you can put in any actor’s name and get results that include directors and writers, seeing how many degrees of separation they have between them and one other person in the industry.
The API provider gives you a limited “movie” end-point where you can supply parameters for “year”, “title”, “language”, returning either a full movie result or an list of abbreviated list consisting only of movie titles.
There aren’t any writer or director endpoints, but movie results will include writers and directors whose names will often match some actors in the database. If a movie was re-made with the same title you get results for every movie of the same title, and since their years are in the results you can tell them apart.
You can build your app. First your app sends out “/movie/actor” passing in “Kevin Bacon”. It gets back a list of films, which you can parse out and insert into the /movie/title endpoint. From each of those results you can get back lists of actors, writers and directors off their movies when you retrieve each of Kevin’s movies. You can do the same with the other actors, doing the same until you find actors, writers or directors in common and return the degree of separation and the graph connecting the two.
You can see that the API provider might not be too happy with your app. Even one instance could hit their server two-hundred to two-thousand times for one “Six Degrees” query as the circle of movies expands exponentially. Imagine if your app has a few hundred users a day. Most likely if the movie API provider didn’t rate limit their service they will now. If they don’t and your app becomes popular you’ll bog down their servers. Now in reality there are only so many movies and they could probably cache them all– heck, you could cache their results on a server of your own acting as a middle-man – but even so the traffic will be kind of high. Every time they update their database you’ll need to re-query their service, and since they don’t give you a handy “all movies since date” (/movies/year/ for example) you’re forced to over-use their API.
What you’d really like is to download a complete list of movies and generate your own database designed to answer this query. Since they probably don’t want to give up their data like that, they might consider building their own “Six Degrees” service due to popular demand. Until they do, your app is running kind of slowly and behaving badly towards the movie data service.
In short, the movie API provided a means to make your app – just enough that you did it — but it’s a bad fit.
“In-House” API Mis-Use
This API provider/user mismatch is easy to see with web APIs. Most anyone who has tried to build something on top of a web API will have run into it. You’ll also run into the problem inside your own local code base. Any time an interface from one library or module provides a “good enough” access to something needed by another part you can get an API mismatch. Developers who weren’t the API designers find the API offers a way to do something, just not a good way.
Sometimes a local API provides the perfect interface, but the implementation is poorly suited to the way you use it. A service might rely on a poorly written function, which matters not at all if used once in a while but destroys performance if used once a second.
Another example, briefly:
Some people may recall the IBM/PC BIOS video services. You could count on having access to these services and that they’d work across a wide variety of hardware. Unfortunately that’s about all that could be said for them as the BIOS video functions ran very slowly and only provided very basic facilities. Their primary reason for existing was as a way to allow the computer to display text and basic graphics while starting up when no display software had been loaded yet.
The PC didn’t ship with any other way to display text and graphics that was easy to program, so when reading a reference manual the BIOS would appear to be the natural starting point for accessing hardware. Here’s a good reference with examples in case you’re curious. Fun times.
You could use the BIOS to create simple plots of data (but you’d see the painfully slow progress of the graphics across the screen.) Logically the BIOS provided enough primitives to make basic games but performance-wise the games would suck. But because writing code directly for display hardware was difficult, if for no other reason than the manuals were not easy to find, lots of novice programmers (which meant a lot of them) used the BIOS routines too often.
Types of API Problems
Often you will experience API mis-matches as one or all of these:
- Poor performance
- Wrong level of abstraction used by the caller
- Undesired side-effects.
As described already, you may be looping over a set of inputs, calling an expensive API function inside the loop, or simply calling the function much more often than intended. Perhaps the function does other work behind the scenes which mattered for it’s intended use but you don’t need and may be unaware of. In the most insideous cases the performance hit is just small enough to go unnoticed for a while – long enough for whoever made the change to forget all about it. That’s exactly what happened to me in the example I referenced in the intro.
Wrong Level of Abstraction
The API might require some more primitive input than you readily have on hand at the site you’re using it; maybe you have to decompose or retrieve information and that could be expensive or at the least messy, requiring a layer between your code and the API. Sometimes the problem goes the other way and you have to independantly construct objects you’d otherwise have no reason to have at that place in your code, forcing you to mock up something. In my example, a reason I resorted unthinkingly to the API was actually that I didn’t need to change much, just pass a different type of reference to an object, so the decision seemed natural and frictionless.
Sometimes a side-effect of a call may be the desired behavior; you might do a “status” call to a service to get back a particular piece of configuration information, even though the status call retrieves lots of other stuff and updates a log and so on. In that case the API isn’t granular enough. Other times you want a call for it’s stated purpose but in your case the side effects are undesirable for your circumstances.
In my example case the slowness was caused by what, in the particular place I was using the API, would be considered a side-effect. What I really needed out of the function amounted to a sub-set of what the function did. It checked for presence of an object with a “name” attribute matching a user supplied name and also checked that an object of that name belonged to a larger collection type. In my bad use case I only needed that last check, which is much faster to do than the first check. In the example the code change dramatically reduced time spent in that function and reduced total runtime of the application four-fold.
API Producer-Consumer Mis-Match as a Significant Source of Poor Performance
As more and more developers use an API – especially one they don’t control or influence – these mismatches will accumulate. The more “distance” between the API producer and consumer, the more the performance of the consumer software will suffer. By distance I mean ease of communication, producer’s understanding of their consumers, and alignment of incentives between the two parties. Really we are just talking about the long term costs of software re-use.
When incentives are aligned the provider will listen to their “customers” and adapt the API. When the API consumers aren’t the primary “customer” as in the movie database example APIs will resist change if it conflicts with their incentives. In that case the provider wanted to only allow retrieval of single movies, because they intended that service to enable third parties to place content onto other sites while crediting the movie database company. They didn’t care much about other uses, at best. They just wanted their brand out there, with links back to their site for advertising.
Software development requires a lot of thought and concentration. Any time we can save ourselves some thinking we do so, in order to use our time more efficiently. As developers we couldn’t get far without using libraries and accessing services through web APIs. But we should review our use and possible mis-use more often than many of us do.
If you create a flame graph or any other performance profile of your application pay particular attention to places where API or libraries are heavily used. You will probably find some bad actors in there. To be sure, we expect calls to core libraries to make up a lot of time spent in an application, but you may be surprised how a few of these calls seem to inexplicably use a large percentage of runtime. If it’s an API of your own design you may identify the best places to tighten up some heavily used parts; if you’re calling into a third party API you can at least be more thoughtful about how you use it.
Finally, situations where API use goes bad may arise easily when developers don’t choose APIs or projects; someone may hand them an assignment, point to an API and say “go.” An API may superficially appear to provide the needed services for the project but the devil really is in the details.