Building A Great API Product
A primer on building technical products.
Introduction
I always wanted to write about the considerations we must make from the product perspective to build and manage products that cater to developers. There are multiple types of developer products, from tools that serve the build chain to IDEs and text editor to infrastructure tools products where code is running. Specifically, we will dive into one kind of developer product that has gained in popularity, the RESTful API. It’s half delivery method, half valuable source of functionality for many products we use every day. We’ll go from first principles to API requirements while highlighting the great developer experiences out there. Colleagues who might not have a technical background will feel at home- I try to make no assumptions on prior knowledge. But people who punch buttons in front of glowing boxes might appreciate the history.
Some Context
Before the advent of the display terminal, punch card development was one of the only ways to program computers. The developer experience ranged from marking up their cards and then feeding them in and hoping that their instructions were in the right order. Developers then would take great importance to make sure their programs were correct before execution. One can imagine that this was a tedious task.
Once display terminals and disk-based computers started to be the norm around the 80s. Software products began offered to extend the ability of those computers. The people programming those computers were usually at the mercy of the languages they could use to program them.
A suite of products started to serve developers of all types. Programming languages and runtimes could help developers save time, and database products could help those computers store computer data without having developers spend time doing the work needed to do that themselves. Users who needed developer tools would order them through catalogs or join USENET groups to speak to users and enthusiasts of those programming environments. A company called Borland acquired a company that made a Pascal compiler. It then began offering TurboPascal as an integrated development environment in 1983. These tools would make discovering language features faster as developers would utilize the libraries and features within those tools. Organizations would often build libraries and use them internally for typical tasks or pay consultants to help them with their software.
Enter the internet
With the standardization of HTTP over TCP/IP. (A way for computers to standardize the way they speak to each other) The internet would massively lower the barriers to entry for developers seeking code and support. The developers were able to form public mailing lists and forums and share their ideas about programs. Even better, the cost to acquire information of data reduced astronomically.
During this period, a format for data encoding called SGML (Standard Generalized Markup Language) was the norm. With SGML, computers could send and read derivatives of SGML like HTML from servers. However, the specification was heavy to implement and include in programs for data. (You might not appreciate it you got an 80-page packet if you only cared about the last page)
Technologists and entrepreneurs looked to take advantage of this new model in distributing software and services. There was a parallel battle for hearts and minds. Internet Explorer and Netscape were duking it out to ship a better web browser. Netscape Navigator 2.0 would ship with a new programming language and run-time called JavaScript and have that language live longer than the company its self. The important part is that JavaScript would come with a fascinating Object model. It was the way how JS would represent data called JSON (JavaScript Object Notation). The alternative seen in many other languages was XML, an HTML like format that would get pretty messy if there were too many nested data types. You also had the spreadsheet’s favorite, comma-separated values. Other languages would adopt this specification in their languages because of the combination of human readability and programmability.
Now would a developer have to do to try to get data from another source? Whenever you go to a restaurant and look at the menu, you don’t order and cook the food yourself. You go to a restaurant and pay for a service. Now, say that developer in 99' was looking to add the current weather to his webpage. There was a variety of ways he could have done it. They could have gone to the National Weather Service webpage with his zip code in the link and “scrape” the webpage. Meaning, downloading the webpage and then extract the data from it using software they wrote. But this is a very convoluted way to go about getting something as simple as the weather. Some kind folks offered FTP servers where you could download that data, but everyone had their idea of how they structured the data. A developer would need to worry about how each data provider structured their data.
They could use the SOAP protocol, but it forced developers to worry too much about how they received the data. He couldn’t just request the data but would have to include a very heavy client in their code.
Enter, REST. Remember the restaurant? In the restaurant, you would do a variety of actions to get your food. You would get a menu and then tell your server to post the order to the kitchen. Well, those actions in the specification are called verbs. And we GET and POST the same way people get and give data about their intentions all the time. The way how we interact with the kitchen through the waitstaff is one example of an application programming interface or API.
So the NWS can offer that data through an interface where the programs can talk to each other and not have to concern themselves about meatballs or do anything manually.
A Web Revolution
The internet matured out of its AOL era adolescence. The web developers of yesterday were empowered to build services for other developers in more efficient ways than before. Before, developers who built for other developers were mostly limited to physical media transfer and would have to resort to a very un-optimized supply chain where they would publish Software Developer Kits to incorporate into the software.
Now web technology was changing how developers consumed software. Instead of only being able to get data, they could have another computer somewhere else do the work and send the result back.
As a developer, I shouldn’t have to spend too much time having to reinvent the wheel to include functionality that might not be core to my product. What if I wanted to have my phone verify my login? What if I wanted to automate some everyday tasks within my company?
We are going to break down two companies that have offered such products to developers and highlight what was required to ensure their success.
Meet Our Cases
Despite programmatic telephony being possible ever since the modern modem, the transition to DSL and Coaxial internet networks reduced the importance of phone lines on the internet. With VoIP, being in fashion, a small prototype by Jeff Lawson became Twilio in 2008 as an application that allowed developers to place calls via an API. If given a phone number, developers would be able to POST to their endpoint (a URL looking like a string of text called a URI) It would later add the functionality of being able to send text messages via an API in 2011. Developers and companies looked to use SMS as a way to communicate and authenticate end-users, Twilio would take advantage of this growing need.
Zapier is an automation platform that can allow you to build workflows. Before Siri Shortcuts, you could set up integrations across different applications to have them communicate with each other without having anyone with a technical background be able to move data/actions around. If you want to fire off an email when a google form is complete, you would be able to wire up the applications through their service without having to build the tooling yourself.
Now, both companies (as of today) offer end-user facing products. As mentioned, Zapier’s primary product is a visual editor that allows users to drag and drop integrations across different products. However, if companies want to build on this ecosystem, they need a way to integrate with their functionality. Whereas, Twilio is seen as a remote interface where it might add functionality where product teams don’t need to implement the functionality themselves. What can we learn from these companies who serve developers for anyone who wants to offer APIs?
One: Know the value prop
If APIs are offering functionality, the API must be more straightforward than having the developer implementing the feature. If APIs are providing data, there must be a precise structure where developers don’t get too much extraneous data. All the while, they are considering how quick and how small the payloads from the API are. Twilio was able to position itself in a crucial balance where its core to digital products/businesses (2FA, Growth/Onboarding). Despite how core it is, Twilio offering to handle the arduous process of setting up phone numbers to interact with programmatically with Modems is something that most companies don’t want to do themselves.
For companies who offer services, they can extend an invitation to partners or their customers to implement their roadmap without having to have the company do it for them. Better yet, if other companies offer functionality or integrations in other products, that could be a way to make it so that the product is stickier in their user’s workflow.
Companies need to understand that APIs are a public window on how an engineering team operates. For example, Amazon’s API memo means that most features on AWS are ready to be exposed programmatically, even if there isn’t much external documentation. However, if there isn’t enough support from an API in terms of tutorials and documentation, developers wouldn’t want to take advantage of the API. Your users are always calculating whether if it is worth it for them to do the work themselves or to pay another service to do it for them. In the case of Twilio, many of the users of Twilio are trying to extend their application with SMS capabilities but might not be interested in handling all the telephony code.
In the case of Zapier, the APIs they offer is the ability for a company to provide integrations with into their product. If you are a developer of a note-taking app, you might want to include it to increase the value of your application if you can advertise the fact that you can have it talk to Google Drive and have people add programmatic logic. However, the developers building on Zapier has to consider what additional requirements and limitations that they will be adding to the product. For example, if you are building a product that is supposed to be a massive data moat, you might not want to make it easier for the application to talk to other products expect ones within your product portfolio.
Two: Offer simple design and implementation.
If you are in a product function at one of these companies, you should find out how long it takes to get a lead into a consumer as fast as possible. Well designed APIs have precise verbs and context. Service architects should understand that if you plan to make such an API public, then the metaphors that they use to know how the system works must also be apparent to a developer who has no context. Poor architecture can lead to an API that does too much or expects objects that are too large that make the developer do too much work. Understand the goal of the user and offer clearly defined routes that map to the user’s intent.
When you have a simple API, it’s more accessible to leads, and you have to spend less time onboarding customers. Twilio has a great example where you can implement a concept in about 5 minutes to show how easy it is to understand. (More on how this can be an advantage later)
For unavoidable situations where developers might need some time wrap around their head around how your system works, making sure that your developer ecosystem is gated correctly is critical. A great example is the Zapier developer platform. They require users to create an account to view documentation, creating an integration is mostly a process that will take some time to learn about the expectations that developers will need to onboard themselves on their platforms. Also, for someone who usually handles partnerships at Zapier, they would want to know who is interested in coming along.
Three: Know what you have to support
Developer APIs might make assumptions on consumption. Some are too limited, where critical features or a need to support a programming language is a blocker.
Twilio has done a great job offering well-maintained SDKs and packages for a variety of programming languages. If not, a developer can call the API themselves.
Whenever the surface area of an API product expands, it is all too common to run the risk of outdated documentation or, worse yet, deprecated routes. Although it is common for users to surprise us, we should never be shocked to find out its potential for abuse. Product requirements should always have clear limits on API use.
The Zapier developer integration exposes an exciting and simultaneous worrying side effect for PMs. By increasing the surface level of interactivity between other applications, you are opening your product in ways that you might not expect. In some cases, by making it easier to export data for workflows, you can generate lifeboats for customers to migrate off your product. Although we don’t talk about the risks/benefits of data portability here, developers and PMs alike should consider this.
Now Zapier offers a smart way to handle any potential discrepancy, in the platform, you can approve or disallow specific applications from talking to each other. For Twilio, the use case is pretty cut and dry.
As they support more use cases, they make sure that they have guides for the most common actions listed on their sites. And their documentation places the least burden of thought on the end-user by having examples be in the most popular languages that developers like to use if a language example isn’t there, clear documentation of what data goes in and out.
Four: Market your value prop
Twilio achieved notoriety for its presence in Hackathons and the conference circuit. By having its product be straightforward to demo, Twilio can send Developer Evangelists all-around the world to present it’s products to developers. Developer Evangelists are software engineers who also market the product and provides support to champion adoption of the product. They can be at trade shows like TC Disrupt to show it off to users and support their future customer base.
The ease of demo also leads to a wealth of content where code samples and other lead-generating content to acquire more users. In their website, there are plenty of code samples, and evangelists are at the ready across most social media platforms, even ones specific to developers.
The developer experience is the best way to market your product as the rate of adoption within developer circles follow hype cycles. Cycles are substantiated on great developer experiences. (Docker) Some cycles are just explorations of ideas. (Microservices) But the overarching goal is to become the most recommended in a space. You want developers to associate your API with the verb of what it does. Stripe’s goal is to hold as much mind-share when they think about billing. Twilio’s goal is that way with SMS and Voice. Around the 2010s, as developer pay increased, a shift in purchase behavior occurred. Developers and teams had more control over personal adoption rather than having the purchasing decision done above them. (Oracle DB comes to mind)
Five: Tracking Usage and Billing
Developers need visibility into how they are calling your APIs and need usage information. Your user should never have to build tooling or export their logs to understand and control the cost. Any confusion around API consumption is an open opportunity for a competitor.
Development teams need visibility when calling services. While the service needs to have a high uptime standard, if there is an outage, understand that APIs not working can mean that they, in turn, have outages that place them in an uncomfortable position with customers. This problem is a highlight for PaaS providers where bills can be a significant development expenditure.
Caveats
I want to take a moment to highlight that just because an API is built and you are able to offer value for customers doesn’t mean that you will win market share. These companies thrived because they understood that support and documentation are core and central to the product experience.
As with most products, its our jobs as PMs to determine where our priorities and efforts should go and additional effort spent on features that need to be maintained can possibly sacrifice the experience elsewhere. The best way to look at APIs (in my opinion) is essentially as an other option that your customer may have to consume your product. However, if that experience isn’t supported well enough- it could be a detractor to your product.
An additional consideration for the product team is access and authentication to your API, if end users don’t have a clear way to deal with the tokens and permissions given to applications, this can pose an issue for the end user experience. As always, the needs of your users vary.
In sum, when executed property, building an excellent developer API will allow you to stay ahead of having your product becoming a commodity. It’s easy for companies to offer APIs, but the execution matters most. Its benefits are boundless, and it enables companies to get considerable recurring revenue and deep integration in their product adding stickiness if well positioned. This even allows your partners to take advantage of your ecosystem and allows you to extend your market to partners to better serve your customers.
