The Future of Open APIs on the Web

Then finally I’d like to start a discussion about what I’m kind of calling an open-API community. There was a stage in software where we understood that OS vendors were kind of owning the space, and we needed to level the playing field in some kind of way. I think some of this is going down in the API world, where proprietary individuals are owning the space, and they’re creating silos, and it’s been difficult to work with. For that reason, I think it’s a great opportunity to start thinking about open-API the way we thought about open-source 20 years ago. So let’s see if that’s really true. I kind of want to have a discussion. I’m kind of working this as a…I talk for about five minutes, and then we discuss. Let’s see if this works out. We’ve got enough people here that I can kind of [inaudible 00:01:43].

So I start…My sort of kick-off point is this notion of the cathedral and the bazar. This really starts a…it’s kind of this mean in the space. It’s this notion, this sort of story we have in the keynote today. We were talking in the security space. We were talking about the idea of lots of eyes on software and the notion of keeping things open. And does it really work?

This notion of the cathedral and bazar was first discussed in an essay that was released in 1997, at [inaudible 00:02:23]. So this idea of that there are a couple different ways to do open-source, the cathedral way of doing open-source and the bazar way of doing open-source. Eric S. Raymond, who delivered the paper, was arguing that one way is better than the other.

Within a year or two, he actually put together that essay and several other essays into a book and called it "Cathedral and Bazar." So this book was released in '99. What I find interesting is in 1999, he had convinced a publisher to release the book as an open-source book, which was a real learning experience as well. So this book is open for anyone to use and read, just as any other open-source is.

So what’s the heart of this cathedral model? Cathedral model is where the open-source is delivered when the product is delivered. Nobody gets to see it during the time it’s being built. We build it all ourselves. We don’t ask for any other input from anyone else. We know exactly what we want to do. We control all those details. And when we finally release the product, we release the source.

At the time two years ago, that was a common way of doing open-source. When we released the product, we’d give away the code. But in the essay and in the book, Raymond actually tried to advocate for another model, and that is where the code is developed in public, and he was saying that public over the Internet at that time, so that every step along the way, we get lots of eyes, and we ask for lots of contribution and get lots of people involved in the process. So this is what he called the bazar model, the model that’s like an open bazar where anybody inside the tent, everybody could get involved, and everybody can contribute in some kind of way. It’s not a free [inaudible 00:03:57] cathedral. It’s an open bazar.

So the first question I have, when we think about…This really relates a lot to source code. The question I have is: When we build APIs…There are lots of companies whose entire business model, entire delivery model is dependent on APIs, often APIs that they don’t control, APIs that are offered by Google or APIs that are offered by telephone companies or Microsoft or lots of other organizations. So what is the API model in this point of view? Are APIs developed in the open for lots of people to contribute to on the web? Or are they more of a cathedral model, where you just get the API at the end, and then you get some restriction information, and that’s it? Who here has actually worked in some kind of web API space today? All right. So how does that work for you? Who builds APIs here? You create APIs. Right? You have [inaudible 00:05:08]. So how does that work for you? Is it…Do you build these kinds of interfaces and kind of [inaudible 00:05:12] through GitHub? Can anybody contribute? Or is it mostly kind of a continued experience in your [inaudible 00:05:17]?

The other thing that’s interesting to talk about, just from the API perspective, rather than just talking about the services behind them, is you may remember recently, a couple years ago, there were legal fights over whether or not the API could be copyrighted or patented or closed out from everyone else. Right? Which had the possibility of threatening business models and other things. Right? While it played out against [inaudible 00:11:07] APIs, we would have had a huge implication as well for web-based APIs.

So this notion of whether the API is a little bit too much of a bazar, a little too much stabilized, whether or not it’s a threat from the patent or copyright standpoint or view, I think is something that is not quite resolved, and I think it’s a big challenge for us.

So I want to take a little step a little bit further in and talk a little bit about Raymond’s [inaudible 00:11:31] included in the book, "Cathedral and Bazar," is what he called his lessons, his open-source lessons, and there were lots of them. There were actually 19. We’re not gonna go through all of them, but it was this notion about scratching an itch and how you behave in a group and how you use the many eyes and so on and so forth. It sort of became a kind of an ethos or a kind of pattern about how we would talk about open-source software.

I think some of these lessons actually come really close to home for APIs or web APIs on the web, when we’re [inaudible 00:12:05] computing, and the easy one is scratching an itch. Like, we want this low barrier of entry, this ability to create an interface to allow people to access the thing. But I think that makes a lot of sense.

But this is what here, in particular, is pretty tough, to throw early iterations away. This idea that we should be able to start small and iterate has this situation where suddenly the interface is [inaudible 00:12:31] or it becomes too much of a bazar. It creates this tension about whether or not I can count on this.

Released early and often. I think, actually, we do a pretty good job in the API space about this, "Release early and often," especially if it’s a centralized sort of stats engine, rather than an API for localized situation. I can continue to add features, continue to make bug fixes and things like this, without actually affecting any of the consumers. So I can actually grow and evolve at a pretty fast pace, which is pretty cool.

I think the, "Many heads are better than one," is kind of a toss-up. I think there are lots of advantages to getting lots of people contributing to the APIs or contributing to the stuff behind the APIs. At the same time, I do think…I didn’t get your first name.

Chris: Chris.

It turns out it’s incredibly difficult to fork an API. In fact, the people who build the original API and [inaudible 00:13:43] don’t want that kind of activity because it bifurcates the community, and it’s hard to get it back together. As Quintin kind of mentioned, suddenly you’ve got too many things in there.

I think, "Smart messages, dumb code," is a pretty common pattern in the API space. We tend to send simple serialized messages or simple activation messages, like hyper-media, or just even, you know, content, simple content in small bits, like event-based. So I think it’s pretty good. That allows people to put a lot into the message itself. So you have a lot of possibilities there.

Here’s another one that I think is difficult. The original…A lot of the ideas behind the original Unix and Linux platform is like, "I’ll take a little [inaudible 00:14:19]," and before you know it, I’ve got an app that nobody has before. Right? There’s a lot of serendipity involved. Some of the early thoughts about the way the Unix system was gonna work is there weren’t going to be activations. There were simply going to be this environment where I can assemble things in any way I wanted to. I think there are parts of the API space that aspire to that. You know? Here’s a thing that does currency translations. Here’s a thing that does credit card transactions. Here’s a thing that does shipping. Here’s a thing that manages a cart. [inaudible 00:14:51]. I built myself a service.

Well, it turns out, I think as Quintin pointed out, it’s incredibly difficult to do on the API space because everybody’s a one-offer. Everybody’s got their own interfaces, their rules, and I have to build a special client adapter for every single one of these. I can’t just simply do one thing and one thing well and pass text messages back and forth.

So I think we’re having some trouble in this space, but I think most of this has to do with design patterns, rather than technical details. Now here’s a big one. "Plan to hand the project off," is another big principle from ESR’s set of lessons. You can’t just abandon an open-source project. You may have started by scratching an itch, but now you need to pass it on to someone else and keep it vital. There’s a real…I think there’s a real danger in the API community. I become dependent on a proprietary API and build a business model on it. Then suddenly that API gets yanked out from under me, or the business arrangement changes. I think we’ve seen examples in the talk earlier. Google has canceled APIs for folks. Twitter has changed APIs specifically in order to break a bunch of clients that have built upon it.

This offers or provides a real danger to the community, in the sense that I’m not gonna vet them out. I’m not gonna vet my business or build a model around it if it’s so brittle that I have to keep managing it or changing it every time.

So what I find fascinating is right around the same time that Raymond was talking about this notion of the cathedral and the bazar, and what’s the better way to think about open-source, there’s another gentleman by the name Clay Sherpy [SP]. Actually a year before, in 1996, I think it was for ACM or [inaudible 00:16:33]. He had written an article called…Oh, it was ACM. "In Praise of Evolvable Systems." What is an evolvable system? Clay sort of laid out this notion that you can start small and then add to it without creating a bunch of hassle.

He had this great line, which I’ve used several times, "Centrally designed systems start out strong, but then improve logarithmically." It has a lot of what we do in terms of central source. We have a set of controls, and we build out. We build out from there. But the point he makes is that systems that are evolvable actually start out weak. They’re less than minimum viable. They’re barely working, but they can improve exponentially by suddenly…Especially because you have this idea of serendipitous reuse and discovery and lots of eyes and lots of lines, suddenly, you can leapfrog. You can actually make lots of incremental changes that suddenly result in a big shift. We even know from fiscal evolution that often it’s a series of small random changes that result in a big success.

So Clay makes this sort of case that the central notion is very different than this evolvable notion, and that sort of takes me to kind of the next question I have. In our experience as people who use APIs and people who develop them, how many of them actually follow that sort of set of 19 principles, especially the ones we talked about, in terms of Raymond? And even better, how many of them actually can safely evolve without breaking folks? What’s been our experience? Anybody want to talk about what works? What doesn’t? Anybody built any that actually work along this kind of way? This is your chance to tell me. Talk to me now.

So that sort of brings this notion of maybe there’s another set of lessons. There’s another set of principles. Maybe we need to sort of create sort of an actionable set that go [inaudible 00:20:22] had all the 10 lies of the distributed software or something like that. Sun Microsystems. Maybe we need a sort of actionable list, sort of like what Raymond talks about, but that is focused specifically on distributed systems. Make sure that the changes you make don’t break anybody and so on and so forth. There could be a whole set of things like that. Make sure you design it to be extensible, as well as forward compatible [inaudible 00:20:47].

And that…I think that goes to the evolvable element. I think part of the bazar problem is if I fork it or if I start one of my own, what I’ve really done is actually make the problem worse instead of better. But often, if I have some open-source software, if I can fork that, I might actually make it better because I might be creating a different community. So the rules may be counterintuitive in some ways, I think.

So sort of the next idea is…I talked a lot about web APIs. Is there a way to talk about open web APIs the way we talk about open-source? Here’s the thing. These APIs are going nuts. In the API space, there’s just tens and tens of thousands of them happening all the time, and there are some big business models being built on them. So we heard Dropbox today and so many other things. Their total [inaudible 00:21:40] was based on an API. That’s how they distribute things. That’s how they actually get things done. They don’t distribute code anymore. They simply distribute the API, and we run it at a remote distance.

I read a recent market insider report that’s predicting $200 billion opportunity over the next five years in this space, and this is just one of the many versions of this story. There’s lots of opportunity. There are lots of chances for people to do well. There’s also lots of chances to do it really badly. I think those of us who spend time in the API space, every couple years, we hear about…Usually it’s an industry initiative. It’s like a cross with telco industry or a banking industry or a health industry. So they just kind of build this one [inaudible 00:22:26] API to rule them all, and they spend hundreds of millions, and it tanks because they’re not learning the basic rules about how to write distributed systems. And so there’s lots of opportunity lost there. There’s lots of troubles. People invest a lot, and they try to get something out of it.

At the same time, just at the individual API level, there’s a recent case where Google actually pulled its entire flight API that people had been building lots of apps on, because they wanted to build their own app. So they offered this API for a couple years, and it was maybe close to five years, and I’m sure they learned lots of things. Then they said, "Thank you very much," and they just pulled the pin. And, lo and behold, suddenly they have a new application that helps you track and manage flights. So that’s a drag.

Twitter has this reputation of actually, you know, not paying attention to their developer community very often. They have a huge API. [inaudible 00:23:21] loves headlines. Right? So I picked them just to kind of get us talking, but they’ve actually broke a chain, an entire client API, just to flush out competitor clients. They basically killed the client market by changing the API, and they bought one of the better clients, and then the rest of them are just barely hanging on by a thread.

So somewhere along the line, we’re becoming, those of us who use APIs or those of us trying to build systems on APIs, even if you’re building inside your company, we’re becoming captives of people that have other kinds of motivations or other kinds of means. This seems very familiar to me, but now it’s in a very different space.

So it sort of takes me back to the cathedral and the bazar model. Right? So what is the model where the source code is only accessible [inaudible 00:24:14] and we never get to see it? What is that? That doesn’t seem to be the cathedral or the bazar. Right? Suddenly I’m…All I get to see is the windows and the doors, and I’m given a key to walk in and out, but I might have that key taken away from me at any time. So I don’t know how to deal with that. I don’t know how to…Like, if I was an investor, this is a dangerous spot for me, I think.

So as a consumer of APIs, how do I know this is a healthy API? You know, one of the things I do in source code is I check to see how often the repo is updated, what the activity is on the site, and so on and so forth. Can I do that for APIs? Is that…For some of them, I can. Right? Because some of the API interfaces are released, but often, they’re not.

How can I tell this is safe? What’s being done with my data? What’s being done with the information I pass back and forth? There are a handful of times when we realize it turns out that using an interface that collects, like on a phone, it collects a lot of other data that I didn’t know it collected. Right? So maybe I’m building an application that is putting my users at a disadvantage.

Is this an ethical API? I have this cool API that I can use, that actually…I send a picture, and it identifies [inaudible 00:25:28]. Is that a mechanical turk or somewhere in Asia where somebody is actually reading a picture and typing it in and getting paid a penny? How do I know if this is going on behind the scenes? I can go well beyond [inaudible 00:25:45]. Right? How do I know what the ethics of the community of the organization behind this API? Conversely, their inclusion, where they source their goods and materials, all these other things. How can I find that out?

What is the governance model for this API? Is it one person who just decides how it’s gonna work? Is there a committee? Is there a committee that I can get involved in? All these other things that we have established for open-source software. Will I get there in the end? Will I have the rug pulled out from under me? Or is there a commitment in this open-API ethos that we will find some home for this interface? We will find some home for this work?

I think when we start to look at this list of things, these are things we all want, but I don’t think there’s any central way to get this. I think this is…These may be examples of things that are important to us, but there is no real leverage. We may need an open-API community the way we build in an open-source community, and the Linux Foundation may be actually a great place for this. I think we need that kind of community.

There are lots of standards [inaudible 00:26:51] talking about. It’s a bit of a bazar. Everything is a little bit different. We might be able to improve that with standards, even the notion of best practices or [inaudible 00:27:01] the way we think of drivers and so on and so forth, the way we solve a lot of this problem at the code level.

Mapping the landscape. There are hundreds and hundreds of API providers. Why do we have to [inaudible 00:27:15] community? Why do we not, you know, put them in the same standards? Why do we not get them to cooperate with each other and help each other? How do we support all the developers, all the startups, all the individual developers who want to create APIs or actually consume them, and look out for them? How do we actually connect stakeholders, whether they’re Microsoft or Google or Kubernetes or [inaudible 00:27:33] or some small company that started something? Everyone has a stake in this.

How do we create those principles about safety and ethics and values and disability? Can we incubate initiatives? Can we be a place if somebody is creating a new standard or creating a new set of principles or creating new libraries to create APIs? Can the foundation do that here? Can we identify and reward good citizens? Can we have sort of a seal of approval kind of experience that is part of the Linux Foundation, just being part of the foundation, sort of like what’s happened with the cloud native? Right? We sort of gathered all these disparate things together in order to create a community that helped drive that up.

Now it turns out many of these pieces exist, but I think they exist in very disparate ways, and they don’t exist in coordinated ways. One of the things that I like is that there is a thing called "open-API initiative" here at the Linux Foundation. It’s focused now on one particular format of description, but there is so many more things that I think that open-API initiative could do in helping the landscape and the community and all these other things. It may not be quite the mission of that group, but I think there’s an example there. What’s that?

We also had some people that are starting to map the space. This is one of my colleagues at API Academy [inaudible 00:32:34] last year, who started to map the API landscape. This looks very similar to the kind of landscape mapping that we see happening here at the foundation. Some of these are…This is again sort of a conflation challenge. Some of these are people who provide the sources behind the APIs. Some of these are people who provide the API framework or the API hosting. But there’s…So there’s some work that needs to be done there, but some people are starting on this effort.

There’s lots of online community API crap. There’s some other [inaudible 00:33:02] online communities. There are lots of local meet-ups, like local level, around the world. Almost every country I go to, every continent I go to, I can find a meet up. In fact, the Linux Foundation has the API Strategy and Practice Conference, one of the more successful conferences, has just joined part of the Linux Foundation. This is a great opportunity to start kind of looking at the landscape, collecting stakeholders, and doing all these other things. I think we just did one this past year.

But I think there’s still more. Like, where are the stakeholder…Where is that set of principles? Where are those incubating? Maybe we also need to talk about the notion of that conflation. We have to fight that. Then how do we reward good citizens? How do we make [inaudible 00:33:48]? In other words, how do we sort of get [inaudible 00:33:51] fly wheel working in this API space, where more and more of us are building businesses based on the notion of collecting things together? We can become victims of what happens behind the scenes. How do we protect ourselves? How do we enable more communities? How do we enable more people to do things on their own, without having to rely on the large private corporations? I think that’s kind of what I’d like to see.

I’m being told my time is up. I’m gonna open…I’m gonna end with this last question. We won’t get a chance to talk a lot about this here. I’ll be here for the rest of the day [inaudible 00:34:24]. You can find me. Do we need this kind of open-API community? Is this something that the Linux Foundation can help us with? Or some other organization? And if so, what are the things it needs to do? I think some of the stuff was mentioned here, like making sure we understand the difference between what’s an interface and what’s source and how we…How does it matter? Maybe it doesn’t matter. Maybe the source is all that matters, but maybe there is something else. So what is the future of creating an open-API space?

I need to stop because we need to keep going to the next session. I want to thank you very, very much. I’ll post the slides online later today. And if I can, I’ll try to do a transcript of the conversation we had. Thank you very, very much, and I hope I get to see you again. Thanks. Thanks.