Driving Digital Transformation at Speed

Driving Digital Transformation at Speed

Microservices, APIs, and Innovation

Mike: Okay. So I again wanna echo Jeremy’s comments. Thank you very much for being here. As a member of the API Academy I get to travel around the world talking to companies about their APIs, about their microservices, about digital transformation and this is the chance for me to get to learn from you as much it is to share the kinds of things that I’ve learned along the way. I travel quite a bit. Last week I was in California. The week before that I was in New York. Next week I’ll be in São Paulo, Brazil and then next month I’m in Europe and Australia and all sorts of other places. What’s really exciting is everywhere I go people are working on these things about APIs and microservices and even though that they’re very different businesses, even though they’re at very different places in the journey, kinds of organizations, there are a lot of similarities, a lot of similar things. So what I wanted to do is [inaudible 00:00:57] to you today the kinds of things that we’re seeing consistently throughout all kinds of industries and all kinds of companies. Sort of share that with you and give us a chance for me to talk about that a little bit.

We’re gonna have a lot of competition. So as I mentioned I thought I’d talk about three different things. This idea of services, this idea of APIs. Should I use the mic?

Woman: Yeah, you might wanna use it. I did tell them to let them know that they’re loud so…

Mike: All right. See if I can turn this mic…turn this mic on. Is that a little better? Is that a little better? All right, okay. We’ll try that and see how that goes. So as I mentioned, this is me. This is actually how you find me. This is the best way to figure out where I am and what I’m doing. This is how you find me on Twitter and LinkedIn and GitHub and I would love to connect with you. I’d love to learn what you’re working on, the kinds of challenges you’re seeing out there and this would be a great way for us to connect. I’d love to hear from you any time.

I mentioned I work in this group called API Academy. It’s a unique group of people. About six people and we’re scattered around the globe. There’s one based in Zurich and one in London and a couple in Vancouver and one in Silicon Valley. I live in Kentucky in the United States, so this is sort of a nice trip around my neighborhood this time. I’m not in some far flung location. I’m having a good time.

We publish a lot of material. The last project that we worked on was a book on microservices. And since CA sponsored the book you can actually download an e-copy of this book at this URL. You’ll get these slides but you can get a copy of this book and it’s a kind of a…sort of an overview. It’s not a big sorta deep down techy version. It’s…first chapter is on business value. The next one is on designing systems. There’s one on culture. There’s one…there’s some on code and DevOps and things like that.

So this is a great way to kinda get started in that space if you’re interested in it. We’re working on another book right now that’s gonna focus a lot on the API lifecycle and that will probably be out at the end of this year or early next year. There’s also a release on security, which I don’t have a copy of, I thought we’d have copies of here today, but if you contact your CA rep or you hit me up one of the channels I’ll be able to point you to the security book on microservices as well.

All right. So just to kinda start this off. We talk to a lot of companies about microservices and they talk about them in a lot of different lights but a pattern I see quite often is that microservices are really a way of toolmaking inside your organization. So services are services. We’ve all used kinda services. We have a lot of people…how many people are doing some kinda SOA service or something like that? Anybody, right? Any SOAP or anything like this? Maybe some REST stuff, right? These are all services, right? So whether we brand them microservices which is kind of the new hotness or not doesn’t really matter too much but the pattern we’re seeing with people who are creating these small interchangeable services is they treat them a lot like a toolkit. Some organizations actually are trying to build a whole set of tools that they can then use to build products upon. So it makes it safer, cheaper and easier to create a product. You don’t always have to start from zero. Others are using services as a way to kinda retool their organization, fill in some technical debt gaps and that sorta thing. But what we found really interesting is this notion of everyone that we’ve talked to has some sense of toolmaking about it. They’re focusing on this idea of creating a service that does one thing really well. This is the service that manages users, or this is the service that handles policy search or something like that. So they’re focusing on one job to do and they can scale that job any way they want. They’re also focusing on this notion of making sure the output of one of the services can actually be the input of another service. In other words, from the very beginning, they’re designing these services to be able to talk to each other. Amazon for example doesn’t…they focus on this idea, this…of the way these services talk to each other. They allow their teams to use whatever languages they want, whatever tools they want. They say, "The only thing we wanna govern is the way we talk to each other." Because they follow the second principle as one of their highest principles. There’s also a pattern that we see which is designing software to be tried early and often and sometimes thrown away. So Facebook is a really good example of that. Facebook has this phrase they call "fail fast," which means to them you learn through doing and the first time you do it you usually don’t get it right. So let’s do that fast. Let’s get that done early. Don’t wait. Don’t do a whole lot of upfront design. You learn from every attempt so get in there early and start making those attempts. So that’s a pretty interesting concept that we see quite often. And that’s one of the reasons services are kept relatively small because it’s easy to build this one quickly. It’s focused on one problem. Try it out. If it doesn’t work, build another one. So that’s a pretty common pattern.

And here’s one that we found really interesting. Almost every one of the organizations that are great at this microservice or small service space use services as a way to create services. They actually write code that helps them write better code. So Netflix is a really good example. Has anybody heard of Netflix Chaos Monkey or Chaos Army, right? They wrote a whole bunch of tools, spent thousands of hours and hundreds of thousands of dollars to write software that tests their running system and tests their tools to make sure that it’s actually gonna be resilient. Amazon spent a whole bunch of money creating tools that make it easier to create new products because at Amazon, one of their highest elements is they wanna create lots of products for people to try. Etsy. Has anybody heard of Etsy before? Etsy is a kind of a shop, right? They have this whole set of tools around releasing because they release to production 30 to 40 times a day. So they have a whole new set of tools for that.

So every organization that we found finds out what their secret sauce is and builds their own tools for that. They buy whatever else they need, but the thing that really embodies their culture or their values or the things that make them powerful, they usually end up creating custom tools for that.

So this might look interesting and this might look familiar. It turns out that this list is a version of a list that’s more than 40 years old. It was…they were really the ideas of creating the Unix operating system, which is now popularly…popularized as Linux. This idea of building small programs that talk to each other that do just one job that you can build little scripts or little bits of code to glue things together quickly and easily and just do that over and over again, and actually write tools to help you write the applications that you need. So what’s really happened over the last 40 years, the last two generations of programmers, we’ve taken the principles from AT&T and Unix and all these other things and started to play them out across the network itself. So there’s a lot of familiarity. There’s a lot of challenges to moving from just one box to the whole network but the good news is we’ve got 40 years of history, 40 years of learning, 40 years of writing and reading and examples to fall back on. So that’s a really, really interesting idea. It turns out that the internet that all the new kids on the block out there in Silicon Valley are finally catching up to what traditional programming, mainframers and others have known for 40 years. So that’s a pretty good sign. That’s actually a really good sign for those of us who already work in a shop that we’re so-called "legacy shops" that have all this intellectual property that we’ve invested in for decades because that’s still super valuable. And it turns out most of that stuff has been built on principles like this already.

As a matter of fact, when Werner Vogels was asked early on when Amazon said that we’re building all these small services that are nimble and agile and talk to each other, he was asked this about a decade ago in the ACM Journal, computing journal, and somebody said, "Well, what do you call this style of programming?" He said, "It’s service-oriented programming?" Like he didn’t care about the new names. They cared about what they were doing, and they were doing principles that were based, you know, quite a while ago. So one of the things we remind our customers all the time is that microservice is about toolmaking, whether it’s making your own tools to build your own products, or writing code to help you write code. These are all ways of creating tools. Now actually a useful definition would be handy too. There’s enough definitions of microservices to probably fill a lake. I’ll just add mine to the list. But the way we talk about it in the Academy is relatively short. It’s loosely coupled components running in an engineered system. So what does that really mean, loosely coupled? There’s a really nice definition for loosely coupled. It means that my team can release our code to production without having a meeting. I don’t have to wait for somebody else. I don’t have to say, "Well, I’m ready but the data team’s not ready." Or, "I gotta wait until all the UI people have gone through the blah, blah, blah." I release on my schedule. That means that when I release code into the system, I’m not gonna break somebody, somebody else that’s already running.

Now it turns out that’s not easy to do. It’s a nice, easy sentence but that’s not easy to do. Especially since most of us have learned over time to be very much dependent upon others. We have really tight coupling, really tight integration, so releasing something can be actually pretty dangerous. But what we found time and time again, the people who are using this DevOps, agility, microservice-y kind of culture are making it possible for teams to have the autonomy to release when they’re ready and not release when somebody else is ready. Because when it turns out if you have to release when somebody else is ready, you’re only as fast as the slowest runner in your group. Whereas if you can actually decouple this and get the individual teams some autonomy, they can run at their own pace. As a matter of fact, Gartner has this notion about…it’s bimodal. Is that the phrase they use? Bimodal, or two-speed system? When we were talking to Gartner when they, a couple of years ago when they started this, our first question was "why two?" Turns out there’s lots of paces, lots of speeds. In fact, closer to systems of record, usually the change doesn’t happen very often. Maybe not at all. Maybe we’ve got some [inaudible 00:11:33] and some mainframe stuff. There’s no agility there. There’s safety and maintenance. But as you get closer and closer to the customer, closer and closer to the handheld app, agility and speed becomes really important, and flexibility becomes really important. So you’ve got lots of possible speeds and those speeds change throughout your organization and what your targets are.

So it turns out there are really two things we’re trying to do. We’re trying to get speed but we can’t sacrifice safety in the process. So what we talk about in the book is this notion of harmonizing this notion of speed and safety. So speed gets us to market faster, gets…closes that loop so I get feedback a lot sooner, my developers get feedback instantly as to whether or not this is a good test, a good product, a good service. I learn from that first edition so I can throw it away early before I put it into production. But we also have to have the sense of safety. And safety is different in different parts of the organization and in different channels. So safety has a completely different meaning in the era of Facebook than it does in a bank, right. At Netflix safety is just making sure that your individual information is kept safe and that they can safely stream to where you’re…whatever your target is. But at a bank safety’s a whole nother thing. I have to have the balance exactly right, I have to protect your information, I have to do lots of other things. So safety is really kind of a relative idea and it’s not a matter of picking one or the other but a matter of tuning, figuring out how much agility can I get while maintaining the safety I need. In fact, the most important way that you can get safety in your system is through the DevOps culture pattern. Safety happens from the time I check in until it’s up and running, right. So if I can automate those tools then I can improve a lot of safety. It’s harder for me to put something into production that’s gonna be damaging, it’s harder for me to sideswipe guidance. I can use a lot of things in the CI/CD pipeline that actually tests and make sure that I’m following the group guidance. So there’s lots of things in that DevOps space that increase safety. For the most part what our customers tell us is DevOps is mostly about safety and not about speed. Speed happens when I have small services that I can code quickly and get into that engineered pipeline.

So fine. So from this service point of view what are three things that you can do now that you can start working on inside your organization? This is not an all-or-nothing deal. This isn’t "we’re all gonna be microservicey by Friday." This is an incremental process. What can you start today? And we talk about three things that we see often. This notion of build pipelines that I just alluded to, the idea of engineered deployments. How can I automate an engineered deployment so that they’re repeatable in safety? And this idea of reducing work in progress which is really the big one.

So building pipelines. The build pipeline isn’t just this notion that let me take source code and check out all the right libraries and create some executable I can deploy. Build pipelines let us do lots of other things too, like run jobs to check for dependencies. Am I using the right dependent libraries, am I using the right versions of all those, are they gonna be available when I go into production? So I can actually run a job in these CI/CD tools, Jenkins or some other things, that checks the dependency list and will give a yellow that says, "You know, I see you’re using version three of our network stack. Version three’s gonna be deprecated in six months. I’m gonna give you a yellow on the build, not a green. And in two months, I’m gonna start giving you a red. You’re not gonna be able to build if you haven’t moved to the new network stack."

So you can teach developers right away. When somebody does a build she sees right away whether or not this is actually a good idea. You can do the same thing for model checking. You’re using the wrong schema for users. You’ve actually moved to a new schema. I’m not gonna be able to release this into production. You can do it for code complexity. You’ve got too many lines of code in your methods or you’ve got too many jumps or too many calls. All these other things that you can automate without having to manually review. Even to the point of code style. So you’re not using the right naming conventions or the right capitalization or all these other things. You can do lots of things that improve the quality of the code that you release into production just through these automated tools and pipelines. And the good news is you get feedback right away. If I’m a developer and I’m trying to build something, as soon as I press enter and check it in and I get a build, it’s gonna tell me right away. I won’t find out three weeks later in some human review, right. So what I’ve done is reduce that feedback loop to just a matter of minutes.

So that’s a really powerful way that you can start to change the way your code looks and change the way your developers interact when you start introducing this notion of build pipelines. And build pipelines lead into the testing strategy, lead into this sort of a engineered deployment strategy. So it’s more than just unit testing, you know, just single standby unit testing where we figure out whether or not I’m actually doing the math properly. That also means acceptance testing. Am I actually solving the business problem? Sometimes this is called behavior testing. And we can automate a lot of that. We can’t automate all of it sometimes, but we can automate a lot of that acceptance testing. And then the notion of capacity, and I actually scale this the way I need to do. By the time we get into capacity testing, one of the key elements that we see our customers use is service virtualization. So what I need to do is have a virtual copy of other parts of production so that I can go ahead and check in, run these tests and not worry about actually fiddling with the production. In fact, one of the key values of virtualization is you can actually create the sad path test, right. We all know about the happy path, right? Does this actually work the way I expect? But now when I do service virtualization I can actually create bad data. I can create situations that test the limits of all of the inputs and outputs and computations and that’s much easier to do in a virtualized environment.

Finally, I might have to get humans to talk about some showcase issues. Does this actually…is this usable? Get a small team together. And then I can actually release into production. Some organizations can actually automate almost all of this track. Etsy’s pretty good at doing this. So there are cases where they can automate from check in all the way to production release. And Etsy releases 30 to 40 times a day, which is really quite incredible. Now it’s good news for them is that they’re really running a sort of a shopping cart shop. It’s not like they’re doing health records or financial work. So it’s pretty easy for them. But they’ve learned to take advantage of that.

The biggest thing that you can start to change today about the way you treat services is this notion of reducing work in progress. A lot of customers we work with release maybe three to four times a year. That means you only have three to four times a year to get things right, and if you don’t get it right it’s gonna be really costly and you might have to wait three months in order to get it right again. Or you’re constantly in a hotfix mode. It makes it really difficult. The other big challenge of releasing only three or four times a year is you have a lot of information in that release. If I release something that has 150 changes in it, there are more than 11,000 possible ways that can go wrong just inside that package before I connect it to the rest of my network. If I have a bug in that release, it could be really hard for me to figure out which interaction is causing that. So it reduces my chance of success and it increases the risk, because I only get a chance to do this three or four times a year. However, if I release in smaller packages and if I release the same amount of changes every year in smaller packages, say maybe every two weeks, all of a sudden I’ve reduced the size of the package and that means I’ve reduced the possible things that could go wrong. Maybe I’ve only got about…maybe I’ve got 10 things in this package and that means there’s probably only about a 100, 150 things that could go wrong inside that package. My odds of finding a mistake are easier. Even better, if it doesn’t work, in the next two weeks I could do it again.

So one of the key values of reducing this work in progress is the loop that I have is a lot shorter. That means I reduce the amount of work I do each time, but I increase the chances it’ll actually be successful. This doesn’t actually represent getting more work done in a year. It’s just having more success in that same year. Less stress, more success. There’s a great book that introduced the notion of DevOps into the enterprise and I can’t remember the name of the book. I’ll include it later, but one of the comments from one of the authors is "if it hurts, do it more often." So one of the challenges for us in release is release is always a really stressy moment. "Oh, we’ve gotta make sure people aren’t on vacation. You can’t be home with your family. You gotta be in the office over the weekend. It’s gonna be a really high stress moment." Whereas if you do it every week or every two weeks, suddenly you reduce the risk, you reduce the stress, and it becomes a daily occurrence. I mentioned earlier that Etsy does this 30 to 40 times a day. They’re very, very small changes. Sometimes it’s one bug fix is a production release. When you join Etsy, the very first day at Etsy, you press a button and release into production, because they say, "because that’s what we do here every day. We’re about releasing to production." So they’ve turned production releases into commodity items. They’re a no-stress everyday occurrence for them. Reducing that work in progress also reduces the chance that somebody will actually be gone once you release that. We released it, there’s a bug and she’s already on another team. It reduces the chance that people will forget. It reduces the chance that somebody will add a fix that actually conflicts with somebody else’s fix, so you don’t have these big merge moments where it takes a week. I know some shops actually stop developing for a week or two while they organize their release to go into production. So this can increase a lot of elements.

So this is a great question that was posed to us by Tom and Mary Poppendieck when they were writing the book "Implementing Lean Software". How long would it take you to deploy a change that would involve just one line of code? One line of code changed and it needs to show up in production. And more importantly do you do that on a repeatable reliable basis? Can you do that over and over and over again? Because it turns out incremental change is the way to success. You got a big problem? How do you eat an elephant? One bite at a time. Exactly right. So one change at a time. So that’s a really great metric. So no matter what tools or what kind of culture, what kind of approach you have in the shop, the real metric is how long does it take to make a change in production? How long does it take to turn a feedback into a feature, an idea into install? And that’s the key measure that can help you figure out if you’re actually getting better, if you’re actually meeting your goals.

Okay. So let’s talk a little bit about APIs. And the way we think about APIs at the Academy is APIs are about multichannel. APIs are about lots of possibilities. Being where the customer is, being on that customer’s device, whether it’s their handheld or their tablet or their laptop or their desktop or a machine shop display screen or the warehouse screen or wherever that is. How do we get to where they are? An APIs help us do that.

Now from the APIs, the key element is the interface, application programming interface. So the interface is what helps us interact with others. Remember in that old Unix example the output of one is the input of another. And we know how to design good interfaces. We know that interfaces make us happy or sad. They are frustrating. I just accidentally deleted all my contacts. I’m really angry. Right, we know what those experiences are like. Developers have those same experiences with APIs. When we go into a shop and we look at people’s APIs and they ask for a feedback, we look at how usable those APIs are, how easy it is to make a mistake, and how easy it is to be successful. That’s gonna give us a great indicator of the number of bugs and errors that get introduced into code when you release them. If it’s easy to do the right thing, people will do that. If it’s easy to make mistakes, people will do that. Or they’ll work extra hard not to make mistakes which means now they’re working harder than they need to.

So just as we design user interfaces to make our customers happy, we need to design API interfaces to make our developers happy, API interfaces that make it abundantly clear how you use them, what they do, and how they work is a key to success in designing these APIs. Remember APIs are just the front door. They’re not the service. They’re the interface. You might have actually multiple interfaces to the same service. Because inside the organization I might have lots of CS degrees, lots of Java programmers but maybe some of my partners are working in Node or Rails or PHP or some other language. Maybe some of my partners are even working from spreadsheets. So I’ll change my API to be where those people are. It’s not one API to rule them all. It’s the interface that matches more than anything else.

I live in Northern Kentucky, I live very close to Cincinnati. Cincinnati is the home of Procter & Gamble. Procter & Gamble is the king of repackaging the same soap in a different box, right. If you’re a college student you’re gonna wanna use this product. If you’re a homemaker, you wanna use this product. If you’re single and living on your own, you’re gonna wanna use this product. It’s the same soap, but they have different interfaces, different ways to market to different segments. And that’s a big part of what APIs can help you do. They can help you target your audience at a very low cost. I don’t need to rewrite the service. Just change the interface, change the packaging. And that’s why APIs can be very, very powerful.

So interface design is a key. We design for the consumer. We design for the jobs to be done by our customer. Not for us. We don’t design services for us and expect everyone else to use them. They wanna use their services. There is no one single API. We have some customers that still have this idea that, "You know what? If I could just design one API, everybody would leave me alone for like six months. I’ll design this one and we’ll be done forever." No. Never works. We’ve tried that with canonical data models, service models, blah, blah, blah. The same thing happens all the time. There’s this weird thing called change. The company changes, the business model changes, the operational elements change. So having this notion where I can create an interface that matches a "jobs to be done," to use Clayton Christensen’s term, is really key.

Design part of a process. Make design a part of whatever you do. Let’s gather information, let’s do some design, let’s build one and let’s see what happens. We won’t have a chance to talk about it here but a pattern that we advocate that we see people use is sketch, prototype and build. Sketch on the back of an envelope exactly what this should look like. There are actually some online tools that let you sketch an API so you can practice interaction, show it to stakeholders and devs. Once you like that sketch, go ahead and do a prototype, which is a low cost build element like using Swagger or something like that. And then finally, now that you know exactly how you want it to work, go on and dig a big hole and pour a lot of cement and bring in lots of iron and build it. But now you know the risk is low because you’ve worked on exactly how it should act and behave already through your sketches and prototypes.

So you need to make things what we say safe, cheap and easy. Make it safe to build one of these and put these into production without a lot of cost. Make it cheap. Make it so that I don’t have to ask for a big budget just to test something out. And make it easy, so that if I wanna make a change it’s not gonna be really painful or costly. So that’s what we see shops do time and time again. They reduce the risk and that means they can increase the number of experiments and that means they can get closer to success faster than their competitors.

So to use the same pattern from Tom and Mary Poppendieck, how long would it take you to release a new API? Especially if you think of APIs as just the interface and not the services. You already have the services built. How long would it take me to sketch an API? Most organizations we work with can sketch out an API in a matter of a day or two. They can build a prototype in about a week. They can build a minimum viable API that’s production-ready in less than 90 days. That means I can get a new product. I can use all of those tools that I have inside the organization. You can get a new product to market, get it on somebody’s phone, get it in front of a customer in a quarter, in 90 days. That reduces the risk and reduces the cost.

So what are the patterns we’re seeing in people creating these APIs, these interfaces? A big one is focusing on InterOp and not integration. You think about the way the web works, the way we connect our websites. We don’t ask permission first. We don’t have a meeting. We don’t even understand exactly what’s happening on the other end, but we have a link. We have a connector between the two. Maybe a form that asks for a few bits of information. That’s all APIs need to do. They don’t need to have an intimate knowledge of the model behind the scenes or the database behind the scenes. They just need to know about those interactions. When you level up to that top level rather than a deep understanding, you can get a lot more things done.

Here are some patterns that we see people doing. This gets a little fiddly but we’re kind of impressed with some of the things that we’re seeing. Now that we have IoT, now that we’re having a lot of really short interactions with small devices all over the place, HTTP is not always the best strategy. MQTT, CoAP, web sockets, direct TCP/IP conversations are being used more and more as we see people working. Adopting formats that are friendly for machines rather than humans is another key. And then we’ll talk about this idea of vocabularies.

So there are lots of possible ways that you could actually build the same API. When you design an API to only work in HTTP, if you suddenly wanna do it for internet of things or on some other device platform, you have to rewrite the whole thing over again. So designing without having the implementation first is really key. Plan to possibly support multiple protocols in the future can get you further…a little bit further along the way.

There’s another big item here. Am I losing my…there we go. And that’s the use of new formats. Has anybody heard of the Atom format before? It’s been around for about 10 years or so. There’s a thing called HAL which…the hypertext application language. Amazon is using HAL for more and more of their internal APIs because it means that people sort of understand the format even before they’ve met them before. There’s another one called Siren, which is used in an IoT space and there’s one called Collection+JSON, which is actually a lot like Atom, but it’s in sort of curly brace format because we all know curly braces are gonna solve the world’s problems, right. It used to be angle brackets were in, but it’s curly braces now, man.

But all of these things are different ways to represent the same bit of information. HTML could go on that list. What’s not on that list is XML and JSON. XML and JSON are not on that list because they don’t have enough information for machines to understand what to do. You have to write a lot of code. Whereas these actually have a lot more information. They’re like HTML. They have links and forms and descriptors and things like that in them. So that’s a really interesting challenge that we see coming up. And finally we’re seeing people actually adopt a notion of vocabularies. So there are some new formats that have been developed that actually focus on explaining the vocabulary, all the data words and all the actions that you could possibly do. Does anybody know Eric Evans? Design Driven, Domain Driven Design? So this actually comes out a lot of that. Usually these are called bounded context or domain boundaries. So ALPS is a…actually it’s a format that I’ve worked on with a couple of other people. Dublin Core Application Profile, JSON Home. If you’re really into the geeky stuff, there’s this thing called RDFS, schemas for RDF and OWL. But what we’re seeing companies do is their canonical model turns out to be their glossary, or their vocabulary. And then we apply it to HTML, we apply it to JSON or we apply it to Siren and then we apply it over HTTP or all these protocols. This becomes the thing that gets managed more than anything else. The tight governance happens on the vocabulary, or the glossary of terms. So these are some pretty interesting leading edge elements that we see when people are moving beyond just creating an API to making APIs talk to each other.

So the last thing I wanted to talk about is this idea of innovation. It turns out all of the companies that we work with have some sense of what it takes to innovate quickly, and they try to make that easy inside their organization. Because innovation is about adapting. Adapting to change, adapting to new customers, new markets, adapting to new competitors. How do we innovate inside our organization?

So the question that we get a lot is "what does innovation look like?" It does not look like this. It does not look like a well-oiled machine. Innovation is not a predictable element. Innovation is not a thing you buy off the shelf. Innovation is not a thing that you get everybody to go dress up the same and march to. Innovation is not a process. Innovation is sort of a state of mind. It’s a way people interact with each other. It looks more like this. Innovation looks like a bunch of ants running around. This is for you specifically. It’s a long story. Innovation. You look down at a bunch of ants running around. You wonder, are they actually getting anything done? Is there something going on here? I can’t really tell. Ants are really weird because ants do not have a leader. We think the queen is the leader. She’s not. She’s busy. Ants kinda decide for themselves in a group what they do. They’re a self-organizing organism, self-organizing group. And that’s what innovation often looks like. You look at the team and you’re not really sure are they really doing something, or not? And why are they dressing so weird? Like what’s going on there and why are they yelling in the next room? Like we’re all kind of upset by what goes on in these kinds of innovation spaces. There’s a great article about this called "Your Innovation Team Shouldn’t Run Like a Well-oiled Machine". It’s out of the Harvard Business Review and you can actually find this online. You don’t have to pay for it. It’s an open webpage. It’s a really great article that talks about what are the keys that make innovation work. And I’ll give you just some kinda quick notes from that along with some other things that we’ve learned.

So the first one is your innovation culture, the way you interact with each other, beats your strategy every day. Drucker had that great comment. I think he said, "Culture beats strategy for breakfast." Like and the very start of the day, you may have the greatest strategy for taking over the market, but if you have a culture that can’t afford to take risk, that isn’t willing to try some things out, that makes it hard to be creative, then your strategy’s not worth anything.

Culture also beats engineering. Doesn’t matter what hardware and software you have. Doesn’t matter if you’ve got the latest tools from Netflix that you downloaded somewhere on some open source website. It’s your culture that will decide whether or not you’re successful. This is one of the reasons by the way that Netflix gives away most of the software they build for their internal use, because they know the software is not the secret sauce. It’s the people using it and it’s how they get to use it that actually makes a difference. So long story about the American auto market and Toyota that talks a lot about the difference between just knowing what you should be doing and having a culture that makes it possible. So we’ve had to learn over the last couple of decades to adopt a different culture in the auto field in order to actually be able to innovate and build quality in.

So the last item on this is culture beats code. Doesn’t matter how many programmers you buy, doesn’t matter what degrees they have, it doesn’t matter if you’re using the latest Rust or Go language or any of that kinda stuff like that. It’s the culture of innovation, the culture of experimentation that really matters. Making it possible for your teams to try things out is really important.

And that’s when…we talk about this in our book. This is one of the diagrams that talks about this notion of the microservice way or the microservice culture. You’ll notice at the top that there’s this idea of what a service is and on the left of that there’s this idea of the engineered system, the macro solution to it all. Those are the only two tech elements on the screen. The other three elements, the organization, the culture, and the processes and tools are all people stuff. And it turns out if you got into this business because you love computers and you didn’t like to work with people, well, that’s gonna be a bummer. Because it turns out people are the system. People are actually what make it work. And organizing people and helping people to build a culture that they love is how these companies actually pay off. There’s a great example, again from Netflix, on processes and tools. When they first moved their stuff to the cloud, it took several years to do it, but within the first two years it was the point where they had the stuff all up in EC2 and you could actually run an EC2 instance. Before they went to the cloud it used to take six weeks from request to getting a 1U racked in there on-premise head in. Six weeks. So they would always over capacity and stuff like that because they were always worried about how long it would take.

So after a couple of years they get everything working on the cloud and they give out these EC2 instances. They can just flip a switch and they get another instance up and running. How long did it take from request time to actually having an EC2 up and running and ready? Anyone wanna guess? What is it?

Man: Six weeks.

Mike: Six weeks is the right answer. It took them six weeks. Why? Because they were having the same meetings, filling out the same forms, having the same reviews before somebody could finally flip the switch on the EC2 instance. So it turns out the tech wasn’t the problem. The tech wasn’t the thing that was keeping them back. It was all the process. Now when you were doing this all with humans there were lots of opportunities to screw that up. Having meetings and reviews and tests made a lot of sense. But once you have it automated there’s no fiddling. That’s gonna be the same every time. Have that meeting to figure out exactly what that image should look like. That’s fine. Do that once. That might take six weeks. After that you just repeat it like flipping the switch. So changing processes become really important for an innovation culture. Why are we doing this? Can we automate this? And if you can, let’s do that.

There’s other examples on culture and organization which we’ll talk about a little bit too. So what can you do now to kind of encourage this innovations culture inside your organization, encourage people to be able to take risks without hurting themselves? We talk about right sizing teams. Anybody hear of Conway’s Law? Conway? Mel Conway? We’ll talk a little bit about that and this idea of unplanned innovation. You don’t get up in the morning and say, "It’s nine. It’s innovation hour so I’m gonna innovate for an hour. Then I’ll go back to like writing code." Unplanned innovation is really what you’re looking for. So right sizing teams. Anybody heard of this? Dunbar’s Number? Dunbar’s Number was this…it’s popularly the 150 in this list. That’s supposed to be the number of friends you can keep in your head. If I meet somebody else, somebody’s out. Every time I visit a new place, somebody’s out. My head’s full. So originally this list of Dunbar’s Number was developed by a social anthropologist who studied chimp groups and he studied…basically he has this theory that says the size of your brain actually governs the size of the group you can successfully interact with. And we have big brains so we can interact with a large group. Most of the…even Facebook originally…the number that comes after 1,500 is 5,000. That was originally Facebook’s limit for friends. They were actually using Dunbar’s theories to help organize online social networks. It turns like organizing virtual networks is very different than human networks. So they’re…and it’s also easier for Facebook to sell more ads if you have more than 5,000 friends so they got rid of that. But the idea still holds. What’s really interesting is not the 150 number about what I can remember in my head. It’s the numbers down in the bottom here, 5, 15 and 35. It turns out that the name that he has for the group of five is called the intimates. That’s where you look each other in the eye every day, you know exactly what each other are thinking and you know exactly how to work together and there’s very little talk, there’s very little discussion. We just know each other. It’s not a coincidence that most startups are around five people. You don’t create a startup day one and have 150 people show up. It just doesn’t make any sense. What they do is they start with that intimate group. There’s five of us around a table. We all know each other…what their job is. We know…I know exactly if Mike is not doing his job. He has no way to hide. And then you start to grow and you get to what’s called the family group, 15. Fifteen, you can still pretty much sorta just have a short discussion. What are we gonna do today? Have we got this solved? Good. By the end of the day have a little talk at the end. This is where we get the notion of standups, right. A little standup at the beginning of the day. It’s the same basic idea. And there’s very little communication and the communication’s pretty clear. Then you move up to the next size which is around 35, 35 to 50. It kinda varies. And all of a sudden it’s a little tough to get everybody on point. So maybe we’re gonna have to break them into smaller groups. I might have to write this down so you remember. Maybe we should have some documentation. Maybe there’s like a guide that we should sorta share. And somewhere around that 35 to 150 suddenly there’s an HR department, there’s a legal department. There’s all these things. And some people when it gets around that level they say things like, "You know. This used to be a great company to work for. Now it’s just like all these forms and paper works and meetings and blah, blah, blah." Culture changes for every step along this way. What we find really interesting is right sizing teams lead down toward the bottom end of this curve. Having a team of five or a team of up to 15, 12 to 15 can be very efficient and very fast. I don’t have to have as many meetings and I get exactly what I’m looking for. We actually recommend to folks if you’re organizing your team culture, aim for what we call "Dunbar Level One and Two." There’s a great company called Spotify. Anybody know this company? They really excel at this team element. There’s a group of articles called "Scaling Spotify," which I highly recommend which talks about how they go about doing this and why they do it this way. They have a much more complex…it’s a much more nuanced system about how they interact with different teams and make sure that they don’t become isolated. It’s great stuff.

And actually the size of the team comes to this thing about Conway’s Law, this idea that a system’s design is a copy of their communication structure. The communication structure you have is the software you get. Right, if it’s a bunch of people that don’t quite understand each other, they keep writing memos back and forth and it’s hard to integrate and it never works quite right, that’s exactly the software you get. It’s a little confused, it’s kinda balkanized, there isn’t clear communications and there’s lots of messes that we have to keep fixing up.

So here’s the challenge. Melvin Conway came up with this 50 years ago when he was working at IBM. It’s now one of the things microservice conferences talk about every time. You almost can’t do a slide deck anymore on microservices without mentioning Mel Conway. Mel by the way is still alive. Mel’s about 83, 84 years old. He’s done lots of things besides write this one paper in 1967. He’s a really engaging and entertaining guy. But his notion, his real observation is that having a team work on its own gives you a lot of power. And this…we mentioned this earlier. If I have to hold a release until some other team is ready I’m not an independent team, right. I’m not a team of five. I’m five people that are part of a team of 150 and I gotta wait for them, right. That’s the team.

Somebody who’s really, really good at this sort of team idea and this team management and this independence is Amazon. At Amazon all of the teams are not only autonomous and independent but they’re black boxes to everybody else. The only way they’re supposed to be interacting with other teams who create other products is through documentation. We don’t have an all hands meeting at Amazon. Each individual team works on their own product. So they really emphasize this independence really, really strongly.

And the last one is unplanned innovation. This is what innovation looks like. It’s not a straight line. It’s a mess. You go forward, you go back, you go around, haven’t we been here already? Somebody peeks in and they say, "Are you done yet?" "No." "When’s it gonna be done?" "I don’t know." That’s what innovation looks like. You can’t schedule it and you can’t box it in. So organizations that understand this try to figure out how they can give as much freedom to their group as possible to be innovative. Google originally had a notion of the 20% rule. Does anybody remember the 20% rule? Twenty percent of your time you could do whatever you want. You don’t need to ask anybody. Gmail came out of the 20%. The maps came out of 20%. What was that one chat thing that they were…they had for a while?

Man: [inaudible 00:45:54]

Mike: Yeah, Wave. That came out of 20%. So, you know, your batting average is not 100%. But the idea is you build what you’re interested in and let us know.

Man: AWS was an innovation.

Mike: AWS was a great way of saying, "You know what? We got all this stuff sitting around. Can we like just start billing other people and let them use it?" Right? Stuff that’s sitting right in front of them. So what’s not necessary this thing that we planned out or mapped out. It’s people thinking creatively. That’s a great example by the way.

So unplanned innovation is really, really powerful. This is a line that we heard from somebody at Amazon. "If you wanna achieve greatness, stop asking for permission. Just do it." The way that was embodied for us is the way Amazon does their new product space. If I come in on Monday and I have an idea for a product, there’s a site I can go to. I press a button and fill out a few things. Gives me a couple of EC2 instances, a build pipeline, a test harness, a JIRA list, an email list and some other things that now make it ready for me to start building a product. The last step in that series is to tell my boss I just started a new product. Not to ask my boss. Say, "Oh, Mike started a new product." And my boss gets that information and she can watch that on a screen or whatever. Now it may turn out that Mike has started 20 products this month. Maybe Mike needs to have a meeting. "Are you gonna finish one of these, Mike? Well, you got something planned? Maybe we should kill a few of these." That’s fine. But the idea is to make it possible for people to be creative and to follow that creative muse.

So that goes to sort of the next level. How long would it take to create a new product? At Amazon they can have a…they get an idea, they go through this process. It’s sort of like the lean canvas process. Give me the basic idea, package it up. How would you package that up? Write the marketing information first. What’s the audience? What’s the problem it solves? These get reviewed and they say, "Okay. Build a web page and see if you can attract a crowd." "New product coming soon. Enter here with your email address." "You attract enough of a crowd I’ll give you a little bit of money. You can build a…like a POC version." See if you can gonna attract a crowd there. If that works, then we can talk about a budget for a minimum viable product. So they can get products out in months. They can start with a minimum viable product. They can usually get it out in six months. That’s a new product for them and then they can decide incrementally if they wanna continue on it. So they don’t have a big plan, they don’t have a big upfront, "I’ll let you know in a year" kind of a way. They’re constantly getting feedback along the way.

All right. One more thing. I’m kinda running out of time here. You know this book? "The Little Prince." Antoine de Saint-Exupéry. I’m gonna fly to Lyon, France in a couple of weeks for the World Wide Web Conference and the airport is named after this author, Antoine. "The Little Prince," the reason I mention this is this…the author is actually this sort of romantic figure in France. He’s a philosopher, he’s a children’s author, he’s wrote a lot of other articles, well-thought-out books and he was a pilot in the…during World War II he was a sort of a spy pilot. It turns out he got lost at sea and they never found him. So he was this sort of romantic figure lost into the history. I think it was in the '80s or the '90s they finally found his plane off the Dover Coast in England like under water and that sort of like closed a very important chapter. But the reason I bring him up is because of this quote. "If you wanna build a great ship don’t get people together, tell them go get wood, don’t assign them tasks and tell them to do this and do that. What you do is you teach them to long for the endless immensity of the sea." You say, "Wouldn’t it be great if we had a thing?" Right? Don’t say, "Go get stuff." Say, "What do you think we need in order to have that thing?" This is the idea of leadership is about giving people a direction, giving people a goal, not telling them what to do. The reason I mention this is this quote is in Netflix’s leadership deck. You can actually find their HR deck online. They still teach this quote. They say, "Your job as a leader is to carry water, is to help your team get somewhere. Not tell your team how to do things. We paid people big bucks." And in Netflix they pay them a lot of money. They’re smart people. They’ll figure it out. Tell them where they need to go. Tell them where the landmines are. Tell them where, you know, where the guardrails are but lead them on.

So that’s really innovation. That’s really a strategy for getting something done. Unleash the power inside your organization. So really this idea of the microservice way is harmonizing the speed and safety. Create a place where you can tune that and you can create services that match your needs. Figure out how to create APIs that actually make it easy to do the right thing, that give a chance, developers a chance to build something interesting without hurting themselves. And then give them that freedom to do that. Give them that chance to start working on things without screwing up the running organization, without tipping the scales, without drilling a hole in the bottom of the boat. Make it possible for them to be creative. And then eventually you can start to really ask yourself basically how long do you think it’s gonna take? Whatever it is you’re trying to do, how long is it gonna take? And the best answer I got on this was "one day." Because I can change one thing in one day. And then I can change another thing and I can change another thing and I can change another thing. The pattern we learn from these people who build small services, who build great APIs, who wanna unleash this innovation is they always just do it one small step at a time. They don’t have these grandiose things and make big announcements. They just start all of a sudden nipping at the heels of their major competitors. Suddenly they have a thing. It’s a small thing, and they can dismiss it and say, "Oh, it’s okay. It doesn’t have all these features." But pretty soon it has more features and more features and more features. And pretty soon it’s a contender.

So hopefully this gives you some ideas about the kinds of things we’ve learned from customers over the last couple of years and it can give you an idea of how you can start on your own journey. You may already be at different parts of this already and maybe this just kinda confirms some things you already know. But I wanna make sure that you get a chance to see the software that CA’s got today, listen to Kevin, and see some demonstrations. So I wanna just close right now. I wanna say thank you very much for the time we had. I will be here all day today so we’ll have a break after the next speaker and we can talk a little bit. And I just wanna say thanks. Thank you very much.