Serverless Is Here To Stay: Sorry, Corey Quinn

Transcript

Jeremy Daly:
One of my biggest complaints about serverless is that for the most part the costs scale linearly.

Rahul Subramaniam:
I think people make too much of the linear pricing problem. This is not a technical problem.

Jeremy Daly:
If you’re thinking about serverless as a full system, then you’re thinking about it wrong.

Rahul Subramaniam:
Serverless is a journey, not an end goal.

Hilary Doyle:
Okay, we’re on this journey together.
This is AWS Insiders, an original podcast by CloudFix about the services, patterns and future of cloud computing at AWS. CloudFix is a tool that finds and implements 100% safe, AWS recommended cost savings. That’s fixes not just analytics. I’m Hillary Doyle, and this is Rahul Subramaniam. Hey, Rahul.

Rahul Subramaniam:
Hey, Hillary. How are you doing today?

Hilary Doyle:
I’m genuinely excited about today’s discussion. We’re talking about serverless. Let’s set it up with the 101s. What is serverless and are there really no servers?

Rahul Subramaniam:
It’s not that there aren’t any servers, it’s just that we’ve gotten to a point in our sophistication of managing that infrastructure that you really never have to worry about it. The cloud providers do that for you.

Hilary Doyle:
A sophistication that is not reflected in my question, obviously.

Rahul Subramaniam:
You’re getting there, Hillary, you’re getting there.

Hilary Doyle:
Thanks, thanks. Day by day. It’s a real journey. We are going deep into serverless with Jeremy Daly, and we’ve got smart tips and hacks for optimizing your serverless setup. We’ve got an excellent use case. Some might call it a thirst trap, by which I mean it will actually make you thirsty. But first, we like to start with the news. So let’s hit some AWS headlines. Get us up to speed.Rahul, you know how this works. We bring the headlines, you bring the commentary. Let’s start with some security news. You can now use AWS identity and access management roles outside of your AWS workloads. The feature is called IAM Roles Anywhere. Rahul, how significant is this? And do we care?

Rahul Subramaniam:
To be honest, I really don’t know where or how this applies, given that it really doesn’t solve the problem of authorization. The nice thing about IAM in the context of AWS resources is that you not only define the roles there, but also the access policies all in the same place. Now, if your services are going to be elsewhere as the service suggests, then you still need to manage all of those access policies in the other system. I’m still scratching my head trying to figure this one out. I’m curious to learn more, and so I want to reach out to them and figure out how this all works.

Hilary Doyle:
That’s very open-minded of you. We’ll move on to some adoption news. We are getting a puppy. I’m just kidding. But if somebody wants to ship us a puppy, please use FedEx. They’ve announced they’ll be mainframe free in the next two years. They’re investing 2 billion into this transition, and they’re anticipating a 2 billion annual ROI.

Rahul Subramaniam:
Finally, finally. Companies have been trying to get out of mainframes for decades now, and a lot of them are actually being forced to do this because unfortunately, people are either retiring or there’s no one else left to know this technology. Looks like AWS’s Mainframe Modernization service is finally giving that momentum to this movement, and we can hopefully see some real results.

Hilary Doyle:
Okay, Delta, they’ve chosen AWS to be their main cloud provider. Flights are now going back to pre-pandemic chaos. Anyone who’s been in an airport knows this. Delta’s going to the cloud to deal with their customer service. What are your thoughts?

Rahul Subramaniam:
Good for the move on AWS. I have not flown Delta for the last 15 odd years, given that they’ve been the only airline to always lose my luggage.

Hilary Doyle:
No. Delta do better,

Rahul Subramaniam:
I hope, I really hope that moving to the cloud solves that problem as well.

Hilary Doyle:
Okay, Rahul, there’s a lot of love for serverless, and yet it’s also an unexpectedly contentious topic. Corey Quinn, ordinarily so docile in his opinion, has said that serverless sucks, and he’s not alone. There are a lot of haters out there. Can you explain to the lovers why there’s so much unease in the cloud about serverless?

Rahul Subramaniam:
It’s just usual for Corey to take an extreme stance on everything, and not just extreme one, a really snarky one at that. Now, he and the others say, serverless just doesn’t work yet. To set the record straight, Corey’s wrong. Let me give you an analogy. It’s like saying airplanes and modern transportation sucks because we haven’t achieved or invented teleportation yet. Now, we differ in what we call serverless and how we approach it. In my view, 100% serverless isn’t going anywhere.
The promise of serverless is that it will eventually allow you to focus on just the application code and everything else will get magically taken care of for you. And we’ve been on that journey over the last 15 years, and there’ve been such amazing services developed, and we’ve come such a long way that I just don’t understand the argument that Corey’s making. Can you imagine a state where you’re going back to having half a dozen DBAs managing your databases? I, for one, never want to go back to that life again. Or just managing your enterprise service buses and having big teams of people just keeping it all up and running. In my world, that is history.

Hilary Doyle:
When you talk about this magic, there’s a belief in it, and developers are making the change to serverless. But the question is whether serverless can actually deliver on its promise of magic yet.

Rahul Subramaniam:
Let me word this way. When you’re thinking about shipping an application, the image that comes to any developer’s mind is, “Hey, I’m going to write code. I’m going to build it. I’m going to test it, then I’m going to ship it.” It just sounds simple. But the reality is that this is only about 10 to 20% of what it takes to deliver software that is scalable, reliable, and cost-effective. 80% of the effort is actually spent in understanding and architecting and managing all the underlying infrastructure.
For me, serverless is a journey. My journey started when I didn’t need one of my colleagues to drive into a data center in the middle of literally nowhere to provision a server on one of our [inaudible 00:06:57]. And instead, we could literally just make an API call and set up some compute and memory that got us started. Now, since then, it has evolved into a number of amazing services at AWS. You’ve got a S3, we’ve got Lambda, we’ve got Aurora Serverless. In fact, there are multiple flavors of servers with Aurora. And a whole lot of other managed services that are just remarkable.
Now, the promise of serverless is an amazing North Star to have, and anyone who thinks of serverless as a destination is really missing the point. And yeah, I think Corey’s missing the point here. Serverless isn’t going anywhere.

Hilary Doyle:
All right. You say that serverless is a journey. If I’d known, I would’ve worn better shoes. But for everyone else, grab your Fitbits. We are on this journey together.
Whether you like it or not, serverless is everywhere. But there are some use cases that really stand out. If all use cases were vying for an Oscar, you would definitely see this one nominated mostly because it’s about movies and Coke, Coca-Cola to be clear, this is a family show. I’m going to take you back to 2009. Coca-Cola unveils the Freestyle touchscreen microdispensing fountain.

Rahul Subramaniam:
That’s a mouthful.

Hilary Doyle:
I know. Not as good as the mouthful that comes from 165 standard drinking options plus custom flavors. The exterior designed by beloved Italian design firm, Pininfarina integrated microdosing technology that had previously been used to dispense pharmaceuticals. This was the future that became the present that landed in movie theaters across the nation. I’m going to tone down the drama. The bottom line is movie theaters were looking for cost savings. The machine was a solution. Suddenly theaters could staff fewer people at the counter. Moviegoers were happy because listen, we’re all chasing the high of mixing Slurpees at the 7-eleven. But problem, when the pandemic hit there weren’t a whole lot of people who were keen to use a public touchscreen that is a lot of sticky kid hands. Coke wanted to put the touchscreen in your hand using AWS to facilitate a mobile experience. So Rahul, what was the problem here and how did they solve it?

Rahul Subramaniam:
The problem is avoid sticky hands and sticky feet. The problem that they really have technically speaking is that when you press that button on your phone, you want to send a message to the machine and have it dispense your drink. And there can be all kinds of combinations of it. You don’t want there to be a lot of time delay between when you press the button and the drink starts showing up out of the spout. So there is that latency issue that you need to address. Now, imagine a million movie goers at the same time, pressing the buttons and having to process all of that at the same time and send it all to the machines and make sure the machines operate perfectly every single time.

Hilary Doyle:
And this was a challenge that was particularly suited to serverless, right?

Rahul Subramaniam:
Hilary, everything is particularly suited to serverless. Unless you are-

Hilary Doyle:
That’s what I’m talking about.

Rahul Subramaniam:
Unless you really, really love managing insanely complex systems, and I, for one, don’t. Yeah, everything is suited to serverless.

Hilary Doyle:
I don’t need my Coke to be complicated. We’re going to leave you with this cliffhanger, and we will come back to Coke at the end of the show because well, that feels more responsible.

Rahul Subramaniam:
Did you say family show?

Hilary Doyle:
Was until now. Anyway, Rahul, you’ll take us under the hood to show us how Coca-Cola actually addressed this alleged latency issue. But first, when it comes to serverless, this guy is literally singing its praises. We are talking to Jeremy Daly. Let’s get to it.
Okay, Jeremy Daly, welcome to AWS Insiders. How are you?

Jeremy Daly:
I’m doing well. Thank you for having me. Excited to be here.

Hilary Doyle:
We’re delighted to have Jeremy with us. Jeremy is the co-host of the Serverless Chats podcast. He is an AWS serverless hero. He’s the author of the weekly serverless newsletter Off-by-none. He blogs at jeremydaley.com. And finally, he is the first hip hop celebrity we’ve had on the show. In 2020, Jeremy released LAMBDA, a serverless musical narrowly overlooked by the Tony’s, I might add.

Jeremy Daly:
(Singing)

Hilary Doyle:
We’ve been waiting on a sequel.

Jeremy Daly:
My family was big into Hamilton, and they just loved the musical and singing it constantly. I’m like, “What if I change the lyrics to this thing?” Werner Vogels loved it, and actually Andy Jassy commented on it as well. So it was pretty cool.
(Singing)

Hilary Doyle:
A generation of kids will learn how to code if you expand this musical, and the Grammys will come in their own time. Jeremy, it’s interesting. Werner Vogels obviously famously announced that with serverless, the only code you’d have to write was business logic. What can you tell us about the stage that we’re at in the serverless evolution? Are these the awkward teenage years, or are we on our way to adulthood?

Jeremy Daly:
You can’t build an application in the cloud anymore without using a “Serverless service” in some way. But I do think that that promise of just write business logic was very much so premature in terms of at least I think the intent was there. But one of the things that you do with serverless applications, really with any applications you write in the cloud now, is you’re using a lot of infrastructure as code. So whether that’s CDK or whether that’s serverless framework or whether that’s the architect framework or whatever you’re using, you are writing in many cases specific control plane instructions for the services that you need to use for AWS. So serverless is more than just writing business logic. It’s also about assembling all of the different services and wiring those all together in order for those to work as a single application.

Rahul Subramaniam:
But Jeremy, with every layer of abstraction that you add to any kind of operation, you will lose some level of control.

Jeremy Daly:
Yeah, in most cases, even if you think you have control, you don’t. So I mean, even if you’re on an EZ2 instance, [inaudible 00:13:33] can control everything. No, you can’t. I think that this idea of control, the trade-off for it in most cases is confusion and suboptimal outcomes. Whereas if they basically optimized it for you and then there was like, “Oh, I need the keys. I really want to change my own oil for some reason.” If that’s the case, then fine, give people that ability. But 99% of people using these services, they don’t need to optimize them.

Hilary Doyle:
So you’re telling us control is an illusion? That’s giving me a hive, but really you’re suggesting the trade-off here is control versus efficiency and simplicity, right?

Rahul Subramaniam:
Yeah. I think to that point, literally enterprises don’t realize today that there’s a kid in a dorm room who could be building an application that could challenge them for a few dollars where their enterprises couldn’t move because of the way they think about it. Hilary and I were having a conversation the other day where she was talking about she has all this content that she’s creating. She wanted all to be searchable and stuff, and engineers trying to figure out what to do. I was like, “Yeah, you should just use Kendra to do all of that. Take all of your stuff. Don’t worry about elastic search. Don’t worry about Lambdas trying to figure out, don’t worry about the API Gateway. Don’t worry about all these things that [inaudible 00:14:50] stitched together. Just take all your documents, load them all in Kendra, you get Google-like search out of the box.”

Hilary Doyle:
I look to Rahul for all of my life decisions, by the way.

Rahul Subramaniam:
Jeremy, does serverless change the way you think about application development at all?

Jeremy Daly:
Yeah, I think it definitely does. If you’re thinking about serverless as a full system, then you’re thinking about it wrong. We’re no longer creating these single monolithic stacks that maybe connect to a database, maybe it uses a queue things like this. This is separate systems that are all running and coordinated together.

Hilary Doyle:
So there is a general perception though that serverless is less optimal compared to the purpose-built optimized stacks of the past.

Rahul Subramaniam:
It’s a common conversation across enterprise customers that you’ll see is optimization of the database, the number of people involved in fine-tuning and refining and getting the last bit of juice out of their hardware and optimizing it is in their view totally worth it. Because if you just look at the server cost, yes, it’s capital expenditure, it’s predictable, it’s cost-effective because they’ve optimized the resource utilization of that to a large degree. And why then does it make sense for them to move to something that loses control?

Jeremy Daly:
I think that the idea of serverless being less optimal, it certainly depends on which part of the application stack you’re talking about. I think from a cost standpoint, one of my biggest complaints about serverless is that from the most part, the cost scale linearly, the more services you use, it just keeps getting more and more expensive as opposed to bending the cost over time. I think that specifically with databases, it’s really hard because let’s say that workload that runs … or something like that, if you move that over to serverless clusters right now, the cost would probably quadruple, more than that.

Hilary Doyle:
It sounds as if cost is the defining issue for enterprise companies considering a shift to serverless. Rahul, how should we be thinking about this linear cost issue?

Rahul Subramaniam:
I think people make too much of the linear pricing problem. This is not a technical problem. The linear nature of public pricing that is published. It’s public pricing, not private. That just seems exorbitant. The reality on the ground, however, is that a AWS already has volume discount tier that are baked in, and that takes care of some of the problem. And moreover, if there is a customer with an insanely large workload and pricing really becomes an issue, all you need to do is approach them and they accommodate private enterprise pricing, and it doesn’t take a technical solution to do this.The big issue really is, and I don’t like the current conversation around serverless being… It’s okay to not move on to serverless in cases where you might have to scale because in some ways it feels like a cop out. The bottom line really is that serverless is really cheap, and that’s been the big problem of all three cloud providers, primarily AWS driving it, which is they get… This message caters beautifully to all the new startups that have completely new workloads, but 99% of the existing workloads are still on-premise with large scale enterprises. And it gives them an easy way to make an excuse to say, “Hey, I already have the scale.”
I think we need to get a little more aggressive with the messaging around serverless. You have to rethink the total cost of ownership. You have to start putting all the code you own and all that customization that you’ve done, put that in the liability column, start treating it like a PNL. When you do that math, you then start pushing people over to the cloud because with that math, this stuff starts seeming super cheap.1

Hilary Doyle:
I like that you position code as a liability, but I also imagine that that’s a pretty hard pill to swallow for a lot of enterprise companies. And that takes me back to the FedEx headline at the beginning of the show. It’s a pretty clear example of this kind of cost reframing that you’re recommending. They’re shutting down the final 20% of their mainframes, and this initiative to go serverless will take an investment of $2 billion, but they project the investment will also return 2 billion a year in efficiencies every year. That’s compelling math and other companies are going to pay attention to these kinds of moves. Jeremy, predict the short to medium term future of serverless for us when it comes to enterprise.1

Jeremy Daly:
I think that what you’re going to see over the course of the next several years is certainly going to be this strange consolidation of, and a rethinking of how we’re building apps in the cloud. I think if you look at something like Kubernetes, you’re seeing now that even Kubernetes, that something like 75 to 80% of Kubernetes clusters are now on managed services like GKE or EKS or something like that. But essentially this is what’s going to happen, is people are going to want less and less responsibility over the management of these complex systems because they are too complex. Like an AWS, like a Google, they’re all going to try to build their own vertical stacks. I think it’s going to be confusing, it’s going to be hard to use, and we’re going to go through another cycle of like, “We’ve got to figure out a better way.” And so I think until that is fleshed out that we’re going to have a few more years of people doing things the wrong way. The good news is that as we reduce more of this and as we take more of this stuff off our books, fewer liabilities, it’s going to be easier to switch from DynamoDB to MongoDB to some other DB. So I would say continue to go down the road of using these other services, even if you pick the wrong ones, or even if you don’t quite wire it and it isn’t quite optimal for you now, there’ll be better options in the future, things will get better and it’ll just take a little bit of time to shake out.
Maybe not the rosy picture that everybody wants, but I think that eventually we’re going to get to a point where it’s going to be very easy to build applications in a universal way across what I and others call the cloud computer. Because essentially that’s what we’ve done. We’ve built a new giant massive distributed computer around the world with all these different data centers and it needs an OS that we can all write against and not have to worry. I think about a lot of these primitives and a lot of the nuances that make it really tough to get it right.

Hilary Doyle:
Jeremy, thanks for making time to share your thoughts and insights. We have discussed the future of serverless, but we didn’t discuss the future of your music career. Personally, I’d like to see a collaboration between the two of you, something along the lines of a rock opera. Anyway, we’ll do this next year at the Grammys.

Jeremy Daly:
Sounds good.

Rahul Subramaniam:
I want some free passes.

Hilary Doyle:
I’m just in it for the popcorn and the dresses.
Oh my goodness, there’s a lot of data running through my head after that discussion. I’m going to send it up to the cloud and in the meantime, let’s talk tips and tricks. How can we optimize our own AWS products and suites? Rahul, talk to us about the design and architecture of these services behind the scenes. What does it take to build a serverless server, and how can we ensure that it is highly reliable, infinitely scalable, and obviously highly performant? The easy stuff?

Rahul Subramaniam:
That’s a really hard question to answer, but let me try to walk you through a very simple framework that I use. The first one is break down your product or solution to a set of very simple components. As little code as you possibly can. Offload as much of the management and operations to manage services as possible. Complexity isn’t additive. In fact, it compounds. I remember back in the day when we were so constrained on network latency that we would try to get the database to do as much as possible. There would be data in there, there’d be TL transformations. I would even go as far as separating databases into an OLTP store just for transactions or where I use something like DynamoDB or maybe an old app database, which is only meant for data warehousing and analytics. So there I might choose something like Redshift and Athena along with it, and if I wanted to do full tech search, I would use something like elastic search. That’s how you think about breaking down what one thing does and just keep it at that.
Second, for each of these little services. Spend the time to understand and analyze what could possibly go wrong. In the early days of Lambda, I remember we built this microservice-based app that was supposed to be able to handle all the [inaudible 00:23:47] that we could throw at it, but as soon as we deployed a second Lambda-based application in the same AWS account, we started hitting what are called concurrency limits. It’s not something that we had even thought about during the design phase, so it taught us a valuable lesson in analyzing ways in which things can go wrong. And that’s a lesson that I’d like to pass on.

Hilary Doyle:
Could we get back to something that came up with Jeremy? We talked about enterprise and enterprise has often spent millions on their stacks. How can you ensure a seamless migration for the old guard when it comes to serverless? Do you have any tips for them?

Rahul Subramaniam:
I really wish there was one simple magic trick, but I will say this.

Hilary Doyle:
You promised magic Rahul, you promised.

Rahul Subramaniam:
I said that’s a promise, I’m not sure I can fulfill that.

Hilary Doyle:
Oh my God.

Rahul Subramaniam:
But here’s what I will say. Trying to transform your monolith into a cloud-native poster child, I would recommend that you find a seam in your application that you can replace with a managed AWS service. And if you have, for example, a BI or dashboarding capability in your app, see if you can replace that with S3 or Redshift, depending on what your latency requirements are, plus Athena plus QuickSight, that’ll probably solve your BI problem. That’s serverless for that solution. If you’re using caches, could you replace it with elastic cache? If you’re using a database that you run and manage on your own, could you move that to RDS Aurora? Start thinking about ways to find those seams and replace them with managed AWS services.

Hilary Doyle:
This is like eating an elephant, so you’re going piece by piece and that’s how to make a migration seamless.

Rahul Subramaniam:
I think that’s the perfect analogy or the perfect way to describe a lot of these old school enterprise monoliths. They are beasts larger than elephants. So yes, that is the strategy that would work for that.

Hilary Doyle:
Okay, we mentioned Corey Quinn earlier, thorn in the side of serverless. He says that “Devs know how to develop for WP Engine, they’re less likely to know how to develop for serverless.” How can we know the right way to write applications? What are your tips here?

Rahul Subramaniam:
How can we know the right way to develop an application? I think that’s the wrong question to ask.

Hilary Doyle:
Ouch.

Rahul Subramaniam:
This is the problem with serverless haters. They are focusing on the wrong things. In my opinion, the right question to ask is, do I own what I really need to own? And if the answer to that question is yes, then great, I think you’re on the right path. But the answer to that question is going to be different for different developers and different organizations. Let me just illustrate that with an example. I personally don’t care about building out a globally optimized CDN or a content delivery network. I love CloudFront and I would just use that all day long. Someone like Netflix or YouTube on the other hand, does care about CDNs and its performance in every region of the world to ensure the lowest possible latencies of video delivery. And so they own it. It makes sense for Netflix, but it doesn’t for 99% of the world, including me.

Hilary Doyle:
Where do you think businesses aren’t properly thinking about ownership? As a business owner I’m always trying to think about planning ahead, and how do we ensure we’re owning the right information? And that can be hard to figure out in the moment.

Rahul Subramaniam:
I don’t think businesses are asking themselves the hard question of, do I really need to own this part of the technology stack? In fact, one of the really neat exercises that I recommend everyone do is write it down. Write down why you need to own a particular piece of the infrastructure or the technology stack and share it with everyone in your organization. And then pick only the stuff that stands the test of a logical argument.

Hilary Doyle:
What does a logical argument sound like to you?

Rahul Subramaniam:
If someone presents a case for a better, cheaper, faster solution that uses a managed service elsewhere, go use it. And if not, consider building it. Really that simple.

Hilary Doyle:
We like simple syrup. We promised to get back to Coke at the end of the show. How’s that for a segue? The Pininfarina-designed, touchscreen-enabled movie theater and restaurant-based drinking fountains, AKA, one of our favorite serverless use cases. The challenge facing Coke was scaling UX and
avoiding latency. Rahul, can you walk us through what the build might have looked like? Would Coke have used Lambda and Kinesis for this?

Rahul Subramaniam:
Using Kinesis for their particular problem, I think it’s like using an ICBM to swat a fly.

Hilary Doyle:
Okay, this is the story of my life. It’s never the elegant solution with me. I bring the elephant.

Rahul Subramaniam:
Hilary, I don’t think go really had a big scale problem. This just seems fairly simple when it comes to AWS architectures. They probably just put together an API gateway with some Lambdas and maybe DynamoDB behind it to process all these requests. You can have tens of thousands of concurrent users operating right out of their mobile phones an ordering or placing orders for the drinks. It’s serverless in this case that brings the simplicity to the solution. And I think even though they were spending what, I think it was something like 13 grand on six servers earlier, I don’t think they were counting any of the resources that they needed to manage those six servers. And now with this new solution, they just don’t need those servers anymore and they don’t need any of those other resources managing them. I’d say that’s a win-win for both serverless and Coke.

Hilary Doyle:
Okay, great. So simplicity and savings are really what a serverless solution can enable. And when you go serverless, in this case, you can go both serverless and sugarless. I’m just going to step that back. That was a terrible joke. Basically the takeaway that we really want to Coke Zero in on here is that this little startup, thanks to these amazing savings may just pull through. Go get them, Coke.

Rahul Subramaniam:
How much does a glass of Coke cost at theaters these days?

Hilary Doyle:
Listen, I don’t know. I don’t drink that stuff. I like the interface. I’m a water girl, come on. So it sounds like at its best, that is the story of serverless. Once it is set up, deployments become relatively straightforward and the outcomes in terms of ease, sustainability, UX, and cost can really be game-changing. Is that a fair assessment?

Rahul Subramaniam:
So the serverless pattern to me is something that moves the generalist work to a more distributed model with specialists. In the medium to long term, I see it transforming what it means to build and ship applications. And in the process there will be tens if not hundreds of flavors of serverless each addressing a particular pain point in the process of shipping and delivering products and solutions.

Hilary Doyle:
That’s it for us for now. We’ll be back. You’ve been listening to AWS Insiders from CloudFix. I’m Hilary Doyle.

Rahul Subramaniam:
And I’m Rahul Subramaniam.

Hilary Doyle:
CloudFix is an AWS cost optimization too, you can learn more about them at cloudfix.com. Please check out the show notes.

Rahul Subramaniam:
And leave us a review.

Hilary Doyle:
Five starts.

Rahul Subramaniam:
And please follow us.

Hilary Doyle:
And reach out to us directly at podcast@cloudfix.com. Send us your feedback and let us know what you’d like to hear about on the show. We’ll catch you later.

Rahul Subramaniam:
Bye-bye.