What to Think About Before You Go Viral
Getting the balance right on being prepared to scale your project but not being overly prepared (i.e. wasting money on excessive hosting) can be tricky. Here are a few things to keep in mind…
Full Transcript Below:
Cody:
Your system is to automatically scale.
Cris:
How do you prepare your app, your website, whatever it is that you are building for that flood of traffic?
Cody:
What is a realistic estimate of what the maximum traffic threshold could be? Which is kind of funny because it’s like hanging for a botnet attack, but that’s more or less what it is.
Cris:
Today, we’re going to talk about scaling your applications. How do you prepare your app, your website, whatever it is that you are building, for that flood of traffic, that’s going to come in, because you’re going to be the next Facebook, the next Instagram?
Cody:
Well, we all hope.
Cris:
Right. How do you prepare your app or your website for that influx of traffic?
Cody:
So this is going to harken back to a previous episode we’ve done, actually. If you haven’t seen it, it’s the one on load versus stress testing.
Cris:
It’s right here,
Cody:
Somewhere there.
Cris:
Or here, it’s somewhere.
Cody:
It’s somewhere. And that episode talked about the differences between that and where they come into play. But in reality, if you want to actually make sure your application is going to scale to your traffic, you’re going to want to implement stress testing.
Cris:
Okay.
Cody:
And the idea, and I’ll just outline the differences here. Still go watch the episode. I’m going to give you a little preview of it. Is that load testing, tests your current site under it’s designed load, whereas stress testing tests it to its breaking point.
Cris:
Got it.
Cody:
And the point of that is to figure out, obviously, can this software handle traffic at the level I expect it to, and when does it actually break? So when you’re scaling an application, you need to know first off, what is the traffic scale we’re actually dealing with? And you have to be realistic and also optimistic because you obviously want to have it handle what the best case scenario is. Because, if you’re making a software product on the web, you probably want it to have a lot of traffic. So you want what is a realistic estimate of what the maximum traffic threshold could be. So, from there you want to find that number and then you need to simulate that traffic somehow.
Cody:
There’s a lot of different solutions out there for various different frameworks and platforms. It depends. If you’re using Python Django, there’s one for that. If you’re using Node.js, there’s something for that. There’s packages all over the place for this kind of thing. And there’s even services you can pay for it too, which is kind of funny because it’s like paying for a botnet attack, but that’s more or less what it is.
Cris:
We have horizontal scaling, vertical scaling.
Cody:
Yes.
Cris:
Let’s talk about that. What does that mean? What are the differences?
Cody:
Okay. So vertical scaling is actually what most people do whenever they detect that their servers or their systems are not adequate for their current amount of load.
Cris:
When someone’s saying, “We need to scale.” Yeah. They’re probably talking potentially, vertical scaling.
Cody:
Yeah, because it’s the easiest one to do. Basically, for most people, vertical scaling means go into your host panel and select the next tier up, because it’s making the server you have your systems in bigger, in the sense of more hardware, more RAM, more CPU, more storage, whatever it may be. It’s scaling that one system. Whereas a horizontal scaling is a little more complex because that involves splitting the load horizontally across multiple servers.
Cris:
Oh, okay.
Cody:
And sometimes that can actually be a lot cheaper because depending on your host and what you are actually processing and doing and what the load is. So a lot of times, many small services are cheaper than one big huge one. So it depends on the situation. But the complexity that comes in with horizontal scaling is now you have all these servers processing all your requests and you need to make sure that they all inter-operate correctly. This is a good example, actually, on YouTube, they horizontally scale the heck out of that platform.
Cris:
Sure. Yeah.
Cody:
It’s huge. Gigantic. There’s probably entire warehouses full of servers and a by-product of that is that whenever you have something that’s a calculated result, you have to deal with something called eventual consistency, and eventual consistency, it sounds somewhat complex, but basically it means eventually this number will be correct.
Cody:
We don’t know it, but it will be at some point and a good example of what an eventual consistency number would be, would be like the YouTube views count. You’ll notice sometimes if you look at a video, sometimes that number will go up. Sometimes it’ll go down a little bit. Sometimes it’ll skyrocket up because it hadn’t updated in a while. And that’s because they’re basically tabulating the result from all of their different clusters. And in this case it would be not only horizontally scaled backend software, but also horizontally scaled database software too. And at a certain point, all those databases have to synchronize. When that is, eventually.
Cris:
Eventually.
Cody:
And the point is to just keep it accurate and approximate. But at a certain point, you just assume that it’s probably never going to be perfectly accurate. And that’s yet another complicated scenario within horizontal scaling is you have to deal with that. Especially as you grow larger and larger, you have to figure out how those whole systems will inter-operate and have the correct data, even though, a request could be coming from server 233, and then all of a sudden the user, just all of a sudden gets switched over to another one. They just processed a result in one thing, takes longer than the other and you have a race condition. How does it all actually fit together as a Tetris puzzle?
Cris:
You pause your video, go grab a drink out of the fridge, come back. Is that still pulling from the same place? Did that just move over somewhere else?
Cody:
We have no idea. And the answer is maybe, maybe not.
Cris:
Yeah. Interesting.
Cody:
And there’s many strategies because sometimes, with horizontal scaling, another layer of the complexity is deciding that fact. Will users just get switched around based on current load? How will that impact their experience? Are they stuck in one server whenever they started their first session with your website? Because you just want to keep it consistent. These are all decisions you have to make when horizontal scaling comes into play. And that’s why most people choose vertical scaling first, because it’s a lot simpler.
Cris:
So what is microservice architecture? What is it, why is it popular? How does it allow you to scale? All that sort of stuff.
Cody:
So microservice architecture is interesting because it is a bit of a buzzword right now, and it has been for a few years. But it is effective at doing certain things, and I’ll get into that. But effectively what a microservice is, is a tiny service, basically a tiny service. I know. Who would’ve thought right?
Cris:
Who would’ve thought of that naming convention?
Cody:
But that’s the idea is to split up loads of work that would normally be in a giant monolithic backend, and move it away from being all in the same piece of software.
Cris:
Gotcha.
Cody:
So an example could be you have a server that needs to accept file uploads. These file uploads can be big. And if you accept them all into one big monolithic stack of servers, that means that whenever it is uploading that giant file and processing it, it bogs down the whole server for everyone. This can be solved through numerous different ways, including just threading it out on the backend, including horizontal scaling, or, which in a way is a form of horizontal scaling, microservice architecture, where there’s one dedicated server, or even a group of dedicated servers, that handles those big file uploads. So the main backend-
Cris:
Go process.
Cody:
Yeah. Go process and do stuff in the background. Good example could be Django Celery, which isn’t really a microservice, but also it totally is at the same time, because it has workers that are asynchronous and can be on different locations.
Cody:
The only way it’s also not a microservice, is because it’s still highly integrated with the current project, but that’s neither here nor there.
Cody:
Node is a good example of another platform where they use microservice architecture all the time. It’s very common for microservices to be part of a Node API. And you’ll have several different little projects that do long running tasks in the background that just get dispatched as people, again, upload big files or something like that, so that the big backend never gets bogged down. That’s the whole point.
Cris:
It’s almost like a boardroom meeting where you have a bunch of people around a table and you throw a question out to the room and then everyone just sits and gets bogged down and waits for that answer. Instead of being like, “Go take a note on this, come back and give me the answer when you’re ready.” In a way. Interesting.
Cody:
Yeah. That’s why we have, for instance, the different executive titles, there’s CEO, COO, CFO, CTO. Having a non-microservice architecture is a lot like everyone just having a C title, they’re all chiefs. No one has a specific role, where they all just handle it. And you can imagine how complex that could get in terms of role separation. And that’s the point, is that it’s separating out different responsibilities for different areas of code that are entirely isolated, but dependent on each other at the same time.
Cris:
Very cool. I think we’ve covered a lot of ground on this. Any other final closing thoughts while we’re talking, scaling your applications?
Cody:
Honestly, I think scaling is definitely something to be considerate of as you’re going through the development process. It’s so hard to get it right as to choose when to scale, because a lot of the time it’s both in anticipation of traffic, which hopefully may come, but who knows if it will and you might end up wasting money on hosting or something of that sort or not doing it at all. And then the traffic does come and your entire site is down. It’s so hard to get that right. And there’s a lot of practice and principles on how to actually scale at the right moment. And I think at least from my perspective, the best way to just prepare for that scenario is to have a managed hosting solution so that you don’t have to worry about when to scale and just have it set up in the beginning.
Cody:
But again, as we talked about, it depends on your business use case. Some things don’t have to scale. So anyway, say with that, be prepared to scale and if you don’t want to prepare to scale, at least set up automatic scaling, so you don’t have to worry about it. Yeah.
Alexandra:
Thank you for joining us for this episode of Bixly Tech Tuesday where Cris and Cody talked about the ramifications of planning your app appropriately, for being able to scale it down the line. What happens when you have thousands, maybe even millions of people accessing your website or downloading your app? I hope you learned a lot and don’t forget to check out the links that we have in our description box, down below. You can find a link to our free custom software guide, which will walk you through the process of planning out your own app idea. And you can check out our website, bixly.com and you’ll see at the top, we have a Validate My Idea button, which means that you get a free 60 minute meeting with Cris to talk about your next app idea. I hope you enjoyed this episode of Bixly Tech Tuesday.
Originally published at https://blog.bixly.com on September 28, 2021.
