There are many potential problems with scaling business software products. Over the past few years, we at rinf.tech have seen some common problems, causing unexpected and sometimes serious damage, although seeming pretty obvious and preventable in hindsight.
This article explores why and when to consider scaling your digital product and how product managers can help support and encourage software scalability within development teams.
In the early stages of software development, developers usually do not prioritize product scalability. First and foremost, they focus on the software’s ability to solve problems quickly. However, after validation and user testing, it is time for developers to shift their efforts to scale the software solution. This process is essential to secure its place in the market and further growth. The question is – How to do it effectively? The answer is – scaling.
Let’s start with a simple definition to ensure we’re on the same page.
Scalability is your ability to effectively increase or decrease resources to meet the current and/or future demands.
In software development, scalability is the product’s quality that provides the flexibility to manage the growing demands and requirements of any number of users. Simply put, it is the software’s ability to respond to changing needs whenever they arise, regardless of their size and scale.
Scalability is about doing more with less. Having the right tools and processes in place, you don’t need to worry about growth, virality, or load fluctuations ruining your product (or your company!)
Scalability is important in everything from your business to your tech stack and software development team. Here are some examples.
As a business, you can think of scalability as the ability to serve more users. If you sell hamburgers and French fries, can you produce enough of them if your customer base grows 5x overnight? How about 10x or 100x?
Will your app/tool/solution be able to handle the spike in traffic without crashing or slowing down? What happens if the number of concurrent requests to your database jumps from 1,000 to 10,000 in a matter of seconds?
This may seem like a severe risk. Building scalability means you’ve done something right, and more people want your offering or value proposition (be it hamburgers or online services).
Scalability is frequently seen as a technical issue. However, having the right processes within your team is just as important as having the right technology stack to manage growth.
That’s why it makes sense to break scalability down into performance and processes. Let’s dive into both.
What does your engineering team think about scalability?
As a software company, your tech team may think about growth differently than you do as an owner or manager.
If usage spikes, will your technology solution give up and freeze under load? If a viral piece of news brings you 30x the expected traffic, can you effectively handle it? Or will you be burning money just to keep your site running?
Growth does not mean the excitement of new users and revenue increase. It means technical uncertainty.
When dozens or hundreds of factors affect the speed and efficiency of your software, any one of them can confuse you. How you handle all these scenarios determines the scalability of your software.
There are many discussions about what tech tools, systems, and infrastructures are best for scalability. As a project/product manager or non-technical lead, you don’t need to know the specifics of each. However, you must understand how your tech team deals with scalability issues.
Software scalability means increasing your infrastructure’s computing power. There’re two ways of scaling a tech solution: vertical and horizontal scaling.
Vertical scaling (a.k.a. scale-up) is when you add more computing power (RAM, CPU) to your existing machines.
The main difference between the two is that the first requires breaking your code into smaller pieces that can be run simultaneously on multiple machines. While this can add redundancy and flexibility to your system, it also adds complexity.
On the other hand, vertical scaling keeps your code together but runs it through a higher spec server. It’s easier and cheaper, but it limits you by the power of one machine. If something goes wrong or you need to upgrade, you might be in big trouble.
There are several factors to consider before scaling up:
Vertical scaling can be useful if you know the amount of tasks the system needs to perform. Unfortunately, this is not always the case in a rapidly changing market. Therefore, there is another approach.
Horizontal scaling (a.k.a. scale out) is when you add more machines (or servers) to your resource pool.
Here’s what you need to know about scaling out:
Whatever approach you choose to build a scalable software product, you need to align it with your long-term scaling strategy and take into account market conditions and user expectations in advance.
Your software development team is the best resource for addressing the performance-related aspect of scalability. So, if you are not actively involved in this process, how can you help?
The solution to the scalability problem is to eliminate the uncertainty. But instead of doing it with tools or stack updates, you can focus on the human element of scalability: processes.
As you build a larger and more complex system, what does your team need to know?
How are they preparing to handle more data?
Where can they learn about system architecture and how to flag problems?
Do you have scripts for errors or increased load?
What can you automate to give your team more time to resolve critical issues?
With rinf.tech, you can create your own knowledge bases and wikis to document all of your team’s processes. Think of it as an “idea gathering hub” – a centralized, customizable place to store information.
You can use such a wiki as a guide if you run into scalability issues. This means that whenever there are typical problems — whether it’s a spike in traffic or API calls — your team knows where to find a process to address them.
Scalability issues are often related to people hiring, process scheduling, resource management, and disguised process issues.
Here are eight ways to solve these problems and make scalability part of your team’s DNA.
The benefits of product scaling are numerous, from reducing maintenance costs to ensuring that software can keep up with growing user needs. But that’s only part of the story.
Here are some more facts about the importance of scalability to product and business success.
Scalability is essential to market expansion. To attract more users in the long run, you need to make it better than the competition and anticipate user pains and needs. As these issues and requirements evolve, you will need to tune your product accordingly, introduce new features, and optimize performance.
Product scaling benefits include, but aren’t limited to:
No business owner wants to pay for idle resources. Scaling allows you to create an easily manageable architecture that can turn resources on and off on demand.
Software product scalability ensures the best user experience. Aligning product performance and functionality with user needs is a zero step to quickly attract customers. Continuous software improvement does even more: creates a positive user experience, sets the bar high for competitors, and instills confidence in customers.
This is a working strategy to manage bottlenecks and avoid cascading failures. The road to software hell is littered with bottlenecks that cause downtime, delays, and even system crashes. For example, overloading a database with requests for which it was not designed slows down or even disables other processes that depend on the database.
Result? Fewer bottlenecks.
So, is there a point when you need to focus on a scaling strategy? Yes, and it also has something to do with your business perspective.
There are four main aspects to consider when designing a scalable system.
More users require higher system performance, so increasing the capacity will make the software more powerful.
On the one hand, it will introduce new features. On the other hand, this may entail additional resources on the part of users (system performance, memory/battery usage, etc.). Balancing these issues should be part of your scalability plan.
The high performance of one function is often achieved at the expense of another. You need to prioritize between consistency, availability, and partition resilience. For example, would you like your users to see some updates right away, or the full set over time?
Scaling means using more resources and technologies that will require more maintenance. When scaling your software, you must prepare a new maintenance plan and budget so that you can address more potential issues.
Scaling your digital product can involve building from scratch or using off-the-shelf solutions that are easy to set up and maintain. In the first case, you will have more customization freedom at a higher cost, while in the second case, you will save money and time on several standardized solutions.
When you plan any marketing campaign, you expect an influx of customers, and you need your software to keep the new volume of visitors. For a scalable system, it will not be a problem to increase capacity depending on the new workload, while a non-scalable system will lead to a decrease in performance.
Resources work within their limits. If you add features that require more CPU, for example, the performance of dependents will be affected. This can cause delays, buffering, and various response failures.
And this is usually done by introducing new features and optimizing memory and data usage so that customers can enjoy fast performance. Their fast and successful integration is possible due to the flexible architecture of the system.
There are bottlenecks in the system. They indicate that the workload is unevenly distributed, or that some centers require more resources to process requests. In this case, scaling is needed to balance workload distribution and track downstream processes to avoid further bottlenecks.
There can be many reasons for poor performance: memory problems, an unresponsive database, poorly tuned or integrated technologies, a large load that the system can handle, and so on. Scaling your product will help spread the load evenly and point out architectural weaknesses that need to be addressed.
Since the data layer is at the heart of any operation, its propagation can speed up processes or cause temporary system unavailability. Scaling includes distributing and partitioning data, which makes it easier to organize and optimize data usage and storage.
When you see that it’s time to scale, you need to understand what can be scaled and how to prepare the software architecture for this process. This will be useful for evaluating the priorities and results of scaling.
Scalability isn’t always an issue that teams need to take care of. At least right away.
If you haven’t found a product/market fit yet, it’s probably more important to experiment with different features or offerings before planning how your app will go viral.
When you have a small pool of users, you can sometimes get away with doing things you plan to automate later. This allows you to run faster, and when you finally automate your processes outside of the loop, you will know exactly what to build because you will have the muscle memory to do it yourself.
While manual actions can help experiment with a prototype or MVP, they’re downright dangerous when you’re working with a mature product.
As a product manager, you should be looking for red flags that it’s time to think about scaling your software solution. Ask questions such as:
Have you accumulated too much technical debt?
Is complex code blocking simple updates?
Are you experiencing downtime or lag even with current usage?
Set up systems to track and flag scalability issues so you don’t add extra resources too soon or too late.
It can be difficult to purposefully build features that you know will break under pressure. However, keeping your team from building cool features because they don’t scale immediately is a bad idea.
Sometimes the “things that don’t scale” are manual processes. In other cases, they create technical debt—the “cost” of choosing a simple or limited option when a better choice would take longer and cost more.
Technical debt is a “necessary evil” when testing ideas or features and should not be avoided. However, just like taking out a large bank loan, you need to know how you are going to repay it.
Make sure you have systems to monitor scalability, keep track of tech debt or “things that don’t scale” in your PM tool, and review these as you plan your sprint. You don’t want to start building on a cracked foundation.
Knowing that you need to start addressing scalability issues is only half the battle. The rest contributes to your current backlog, roadmap, or code reviews, which is not always easy.
Scalability is not sexy. No one will praise you for doing it well, but you’ll face backlashes if you fail.
However, the scale exaggerates the problems. What seems like a minor annoyance when serving dozens of customers can ruin your entire product when you move on to hundreds or thousands.
That’s why it’s a good idea to consider scalability from the start. This does not mean postponing your product launch to “improve” your infrastructure. But instead, every sprint should include the question: does it scale?
If this is not the case and you do not have an easy solution, document what you are doing and why.
Later, when you schedule the “scalability” sprint, you will have a list of issues to work on.
Articulate the goals of a solution’s scalability project to your team, brainstorm, ask for feedback and celebrate success.
As a product manager, you should step in and discuss with your team ways to optimize performance and achieve high-availability. Focus on processes required for scaling while keeping an eye on your team effort and providing guidance. Always have a strong Team/Tech Lead to support you.
Here’re some of your focus areas during a software scaling project:
As your product scales, you will inevitably attract a more diverse group of users. Understanding their needs is how you will keep up the momentum. But if your software team still needs to provide detailed requirements, they will be overwhelmed with work.
User interviews are a powerful tool for understanding how people use and interact with your product. They should be done by as many experienced people on your team as possible.
User interviews are the formal process of interacting with your users to understand their problems, habits, skills, and desires. User interviews are more than just casual conversation; they are targeted, recorded for research and future use, and provide helpful information to IT teams.
Give your team autonomy to conduct research and better understand the user. Or at least give them access to recorded user interviews to help prioritize future sprints.
Don’t assume that your development team will know how to integrate your software systems in a way that’s perfect for your business operations without spending reasonable time understanding your technology needs. Map the standard business processes in the systems to be integrated and define exactly what the systems need to do for each of the processes – what data is sent / received and when, where data should be entered by users and where it should be pulled automatically, etc.
Your tech consultants or you may have to spend an extra day or two ahead of time to come up with a simple plan, but your team will likely spend even more testing and resolving the same issues later. Do this ahead of time to control the time spent investigating, save on rework costs for your developers, and get a clearer picture of the total amount of money you will need to spend on work before processing your payroll.
There is always a trade-off when it comes to scalability. If you’re a SaaS company sacrificing development speed to mitigate the risks of a potential burst of mainstream usage, you’re probably making a poor choice.
However, if you’re building a video game where uptime is more important than new features, this might be a compromise you’re willing to make.
Choose a path and stick to it.
Look for ways to automate your customer journey to make room for “critical” tasks.
Scalability is (usually) in the “important but not “urgent” section of your priority matrix. Unfortunately, there are often many “more urgent” tasks along the way. To give your team the space they need to focus on scalability, seek ways to automate the “urgent” parts of your product or customer journey.
Some common examples would be creating an account, messaging with a user, registering, reporting an error, or any other thing where users might need “urgent” technical assistance.
The knowledge base of your scalability processes is only good if it is up-to-date and readily available. The same applies to any other knowledge management effort.
Here are some tips to help your team get started with the documentation process:
Create a sense of psychological security when no one is afraid to ask questions or offer ideas. Sometimes it’s as simple as changing the language allowed in meetings to make it more inclusive or giving everyone the time and opportunity to contribute.
Use a simple knowledge management system (like wikis) that makes it easy for everyone to contribute and find the information they need.
While the final choice of scaling approach will greatly depend on the specific product types, existing architecture, and business goals, general advice should be kept in mind.
A single point of failure (SPOF) is the first thing to avoid when scaling, as the failure of one such component brings the entire system to a standstill. It is important to address such weaknesses to ensure the consistency and continuity of service delivery.
To achieve the latter, you must break down the architecture into smaller, manageable, and more independent parts. Fortunately, this approach is at the heart of software scaling.
Your product architecture pattern will dictate many scaling decisions. Here is a list of popular patterns based on operational logic.
Multilayer (n-tier): data is transferred from the innermost layer (database) to the uppermost layer (data task).
Event Driven: One agent distributes input to different modules to perform specific tasks.
Microservices: A set of services that run independently on different servers.
Microkernel: A pattern that is best suited for repetitive operations that are stored in the microkernel and executed repeatedly.
Space-based: A pattern in which the processing logic focuses on accessing segments of databases stored on different servers.
Because scalability has many facets, you need to decide how you’ll track and measure changes as a result of scaling. You can measure CPU and memory usage, network and disk input and output, latency, and more. Many application performance monitoring tools (APM tools such as Traceview, NewRelic, and others) will help you get this data.
Platform as a Service (PaaS). In this case, you get an evolving platform for creating software (including a framework and a runtime). This is useful in case you want to customize your product and plan to use the API.
Infrastructure as a service (IaaS). This computing infrastructure offers cloud servers, networks, operating systems, and storage. You can buy it on demand and as needed instead of spending money on hardware.
Mobile Backend as a Service (MBaaS). This is an option to manage the mobile backend and enable certain features without developing them. It is widely used to develop mobile applications based on website infrastructure.
The caching technique works great when it comes to preventing bottlenecks during expensive or repetitive database queries or page rendering. Switching the processing logic from database lookups to cache lookups will first buy database capacity, as it will not participate in processing every data request. In addition, cache lookups take less time, and speed is what all users like about the software.
If you want to create a scalable product, you must use technologies that support scalability approaches from scratch. This includes choosing popular libraries, suitable search algorithms, asynchronous codes that upstage non-critical tasks, and so on.
Offloading work to clients means sending fewer requests to servers. For example, instead of running sessions on the server side, you can use client cookies and a JSON Web Token (JWT) for authentication.
Software product scaling ensures growth and success, creating conditions for entering new markets and reaching new audiences. But this is not an easy task. With so many features and factors in mind, you need a skilled and reliable team of seasoned engineers to ensure consistency and prevent bottlenecks downstream. rinf.tech can help!
Working across various industries has given us valuable insight into user needs, requirements and expectations. We will be happy to use this knowledge and experience to help scale your software solution.
This article explores the benefits of FaaS for development, deployment and management of microservices applications in the Cloud.
This article shares tips and the best practices of software team scaling from business and technical perspectives.
This article will explore key challenges facing business and enterprise software development in 2022 and how they can be solved.
