The Power of Just-In-Time Infrastructure

December 6, 2022December 4, 2022 Lee Atchison application resource management, capacity planning, cloud infrastructure, cloud-native applications

by Lee Atchison

Just-in-time (JIT) manufacturing is a production model that creates manufactured items to meet demand as demand dictates. This is opposed to traditional manufacturing, which makes products ahead of time and stores the product as inventory to be delivered as needed when sold.

With traditional manufacturing, if an insufficient amount of product is made in advance, then shortages occur. If too much product is made in advance, then waste is created. JIT manufacturing aims to reduce these problems. The advantage of JIT manufacturing is reduced waste and improved overall efficiency.

The same concept applies to operating the infrastructure required to run large-scale cloud-native applications. The public cloud—and related technologies such as managed containers in Kubernetes—allow you to dynamically change your application infrastructure to handle the specific, current needs of your cloud-native application, reducing waste and improving efficiency.

Just-in-time(JIT) infrastructure—the cloud computing equivalent of JIT manufacturing—solves many of the same problems for application resourcing as it does for the manufacturing equivalent.

Traditional Static Applications

A traditional static application works similarly to a conventional manufacturing plant.

The application normally has to allocate enough infrastructure resources to handle the variable application load applied to the site. Site capacity is adjusted in advance of the need to ensure sufficient capacity. This is similar to the in-advance production process of traditional manufacturing.

But if the forecasted load is incorrect, then the site load can outstrip capacity and an outage or brownout can occur. This is demonstrated in Figure 1.

Figure 1. Static resourcing causes capacity issues.

In manufacturing, excess product is produced to prevent shortages. In cloud computing, extra infrastructure is allocated to ensure you always have enough capacity to avoid outages in applications. In Figure 2, excess site capacity can provide enough capacity even when application traffic spikes occur.

Figure 2. Conservative resourcing resolves the problem, but at a cost.

Just like traditional manufacturing, poor predictions can result in either wasted capacity or shortages and an outage.

Precise planning is challenging to achieve, whether we’re talking about manufacturing or application resourcing.

Cloud-Native JIT Infrastructure

The cloud and cloud-native applications can employ a technique similar to just-in-time manufacturing to ensure neither problem occurs. Dynamic infrastructures, which are possible in nearly all cloud-native application scenarios, allow you to constantly adjust the resources allocated to an application to ensure they always have sufficient resources while minimizing wasted resources. In Figure 3, site capacity is continuously changed to meet current application demand.

Figure 3. Dynamic resourcing reduces the risk of failure while reducing waste.

This strategy is possible because resources can be allocated to an existing application quickly in a cloud-native environment. This means constantly changing resource requirements can be matched by equivalent constantly adjusted resource availability. This model reduces waste and reduces the likelihood of brownouts during unexpected traffic spikes.

Just like in manufacturing, how closely should you adjust site capacity to match the actual resource needed? If you adjust capacity in tight lock-step with current needs, you can miss some usage spikes, and brownouts can still occur. This is what happened in Figure 4 when the capacity adjustment algorithm missed a sudden usage spike, resulting in insufficient site capacity and a brownout.

The tighter you manage your resource allocations, the greater the chance that a sudden spike can create an outage.

Figure 4. Guessing wrong in JIT infrastructure still can be an issue.

But by adequately tuning resource requirements, it is possible to avoid—or at least reduce—spike-driven brownouts.

Algorithm Adjustments

The key to proper resource tuning is understanding traffic patterns. It’s hard to plan for sudden and unexpected spikes, but many traffic spikes are predictable. For instance, do you have predictable traffic cycles, such as heavy usage Monday mornings? What about the first of the month? Are you planning a marketing push on a given date that can generate a spike in traffic? What about a social media campaign?

Additionally, algorithm refinements and improvements are getting better, especially as tools such as machine learning and artificial intelligence are used to find patterns and predict when traffic changes may occur.

JIT infrastructure, just like its manufacturing counterpart, can reduce cloud costs, reduce waste and ensure sufficient capacity to meet variable demand. And improved algorithms to predict capacity can make the benefits of JIT even better.