Why Kubernetes over bare metal infrastructure is optimal for cloud native applications

Kubernetes over bare metal infrastructure simplifies how service providers can build their telco cloud, and interest in it is growing. Optimized for cloud-native applications, it can unlock efficiency CAPEX and OPEX savings, and will be key to harnessing the full benefits of 5G Core.

The telecom industry is now moving on with this technology due to several benefits of using Kubernetes over bare metal infrastructure compared to virtualized infrastructure. But it is important to note that virtual network functions (VNFs) will be around for years to come, and they will continue to run in parallel on network functions virtualization infrastructure (NFVI).

Kubernetes over bare metal infrastructure: driven by the rapid introduction of 5G Standalone

With the introduction of 5G, telecom networks are evolving to incorporate the new standardized 5G Core based on the cloud-native Service Based Architecture (SBA) and New Radio. 5G Standalone is needed for service providers to address multiple vertical opportunities for industries and enterprises. Many new opportunities are ahead of us as a result, but are facing the following challenges:

• How to be more efficient and keep capital expenditure as well as operational costs low when data traffic keeps growing.
• How to deal with increasing network complexity. For example, when it comes to the number of layers and functions in the stacks of hardware, virtualization, applications and support systems. In addition, the number of potential combinations of products from various vendors within and between the layers in the cloud stack also have to be considered.

Making operations easier is key in enabling new 5G use cases, as they require additional network capabilities that increase complexity. To get the full benefits of cloud-native applications, service providers need an infrastructure optimized for it. Which is where Kubernetes over bare metal infrastructure comes in.

Figure 1. Drivers for Kubernetes over bare metal infrastructure

Kubernetes over virtualized infrastructure versus bare metal infrastructure

Deploying Kubernetes over virtualized infrastructure is complicated compared to Kubernetes over bare metal infrastructure. This is largely due to additional infrastructure overheads such as the host operating system, hypervisor, guest operating system, SDN controller and more.

5G applications follow cloud-native design principles and are deployed in containers. But deploying containers in virtual machines, as is the case for virtualized infrastructure, adds additional complexity to the virtualized infrastructure. The number of dependencies between various software components is high and thus drives the need for optimization.

With a Kubernetes over bare metal infrastructure, without the virtualization layer, service providers get a simplified architecture common for central, edge and private network deployments providing substantial CAPEX and OPEX benefits.

Figure 2. Kubernetes over virtualized infrastructure versus bare metal infrastructure

Today it is possible to do away fully with hypervisors since Kubernetes has built-in options to manage and schedule workload placements and resources. There are no requirements to run different operating systems on top of a server since the micro services share the underlying infrastructure.

Figure 3: Benefits of Kubernetes on bare metal infrastructure

The interest in Kubernetes over bare metal infrastructure is accelerating in the telecom industry. By the end of the first quarter of 2022, Ericsson has more than 20 service provider customers including Post Luxembourg, SKT, Far EasTone, STC, and Masmovil for its solution, Ericsson Cloud Native Infrastructure. A North American tier 1 service provider has already taken our solution into commercial operation. From customers, we hear that they appreciate our solution being optimized for 5G cloud native applications by providing a high degree of automation and efficient operations.

Not everybody is as positive about Kubernetes over bare metal infrastructure as our customers. In this article in Fierce Wireless for example, several arguments are made about the technology’s shortcomings compared to traditional virtualization. Let’s go through some of the statements and see if we can address some misconceptions.

“The concepts of bare metal and cloud are mutually exclusive.”

In a simplistic sense – yes they are. Virtualization has touted software-based agility to build on cloud principles. However, what this statement fails to address is that the bare metal cloud and Kubernetes paradigm don’t wind the clock back to a hardware-based appliance mode. Rather, they replace the software agility provided by the hypervisor with another approach using Kubernetes. It is not the case that we will lose cloud agility by introducing a Kubernetes over bare metal-based approach to cloud native applications.

“Almost all the benefits we expect from cloud flow from the ability to abstract away the complexities of the underlying hardware infrastructure. Without a hypervisor, you can’t do that.”

This would be true in absence of an alternative to replace the hypervisor. The example compares a bare metal appliance-type deployment to a hypervisor-based deployment. But the industry has moved on and so has the maturity of Kubernetes. While often strongly regarded as a container orchestration platform, it does a great job in providing a universal abstraction as well. That is why developers can write applications without having to worry about what the underlying hardware looks like. Learn more about that subject by reading the post Kubernetes The Universal Abstraction and by watching this webinar.

“Worse, when you deploy on bare metal, your workloads become more dependent on the specific hardware they’re running on, making them harder to move or change or automate.”

One of the main justifications of the hypervisor is that if it is removed, applications will become locked into the hardware. But this is not true. The modern design of Kubernetes and containers as a service distribution gives a similar hardware abstraction and safety net as hypervisor-based designs do. More than that, proprietary hypervisors have made them lock-in risk for applications, where they are forced to always use components that are built on the hypervisors – such as proprietary load balancers and virtual networks.

In addition, as 5G is being rolled out, there is an increasing need for supporting specialized hardware resources like graphics processing units (GPU) and network acceleration technologies. With bare metal technology, introducing these specialized hardware resources can be done smoother due to fewer dependencies.

“Vendors pushing bare metal point to a multitude of benefits of using a physical server approach.”

Bare metal doesn’t mean managing physical servers directly without any abstraction. An important factor here is the software defined data center (SDDC) or software defined infrastructure (SDI). Today's hardware cannot be compared to standalone bare metal servers from the past. Even if we assume that Kubernetes doesn’t provide hardware management capabilities to match the hypervisor, today's SDDC solutions allow for software definition of hardware resources. Many tasks that we have relied on the hypervisor to do in the past are today managed by SDDC solutions. Examples include tenancy, automating scaling and provisioning and programmability of the hardware layer. 

“Some vendors argue eliminating hypervisors boosts performance. While older hypervisors did sometimes diminish performance, modern virtualization platforms don’t.”

Eliminating the virtualization layer improves the performance of the applications by giving direct access to a server’s processing power, and enhances overall throughput as data packets are processed much faster. The performance boost is primarily driven by the removal of the hypervisor and guest operating system that come along with the virtualization layer.

“When vendors say bare metal deployments cost less, they mean your initial capital outlay can be lower since you’re not buying virtualization software. Take note that they never mention OPEX.”

A Kubernetes over bare metal infrastructure deployment is much easier to manage and operate, largely due to a simplified stack. An example is software upgrades and maintenance of the virtualization software is no longer required, which will have a positive impact on OPEX too. Network operations are also streamlined with fewer teams – (a dedicated virtualization operations team is no longer needed ) having to manage the infrastructure.

“Running Kubernetes on bare metal does nothing to alleviate server sprawl.”

This is only the case if the abstraction and management capabilities provided by the hypervisor are not replaced by either Kubernetes or SDDC. If anything, modern approaches like SDDC are very efficient at optimizing resource usage in conjunction with Kubernetes infrastructure management APIs. Allowing a cluster to directly drive the underlying infrastructure layer using APIs is an efficient approach for adapting infrastructure to workload needs. 

“Arguably, virtualization increases security, since it provides one more layer for multi-layer defenses, including VM-level micro-segmentation, firewalling, and access control.”

This is a fair point, and it is well acknowledged that adding one more layer of software-based security in the hypervisor enhances the security posture. This is made more relevant due to the lack of proper multi-tenancy in Kubernetes. However, the cost benefit of leaving the hypervisor in there just for the sake of security must be taken into account. There are mature approaches available for Kubernetes security including network and service meshes, endpoint detection and response (EDR) and micro-segmentation. Furthermore, multi tenancy can also be achieved to great extent by isolating the workloads using Kubernetes inbuilt functions like node pools and node groups within the same Kubernetes cluster.  

“Bottom line, bare metal leaves you with an inefficient, inflexible foundation for your business.”

Quite the opposite: Kubernetes over bare metal infrastructure is the most efficient way to deploy at the edge, especially RAN workloads. All of the use cases cited by the article including adding new vendors, RAN hosted edge applications, expansion of capacity and provisioning of dedicated servers are facilitated by a bare metal Kubernetes approach, with SDDC adding additional value. This approach is even more relevant at the edge where every vCPU counts. While building 5G-based core or RAN, it does not make sense to continue pushing older approaches with the hypervisor when there are more efficient alternatives available.

There are many reasons to use Kubernetes over bare metal infrastructure in the long run as a platform for cloud-native applications, but virtualized infrastructure will also continue to play an important role. Read this customer case about Rogers in Canada for an example of why. The service provider is using NFVI in the initial phase for its 5G SA deployment, and over time will evolve to also use Kubernetes over bare metal cloud infrastructure, based on Ericsson Cloud Native Infrastructure.

Learn more

• Find out more about bare metal cloud infrastructure and its importance in this Q&A with Ericsson's experts
  
  
• Learn more about the benefits of Kubernetes on bare metal cloud infrastructure here
  
  
• Read our guide to building a cloud-native infrastructure, including the role of bare metal, here 
  
  
• Read case study from, SK Telecom -  the world first live bare-metal cloud-native 5G Core solution

Explore 5G