Running ESXi in Kubernetes
I have been running VMware ESXi for Datacenters for more than a decade to provide services to production and Dev/Test workloads even before the raise of Devops and SRE mindshift.
This article is to explain, how we ran the ESXi with principles of Kubernetes to get 100% uptime, rolling upgrades, roolbacks, treating the ESXi as stateless.
We have used below components to acheive it:
Cisco FI as ESXi orchestrator or Kubernetes for ESXi
Cisco UCS Blades as Compute Nodes, i.e., ESXi servers
The configmap or configuration as a boot LUN on SAN
What we achieved with this mode of operations:
ESXi has been upgraded 1000 times( the number will be a bit higher) with no VM downtime
ESXi has been treated as cattle, never needs a care ( If not working terminate it and through the profile and get a new clone and replace the ESXi configuration )
Lean Team, managing the ESXi is possible with SRE ( haha- VMware admins )
Server replacement as Kubernetes drain of node and promote another server as compute node with attaching new profile to Server.