Part 0: Overview and hardware
In this blog series I want to take you on the adventures I took in order to build my own nested NSX & Tanzu homelab. This undertaking had some problems as can be expected from a nested configuration so I want to document my steps and maybe it can even help someone who wants to undertake a similar undertaking.
1. Premiss
I already had some homelab components set up, mostly in a small and energy efficient manner. This is due to my living arangements, I live in an apartement without the possibility to run a full rack configuration in the basement or a dedicated server room. Therefore my homelab lives in my living room and as you can imagine neither I, nor my partner, would be happy with a big and noisy server rack living there.
So I had to control my urge on buying second hand enterprise equipment because, even if you can get good deals with a lot of compute performance available for a low price, these tend to be on the loud side and also not very energy efficient. I settled on two HPE Proliant MicroServers Gen10 with 64GB of RAM each, a NAS for shared storage and some network components. And while those are without a doubt some fine little servers and it worked great for my basic infrastructure lab needs (such as vSphere, AD and the sort) there was no way these would be powerful enough for any serious NSX workloads.
So it was time for something a bit bigger...
2. Hardware
Now I was at the same problem again as previously. What can I get, which for one doesn't cost me more than my car but also has enough resources to get my NSX and Tanzu wishes going. As I said before, old enterprise servers were out of the questions due to heat and noise. Quiet low power servers would have been an option, but again, they are not cheap and I would need a cluster of these to get enough ressources to run everything I wanted (looking at the SuperMicro SuperServer E300-9D-8CN8TP, a great server surely, but for nearly 2000 CHF without any RAM per server not the most affordable option...).
So I began to think about a nested homelab. Sure, I wouldn't get any real high-avalibility and lose some performance due to the nature of nested hypervisors but in the end, for just a bit of labbing, does it really matter? In addition I could put those components in a standard desktop case or even a 4U server case and get some slow spinning fans and have a powerful and quiet server. And this is exactly what I'm doing. In the end, my haul looked like this:
Components:
- SuperMicro H11SSL-i SP3 Motherboard
- AMD Epyc 7551P 32 Core 64 Thread CPU
- 256GB RDIMM DDR4 2133 RAM
- Seasonic PX-550 550W ATX Powersupply
- Noctua NF-F12 120mm FAN Intake
- 2x Noctua NF-A8 80mm FAN Exhaust
- Noctua NH-U9 TR4-SP3 CPU Fan
- Intertech IPC 4U-4088-S 4U Server Case
- Various SSDs for Boot and Storage
And all this run me less than half the cost of just one of the SuperMicro E300 Servers mentioned above... Not a bad deal I would think to myself.
With the hardware settled I was ready to go on to the next step, installing all the required infrastructure components. In order to get a "close-to-real-life" feeling with this lab, I want to set this up in a 3-node configuration. So I would set up a vCenter, DNS, AD and all the other required infrastructure components themselves on the host directly and then install 3 nested ESXi VMs with 8 Cores and 64GB of RAM each. This gives me, at least in theory, a nice and even distribution of ressources between the leftover on the host itself and the nested ESXi instances. If it will work however, well, I will find out in the next part...