Hello again,
I have used virtualization for a while now in the enterprise.
Mostly using Esxi and vSphere, using servers with 256 GB of memory and Terabytes of backend space (iSCSI, Fiber, NFS, etc.). Also, some KVM setups when Open Source was the only viable option due to pricing, etc.
At home, I have used either the free version of VM-workstation or VirtualBox.
I have not needed something more powerful. Just enough to test a new Linux distro for example, or run a KVM image.
However, I started to use GNS3 to run big labs at work. Then I switched to EVE-NG. This was not an issue of course since I had the virtual resources.
At home, it was a different story could not run big labs just the simplest ones.
Thus, I started to look to get some computer power. I could have got a server of e-bay for some decent price. However, they are noisy as hell.
About a couple of years ago I found a nice bargain, a gaming PC that was marketed to beginners, I was not going to use it as such what caught my eye is that to it had 4×2 CPU AMD cores.
The whole setup included 8 GPUs was about 300 bucks!
It only had 8 G of memory but I could get 2×16 G memory modules, which would give it 32 G of memory, and it could become my virtual server and also file server since I had about two TBytes of space that I exported via NFS.
So I got it, then I paid for the VMware workstation pro appliance and I was set.
This setup served me well for a bit. I was able to run NFS, and then run this blog and another server having WI-FI using Hostapd on it. Also, I was able to test several distros and other stuff.
Installed the EVE-NG server on it and was able to run bigger labs also.
However, the bigger the labs and the more appliances I was running I started to notice EVE-NG could not cope.
It was odd since the virtual guests that I had run without issues on workstation pro. Running Cisco IOL or something like that also did not create issues. I run into problems trying to run the bigger KVM images like FMC or trying to run a Linux image inside EVE-NG.
Obviously, virtualization inside virtualization was taxing EVE-NG.
I decided to move EVE-NG.
KVM
Thus, I decided to run EVE-NG as a KVM image. Technically, they do not support it. However, a few people have done it.
I figured that the CPUs on my AMD machine would have enough juice to run both VMware and EVE-NG.
Thus, converted my EVE-NG VMware image to KVM and proceeded to install it.
After getting it up and running, I restored the labs of the old EVE-NG image, and voila, I had it running.
Not so fast, while the labs run fine all of the KVM images could only start without KVM acceleration.
Tested some images and the performance was good, although not as fast as with KVM acceleration.
After poking a bit, I realized that EVE-NG was not detecting the AMD virtualization thus it was not allowing acceleration, weird.
It was weird because the AMD server had it enabled, I was passing it to the KVM image.
I could have settled for this. I decided to try something else.
Proxmox.
Proxmox is an open-source alternative to Esxi. I read good things about it so I decided to try it.
Once again found a great bargain, a Lenovo 4×2 Intel CPU with 32 G of memory refurbished.
Got Proxmox installed. I added EVE-NG and did add my NFS mount points as data stores.
So far so good.
I run into an issue out of the box, even though Proxmax has the ability to use thin provisioning on local stores, it cannot do it on NFS data stores. Not a showstopper, it would have been nice though. I was able to stand up a couple of Linux guests quick.
Run into a more serious issue, once again EVE-NG did not recognize virtualization.
More troublesome, some of the KVM images on EVE-NG would not even run.
Too bad, Proxmox showed promise.
ESXi.
This time I decided to use Esxi, got the free version and a license.
I installed ESXi and then EVE-NG on it and added my NFS mount points as data stores. Hopefully this time it was going to be a charm.
Well not so fast, running EVE-NG of the local data store was fine; NFS on the other hand had terrible issues. Read and write access was slow as molasses.
Try several things some would not work even though the motherboard on my Lenovo was supported by VMware. I could not set the right parameters on the NIC to fix the NFS issue for example.
Windows 10 guests would use 100% of disk access, Linux guests were better but you could tell performance was not quite right.
I could stand up a Linux Ubuntu in minutes on my NFS server but on the Esxi server it would take a couple of hours and Windows forget it.
On the other hand, EVE-NG was purring along nicely.
I was surprised at the speed at which KVM images would run on EVE-NG, it should not have been a surprise, 4×2 CPU Intel cores is more than enough.
Moral of the story.
Well, there is no moral.
I knew that I was using a workstation as a substitute for an enterprise server.
Doing research for the NFS issue, you can fix it by changing a couple of settings on your NIC. Unfortunately, as I stated Esxi could not make the changes.
The disk issue is also well known, VMware claims it is fixed in release 7.0, I did upgrade and it did not solve the issue.
The two issues basically do not allow me to run guests of NFS mounts.
I do not have iSCSI running on my home setup, perhaps that will be the next step since the implication is that in that kind of scenario there are no issues.
Having said that EVE-NG is working fine and now I can run some complicated labs with the memory and speed that I have.
Running guests of the local store is also fast.
I still have my NFS server with enough disk space to run other guests using workstation pro and that should not be a problem.
It was a fun weekend if you think doing this sort of thing is fun.
Take care.
Ciao.