DIY Home Lab Server (2017 Version)

Owning a server physically has always been one of my dreams since an increasing number of free cloud services in China were quitting the game. Obviously, storage is always a "human basic need", and soft routers as well as other infrastructure provided by a home lab server could help more in daily life. So, beyond a storage server, a home lab server is what I want to build this time.

Hardware

The cheaper the better, the free are the best.

It's my baseline for picking hardware for home lab server, where "cheap" refers to high cost-performance ratio. I've already got an Intel i5-3470, and 16 GiB of DDR3. The next step is picking an suitable motherboard for it. For a server, server boards with IPMI equipped is desired. But they are often relatively expensive, and advanced features such IPMI and ECC RAM support aren't always necessary. If the board and the whole system is stable enough, once it has booted into a system that can provide Web UI management, we won't actually rely on IPMI or other similar solutions.

At first my plan was to make full use of a spare ITX NCASE M1. My NCASE M1 v5.0 is actually a well-designed case for ITX form format with 3 PCI slots, and can hold a long graphics card or two 3.5" drives along with SFX power module and more than two 2.5" drives. There is also a forum thread to discuss 3-slot micro ATX boards that suits the case, and since I've got an Intel i5-3470, I bought a Gigabyte B75M-D2V motherboard for my home lab server. The board features a Realtek Gigabits interface, a native SATA 3.0 6Gbps port as well as  built-int USB 3.0 support, and it cost me only RMB 138 (US$ 20 approximately) from the internet.

The rest of hardware are the easy part. The mother board provides only 2 RAM slots and support up to 16 GiB RAM, so two Samsung DDR3 will work perfectly fine. An old Samsung EVO 830 SSD is used as startup disk, and a Fujitsu D2607-A11 (LSI chip, flashed into IT mode) is used as HBA and will be assigned to FreeNAS through VMDP.

The best part is the case - the new case. NCASE M1 is a nice choice for ITX desktop and workstation. But for home lab server, it is expected to hold more hard drives and could support hot-swap. There're some other choices from SilverStone, Lian Li and other brands. My first target is Silver Stone DS380, a mid-tower chassis that can provide at least 8 hot-swap 3.5" hard drive bays, plus two 5.25" bays. However, my workstation is running in a Fractal Design Define R5, which is also a (huge) mid-tower chassis that provides over 8 3.5" bays. The space are precious. Then I found some products in small form factor, such as SilverStone CS380 and SilverStone CS01-HS. They seems to be great choices, but that doesn't count the price in, let alone the comments that the cooling performance of those chassis is poor. By chance I was finding U-NAS chassis, which are in ITX form factor, and provides options for 2 bays, 4 bays and up to 8 bays. It was very luckily for me that their latest product, U-NAS NSC-810A, switched from ITX form factor to micro ATX, and is on sale. That means I can get a chassis bundled with a 350 watt 1U power supply unit for a reasonable price, and can adopt my existing platform to it without buying or selling. After some hesitation, I purchase the chassis and power supply unit bundle for RMB 1,498 (US$ 350).

The chassis are layered into hard drive space and motherboard space, and the internal of the it is somehow compact. The build of a small form format (though this one is micro ATX) is always hard, but thankfully everything works.

Note that the 2 PCI expansion slots require a soft link adapter to connect the card and motherboard, which isn't included in the accessories. I have read about this before but not sure about if a connector will be delivered with the chassis or not. The truth is that it wasn't delivered, and I have to order one and wait for additional days.

Software

Platform

Virtualization: VMware ESXi

VMware ESXi is a popular hypervisor and it offer free limited license for non-commercial usage. It provides easy-to-use WEB UI management and deep integration with VMware workstation. With the hardware supporting VT-d and VT-x, VMware ESXi provides hardware pass-through and other advanced features. It can run multiple operating systems on a single bare metal, and will meet the needs of a home lab server.

Storage: FreeNAS Corral

FreeNAS is a FreeBSD based system that targets at storage usage. It takes the advantages of the native support for ZFS in FreeBSD to provide flexible and reliable storage services. FreeNAS Corral is now (Jul 12, 2017) a technical preview branch of FreeNAS, and the latest release would be FreeNAS 11. But FreeNAS 11 haven't yet provides a built-in docker feature which I deeply rely on. Besides, my existing storage system is running on FreeNAS Corral, and it's of usability, so I decided to stick on it.

Ubuntu Server

Nothing more than preferences to say. Use other systems, or both of them if you need or you like.

Post Installation Job

Flash Fujitsu D2607-A11 to IT Mode

There're pages available on the Internet to teach how to flash, and the original post may be this one. It has actually been a while since I flashed my RAID card into IT mode so I can't find the one I follow. There are minor differences among them, and some of the tutorials may fail. But try more and there will always be a working one. I didn't success either without one-day hardworking try.
Another issue is about Boot from PCI Op-ROM. This RAID card has a built-in Op-ROM which is a management GUI. This problem will not exist after flashing it into IT mode, as it works as an HBA under IT mode.

Patching VMware ESXi Installer Image

Gigabyte B75M-D2V comes with an integrated Realtek network interface card, and VMware ESXi doesn't have built-in drivers for it. Drivers for Realtek 8168/8111/8411/8118 can be found here, and a guide to integrate it into installer ISO could be found here. Both the driver and the tool works for VMware ESXi 6.5.