Virtualizing Your Router: SmoothWall on Hyper-V

by Justin Unterreiner

I'm going to explain how you can run SmoothWall as your primary home router on your existing home server with the help of Hyper-V.

But first, some background. (Alternately, you can skip down to the block diagrams.)

Background

Ever since I upgraded from 56k to Cable Internet, I've run a personal home server. The first home server was a spare Pentium III running Microsoft Personal Web Server (PWS) and no-ip.com's dynamic IP update client on Windows 98 SE. I used it as my own personal cloud storage, before the phrase was coined.

Fast forward a few (many!) years, and the setup had grown to three servers instead of one.

This setup was great, except for two things: power usage and heat. The older Pentium 4, acting as a router, wasn't very energy efficient, and the heat from three separate computers left the office closet quite toasty.

I decided to start a new project: build a single, energy efficient PC that would combine all three server roles into a single box.

Combining the NAS and application server roles was easy enough; both roles used Windows-based software and could easily run on the same box.

The router would prove the most difficult prospect. After years of using SmoothWall, I didn't want to go back to a consumer router. What I really wanted was for SmoothWall to run side-by-side with Windows on the same box. Enter virtualization with Hyper-V!

The Hardware

First, the hardware. Since this machine would be an NAS, I needed a case with multiple hot-swappable SATA drive bays. I went with the U-NAS 800 Server Chassis.

To address the power efficiency and heat concerns, I went with the Intel i3-4130T CPU.

Finally, for router function I needed a minimum of two Ethernet ports. Since the case was so small, they would need to be ports on the motherboard instead of expansion cards. I went with the Gigabyte Z97N Mini-ITX board.

The Software

The server runs Windows 10 as its host operating system. I chose Windows here because all of the software I use for the NAS and application server roles is Windows-based.

The router portion is where it gets interesting. The SmoothWall OS is hosted inside of a Hyper-V instance on the same Windows 10 installation. But how do we connect with the cable modem?

As mentioned above, two Ethernet interfaces are needed: one for the cable modem and one for the connection to the internal network.

It would be an bad idea, however, to connect a PC running Windows (or any desktop operating system) directly to a broadband modem without an NAT or firewall device in between. Fortunately, Hyper-V helps us out with virtual network partitions.

The diagram below shows how this can be achieved. Note that SmoothWall uses colors to indicate network types. For example, green is for private networks (the internal LAN), and red is for public networks (the Internet).

Below you can see that the physical interfaces are not bound to the TCP/IP protocol on the Windows side which prevents Windows from using them to communicate. Instead, they are bound to the Hyper-V virtual switches, the same switches that the SmoothWall VM is bound to.

Below you can see how the virtual interfaces are configured. The virtual interfaces to the SmoothWall are straightforward; the red interfaces connects to the red virtual switch and the green virtual interfaces connect to the green virtual switch. At this point, all of your clients on the LAN would receive IPs via DHCP and have Internet access, except for the Windows 10 host server. In order to connect the Windows side, we need one more virtual interface that binds its TCP/IP to the green virtual switch.

All you need to do now is set up Hyper-V to auto-boot the SmoothWall VM when Windows starts, and you have a virtual router ready to go!