September 2018

Part 3: Setup router to use the routing VM(s)

With the VM building out of the way, on to configuring the VMs.

First a little more software install/prep work

yum install quagga
systemctl enable zebra
systemctl enable ospfd
systemctl disable firewalld
cp /usr/share/doc/quagga-*/zebra.conf.sample /etc/quagga/zebra.conf
cp /usr/share/doc/quagga-*/ospfd.conf.sample /etc/quagga/ospfd.conf
chown quagga.quagga /etc/quagga/*.conf
setsebool -P zebra_write_config 1
systemctl start zebra
systemctl start ospfd
systemctl stop firewalld

Basically, install quagga, a routing daemon that works similar to cisco ios. From there enabling the two services we need for ipv4 connectivity. Zebra is the main quagga daemon, the ospfd is the daemon to run this as an OSPF router. There is an ospf6d daemon that I have yet to mess with to route the fd00::/8 ipv6 network.
In the process of setting this up, I found the default firewall is pointless and causes problems with the routing daemons unless you open the right ports and subnets. I chose to disable the firewall as these VMs are not directly exposed to the internet and are behind firewalls already.
The last two odd commands that may not be familiar:
The setsebool, allows the quagga daemon to modify the /etc/quaga/*.conf files.
The vtysh, this is how you get into the quagga software. Thus into the next bit.

routervm-site1# conf t
routervm-site1(config)# int eth0
routervm-site1(config-if)# ip address
routervm-site1(config-if)# ip ospf authentication message-digest
routervm-site1(config-if)# ip ospf message-digest-key 1 md5 quagga
routervm-site1(config-if)# exit
routervm-site1(config)# int zt-interfacename
routervm-site1(config-if)# ip address
routervm-site1(config-if)# ip ospf authentication message-digest
routervm-site1(config-if)# ip ospf message-digest-key 1 md5 quagga
routervm-site1(config-if)# exit
routervm-site1(config)# router ospf
routervm-site1(config-router)# ospf router-id
routervm-site1(config-router)# network area
routervm-site1(config-router)# network area
routervm-site1(config-router)# area authentication message-digest
routervm-site1(config-router)# neighbor
routervm-site1(config-router)# neighbor
routervm-site1(config-router)# exit
routervm-site1(config)# ip forwarding
routervm-site1(config)# ipv6 forwarding
routervm-site1(config)# exit
routervm-site1# write

To sum this up:

    • Enter configuration terminal
    • interface eth0, (may need to change this to match, silly eno2345 names now days are possible.
    • Configure the IP address
    • Setup MD5 authentication and defines the key “quagga”
    • Drops back to config terminal, and then into the ZeroTier interface
    • A repeat of the eth0 interface, just different IP
    • Drops back to config terminal, and into the OSPF routing bit
    • Defines the OSPF router ID, I chose the ZeroTier IP.
    • Defines the networks OSPF is to run on and in what area
      • I went with the backbone area, as I don’t really want to get to complex
    • Defines the area to use MD5 authentication
    • Defines the OSPF neighbors,
      • In this case the other two VMs in the ZeroTier network.
    • Drops back and sets IP forwarding for IPv4 and IPv6
    • Drops back again and writes configs to disk

At each site where one of these VMs is set up, change the IP address as needed.  (network interfaces, neighbors, and router-id)

Continue reading…

Part 2: Build the router VM(s)

To bring us into part 2, the next thing is to create the routing VMs.

I simply did a simple CentOS 7 minimal install with one network interface configured.  At the router, I created network on the same LAN segment.  During setup of the VM, I configured the VM to be with a gateway of  I also did the same with IPv6, fd00:10:10:9::2/64 and fd00:10:10:9::1/64 respectively.  Now when I get to other sites to deploy, the network will change to be &  The reason I chose to go with a /29 here is simply due to I plan to layer in a “services” into these subnets.  This particular set of VMs I plan to layer on a Pi-Hole and Chrony NTP server.

If you have trouble installing CentOS 7 at a minimal install, then this series is not for you.

Once you have the VM created, the first thing to do is install ZeroTier.  Thankfully they make this dead simple.

If you want to verify things:
curl -s '' | gpg --import && if z=$(curl -s '' | gpg); then echo "$z" | sudo bash; fi

If you like to live risky:
curl -s | sudo bash

Run either as root and it is installed.

Other small things I have found that are needed after install,
In /etc/sysconfig/network-scripts/ifcfg-eth0, add the following: NM_CONTROLLED=”no”
In /etc/sysconfig/network, add the following: NETWORKING=yes
In /etc/selinux/config, change SELINUX to be permissive, SELINUX=permissive
Reboot a couple of times, make sure that default gateway and DNS servers show up in their respective places.

At this point, the next thing to do is create the ZeroTier network.  Simply log in and go the networks page and create a new network.  Once that is done, the next thing is to join the network.

zerotier-cli join abcd1234abcd1234

Finally, go authorize the new clients and label them in the “My Networks”.

Part 1: ZeroTier and making a multi-site LAN (MAN?)

Recently I have come across an interesting little software tech that lets me do some fun things that normally one would only see in the data center or large companies.  That software is called ZeroTier.

When I first started playing with this I was doing some really bad things abusing networks to get a nice simple method to access all my remote servers and devices without hassle.  Let’s just say all of what I did was horrible and was burned to the ground in favor of this. My goal was to set up a small network that would span 3 LANs and allow me to connect them together either in a routed or one big bridge.  I do not recommend the bridge but it can be done.

So the first step I took was to draw out what the network should look like. Then I took a little time and thought about it and had to modify it a little more.

Rough hand drawn what I wanted it to look like.

After a little thinking.

In the picture on the left, I identified I would need to use 4 subnets, 3 for the LANs and one for the inter LAN routing.  I then saw a flaw with this initial diagram, how do I route to ZeroTier?  My routers of choice are MikroTik, there is no built-in ZeroTier package for them.  It is then I thought of doing a router on a stick at each site to handle doing the handoff to ZeroTier.  Granted I drew a device with traffic routing through it; not to and from it on the same interface.  This will take you to I need more subnets.  So finally I had what needed to complete this task. 3 LAN subnets, 3 point to point subnets, and 1 ZeroTier subnet.

Now, why did I want to go this way?  Sure one could set up an IPsec tunnel or any various other VPN tunnels and send traffic over them.  This software offered the opportunity to let me make a MAN like network across the internet that is very little configuration and almost zero NAT hole punching.  At work, I have seen what happens when VPNs suddenly die and it is due to someone/something doing something bad.

Continue reading…

Series about ZeroTier

Currently, I am writing a little series involving a product I recently discovered, ZeroTier.  Many people seem interested in how I am using it for personal use.  So spending a few nights writing up my thoughts, the how to’s, and grabbing some screenshots along the way.

Below is a bulleted list of post titles for the series.

I hope this is a nice little list of topics/points that will cover the subject well.  Once I have all the core parts written I will post them all back to back (minutes apart) and link them here as well.

(edit 2018-12-07, Updated Part 4 & 5 to have links, Added Part 6 text.)