Setting up network bridging for VMs in Ubuntu

I had cause to set up a network bridge in a new Ubuntu system this weekend. For those unsure of the concept, the idea of bridging (in this context) is to connect virtual machines to a network. The name comes from the hardware concept of bridging, where multiple networks are glued together by the use of a "bridge", a hardware device that does all the routing, mostly transparently.

In Linux, there's a "bridge network device" set up to support providing networking to VMs and other more advanced networking concepts.

In the process of configuring my bridge, I made numerous mistakes based upon a hazy idea of what I was setting up. So, if you need to set up bridges for LXC, KVM, or other systems, this is what you need to know and do.


1. What the bridge device is

The bridge device appears as "br0" (like "eth0") on the host machine and it all but replaces eth0 as your primary network device. It handles all routing to and from the external network, and within the machine to the VMs that are using it. How it appears to each VM depends on the virtual machine system you're using - typically, those VMs will think they have a device called "eth0" that's connected to the external network, but the network they're connecting to is the one controlled by the bridge.

Each bridge device (br0, etc) is associated with a "real" device like eth0. If the bridge device sees traffic coming in from the real device addressed (at a low level) to one of the virtual machines or to the host, it routes traffic to that VM or the host. Likewise,  it routes internal traffic internally, and routes traffic from VMs or the host to external addresses via the "real" device. The addressing is via standard Ethernet MAC addresses, not IP addresses - those are higher level addressing concepts, and the bridge device tries to be protocol neutral. The overall effect is one where the network controlled by the bridge is merely an extension of your existing Ethernet network.


2. How you configure it

I'm not going to cover the VM side as that'll depend on your virtual machine manager, but here's the key points for setting up the system. I'm assuming that you DON'T want NAT - virtually none of my readers will want NAT given their external router already handles that for them. The only exception I can think of is if you're experimenting on your work PC and you aren't the network admin and don't want to particularly bother him or her. Anyway, here goes:

Step 1: You need to disable Network Manager and install bridge networking.

Completely. Network Manager isn't really what you want running on a server in the first place. (To be honest, I've never liked the tool, I see why it's there, but it's not particularly reliable in my experience.

Disable it using:

  $ sudo -s

 <your password>

 # /etc/init.d/NetworkManager stop

 # apt-get remove network-manager

sudo -s makes you root until you exit, I'm going to assume you remain root for the remainder of this article. Be careful! Should I ask you to reboot, remember to sudo -s upon logging back in if you need to repeat any of the steps in Step 1 or Step 2.

You'll also need to add the bridge support. You can do this using:

 # apt-get install bridge-utils

Step 2: Set up /etc/network/interfaces and other configuration files

I'm going to make the following assumptions here.

• You're using a static IP address. For the examples, it's 10.10.0.3
• Your router is 10.10.0.1
• Your DNS server is 10.10.0.2.
• Your netmask is 255.255.0.0 which makes your broadcast address 10.10.255.255. 

You'll need to modify the examples accordingly for your situation. IPv6, incidentally, should "just work" if you're using stateless configuration - at least for the bridge and host, your VMs are another matter but that's because of peculiarities in the VM implementations, not the bridge system.

First, edit your /etc/resolv.conf and put in something sensible like:

nameserver 10.10.0.2

Second, edit your /etc/network/interfaces file and make it look like this (with the IP addresses changed to suit your network, obviously.)

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual

auto br0
iface br0 inet static
   address 10.10.0.3
   netmask 255.255.0.0
   gateway 10.10.0.1
   bridge_ports eth0
   bridge_stp off
   bridge_maxwait 5
   post-up /usr/sbin/brctl setfd br0 0 addif br0 eth0

The biggest thing you'll probably notice about the above is that there's no configuration for eth0. This is intentional. Other than the interface coming up, you don't want it to have anything assigned to it. The bridge device is going to be the one that receives traffic for your host, it'll handle the routing to and from the Ethernet port.


Step 3: Reboot and check

Now, I know you're probably going to ask "Can't I just test it without rebooting?", well, you can, but you will not know for sure that everything's going to work the next time the system goes down. Given your network is all screwy right now anyway, you're not going to lose anything by doing this. So reboot, and upon restart check that your networking is doing what you think it should.

Type the following to verify the network is up and running properly:

 $ ifconfig

This should show, at minimum, the following devices as being "up":

br0 - which should be configured as above
eth0 - which should be up, but have no IPv4 address, and if it has an IPv6 address, it should only be the link-local address (begins with fe80:)
lo - which should be configured as 127.0.0.1.

$ route -n

This should show, at minimum, the following routes:

Destination: 10.10.0.0: Gateway 0.0.0.0, Genmask 255.255.0.0, Iface br0
Destination: 0.0.0.0: Gateway 10.10.0.1, Genmask 0.0.0.0, Iface br0

$ ping www.google.com

This should work. If it can't find the host, your /etc/resolv.conf is wrong. If the destination host is unreachable, then there's probably a major problem with your configuration so check the numbers and, obviously, make sure your Internet is working fine from another machine on the same network.


Step 4: Configure your VMs

At this point your bridge is working, so you can get to the next messy stage and start configuring your VMs to use it. That's another story, and what you do will depend upon what VM system you're using. Good luck!