QEMU/Advanced networking - ArchWiki

Jump to content

# QEMU/Advanced networking

From ArchWiki
< QEMU

## Advanced bridge network configuration

### Creating bridge manually

This article or section needs language, wiki syntax or style improvements. See Help:Style for reference.

Reason: This section needs serious cleanup and may contain out-of-date information. (Discuss in Talk:QEMU/Advanced networking)
Tip Since QEMU 1.1, the network bridge helper can set tun/tap up for you without the need for additional scripting. See QEMU#Bridged networking using qemu-bridge-helper.

The following describes how to bridge a virtual machine to a host interface such as `eth0`, which is probably the most common configuration. This configuration makes it appear that the virtual machine is located directly on the external network, on the same Ethernet segment as the physical host machine.

We will replace the normal Ethernet adapter with a bridge adapter and bind the normal Ethernet adapter to it.

- Install bridge-utilsAUR, which provides `brctl` to manipulate bridges.

- Enable IPv4 forwarding:

    # sysctl -w net.ipv4.ip_forward=1

To make the change permanent, change `net.ipv4.ip_forward = 0` to `net.ipv4.ip_forward = 1` in `/etc/sysctl.d/99-sysctl.conf`.

- Load the `tun` module and configure it to be loaded on boot. See Kernel modules for details.

- Optionally create the bridge. See Bridge with netctl for details. Remember to name your bridge as `br0`, or change the scripts below to your bridge's name. In the `run-qemu` script below, `br0` is set up if not listed, as it is assumed that by default the host is not accessing network via the bridge.

- Create the script that QEMU uses to bring up the tap adapter with `root:kvm` 750 permissions:

    /etc/qemu-ifup

    #!/bin/sh

    echo "Executing /etc/qemu-ifup"
    echo "Bringing up $1 for bridged mode..."
    sudo /usr/bin/ip link set $1 up promisc on
    echo "Adding $1 to br0..."
    sudo /usr/bin/brctl addif br0 $1
    sleep 2

- Create the script that QEMU uses to bring down the tap adapter in `/etc/qemu-ifdown` with `root:kvm` 750 permissions:

    /etc/qemu-ifdown

    #!/bin/sh

    echo "Executing /etc/qemu-ifdown"
    sudo /usr/bin/ip link set $1 down
    sudo /usr/bin/brctl delif br0 $1
    sudo /usr/bin/ip link delete dev $1

- Use `visudo` to add the following to your `sudoers` file:

    Cmnd_Alias      QEMU=/usr/bin/ip,/usr/bin/modprobe,/usr/bin/brctl
    %kvm     ALL=NOPASSWD: QEMU

- You launch QEMU using the following `run-qemu` script:

    run-qemu

    #!/bin/bash
    : '
    e.g. with img created via:
    qemu-img create -f qcow2 example.img 90G
    run-qemu -cdrom archlinux-x86_64.iso -boot order=d -drive file=example.img,format=qcow2 -m 4G -enable-kvm -cpu host -smp 4
    run-qemu -drive file=example.img,format=qcow2 -m 4G -enable-kvm -cpu host -smp 4
    '

    nicbr0() {
        sudo ip link set dev $1 promisc on up &> /dev/null
        sudo ip addr flush dev $1 scope host &>/dev/null
        sudo ip addr flush dev $1 scope site &>/dev/null
        sudo ip addr flush dev $1 scope global &>/dev/null
        sudo ip link set dev $1 master br0 &> /dev/null
    }
    _nicbr0() {
        sudo ip link set $1 promisc off down &> /dev/null
        sudo ip link set dev $1 nomaster &> /dev/null
    }

    HASBR0="$( ip link show | grep br0 )"
    if [ -z $HASBR0 ] ; then
        ROUTER="192.168.1.1"
        SUBNET="192.168.1."
        NIC=$(ip link show | grep en | grep 'state UP' | head -n 1 | cut -d":" -f 2 | xargs)
        IPADDR=$(ip addr show | grep -o "inet $SUBNET\([0-9]*\)" | cut -d ' ' -f2)
        sudo ip link add name br0 type bridge &> /dev/null
        sudo ip link set dev br0 up
        sudo ip addr add $IPADDR/24 brd + dev br0
        sudo ip route del default &> /dev/null
        sudo ip route add default via $ROUTER dev br0 onlink
        nicbr0 $NIC
        sudo iptables -I FORWARD -m physdev --physdev-is-bridged -j ACCEPT
    fi

    USERID=$(whoami)
    precreationg=$(ip tuntap list | cut -d: -f1 | sort)
    sudo ip tuntap add user $USERID mode tap
    postcreation=$(ip tuntap list | cut -d: -f1 | sort)
    TAP=$(comm -13 <(echo "$precreationg") <(echo "$postcreation"))
    nicbr0 $TAP

    printf -v MACADDR "52:54:%02x:%02x:%02x:%02x" $(( $RANDOM & 0xff)) $(( $RANDOM & 0xff )) $(( $RANDOM & 0xff)) $(( $RANDOM & 0xff ))
    qemu-system-x86_64 -net nic,macaddr=$MACADDR,model=virtio \
        -net tap,ifname=$TAP,script=no,downscript=no,vhost=on \
        $@

    _nicbr0 $TAP
    sudo ip link set dev $TAP down &> /dev/null
    sudo ip tuntap del $TAP mode tap

    if [ -z $HASBR0 ] ; then
        _nicbr0 $NIC
        sudo ip addr del dev br0 $IPADDR/24 &> /dev/null
        sudo ip link set dev br0 down
        sudo ip link delete br0 type bridge &> /dev/null
        sudo ip route del default &> /dev/null
        sudo ip link set dev $NIC up
        sudo ip route add default via $ROUTER dev $NIC onlink &> /dev/null
    fi

Then to launch a virtual machine, do something like this

    $ run-qemu -hda myvm.img -m 512

- It is recommended for performance and security reasons to disable the firewall on the bridge:

    /etc/sysctl.d/10-disable-firewall-on-bridge.conf

    net.bridge.bridge-nf-call-ip6tables = 0
    net.bridge.bridge-nf-call-iptables = 0
    net.bridge.bridge-nf-call-arptables = 0

In order to apply the parameters described above on boot, you will also need to load the br-netfilter module on boot. Otherwise, the parameters will not exist when sysctl will try to modify them.

    /etc/modules-load.d/br_netfilter.conf

    br_netfilter

Run `sysctl -p /etc/sysctl.d/10-disable-firewall-on-bridge.conf` to apply the changes immediately.

See the libvirt wiki and Fedora bug 512206. If you get errors by sysctl during boot about non-existing files, make the `bridge` module load at boot. See Kernel module#systemd.

Alternatively, you can configure iptables to allow all traffic to be forwarded across the bridge by adding a rule like this:

    -I FORWARD -m physdev --physdev-is-bridged -j ACCEPT

### Network sharing between physical device and a Tap device through iptables

This article or section is a candidate for merging with Internet_sharing.

Notes: Duplication, not specific to QEMU. (Discuss in Talk:QEMU/Advanced networking)

Bridged networking works fine between a wired interface (Eg. eth0), and it is easy to setup. However if the host gets connected to the network through a wireless device, then bridging is not possible.

See Network bridge#Wireless interface on a bridge as a reference.

One way to overcome that is to setup a tap device with a static IP, making linux automatically handle the routing for it, and then forward traffic between the tap interface and the device connected to the network through iptables rules.

See Internet sharing as a reference.

There you can find what is needed to share the network between devices, included tap and tun ones. The following just hints further on some of the host configurations required. As indicated in the reference above, the client needs to be configured for a static IP, using the IP assigned to the tap interface as the gateway. The caveat is that the DNS servers on the client might need to be manually edited if they change when changing from one host device connected to the network to another.

To allow IP forwarding on every boot, one need to add the following lines to sysctl configuration file inside `/etc/sysctl.d`:

    net.ipv4.ip_forward = 1
    net.ipv6.conf.default.forwarding = 1
    net.ipv6.conf.all.forwarding = 1

The iptables rules can look like:

    # Forwarding from/to outside
    iptables -A FORWARD -i ${INT} -o ${EXT_0} -j ACCEPT
    iptables -A FORWARD -i ${INT} -o ${EXT_1} -j ACCEPT
    iptables -A FORWARD -i ${INT} -o ${EXT_2} -j ACCEPT
    iptables -A FORWARD -i ${EXT_0} -o ${INT} -j ACCEPT
    iptables -A FORWARD -i ${EXT_1} -o ${INT} -j ACCEPT
    iptables -A FORWARD -i ${EXT_2} -o ${INT} -j ACCEPT
    # NAT/Masquerade (network address translation)
    iptables -t nat -A POSTROUTING -o ${EXT_0} -j MASQUERADE
    iptables -t nat -A POSTROUTING -o ${EXT_1} -j MASQUERADE
    iptables -t nat -A POSTROUTING -o ${EXT_2} -j MASQUERADE

The prior supposes there are 3 devices connected to the network sharing traffic with one internal device, where for example:

    INT=tap0
    EXT_0=eth0
    EXT_1=wlan0
    EXT_2=tun0

The prior shows a forwarding that would allow sharing wired and wireless connections with the tap device.

The forwarding rules shown are stateless, and for pure forwarding. One could think of restricting specific traffic, putting a firewall in place to protect the guest and others. However those would decrease the networking performance, while a simple bridge does not include any of that.

Bonus: Whether the connection is wired or wireless, if one gets connected through VPN to a remote site with a tun device, supposing the tun device opened for that connection is tun0, and the prior iptables rules are applied, then the remote connection gets also shared with the guest. This avoids the need for the guest to also open a VPN connection. Again, as the guest networking needs to be static, then if connecting the host remotely this way, one most probably will need to edit the DNS servers on the guest.

## Networking with VDE2

This article or section needs language, wiki syntax or style improvements. See Help:Style for reference.

Reason: This section needs serious cleanup and may contain out-of-date information. (Discuss in Talk:QEMU/Advanced networking)

### What is VDE?

VDE stands for Virtual Distributed Ethernet. It started as an enhancement of uml_switch. It is a toolbox to manage virtual networks.

The idea is to create virtual switches, which are basically sockets, and to "plug" both physical and virtual machines in them. The configuration we show here is quite simple; However, VDE is much more powerful than this, it can plug virtual switches together, run them on different hosts and monitor the traffic in the switches. You are invited to read the documentation of the project.

The advantage of this method is you do not have to add sudo privileges to your users. Regular users should not be allowed to run modprobe.

### Basics

VDE support can be installed via the vde2 package.

In our config, we use tun/tap to create a virtual interface on my host. Load the `tun` module (see Kernel modules for details):

    # modprobe tun

Now create the virtual switch:

    # vde_switch -tap tap0 -daemon -mod 660 -group users

This line creates the switch, creates `tap0`, "plugs" it, and allows the users of the group `users` to use it.

The interface is plugged in but not configured yet. To configure it, run this command:

    # ip addr add 192.168.100.254/24 dev tap0

Now, you just have to run KVM with these `-net` options as a normal user:

    $ qemu-system-x86_64 -net nic -net vde -hda [...]

Configure networking for your guest as you would do in a physical network.

Tip You might want to set up NAT on tap device to access the internet from the virtual machine. See Internet sharing#Enable NAT for more information.

### Startup scripts

Example of main script starting VDE:

    /etc/systemd/scripts/qemu-network-env

    #!/bin/sh
    # QEMU/VDE network environment preparation script

    # The IP configuration for the tap device that will be used for
    # the virtual machine network:

    TAP_DEV=tap0
    TAP_IP=192.168.100.254
    TAP_MASK=24
    TAP_NETWORK=192.168.100.0

    # Host interface
    NIC=eth0

    case "$1" in
      start)
            echo -n "Starting VDE network for QEMU: "

            # If you want tun kernel module to be loaded by script uncomment here
    	#modprobe tun 2>/dev/null
    	## Wait for the module to be loaded
     	#while ! lsmod | grep -q "^tun"; do echo "Waiting for tun device"; sleep 1; done

            # Start tap switch
            vde_switch -tap "$TAP_DEV" -daemon -mod 660 -group users

            # Bring tap interface up
            ip address add "$TAP_IP"/"$TAP_MASK" dev "$TAP_DEV"
            ip link set "$TAP_DEV" up

            # Start IP Forwarding
            echo "1" > /proc/sys/net/ipv4/ip_forward
            iptables -t nat -A POSTROUTING -s "$TAP_NETWORK"/"$TAP_MASK" -o "$NIC" -j MASQUERADE
            ;;
      stop)
            echo -n "Stopping VDE network for QEMU: "
            # Delete the NAT rules
            iptables -t nat -D POSTROUTING -s "$TAP_NETWORK"/"$TAP_MASK" -o "$NIC" -j MASQUERADE

            # Bring tap interface down
            ip link set "$TAP_DEV" down

            # Kill VDE switch
            pgrep vde_switch | xargs kill -TERM
            ;;
      restart|reload)
            $0 stop
            sleep 1
            $0 start
            ;;
      *)
            echo "Usage: $0 {start|stop|restart|reload}"
            exit 1
    esac
    exit 0

Example of systemd service using the above script:

    /etc/systemd/system/qemu-network-env.service

    [Unit]
    Description=Manage VDE Switch

    [Service]
    Type=oneshot
    ExecStart=/etc/systemd/scripts/qemu-network-env start
    ExecStop=/etc/systemd/scripts/qemu-network-env stop
    RemainAfterExit=yes

    [Install]
    WantedBy=multi-user.target

Change permissions for `qemu-network-env` to be executable.

You can start `qemu-network-env.service` as usual.

### Alternative method

If the above method does not work or you do not want to mess with kernel configs, TUN, dnsmasq, and iptables you can do the following for the same result.

    # vde_switch -daemon -mod 660 -group users
    # slirpvde --dhcp --daemon

Then, to start the virtual machine with a connection to the network of the host:

    $ qemu-system-x86_64 -net nic,macaddr=52:54:00:00:EE:03 -net vde disk_image

## VDE2 Bridge

Based on quickhowto: qemu networking using vde, tun/tap, and bridge graphic. Any virtual machine connected to vde is externally exposed. For example, each virtual machine can receive DHCP configuration directly from your ADSL router.

### Basics

Remember that you need `tun` module and bridge-utilsAUR package.

Create the vde2/tap device:

    # vde_switch -tap tap0 -daemon -mod 660 -group users
    # ip link set tap0 up

Create bridge:

    # brctl addbr br0

Add devices:

    # brctl addif br0 eth0
    # brctl addif br0 tap0

And configure bridge interface:

    # dhcpcd br0

### Startup scripts

All devices must be set up. And only the bridge needs an IP address. For physical devices on the bridge (e.g. `eth0`), this can be done with netctl using a custom Ethernet profile with:

    /etc/netctl/ethernet-noip

    Description='A more versatile static Ethernet connection'
    Interface=eth0
    Connection=ethernet
    IP=no

The following custom systemd service can be used to create and activate a VDE2 tap interface for users in the `users` user group.

    /etc/systemd/system/vde2@.service

    [Unit]
    Description=Network Connectivity for %i
    Wants=network.target
    Before=network.target

    [Service]
    Type=oneshot
    RemainAfterExit=yes
    ExecStart=/usr/bin/vde_switch -tap %i -daemon -mod 660 -group users
    ExecStart=/usr/bin/ip link set dev %i up
    ExecStop=/usr/bin/ip addr flush dev %i
    ExecStop=/usr/bin/ip link set dev %i down

    [Install]
    WantedBy=multi-user.target

And finally, you can create the bridge interface with netctl.

## See also

- libvirt networking handbook

Categories:

- Emulation
- Hypervisors

Hidden categories:

- Pages or sections flagged with Template:Style
- Pages or sections flagged with Template:Merge

QEMU/Advanced networking
Add topic