If we want to experiment with network applications in a more realistic environment (for example, if we develop a new routing protocol), then we need a tool to simulate the properties of a WAN connection. As we know, a WAN is different from a LAN: bandwidth is lower; there can be bit errors in the transmitted packets in that they can be lost or reordered; and the delay is significantly higher.
There are numerous WAN emulators on the market (both hardware and software), but if we want a cheap and relatively simple one, we can use a Raspberry Pi (RPi) with Linux, which has the NETEM (Network Emulator) function built into the modern kernels. In this article, we try the functions provided by the NETEM, and because in this case the RPi is functioning as a router, we can configure it as a simple firewall as well.
CCNA Training – Resources (Intense)
The physical topology in our labs is very simple, but we need a piece of hardware to build it. Because the RPi has just one built-in Ethernet port, we need a second one, so we have to buy a USB-to-Ethernet adapter. It’s not expensive and maybe we can get it cheaper from Ebay or similar sites.
If we need to connect more than two USB devices to the RPi, we need a USB hub as well. When you connect the adapter, it should be recognized as eth1, and you can quickly check this with the dmesg command or the ip command (as you can see I’ve already configured the IP address):
I connected my devices according to the diagram below because I want Internet connection for the RPi and my PC2 as well, but you can connect them directly (don’t forget to use a crossover cable in this case). My router as the default gateway is at 192.168.0.1.
If you look at the illustration above then it’s obvious that the default gateway for PC1 is the RPi, but because on PC2 the default gateway points to 192.168.0.1, we have to define a static route to 192.168.40.0/24 on it, and we need the RPi to act as a router. The first problem can be solved by the following command on the Windows PC:
route add 192.168.40.0 mask 255.255.255.0 192.168.0.31
The command necessary to make the RPi as a router temporarily (I mean until the next reboot):
We used the system control (sysctl) utility to write the logical 1 value to the proper kernel variable. If we want this setting permanent, locate the /etc/sysctl.conf file, uncomment the line containing the above value, and reboot or issue the sysctl -w -p /etc/sysctl.conf command.
Now try pinging PC2 from PC1 and vice versa (if the Windows firewall prevents it, you can disable it permanently). The pings should work and the time values will be around 2-3 milliseconds.
Let’s first simulate the increased delay on a WAN link which can be easily checked by ping command. For the simulation we’ll use the NETEM function, which can be controlled by the tc (Traffic Control) command, which is part of the iproute package. Traffic control can be a complicated but powerful tool and I recommend reading the Traffic control HOWTO. For our purposes though, we need not understand all of its features.
Traffic control is based on queues, into which the packets arrive and later leave at a given order and given rate. This can be referenced as Quality of Service as well, because in this way we can separate various traffic and give them different priorities, for example.
Every network interface has queues and an attached scheduler, which arranges the packets in the queue for output. The simplest scheduler is FIFO (first-in-first-out), but many more exist, such as SFQ, WRR and so on. This scheduler has another name under Linux traffic control: qdisc (queuing discipline). There are two qdiscs on every interface: the egress or root qdisc is related to the output traffic from the interface, while the ingress qdisc is related to input packets. As we have more control on the packets leaving the interface, we mostly use the egress or root qdisc.
With this information in mind we can start our lab, and define the delay for eth1 interface of RPi. The necessary command is as follows:
tc qdisc add dev eth1 root netem delay 1000ms 15ms
This means that we add 1000 milliseconds of delay to the root qdisc of eth1 device, but with some jitter (which is 15 milliseconds). If we want a more realistic distribution of packets, we can use the following command instead:
tc qdisc change dev eth1 root netem delay 1000ms 15ms distribution normal
After this try pinging PC1 from PC2 (I’ve set 5 pings just like on Cisco IOS):
As we can see, the time values reflect the delay settings. The high value of the first ping is probably because of ARP.
Now try to simulate corrupt packets with the following command (note that we replaced the previous NETEM setting with the change keyword):
tc qdisc change dev eth1 root netem corrupt 20%
These packets will contain bit errors, so in the ping output we can see timeouts, and the summary tells the amount of the loss:
There’s another tool we can use in testing called mtr (My Traceroute), which has Linux and Windows versions also. The latter is called WinMTR and can be downloaded from here: http://winmtr.net/download-winmtr/. Running it gives us the following result:
As we can see in my case, the packet loss due to corruption is a bit higher than the configured value, but it roughly fits the configured setting.
Now try the loss parameter of NETEM:
tc qdisc change dev eth1 root netem loss random 20%
Run WinMTR again and check the results.
NETEM can simulate other transmission errors, such as duplicated and reordered packets. The configuration is as follows:
tc qdisc change dev eth1 root netem duplicate 10% reorder 5%
As we can see, we can combine multiple options in one line. Of course this feature is needed as these events on the WAN link can occur simultaneously.
Now try some bandwidth limiting with the help of tc. There are various methods for this under Linux but we will use a simple qdisc for our lab called Token Bucket Filter (TBF). TBF, as the name implies, uses a symbolic bucket with tokens associated with it.
According to the manual, “TBF is stocked with tokens which correspond to the amount of traffic that can be burst in one go. Tokens arrive at a steady rate, until the bucket is full. If no tokens are available, packets are queued, up to a configured limit. The TBF now calculates the token deficit, and throttles until the first packet in the queue can be sent.”
You can read more about this qdisc among the links provided at the end of the article. In its simplest use we can limit the bandwidth to a certain value that’s available to the clients. There are more sophisticated methods also, in which we can provide bandwidth based on IP address or other attribute of the hosts, but in this case this is not the goal.
First we need to clear the previous settings, then add TBF qdisc with the following command:
tc qdisc del dev eth1 root
tc qdisc add dev eth1 root tbf rate 512kbit burst 50kbit latency 50ms
TBF parameters are as follows:
Rate: defines the configured speed; in this case we want to limit the bandwidth to 512 kbit/s.
Burst: the size of the so-called bucket; in other words, it’s the amount of packets that can go out simultaneously on the interface.
Latency: specifies the maximum amount of time a packet can sit in the bucket.
Now we need a tool to measure the bandwidth. I chose iperf, which is a multiplatform software and is easy to use. On PC1 I started the server through the iperf –s command, then on PC2 I connected to it by issuing iperf –c 192.168.40.2. The result was the following:
As we can see, our traffic shaping is working. Try to experiment with the various parameters of TBF.
Finally we can put NETEM and TBF together and simulate a realistic, slower WAN link. Because we cannot put them together in one rule, we need to define handles and we can build a simple tree structure. The commands are:
tc qdisc add dev eth1 root handle 1: tbf rate 512kbit burst 50kbit latency 50ms
tc qdisc add dev eth1 parent 1: handle 11: netem delay 1000ms loss 10%
Now we need to define a handle for the root whose value is 1 and attach the TBF rule to it. Then we add a second qdisc with the handle value of 11, whose parent is 1. In this way both of the rules are attached. The handles have a second, so-called minor number also, but in this case we didn’t use it.
The result is the following (now we see the server side):
The above shows that we used UDP traffic for testing, and in the first case we have more lost packets but in the second case the result is pretty good. There are a lot of configurable variables using tc, and we can do very esoteric things with it, but maybe that’s enough to demonstrate its capabilities.
When network traffic goes through a device, we can filter that traffic and use the device as a firewall. The RPi can act as a simple firewall, and we can configure it in various ways to do so. In Linux, the Netfilter code in the kernel provides firewall functions and we can use the iptables command to configure it. There are a lot of good tutorials on using iptables, so here I will only show a really simple code just to show the logic behind the command.
First of all, a good firewall blocks everything that is not allowed. If we issue the iptables –L command, we can see the default behavior of Netfilter: the traffic destined to the firewall (chain INPUT), the traffic that goes through the firewall (chain FORWARD) and the traffic sourced from the firewall (chain OUTPUT) are all accepted.
Let’s change this: close all the gates, except the output from the firewall. We can easily do this by changing the policies to DROP:
Did you connect to the RPi by SSH? Then the first command broke your connection. J
The first rule when writing firewall code: think before you press ENTER, because you can lock yourself out from the firewall. You can revert back to the default settings on the console, fortunately.
Now add some rules for the allowed traffic. For example, we want to allow an administrator PC using the IP address of 192.168.0.3 to connect. We have to add a rule to the INPUT chain and define the source address, and then define what to do when this rule is true (where to “jump”):
iptables –A INPUT –s 192.168.0.3 –j ACCEPT
Another rule lets us connect to the machine from everywhere, but by SSH only. We need to know some properties of SSH to do this. First, it uses TCP as transport protocol and it uses port number 22. The rule is as follows:
iptables -A INPUT -p tcp –dport 22 -j ACCEPT
Now we can connect from any PC to the RPi by SSH, but only PC2 can ping it (or can connect by any method).
Let’s define a simple rule for the transit traffic also. PC1 and PC2 cannot even ping each other, so let’s allow PC1 to ping PC2 – but not vice versa. We need to know that ICMP has many types of error, control and diagnostic messages, and ping uses two of them: echo-request (code 8) and echo-reply (code 0). Therefore we need to allow echo-requests into the firewall on interface eth1, and let echo-requests come in on eth0 with proper destination addresses also:
Now our firewall code looks like this:
I used the –n parameter in the command since I didn’t want to wait while iptables resolves the IP addresses to hostnames (it won’t succeed anyway).
There are a lot more options to filter traffic (I didn’t even mention stateful filtering), but I hope that the basics are clear.
As we can see now, writing complex traffic control and firewall rules can be a fun but difficult and cumbersome experience, so make sure to use a lot of tools to help in this work. You can find some of these among the useful links below.
Linux Advanced Routing and Traffic Control HOWTO:
The basics of TBF:
TCGUI: a visual traffic control configurator:
Modules for Webmin, the web-based system admnistration tool:
Firewall Builder: a software to help building firewalls for various operating systems: