Thursday, October 08, 2009

How to create Netem distribution tables

The Netem and Dummynet emulators are widely used in networking research to achieve emulated network environments that can be used to evaluate new protocols and applications under controlled and repeateable conditions. In the case of Netem it is possible to use custom ditribution tables to better approximate the round-trip-time variations in the nework.

Unfortunately there is little information on how to generate this distribution tables based on real measured data. The only reference available is in the iproute2 source code itself and is a little confusing. After some reading and re-reading I think I finally got the magic formula and here I present it so others can benefit of my work.

First download the iproute2 source code:

git clone \

Then enter the netem subdirectory and compile the maketable and stats utilities:

cd iproute2/netem
gcc -lm -o maketable maketable.c
gcc -lm -o stats stats.c

Next you need to extract some delay statistics of the channel you want to emulate. Suppose you want to emulate the network between a server in Japan and a client PC in Chile. Then simply run a ping between both machines for some period of time. A whole day of ping data would be enough:

ping > mydata.txt

The command above does a ping to the address and stores the output to a file named mydata.txt. The file will contain the round-trip-times between the local machine and the machine with address like shown below:

PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=43 time=306 ms
64 bytes from icmp_seq=2 ttl=43 time=273 ms
64 bytes from icmp_seq=3 ttl=43 time=310 ms
64 bytes from icmp_seq=4 ttl=43 time=300 ms
64 bytes from icmp_seq=5 ttl=43 time=300 ms
64 bytes from icmp_seq=6 ttl=43 time=304 ms

The more data you have the better so leave the ping running for some long period of time. Next we need to extract the round-trip-time data only:

cat mydata.txt | grep icmp_seq | cut -d'=' -f4 | cut -d' ' -f1 > myrtt.txt

The above command will extract all the round-trip-times from mydata.txt and put them in a single column in a file named myrtt.txt. Finally we can create a distribution table that Netem understands using the maketable utility:

./maketable myrtt.txt > mydist.dist

This will create a distribution table and store it in the file mydist.dist that must be copied in a special folder (/usr/lib/tc) so Netem can find it:

sudo cp mydist.dist /usr/lib/tc

Additionally we can obtain the mean and standard deviation of the round-trip-time using the stats utility:

./stats myrtt.txt
> mu = 296.636364
> sigma = 12.808378
> rho = -0.103472

This command gives use the mean (mu) the deviation (sigma) and the correlation (rho) of the round-trip-time. And at last we can now use Netem to emulate the channel's round-trip-time using the following command:

sudo tc qdisc add dev eth0 root netem delay 296.636364ms 12.808378ms distribution mydist

Note the the distribution name is the same as that of the file we copied inside the /usr/lib/tc folder but without the .dist extension. Now Netem will provide you with a channel that has similar round-trip-time characteristics as those of the real channel.