Control groups, or cgroups, are a way in Linux to control processes' hardware resources utilization by defining the resources limits, grouping them in a hierarchical structure and assigning processes to them. Cgroups can be used, in particular, to specify the skb priority of all network packets generated by specific process. This provides a convenient way to prioritize network traffic generated in communication with the WHLE board itself (as opposed to the traffic passing through it when it’s used as a router, a case described in Ssh Prioritization (iptables)).
Connection Diagram
...
The network used is very straightforward and consists of a single 1 Gb/s link between a testing machine (PC) and a WHLE board (whle_ls1046). Two iperf3 streams sending data from whle_ls1046 to PC will be competing for the link’s throughput. Different traffic classes will be used using the cgroups mechanism for the associated iperf3 processes and the resulting changes in data transfer speed will be observed.
| Inc drawio | |||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...
| Code Block | ||
|---|---|---|
| ||
root@whle-ls1046a:~#
test_iperf() {
local port1=$1
local prio1=$2
local port2=$3
local prio2=$4
local iperf_time=$5
kill_iperf "${port1}"
kill_iperf "${port2}"
tc qdisc del dev eth1 root handle 1:
launch_iperf_with_priority "${port1}" "${prio1}" "${iperf_time}"
launch_iperf_with_priority "${port2}" "${prio2}" "${iperf_time}"
tc qdisc add dev eth1 root handle 1: mqprio num_tc 4 \
map 0 0 0 0 1 1 1 1 2 2 2 2 3 3 3 3 hw 1
sleep ${iperf_time}
} |
The example usage will be given below.
Tests
Same priority
...
1 2 2 2 2 3 3 3 3 hw 1
sleep ${iperf_time}
} |
The example usage will be given below.
Tests
Same priorities
Assuming that iperf3 servers at ports 5201, 5202 are running on PC, run the following command on WHLE:
whle_ls1046
| Code Block |
|---|
root@whle-ls1046a:~# test_iperf 5201 4 5202 4 6
Launching iperf3, port 5201, priority 4
[1] 735
Launching iperf3, port 5202, priority 4
[2] 738 |
This would create two iperf3 streams with the same skb priority 4, mapping to the traffic class 1. Meanwhile, on the PC side:
PC, console 1
| Code Block |
|---|
Accepted connection from 192.168.3.2, port 54202
[ 5] local 192.168.3.1 port 5201 connected to 192.168.3.2 port 54208
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 55.0 MBytes 462 Mbits/sec
[ 5] 1.00-2.00 sec 56.1 MBytes 471 Mbits/sec
[ 5] 2.00-3.00 sec 56.1 MBytes 471 Mbits/sec
[ 5] 3.00-4.00 sec 56.1 MBytes 471 Mbits/sec
[ 5] 4.00-5.00 sec 56.1 MBytes 471 Mbits/sec
[ 5] 5.00-6.00 sec 56.1 MBytes 471 Mbits/sec
[ 5] 6.00-6.04 sec 2.46 MBytes 467 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-6.04 sec 338 MBytes 469 Mbits/sec receiver
-----------------------------------------------------------
Server listening on 5201
----------------------------------------------------------- |
PC, console 2
| Code Block |
|---|
Accepted connection from 192.168.3.2, port 46446
[ 5] local 192.168.3.1 port 5202 connected to 192.168.3.2 port 46458
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 53.9 MBytes 452 Mbits/sec
[ 5] 1.00-2.00 sec 56.1 MBytes 471 Mbits/sec
[ 5] 2.00-3.00 sec 56.1 MBytes 471 Mbits/sec
[ 5] 3.00-4.00 sec 56.1 MBytes 471 Mbits/sec
[ 5] 4.00-5.00 sec 56.1 MBytes 471 Mbits/sec
[ 5] 5.00-6.00 sec 56.1 MBytes 471 Mbits/sec
[ 5] 6.00-6.04 sec 3.69 MBytes 793 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-6.04 sec 338 MBytes 470 Mbits/sec receiver
-----------------------------------------------------------
Server listening on 5202
----------------------------------------------------------- |
The experiment shows that the link’s throughput is shared evenly for traffic in the same class. Similar results would be obtained with calls:
| Code Block |
|---|
test_iperf 5201 0 5202 0 6
test_iperf 5201 8 5202 8 6
test_iperf 5201 12 5202 12 6 |
(That would cover all 4 traffic classes defined by tc, with skb priorities different from 0, 4, 8, 12 resulting in the same classes set.)
Different priorities
Run test_iperf with different skb priorities, making sure that they map to different traffic classes, for example:
whle_ls1046
| Code Block |
|---|
root@whle-ls1046a:~# test_iperf 5201 40 5202 4 6 Launching iperf3, port 5201, priority 40 [1] 735774 Launching iperf3, port 5202, priority 4 [2] 738 |
...
] 776 |
Meanwhile, on the PC side:
PC, console 1
| Code Block |
|---|
Accepted connection from 192.168.3.2, port 5420248344 [ 5] local 192.168.3.1 port 5201 connected to 192.168.3.2 port 5420848350 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 558.084 MBytes 74.1 462 Mbits/sec [ 5] 1.00-2.00 sec 56.1 MBytes 471 Mbits/sec [ 5] 2.00-3.00 sec 560.100 MBytesBytes 471 Mbits0.00 bits/sec [ 5] 32.00-43.00 sec 560.100 MBytesBytes 471 Mbits0.00 bits/sec [ 5] 43.00-54.00 sec 560.100 MBytesBytes 471 Mbits0.00 bits/sec [ 5] 54.00-65.00 sec 560.100 MBytesBytes 471 Mbits0.00 bits/sec [ 5] 65.00-6.0400 sec 20.46 MBytes00 Bytes 4670.00 Mbitsbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-6.0408 sec 8.84 338 MBytes 46912.2 Mbits/sec receiver ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- |
PC, console 2
| Code Block |
|---|
Accepted connection from 192.168.3.2, port 4644642704 [ 5] local 192.168.3.1 port 5202 connected to 192.168.3.2 port 4645842720 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 53.9 100 MBytes 452839 Mbits/sec [ 5] 1.00-2.00 sec 56.1 112 MBytes 471942 Mbits/sec [ 5] 2.00-3.00 sec 56.1112 MBytes 471942 Mbits/sec [ 5] 3.00-4.00 sec 56.1 112 MBytes 471942 Mbits/sec [ 5] 4.00-5.00 sec 56.1 112 MBytes 471942 Mbits/sec [ 5] 5.00-6.00 sec 56.1 112 MBytes 471942 Mbits/sec [ 5] 6.00-6.04 sec 34.6971 MBytes 793937 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-6.04 sec 338666 MBytes 470925 Mbits/sec receiver ----------------------------------------------------------- Server listening on 5202 ----------------------------------------------------------- |
The experiment This shows that the link’s throughput is shared evenly for traffic in the same classtraffic class 1 (skb priority 4) has a strict priority over traffic class 0 (skb priority 0). Similar results would be obtained with any of the calls:
| Code Block |
|---|
test_iperf 5201 0 5202 8 6 test_iperf 5201 0 5202 12 6 test_iperf 5201 84 5202 8 6 test_iperf 5201 4 5202 12 6 test_iperf 5201 8 5202 12 6 |
(That would cover all pairs of 4 traffic classes defined by tc, with skb priorities different from 0, 4, 8, 12 resulting in one of the same classes setpairs from above.)