Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Control groups, or cgroups, are a way in Linux to control processes' hardware resources utilization by defining the resources limits, grouping them in a hierarchical structure and assigning processes to them. Cgroups can be used, in particular, to specify the skb priority of all network packets generated by specific process. This provides a convenient way to prioritize network traffic generated in communication with the WHLE board itself (as opposed to the traffic passing through it when it’s used as a router, a case described in Ssh Prioritization (iptables)).

Connection Diagram

...

The network used is very straightforward and consists of a single 1 Gb/s link between a testing machine (PC) and a WHLE board (whle_ls1046). Two iperf3 streams sending data from whle_ls1046 to PC will be competing for the link’s throughput. Different traffic classes will be used using the cgroups mechanism for the associated iperf3 processes and the resulting changes in data transfer speed will be observed.

Inc drawio
zoom1
simple0
custContentId579567620
pageId575668366
lbox1
diagramDisplayNamedirect_whle-pc_1G.drawio
pCentercontentVer1
hiResPreview0
revision1
baseUrlhttps://conclusive.atlassian.net/wiki
diagramNamedirect_whle-pc_1G.drawio
pCenter0
aspectrlxTkgMxotecUizovGCC 1
width891
linksauto
tbstyletop
isUpload1
height701

...

Code Block
root@whle-ls1046a:~# chmod +x cgroups-setup.sh
root@whle-ls1046a:~# ./cgroups-setup.sh

...

Iperf3 Setup

PC

Two iperf3 streams will be created, with servers launched on PC and clients on whle_ls1046, with the default client → server data flow direction.

...

Building on this a third, final procedure will be defined, which coordinates launching two iperf3 streams with different priorities, for the same time period, and the creation of mqprio qdisc.

whle_ls1046
Code Block
languagebash
root@whle-ls1046a:~#
test_iperf() {
    local port1=$1
    local prio1=$2
    local port2=$3
    local prio2=$4
    local iperf_time=$5
    kill_iperf "${port1}"
    kill_iperf "${port2}"
    tc qdisc del dev eth1 root handle 1:
    launch_iperf_with_priority "${port1}" "${prio1}" "${iperf_time}"
    launch_iperf_with_priority "${port2}" "${prio2}" "${iperf_time}"
    tc qdisc add dev eth1 root handle 1: mqprio num_tc 4 \
       map 0 0 0 0 1 1 1 1 2 2 2 2 3 3 3 3 hw 1
    sleep ${iperf_time}
}

The example usage will be given below.

Tests

Same priority

...

 1 2 2 2 2 3 3 3 3 hw 1
    sleep ${iperf_time}
}

The tc qdisc ... command is the same as the one used in https://conclusive.atlassian.net/wiki/spaces/CW/pages/580124673/Traffic+Control+with+tc#Example - see that article for detailed description.

Tests

Same priorities

Assuming that iperf3 servers at ports 5201, 5202 are running on PC, run the following command on WHLE:

whle_ls1046
Code Block
root@whle-ls1046a:~# test_iperf 5201 4 5202 4 6
Launching iperf3, port 5201, priority 4
[1] 735
Launching iperf3, port 5202, priority 4
[2] 738

This would create two iperf3 streams with the same skb priority 4, mapping to the traffic class 1. Meanwhile, on the PC side:

PC, console 1
Code Block
Accepted connection from 192.168.3.2, port 54202
[  5] local 192.168.3.1 port 5201 connected to 192.168.3.2 port 54208
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  55.0 MBytes   462 Mbits/sec                  
[  5]   1.00-2.00   sec  56.1 MBytes   471 Mbits/sec                  
[  5]   2.00-3.00   sec  56.1 MBytes   471 Mbits/sec                  
[  5]   3.00-4.00   sec  56.1 MBytes   471 Mbits/sec                  
[  5]   4.00-5.00   sec  56.1 MBytes   471 Mbits/sec                  
[  5]   5.00-6.00   sec  56.1 MBytes   471 Mbits/sec                  
[  5]   6.00-6.04   sec  2.46 MBytes   467 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-6.04   sec   338 MBytes   469 Mbits/sec                  receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
PC, console 2
Code Block
Accepted connection from 192.168.3.2, port 46446
[  5] local 192.168.3.1 port 5202 connected to 192.168.3.2 port 46458
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  53.9 MBytes   452 Mbits/sec                  
[  5]   1.00-2.00   sec  56.1 MBytes   471 Mbits/sec                  
[  5]   2.00-3.00   sec  56.1 MBytes   471 Mbits/sec                  
[  5]   3.00-4.00   sec  56.1 MBytes   471 Mbits/sec                  
[  5]   4.00-5.00   sec  56.1 MBytes   471 Mbits/sec                  
[  5]   5.00-6.00   sec  56.1 MBytes   471 Mbits/sec                  
[  5]   6.00-6.04   sec  3.69 MBytes   793 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-6.04   sec   338 MBytes   470 Mbits/sec                  receiver
-----------------------------------------------------------
Server listening on 5202
-----------------------------------------------------------

The experiment shows that the link’s throughput is shared evenly for traffic in the same class. Similar results would be obtained with calls:

Code Block
test_iperf 5201 0 5202 0 6
test_iperf 5201 8 5202 8 6
test_iperf 5201 12 5202 12 6

(That would cover all 4 traffic classes defined by tc, with skb priorities different from 0, 4, 8, 12 resulting in the same classes set.)

Different priorities

Run test_iperf with different skb priorities, making sure that they map to different traffic classes, for example:

whle_ls1046
Code Block
root@whle-ls1046a:~# test_iperf 5201 40 5202 4 6
Launching iperf3, port 5201, priority 40
[1] 735774
Launching iperf3, port 5202, priority 4
[2] 738

...

] 776

Meanwhile, on the PC side:

PC, console 1
Code Block
Accepted connection from 192.168.3.2, port 5420248344
[  5] local 192.168.3.1 port 5201 connected to 192.168.3.2 port 5420848350
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  558.084 MBytes  74.1 462 Mbits/sec                  
[  5]   1.00-2.00   sec  56.1 MBytes   471 Mbits/sec                  
[  5]   2.00-3.00   sec  560.100 MBytesBytes   471 Mbits0.00 bits/sec                  
[  5]   32.00-43.00   sec  560.100 MBytesBytes   471 Mbits0.00 bits/sec                  
[  5]   43.00-54.00   sec  560.100 MBytesBytes   471 Mbits0.00 bits/sec                  
[  5]   54.00-65.00   sec  560.100 MBytesBytes   471 Mbits0.00 bits/sec                  
[  5]   65.00-6.0400   sec  20.46 MBytes00 Bytes  4670.00 Mbitsbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-6.0408   sec  8.84 338 MBytes   46912.2 Mbits/sec                  receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
PC, console 2
Code Block
Accepted connection from 192.168.3.2, port 4644642704
[  5] local 192.168.3.1 port 5202 connected to 192.168.3.2 port 4645842720
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  53.9 100 MBytes   452839 Mbits/sec                  
[  5]   1.00-2.00   sec  56.1 112 MBytes   471942 Mbits/sec                  
[  5]   2.00-3.00   sec   56.1112 MBytes   471942 Mbits/sec                  
[  5]   3.00-4.00   sec  56.1 112 MBytes   471942 Mbits/sec                  
[  5]   4.00-5.00   sec  56.1 112 MBytes   471942 Mbits/sec                  
[  5]   5.00-6.00   sec  56.1 112 MBytes   471942 Mbits/sec                  
[  5]   6.00-6.04   sec  34.6971 MBytes   793937 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-6.04   sec   338666 MBytes   470925 Mbits/sec                  receiver
-----------------------------------------------------------
Server listening on 5202
-----------------------------------------------------------

The experiment This shows that the link’s throughput is shared evenly for traffic in the same classtraffic class 1 (skb priority 4) has a strict priority over traffic class 0 (skb priority 0). Similar results would be obtained with any of the calls:

Code Block
test_iperf 5201 0 5202 8 6
test_iperf 5201 0 5202 12 6
test_iperf 5201 84 5202 8 6
test_iperf 5201 4 5202 12 6
test_iperf 5201 8 5202 12 6

(That would cover all pairs of 4 traffic classes defined by tc, with skb priorities different from 0, 4, 8, 12 resulting in one of the same classes setpairs from above.)