Setup

For the test a single WHLE board was used, identified as whle_ls1046_1, and a PC equipped with a network card having two 10G ports. A single PC machine was set up to emulate two separate machines serving as endpoints for the WHLE board to route between. Please note that the actual speed test results depend on the CPU power of the PC, which, for the tests to measure only WHLE performance undisturbed, must be able to handle traffic at both ends at level exceeding the ability of a single WHLE board, with a good margin. In the carried out tests the 12-core 4.5 GHz x86_64 machine was more than sufficient for the task.

Before proceeding please make sure you followed the common setup described in Kernel DPAA drivers .

Router

Connection diagram

Connection speed from ens1f1 interface to ens1f0 on PC will be measured, with whle_ls1046_1 configured as a router. The isolated_ns denotes network namespace in which the ens1f0 interface had to be enclosed to force PC to send the packets through whle_ls1046_1 instead of short-circuiting to the local interface.

Open

Network Setup

PC

whle_ls1046_1

Tests

Iperf servers

On PC launch four instances of iperf3 servers, listening on ports 5201-5204. The ip netns exec command requires root access.

PC

Iperf clients

Launch four instances of iperf3 simultaneously.

PC

The port numbers are picked in such a way that every iperf3 flow is handled by a different core on whle_ls1046_1 - cores 0, 2, 1, 3, respectively. (The iperf3 calls fix the source port used for the data transfer connection (the --cport parameter). There is a small chance some of them are already used in the system. In this case it’s necessary to locate these processes with netstat -tnp and kill them.)

Stop all the clients after some time.

PC

Sum the values from the lines with "sender" at the end.

The total bandwidth achieved is 2.29 + 2.37 + 2.33 + 2.35 = 9.34 Gb/s. This is the upper limit for the TCP protocol on a 10 Gb/s physical link, proving that WHLE-LS1046A board is able to handle routing at its network interface's limit using standard kernel drivers.

WHLE work analysis

Consider the snapshot from the top command ran on whle_ls1046_1 during the performance test:

Open

The si column shows the CPU time spent in software interrupts, in this case the network interrupts almost exclusively. Nearly zero time spent by the system or user shows that the routing task is carried out in the interrupts alone. The even spreading of load at ~73% between all cores stems from picking the right parameters (ip source address, ip dest address, tcp source port, tcp dest port) defining four data flows assigned by driver's RSS to four separate CPUs. The idle time id at ~25% shows that WHLE operates at 75% capacity, providing a decent margin to account for more realistic routing tasks, with bigger routing tables and less than perfectly CPU-even traffic.

L2 Bridge

Connection diagram

Network Setup

PC

whle_ls1046_1

Tests

Iperf servers

On PC launch four instances of iperf3 servers, listening on ports 5201 to 5204. The ip netns exec command requires root access.

PC

Iperf clients

Run four clients simultaneously:

PC

The addresses and ports are picked in such a way that each iperf3 flow is handled by a different core on whle_ls1046_1 - cores 3, 1, 0, 2, respectively.

Stop all the clients after some time.

Sum the values from the lines with "sender" at the end

The total bandwidth achieved is 2.45 + 2.28 + 2.25 + 2.37 = 9.35 Gb/s. This is the upper limit for the TCP protocol on a 10 Gb/s physical link, proving that WHLE-LS1046A board is able to handle bridging at network interface's limit using standard kernel drivers.

WHLE work analysis

Consider the snapshot from the top command ran on whle_ls1046_1 during the performance test:

Just like in the case of router (Router/Bridge Mode: PC + WHLE Setup | WHLE work analysis) the only meaningful columns are id (idle) and si (software interrupt). Unlike with the router, however, the CPU load in the bridge mode has a high variance and thus a single top command snapshot can be misleading. It’s useful to record the numbers for a minute or so:

whle_ls1046_1

Plotting them, along with the averages, would obtain a graph similar to this one:

From this graph it’s clear that every core’s idle time oscillates at 30% average, leaving healthy margin to account for more realistic bridging scenarios with less than perfectly CPU-even traffic.