Kernel DPAA drivers

Owned by Marcin Wcisło
Last updated: Oct 31, 2024
2 min read

About

This section describes steps to replicate WHLEs connection setup showcasing the ability of WHLE-LS1046A board to serve as a router/bridge device providing throughput at 10 Gb/s level using standard DPAA kernel drivers.

Common setup for all WHLE boards

System

Conclusive Ubuntu image was used, version ubuntu-jammy-whle-ls1046a-2024-05-15-6ccb862.img, available at https://gitlab.conclusive.pl/devices/ubuntu-build/-/packages/109. For the flashing procedure please refer to Installing Ubuntu image.

Kernel

In tests the kernel version 6.5.6-32773-whle-ls1-g1dbc2136b533 was used, although any recent Conclusive kernels 6.5 or 6.1 should work.

No additional kernel arguments are needed, for the tests the default arguments were used:

root@whle-ls1046a:~# cat /proc/cmdline root=UUID=4e117e32-2acc-400d-8d97-915b6c99c41e console=ttyS0,115200 earlycon=uart8250,mmio,0x21c0500 rootwait rw

Software

root@whle-ls1046a:~# apt-get update root@whle-ls1046a:~# apt-get install -q0 iperf3 bridge-utils

DPAA Drivers

The tests were carried out using iperf3 tool, TCP protocol, with servers on whle_ls1046_0 and clients on whle_ls1046_2, with the direction of the main network flow being whle_ls1046_2 → whle_ls1046_1 → whle_ls1046_0. The overall speed of transfer depends on the speed at which whle_ls1046_1 and whle_ls1046_2 are able to receive data, which is bound by CPU frequency and depends on how the traffic flows are shared between cores. To achieve optimal bandwidth it's necessary to (for each of whle_ls1046_1 and whle_ls1046_2) bind the specific iperf3 flow to a single core and to make sure the bindings don't overlap.

Kernel DPAA driver allows some limited traffic control with the help of ethtool (https://docs.kernel.org/networking/device_drivers/ethernet/freescale/dpaa.html). Core-binding is achieved by using Receive Side Scaling feature, enabled on 10G ports with

root@whle-ls1046a:~# for dev in eth4 eth5; do ethtool -N ${dev} rx-flow-hash tcp4 sfdn; done

The RSS is enabled by default and the command above shouldn't be necessary. The sfdn argument specifies the packet header fields defining the flow and used to map it to a specific core.

  • s: Source IP address

  • f: Destination IP address

  • d: Destination port

  • n: Source port.

The current set of fields used in RSS for the given network device can be printed with

The exact core the flow will be bound to, being based on the hash value from the above fields, is not known in advance, although it's deterministic.

Unfortunately it's apparently not possible to reduce these four fields - using any of s, f, d, n in the ethtool call enables all the other three.