DPDK

Owned by Michal Podziomek (Unlicensed)
Last updated: Jul 25, 2023 by Marcin Jabrzyk

Table of Contents:

 

Data Plane Development Toolkit

DPDK, the Data Plane Development Toolkit, is an open source project of the Linux Foundation that provides offloading of TCP packet processing from system kernel to the the user space. This significantly increases performance.

WHLE-LS1 uses DPDK as the software layer of DPAA and DPAA2 (Data Path Acceleration Architecture) feature of the Layerscape SoCs.


Prerequisites

Required tools & libraries to build the kernel & DPDK:

libxml2-dev gcc g++ make meson bison flex ninja-build libssl-dev git bc cmake pkg-config libnuma-dev

Kernel

Download & build the the 4.19 kernel.

There are two FDT files configured for USDPAA:

  • freescale/fsl-ls1046a-whle-usdpaa.dtb

  • freescale/fsl-ls1046a-whle-usdpaa-2.dtb

First one sets all of the ethernet/SFP ports up for userspace, second one leaves one ethernet port for kernel use.

Booting

Set bootargs in U-Boot to setup hugepages, DPAA manager portals, isolate Linux so it runs on a single core & leave the other cores for DPDK:

env set bootargs console=ttyS0,115200 root=/dev/mmcblk0p1 earlycon=uart8250,mmio,0x21c0500 default_hugepagesz=1024m hugepagesz=1024m hugepages=4 isolcpus=1-3 bportals=s0 qportals=s0 iommu.passthrough=1 run mmc_boot

Building DPDK

In order for DPDK to work correctly with WHLE-LS1 we will have to build it with the DPAA driver & the rte_kni kernel module, that means we have to supply the path to 4.19 kernel headers & source files for DPDK. Newer versions of DPDK use meson/ninja for the build process, so make sure these are available on your build machine.

Keep in mind DPDK expects a specific directory structure for the kernel headers/source in order for the build process to finish correctly:

  1. Export 4.19 kernel headers to a certain directory, which we're going to call KERNEL_PATH.

  2. At KERNEL_PATH create a symlink called build which links to the kernel source directory KERNEL_SRC.

  3. You'll need to do a full kernel build once before running a DPDK build, else rte_kni part will fail.

Expected KERNEL_PATH directory tree looks like this:

KERNEL_PATH: build -> KERNEL_SRC {Kernel headers}

Links:

Build on target device

  • KERNEL_PATH is the kernel headers directory, as explained above.

  • By default DPDK builds with a debug-optimized build, which contains only partial debug information. If you want a full debug build, add -Dbuildtype=debug . Keep in mind this will drastically increase binary sizes up to 70MB+ for a single example app, so you might encounter space problems on the target device!

  • Add -Dexamples=all to build all the example dpdk apps, or -Dexamples=l2fwd,ipsec-secgw,etc to build only specific ones.

Prerequisites for running DPDK applications

  1. Download dpdk-extras and put rte_kni.ko from the DPDK build folder in dpdk-extras/dpaa

  2. Install fmc either by:
    - Using  make install in dpdk-extras/fmc
    - or build & install from source

  3. Run dpdk-extras/dpaa/dpdk_env.sh

Example applications

We're assuming your device has at least one Ethernet port configured in the FDT for userspace. If you're not using the fm1-mac9 and fm1-mac10 (SFP) ports, you'll most probably have to block them due to DPDK applications quitting abnormally if they're not connected. For testing purposes you might also want to create a TAP kernel virtual network device.

Links:

l2fwd

This will run the L2 forwarding app setup to forward all packets between adjacent ports, e.g. 0 and 1.

l2fwd-crypto

This is analogous to l2fwd with a crypto operation specified from the command line ran on each packet forwarded.

ipsec-secgw

dpdk-ethtool

Allows you to query and set parameters of userspace controlled NICs.

 

If you’re looking for more information please contact us at https://conclusive.pl/contact/