Building OpenWrt with the Yocto

Project
Yocto Summit 2021

Tomasz Żyjewski

1 / 34
 Agenda

whoami
Who we are?
OpenWrt description
Advantages of using the Yocto Project
meta-openwrt overview
HW - short description
Description of tests
Boot OpenWrt community version
Build OpenWrt with meta-openwrt
Boot meta-openwrt image
Summary
Q&A

Yocto Summit 2021

CC BY | Tomasz Żyjewski 2 / 34
 whoami

Tomasz Żyjewski
Embedded Systems Engineer

@tomzy_0 2 years in 3mdeb

tomasz.zyjewski@3mdeb.com Integration of update systems and
OS creation for embedded devices
interested in:
Yocto Project
OS updates
boot-time optimization

Yocto Summit 2021

CC BY | Tomasz Żyjewski 3 / 34
 Who we are?

coreboot licensed service providers since 2016

coreboot project leadership participants
UEFI Adopters since 2018
Official consultants for Linux Foundation fwupd/LVFS project
Yocto Participants and Embedded Linux experts
Open Source Firmware enthusiasts and evangelists

Yocto Summit 2021

CC BY | Tomasz Żyjewski 4 / 34
 OpenWrt description

Highly extensible GNU/Linux distribution for embedded devices

primarly used to route network traffic

Fully writable filesystem with optional package management

Project started in 2004, first release in 2006

fun fact: releases were historically named after cocktails, such as

White Russian, Kamikaze
naming scheme droped after remerge of the LEDE subgroup and
OpenWrt

Support over 1700 different hardware

for newer releases (18.06 or later) it is recommended that device

has at least 16 MB of flash memory and 64 MB of RAM (128 MB of
RAM is preffered)

Yocto Summit 2021

CC BY | Tomasz Żyjewski 5 / 34
 OpenWrt description

Why use OpenWrt?

extensibility - over 3000+ standarized applications, can replicate
setup on any supported device
security - secury by default, keeping software components up-to-
date
performance and stability - each module receive a lot of testing and
bug fixing
strong community support - OpenWrt forum and mailing lists
research - lots of network performance research are executed on
OpenWrt systems, every successful expirements are available in
OpenWrt first
open source - no additional cost needed, OpenWrt is provided
without any monetary cost

Yocto Summit 2021

CC BY | Tomasz Żyjewski 6 / 34
 OpenWrt description

Releases

current stable release: OpenWrt 19.07 (latest version 19.07.7

released on 18 Feb 2021)
next stable release: OpenWrt 21.02 (21.02-rc1 released on April,
26th 2021)

No stable release cycle

breaks between releases last from a month to 20 months

Stable version numbers are mode from the year and the month when a
new stable branch was created

Table of Hardware contains the release version that supports the device

every release is available on the OpenWrt webpage

Yocto Summit 2021

CC BY | Tomasz Żyjewski 7 / 34
 OpenWrt description

Packages

OpenWrt provides several thousand packages

there is a package table which always show packages available in
the latest update of the stable release

Packages can be installed in two ways

via web interface and LuCI

via command-line interface and opkg packages manager

Packages source code is also available

source code of some of the OpenWrt packages like LuCI or procd

community repository of ported packages
base OpenWrt repository, contains patches for basic packages

Lots of patched packages, every repository has branches for supported

releases

Yocto Summit 2021

CC BY | Tomasz Żyjewski 8 / 34
 OpenWrt description

OpenWrt build system, known as OpenWrt Buildroot

based on modified Buildroot system

set of Makefiles and patches that automates the process of
building

Uses kconfig for the configuration of all options

Provides an integrated cross-compiler toolchain

Handles standard OpenWrt image build workflow: downloading, patching,

configuration, compilation and packaging

Provides a number of common fixes for known badly behaving packages

Yocto Summit 2021

CC BY | Tomasz Żyjewski 9 / 34
 Advantages of using the Yocto Project

Better management of the components that make up the system

division into layers

Most hardware vendors provide direct support for Yocto

support for a new machine is easier to achieve by adding the

appropriate bsp layer to the build

Yocto Project allows to prepare more complex systems

extending the functionality of the system by adding layers

Yocto Summit 2021

CC BY | Tomasz Żyjewski 10 / 34
 meta-openwrt overview

Repository - https://github.com/kraj/meta-openwrt

Maintainer - Khem Raj

3 branches - master, dunfell, hardknott

master branch should be compatible with gatesgarth and hardknott

LAYERSERIES_COMPAT_openwrt-layer = "gatesgarth hardknott"

19 contributors, last contributions

Yocto Summit 2021

CC BY | Tomasz Żyjewski 11 / 34
 meta-openwrt overview

README.md review

kind of outdated, not needed dependencies on meta-nodejs and

meta-nodejs-contrib
to build an image openwrt-distro-defaults needs to be add to the
INHERIT variable
consideration about TCLIBC - it is not pointed if using musl is a must
OE release limitations - it should work with Sumo version and later

Available images

openwrt-image-minimal - has openwrt networking and cli but no UI

openwrt-image-base - has openwrt networking, cli, and UI (LuCI)
openwrt-image-full - has minimal and base images functionality
plus some network related packages, including relayd or tcpdump

Yocto Summit 2021

CC BY | Tomasz Żyjewski 12 / 34
 meta-openwrt overview

Recipes

$ ls meta-openwrt/recipes-*/
meta-openwrt/recipes-core/:
firewall3 fstools images iwinfo jsonpath libubox make-ext4fs netifd
odhcp6c odhcpd packagegroups procd rpcd ubox ubus uci uclient
ustream-ssl xtables-addons
meta-openwrt/recipes-extended/:
images libnl-tiny libroxml luci packagegroups ugps usbmode usign
meta-openwrt/recipes-kernel/:
linux

meta-openwrt/recipes-networking/:
ipset relayd uhttpd umbim umdnsd uqmi

meta-openwrt/recipes-support/:
lua lua-socket

meta-openwrt/recipes-tweaks/:
base-files busybox dnsmasq hostapd iptables modutils-initscripts packagegroups udev

Yocto Summit 2021

CC BY | Tomasz Żyjewski 13 / 34
 meta-openwrt overview

Lots of core recipes use source code from OpenWrt projects repositories

each recipe has its own revision from which it retrieves the code, it
should be remembered when the OpenWrt version is updated

Installing OpenWrt specific configurations

some recipes like netifd use also OpenWrt Github repository to

install needed scripts or configuration files

Recipes from recipes-tweaks are fetched from default OE source, but

tweaked for OpenWrt by installing additional files and scripts

looks like some of them still needs a lot of patches e.g. hostapd

Yocto Summit 2021

CC BY | Tomasz Żyjewski 14 / 34
 meta-openwrt overview

List of provided bbclasses

$ tree meta-openwrt/classes/
meta-openwrt/classes/
├── openwrt-base-files.bbclass
├── openwrt.bbclass
├── openwrt-distro-defaults.bbclass
├── openwrt-kmods.bbclass
├── openwrt-lua.bbclass
├── openwrt-services.bbclass
└── openwrt-virtual-runtimes.bbclass

Inherited openwrt-distro-defaults sets procd as init manager, sets up the

DISTRO_FEATURES and inherits another bbclasses

openwrt-base-files adds OpenWrt github repository to couple of recipes

(dnsmasq, hostapd, netifd, uci) and than it is used to install OpenWrt specific
init scripts or configuration files

on master branch it use dd3464023f18 git revision which is near

v19.07.5

Yocto Summit 2021

CC BY | Tomasz Żyjewski 15 / 34
 meta-openwrt overview

Issues review
there are 12 open issues
some of them are open since 2017, some of them are couple
months old
the maintainer is responsive and open to any PR improving the
state of the layer

Yocto Summit 2021

CC BY | Tomasz Żyjewski 16 / 34
 HW - short description

Raspberry Pi 4 ver. B
SoC: Broadcom BCM2711
RAM: 4GB
WLAN: Cypress CYW43456
Ethernet: 1 Gbit port

Powerful platform, nice for learning

OpenWrt releases officially support previous versions of RPi

Yocto Summit 2021

CC BY | Tomasz Żyjewski 17 / 34
 Description of tests

Goals to achieve with this proof of concept

log to the device via serial console

read system logs
connect with Ethernet cable, get IP address
log to the device via SSH
check data on LuCI
change configuration via LuCI, e.g. root password

Basic features that allows to continue development on the system

many topics on the Internet propose to start working with the

OpenWrt system by logging in via SSH or changing the
configuration in LuCI
the ability to read logs allows to analyze and solve problems

Yocto Summit 2021

CC BY | Tomasz Żyjewski 18 / 34
 Description of tests

The reason for the presentation and testing of meta-openwrt

I am more familiar with Yocto than Buildroot and would prefer to

set up my systems for embedded devices using the first one
preparing an OpenWrt image with Yocto will allow to get to know
most of the elements inside target system

Helping each other

many users may come across the meta-openwrt layer in

OpenEmbedded Layer Index
raising interest in the meta layer

Yocto Summit 2021

CC BY | Tomasz Żyjewski 19 / 34
 Boot OpenWrt community version

For RPi 4 there is no stable version of OpenWrt released

Can use snapshot version

community driven version, new image every few days

discussion in the OpenWrt topic
for this presentation image rpi-4_snapshot_3.1.57-32_r16707_extra
was used

Booting OpenWrt image allow to make comparison

Yocto Summit 2021

CC BY | Tomasz Żyjewski 20 / 34
 Boot OpenWrt community version

Boot log

[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]

[ 0.000000] Linux version 5.4.117 (builder@buildhost) \
(gcc version 8.4.0 (OpenWrt GCC 8.4.0 r16707-e57e460dc7)) #0 SMP Mon May 10 12:41:06 2021
[ 0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
(...)
BusyBox v1.33.0 () built-in shell (ash)

.--. .- ..-. .-. /

/ )`-'.-. .-.. .-. ) ( ).--.---/---
/ / / )./.-'_)/ ) / \ / /
( / /`-' (__.''/ ( ( . )/ /
`-.' / `-`-' `-'
rpi-dca632b6ec RaspberryPi4ModelBRev1.2 bl:Apr162020 vl805:000137ad
3.1.57-32 r16707 unknown SNAPSHOT
root:mmcblk0p2:05141719-02 boot:mmcblk0p1 cmdL:PARTUUID=05141719-02
t:2021-05-10 12:43:23 l:0.14,0.12,0.05 @up00:02:26
rootfs:944.9MB/25.1% boot:383.8MB/6.7% mem:3.7GB/59.6MB
LAN: 192.168.1.1/24(br-lan), dhcp: 192.168.1.100-249, dhcp.leases: 0
MonMay1012:42:592021 to send to ff02::1%lan@br-lan (Address not available)
MonMay1012:43:152021 to send to ff02::1%lan@br-lan (Address not available)
MonMay1012:43:202021 plugin: exec_read_one: error = Cannot find device "eth1"
update-unavailable
=== WARNING! =====================================
no root password: Use the "passwd" command
=== WARNING! =====================================
root@rpi-dca632b6ec /39#

Yocto Summit 2021

CC BY | Tomasz Żyjewski 21 / 34
 Boot OpenWrt community version

Examinate system logs

use logread from busybox

# logread -f -l 5
Mon May 10 12:42:21 2021 daemon.info urandom_seed[17614]: Seed saved (/etc/urandom.seed)
Mon May 10 12:42:22 2021 daemon.info dnsmasq[16440]: read /etc/hosts - 4 addresses
Mon May 10 12:42:22 2021 daemon.info dnsmasq[16440]: read /tmp/hosts/odhcpd - 1 addresses
Mon May 10 12:42:22 2021 daemon.info dnsmasq[16440]: read /tmp/hosts/dhcp.cfg01411c - 2 addresses
Mon May 10 12:42:22 2021 daemon.info dnsmasq-dhcp[16440]: read /etc/ethers - 0 addresses

Connect to the local PC via Ethernet cable

get IP address right away

$ ifconfig eno1
eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.105 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fd1d:bf:8d15:0:85ac:98c6:8e4c:89bd prefixlen 64 scopeid 0x0<global>
inet6 fd1d:bf:8d15:0:6e9c:c0fe:e686:8c28 prefixlen 64 scopeid 0x0<global>

Yocto Summit 2021

CC BY | Tomasz Żyjewski 22 / 34
 Boot OpenWrt community version

Connect to the RPi4 gateway via SSH

BusyBox v1.33.0 () built-in shell (ash)

| |. .| | | |. |_ .
| - || _ | -__| || | | || _|| _
_____|| __|_____|__|__||________||__| |____|
|__|

rpi-dca632b6ec RaspberryPi4ModelBRev1.2 bl:Apr162020 vl805:000137ad

3.1.57-32 r16707 ssh SNAPSHOT 192.168.1.105 59768 22 /dev/pts/0
root:mmcblk0p2:05141719-02 boot:mmcblk0p1 cmdL:PARTUUID=05141719-02
t:2021-05-10 12:43:04 l:0.20,0.12,0.04 @up00:02:06
rootfs:944.9MB/25.1% boot:383.8MB/6.7% mem:3.7GB/59.9MB
LAN: 192.168.1.1/24(br-lan), dhcp: 192.168.1.100-249, dhcp.leases: 1
1620693690 34:17:eb:d7:a5:9f 192.168.1.105 tomzy-OptiPlex-7010 01:34:17:eb:d7:a5:9f
1620693742 00:00:00:00:00:00 0.0.0.0 tomzy-OptiPlex-7010 fdad:74b2:130f::a0f/128
MonMay1012:42:452021 plugin: exec_read_one: error = Cannot find device "eth1"
update-unavailable
=== WARNING! =====================================
no root password: Use the "passwd" command
=== WARNING! =====================================
root@rpi-dca632b6ec /38#

Yocto Summit 2021

CC BY | Tomasz Żyjewski 23 / 34
 Boot OpenWrt community version

Open LuCi in web browser

Yocto Summit 2021

CC BY | Tomasz Żyjewski 24 / 34
 Boot OpenWrt community version

Change root password via LuCI

Yocto Summit 2021

CC BY | Tomasz Żyjewski 25 / 34
 Build OpenWrt with meta-openwrt

Needed layers to build meta-openwrt based image for Raspberry Pi 4

meta-openwrt
poky
meta-raspberrypi
meta-openembedded: meta-oe, meta-python and meta-networking

Used refspecs

poky:
url: https://git.yoctoproject.org/git/poky
refspec: 6a751048e50c00261d99c2d8d69534f7a8da38a9

meta-openembedded:
url: https://git.openembedded.org/meta-openembedded
refspec: f3f7a5f1a4713f145107bb043e0d14cb3a51c62f
meta-openwrt:
url: https://github.com/kraj/meta-openwrt.git
refspec: 6e8159a07ce8991cb6b04e3cb15f82b9eadad1e5

meta-raspberrypi:
url: git://git.yoctoproject.org/meta-raspberrypi
refspec: 3ae135e590e375c8da26b003bda41c18fb977ae1

Yocto Summit 2021

CC BY | Tomasz Żyjewski 26 / 34
 Build OpenWrt with meta-openwrt

According to the documentation

TCLIBC should point to musl

openwrt-distro-defaults should be add to the INHERIT variable

Additionaly the ENABLE_UART variable can be set to 1

enable UART on RPi

used in meta-raspberrypi

Tweaks needs to be added to local.conf

kas layer management tool could be used to manage that

layer management tools described at Yocto Summit 2020

Yocto Summit 2021

CC BY | Tomasz Żyjewski 27 / 34
 Boot meta-openwrt image

Boot log

[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]

[ 0.000000] Linux version 5.10.17-v8 (oe-user@oe-host) \
(aarch64-ys-linux-musl-gcc (GCC) 10.2.0, GNU ld (GNU Binutils) 2.35.1) \
#1 SMP PREEMPT Mon Mar 1 09:44:55 UTC 2021
[ 0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
(...)
Please press Enter to activate this console.
[ 6.340173] kmodloader: loading kernel modules from /etc/modules.d/*
[ 6.385165] NET: Registered protocol family 10
[ 6.391193] Segment Routing with IPv6
[ 6.408153] bridge: filtering via arp/ip/ip6tables is no longer available by default. \
Update your scripts to load br_netfilter if you need this.
[ 6.514195] xt_time: kernel timezone is -0000
[ 6.557067] kmodloader: done loading kernel modules from /etc/modules.d/*

Distro for Yocto Summit 2021 0.0.1 \n \l

root@(none):/#

Yocto Summit 2021

CC BY | Tomasz Żyjewski 28 / 34
 Boot meta-openwrt image

Examinate system logs

use logread from busybox

# logread -f -l 5
Failed to find log object: Not found
Failed to find log object: Not found
Failed to find log object: Not found

Connect to the local PC via Ethernet cable

get IP address right away

$ ifconfig eno1
eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::f0d:a471:f7ae:6289 prefixlen 64 scopeid 0x20<link>
ether 34:17:eb:d7:a5:9f txqueuelen 1000 (Ethernet)
(...)
$ ping 192.168.1.1
ping: connect: Network is unreachable

Yocto Summit 2021

CC BY | Tomasz Żyjewski 29 / 34
 Boot meta-openwrt image

Checking SSH connection is not possible

networking not starting on the RPi /lib/netifd/mac80211.sh is

missing
IP address is not given by the gateway
DHCP server may not work correctly on the meta-openwrt based
image

Without IP address accessing LuCI is also impossible

Running network script needed lock directory under /run/lock

adding it allows to read system logs

Yocto Summit 2021

CC BY | Tomasz Żyjewski 30 / 34
 Boot meta-openwrt image

System logs analyze

dnsmasq issue

Fri Mar 9 12:35:19 2018 daemon.crit dnsmasq[1160]: Ubus not available: \

set HAVE_UBUS in src/config.h
Fri Mar 9 12:35:19 2018 daemon.crit dnsmasq[1160]: FAILED to start up

uhttpd problems

Fri Mar 9 12:34:59 2018 daemon.notice procd: /etc/rc.d/S50uhttpd: \

Skipping invalid Lua prefix "/cgi-bin/luci=/usr/lib/lua/luci/sgi/uhttpd.lua"

Missing some packages

collectd
dropbear

Yocto Summit 2021

CC BY | Tomasz Żyjewski 31 / 34
 Summary

OpenWrt vs meta-openwrt image comparison

Functionality OpenWrt community image meta-openwrt image

accessing via console working working

read system logs working not working

receiving an IP address working not working

accessing via SSH working not working

accessing LuCI working not working

setting root password via LuCI working not working

Lots of things do not work out of the box

this is a good initial set of functionality to check

solving the existing problems will allow for further development

We will try to make improvements to meta-openwrt gradually

does not look usable for now

Yocto Summit 2021

CC BY | Tomasz Żyjewski 32 / 34
 Contact us

We are open to cooperate and discuss

contact@3mdeb.com

facebook.com/3mdeb

@3mdeb_com

linkedin.com/company/3mdeb

https://3mdeb.com

Book a call

Sign up for the newsletter

Feel free to contact us if you believe we can help you in any way. We are always
open to cooperate and discuss.

Yocto Summit 2021

CC BY | Tomasz Żyjewski 33 / 34
 Q&A

Yocto Summit 2021

CC BY | Tomasz Żyjewski 34 / 34