I love t-shirts, especially those that you’d call logowear. But it’s not the kind of big name fashion logos that I’m referring to. Rather, it’s logowear from my favorite IT companies. I have well over a hundred of these t-shirts, and except when I’m preparing for a special event, I pull a random t-shirt from my collection. Yesterday I happened to wear a power.org t-shirt, while today I’m wearing an OpenPOWER t-shirt, two POWER t-shirts in two days :-) Both of these brought back some nice memories.

It's been a while since I've posted about arm64 hardware. The last machine I spent my own money on was a SolidRun Macchiatobin, about 7 years ago. It's a small (mini-ITX) board with a 4-core arm64 SoC (4 * Cortex-A72) on it, along with things like a DIMM socket for memory, lots of networking, 3 SATA disk interfaces.

The Macchiatobin was a nice machine compared to many earlier systems, but it took quite a bit of effort to get it working to my liking. I replaced the on-board U-Boot firmware binary with an EDK2 build, and that helped. After a few iterations we got a new build including graphical output on a PCIe graphics card. Now it worked much more like a "normal" x86 computer.

I still have that machine running at home, and it's been a reasonably reliable little build machine for arm development and testing. It's starting to show its age, though - the onboard USB ports no longer work, and so it's no longer useful for doing things like installation testing. :-/

So...

I was involved in a conversation in the #debian-arm IRC channel a few weeks ago, and diederik suggested the Radxa Rock 5 ITX. It's another mini-ITX board, this time using a Rockchip RK3588 CPU. Things have moved on - the CPU is now an 8-core big.LITTLE config: 4*Cortex A76 and 4*Cortex A55. The board has NVMe on-board, 4*SATA, built-in Mali graphics from the CPU, soldered-on memory. Just about everything you need on an SBC for a small low-power desktop, a NAS or whatever. And for about half the price I paid for the Macchiatobin. I hit "buy" on one of the listed websites. :-)

A few days ago, the new board landed. I picked the version with 24GB of RAM and bought the matching heatsink and fan. I set it up in an existing case borrowed from another old machine and tried the Radxa "Debian" build. All looked OK, but I clearly wasn't going to stay with that. Onwards to running a native Debian setup!

I installed an EDK2 build from https://github.com/edk2-porting/edk2-rk3588 onto the onboard SPI flash, then rebooted with a Debian 12.7 (Bookworm) arm64 installer image on a USB stick. How much trouble could this be?

I was shocked! It Just Worked (TM)

I'm running a standard Debian arm64 system. The graphical installer ran just fine. I installed onto the NVMe, adding an Xfce desktop for some simple tests. Everything Just Worked. After many years of fighting with a range of different arm machines (from simple SBCs to desktops and servers), this was without doubt the most straightforward setup I've ever done. Wow!

It's possible to go and spend a lot of money on an Ampere machine, and I've seen them work well too. But for a hobbyist user (or even a smaller business), the Rock 5 ITX is a lovely option. Total cost to me for the board with shipping fees, import duty, etc. was just over £240. That's great value, and I can wholeheartedly recommend this board!

The two things that are missing compared to the Macchiatobin? This is soldered-on memory (but hey, 24G is plenty for me!) It also doesn't have a PCIe slot, but it has sufficient onboard network, video and storage interfaces that I think it will cover most people's needs.

Where's the catch? It seems these are very popular right now, so it can be difficult to find these machines in stock online.

FTAOD, I should also point out: I bought this machine entirely with my own money, for my own use for development and testing. I've had no contact with the Radxa or Rockchip folks at all here, I'm just so happy with this machine that I've felt the need to shout about it! :-)

Here's some pictures...

The September syslog-ng newsletter is now available: Improved FreeBSD and MacOS support in 4.8.0 Setting the version number in the syslog-ng configuration Switching containers from Debian Testing to Stable You can read it at: https://www.syslog-ng.com/community/b/blog/posts/the-syslog-ng-insider-2024-10-4-8-0-release-version-number-debian-stable syslog-ng logo

Version 4.8.1 of syslog-ng was released last week. It is a bugfix release, and it contains fixes for problems also reported by members of the Fedora community. The Fedora 41 release is near, so package updates now need some additional testing, and “karma” in Bodhi. You can find information on how to install syslog-ng 4.8.1 from a testing repo on Fedora 41 beta at https://bodhi.fedoraproject.org/updates/FEDORA-2024-4e812b8a23. This is also the place where you can provide feedback and karma.

This weekend I visited the first Audio Expo in Budapest. It was the first music event I truly enjoyed in years. Even if corridors and rooms were packed, there was enough fresh air. What sets this event apart from other events is the focus on listening to music on the vendors’ products rather than just the speeds and feeds on why you should buy their products. While, of course, the expected outcome is the same, with the emphasis on listening to live systems, I found the event much more comfortable to walk around.

Two weeks ago, I was at EuroBSDcon and received a feature request for syslog-ng. The user wanted to collect FreeBSD audit logs together with other logs using syslog-ng. Writing a native driver in C is time consuming. However, creating an integration based on the program() source of syslog-ng is not that difficult. This blog shows you the current state of the FreeBSD audit source, how it works, and its limitations. It is also a request for feedback.

Recently I was at EuroBSDCon, where several participants recognized that I am a POWER guy. And they were right, I have been an IBM POWER Champion focusing on open source software on POWER for the past three years. Talos II POWER9 mainboard I got the usual question from people: is there anyone working on an affordable and open source friendly POWER machine? My answer was a definite yes, but also had to admit that I do not know the actual status for any of the projects.

EuroBSDCon was fantastic, as always :-) I talked to many interesting people during the four days about sudo and syslog-ng, and of course also about many other topics. I gave a sudo tutorial, and it went well, with some “students” already planning which features to implement at home. There were many good talks, including one from Dr. Marshall Kirk McKusick, who was with the FreeBSD project right from the beginning, and worked on BSD even earlier.

Last week I wrote about a campaign that we started to resolve issues on GitHub. Some of the fixes are coming from our enthusiastic community. Thanks to this, there is a new syslog-ng-devel port in MacPorts, where you can enable almost all syslog-ng features even for older MacOS versions and PowerPC hardware. Some of the freshly enabled modules include support for Kafka, GeoIP or OpenTelemetry. From this blog entry, you can learn how to install a legacy or an up-to-date syslog-ng version from MacPorts.

It (was) that time of year again - last weekend we hosted a bunch of nice people at our place in Cambridge for the annual Debian UK OMGWTFBBQ!

Lots of friends, lots of good food and drink. Of course lots of geeky discussions about Debian, networking, random computer languages and... screws? And of course some card games to keep us laughing into each night!

Many thanks to a number of awesome friendly people for again sponsoring the important refreshments for the weekend. It's hungry/thirsty work celebrating like this!

• Codethink
• Mythic Beasts

Warning: If you're not into meat, you might want to skip the rest of this...

This year, I turned 50. Wow. Lots of friends and family turned up to help me celebrate, with a BBQ (of course!). I was very grateful for a lovely set of gifts from those awesome people, and I have a number of driving experiences to book in the next year or so. I'm going to have so much fun driving silly cars on and off road!

However, the most surprising gift was something totally different - a full-day course of hands-on pork butchery. I was utterly bemused - I've never considered doing anything like this at all, and I'd certainly never talked to friends about anything like it either. I was shocked, but in a good way!

So, two weekends back Jo and I went over to Empire Farm in Somerset. We stayed nearby so we could be ready on-site early on Sunday morning, and then we joined three other people doing the course. Jo was there to observe, i.e. to watch and take (lots of!) pictures.

I can genuinely say that this was the most fun surprise gift I've ever received! David Coldman, the master butcher working with us, has been in the industry for many years. He was an excellent teacher, showing us everything we needed to know and being very patient with us when we needed it. It was great to hear his philosophy too - he only uses the very best locally-sourced meat and focuses on quality over quantity. He showed us all the different cuts of pork that a butcher will make, and we were encouraged to take everything home - no waste here!

At the beginning of the day, we each started with half a pig. Over the next several hours, we steadily worked our way through a series of cuts with knife and saw, making the remaining pig smaller and smaller as we went.

We finished the day with three sets of meat. First, a stack of vacuum-packed joints, chops and steaks ready for cooking and eating at home. Second: a box of off-cuts that we minced and made into sausages at the end of the day. Finally: a bag of skin and bones. Our friend's dog got some of the bones, and Jo turned a lot of the skin into crackling that we shared with friends at the OMGWTFBBQ the next weekend.

This was an amazing day. Massive thanks to my good friend Chris Walker for suggesting this gift. As I told David on the day: this was the most fun surprise gift I've ever received. Good hands-on teaching in a new craft is an incredible thing to experience, and I can't recommend this course highly enough.

There are discussions in development circles about Arm powered laptops since forever. But most of time they do not mention “normal” users. Like your parents, spouses, kids who are not developers. People who turn computer on (cold boot or from suspend does not matter) and expect them to “just work”.

My teenage daughter is one of them. Her current laptop is one of Thinkpad models, previous one was Thinkpad as well. Fedora Linux as operating system serves her needs just fine. But despite my 20 years of work with Arm architecture I am unable to get Arm based laptop for her.

Choices

There are several Arm powered laptops on a market:

• Macbooks
• Windows on Arm laptops
• Chromebooks
• SBCs with screen like Pinebook Pro

And all of them have issues when it comes to using Fedora Linux on them.

Macbooks

Thanks to Asahi Linux team we can run Fedora Linux on M1/M2 based Macbooks. Which means second hand market as Apple does not sell those models any more (unless 8GB of ram is enough for you).

There are many things which are not supported:

• Thunderbolt
• DisplayPort over Thunderbolt
• HDMI audio (work in progress)

So you pay for hardware and have features which you cannot use. I use Macbook Pro 2021 (with M1 Pro cpu) for local development and stopped checking how work goes.

Windows on Arm laptops

Qualcomm managed to convince Microsoft to not offer licenses for other vendors which means all we can have are Snapdragon based laptops. Which may work nice under MS Windows but if you want to use Linux then “good luck” is all you can get from me.

Some things work, some do not. I was told that Thinkpad x13s is one of best supported models. Johan Hovold has a Thinkpad x13s status page which lists what works and what needs to be done to have some kind of working laptop.

Definitely not a system for daily use for normal Linux user.

Chromebooks

Laptop to run web browser and Android apps. If this is all you need then go for it. But avoid if you are “normal” user and want to run Linux.

Finding how to enable running anything other than ChromeOS may involve digging through Internet pages, finding how to override ‘write protection’ etc.

Just no. Also Arm ones are usually ram limited.

Pinebook Pro and other SBCs with screen

Those are systems for developers only. Normal users should avoid using them as those systems require someone who knows how to prepare them to work at all.

Find/build proper firmware, put it properly in device (SPI Flash or storage media), keeping things up-to-date may end with partially not working device etc.

For developers those are ‘issues’ to workaround/solve but for normal users it may be ‘update went in background and now all I have is black screen’.

And like with Chromebooks you may be limited by ram size (Pinebook Pro has only 4GB ram).

Conclusion

If you are a normal user who wants to run Linux on a laptop then maybe stay away from Arm powered ones. Leave them for developers and check once/twice per year to see how situation looks.

At work I spend most of time on SBSA Reference Platform. Especially in firmware part (Arm Trusted Firmware also known as TF-A and Tianocore EDK2 also known as UEFI). However, for some time, I have felt the need to experiment with some UEFI-related task on existing hardware.

I first searched for “affordable” SystemReady SR system. But options were either Ampere Altra or NVIDIA Grace, both prices at 3000 EUR or more.

So I looked at the budget market and bought a FriendlyELEC NanoPC-T6 SBC.

Some words about the hardware

The FriendlyELEC NanoPC-T6 is a SBC (Seriously Bad Computer) based on Rockchip RK3588 SoC. It has some interesting on-board features:

• 8 cpu cores (4x Cortex-A76, 4x Cortex-A55)
• 16GB of memory
• 32MB of SPI flash for storing firmware
• 64GB of eMMC (not visible under Linux)
• USB 2 ports
• USB 3 ports (one type A, one type C)
• 2 HDMI outputs
• 1 HDMI input
• m.2 type M slot for 2280 card (such as NVMe)
• m.2 type E slot for 2230 card (such as Wi-Fi)
• 2 2.5GbE network ports
• USB-C serial console port (CH341 chip)
• microSD card slot
• Power, Reset, MaskROM buttons
• audio jack

It comes with metal case which works also as a heatsink.

Out of the box experience

As you know I expect a good “out of the box” experience. And NanoPC-T6 was like any other SBC I used in the past — unpleasant, horrible and frustrating.

The device came with a fork of U-Boot 2017.09, configured in such terrible way that it was incapable of booting any standard distro images I tried. I managed to boot the pre-installed Android 12 on the eMMC but quickly rebooted to avoid dealing with it.

I managed to boot Debian ‘testing’ manually but there was no networking available under 6.9.x kernel.

I Then moved on to other things as my schedule was quite busy.

UEFI experience

This week I reserved some time to get NanoPC-T6 running properly. I downloaded a Rockchip tool called “upgrade_tool” and used it to flash a UEFI image from the EDK2-RK3588 project.

Experience was much, much better. The firmware was now capable of booting distro images, allowed me to choose between ACPI or DeviceTree for hardware description and had proper EFI Shell — almost like a well-developed systems.

I went with ACPI mode and booted directly to Fedora ‘rawhide’ system stored on a USB drive. Linux 6.11-rc booted, found devices plugged into USB 3 ports, recognized both network interfaces (Realtek 8125 ones) and the NVME drive as well. There was video output on the HDMI screen (in a hardcoded 1080p resolution).

I then copied the system from the USB3 drive to the NVME, set the proper boot order and enjoyed a nicely working system.

DeviceTree?

But aren’t Seriously Bad Computers (SBCs) expected to run with DeviceTree? ACPI is for MS Windows, not for Linux or *BSD systems, right?

So I decided to boot into DT land. It took me a while as I had to remind myself how it works and ensure that UEFI firmware will use the 6.11-rc DTB instead of one for 5.10-rk or 6.1-rk vendor kernels.

Finally it booted — or rather, it “kind of” booted…

No USB, no PCIe == no NVME == boot into emergency mode because the root filesystem is not present…

Future plans

What will future bring? I am going to find out. I have ordered a Wi-Fi card for m.2 type E slot to see how it performs and I am going to spend some time around this EDK2 fork to make some experiments on real hardware.

Nov 26th saw the release of 4.4.165, 4.9.141, 4.14.84 and 4.19.4

For these LTS kernel versions, results were reported upstream, no regressions were found.

• Bug status
  • 2018-11-27: Naresh Kamboju sent a kselftest patch to add a config frag for FTRACE re: bug 3920
  • 2018-11-26: Rafael Tinoco sent a patch to fix thread_info->syscall on arm re: bug 3783
  • 2018-11-26: Dan Rue sent a kselftest patch set to fix fw_Firmware.sh re: bug 3503
  • 2018-11-21: Naresh Kamboju reported kselftest get_cgroup_id_user issue re: bug 3920
  • 2018-11-21: Naresh Kamboju reported a libbpf problem in kselftest bpf tests
  • 2018-11-26: Rafael Tinoco – bug 3765 – proposed re-asked Russell about re-submitting this or not (zmalloc pae fix)
  • 2018-11-26: Rafael Tinoco – bug 4053 – backported 5 patches from Daniel Lezcano from v4.14 Android (fix thermal)

2018-11-26: Rafael Tinoco – bug 4043 – Asked Greg to backport a fix for v4.4, Sasha forwarded to the mm list.

For Android Kernels, regressions were detected.

Issues:

• 4.14.84 + HiKey boot regression – observed in with 9.0 and AOSP
• 4.4.165 Regression:
  VtsKernelSyscallExistence#testSyscall_name_to_handle_at – Unknown
  error: test case requested but not executed.
  VtsKernelSyscallExistence#testSyscall_open_by_handle_at – Unknown
  error: test case requested but not executed.
  VtsKernelSyscallExistence#testSyscall_uselib – Unknown error: test
  case requested but not executed.

No Others Regressions: 4.4.165 and 4.9.141 on Android 9.

X15: 4.14.84 + O-MR1 – Baselining activity has been particularly effective over the past two weeks, dropping the number of errors from 65 failing tests to 16 as of today. That’s really good progress towards setting a clean baseline.

Bug 4033  Sumit has been looking at the failing CtsBluetoothTestCases android.bluetooth.cts.BluetoothLeScanTest#testBasicBleScan and android.bluetooth.cts.BluetoothLeScanTest.testScanFilter failures.

These tests both pass across all kernels with 8.1. They however fail with both 9.0 and AOSP. Looking at historical AOSP results it appears that failures there started approx in the September timeframe.

Last, successful test builds and test boot to UI with 4.4.165 and 4.9.141 with Android 9) using the newly released clang-r346389 compiler.

Nov 26th saw the release of 4.4.165, 4.9.141, 4.14.84 and 4.19.4

For these LTS kernel versions, results were reported upstream, no regressions were found.

• Bug status
  • 2018-11-27: Naresh Kamboju sent a kselftest patch to add a config frag for FTRACE re: bug 3920
  • 2018-11-26: Rafael Tinoco sent a patch to fix thread_info->syscall on arm re: bug 3783
  • 2018-11-26: Dan Rue sent a kselftest patch set to fix fw_Firmware.sh re: bug 3503
  • 2018-11-21: Naresh Kamboju reported kselftest get_cgroup_id_user issue re: bug 3920
  • 2018-11-21: Naresh Kamboju reported a libbpf problem in kselftest bpf tests
  • 2018-11-26: Rafael Tinoco – bug 3765 – proposed re-asked Russell about re-submitting this or not (zmalloc pae fix)
  • 2018-11-26: Rafael Tinoco – bug 4053 – backported 5 patches from Daniel Lezcano from v4.14 Android (fix thermal)

2018-11-26: Rafael Tinoco – bug 4043 – Asked Greg to backport a fix for v4.4, Sasha forwarded to the mm list.

For Android Kernels, regressions were detected.

Issues:

• 4.14.84 + HiKey boot regression – observed in with 9.0 and AOSP
• 4.4.165 Regression:
  VtsKernelSyscallExistence#testSyscall_name_to_handle_at – Unknown
  error: test case requested but not executed.
  VtsKernelSyscallExistence#testSyscall_open_by_handle_at – Unknown
  error: test case requested but not executed.
  VtsKernelSyscallExistence#testSyscall_uselib – Unknown error: test
  case requested but not executed.

No Others Regressions: 4.4.165 and 4.9.141 on Android 9.

X15: 4.14.84 + O-MR1 – Baselining activity has been particularly effective over the past two weeks, dropping the number of errors from 65 failing tests to 16 as of today. That’s really good progress towards setting a clean baseline.

Bug 4033  Sumit has been looking at the failing CtsBluetoothTestCases android.bluetooth.cts.BluetoothLeScanTest#testBasicBleScan and android.bluetooth.cts.BluetoothLeScanTest.testScanFilter failures.

These tests both pass across all kernels with 8.1. They however fail with both 9.0 and AOSP. Looking at historical AOSP results it appears that failures there started approx in the September timeframe.

Last, successful test builds and test boot to UI with 4.4.165 and 4.9.141 with Android 9) using the newly released clang-r346389 compiler.

Nov 26th saw the release of 4.4.165, 4.9.141, 4.14.84 and 4.19.4

For these LTS kernel versions, results were reported upstream, no regressions were found.

• Bug status
  • 2018-11-27: Naresh Kamboju sent a kselftest patch to add a config frag for FTRACE re: bug 3920
  • 2018-11-26: Rafael Tinoco sent a patch to fix thread_info->syscall on arm re: bug 3783
  • 2018-11-26: Dan Rue sent a kselftest patch set to fix fw_Firmware.sh re: bug 3503
  • 2018-11-21: Naresh Kamboju reported kselftest get_cgroup_id_user issue re: bug 3920
  • 2018-11-21: Naresh Kamboju reported a libbpf problem in kselftest bpf tests
  • 2018-11-26: Rafael Tinoco – bug 3765 – proposed re-asked Russell about re-submitting this or not (zmalloc pae fix)
  • 2018-11-26: Rafael Tinoco – bug 4053 – backported 5 patches from Daniel Lezcano from v4.14 Android (fix thermal)

2018-11-26: Rafael Tinoco – bug 4043 – Asked Greg to backport a fix for v4.4, Sasha forwarded to the mm list.

For Android Kernels, regressions were detected.

Issues:

• 4.14.84 + HiKey boot regression – observed in with 9.0 and AOSP
• 4.4.165 Regression:
  VtsKernelSyscallExistence#testSyscall_name_to_handle_at – Unknown
  error: test case requested but not executed.
  VtsKernelSyscallExistence#testSyscall_open_by_handle_at – Unknown
  error: test case requested but not executed.
  VtsKernelSyscallExistence#testSyscall_uselib – Unknown error: test
  case requested but not executed.

No Others Regressions: 4.4.165 and 4.9.141 on Android 9.

X15: 4.14.84 + O-MR1 – Baselining activity has been particularly effective over the past two weeks, dropping the number of errors from 65 failing tests to 16 as of today. That’s really good progress towards setting a clean baseline.

Bug 4033  Sumit has been looking at the failing CtsBluetoothTestCases android.bluetooth.cts.BluetoothLeScanTest#testBasicBleScan and android.bluetooth.cts.BluetoothLeScanTest.testScanFilter failures.

These tests both pass across all kernels with 8.1. They however fail with both 9.0 and AOSP. Looking at historical AOSP results it appears that failures there started approx in the September timeframe.

Last, successful test builds and test boot to UI with 4.4.165 and 4.9.141 with Android 9) using the newly released clang-r346389 compiler.

Nowadays we have only one Connect per year. And this year we met in Madrid, Spain.

How did it go for me? Good, better than the previous one.

I attended most of the talks I wanted to see, spoke with countless people, and met most of the people on my “to meet” list.

SystemReady related talks

For me, the main topic at Linaro Connect was SystemReady. There was a track for SystemReady IR on the first day and the next days included additional presentations.

Does SystemReady IR “Just Work”?

Jon Humphreys from Texas Instruments gave an introduction what SystemReady. Explained how systems operated before it, the definition of “Just Works”, and some aspects of testing. He then discussed issues with the current certification process, such as missing firmware files used during testing and problems with errata documents. He also offered recommendations on how to improve the situation.

Looking back on SystemReady IR

Vincent Stehlé from Arm spoke a lot about of history and timelines of involved specifications, the U-Boot features timeline and several statistics about SystemReady certifications along with lessons learned.

30.9% of certified systems were for IR band (only SR was higher with 36%), mostly for older versions of the specification.

We learned that there are three certification labs besides Arm itself. And the plan is to eventually remove Arm from this role.

It was interesting to see which distributions were used during IR certification. Fedora Linux leads with 33.9%, followed by openSUSE at 33% and Debian at 21.1%. Other distributions include Ubuntu, SLES, Rocky Linux and RHEL.

U-Boot for SystemReady IR — Status and struggles

Ilias Apalodimas from Linaro presented the history of U-Boot getting features required for SystemReady IR support. He highlighted some common issues and offered suggestions for improvement.

This talk nicely expanded on the topics mentioned in the previous two talks.

SystemReady as default option for BSPs: Findings from the last SystemReady IoT workshop

Ilias Apalodimas, Pere Garcia Gutiérrez and Peter Robinson presented conclusions from SystemReady’s Advisory Committee workshop.

One idea was that SoC vendors should provide BSP setup in a way that the resulting firmware would already be SystemReady compliant. This would make it easier for OEM/ODM vendors, with some following their own paths while others adhered to the guidelines.

Qualcomm and SystemReady IR, an overview of Qualcomm support in U-Boot and what the future

Caleb Connolly from Linaro shared the story how Qualcomm support in U-Boot improved from terrible to good. He mentioned several quirks and issues encountered along the way.

Arm System Architecture and SystemReady Update

Dong Wei from Arm provided updates on changes to the SystemReady specifications.

I somehow missed that session. It was on my list, but I got distracted by a discussion.

He explained how changes are managed and reminded us how SystemReady bands depend on specifications. Then presented planned changes to BSA, such as creation of VBSA for virtual environments and increase of requirements for future versions.

Another change mentioned was that the SystemReady LS and LBBR recipe of BBR might be deprecated due to lack of interest.

There will be some changes for the SBSA requirements. So far there are no plans to create SBSA level 8, but if a silicon vendor can fulfil all future requirements, Arm may consider publishing SBSA level 8.

The BBSR (Boot Base Security Requirements) specification may become a requirement for future SystemReady versions.

SBSA Reference Platform in QEMU status update

I gave a talk presenting what we achieved in the past year regarding the SBSA Reference Platform in QEMU, Trusted Firmware and EDK2 projects.

I discussed changes to virtual hardware, firmware and the methods we use to transfer configuration data between layers. I explained why I use “Datatree” term instead of “Devicetree” and outlined our plans for the future.

Other sessions

I attended other sessions as well and watched several ones I missed due to discussions or schedule conflicts. Here I want to write about some of them.

TianoCore community update

Leif Lindholm provided an introduction and updates on the TianoCore EDK2 project.

He covered which specifications it implements, how repositories are structured, the licenses used and the communication methods. Also discussed updates to SSL/TLS libraries and toolchain profiles, contribution rules and recent changes to there rules.

Additionally, he outlined plans for changes to edk2-platforms such as creating stable tags and implementing automated CI.

An Arm laptop project conclusion

Johan Hovold shared information about the state of Linux on the Lenovo Thinkpad x13s laptop (which comes with Windows for Arm by default).

The good part is that most of the support is already merged into upstream projects and the required kernel configuration changes have been included in several distributions.

The bad part? There is still a lot of work to be done. I wonder when vendors will learn how to write proper ACPI tables (x13s uses Devicetree to run Linux).

Rethinking the kernel system call entry

Arnd Bergmann from Linaro spoke about his work on rethinking the Linux kernel system call entry.

He covered how system calls are currently defined now in the Linux kernel. Then showed how he changed it to be more readable.

I have my system calls table but had never looked how kernel code for them looks. Turned out that it can be quite intimidating with macros expanded to macros resulting in complex C code.

Arnd showed that it can be made readable. I hope that his work will land in the Linux kernel soon.

End words

Linaro Connect MAD24 was a success. I got ideas for blog posts, met people who read my posts.

And if you want to watch more talks from conference then there is official Linaro Connect MAD24 playlist on YouTube.

Visit Linaro Resources Hub for video recordings and presentation slides.

There are memberships I forgot about. Some of them remind from time to time with “we have changed rules” mail. Which usually are moments when I remove account from their system.

And there are memberships I remember never mind if I use them on not anymore.

One of them is OpenEmbedded.

History

I started using OpenEmbedded around February/March of 2004. Will not cover history here as I wrote several posts closer to those years:

• Year with OpenEmbedded in 2005
• 30 months of OpenEmbedded and me in 2006
• 3 years of OpenEmbedded and me in 2007
• 5 years with OpenEmbedded in 2009
• 10 years ago I got write access to OpenEmbedded in 2014

I learnt a lot, helped people and companies, got paid by both people and companies, mentored new users and developers.

And got some friends in the OpenEmbedded community.

Membership

In 2007 we were discussing about creating official organization. During FOSDEM 2008 a group of developers met and it happened.

First it was OpenEmbedded e.V. based on Germany law, years later it became one of Software in the Public Interest (SPI) projects.

Developers joined, became members, attended meetings to vote etc..

During years, due to member being spread all over the globe, we moved from in-person meetings (with potential proxies) to online voting system.

And in last week I got an email with voting invitation. Of course, I knew that it will come — we have openembedded-members mailing list for a reason ;D

Sentimental value

For me my OpenEmbedded membership has highest value when it comes to my memberships. Sure, mostly for sentimental values but when I am at FOSDEM, I always visit OpenEmbedded stand, usually see some OE related talk in the Embedded devroom etc.. Similar at other events.

And my e-mail address in the openembedded.org domain still works. I do not use it but it has value to me.

During my work on SBSA Reference Platform I have spent lot of time in firmware’s code. Which mostly meant Tianocore EDK2 as Trusted Firmware is quite small.

Writing all those ACPI tables by hand takes time. So I checked ConfigurationManager component which can do it for me.

Introduction

In 2018 Sami Mujawar from Arm contributed Dynamic Tables Framework to Tianocore EDK2 project. The goal was to have code which generates all ACPI tables from all those data structs describing hardware which EDK2 already has.

In 2023 I was writing code for IORT and GTDT tables to generate them from C. And started wondering about use of ConfigurationManager.

Mailed edk2-devel ML for pointers, documentation, hints. Got nothing in return, idea went to the shelf.

SBSA-Ref and multiple PCI Express buses

Last week I got SBSA-Ref system booting in NUMA configuration with three separate PCI Express buses. And started working on getting EDK2 firmware to recognize them as such.

Took me a day and pci command listed cards properly:

Shell> pci
   Seg  Bus  Dev  Func
   ---  ---  ---  ----
    00   00   00    00 ==> Bridge Device - Host/PCI bridge
             Vendor 1B36 Device 0008 Prog Interface 0
    00   00   01    00 ==> Network Controller - Ethernet controller
             Vendor 8086 Device 10D3 Prog Interface 0
    00   00   02    00 ==> Bridge Device - PCI/PCI bridge
             Vendor 1B36 Device 000C Prog Interface 0
    00   00   03    00 ==> Bridge Device - Host/PCI bridge
             Vendor 1B36 Device 000B Prog Interface 0
    00   00   04    00 ==> Bridge Device - Host/PCI bridge
             Vendor 1B36 Device 000B Prog Interface 0
    00   01   00    00 ==> Mass Storage Controller - Non-volatile memory subsystem
             Vendor 1B36 Device 0010 Prog Interface 2
    00   40   00    00 ==> Bridge Device - PCI/PCI bridge
             Vendor 1B36 Device 000C Prog Interface 0
    00   40   01    00 ==> Bridge Device - PCI/PCI bridge
             Vendor 1B36 Device 000C Prog Interface 0
    00   41   00    00 ==> Base System Peripherals - SD Host controller
             Vendor 1B36 Device 0007 Prog Interface 1
    00   42   00    00 ==> Display Controller - Other display controller
             Vendor 1234 Device 1111 Prog Interface 0
    00   80   00    00 ==> Bridge Device - PCI/PCI bridge
             Vendor 1B36 Device 000E Prog Interface 0
    00   80   01    00 ==> Bridge Device - PCI/PCI bridge
             Vendor 1B36 Device 000C Prog Interface 0
    00   81   09    00 ==> Multimedia Device - Audio device
             Vendor 1274 Device 5000 Prog Interface 0
    00   81   10    00 ==> Network Controller - Ethernet controller
             Vendor 8086 Device 100E Prog Interface 0
    00   82   00    00 ==> Mass Storage Controller - Serial ATA controller
             Vendor 8086 Device 2922 Prog Interface 1

Three buses are: 0x00, 0x40 and 0x80. But then I had to tell Operating System about those. Which meant playing with ACPI tables code in C.

So idea came “what about trying ConfigurationManager?”.

Another try

Mailed edk2-devel ML again for pointers, documentation hints. And then looked at code written for N1SDP and started playing with ConfigurationManager…

ConfigurationManager.c has EDKII_PLATFORM_REPOSITORY_INFO struct with hundreds of lines of data (as another structs). From listing which ACPI tables I want to have (FADT, GTDT, APIC, SPCR, DBG2, IORT, MCFG, SRAT, DSDT, PPTT etc.) to listing all hardware details like GIC, PCIe, Timers, CPU and Memory information.

Then code for querying this struct. I thought that CM/DT (ConfigurationManager/DynamicTables) framework will have those already in EDK2 code but no — each platform has own set of functions. Another hundreds of lines to maintain.

Took some time to get it built, then started filling proper data and compared with ACPI tables I had previously. There were differences to sort out. But digging more and more into code I saw that I go deeper and deeper into rabbit hole…

Dynamic systems do not fit CM?

For platforms with dynamic hardware configuration (like SBSA-Ref) I needed to write code which would populate that struct with data on runtime. Check amount of cpu cores and write cpu information (with topology, cache etc), create all GIC structures and mappings. Then same for PCIe buses. Etc. Etc. etc…

STATIC
EFI_STATUS
EFIAPI
InitializePlatformRepository (
  IN  EDKII_PLATFORM_REPOSITORY_INFO  * CONST PlatRepoInfo
  )
{
  GicInfo                 GicInfo;
  CM_ARM_GIC_REDIST_INFO *GicRedistInfo;
  CM_ARM_GIC_ITS_INFO    *GicItsInfo;
  CM_ARM_SMMUV3_NODE     *SmmuV3Info;

  GetGicDetails(&GicInfo);
  PlatRepoInfo->GicDInfo.PhysicalBaseAddress = GicInfo.DistributorBase;

  GicRedistInfo = &PlatRepoInfo->GicRedistInfo[0];

  GicRedistInfo->DiscoveryRangeBaseAddress = GicInfo.RedistributorsBase;

  GicItsInfo = &PlatRepoInfo->GicItsInfo[0];
  GicItsInfo->PhysicalBaseAddress = GicInfo.ItsBase;

  SmmuV3Info = &PlatRepoInfo->SmmuV3Info[0];
  SmmuV3Info->BaseAddress = PcdGet64 (PcdSmmuBase);

  return EFI_SUCCESS;
}

Which in my case can mean even more code written to populate CM struct of structs than it would take to generate ACPI tables by hand.

Summary

ConfigurationManager and DynamicTables frameworks look tempting. There may be systems where it can be used with success. I know that I do not want to touch it again. All those structs of structs may look good for someone familiar with LISP or JSON but not for me.

During last weeks we worked on getting rid of DeviceTree from EDK2 on SBSA Reference Platform. And finally we managed!

All code is merged into upstream EDK2 repository.

What?

Someone may wonder where DeviceTree was in SBSA Reference Platform. Wasn’t it UEFI and ACPI platform?

Yes, from Operating System point of view it is UEFI and ACPI. But if you look deeper you will see DeviceTree hidden inside our chain of software components:

/dts-v1/;

/ {
        machine-version-minor = <0x03>;
        machine-version-major = <0x00>;
        #size-cells = <0x02>;
        #address-cells = <0x02>;
        compatible = "linux,sbsa-ref";

        chosen {
        };

        memory@10000000000 {
                reg = <0x100 0x00 0x00 0x40000000>;
                device_type = "memory";
        };

        intc {
                reg = <0x00 0x40060000 0x00 0x10000
                       0x00 0x40080000 0x00 0x4000000>;

                its {
                        reg = <0x00 0x44081000 0x00 0x20000>;
                };
        };

        cpus {
                #size-cells = <0x00>;
                #address-cells = <0x02>;

                cpu@0 {
                        reg = <0x00 0x00>;
                };

                cpu@1 {
                        reg = <0x00 0x01>;
                };

                cpu@2 {
                        reg = <0x00 0x02>;
                };

                cpu@3 {
                        reg = <0x00 0x03>;
                };
        };
};

It is very minimal one, providing us with only those information we need. It does not even pass any compliance checks. For example for Linux, GIC node (/intc/ one) should have gazillion of fields, but we only need addresses.

Trusted Firmware reads it, parses and provides information from it via Secure Monitor Calls (SMC) to upper firmware level (EDK2 in our case). DeviceTree is provided too but we do not read it any more.

Why?

Our goal is to treat software components a bit different then people may expect. QEMU is “virtual hardware” layer, TF-A provides interface to “embedded controller” (EC) layer and EDK2 is firmware layer on top.

On physical hardware firmware assumes some parts and asks EC for the rest of system information. QEMU does not give us that, while giving us a way to alter system configuration more than it would be possible on most of hardware platforms using a bunch of cli arguments.

EDK2 asks for CPU, GIC and Memory. When there is no info about processors or memory, it informs the user and shutdowns the system (such situation does not have a chance of happening but it works as an example).

Bonus stuff: NUMA

Bonus part of this work was adding firmware support for NUMA configuration. When QEMU is run with NUMA arguments then operating system gets whole memory and proper configuration information.

QEMU arguments used:

-smp 4,sockets=4,maxcpus=4
-m 4G,slots=2,maxmem=5G
-object memory-backend-ram,size=1G,id=m0
-object memory-backend-ram,size=3G,id=m1
-numa node,nodeid=0,cpus=0-1,memdev=m0
-numa node,nodeid=1,cpus=2,memdev=m1
-numa node,nodeid=2,cpus=3

How Operating System sees NUMA information:

root@sbsa-ref:~# numactl --hardware
available: 3 nodes (0-2)
node 0 cpus: 0 1
node 0 size: 975 MB
node 0 free: 840 MB
node 1 cpus: 2
node 1 size: 2950 MB
node 1 free: 2909 MB
node 2 cpus: 3
node 2 size: 0 MB
node 2 free: 0 MB
node distances:
node   0   1   2
  0:  10  20  20
  1:  20  10  20
  2:  20  20  10

What next?

There is CPU topology information in review queue. All those sockets, clusters, cores and threads. QEMU will pass it in DeviceTree, TF-A will give it via SMC and then EDK2 will put it in one of ACPI tables (PPTT == Processor Properties Topology Table).

If someone decide to write own firmware for SBSA Reference Platform (like port of U-Boot) then both DeviceTree and set of SMC calls will wait for them, ready to be used to gather hardware information.