VideoCore VII GPU, supporting OpenGL ES 3.1, Vulkan 1.2
Dual 4Kp60 HDMI® display output with HDR support
4Kp60 HEVC decoder
LPDDR4X-4267 SDRAM (4GB and 8GB SKUs available at launch)
Dual-band 802.11ac Wi-Fi®
Bluetooth 5.0 / Bluetooth Low Energy (BLE)
microSD card slot, with support for high-speed SDR104 mode
2 × USB 3.0 ports, supporting simultaneous 5Gbps operation
2 × USB 2.0 ports
Gigabit Ethernet, with PoE+ support (requires separate PoE+ HAT)
2 × 4-lane MIPI camera/display transceivers
PCIe 2.0 x1 interface for fast peripherals (requires separate M.2 HAT or other adapter)
5V/5A DC power via USB-C, with Power Delivery support
Raspberry Pi standard 40-pin header
Real-time clock (RTC), powered from external battery
Power button
Similarly to the change from Pi3 → Pi4 there’s probably a ton of internal differences that will not make it possible to run info-beamer OS out of the box:
There doesn’t seem to be a H264 hardware decoder any more, so the existing method of decoding that will have to be completely rewritten and probably unified with how HEVC/H265 now works.
HDR seems interesting. Not sure yet how all that will work
Lots of other unknowns
There is no release date for anything yet. Any news will be posted here.
No idea, to be honest. I don’t expect anything to be usable before next year and there’s a lot of changes required to make the video decoding work on the Pi5 at all.
A quick update on the current status of Pi 5 support:
info-beamer Pi, so the raw content player:
Got OpenGL support working again. The rework now used DRM atomic syncs for scheduling page flips. It seems this also results in page flips that are synced across multiple displays. This is quite neat as the current Pi4 info-beamer implementation doesn’t support this and both displays can be slightly out of sync as a result.
Running the current development version on a Raspberry Pi OS can now play back content that doesn’t use any videos. So image only setups seem to work fine.
Got an understanding on how video decoding for both H264 and HEVC using the video 4 linux APIs should work. Nothing is implemented yet, but the current decoding architecture for the Pi4 version might be reused in some form to avoid doing everything from scratch. I’m not entirely sure how to best handle audio to video synchronization, but that’s a problem for future-me
One issue is that there doesn’t seem to be an easy way to rotate videos by 90 or 270 degree. The previous API used by info-beamer to place video layers on the display internally handled such rotations. The new DRM API doesn’t support this (it only supports 0 and 180 degrees). It remains to be seen if a workaround is feasible or if such rotation will be no longer supported.
info-beamer OS:
Nothing has been implemented yet, but the changes might not be as big as I have feared. For now info-beamer OS will most likely still use a mostly 32bit userland, to avoid having to distribute both 32 and 64 bit binaries for the OS. The kernel might be 64 bit though. Still undecided.
Separated DRM planes from logical info-beamer layers. It’s now possible to move other layers behind and in front of the GL layer and the DRM planes are dynamically updated accordingly, potentially shuffling them around, as the zpos properties of DRM’s primary layers seem to be fixed to 0.
Used all that to restore the “you didn’t purchase the commercial version” overlay logo. Makes a good test case for all that. And it helps to understand how importing arbitrary RGBA buffers into DRM works.
It should be again possible to seamlessly transition from the text console the the info-beamer overlay. Although I’m not sure if the fact that this works is some oversight somewhere: The kernel has a framebuffer plane (fbcon, see /sys/kernel/debug/dri/1/state) and when I leave this plane untouched and only modify overlay layers, the console seems to stay visible!?
Testing out some the limits of how many video layers are possible. For a Dual FullHD setup, it seems multiple 4K layers work. So Dual FullHD should work without issues. 4K layers while also using a FullHD + 4K Dual setup seems to run into some issues. Still trying to understand what kind of limit that is: github link to the error message.
One additional problem that needs to be solved in the future is how snapshots of the current display output work. With the old dispmanx API, any process could trigger a snapshot of what’s currently on the display. With DRM this is no longer possible, I think. Instead there are virtual “writeback connectors”. From what I understand they behave like a virtual display but instead of rendering to a screen, they render into a buffer. Not sure how that all works yet, but if it does, screenshots are then produced by the info-beamer process itself. A nice side effect is that screenshots of a dual display setup would then be consistent. The downside is the different screenshot handling between Pi4/Pi5 and all older models.
Well. Snapshotting didn’t work out (yet). Two problems:
Using two writeback connectors in addition to the two connectors already used to power two displays seems to exceed the abilities of the hardware. From my vague understanding on only has 3 scalers. The alternative is probably to first snapshot one screen and then the other. Gone is the consistency.
Speaking of writeback connectors: While the snapshotting seems to work, I don’t see any updated data in the buffer object I provide to DRM. Its content remains unchanged. I’m not sure why that’s the case and if there’s maybe some kind of synchronization required to make changes visible to the userland? Very confusing.
On the brighter side:
Got the V4L2 video decoder working for both H264 and HEVC videos! I can now receive the decoded frames either as a software pixel buffer, in case of a H264 video, or as already DMA-buf’ed objects in case of HEVC. The latter can be fairly easily placed into a GL texture. For my first time ever, I can now render a HEVC 4K video into a GL texture in a fully hardware accelerated way. It reaches 30FPS on a 4K display with a 4K video. So that’s quite nice!
I’m still not sure how to do that same for the less exciting H264 format. It is software-decoded into a YUV420P buffer (the Pi5 no longer has hardware acceleration for H264) and I don’t see how or if it is possible to place that directly into a GL texture somehow. A way that works but sucks a bit is to software transform the YUV420P into a RGB(A) buffer and import that. The conversion process takes 2ms for a FullHD video, so I guess that’s ok, but it would feel better to avoid that extra step.
Another neat improvement is that HEVC can now play more than a single video at once, making it a true alternative to H264 on Pi4 and later.
Next up: Implementing playback for non-GL (aka raw) videos.
Note that this is all still very unfinished, but it’s quite nice to slowly see that my current approach seems to work. I’m still not sure how one synchronizes audio (which isn’t done at all) to video playback. We’ll see.
Raw playback is now implemented. Still without any kind of scheduling, so its blasting through the video as fast as possible. It works with both HEVC (hardware frames) and H264 (software YUV420P frames).
I also managed to avoid the conversion process from YUV420P software frames to a RGB(A) format. Instead a frame is now directly imported into either DRM or a GL texture in the YUV420. Turns out that’s possible. So that’s quite nice.
While doing that, there’s still a memcpy from the decoded frame data into the mmap memory of a “dumb buffer”. As that’s around 3MB per FullHD video frame (1920x1080*1.5), that’s quite a bit of data (180MB/s!) for a 60fps video. I tried to avoid that using a custom get_buffer2 function and doing all the buffer preparations and mmap’ing there. It works and avoids the extra memcpy… but it’s slower and jittery. Right now I have no explanation why that’s the case and there the performance is lost.
Next step: Scheduling and re-adding all the additional video methods like pausing and so on…
Found a way to make the new Pi4/Pi5 version able to use explicit CEA/DMT modes available on Pi3 and older. This will at some point in the future allow a unified mode selection dropdown on the dashboard across all Pi models.
Tested the new code on Pi4. It also works and can use the H264 acceleration support still available there. Also nice: It’s finally possible to import HEVC video as non-raw videos.
Started working on a playback clock. It now properly plays videos in their intended speed and I can pause, resume and speed up playback. Still not entirely sure how that all works together once audio is added back.
One major issue is still the unsupported 90 and 270 degree rotation of raw videos. It might be possible to transparently fall back to a GL texture in that case. This would be a quite complicated and will cost performance, but it would ensure backwards compatibility. Also it is not possible to ensure the layer ordering is correct once that happens. In the general case of a video being rotated 90 degree and then drawn on top of some kind of background content it would probably be good enough.
The scheduling turns out to be as complicated as I initially thought. I’m not sure how to, for example, ensure that a 60fps video on a 60Hz screen doesn’t skip any video frame due to timing issues. Usually the rendering of content has a few milliseconds to spare before the next page flip happens. The next video frame better be ready by then… That is, unless you have two display attached. In my experiments it seems you better have more than 8ms remaining before you call drmModeAtomicCommit. Otherwise the flipping takes more than a single frame. So much for synced vsync when using two displays. It seems even with an atomic DRM implementation, flipping of two displays happens at a random offset and that offset between both displays seems to be different every time info-beamer starts. Meh.
Added support for yuv422 frames. I’ll have to probe into all the uploaded mp4 files on info-beamer hosted to see if yuv420 and yuv422 is enough.
Another issue seems to be that I cannot forcefully set (for example) a 4K video mode on a FullHD display. I didn’t have the time yet to figure out where in the kernel code that restriction exactly comes from but the dmesg log is:
I assume there are some sanity checks somewhere that (probably) rely on the information provided by EDID to restrict unsupported video modes? The EDID blob within DRM is also immutable, so it doesn’t seem possible to overwrite it to zero from within info-beamer itself.
The problem is that a user might configure their Pi for 4K, but then temporarily connect a FullHD display. In that case info-beamer doesn’t start and the current implementation of the OS assumes something is broken and restarts. Not ideal. Maybe I’ll have to try setting the mode, then fall back to mode safe modes if they fail to work.
Fixed another problem regarding layer placement. It turns out you cannot move a plane from one CRTC to another one. So info-beamer has to temporarily use a different plane when moving a framebuffer from one CRTC to another. Slowly but steadily I’m getting used to all this :-}
Make gl.setup(w, h) with a width/height smaller than the native resolution use only part of the GL surface. This can speed things up if the higher resolution isn’t needed.
Implemented “fake” drm rendering for layers with a 90/270 degree rotation. Those are not natively supported by the DRM hardware, so info-beamer now mostly transparently falls back to rendering them as GL textures. Most info-beamer packages should not notice a difference. The downside is that this is all a bit slower than 0/180 degrees, but there doesn’t seem to be an alternative.
Video looping now works.
Still no audio
Implemented screenshots. info-beamer now provides a built-in feature to capture the current content what’s on the screen. With KMS/DRM there doesn’t seem to be any other way to implement this. In prehistoric times, dispmanx allows out-of-band screenshots from any process, but that’s no longer possible. The snapshotting uses the TCP interface, so there’s a new command *screenshot/0 and *screenshot/1 that return the size followed by a binary RGB stream of pixels. 4K displays transfers ~25MB that way and I’ll have to optimize this tomorrow to avoid fps lag as much as possible.
Snapshotting now returns a JPEG instead of the raw RGB stream. So retrieving snapshots isn’t as massive as before and someone has to compress the data anyway so info-beamer might as well do it.
Backported the snapshotting to the dispmanx based version for Pi3 and earlier.
Embedded more features required for the info-beamer hosted OS version like the blinking WiFi icon as that too cannot be externally painted over the output.
Some release documentation.
I think a first version for Raspberry Pi OS 32 and 64bit without audio support is ready soon. Stay tuned
Got info-beamer OS to boot on the Pi5 and show some demo content!. That was quite a challenge. I got massively stuck trying to get the vc4 graphics driver to completely initialize. Turns out you need the i2c_brcmstb module(!?). Otherwise loading vc4 is eternally stuck with:
vc4-drm axi:gpu: deferred probe pending
Now comes the massive task of making everything sure everything works 100% again, including on all previous Pi generations. The kernel jump to 6.1 probably requires quite a few changes. This might take a while. Some of the large number of problems:
It might also require some special embedded quirks to work around existing packages that currently work on the Pi4 but wouldn’t when using the new playback mechanism on Pi4/5. For example the HD Image/Video player package currently uses a gl.clear(0, 0, 0, 1) and later a gl.clear(0, 0, 0, 0). When playing a 90 degree rotated video, this works fine on the current OS, as the first clear is basically ignored and the GL layer is transparent.
But the new version requires the fallback to GL videos for this rotation. Info-beamer already does this, but as it’s rendered into the same GL surface as all the other content, so a clear would also clear the rotated video.
The clear(0,0,0,0) is already ignored in that case, but the gl.clear(0,0,0,1) isn’t at the moment as this in general would paint a black background and it’s not safe to ignore in all cases. Not sure how to handle all this yet.
Still no audio. As there is still only a single OS variant across all Pi models, even the Pi1 will get the new FFmpeg library. That deprecated avresample, so currently audio would be broken there as well and needs a rework to use swresample. Probably rather simple, but needs work to ensure it all works.
The eeprom firmware update mechanism needs to be reworked. Currently info-beamer OS places pieeprom.* files together with recovery.bin on the SD card when installing from the zip file. This will result in the Pi4 updating its eeprom on first boot and then renaming the recovery.bin to something like RECOVERY.001, which is then ignored on any followup boot. Unfortunately this won’t work any longer as the Pi5 has a different eeprom with a different size, but no mechanism to use different files. So it would try to use the wrong eeprom image on the Pi5, which then fails. So instead info-beamer would now probably need to place the correct eeprom files per Pi model on-demand on the SD prior to a reboot.
Getting weird artifacts when using the default HD Player package across two displays while switching from one video to the next: The blue part is already part of the next video. That definitely shouldn’t look like this. I’m pretty sure there’s still some bugs in the Pi4 and Pi5 code that no one ever triggered as there isn’t a comparable video player on the Pi anywhere :-}
Implemented the built-in “Lua fixes feature” that allows the Pi4/5 version to apply modified Lua code in some special cases to ensure backwards compatibility with code already running on the current version of the Pi4 software. For example with the HD player, as mentioned above. Adding more fixes is now pretty simple to do.
Reworked the EEPROM update mechanism. It now checks if an update is required on reboot and selected the corresponding Pi4 or Pi5 files depending on the hardware.
Restored the ability to take boot screen snapshots on the Pi5 by taking a screenshot of the frame buffer.
Made the Pi5’s video mode selection compatible to the existing configuration method (e.g. setting values like hdmi_group=1 in /config/screen-settings.txt). These are now parsed and supplied to the info-beamer process which already knows how to use all the CEA/DMT modes from the Pi3 days.
The install.zip and install.img.gz installation methods should fully work and running most packages should work out of the box. Well, except sound. That’s still non-existing for Pi4 and Pi5 at the moment.
At least the sound on Pi3 and earlier should be fully restored and uses libswresample now.
Changed the initramfs format from lzma to gz as most files inside are now already compressed. This makes it a lot faster on older devices like the Pi1.
Basically it’s almost ready for testing. Please get in contact if you have a Pi5 and want to check out this preview version.
Made updating EEPROM on Pi4/Pi5 dependent on the version as well as release date of the bundled versions compared to those running on the Pi. This should ensure the info-beamer provided version with its configuration is installed, even if the running one has the same version number.
Added missing kernel modules for H264 decoding on the Pi4 . It actually works there now.
Seems like the latest EEPROM for the Pi5 now also supports HTTP network booting, so that mode is now working as well.
Combined with the website update earlier today, the Pi5 can now almost completely configured using the configuration editor on the dashboard. Including resolution and dual display settings. It’s now also possible to assign Pi4 content to a Pi5 as they are considered compatible.
The release is now ready for testing. Get in contact if you’re interested.
Updates since last post:
Optimized memory usage on Pi1 (those have 128MB usable memory )
Improved snapshotting while info-beamer isn’t running
Added fan support for Pi5
The last remaining larger topic is now sound support on the Pi4/Pi5. Compared to previous versions this will now use Alsa and as a result should now support outputting audio to two displays at the same time. Still not entirely sure how to best approach this topic and it will take a bit before it’s being completed.
Another interesting topic is HDR support on the Pi4 and Pi5. Given that info-beamer now uses DRM/KMS for placing output on the screen, this should now be possible. Still not entirely sure how that works and if OpenGL content, for example, might also make use of that. All very interesting…
Known issues for the preview info-beamer hosted OS release running on the Pi4 and Pi5:
No audio support yet
Flickering when using transparent videos across multiple displays on the Pi5: This is most likely a Linux kernel bug in the VC4 drm implementation. I’ve opened an issue here.
Putting displays to sleep doesn’t really work yet as CEC support isn’t implemented yet.
A first beta1 of the info-beamer pi tool is now available too. If you have a Pi5 with Bookworm 32bit running, you can download the version here. A 64bit version is also possible, but there wasn’t enough time for testing yet and I don’t really expect any notable performance difference.
Update March 2024: A Pi5 compatible version is now available in the testing OS channel. See Info-beamer hosted OS 14 release thread.
. It actually works there now.
Known issues for the preview info-beamer hosted OS release running on the Pi4 and Pi5: