Raspberry Pi 4 release and info-beamer information

infobeamer-fw · June 27, 2019, 2:52pm

Got to work a bit with the Pi4 today. Got the OS itself to boot albeit with a lot of minor and probably also not yet detected major issues that need fixing…

Right now the info-beamer program itself doesn’t run and it seems that a rather big rewrite of some components is required. Reason is that OpenGL not works differently compared to earlier releases. Previously there was a Broadcom provided OpenGLES2 driver that directly talked to the hardware. This option is unfortunately no longer available.

I expect this to take a while. sigh.

johnniewalker.email · June 27, 2019, 3:02pm

I’m sure if anyone can figure it out then you can Florian, it’s way above my abilities that’s for sure

I guess for when it’s more common to deliver 4k content then it’s important to update the software to support the Pi4, to allow us all to continue using info-beamer!

infobeamer-fw · June 27, 2019, 3:07pm

No worries, there will eventually be a version. I just can’t make any promises about any kind release date yet. Understanding all the APIs required to make that work is a bit of a challenge. I’ll keep this thread updated of course.

LoopForYou · June 28, 2019, 3:24am

2 × micro-HDMI ports (up to 4kp60 supported)

Would be cool if you we could make video walls with just 1 Pi4 or scheduled player that support two different playlists; one per screen.

infobeamer-fw · June 28, 2019, 2:19pm

Totally, but let’s start with the basics first: Getting anything drawn on the Pi4

Speaking of: I managed to overlay a GL layer (the cube) over a video (that lego thingy in the background).

That might not look like much, but proofs that what info-beamer did in the past can also be done on the Pi4. The only major uncertainty now is the ability to render videos into a texture. This is for example used in the magic video wall to rotate the videos into their correct position. No idea how to approach this yet…

infobeamer-fw · June 29, 2019, 4:14pm

Another update and some observations, but first a snapshot:

This is existing info-beamer code running on a current build of the Pi4 version. This version now uses the DRM & GBM APIs to open up an OpenGL context. This GL layer can then be combined with video layers as seen in the above image where a video is used as the background. This means that packages like the HD Player should in theory work with that version. \o/

Some observations:

The texture size limit was raised from 2048x2048 to 8192x8192 pixels. So it’s now possible to load and render 4K images directly. So that’s also something that has to be reflected in the web dashboard as it still restricts to the 2048x2048 limit.
I’ve managed to generate a 4K video signal and render out some content. I don’t have a native 4K display yet, but the NEC P404 was able to display that signal by downscaling it.
Not sure how H265 decoding works yet. Just feeding the hardware decoder H265 video frames doesn’t work as the decoder seems to be confused and doesn’t decode anything.
Still not sure how video decoding into GL textures works.
There’s some flickering going on and the current output doesn’t feel as smooth as on earlier Pi versions. That’s probably my fault for not yet understanding how everything plays together.
No idea how to generate output on two attached displays yet.
DRM seems to have realtime information about the attached display. This might be used in the future to dynamically switch resolution depending on the attached display
I’m not sure if it’s possible to generate a GL surface that doesn’t match the pixel size of the selected output. In Lua, this means that right now only gl.setup(NATIVE_WIDTH, NATIVE_HEIGHT) is guaranteed to work.

If you want to play around with the current version on Raspbian buster, feel free to download this preview release for evaluation purposes only.

johnniewalker.email · June 29, 2019, 7:47pm

Making good progress

infobeamer-fw · July 1, 2019, 9:42am

I’ve also found examples that show how to decode video into GL textures on the Pi4. But that’s in no way added to info-beamer pi yet. It’s all still very early and the complete programming environment feel very unstable and undocumented. Basically the same mess I had to go through when initially porting info-beamer to the Pi a few years ago. I also expect quite a few bug fixes on the Pi4 firmware/kernel itself before I would recommend using it in production.

Don’t expect any major updates in the next 2-3 weeks. I’ll have to ingest a lot of new (underdocumented) APIs.

Until then, my recommendation would be: Unless you need 4K in the future, the Pi4 doesn’t seem to bring anything major to the table yet. It needs more power, gets hotter and will have to proof itself first. For now, just use Pi3s

infobeamer-fw · July 29, 2019, 5:19pm

Smallish steps. A first official info-beamer pi for the Pi4 prerelease is now available on the info-beamer pi download page. A few things that don’t work right now:

GL videos (those that you can scale/rotate/transform arbitrarily) are still not possible.
I’m still now sure how all the 4K stuff and targeting multiple monitors work.
There’s some screen tearing (see this example picture) going on. Not sure why.
Some GL operations are slower on the Pi4. My guess is that the new GL driver is still rather untested and there’s a lot that will improve in the future.
No 4K/H265 video support.

Documentation on all of that (e.g. linux DRM/GBM/mmal/how to play H265) is utter garbage, so it’s continuing to be a slow progress due to a lot of trial and error and waiting on more usable example code or documentation. It’s all quite frustrating.

A new info-beamer hosted OS using buster and this new info-beamer pi release will eventually be available for testing. Don’t expect that to be usable in any kind of production environment for a while though. The recommendation to use the battle tested Pi3 still stands

infobeamer-fw · August 2, 2019, 11:49am

I’ve successfully managed to get the dual display support for GL content. See this picture:

The Pi is connected to the left and top monitor and powers both of them. In this example the combined screen area is then 3840✕1080 (so 2✕ FullHD) and I load a 3840✕1080 JPEG and show that across both screens. The updated info-beamer pi version is available now.

Here’s how that works currently (subject to change in the future):

INFOBEAMER_DUAL_DISPLAY=1 ./info-beamer example

This instructs info-beamer pi to try to open two attached displays. It will use the preferred mode from the display connected to the HDMI0 port on the Pi (in the above case that’s 1920✕1080) and will try to find a matching mode (same width/height/refresh rate) for the display connected to HDMI1.

You can also manually specify modes using

INFOBEAMER_PRIMARY_MODE=xxx
INFOBEAMER_SECONDARY_MODE=xxx

with xxx being one of the modes listed in the info-beamer pi output.

You can freely place the secondary screen relative to the primary screen by using:

INFOBEAMER_SECONDARY_OFFSET_X=x
INFOBEAMER_SECONDARY_OFFSET_Y=y

with x/y being the pixel offset of the top left corner of the secondary screen relative to the top left corner of the primary screen. Having both values set to 0 (the default) effectively clones the output, so both show the same content. Setting INFOBEAMER_SECONDARY_OFFSET_X=1920 will place the second display right of the primary display. And so on.

It seems that right now the Pi4 software doesn’t allow hotplugging of displays. It sounds like the displays are only detected when the Pi boots. This might make reliable installations using dual displays difficult, if the screens are not guaranteed to be turned on when the Pi starts. If a screen is not detected at boot, info-beamer (and I’m pretty sure any other Pi based solution) won’t be able to detect the screen later once it gets turned on). As a result, I’m not yet sure how practical the dual screen mode will be. We’ll see.

Right now only GL content can be shown across dual screens. Videos of any kind won’t work as of now.

infobeamer-fw · August 6, 2019, 6:16pm

A first GL based video decoder works!

I consider that a major breakthrough. Here’s it in action (I’ll probably upload a YouTube video tomorrow):

I’m using the placement feature from the previous post to create s surface spanning two screens. Then I load and play a video across both of them:

gl.setup(NATIVE_WIDTH, NATIVE_HEIGHT)

local vid = resource.load_video{
    file = "costa-rica.mp4",
}

function node.render()
    gl.clear(1,0,0,1)
    vid:draw(0, 0, WIDTH, HEIGHT)
end

This was probably one of the biggest hurdles porting info-beamer pi to the Pi4 and at least the hard part seems solved. I cannot really make any definitive statement about performance, but I looks pretty good so far.

Of course there’s still a lot to do with regards to video decoding:

Right now the video is played as fast as the Pi can. There’s no scheduling. I’ve yet to find out how this works with the new MMAL API I’m using.
There’s no audio support yet.
There’s no proper shutdown procedure for video playback. It just fizzles out and produces error messages. Oh well.
It seems due to alignment(?) some videos have a black bar at the bottom (and possibly on the right side).
I want to reuse the same code for non-GL videos. Previously those were two implementations and it’s been annoying to handle. Having a single player code that can produce both GL and dispmanx output would be nice and seems possible.

Unsolved: 4K and H265 video support. Decoding required a completely different approach and that sounds like a hell to implement. The library/kernel(?) code required to make this work doesn’t seem to exist yet, so that’s going to be interesting.

infobeamer-fw · August 8, 2019, 6:23pm

infobeamer-fw · August 17, 2019, 7:57pm

Exciting news: A new prerelease is now available for download. It’s almost feature complete! What works:

H264 Video playback both directly on the screen or into an OpenGL texture that can then be freely placed/rotated. The latter is for example used in the magic video wall.
Audio output now works. Although I didn’t test all cases yet:
- No idea if analog output works, but I guess it should.
- Not sure how the HDMI target for audio is selected. Right now I only hear audio on the primary (HDMI0) display.
Correct scheduling of video/audio. It should now be in sync.
Transparent playback of content on two cloned displays or across two side-by-side displays for a larger virtual screen. This even works for accelerated video playback. I don’t think any other software on the Pi supports this at the moment.
It’s largely compatible with existing info-beamer code. Only minor changes will be required for some special cases.

What doesn’t work yet:

Loading videos in paused state works, but some audio gets swallowed when doing that. Not sure why yet. But I remember the hassle of getting that work for earlier version, so this will get solved eventually.
The resolutions available seem to be independent from those set using /boot/config.txt. I’ll have to see how this all fits together when integrated with info-beamer hosted, so that the resolution selector works as before.
The :target, :source and :rotate methods for raw videos (see here) are difficult to support with the new potential dual screen setups. There has been a more sensible :place method for multiple years now and I think it’s time to remove the obsoleted methods. This might require some minor changes for existing packages.
HVEC/H265 playback. This is it’s own little hellhole. There’s no documentation and the only implementation seems to be a patched version of ffmpeg. Ugh. But from what I understand, once there’s an easier way to interface with the hardware decoder, the rest of the video playback remains largely unchanged, so all the features metioned above should work together with a future H265 decoder. We’ll see.
Dual screen vsync has its issues and it sometimes results in screen tearing. Not sure yet how this can be reliably solved.
The neat fade-to-white effect in info-beamer hosted welcome screen is no longer possible
Playing lots of GL videos in rapid succession sometimes crashes the Pi firmware. No idea why.
It’s very untested and I’m sure there’s a few crashes here and there.
And probably a lot more I forgot…

The next step is now to release a new testing version of info-beamer hosted using the new Pi4 version. Exciting indeed

infobeamer-fw · August 20, 2019, 4:06pm

There’s now a first working version of info-beamer hosted OS for the Pi4! It’s still a bit rough around the edges but some packages immediately work out of the box. The OS grew massively from 37MB in total size to 67MB. Eventually this is something that might get improved, but for now it’s acceptable.

The next step is now to add compatibility information to the web dashboard and the package description, so you can immediately see which packages are ready and get a warning if a package isn’t marked as compatible with the Pi4. This will be finished this week and I’ll hope to have a first testing release publicly available by Friday.

Known issues at the moment:

Right now loading the GPU driver crashes with multiple displays attached. This is odd as I don’t see this on stock Raspbian. I’ll have to see what the difference is. If that’s resolved, having a cloned output mode could be immediately added to the web configuration interface.
Reported memory usage is wrong. The current method can only see the first GB, so for the 2GB/4GB variant of the Pi4 are incorrectly shown as 1GB. This should be rather easy to fix.
The info-beamer pi software is still a bit unfinished (see the previous post) and there are multiple ways to crash it accidentally. Although it’s unlikely that it’s triggered by any of the existing packages, I’d really want the software completely crash free, so this still requires some work.

If you have a Pi4 and want to try out a first release, please get in contact and you’ll get a few free credits so you play around with it. This is limited to the first 20 requests.

infobeamer-fw · August 22, 2019, 6:04pm

Progress is being made. The UI has been updated. Each package now specifies which devices it’s compatible with:

See here for a full list. By default all packages are compatible with pi/epoch-1, so all devices up to the Pi3B+. Next step is now to update all packages in the store. As I mentioned before, most changes should be fairly minimal. See this change for example.

When assigning a setup to a device, you might now see a red Assign button like this:

This means that the setups uses a package that isn’t marked as compatible. Clicking the Assign button then warns you about this. If you want, you can assign the setup anyway, but don’t expect that to work.

Next up: A first testing release. Most likely tomorrow. Stay tuned and ready up your Pi4s

infobeamer-fw · August 23, 2019, 12:03pm

The testing release is now available. If you have a Pi4, feel free to play around with it. Known issues right now:

Random firmware lockups when playing videos. It might take a while for this to happen in your installation. Still trying to figure out how to handle/debug this.
Memory reported on the dashboard is still wrong
Seamless videos (in video wall installations) with audio swallow a bit of the audio at the start.
Screen tearing with some GL content. Not sure yet how to fix this.

Not yet supported:

Dual display mode for a larger presentation area. While the info-beamer software supports this already, it’s not exposed yet in info-beamer hosted and cannot be configured at the moment. Stay tuned.

New features:

Dual display cloned mode. Just connect a second display to the connector labeled HDMI1. info-beamer will try to match the resolution of that secondary display to the one connected to HDMI0. So if you have a FullHD screen connected to HDMI0, HDMI1 will also use FullHD if possible. Incompatible screens won’t work and info-beamer won’t start in that case at the moment.

infobeamer-fw · August 28, 2019, 4:58pm

New versions of the info-beamer hosted testing release as well as a new standalone info-beamer-pi version is now available.

Changes from previous version:

Improvements regarding screen tearing and most of the issues should now be fixed. In addition on a dual monitor installation, the framerate should improve quite a lot.
OpenGL based videos (like used in the Magic Video Wall) are now faster than before on the Pi4. FullHD should now work perfectly smooth.
The info-beamer hosted distribution size shrunk by almost 16MB and it’s now back under 50MB for the complete OS. That keeps things nice and fast.

If you have a Pi4, please test with this version. If you have the previous version installed, simply visit the device page and click on Manage > Activate testing channel… to upgrade. As mentioned before: Contact support for some free credits so you can play around without thinking about your wallet.

The big remaining feature that should eventually end up in info-beamer is H265 support. Some standalone experiments seem promising, but it’s in no way integrated into the rest of the info-beamer program. Stay tuned.

infobeamer-fw · August 29, 2019, 3:14pm

Configuration editor UI has been updated for new Pi4 dual screen features

For a Pi4, the screen resolution settings now look a bit different than before. You can now not only select a video mode, but also select how and if you want to handle multiple screens. Right now there’s 6 different modes:

Single screen (like before on earlier Pis)
Cloned output. So two screens showing the same content
4 different arrangements of both screens next to each other.

Right now the UI only allows the same resolution on both screens. Additionally, you can’t freely place the screens, which might be interesting if they don’t have the same size. That might be added in the future though as the system already supports that.

Update: Of course that now means that dual screen video playback is now possible with a single Pi4. Have a look at this quickly created video showing this feature in action. Right now it’s still limited to H264 and thus 1920x1080 (or most likely something like 1920x540) for videos like the one below, but once H265 support is done, you’ll be able to use 3840x1080 for example.

infobeamer-fw · September 16, 2019, 11:49am

4K / H265 video playback proof of concept

This is another important milestone on using the Pi4’s full potential: My in-dev version of info-beamer pi can now play 4K videos encoded in the H265/HEVC format for the first time. As it’s getting integrated into the normal info-beamer video runtime, splitting a video like this across two screens also works:

This can be used to play 3840x1080 (so 2xFullHD) across two FullHD displays. Previously you have been limited to 1920x1080 stretched across two displays thus cutting the horizontal resolution in half. With the H265 decoder, you can now use the full native resolution of both displays.

On 4K displays, playing 4K videos also works.

It’s still some work to get everything running smoothly and to properly understand all the limitations of the new decoding mechanism. Right now I’m not sure if it’s possible to decode two videos at once, for example. In addition, decoding H265 into a GL texture (for use with the magic video wall for example) also doesn’t work yet.

And for course for integration into the info-beamer hosted service, some changes have to be made regarding device compatibility. This would prevent you from installing a 4K setup on an older Pi that cannot decode them. So there’s still a ton of work to do, but it’s getting there. Stay tuned!

infobeamer-fw · September 20, 2019, 5:07pm

Quick status update. The core video player component has been rewritten to be more generic. It can now decode H264 into GL textures, H264 into fullscreen over/underlays as well as HEVC (so 4K videos!) into over/underlays. See this screenshot:

All of that works seamlessly across two displays as well!

It’s still a bit more work to make pausing, preloading, audio and looping possible with all modes, but I see light at the end of the tunnel. Hopefully there’s a new info-beamer pi release supporting all that in the near future. Stay tuned.