Updating info-beamer open source for new ffmpeg

Help
1

I currently develop the nodes for a few conventions each year, using info-beamer to integrate with other services and provide dynamic session information. We typically have 5 to 10 TVs running info-beamer hosted, and it’s been a great platform. To develop the signage quickly, I’ve been compiling the open-source info-beamer in a fork. This lets me rapidly test the node on my Linux Mint desktop, or a laptop when traveling without an RPi, then push the package when it’s working.

Over the years, I’ve had to make small tweaks to keep it compiling (and compile Python 2.7), but recently ffmpeg deprecated some APIs required for the video functions. I tried to work through the changes myself in info-beamer/video.c at master · superlou/info-beamer · GitHub , and nothing seg faults anymore, but info-beamer only shows a white frame when calling video:draw.

This is kind of an awkward ask, and I understand the need to abandon support of the open source info-beamer code base, but does anyone have some tips on what I might be missing? My intent is not avoid licensing info-beamer hosted, as the hosted environment has been a great tool.

2

Oh my. That old code. None of that exists in the Pi version any more. The reason the swscale stuff is in there is because ffmpeg spits out three YUV420 planes, which cannot be imported as GL texture directly. So swscale converts that into linear RGB. From a brief test run, I think there’s something wrong there.

3

Thanks, I’ll start there. I was kind of crossing my fingers swscale was ok, since that API hadn’t changed, but I’ll need to dig into it.

One thing that was confusing me, is I couldn’t figure out what calls video_next_frame(…). I think video_draw(…) is the node API function to advance manually, but video_next_frame(…) looks like it’s whats responsible for actually decoding the next frame. However, if I put a breakpoint in there, I never see that function called. I do see video_draw(...)being called each frame. Is that expected?

4

That was one of the changes needed when porting to the Pi: On the Pi version, video decoding is completely handled in a background thread, independent from the main Lua/OpenGL drawing loop. video:draw() just renders the current decoded frame, while the video thread advances in the background automatically.

In the initial info-beamer version, being run on vastly faster machines, I didn’t bother with that and decoding was done on demand with video:next() in the main thread. If you take a look at userlib.lua, the “videoplayer” has a draw function that handle advancing the video based on its framerate there. So that’s not really semantically compatible with how video decoding now works on the Pi version. You could probably write a small Lua-side wrapper.

5

Ah, I didn’t realize the Lua-facing API for video playback had changed so much. I was able to get video playback working: I wasn’t checking the return value of avcodec_receive_frame(..) properly or allocating a new AVPacket if I needed to get a new packet in the same read attempt.

Does the following capture the difference in API between then and now?

gl.setup(1920, 1080)

local video = resource.load_video("video.mp4", false, true, false)
local video_player = util.videoplayer("video.mp4")

function node.render()
    gl.clear(0, 0, 0, 1)

    -- Old open-source info-beamer way
    video_player:draw(WIDTH / 2, HEIGHT / 2, WIDTH, HEIGHT)

    -- New info-beamer way, but shouldn't require next()
    video:draw(0, 0, WIDTH / 2, HEIGHT / 2)
    video:next() -- Handled automatically in a separate thread?
end

Both ways of drawing play the video, but the modern way currently plays back twice as fast since the video is at 29.97 fps and node.render() is at 60 fps.

A Lua-side wrapper makes sense, though I would need a way to detect in Lua if the node is running on the open-source build or the real RPi build.

If I can assume fast hardware for the debugging PC, could I just add the util.videoplayer target frame logic to the node_render_self function which I think is called each tick? That said, I’m not really sure how to get all declared video structs. Something in my build environment seems a bit funny, because I can’t find the definition of certain functions/macros, and maybe that’s what is holding onto the video structs? For example:

int video_load(lua_State *L, const char *path, const char *name) {
    video_t video;
    memset(&video, 0, sizeof(video_t));

    if (!video_open(&video, path))
        return luaL_error(L, "cannot open video %s", path);

    ...

    *push_video(L) = video; // <-- This looks like it might save the video struct
                            //     but I haven't been able to find the definition.
    return 1;
}

Do you remember how push_video was defined?

6

That’s because if you call video:next() every time in node.render, advancing to the next video frame happens 60 times per second, when it should be happening 29.97 times. The draw function in util.videoplayer mostly handles this: Each time it it called, it checks how many further frames (0 or 1 mostly) need to be decoded (by calling :next()) to match the frame rate of the video (retrieved by video:fps()). You could probably do something like this (see also next question):

if sys.PLATFORM == "desktop" then
   local original_load_video = resource.load_video
   resource.load_video = function(opt)
      -- desktop version only handles a single file  argument, while Pi
      -- recommends using an option table instead (not the curly braces):
      -- local vid = resource.load_video{file = "foo.mp4", looped = true}

      -- So you might want to do something like this for all arguments
      -- you want to support:
      local file = opt.file
      local looped = opt.looped or false
      -- You can then use 'original_load_video(file)' to open the video decoder
      return {
           -- similar to the table in util.videoplayer:
           draw = function(self, x1, y1, x2, y2, alpha)
               -- implement looping, advancing to next frame and then
               -- finally call the original :draw method.
           end,
      }
   end
end

Does that help? That way video handling is contained in the video object itself and you don’t need to touch any other Lua code (like node.render as you suggested).

sys.PLATFORM is either “desktop” or “pi”

Some common C functions are generated by the prepocessor. If you look at misc.h, there’s LUA_TYPE_DECL and LUA_TYPE_IMPL. They are used in video.c as well to create some metatable and helper functions for the Lua video “type”.

7

Ooh, I didn’t realize I could override the resource.load_video when on desktop! That’s a really clean solution, and I’m going to try that for some of my other kludgy wrappers. Thanks!