Hi,
I’m developing a project to stream video and audio in real time between casino gaming units within the same corporate LAN.
RTSP works relatively well, but I’d like to stream based on WebRTC, as it allows for much greater flexibility, efficiency, and peer-to-peer communication, and I want to keep this within the info-beamer environment.
Is it possible to enable WebRTC capabilities in Chromium, part of the Web Browser Package?
I just had a similar request via email. Here’s an excerpt from my response:
Right now there’s no WebRTC support. I’m also not really sure how that protocol works. From what I understand FFmpeg added WebRTC just very recently (see FFmpeg merges WebRTC support | Hacker News), although I’m not entirely sure if that’s not just the code for sending WebRTC as opposed to receiving it.
I’m also not sure how long it will take until this eventually gets merged into the RPI customized FFmpeg version info-beamer uses. Right now there’s no WebRTC support.
If that’s the feature needed and it eventually ends up included in info-beamer’s FFmpeg version, I think it should just work like any other stream urls as it’s just another transport method and the included video stream is still just H264/HEVC, so there’s no change required at all on how such video would be handled.
Managing a custom FFmpeg fork is not viable, so if the above WebRTC patch is indeed what’s required, I’ll have to wait until that’s merged into Pi’s forked version. Currently that’s GitHub - jc-kynesim/rpi-ffmpeg at test/7.1/main.
Not sure about the chromium part: I would have guessed WebRTC is already enabled there. I guess not? But even if so, there’s neither hardware acceleration when decoding a received video data, nor a fast path to then place a decoded video frame on the display. It’s going to be slow.
A very viable alternative is to use either RTP or UDP broadcast or multicast of H264 streams. Those are already natively supported from within info-beamer’s player and can achieve close to realtime playback with native decoding support.
My use case in this case would be for receiving (video-audio) WebRTC.
Previously, I was getting an error message stating that WebRTC was not supported in the Web Browser package using Chromium. However, after testing again, it was indeed able to receive a WebRTC stream. However, as you mentioned, there is no hardware acceleration when decoding the received video data, resulting in many dropped frames, but with very low latency (~200ms) 
.
The network environment where I’m trying to implement this is a bit old and with very little flexibility for changes (firewall, port, etc.), so WebRTC provides the ease and flexibility I need (and my users too).
There’s a project that integrates WebRTC very well with the Raspberry Pi (https://github.com/steveseguin/raspberry_ninja) to perform a WebRTC transmission, but in this case, as in almost all others, it uses gstreamer, but the decoding is done by the Chromium browser, as I understand it.
With RTSP, video works relatively well, with low latency (with the libav fflags:flush_packets+nobuffer option), but audio often fails occasionally. Packet loss is a problem in the network environment where I use it, so it improves by using the TCP protocol instead of UDP (rtsp_transport:tcp).
I haven’t tried multicasting, but since there’s no flexibility in routers or switches to implement it, I find it more difficult to implement.
Hopefully, stable WebRTC support can be implemented in ffmpeg in the short or medium term, but that’s no longer so dependent on Info-Beamer.
Thanks Florian!
How many network segments are there? How many devices in there should then receive the stream? If it’s most of them and it’s only a single stream or so, you might also just broadcast. Although both multi- and broadcast won’t work with TCP for obvious reasons. Why is there packet loss?
I’ve now looked a bit deeper into the WebRTC patch and from what I understand, it’s the wrong side of transferring video via WebRTC. I think the part the patch implements is for contacting a receiving peer, offering the stream and then sending it there (WHIP). info-beamer would need the opposite (WHEP) as it would need to receive such a stream.
I also found pion and one of the examples (rtp-forwarder) should fill the gap here. It’s a bit convoluted, but I managed to stream from a browser (the linked jsfiddle page acting as WebRTC sender) to rtp-forwarder running locally on my machine (acting as WebRTC receiver). rtp-forwarder the sends the stream via UDP to 127.0.0.1:4002 and that can the played by ffplay. The default video codec seems to be vp8, so that’s not decodable by info-beamer. Not sure what’s required to use H264 instead.
I’m not sure how the to do the SDP exchange automatically as I had to manually copy/paste the jsfiddle base64 SDP blob into rtp-forwarder and its output into the jsfiddle text field.
There’s quite a bit of mystery surrounding the network topology where I’m trying to implement this; not even the IT people are entirely clear about it; it’s old and has many cascades everywhere and 100 Mbps links. I’ve implemented other network transmission protocols such as DANTE (audio streaming over IP) and NDI, but with some effort. In addition, the corporate network is occupied by the CCTV system that manages the cameras nationwide, which occupies a large part of the bandwidth. All of this partly explains the packet loss. I chose TCP over UDP only because of the artifacts that are generated (from time to time) when playing RTSP with UDP in this network environment.
Yes, you’re right, the webrtc merge in ffmpeg is only the whip protocol part. A Whep player in ffmpeg would be the closest thing to natively playing it in info-beamer, but from what I’ve read, it’s still in early development for ffmpeg.
That’s why my hopes were on the “Chromium” in web browser package.
Yes, pion was on my list to investigate. I also ran the test with rtc-forwarder; apparently, it’s a bit more complex to do with h264 (I also didn’t succeed), and the SDP exchange part (offer-answer model) would have to be developed with the webrtc API (I’m not sure how to proceed with that either).
In any case, RTSP with Info-Beamer has worked well for me, but since I need to implement it in another area (on the same corporate network), I wanted to implement it with WebRTC and Info-Beamer, as it offers many optimizations and flexibility. I currently have a system with a streaming server (MistServer), and it works very well with WebRTC and a variety of streaming protocols, but outside of the Info-Beamer environment, except for RTSP re-streaming, but latency increases.