I ran into the need for some libraries only available for python3. It would also be nice to define some pip packages in the package.json, that the system automatically installs. Another thing would be cython support, because many libraries use it. I want to use multithreading/asyncio, but python2.7 is not very userfriendly in this area.
Do you have this on your roadmap?
Eventually, but I don’t consider that urgent at the moment as I feel there’s little to gain for a massive amount of work. For concurrent execution without threads you can use gevent. Multithreading itself should be mostly identical across versions (AFAIK) and the core python process relies heavily on it.
Explicit pip support won’t get added. The reason is that installation of a package should result in a defined state. pip would introduce an external (network) dependencies and the state you end up on a device suddenly depends on an outside source. Additionally there’s then no way for the system to verify the integrity of the local installation. Compared to the way it works right now: Every file within /space/root can and is verified and repaired automatically should there be anything wrong due to power loss or other issues. That’s an important property if you want a reliable and reproduceable system.
There’s an undocumented (I’ll change that) but fully supported feature that can help you with your request though: If you include a file named overlay.squashfs (a squashfs filesystem) within your package, it will be mounted as an overlay on top of the normal operating system files when running your package service. If you create a self-contained Python3 installation with all required dependencies within overlay.squashfs, you can use Python3 within your package service. For debugging in case of an error, you can enter your service’s environment by running /service/service.root/run shell. You’ll end up in a shell with the normal package sandboxing and the overlay active and you can run ./service manually.
Thanks for the detailed answer. I will look at the gevent API first. The squashfs feature sounds really interesting and I see that this can fullfill pretty much every need for extending the environment.
Did you change your mind? I’m asking before I dive into work finding a solution. I would like to update my custom packages and use Python modules without having to retrofit them to Python 2.7. If there’s no easy way [0] to run Python 3.6 (f-strings, type hints, etc.) on the Infobeamer I would start generating content else where and just display pixels data or websites. It’s not what I want, I would like the devices to be standalone but it’s less pain than rewriting code.
I just want to know what your plans are so I can make my next steps in the best possible way.
[0] I’ve never done it, if you can share a manual or tipps on how to get started I will give it a try. Or maybe try out pyInstaller?
Awesome! If I log in over ssh and want to enter the service environment with /service/service.root/run shell I get
$ /service/service.root/run shell
starting service
setting up iptables rules for 1
cleaned iptable rules for 1
removing existing cgroup svc1
removing existing cgroup svc1
removing existing cgroup svc1
removing existing cgroup svc1
....
Is there something I’m missing? Do I need to stop something before entering the environment?
The package runs as expected but if I open a remote terminal (not root) from the website I can’t start Python3 (interactive mode) since it can’t find python3. What do I miss?
Can you add details on how to get to the package environment to info-beamer.com/doc/debugging?
Figured it out with help from @infobeamer-fw: I need to kill the service first with sv d /service/service.root (d for down)
Is there any update to this?
I have struggled a lot with the limitations of python2, to the extent i am now running virtual machines in our datacenter dedicated to proxy-ing stuff for info-beamer, because most of the things just don’t work anymore if you have a somewhat-recent python3 installation.
Maybe we can start off with having both python2 and python3 installed, so existing packages can continue to use /usr/bin/python and get python2, but developers can choose to use /usr/bin/python3 to get python3?
The next OS update will include a method of layering additional filesystems into the environment a package service runs in and a new backend feature to support this.
Such a layer can then provide python3 in a transparent way: Basically all you need to add is "service_runtime": "python3.11" to the node.json of that node’s directory and the running service script on the device will have python3 in its default path. Stay tuned 
ohhhhh, this sounds amazing! Thank you!