Python integration for calling Machine Learning Processes

Armory Engine Haxe
1

I was wondering to what extent it would be possible to call the Blender Python Interpreter and eventually a virtual library from Armory.
The Goal would be to call Pytorch and create Real-time Learning behaviours in-Game.
My speciality is machine learning, but much of the library we use only exist in Python and would be a hassle to reimplement in other languages.
A crutch solution would be to call a shell, activate a virtual environent and then call python code from there, and communicate with it through pipes or fifos …
But that’s a crutch, that’s what I ended up doing in Unity … Even Unity’s in house solution is a hassle to use and is actually “a hack” to bypass the fact you can’t put python scripts, nor environment, nor interpreters, nor libraries in the unity assets … What they ended up doing was a ML agent that can only learn when used by the game designer … which is not the way that is most exiting to use Machine learning in games … We want the user to experience the learning.

So how much of a hassle do you think it woould be to use python code in Armory.
Let’s be clear: Not python code to manipulate armory objects … python code to do machine learning processes on standard data types …
basically multidimensional arrays of ints or floats

2

For C builds of Armory it should be possible to embed CPython, though it would probably involve some work. For the development builds of Armory games, which run in Krom, a JavaScript environment, you could maybe compile CPython to JavaScript ( which has been done before, to run it in the browser ), or build a custom version of Krom with JavaScript bindings to Python that you would call in your Armory game.

To sum it up, it would probably be a hassle, but it isn’t like it is way to far fetched to be done.

If you served like a machine learning network API that the game could communicate with, and then provided a game launcher that starts the game and the learning API server, that could be another option that doesn’t involve linking the languages. That wouldn’t work on consoles, though, and I could see why you want it fully integrated.