[Basic] Yet another A.I. project KERAS-RL

Showcase
1

Hi

I created a new project to use Keras-RL :

You could can implement your env/agent as you want.

After that you can export your model to TensorFlow format and run with GitHub - armory3d/tensorflow_example: Using tensorflow in armory

I’m a newbie (in ML) and I would use RL to train a (fast) runner :
deep

++

7 Likes
2

Hi,

I chose an easier problem to test my basicia lib : “goto problem”.

and I implemented a method to visualize a trajectory :
VDebug.addDrag(mypos,color, 3, "myid", 200);

run

This article helped me to understand the reward function :

2 Likes
3

I finished the pipeline: Train (keras) > Export > Exploit (tensorflow.js)
But I have to find a new pipeline: tensorflow.JS is too slow :dizzy_face: ( 500ms / 25predicts)!

gif (unrepresentative speed :):laughing:)

run-live
run-live.gif1280×780 1.78 MB

…

4

Thanks for the link @Marc … it’s the domain where I am, that’s as written in the video, “bridging the gap from games to industry” . This conference shows perfectly well the requirements + challenges that exists for doing that.

If you look at this post IA, Python and Armory 3d/Blender, I try to show the progress I had with Armory, with a focus on which tasks could be taught using the available logic nodes, Haxe, deep reinforcement learning with neural network inside. It’s a way to teach some experimental systems and results obtained was quite good, as everything stay inside logic nodes for performances and be able to make at end a way to :clap:

  • attach a kind of brain to some parts of a system that is a 3D object in Armory;
  • train them separately;
  • and finaly reassemble trained brain inside a whole system like if you were adding a cube to an existing mesh, thus adding it some kind of intelligence.

I am actually looking for it, with a combination of an ECS approach joined to the 3D Armory environment capabilities.

When looking at the Bonsai approach, with it’s Inkling langage, https://docs.bons.ai/guides/inkling2-guide.html#inkling-a-new-language as a way to express “the solution to a problem in terms of how you teach the computer to find the solution as opposed to how to calculate the solution”, it’s seems to me a little bit complicated and time consuming for people in industrial environment.

I would prefer to propose an intuitive and a kind of a child-friendly way to do it with 3D objects.
For example “L’usine de robots” de Tinybop https://youtu.be/P1G_ofvsWxU

image
image.jpg1066×588 89.6 KB
is child-friendly way to compose a robot and that’s inspiring to make an easy machine teaching tool.

Thus to make the machine to learn, we choose what kind of trained parts to add to it.
It could be as simple as using Add a new Mesh in Blender and then
image
have a list of command, like we have with Object

image
image.png240×650 47.8 KB

in order to define for example the input/output we have in the created Brain, that will in back-office make the good choice of the algorithm for the brain and prepare the training environment for us, where to train seperately each brain with available easy to use tools usable to adapt goals, rewards, values or termination conditions for example.

5

Hi,

I’m delighted to read from you. And I tried to read your very (very) long thread ( IA, Python and Armory 3d/Blender) I’m not sure to understand everything.

you implemented an NN lib in Haxe and you’re implementing (very) high level " RL
framework" with the node Editor. :partying_face:.

Did you have any performance issues (train/prod)?
Did you find how to use CUDA/Nvidia?
Did you use genetic algorithms (for hyper-params)?
Where is the code?

I chose a different way… I’m developer :slight_smile: LogicNode it’s not the big deal. And there are a lot of ressources to learn Keras/Tensorflow.

Currently, my plan:

  • In production: I have to avoid performance issues
  • In training: implement genetic algorithms ( noob-friendly and faster),
  • Create a sandbox video-game playable.
6

I hope to answer your questions with these few clarifications.

I developed my own DRL (Deep Reinforcement Learning) code in Haxe langage with the objectiv to encapsulate them into logic nodes and traits in Armory, and be able to test different kind of algorithms and architecture for the DRL or NN parts as well as the different sensors simulated that we can actually use in an industrial environment to define each state during a NN training.

As I shown in different posts, firts results are ok. Main issue is a huge quantity of nodes (existing or new) and traits used to make an application like the “tank” or “robotic arm” one.

Thus actually I consider to redesign the software “de fond en comble” for distributing RL components in a composable way by using an ECS-like approach (Entity Component System), with the objectiv too to get independant threadable parts and to have the possibilty to develop a new concept of “Dynamic of a Layer in a Neural Network”, which defines a form of evolution over time of the NN Layer, like what we make with a target in a software for a radar system.

First main task is to get a kind of top-down hierarchical control as I found they already realize in the Bonsai solution. For me I think that the simplest way seems today possible by reusing the capabilities of linked 3D objects and parenting in Armory combined with a kind of ECS design approach in order to encapsulate resource requirements within short-running compute tasks with their own independant data, that will facilitate then some usage of a GPU or CPU cluster during the training of “brains” for sub-systems or during the gameplay of trained “brains” for the whole system.

I think to use this possible clusterisation of the brain training for example to get the best parameters for a NN, given a sub-system brain / training environment used/… , thus making a kind of Genetic Algorithm that will allow to get a NN that will perform properly according to a sub-system where the brain belongs to.

Thus the code is always in evolution/redisign and includes special “bidouilles” to obtain results and performances with Haxe/logic nodes/traits, thus this largely differ from the well-made generalist solutions found in libraries such as Tensorflow, but allow for me to explore, get a better understanding of DRL algoritms we find today in the research field, a very active area, and test new ways of using them inside innovativ software design, thanks to what is offered to us using the promising 3D Armory environment.

My biggest concern is the lack of interest right now of its creator Lubos to share with the community, with some people ready to help him for some support on Armory but now who are beginning to tire of deafening silences to their proposals. It’s worrying for the future.

1 Like