I dont know how to code .so can you give me download link for nobe and tutorial will be helpfull.Btw awsome work.Ps if you see armory github paje you will see that krom support tensorflow.
You can start here with coding in Haxe https://code.haxe.org/
For info, NNFactory is made all in Haxe with no dependencies with others libraries in the NN domain and with the idea to have something very specific to Armory capabilities for example to develop new systems equiped with IA and have a tool for tests and explorations of new concepts.
Amazing progress. Yes there will be tensorflow support, but having all-in haxe library with nodes would be super cool.
First training & test for a Neural Network in Armory : training output at 3100th step with 100% success rate of predictions on the learning set. Then 100% of the samples with defects are detected. Now the distribution of the NN haxe code into the Armory nodes begins.
The speed of learning if very good too with env 1 sec/1000 iterations.
That sounds absolutely awesome! I hope you will share your work with the community .
@donalffons Sure, this is already the case. And new players to prepare 3D experimentations, inspired by those showned before in this thread, are welcome !
Armory 0.4 is out and free!
This nodes architecture called NNA (for Neural Network Armory), is now operationnal :
- on the left we train the neural network with a training set coming from a Json file
- on the right we ask a question to the neural network (again the question is inside a json file)
Next step will be to use data coming from objects in the Amory scene … see preceeding examples.
Register the camera view for some post-processing
This looks highly promising, good work!
Which network types besides Perceptron are currently supported?
@NothanUmber The Perceptron is the current one with a sigmoid function. Feed-Forward and Backpropagation are available inside the Node Factory for training and Feed-Forward inside the Node Activation to maintain high performance for real-time applications.
Armory Nodes could then offer the possibility of testing different networks and architectures, such as coupling several different networks.
What we could be inspired for NNA tests … how to propose to game developers in Armory a method (1) that will learns a 3D human pose without any 3D datasets or (2) to be able of reconstructing a full 3D mesh of a human body from a single RGB image.
see here an example for (1): “Our method can predict a 3D pose from 2D joint locations in a single image. Our system is based on the generative adversarial networks, and the networks are trained in an unsupervised manner. Our primary idea is that, if the network can predict a 3D human pose correctly, the 3D pose that is projected onto a 2D plane should not collapse even if it is rotated perpendicularly. We evaluated the performance of our method using Human3.6M and the MPII dataset and showed that our network can predict a 3D pose well even if the 3D dataset is not available during training.”
Interesting! Extracting 3d meshes from rgb images would be a more difficult task than the one they solve in this paper though. (they are “just” guessing the z coordinate from an already available 2d mesh). My guess is that you will need some kind of CNN at least for the generator network instead of just MLPs. (Not sure why they use MLPs in the paper instead of CNNs - doesn’t this make the task more difficult as the networks first have to learn how to decode the spatial structure of the 2d mesh from the linearized input vector?)
In which scenario would you want to use the 3d-mesh from rgb image network? Wouldn’t the 3d mesh infos of all objects in an Armory3d game already be available to the game AI? Or do you want to build something that plays 3rd party games just from screenshots? (Like the Atari games playing RL agents from Deepmind)
Edit: Ah, you want to build some motion capturing software that just works with 2d images from a single camera?
Edit2: Ah, the approach in the second paper looks more like it
@NothanUmber Thanks for your comments.
In short, I’m trying to discover if there can exist and develop a kind of ecosystem of enthusiasts around the use of NNA and what could be the most expected uses (cf. the examples we find of what is done elsewhere with NN and 3d-mesh from rgb image is one of them that could interest some of us to develop)?
First, on my side, I test a tracking that could then be portable on a machine equipped with a camera. Taking care of RealTime constraints is determinant. One of the advantages here is to be able to quickly generate object views via 3D for NN training without having to go through labeled or unlabeled web image libraries.
Another is to be able to do the training on a 3D modeled environment, which takes into account as many difficulties as possible before going on the field.
In a second time, it will be the opportunity to test various NN architectures thanks to the ease offered by the use of nodes. See an example of a concept on this diagram.https://einstein.ai/research/domain-specific-language-for-automated-rnn-architecture-search)
@Didier can I ask, are you using any python libraries behind this or reimplementing entirely in haxe?
I’ve been recently learning PyTorch so was curious.
Everything is in Haxe and not coming from Python library … the simplest and small code as possible to do it.
As Lubos wrote “Behind the scenes, Armory is powered by open-source technology. Utilizing Kha - multimedia framework and Haxe - cross platform toolkit, to provide top class performance and portability. “ and it is mandatory for me to stay in line with that … look in the forum at what I suppose those who could participate to a kind of ecosysteme are also waiting for.
For the moment, I need to find a successful way to easily acquire the camera view in Armory, transform it into an array of pixels and feed the NNA with it (the Armory code doesn’t seem to me to be complete right now to do this and I hope @Lubos will now have more time after delivering the 0.4 release to address the issue).
Thus with NNA, tests are already done with small matrix representing ASCII characters and performances are at rendez-vous thanks to the NNA logic nodes coded all only in haxe, focused on only one thing : near real time performances to simulate reality + portability on different kinds of small platforms.
For your information, during this time I progress also on the realization of other logic nodes which will allow us to realize interesting things on the one hand for games, but also for real applications. Thus @AnadinX if you are learning Pytorch, I propose to you to focus on this example in Pytorch https://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html#sphx-glr-intermediate-reinforcement-q-learning-py to better understand what the following logic nodes mean.
I take the example of the fireman who must move as quickly as possible through a burning building. He obviously has not the time to assimilate all the building’s plans and cannot memorize all the evolutions of his environment given in real-time by sensors and others field actors. For example, a hearing aid could indicate the best options at any time according to current needs (find, escape, etc.) and constraints (remaining oxygen, temperature, obstacles, etc.). They operate in an environment for which we will only know later if it was the right decision he took and so where it can be difficult at a given time to understand what action leads to what result through many successful steps according to given targets, and each of which can be decisive for the final result. We find the same problem for industrial processes that lead to quality defects and rejects, and it’s a challenge for the enterprise 4.0.
We will have a logic node for the Environment that is able to transform an action taken by the 3D character in the actual state into the next state and a reward. The logic node for the 3D character (Agent) transforms to a new state and reward into the next action.
Thus, to simplify for the fireman, the first step for me is to modelize in Armory some kind of simple evacuation of a character from any room in the building. This is the objective of what is called in Machine Learning jargon, Reinforcement Algorithms that incorporate Deep Learning like the actual NNA.
Thus the NNA will become a kind of Expert, pre-trained. With several of those agents specialised in sub-tasks (imagine one agent for one sensor (vision, IR, doppler, gaz, Tp, RFID, IoT com, …), we get what is called a policy that can map a state to a best action according to what the NNA Expert Agent is trained for.
I expect that if an ecosystem around NNA is growing and that step by step this ecosystem develops useful solutions to achieve real-world objectives by playing with the NNA logical node to create awwesome architectures for reinforcement learning , operating better and better in simulated and random harsh environments like in video games and then worn in the real world ( and thus do not remain limited to the video game ).
Maybe the core of allowing a fast growing ecosystem is to allow via and thanks to easy use of the logic nodes inside Armory to formalize a knowledge transfer mechanism which allows a new coming player in the network to exploit access to a kind of cloud/lake of Expert Agents, becoming the teachers. The role of the ecosystem administrator becomes to be in charge to validate when an Agent is becoming an Expert of what and then put it into the Expert NNA Cloud. The role of a NNA Switcher logic node then becomes to make/assure the best automatic connection between an Actor/Environnement needs and the Experts available in the Expert NNA Cloud.
To complete and for those interested with the preceeding post with actors/ environment/ teachers, I propose you to visit and test the DeepMind Lab which provides a suite of challenging 3D navigation and puzzle-solving tasks for learning agents. Primary purpose is to act as a testbed for research in deep reinforcement learning, it can give you an idea of what we can do and simplify/favorize creativity of the community by the use of logic nodes in Armory.
This preliminary design of logic nodes for a Deep Reinforcement Learning Armory (DRLA) is made according to the following requirements
Now we don’t need to build a json file filled with labeled training data (like we do with the actual NNA).
The Agent will interact a lot of times with the Environment (during rounds) and we observe positive and negative experiences during those experimentations. It’s like the imitation we have done with the NNA.
We figure that our Environment modifies from state s to state s’, following action a. The immediate reward we observe in the environment is r. Our objective is for a current state s, to estimate from all possible actions which action will give the maximum immediate reward plus future best rewards.
That’s the expected reward taking the action a in state, the Q(s,a), named the Q-function.
Q(s,a) = r + γ * Q(s’,a’) where γ is a factor for future uncertainty.
DEFINING ARMORY LOGIC NODES CAPABILITIES FOR DEEP Q-LEARNING
Taking the example of the tanks game, thereafter the main nodes for the DRLA (Deep Reinforcement Learning Armory).
Action: the possible actions are: move_right, move_left, move_front, move_back, wait, canon_up, canon_down, fire;
Environnement: gets the action and modifies the scene; takes a picture of the 3D scene with the selected sensor, that’s here a camera; modifies the game’s score.
State: the scene interface gives the sensor picture of the scene at t and the game score. The Reward becomes: If the action = fire and the game score increases, then reward r = 1; Missing the target when fire makes score not changing or decreasing and r = -1; moving right, left or wait makes r = 0.
Agent : the sensor picture is transformed into an input vector for the Agent’s NNA, the State’s vector; given an input state vector, the NNA predicts rewards for all 8 actions. The Action with the best reward is selected for the following t+1 round.
DRLA (Deep Reinforcement Learning Armory) preliminary design of the logic node system:
promising and to test in the futur with DRLA to discover all the potential, those new robots are mergeable in nervous system (MNS) of devices.
If you have a 3D printer, it’s then interesting to imagine make the link between a trained DRLA in Armory and those reality within your reach (Open-Source)
with a robotic arm https://github.com/NiryoRobotics/niryo_one
or with the InMoov robot
last info : thereafter the new preliminary design of the DRLA logic nodes system.
The main design evolutions concern the RLAgentTraining and the RLAgentActive.
The Send Event send the adhoc event to the tank game and Scoredujeu retreive the actual game score.
The RL Metteur en Scene is the starting point for the training/activation of the Deep Reinforcement Learning.
Wow! Push the button!
Yes ! The fun part coming soon