Armor3d capability questions

Armory Engine General
1

My game is not a traditional “game”, but a graphic novel creator. It’s a bit similar to source filmmaker, but with a focus on still shots instead of animations, parametric characters, a free asset library, cloth sim, and IK-posing with collision response. (meaning, if you put a character’s hand on the wall, the wall should push the fingers flat)

Inside the “game”, users will add environments, lights, characters, character clothes, then pose characters and either snap still shots or key frame simple animations.

There is a remotely similar “3d VR authoring game” built in Unity. However, the Unity based asset import process is horrible, and building asset in the Unity Editor is also horrible. It might work for a simple game with simple assets, but my game needs a giant asset library (and editable assets), and this is not going to cut it…

The native way Armor3d assets are built in Blender with essentially “automatic conversation”, is really ideal… because it means if you can build it in Blender you can import it into the “game”.

I’m looking into what things Armor3d would make easy, and what things would require work on Armory3d… for example:

  1. I see there is IK/FK posing in Armory3d. Is it practical today to connect collisions to IK so you can do things like have fingers react to touching something when being IK/FK posed? Can it do collisions with detailed meshes not just collision shapes?

  2. Can armory3d load assets at runtime from blend files? If not, is there some export file format where individual assets can be encapsulated (including all necessary textures, materials, etc), so Armory3d can load them at runtime?

  3. Is there any way to programmatically create 2d UI? I see some examples based on building UI out of blender objects… but it’s very important for me to be able to dynamically generate 2d menus with just code. For example, to scroll a programmatic list of character parametrics, or do display a 2d asset browser with thumbnails.

  4. Does Armory3d have provisions for native code modules? I want to integrate my own cloth simulator code, which is written in C (eventually with compute shaders), not in Haxe. Is there a way to fit this in, or is this going to mean forking Armory3d?

2

Hi @david_j and welcome to the forum!

I think IK in Armory in general needs more work, so that would probably be outside of what is doable at the moment.

@Didier probably knows more about the state of the IK in Armory, and he has done some work writing his own and combining it with machine learning. Lots of potential there once it develops enough for general use.

Armory uses Bullet for physics and is capable of using mesh shapes for collisions instead of the basic primitives.

Armory cannot load assets from Blend files at runtime, but it has its own proprietary ( not closed source, just specific to Armory ) format that it uses for scenes. I have some work-in-progress effort on exporting projects as “Mods” that can be loaded dynamically at runtime. Each “Mod” is made in Blender just like the game itself. See my post:

The implementation has fallen a bit out of date and will needs a bit of rework to get it working with the latest Armory version. Hopefully it will be able to get merged into Armory core when I can get it updated. I’ll need to find some time and get confirmation that that is a possibility for it to get merged from @lubos before any more work is done on that.

Luckily, right now, that is the most reliable and mature way to create 2D UI in Armory. ArmorPaint’s UI is a prime example of what you can accomplish, and all of it is generated by Haxe code, so you should be set on that front.

For something like a cloth simulation, you should be fine. You have to bind the C/C++ library to Haxe, but I’ve got a nearly complete set of PRs for automatically generating the bindings based on a WebIDL file that defines the interface to the C/C++ API. For the development build of the game it uses Emscripten to compile the library to JavaScript, so the library would have to be compatible with Emscripten. As long as it doesn’t do anything super weird I think it should be fine. There are plenty of entire video games that have been compiled to JavaScript with it.

We are planning on migrating the Bullet Physics bindings to the new system from the manually written bindings that we are currently using. Right now the auto-binder pretty much works as far as I can tell, but before it can get merged we are waiting for a newer pre-release of Haxe, because the latest one that the binder depends on has some glaring issues elsewhere.

3

Thanks for your answers… I’m not daunted if Armory can’t do all of what I need, as I’m willing to put dev work (and money) into improving it. I’m just trying to understand whether its reasonable / viable to get where I’m going…

For example…I would really prefer to keep my modules (like cloth) as fully native code. The cloth sim is threaded, using OpenMP… and jumping through hoops to get it to do native shared memory threading through Emscripten pthreads sounds like a world of pain. I don’t care about losing web-player support. Is this viable? Or is it just too difficult given the way Armory3d is built around Haxe?

On a similar note… is Armory3d entirely single-threaded because of Krom/Haxe? Does it have notable garbage collection pauses?

I was naively assuming that Armory3d was a more typical C/C++ core engine, with Haxe for scripting… but looking through the code, it appears to all be built in Haxe. That certainly has it’s advantages, but for some “heavy lifting” stuff, fast native threading is really important.

4

Iron is core engine written in Haxe Kha, see Armory3D’s architecture. Yes, it have multi-threading and is really fast (Legend say that Krom target is sometime faster than C/C++ target)

2 Likes
5

Definitely checkout the Armory3D architecture link that @BlackGoku36 posted to understand a bit more about the way everything is put together.

As far as your cloth simulation library and not compiling to JavaScript, that is still a possibility, you would just have to compile your own version of Krom, the technology that runs Armory games during development, and provide JavaScript bindings to your library for it. That would be going out of your way a bit more to make that happen, but it isn’t a huge deal.

You can still use Krom during development and export the game to C for the production build. If you want to avoid Krom completely you are going to have to wait for much longer build times because you have to build the generated C code every time you change something.

6

I just checked out ArmorPaint, and it uses a fork of Krom to provide native file dialogs and Direct3D11 support, so that is a good example of Extending Armory to use native libraries for a Krom application.

Also, a while back @RobDangerous, the author of Kha, mentioned that he could probably add mechanisms to Krom for loading native libraries, which would mean that you wouldn’t have to fork Krom, to add those features.

For your cloth simulation library, it might be slowed down by the calls from JavaScript to C++, but if it was a problem, you could still do away with JavaScript for your production build with the C export of the game and only use Krom during development.

7

Sorry, but multithreading in Krom is very restricted - it’s JavaScript after all.

8

I read the Armory3d architecture page, and dug through the code, and I’m amazed by this Haxe based technology stack.

However, there are some sticky and intertwined issues that affect the performance of data exchange across the boundary between C-code (native or WASM) and Haxe code.

Similar issues occur in thunking to other managed languages, but in other engines (UE4, Blender, Source, etc) the core of the engine is written in the low-level language. Having the core of Armory3d / Iron / Kha written in the high-level language complicates efficient integration of low-level modules.

This is especially important for a cloth simulator, because it needs to see all meshes and acceleration structures (aka BVH/kdtree) because it needs to do collision detection for all collision targets during the cloth solve. This is also stuff that effects haxebullet physics.

After my digging, I’ve decided I don’t think it’s practical to use a C cloth simulator with Armory3d, because of the data-marshalling issues… and I don’t think it’s practical to expect good multi-threading out of Armory3d, because the Krom target doesn’t support it, and the CPP/native target has a very very poor garbage collector.

Probably it would be more practical to convert my cloth simulator to Haxe+GPU compute. This might be a decent route for me, because the way Armory3d works with Blender scene files is extremely advantageous for my application - because it will be a very long time before the UE4 editor is as good as Blender.

Here is a summary of the detailed facts that lead me to this conclusion. I welcome any corrections or thoughts…

(1) Armor3d / Iron / Kha allocates the scene data structures in Haxe, which turns into Javascript GC heap objects in the Krom target. Mesh data is allocated by Kha as a Javascript “Int16Array”…

(2) Inside Krom/V8, WASM code (aka Emscripten compiled C code), can only see data allocated inside the WASM heap. This means to see any of the above scene or javascript mesh arrays, they need to be copied into the WASM heap, causing performance cost and memory bloat. This makes a C/Emscripten compiled cloth simulator pretty impractical.

For details, read Most performant way to pass data JS/WASM context · Issue #1231 · WebAssembly/design

(3) Integrating a C cloth simulator into Krom as a native C library could avoid copying mesh buffers, but there is still a big impedance mismatch, because it will have to use V8 APIs to traverse the scene data that lives in Javascript GC objects.

(4) CPU side collision detection also requires seeing animated mesh results, and mesh acceleration structures. Something needs to get the skinned mesh results into CPU memory (whether computed on the CPU or GPU), and update CPU memory acceleration structures (like a BVH). If this data lives in Javascript/GC memory, then it’ll have the same issues as described above to be seen by C/native or C/Emscripten.

(5) As a result…it seems most viable with Armory3d to write a cloth simulator in Haxe, so it has most efficient access to the data. However, unfortunately, Armory3d on Krom is single threaded, because Haxe/Krom compiles to Javascript with GC heap objects, and the Javascript V8 GC heap is single threaded. This would mean the cloth simulator would also be single-threaded, which is pretty painful. (Likewise, single-threaded CPU mesh animation skinning would be pretty painful)

(6) For Haxe code to multi-thread with shared heap objects on Krom, it would need to stop using the Javascript GC heap. This would basically mean running the Haxe CPP target on WASM, and relying on hxcpp’s inferor garbage collector, instead of the excellent (but single threaded) V8 garbage collector. My understanding is that this is not currently possible, as hxcpp currently does not generate Emscripten compatible code.

(7) The Haxe cpp target running native can support threads, and can share data buffers by reference with C, but relies on it’s very poor internal garbage collector. It would still have the same Haxe data thunking issues as C/native/Krom.

9

It looks like you did some good research and it sounds like you’ve got a pretty good understanding. These are my thoughts so far ( the numbers not related to the numbers on your post ):

  1. Armory actually uses the HashLink to C ( HL/C ) target of Armory, not the hxcpp target. They’ve got a different garbage collector, but I don’t exactly how they compare. You would probably have similar problems so that doesn’t necessarily fix anything.

Note: I’ve also heard that the comment on the HashLink site saying that it outperforms V8 is inacurate.

  1. It might not matter that the cloth is slow on the Krom build if you can export your project to C and everything will be native code. Still I don’t know that that would fix everything for the C build because there still has to be calls from the garbage collected HashLink C code and your non-garbage collected C cloth library.

  2. I think your probably right about Haxe and GPU compute being the best option. That is how ArmorPaint achieves its extremely high performance painting. It would probably perform faster than CPU compute with the native implementation, anyway, as long as it wasn’t too difficult to port.

Edit: If the cloth library needs to integrate with Bullet physics, then you might still have similar issues with memory transfer even if you wrote the cloth sim itself in GLSL because of Bullet being run as ASM.js. So far Bullet’s cloth simulation in Armory has ran OK with small tests, but I don’t think that anybody has stress tested it or anything.

10

The hxcpp garbage collector is not a very poor garbage collector, particularly when you enable the generational gc mode and data exchange is much more efficient than it normally is in those situations - pointers to Haxe-internal things can be directly pinned and accessed and the other way round it’s even easier (see for example Kha’s array buffer classes). The hl/c garbage collector currently is indeed pretty slow though, also C/Haxe data exchange is still limited compared to hxcpp.

11

@RobDangerous - it’s nice to know there is an experimental generational mode. I still don’t expect it to compare to V8’s collector, which is arguably one of the best GCs on the planet, next to Hotspot and Azul-Zing. But it’s nice to know they are improving it.

There shouldn’t be any need to pin things with hxcpp, as it uses a non-copying conservative collector. Pinning would only be relevant for interfacing with a precise copying/compacting collector like V8/Krom.

How does accessing Haxe structs / objects from C++ work? Are Haxe fields translated into C++ fields by the compiler? If so, this could be a big advantage of using a non-compacting conservative collector, as C++ code could read the Haxe cpp header files, and walk Haxe data-structures “natively”. Though I doubt this is a supported pattern, because it would only work for hxcpp, not HL/C or Krom.

12

Does the current Armory3d bullet support Mesh Colliders? Or only collision shapes? If it does, how the Mesh data get transferred from Armory3d/Krom into ASM.js? I couldn’t find this in the code. The haxebullet mesh stuff seems to just be empty stubs.

If these mesh-data copying issues become a problem, another possibility to consider is using a Haxe physics library (maybe this is what OimoPhysics is to become?).

There is also a C# to Haxe transpiler, and while I doubt the amazing bepuphysics2 would transpile very well, there might be useful pieces of bepuphysics1 that could either be ported or transpiled over… like the excellent BEPUik.

13

It looks like the code for creating the Bullet mesh shape is here.

14

Sure, the V8 GC is better but I’d argue there’s some area between very poor and best of the world. The hxcpp GC optionally supports compaction by the way. And yes, Haxe fields are translated to C++ fields, you can access fields natively (and Kha uses that) and even just stuff some C struct into Haxe classes then later read it back in C, again directly (Kha uses that intensively).

15

To add one more tidbit to that - even with compaction turned on one doesn’t always have to pin objects because when the garbage collector is running can be controlled completely from the C++ side of things.

16

@zicklag - thanks… That code is not just copying the whole mesh (duplicating all the data), but it’s doing it very slowly, by using addTriangle() on every triangle. That seems viable for small to moderate static collision meshes (like a non-streaming level mesh), but not for handling collisions with animated / skinned meshes which change every frame. This is fine for games with small to medium size level meshes and which use armature translated collision shapes for avatars, but for my application I need precise mesh collisions with avatars. That said, I don’t really need bullet, so for my app there might be a different way to skin this cat.

@RobDangerous - That’s interesting, the Haxe Manual just mentions conservative-stop-the-world, but now that you pointed in the right direction I do see some patches and command line flags referencing generational and compaction (which must mean some type of non-conservative precise collector). The only documentation i could find is the Threads and Stacks section of the Haxe Manual, I would update it with some GC details, but it’s written in LaTex so it lacks a wiki-edit/preview capability.

17

hxcpp is lacking in the documentation department, particularly in the details. I only know those things because I regularly move around in the sources and sometimes talk to Hugh. But there’s lots of good stuff in there for performance and interop, even something like http://api.kha.tech/kha/simd/Float32x4.html is possible without much effort - pretty much unthinkable in something like JS, Java or C#.

18

Unfortunately, even if hxcpp could work, @lubos dropped the hxcpp target in favor of HL/C for Armory. I don’t know what the reason was. It may have had something to do with the C++ binding generator for HashLink.

19

…this is a side-point, but you’re mis-informed about that “unthinkability”…

C# has supported SIMD for quite a while. It was available in Mono.simd.dll back in 2013, and was standardized into System.Numerics.Vectors in the windows 64bit RyuJIT backend since ~2014 and in mono since 2016 with . Bepuphysics2 has insanely good peroformance (video) by making heavy use of the new S.N.V SIMD. In Unity, it looks like it’s possible to use the old Mono.simd.dll on win/mac, but I don’t think SIMD works in their new IP2CPP backend (required for iOS and web).

Java also has some support for SIMD, as it’s SuperWord optimization can automatically inject SIMD operations in some cases, but it’s not possible in Java to explicitly use them.

Chrome/V8 had an experimental SIMD.js which was removed in favor of shifting to [WebAssembly SIMD], (6020 - v8 - V8 JavaScript Engine - Monorail), which is bleeding edge, but apparently working. While some users preferred the former because it was usable from “normal” javascript, in practice it was very hard to structure normal javascript to benefit from it, which is why they shifted to a WebAssembly centric implementation.

While SIMD is often important, equally important is using cache-efficient value-type struct arrays, instead of “randomly heap allocated” OO object arrays. You can see an overview of the issue in the “Value Types” section of this BepuPhysicsV2 article. To quote the article…

C# expresses packed struct arrays a bit more naturally than Java or Javascript… Though Java structs/junion, looks half decent, and Javascript has TypedArrays. I don’t know if Haxe automatically uses TypedArrays for struct arrays in the Krom target (i suspect not).

20

That’s not my point. My point is, Haxe/hxcpp does not have simd support at all. And yet I could just easily add it.

1 Like