Armor3d capability questions

Armory Engine General
21

So, to sum up the current findings, writing a cloth sim for the GPU in Haxe is probably the best option for performance, but we would have to find another way to manage physics, because the performance is not going to be good still, if we have to transfer the collision meshes back and forth from the WASM Bullet build to the Haxe -> JavaScript cloth simulator when deploying to Krom. Additionally, hxcpp has some potential for making the data communication between C and Haxe efficient enough, possibly, but Armory isn’t even using the hxcpp target anymore, so that kind of defeats that.

This has opened up a concern for me, in general, because my team hope to one day make potentially large scale games with an Open Source engine, and Armory seems to be the best available option for us. Having to rely on native libraries for certain functionality and yet not being able to communicate efficiently with those libraries could be a serious problem.

I’ve been trying to figure out if there is any solution for Armory to handle this any better, but it seems like the only option is to have Armory core written in a language without a garbage collector. I know it would be a huge change, and I’m not saying that it is necessarily the right thing to do, but I have been trying to figure out whether or not it makes sense to rewrite Iron and Armory in Rust.

We could still utilize Kore just like we are now, and we could even still support Haxe traits by embedding JavaScript, similar to Krom except that the whole of Armory would be native code.

The thing I’m afraid of is that I have seen plenty of small examples with Armory, and examples that show off its amazing graphics capabilities, but none of the examples I have seen are ones that would really exercise the CPU or demonstrate being able to handle large scenes with lots of objects and lots of interaction between the objects in the scene. Could Armory effectively support a large game that could compete with Unity or Unreal while running on HashLink and/or JavaScript?

Rust has the efficiency of C and C++ while eliminating most of the memory errors and security vulnerabilities related to memory errors. It really makes efficient and concurrent programming approachable to a much larger audience and increases productivity. Additionally, Rust compiles to WASM, so we might not have to give up the web target, which too many people are using at this point to justify getting rid of it for good.

I’m still mulling over this and I want to get your guys’s input on it if you have any. I could be missing something major that makes this a completely bad idea, but I want to figure out what will be best for Armory in the long term. @lubos it would be good to get your input on this too if you can find the time.

22

@zicklag - I recommend you take a careful look at your goals and prioritize. If your goal is to work on a game engine, Armory3d needs it. If your goal is to build games, it seems much better to use a game engine that can already build successful games, like UE4 or Unity. If you insist on using an open-source engine, there are others further along, like Xenko. It’s written in C# and very similar to Unity. Though it is still clunky and unfinished compared to Unity or UE4.

The exact same thing is true of my project… My goal is to get the project working, so I feel wrong even looking at unfinished open-source engines. I’m here because for my project, the ability for the community to author and share editable game assets as .blend files is such a potential game changer that I’d be willing to experience some pain if I could make it work.

As for your other points…

I don’t think a game engine has to be written in a low-level language without a garbage-collector. Every game engine is different, and has different goals and tradeoffs. And I must say, the amount of functionality present in even today’s unfinished Armory3d in a small code-size is pretty astounding.

Armory3d is using Haxe and Emscripten to have an almost surreal ability to deploy to any target (including HTML5/webgl)… and it has a deep connection to Blender, to get rid of the hell that is asset-conversion and working with game-asset-editors that are feeble shadows of Blender. I conceptually like this direction.

I think Armory3d, even with Emscripten bullet copying mesh data, even with 100% single-threading, even with only the Krom target… can be a far better BGE than the BGE ever was. It’s an ultra-rapid prototyping system for relatively small games that you want to write entirely in Nodes and Haxe, and deploy everywhere trivially. This is a pretty good niche to be in.

However, when you go beyond that niche, what you get is a confusing paradox of choice, and Armory3d pulled in different directions. Every Haxe target has different complexities and limitations. Krom doesn’t multi-thread Haxe code. Emscripten causes extra copying/marshalling. They all use different GCs, and have different ways of integrating C/C++ code.

One problem with this is understanding it. I have enough experience in low level runtimes to come up to speed on these details quickly, but for most people, this is going to be baffling.

Another problem is integrating foreign code. If you want to integrate non-Haxe modules, you have to pick a method, which effectively means picking a target… so some of the target neutrality promise goes out the window.

Another issue is that optimizing Armory3d for new scenarios is target dependent!

For example, one person is posting on the forums about troubles using a really large terrain mesh. You can cut it up in pieces and use LOD, or you can make it a ROAM heightmap, or whatever… but if the data all lives in Haxe land, and you want to use Emscripten bullet to make a character walk on the ground, there is going to be copying and duplication. What do you do about this?

It feels like this decision depends on the target you care most about. If you want to keep target flexibilty, and efficiently share Haxe mesh data and acceleration structures, maybe you switch to a Haxe native physics implementation like OimoPhysics. Or, if you want to heavily support Emscripten bullet, maybe you want to design a mesh layout and allocation scheme that locates these assets in WASM memory always. Or maybe if you want to keep things simple, maybe you just accept Armory3d isn’t going to be good for very large mesh colliders and send the guy with the huge terrain mesh to a different game engine.

My point is that I don’t think Armory3d can follow all these paths simultaneously, it needs to prioritize where it’s focus lies.

3 Likes
23

@david_j Thank you for the advice. Me and my team appreciate it. We are actually using Armory for a reason similar to yours. We think that Armory has the potential to create essentially the most amazing modding workflow in the history of game engines ( as far as we know ). Having Blender as the interface instantly blows all kinds of other engine interfaces out of the water on asset creation. Sculpting, modeling, mapping, animation, etc., all in the same interface with the ability to export to a game/mod with the push of a button!

Anyway it is going to be hard to give up and go to something else, especially when we are really hoping to use an Open Source engine, we need it to go to consoles, we don’t want to write C for Unreal, and Unity seems to have a bad reputation. There’s just a lot going into our desire to use Armory, especially when being Open Source means that we can contribute and influence its direction towards what we value in the engine, like modding. It may not end up being what we go with in the end, but it will take a lot to make it not worth pursuing Armory.

On to the more technical side, my argument in Rust’s favor for Armory, is that I don’t think you would end up having to sacrifice any of the portability of Armory if you went with Rust. While Haxe has many targets, Armory actually only uses two of them: HL/C and JavaScript. This is very similar to Rust’s targets of native code and WASM. All of Armory’s cross platform ability actually comes from Kore, the C++ library powering Kha. There is also Kha’s abstraction on top of WebGL allowing Amory to go to the browser. We might have to rewrite the browser backend in Rust but by combining Rust’s WASM target with Emscripten for compiling any required C++ libraries such as Bullet, I think it would still be feasible to pull off the web target with Rust.

Other than the web target, all of Armory’s ability to target Windows, Linux, Mac, and game consoles is provided by Kore. By wrapping Kore with Rust, we wouldn’t sacrifice any of the abilities that Armory gets from Kore today. A lot of Armory could actually stay conceptually similar, with the major difference being the language it is written in. The Python portion of Armory at least could stay unchanged.

Other than the rewrite we would also have to change the scripting setup. By embedding Chakra/V8 like Krom does, though, we could support scripting through JavaScript, TypeScript, and Haxe. We would have to create bindings to Armory for JavaScript, but maybe Rust macros could help make that a bit easier to do. I would want to look for some way to automate that as much as possible.

I think that, if it weren’t for the amount work necessary to actually migrate the current codebase, having Armory core written in Rust would be a major benefit to the engine. It would mean that native libraries could be integrated with absolutely no lack of efficiency. The engine itself would become more lightweight and could be more optimized. There wouldn’t be this question of whether or not there is a practical limit to what kind of game Armory could handle efficiently because of the Garbage Collector.

I agree that this does really depend on what direction Armory wants to take. If the goal is not to scale to a game of any size or complexity, then Armory is fine how it is and it wouldn’t make any sense to put so much work into changing it, but if it is looking to compete with the likes of Unreal or Unity, then I think it is worth seriously considering whether or not it can compete while its core is written in Haxe.

This is really depends on what @lubos is looking to accomplish and what his goals are for Armory.

24

No idea whether it would make sense for Armory to move to Rust. Nonetheless here’s two small technical tidbits about your post: There’s no need to rewrite the browser backend, Kore already supports wasm. There’s also no need to wrap Kore with Rust for any of you guys, because that will be part of Kore in just a few weeks.

2 Likes
25

Well that’s awesome. :smiley:

I was literally just right now playing with Rust’s bindgen library to see if it could generate the bindings for Kore. I just finished the bzip2 binding example.

1 Like
26

Some other threading thoughts:

A route to get full Armory3d Haxe multi-threading option is to target HL/C threading. It looks like Hashlink threading is disabled by default, and has some issues, but it exists and it’s getting closer. This should even enable WebGL threading through HL/C to WASM, which is not officially working but theoretically reasonable.

This route would build on a high level concurrency abstraction, like haxe.concurrent.executor that hides the details of whether workers are executed on threads or single-threaded async tasks, so it could still run on JS/V8/Krom.

It also looks like Kha.core.graphics5 (G5onG4 backend) might need some additional thread-awareness, since multiple threads can submit Vulkan command lists simultaneously, but G5onG4 doesn’t guard for this. Though maybe this is handled in some way I didn’t see.

It also might be enough to multi-thread the heavy-lifting pieces, like using multi-threaded Bullet3, while leaving Armory3d Haxe single threaded in Krom. Though there is still the issue of allowing Bullet BVH mesh-colliders to share/see triangle data without copying, at least for the large static environment mesh. Probably the simplest approach is to change Armory3d to use an external allocator for these big chunks, so they can live in the C-heap (and WASM memory) always.

27

Kha already has a threading abstraction that works via hxcpp and javascript - but webworkers are too restrictive to really make that feasible for rendering. And G5onG4 is just the backwards compatibility implementation, that of course doesn’t help with making things run fast either. Things will change when SharedArrayBuffer is finally back but we’re not there yet.

28

I’m doing some experimenting with Rust and posting what I find in another topic:

1 Like
29

It’s strange that you obtain no answer and participation from @lubos on this topic … not a good signal for the community I think.

30

I agree. :confused: I don’t know if we could create a GitHub issue to try and get his attention or something, but I did already try emailing him directly over a couple weeks ago and I didn’t get a response. That’s one of the reasons I started working on my own project.

31

Lubos is one guy and his strong point is clearly development. And he’s so strong in that, I suspect he will manage to get better performance out of his Haxe engine than your whole team will get out of a Rust engine (just a suspicion though, feel free to prove me wrong). Spending his time with discussions doesn’t seem to be his favorite hobby but even though, he is doing way more than anyone can ask for considering Armory’s meager budget so those last few comments, I think they are not fair and quite mean, all things considered.

4 Likes
32

I agree with you, i think he’s focusing improving Armorpaint and Armory. Actually he seems to be focusing more on armorpaint but armory receive armorpaint improvements. Don’t forget that only armorpaint provide incomes to lubos to develop everything. I think armory needs more exhibit and demos to show capabalities and bring more people to contribute. Personnally i follow armory’s developement since 2 years and try to get a level game demo to work. I’m a graphist so i can’t really contribute to code, that’s why i’m learnig coding begining with lua before next step with C# and then haxe to contribute myself later.

Cheers.

3 Likes
33

Well I am thinking the same as well; I followed Armory development closely too until now where I finally get my hands into doing something with it. I’m more of a coder, altough I still lack some of the experience needed about the inner workings of engines to just jump right in and start tinkering with it and contributing to push Armory forward, but I plan to get there.

In my opinion I think the biggest hindrance that prevents Armory garnering a lot of attention is the 0.X, and the problems that comes with that. Like broken or just unusable core features that may be a show-stopper for somebody. To me, at least, in Armory features should be starting to get polished first, then adding features that may be a must have in a lot of projects, (like 3D Audio support), and just then, finally the flashy ones that attract people’s eyes.
Because if you don’t have strong foundations people can’t make serious projects to actually show Armory capabilities, even if they may be limited for now.

2 Likes
34

Don’t take it the wrong way. I have the utmost respect for Lubos and the software he writes. The concerns demonstrated are purely based on the facts of the situation, not out of disrespect or spite for Lubos or his project.

I am trying to build a business on top of a game engine, and if it were based purely on what makes business sense, it would not be wise to invest in an engine where we cannot discuss major development direction with the developer. The concerns raised are crucial to my business and without a response I cannot bet my business on the engine. I do not know whether or not Lubos even wants to make an engine that could support the kind of game that I am making. That was my point about not knowing what the vision of Armory is. It is absolutely nothing personal.

I was fully willing to contribute to Armory to get it to where I needed it, but it looked like it was going to take too much work and I could not get a response from Lubos on whether or not he thought that it would make sense to do the work or even whether or not he thought that Armory would work for my use-case with or without the work. Not being able to communicate over a project makes it impossible for me to contribute to it.

And there are things like the modding development that I did that I never got a response on, good or bad. Modding is essential to the games that I am planning on making. I don’t know whether or not it is even something that Lubos will be open to, so I can’t continue to develop that and keep it up-to-date in my own fork forever.

It is amazing what Lubos has done with Armory, and I understand that it is underfunded and that he needs to work on Armorpaint so that he can make an income, but at the same time, I can’t continue to contribute and work on an engine when I don’t know what the plan and direction of the engine is.

Additionally, somebody recently got on the forums with a valid use-case who was fully willing to put development money into Armory to get it to the state that he needed it to be in for his project, but that wasn’t enough to warrant Lubos’s involvement in understanding whether or not it could fulfill his use-case.

Even if there was a comment like, “I’ve been really busy lately and I’m not going to have time for that in the near future”, that would make me feel way better, but total radio silence is unnerving. Even if Lubos came on and stated what his vision for Armory was and it was completely different from what I needed for my project, it would tell the community what we are all working towards with this engine.

It is important for the community to be able to communicate with the project lead if they are to be expected to contribute. We say that Armory needs more contributors, but if the contributors do not know what they are working towards or what the goals of the Armory project are, how can we contribute? We could guess and assume that we have a good enough idea that Armory will go where we want it to, but that is potentially wasting all of the work that we did when we find out that Lubos doesn’t want it to do what we want it to. I can’t continue to invest in a project where I do not understand the goals that it seeks to accomplish.

Again, none of this is to target Lubos personally, but as a contributor, a member of the community, and someone who wants to use Armory for a real life, important project, I can’t continue to bank on a project with no feedback from the maintainer.

4 Likes
35

The point that money is involved is new to me and interesting but of course I do not know any of the details of the amount of money involved or when you tried to contact Lubos or how you did that but I have one piece of advice for your business communication: Write less. Get to the point. Make communication easy and you are much more likely to get answers (I just read your latest reply because it would have been too ironic not to but it was really off-putting for me).

36

Fair enough, I thought Armory was mostly community-driven, given that things get prioritized by the votes in the roadmap, but nothing wrong with wanting to know what the plan of the project leader is.

About the modding thing, I have seen discussions in other forums about removing features or if some big feature should be added or not “because bloat”, so it’s a very touchy subject. I can’t really talk too much because I haven’t seen how your implementation works or read a lot about the subject, but if it’s something relatively small or can be “avoided” (meaning the compiler not including it if not used) then I don’t see the problem with it being a core feature. If it can’t be small and avoided then maybe adding it as a plugin for Armory?

On a side note, personally I don’t mind the extra features even if I don’t use them, as long as they work as intended and are not best replaceable by some free or cheap plugin like in some other big engine, and if they can be dismissed when compiling if they are not used, that is pretty cool too.

1 Like
37

I love Armory, so very much. It just makes me sad to feel like the lead -and almost only- dev loves it less than me.

To get more funding, Armory just needs more users, contributors and demonstrations of it’s capability. The problem is that contributions largely go ignored, which puts off contributors. Even when work is commited (mainly by @lubos) no one knows about it because there isn’t even a single tweet about the new releases. Nothing to share. Nothing to show anyone else. Nothing to get anyone interested enough to use it or potentially donate.

It’s such a shame as this engine could seriously rock, and be the go to choice of every ex BGE user as well as many small indie teams. There are easy ways this could be harnessed to massively increase funding, but it feels like there’s a huge bottleneck - mainly in @lubos keeping the community out. Please, if you end up reading this Lubos (and a big part of the problem is that I doubt you will), we all respect your crazy coding talent. Just share and listen a minimal amount on social/here, review contributions and help the community do be able to do some of the work.

5 Likes
38

What is your point? Armory was never advertised to be production ready yet. He open sources everything, writes docs about it, accepts pull requests regularly … Seriously how can you be so disrespectful of a one-man band making an open source game engine?

If you work in the game industry at a level where armory sounds interesting to you, you should both donate to Lubos and read the source code and pull request changes… A lot more useful than making too much noise on an unfinished project on “social media”. I feel like the reputation of armory is growing accordingly to its capabilities, exponentially

4 Likes
39

I agree. Maybe there’s some misunderstanding about how major open-source projects work. Lubos is what we would call “the benevolent dictator.” The product owner. It’s his baby, so to speak. Other contributors are accepted into the project under his management and they contribute their changes as so-called “pull requests,” which he may or may not “accept.” Others contribute testing, documentation and various other things. And that’s how it goes. “Social media” really doesn’t play a part in the process at all. Apart from the ticket-system, the usual form of communication is closed IRC chat-rooms, and the communications are purposeful.

Here are three links that I think are worth reading. You will see that the processes are formalized.

http://oss-watch.ac.uk/resources/governancemodels

http://oss-watch.ac.uk/resources/rolesinopensource

If you are technically qualified to be involved, and can adapt yourself to the rigors of contributing to such a project – (it is much harder than it looks) – then you can usually find a place in an open source software project that interests you. But the commitment should not be underestimated. You have to perform at a professional level even though you’re probably not getting paid. And, one thing’s for damn sure: you will never look at open-source software the same way again. You’ll have an appreciation for it that you never had before. And definitely a sense of pride, for you are not just a user. You are a contributor. You are part of the magic.