Benchmark 1 - Spaceship Interior

Showcase
1

(Updated below).

This is a ‘soft-release’ for the Spaceship Interior demo. Tons of work still needed here, but did not want to post-pone it further as it was already promised to be out. :slight_smile:

Goals of this project:

  • Benchmark to measure and improve renderer performance on real data
  • Testbed for porting to new platforms
  • Improving graphics quality
  • Standalone demo / distribution test

The scene is relatively heavy (250k tris) with deferred renderer, tons of processing and no prebaking - a decent GPU is needed here. I tested on GTX 1060 which is of course unstoppable.

The demo will still run on an integrated card - tested on Intel HD5100 at 20-22fps. This brings in one of the goals why this benchmark was created. In upcoming Armory builds, the target will be to raise this to 30fps without dropping quality. Should be fairly big challenge yet still doable.

As usual the first builds are likely prone to crashes / glitches. Right now there are no quality settings - till that is in the demo runs windowed at 720p with the same settings like in the video apart from disabled soft shadows to save some perf. Once the settings will be implemented all effects will be configurable, so more powerful cards can tune it up further. Currently the sound is also not it, apart from ambient background.

Source scene will be out with the next Armory build, since it depends on some of the new features. Overall the scene could be greatly improved, modeling is not that good, flashlight is just flying around, rigid bodies/physics setup is inaccurate… Will eventually fix that (pull-requests with scene edits/improvements also highly welcome if anyone gets to that. :slight_smile:). The look should also improve further with the voxel based GI.

Binaries can be downloaded at:

Once downloaded, simply extract and execute ‘_run.exe’ file (alternatively ‘_run’ on Linux and Armorbench.app on MacOS). Mouse lock is not enabled yet - to rotate camera, drag mouse around with left button pressed (same like viewport camera in Armory).

If you get to test it, please let me know how it goes! Will fix any troubles that pop up. In the future the plan is to create several of those - each with specific setting. The next one should be some kind of open world level to start working on the scene streaming.

PS: The gate password is 340.

1 Like
2

Quick Test:

I do not see Armorbench.app but only Armorbench folder and Krom.app on OS X®.
Only If i run “Krom.app/Contents/MacOS/Krom” from terminal it works.
If i run Krom.app i receive the message “file damaged”.

OS X® 10.11.6 Apple® iMac 21.5 Intel® Core 2 duo Ram 8 GB AMD® 4670 256 MB 4 FPS
Note: crashed sometimes during loading and play.

OS X® 10.11.6 Apple® iMac 27 Retina Intel® i7 Ram 16 GB AMD® Radeon R9 M295X 4 GB 62 FPS

OS X® 10.10.5 Apple® MacBook Pro Retina Intel® i7 Ram 16 GB Nvidia® GT 750M 2 GB 41 FPS

Windows® 8.1 Apple® MacBook Pro Retina Intel® i7 Ram 16 GB Nvidia® GT 750M 2 GB 43 FPS
Note: Crashed during play.

I figured out to check FPS during play watching the gate screen :slight_smile:

Overall great work!

Console report:

iMac-di-iMac:~ imac27$ /Users/imac27/Downloads/armorbench\ 2/Krom.app/Contents/MacOS/Krom
deviceBufferSize = 4096

mSampleRate = 44100

mFormatFlags = 00000009

mBytesPerPacket = 8

mFramesPerPacket = 1

mChannelsPerFrame = 2

mBytesPerFrame = 8

mBitsPerChannel = 32

Load Sound ambient.wav
Uniform texStep not found.
1 Like
3

I haven’t bought armory yet. Hope to buy the next build when it comes out.

Anyways I tested this in my laptop. It works pretty smooth. No crashes.

OS: Windows 10
Processor: Intel i5
RAM: 4GB
GPU: Nvidia 820M 2GB (It also has an Integrated Intel GPU)

Frame rate:
47ms(21FPS) when run with Integrated Intel GPU
60MS(16FPS) when run with Nvidia GPU

I also opened up the task manager to see how much of the processor and memory its using. Here are my findings.

CPU usage: 4% average
Memory usage: 253 MB average

When it comes to audio, its ok for the most part. But occasionally I get some sort of a “tick” sound.
And also the lighting could be improved a little bit.

Otherwise this is a great start.

1 Like
4

In my linux system I’m getting 62 FPS
Linux System: Solus 2017.01.01.0 64-bit
Linux Kernel: 4.9.7
Graphic Card: GeForce GTX 1060 6GB/PCIe/SSE2

Under macOS I’m executing in the same way @AndreaMonzini explained and getting 20-22 FPS
Mac OSX 10.12.3
Intel HD Graphics 6000

2 Likes
5

windows 10
cpu :6350
gpu:1060
ram:12gb
I’m getting 62 FPS but it crashed after 3 minutes

1 Like
6

Linux mint 18.1, Nvidia Titan X and I’m also getting 62fps.

2 Likes
7

Titan X! Wonder what it takes to make it drop under 60fps. :slight_smile:

Thanks a lot for testing, this is super helpful - surprised it ‘mostly’ worked on the first try. Forgot to rename the Krom.app on MacOS, will fix. Even though Armory can do fully native builds, I went with Krom to distribute this - it is built on single machine and packaged for Windows/Linux/MacOS all at once.

The crashes seem to be due to audio streaming in Krom, will get that sorted. Getting Build 8 wrapped up now, then fixing the sound and starting the profilers to dig more perf! Will probably integrate renderdoc along it.

1 Like
8

Windows 7, GTX 660, i5 3570: 62 fps
Seems were all stuck on it!

1 Like
9

Macbook Air 2013
OSX 10.11.6
cpu: 1.7Ghz i7
integrated gfx: Intel HD 5000
ram: 8GB RAM
FPS: 20-22

Alright this is interesting, initially I experienced graphical errors, like shown in the image below, no labels and a lot of black artifacts, this happened twice. I tried running it a third time and now it looks normal. I also experienced a crash on launch at one point.
http://pastebin.com/cJZnYKw6

Oh and yes the package is corrupted if extracted with the standard unpacking tool, use KEKA instead.

1 Like
10

Just to note, it’s (currently) locked to 62fps max. :slight_smile:

2 Likes
11

Unfortunately for me it’s been blurry. Is this expected, or is it rendering wrong?
I have a Surface Pro 4, which has a pretty high resolution.

12

Hm, maybe the depth of field is acting weird, or Krom/Windows is somehow upscaling the output (the window should be quite small - 1280:720 vs 2736x1824 total on SP4). Getting back to this as soon as the new builds are out.

13

Updated binaries with disabled sound, should fix some of the crashes especially on startup. Project sources are also out. More fixes soon.

14

GTX 1060
i5 6500
Windows 10

Runs at 62 FPS too, but is it normal that when you toggel your flahlight often, the room lights go off and after some times you teleport to a 360 degree image of a museum or something like that where your Battery HUD turns off?

15

Testing new builds!

  • Runs on upcoming Armory Build 9
  • Slightly faster load times due to improved material building
  • Enabled light attenuation, creates a better sense of depth but has a bit darker vibe
  • Runs 20-40% faster. Creating the benchmark scene paid off and the first round of speed-ups is there. Each part of the renderer has been shaved down to perform better. I tested on a Mac with Intel HD5100 - went from 20-22fps to 27-30fps. :slight_smile:

@Simonrazer: It’s not normal, but should be fixed now.

1 Like
16

Nice improvements! I ran the newest benchmark just now and got between 28-32 fps. Initially it ran 31-32 but the more I moved around and went back and forth, it started to fluctuate between 28-30.

Macbook Air 2013
OSX 10.11.6
cpu: 1.7Ghz i7
integrated gfx: Intel HD 5000
ram: 8GB RAM

1 Like
17

No problems under Linux: Solus 2017.01.01.0 64-bit NVidia GTX 1060 - Everything looks ok and keeping the 62 fps limit.

Under macOS 10.12 I have a problem running the app. I’m having the famous error: “App is damaged and can’t be opened. You should move it to the Trash."

I know that this was solved on previous system versions changing the security option to allow running software from anywhere. But this options seems to be gone in macOS 10.12. The problem here is that the app is not working just right clicking on it + open (like I do with other unsigned apps). To solve this problem under 10.12, I see some solutions that require the terminal to change permissions to the file or disabling main security options through terminal. This is something I don’t have a problem doing, but I think armory builds should, at least, build to the point that just right clicking + open should work (if not signed, of course).

I will try to investigate what makes this difference; how macOS says the app is damaged instead of just complaining about security things and forcing the right click + open workaround as usual. Maybe some problems with the bundle structure or Info.plist?

Edited: uhm… maybe a signature is needed for the right click + open to work. So the last option is to execute this on the terminal:

sudo spctl --master-disable

You will need to

sudo spctl --master-enable

to revert this action.

This does the same than selecting to run apps downloaded form anywhere.

Edited 2:
With all this I forgot to mention that after this, the first execution crashed, and the next ones worked. With my Intel HD Graphics 6000 we are around 30-32 fps. Trying to replicate the crash to get log message without success now (every time is working now…).

1 Like
18

@artblanc

Having to fiddle with terminal is of course unacceptable. We could try signing the Krom binary in Armory SDK - however I am not sure if you can stuff additional files into the .app folder (that is all Krom needs) once it is signed without breaking anything and what is the legal side of it. Maybe it just won’t do without signing anymore. In that case Armory would generate XCode project for builds to be distributed, user configures required signing and builds (same like using the native/c++ target right now).

There is one report on the first-run crash. What I gathered so far - it always happens only once, it is MacOS only, and it seems to crash because paths are somewhat modified (just a guess) and Krom is unable to load files. Curiously, it starts up the second time.

Happy to see the fps boost!

19

…I’m still getting 62fps…
No issue whatsoever as before.

20

Yes @lubos , you are right. If when signing any hash/checksum with the bundle size is created you wont be able to inject the content after signing. I tried to look if there was a way to partially sign or use other mechanism… and seems something was allowed in the past, but not anymore.

Extract from https://developer.apple.com/library/content/technotes/tn2206/_index.html#//apple_ref/doc/uid/DTS40007919-CH1-TNTAG401

I store data in or otherwise modify my bundle after I sign it.

  • This is no longer allowed. You will need to locate this data elsewhere.
  • It won’t work to write the data before your app runs for the first time. Gatekeeper will reject your app because the signature will be invalid.
  • It also won’t work to write that data after your app first runs. That still breaks the signature. Gatekeeper will check your app again if your app is copied to another computer such that the new copy is quarantined. And on macOS Sierra and later, Gatekeeper will check your app every time it’s launched if it is run from the location where it was downloaded.
  • macOS APIs that rely on a valid identity will fail.
  • In general, you should plan for future stricter runtime checks of code validity.

Maybe doing the same than armory is doing on the other platforms? In linux and windows the content is distributed alongside the run.exe and _run.sh. So, in macOS, the Krom binary could be unique/signed and provided alongside the resources. Armory.app will need to look for the content outside the bundle, alongside the signed Armory.app. Maybe this adds other complications or is also not possible.

And of course, leave the need to construct the signed bundle with resources inside only when generating xcode projects targeting native/c++.