PyOpenGL performance 2 and optimizations

Premature optimization is the root of all evil

Is it?

Note: frame times at the end.

The conversion of all modules from SharpDX to PyOpenGL is finished, so I ran the test program again. Result: 430 FPS (previously 649). Time to optimize.

First: check with cprofile module. Most time spent is in the shaders uniforms update, then in the events raising.

  • After reducing some gpu calls, >> +20 FPS
  • Converting many For loops to list comprehensions and map() >> +10FPS

I was worried because the achievements in frames where small, then I remembered the events reporting.

  • After removing all the onRenderEnd calls, where I was doing rotation of the dragon and window title update >> +1000 FPS

The engine ended up running at 1500 FPS (max). Amazing, but unrealistic, since casting events is completely necessary. Maybe a threaded approach will give improvement in this point.

But since frame times is the right way to measure, I converted all numbers, getting:

  • First gain: 0.1ms
  • Second: 0.04ms
  • Last: 1.5ms

Was it right? After trying the same program in my slow laptop, the timings probed to be correct… and meaningless. All of them:
The result in my laptop was of 58FPS, when before the improvements was of 50FPS. Only 2.7ms Total, after all the stuff I did and removed.

Final result: I’m done with optimizations. Maybe, optimizing before having something good is not good. I will move on to the next step, that will not be an editor this time.

PyOpenGL performance 1

The first concern about rewriting the engine in Python + OpenGL was about having to learn one more language that in the beginning I didn’t wanted to learn. The speed of OpenGL was never any worry, mainly because I trusted in the findings of Valve.

The second concern, was Python speed. Being an interpreted language, it seemed that it was going to be inherently slower that C++ or even worst, slower than C#. Working with Basic.net previously, I was accustomed to the idea of having slow code, but SharpDX is being said to be not so distant to raw C++ in speed. And it proved to be truth after my initial tests, so… What to expect from PyOpenGL?

The first test I made was to load the Stanford dragon (after converting my meshes loader to Python, with Pyassimp) and the result was not satisfactory (or so I thought): The FPS is around 40 (up to 60 without recording), wich seemed to be slow, even in my slow laptop. Still, I could not say anything yet, until making a test in ‘the big one’ and having a comparison with SharpDX. After some days, here it is:

Amazing!

PyOpenGL 2/3.1 is faster than SharpDX (DX9), just like the C++ versions. Even better, I have not made any optimization to the Python version. I’m not using PyOpenGL-Accelerate nor Cython!, so it is possible to get even better speed with those tools 😀 .

The only real difference between the two tests is the shader used, but I doubt that the impact from it is that big.

For reference, the next data:

Graphics card: Nvidia GTX 560 se

Processor: Intel Core 2 Duo 2.6GB

OS: Windows 7 32 bits

Memory: 3GB

Direct X 9 without fixed function Pipeline and shader model 2

OpenGL 2 without fixed function Pipeline and GLSL 1.2

Max FPS without recording:

– DirectX: 570 (first)

– OpenGL: 649 (second)

dx9 ogl2

Something interesting is the fact that the OpenGL app will reduce its speed if the DirectX one is running, as seen on the “Both” part of the video. Also, the OGL will slow down with any movement of any window or any interaction with the desktop. The DX window will remain steady. Does that mean that Windows gives preference to DX tasks over OGL or was it my card?

In the PyOpenGL site, they accept the wrapper being slow, but for some deprecated features, that I’m not using anyway, so I’m safe from that. And with this results and the Cython option, I’m really happy, so the conversion continues.

SharpDX Forum closed.

Note: I had this and the Ducky story ready since a few days, but delayed posting them, until now…

(Edited to move the effect’related info to another post)

While it is truth that SharpDX is just a wrapper (a good one) around DirectX and you can find all the info you need about the api without visiting the forum, it was not the reason to have a forum. It was (or at least I think that’s the reason of any forum) to have a COMMUNITY. A place to ask about specific aspects of the wrapper, comment about something or maybe show what you do with the tool. The post by xoofx (linked above) specially shocked me, mainly because almost every word hit me, like a slap in the face. I have been insulted many times in my life (mainly by my mother) and the words in xoofx’s post felt pretty much the same like my mom’s. I mean, he called us wannabes, (maybe without talent?), that are not having success with SharpDX for not working hard, unlike other guys pointed out in the same post. I now I have no talent. I don’t need one more person telling me that.

Yes, I asked something once and delayed some time to go back to the forum, but that’s because SharpDX does well it’s job, so it is not really necessary to visit the forum so often and, before ask, I read all I found in the forum about the subject I was in doubt about (including the post I was refered to).

Yes, there is a lot of info about DirectX in the web, and probably a lot of users don’t look there enough, and neither in the forum before ask, but complaining about having to answer every question is totally wrong for a very simple reason: It is a forum, not a personal contact way to the devs. Forum managers are not forced to answer every single question posted. That’s why there are other registered users. According to the post’s statistics, few users are answering, other that the admins. Well, that’s the problem of the community, not the admins. DX Studio forum is visited very few times by the admins, but still some people, even myself, answer some questions from when to when. It is not much, but at the end everybody is fine with that.

Yes, admins, devs, moderators and everybody has the right to use their time as they want (and using some of it to maintain and enhance SharpDX is a very good way to use it), but I thing that closing the forum for that reason is a very bad decision. Still, SharpDX is great and there is no reason to stop using it, even if the dev is like doing us a favor by letting us use the wrapper, as I sense he feels, by he using this sentence: “we are using SharpDX at my work, mainly these APIs (Direct3D11/DXGI/D3DCompiler), so any bugs in theses API are critical for my work as well”.

Well, thank you, sincerely, because whatever fate lies onto my work, it could not be possible without SharpDX.

(Sorry for the rant, but the only people I talk to is the Internet people (you). Losing a space for that, even if I don’t use it a lot, is not nice).

PS: Happy Birthday to me!

Engendro3D game engine / Motor de juego

-Engendro (spanish):

  1. Fetus.
  2. Deformed creature born without due proportion.
  3. Very ugly person.
  4. Plan, design or intellectual work poorly designed.

e_logo_proto_TM

Today I announce the conception of Engendro3D™ , a .net based game engine with planned support for DX9, DX11 and OpenGL. But why to start a game engine of my own if it is (I’ve been told) an insanely hard (and perhaps impossible) project for a single person?

Well, basically, to learn.  While I was making my second plugin for DX Studio, I realized that I have to know a lot more about how game engines work. And what is the best way for me to learn? Doing things. So I will put together all that is needed to make functional games to at least be able to improve the appearance of my first game (now in paper), which will use DX Studio.

Right now, lets say that the engine is in the differentiation stage, where the various systems are being formed:

  • Rendering manager
    • Starts and updates the selected device (DX9,DX11,OGL). Also should handle occlusion and instancing.
  • Textures manager
    • Everything related to textures, including atlases and such.
  • Effects manager
    • .Fx file loading, compiling, reload if changed, and parsing.
  • Models manager
    • Meshes loader and importer.
  • Sounds manager
    • Loader (and updater?).
  • Physics manager
    • Setting and updating.
  • Scenes manager
    • Loading and writing to disk of complete scenes. Addition and removal of scene objects.

I have received some critics because of the language I chose: Basic .net. But the facts are that using that language has many advantages for me, so I will stick to it. Anyway, is not like I’m trying to compete against Unity, Cryengine or UDK, right?

Currently I’m working on the rendering manager over DX9. Soon, I will come back with updates.

And about the name… Clever, isn’t it? 😀

After the cube, it comes the indexed mesh

Keeping in the learning of DirectX trough SharpDX, it came the time for rendering something more complex. Since SharpDX has no implementation of a method to load meshes, I had to use Assimp.net.

The first obstacle for that, was that Assimp wrapper is for .net 4, and I was using Visual Studio  2008. Anyway, after switching  to Visual “Studio 2010 (entirely against my will, since I don’t like changes),  I tought that the process to render a mesh was going to be excruciatingly difficult, but still, I just pushed myself into the pool. Turns out that, after reading the Getting Started tutorial on Assimp .net site and using some logic, it was really simple to extract vertex info from the Assimp imported scene. Anyway, after placing all vertices on its place, the first render spat out this (with depth output shader):

wrong

Clearly something was wrong, and it was not the shader.

I suspected that the method to draw a bunch of triangles with textures was different to the one used to draw a simple colored cube, so I searched for “DX9 render mesh” and the first page I checked was talking about Indexed primitives. Okay, so let’s get the indexes from the Assimp scene structure and set those in the device. I also changed from DrawPrimitives to DrawIndexedPrimitive. The result, was exactly the same.

What was I doing wrong? The insanely small amount of DX tutorials (or my lack of ability to find them) was driving me to drop the entire thing. And I did, for a day. Yesterday, I checked all the code and I found a line I forgot to change when I was copying the code for the vertex declaration. I was using Vector3d for the vertices, but I was using float4 in the declaration. After changing that, and adding simple camera functionality, the magic:

good

Original from Assimp viewer (inverted because of my camera position):

orig

So that’s it. I’m now ready to proceed to what I was interested since the beginning: Build a framework to try complex post processing shaders and implement stuff like Deferred rendering that is a little harder using C++ than using .net languages. But now I have a big question that I will ask around. Should I keep learning DX9 or skip it and go directly to DX10 and above?