Engendro 3D™ on Github

Finally, Engendro 3D, the Python based OpenGL game engine arrives to Github. If you want to try the engine so far, to criticize, help or  whatever, visit

https://github.com/jr-garcia/Engendro3D

to clone or fork and try it.

While the published version is still a very early and incomplete alpha, I needed to upload the work I have been doing on this engine and the supporting projects, at least as a backup… and I’m very happy for  doing it, since just yesterday my disk failed, taking with it a lot of work. But now I have an almost updated copy!

I’m not a friend of backups, and this have slowed down the progress on many little things I have started (and never finished) in the past, including the engine, when I sufer a failure, but now I can just clone from the repos 😀

Distance fields for font rendering (Part 2)

To properly test the font rendering from distance fields,  I adapted chunks of code from Here and Here, to make a Python signed distance field creator:

from PIL import Image
import os
from subprocess import call
from time import time
from math import sqrt


def check(limit, ox, oy, px, py, current):
    lp = getpixel(px, py)
    if lp == -1:
        return current
    if lp != limit:
        nw = float(ox - px)
        nh = float(oy - py)
        new = (nw * nw) + (nh * nh)
        if new < current:
            current = new
    return current


def getField(limit, searchLen):
    print('Calculating field {0}...'.format(str(limit + 1)))
    if limit == 1:
        limit = 255
    distances = []
    for w in range(oim.size[0]):
        distances.append([])
        for h in range(oim.size[1]):
            current = limit if limit > 0 else (searchLen * searchLen)
            for x in range(w - searchLen, w + searchLen):
                for y in range(h - searchLen, h + searchLen):
                    current = check(limit, w, h, x, y, current)
            current = int(sqrt(current))
            current *= int(g)
            distances[w].append(current)
    return distances


def getpixel(x, y):
    try:
        p = inpixels[x, y][0]
        return p
    except IndexError:
        return -1


inPath = os.path.dirname(__file__)
outPath = os.path.join(inPath, 'out.png')
inPath = os.path.join(inPath, 'in.png')
oim = Image.open(inPath).convert('RGBA')
img = Image.new('RGBA', oim.size, (0, 0, 0, 0))

ct = time()

inpixels = oim.load()
outpixels = img.load()

search = 8
last = 0.0
g = 255 / search

field1 = getField(0, search)
ct = time() - ct
print('Took: {0}'.format(str(round(ct, 1))))
ct = time()
field2 = getField(1, search)
ct = time() - ct
print('Took: {0}'.format(str(round(ct, 1))))
ct = time()
print('Mixing fields...')

for cw in range(oim.size[0]):
    for ch in range(oim.size[1]):
        fc1 = 255 - field1[cw][ch]
        fc2 = field2[cw][ch]
        fc = int((fc1 + fc2)/2)
        outpixels[cw, ch] = (fc, fc, fc, 255)

ct = time() - ct
print('Took: {0}'.format(str(round(ct, 1))))
ct = time()

print('Resizing and saving output image (128 width)...')
rf = float(oim.size[0]) / 128
img = img.resize((128, int(oim.size[1] / rf)), Image.BILINEAR)

img.save(outPath)

ct = time() - ct
print('Took: {0}'.format(str(round(ct, 1))))

if os.name.startswith('darwin'):
    call(('open', outPath))
elif os.name == 'nt':
    os.startfile(outPath)
elif os.name == 'posix':
    call(('xdg-open', outPath))

(Since this is only a test, the input and output file paths must be changed manually as needed).

The resulting image is certainly better suited for the font rendering:

SDF:

Signed Distance Field Test

Result (same shader shown in previous part):

SDF font render

Zoomed in (no weird artifacts like with the faked DF):

SDF font render zoomed

Unfortunately, the creation time for the whole DF is around 2 minutes for small images (256*256), and that is too much for the usage that I’m thinking. Now,  that usage involves the GPU so, could be faster to do this with the graphics card? Isn’t the purpose of the GPU to do per-pixel work anyway?

 

In the next part I will publish a complete tool to calculate the distance fields in the GPU, hopefully, in real time .

Edit:

For Python 3 compatibility, data passed to ‘outpixels’ needs to be send as int. Code updated to reflect it (and improved file paths).

Distance fields for font rendering (Part 1)

Pushing myself forward again, I will publish some entries about implementing font rendering into a texture atlas, encoded with distance fields and with support for Unicode chars (the possibility to reduce the texture size will probably allow large amounts of glyphs in one single texture) to finally show proper labels and text boxes in Engendro3D.


 

By checking info about how to render fonts in OpenGL, I found this question, which lead me to the Valve’s paper (.pdf).

The results, along with the need to have huge amounts of chars pre-rendered to a font atlas for certain languages, got my interest.

Since the Distance fields looks slightly like blurred images, I made a quick png texture in Gimp containing 6 letters (with Eufm10 font) and I used the Gaussian Blur filter (10 radius) to produce this faked distance Field:

wtx_blured

Then, with a very simple GLSL fragment shader, this is the result:

dfblured

No Bilinear interpolation:

gblured_noint

Zoomed into the ‘B’:

zoomblur

No Bilinear interpolation:

zoomblur_noint

Has outline and glow and, while the result is not the best, the logic works.

The shader I wrote can probably be improved a lot, but I will use the same for all the tests I’ll do:

#version 110

uniform float time;
uniform sampler2D tex;

void main()
{
float color = texture2D(tex,gl_TexCoord[0].st).r;
	if (color <= 0.5)
		gl_FragColor = vec4(0,0,1,1);
	else
		if (color <= 0.6){
			gl_FragColor = vec4(1,1,1,1);}
		else
		{
			if (color < 1.0)
				gl_FragColor = max(0.0,1.0 - color) * 
				vec4(.9,.4,.2,1) * 2.0 * max(sin(-time),sin(time));
			else
				gl_FragColor = vec4(0.0);
		}
}

 

In the next part I will show a Distance Field generator in Python and Pillow (PIL).

GUIs quirks and quacks

 

In the search of a ready-to-use GUI system, I came across several options, but in the end I have decided to make my own, simply because none of the options was completely fittable to my needs, specifically for the next reasons:

Maybe one of the most famous options. Looks very good and has official Python bindings.
I never managed to make it work in either Linux nor Windows. In Linux, after four hours of compiling, and a previously failed process due to missing dependencies, I gave up.
In Windows, the compiling took only two hours, only after I fixed some errors thanks to this instructions:

http://knightforged.com/cegui/

After finishing, I realized that the python bindings were not selected, so I started again, just to found severals errors raised for the lack of Boost. Unfortunately, I neither managed to compile the Python bindings for Boost.

 

Is recommended, looks promising and it appears to be usable from Python, but… It needs boost, so no further research on that.

Maybe not very well known like the others, but they are completely Python based, so no need for Boost. I tried to convert both to plain PyOpenGL, since SimplUI uses Pyglet’s event system (very different to mine) and BGEUI is based on Blender Game Engine and Fixed function (wich I won’t use), but after some days I realized that the effort needed for a conversion-integration into my engine would end up taking too much time. Maybe near to the time needed to make my own. So I gave up with pre-made solutions.

 

Result, after three days of work, the first screenshot of the first ‘Panel control’ of my GUI:

gui

Right now, the widgets support opacity and are stretched automatically. In the picture seen over a 3D duck with specular reflection. I have already one directional light. Current cost of the panel: 1ms

It only uses four vertices in a VBO and an index buffer. Then 3 unifforms for each control’s position/size, color and background image. This setup establish the possibility to render the GUI controls either as normal 2D ‘widgets’ or as 3D objects attached to the scene models and, with a little more work, controls could be rendered using instancing where available (all the GUI in one draw call).

and the gestation continues.

Python SDL2 TTF test

For cross platform windowing, input, font rendering and probably the entire 2d gui render of the engine, I have decided to use PySDL2, since I had some troubles with SFML (crashes in debug mode in PyCharm).

Next is an almost direct conversion from C to Python of the example found here:

http://www.willusher.io/sdl2%20tutorials/2013/12/18/lesson-6-true-type-fonts-with-sdl_ttf/

The Glametrix font I used comes from here:

http://www.glukfonts.pl/font.php?font=Glametrix

I placed the SDL2 and SDL2_ttf libraries in the script’s folder, inside another folder called ‘libs’ and the font in a folder called ‘font’.

Result:

sdlTTF

Code:

import os
from sys import exit
from ctypes import c_long, pointer

sdlpath = os.path.join(os.path.dirname(__file__), 'libs')
os.environ['PYSDL2_DLL_PATH'] = sdlpath

from sdl2 import *
from sdl2.sdlttf import *

def renderTexture(tex, ren, x, y):
    """
    :type ren: SDL_Renderer
    :type tex: SDL_Texture
    """

    #Setup the destination rectangle to be at the position we want
    dst = SDL_Rect(x, y)
    w = pointer(c_long(0))
    h = pointer(c_long(0))
    #Query the texture to get its width and height to use
    SDL_QueryTexture(tex, None, None, w, h)
    dst.w = w.contents.value
    dst.h = h.contents.value
    SDL_RenderCopy(ren, tex, None, dst)

def renderText(message, fontFile, color, fontSize, renderer):
    """

    :rtype : SDL_Texture
    """
    # Open the font
    SDL_ClearError()
    font = TTF_OpenFont(fontFile, fontSize)
    p = SDL_GetError()
    if font is None or not p == '':
        print("TTF_OpenFont error: " + p)
        return None

    #We need to first render to a surface as that's what TTF_RenderText
    #returns, then load that surface into a texture
    surf = TTF_RenderText_Blended(font, message, color)

    if surf is None:
        TTF_CloseFont(font)
        print("TTF_RenderText")
        return None

    texture = SDL_CreateTextureFromSurface(renderer, surf)
    if texture is None:
        print("CreateTexture")

    #Clean up the surface and font
    SDL_FreeSurface(surf)
    TTF_CloseFont(font)
    return texture

SDL_Init(SDL_INIT_VIDEO)
#Create an application window with the following settings:
window = SDL_CreateWindow(
    "SDL2 TTF test",  # window title
    SDL_WINDOWPOS_CENTERED,  # initial x position
    SDL_WINDOWPOS_CENTERED,  # initial y position
    640,  # width, in pixels
    480,  # height, in pixels
    SDL_WINDOW_RESIZABLE  # flags
)

renderer = SDL_CreateRenderer(window, -1, SDL_RENDERER_ACCELERATED)

tfi = TTF_Init()
if tfi != 0:
    print("TTF_Init")
    exit(1)

#We'll render the string "TTF fonts are cool!" in white
#Color is in RGB format
color = SDL_Color(255, 255, 255)
fontpath = os.path.join(os.path.dirname(__file__), 'font', 'Glametrix.otf')
image = renderText("TTF fonts are cool!", fontpath,
                   color, 64, renderer)

if image is None:
    exit(1)

#Getting the window size.
SCREEN_WIDTH = pointer(c_long(0))
SCREEN_HEIGHT = pointer(c_long(0))
SDL_GetWindowSize(window, SCREEN_WIDTH, SCREEN_HEIGHT)

#Get the texture w/h so we can center it in the screen
iW = pointer(c_long(0))
iH = pointer(c_long(0))
SDL_QueryTexture(image, None, None, iW, iH)
x = SCREEN_WIDTH.contents.value / 2 - iW.contents.value / 2
y = SCREEN_HEIGHT.contents.value / 2 - iH.contents.value / 2

r = 1
event = SDL_Event()
while r:
    if SDL_PollEvent(event):
        if event.type == SDL_QUIT:
            r = 0
        elif event.type == SDL_WINDOWEVENT:
            if event.window.event == SDL_WINDOWEVENT_RESIZED:
                SDL_GetWindowSize(window, SCREEN_WIDTH, SCREEN_HEIGHT)
                x = SCREEN_WIDTH.contents.value / 2 - iW.contents.value / 2
                y = SCREEN_HEIGHT.contents.value / 2 - iH.contents.value / 2
        if r:
            SDL_RenderClear(renderer)
            #We can draw our message as we do any other texture, since it's been
            #rendered to a texture
            renderTexture(image, renderer, x, y)
            SDL_RenderPresent(renderer)

SDL_DestroyTexture(image)
SDL_DestroyRenderer(renderer)
SDL_DestroyWindow(window)
SDL_Quit()

I moved the drawing stuff to the event processing so the text is drawn only if there is a change (window re-paint, resize).

PS: Somebody knows a way to avoid using ctypes pointers in the few PySDL2 functions that need them?

PyOpenGL performance 2 and optimizations

Premature optimization is the root of all evil

Is it?

Note: frame times at the end.

The conversion of all modules from SharpDX to PyOpenGL is finished, so I ran the test program again. Result: 430 FPS (previously 649). Time to optimize.

First: check with cprofile module. Most time spent is in the shaders uniforms update, then in the events raising.

  • After reducing some gpu calls, >> +20 FPS
  • Converting many For loops to list comprehensions and map() >> +10FPS

I was worried because the achievements in frames where small, then I remembered the events reporting.

  • After removing all the onRenderEnd calls, where I was doing rotation of the dragon and window title update >> +1000 FPS

The engine ended up running at 1500 FPS (max). Amazing, but unrealistic, since casting events is completely necessary. Maybe a threaded approach will give improvement in this point.

But since frame times is the right way to measure, I converted all numbers, getting:

  • First gain: 0.1ms
  • Second: 0.04ms
  • Last: 1.5ms

Was it right? After trying the same program in my slow laptop, the timings probed to be correct… and meaningless. All of them:
The result in my laptop was of 58FPS, when before the improvements was of 50FPS. Only 2.7ms Total, after all the stuff I did and removed.

Final result: I’m done with optimizations. Maybe, optimizing before having something good is not good. I will move on to the next step, that will not be an editor this time.