Overflowing Colors (fixed point)
#1
Hwy guys since i first was working on my GL2 rendering I have encountered a bug where the polygon faces of the robots would flash black depending on the view angle.

Forever Ive been trying to figure it out and where I made a typo. But i couldnt see anything and what was more odd the 1.X paths worked.

Today I found the issue: file ogl.cpp in funciton g3_draw_tmap. The color is converted to fixed point like:

Code:
f2glf(light_rgb[c].r)

f2glf is taking the fixed point and dividing by 65536.0, this should normalize to something between 0-1.0. But if the value is greater than 65536, then there will be values > 1.0. In my code I convert once more to turn the float into a ubyte. I multiply by 255.0f to get something between 0-255.
So if I get a value greater than 255 it will wrap around to something low. The explains the dark colors.

Putting a limit check on the float calculation removed the problem.

So i dont know if the answer to this is to clamp it on the ogl side before passing the to gpu, or is there a problem farther down when this light values are calculated and they never should have been overflowed.

Reply
#2
This may be working but it really is undefined behavior and the drivers must be suppressing it.

If I need to dig myself deeper I can, but it would be nice to get some second opinions.
Reply
#3
I put some restrictions for dyn_light in render.cpp already. If that doesn't work right, it could be addressed there. If it comes from somewhere else, can you point me to where this overflow occurs?
The greatest pleasure in life is to do what people say you cannot do.
Uhm... Honey, there's a head in the toilet!
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)