February 13, 2017

Windows Won't Let My Program Crash

It's been known for a while that windows has a bad habit of eating your exceptions if you're inside a WinProc callback function. This behavior can cause all sorts of mayhem, like your program just vanishing into thin air without any error messages due to a stack overflow that terminated the program without actually throwing an exception. What I didn't realize is that it also eats assert(), which makes debugging hell, because the assertion would throw, the entire user callback would immediately terminate without any stack unwinding, and then windows would just... keep going, even though the program is now in a laughably corrupt state, because only half the function executed.

While trying to find a way to fix this, I discovered that there are no less than 4 different ways windows can choose to eat exceptions from your program. I had already told the kernel to stop eating my exceptions using the following code:
HMODULE kernel32 = LoadLibraryA("kernel32.dll");
  assert(kernel32 != 0);
  tGetPolicy pGetPolicy = (tGetPolicy)GetProcAddress(kernel32, "GetProcessUserModeExceptionPolicy");
  tSetPolicy pSetPolicy = (tSetPolicy)GetProcAddress(kernel32, "SetProcessUserModeExceptionPolicy");
  if(pGetPolicy && pSetPolicy && pGetPolicy(&dwFlags))
    pSetPolicy(dwFlags & ~EXCEPTION_SWALLOWING); // Turn off the filter 
However, despite this, COM itself was wrapping an entire try {} catch {} statement around my program, so I had to figure out how to turn that off, too. Apparently some genius at Microsoft decided the default behavior should be to just swallow exceptions whenever they were making COM, and now they can't change this default behavior because it'd break all the applications that now depend on COM eating their exceptions to run properly! So, I turned that off with this code:
CoInitialize(NULL); // do this first
  IGlobalOptions *pGlobalOptions;
  hr = CoCreateInstance(CLSID_GlobalOptions, NULL, CLSCTX_INPROC_SERVER, IID_PPV_ARGS(&pGlobalOptions));
There are two additional functions that could be swallowing exceptions in your program: _CrtSetReportHook2 and SetUnhandledExceptionFilter, but both of these are for SEH or C++ exceptions, and I was throwing an assertion, not an exception. I was actually able to verify, by replacing the assertion #define with my own version, that throwing an actual C++ exception did crash the program... but an assertion didn't. Specifically, an assertion calls abort(), which raises SIGABRT, which crashes any normal program. However, it turns out that Windows was eating the abort signal, along with every other signal I attempted to raise, which is a problem, because half the library is written in C, and C obviously can't raise C++ exceptions. The assertion failure even showed up in the output... but didn't crash the program!
Assertion failed!

Program: ...udio 2015\Projects\feathergui\bin\fgDirect2D_d.dll
File: fgEffectBase.cpp
Line: 20

Expression: sizeof(_constants) == sizeof(float)*(4*4 + 2)
No matter what I do, Windows refuses to let the assertion failure crash the program, or even trigger a breakpoint in the debugger. In fact, calling the __debugbreak() intrinsic, which outputs an int 3 CPU instruction, was completely ignored, as if it simply didn't exist. The only reliable way to actually crash the program without using C++ exceptions was to do something like divide by 0, or attempt to write to a null pointer, which triggers a segfault.

Any good developer should be using assertions to verify their assumptions, so having assertions silently fail and then corrupt the program is even worse than ignoring they exist! Now you could have an assertion in your code that's firing, terminating that callback, leaving your program in a broken state, and then the next message that's processed blows up for strange and bizarre reasons that make no sense because they're impossible.

I have a hard enough time getting my programs to work, I didn't think it'd be this hard to make them crash.

February 12, 2017

Owlboy And The Tragedy of Human Nature

Owlboy is a game developed by D-Pad studios over a protracted 9 year development cycle. Every aspect of the game is a work of art, meticulously pieced together with delicate care. While it has mostly gotten well-deserved positive reviews from critics, some people have voiced disappointment at the story arc and how the game was eventually resolved. Note: I'm about to talk about how the game ends, so an obligatory warning that there are massive spoilers ahead. If you haven't already played it, go buy it, right now.

On the pirate mothership, it is revealed that the titular "owlboy" is not referring to Otus, but is actually referring to the mysterious cloaked figure, who is actually Solus. Otus merely serves as a distraction so that Solus could steal the relics from the pirates. Once the heroes follow Solus up to Mesos and beat him into submission, it is revealed that Solus masterminded the events of the entire game, promising the pirates power in exchange for retrieving the relics, then using Otus as a distraction to steal the relics and use them to power the Anti-Hex to save the world.

Unfortunately, as the heroes immediately point out, this resulted in the destruction of Advent and the deaths of countless innocent people. Had Solus just asked for help, all of this could have been avoided, and the world could have been saved without incident. Solus admits that he felt he had no choice, as he simply had nobody he could trust. Our heroes offer to help finish the ritual, at which point Molstrom shows up and ruins everyone's day. Solus' methods have finally backfired on him, and he is now too injured to complete the ritual—but Otus isn't. In an act of desperation, he imbues Otus with the power of the artifacts, and Otus is able to complete the anti-hex in his stead, obliterating Molstrom in the process and saving the world.

A lot of people take issue with this ending for two reasons: One, it means almost everything you fought for in the game technically meant nothing, because you are actually working against Solus the entire time. Two, the entire thing could have been avoided if Solus had just trusted someone, anyone, instead of engineering a ridiculously convoluted plot to get what he needed through deceit and betrayal. Otus and Solus here represent a hero and an anti-hero. They are both fundamentally good people with flawed goals for opposite reasons. Solus is doing the right thing for the wrong reasons, whereas Otus is doing the wrong thing for the right reasons. Solus is doing the right thing, which is saving the world, but he achieves it by sacrificing thousands of innocent lives and betraying everyone. Otus is unknowingly dooming the world to destruction, but only because he and everyone else is operating on faulty information. He always chooses to do the right thing, and to trust others.

This is important, because the way the game ends is crucial to the narrative of the story and the underlying moral. Solus may have nearly saved the world, despite his questionable methods, but he would have failed at the end. Molstrom would have found him and easily defeated him, taking all the relics and then destroying whatever was left of the world as it rose into space. Only after Solus explained everything to Otus was Otus able to finally do the right thing for the right reasons, thanks to the friends he made on his journey holding Molstrom back. By doing this, Solus allows Otus to atone for failing to secure any of the relics, and Otus absolves Solus of the evil he had committed in the name of saving the world by annihilating Molstrom with the anti-hex.

The whole point of this story is that it doesn't matter how many times you fail, so long as you eventually succeed. Otus may have failed to save the world over and over and over, but at the very end, as the world is coming apart at the seams, he is finally able to succeed, and that's what matters. This moral hit me particularly hard because I instantly recognized what it represented - failing to ship a game over and over and over. Owlboy's story is an allegory for it's own development, and on a broader scale, any large creative project that has missed deadlines and is falling behind. It doesn't matter how many times you've failed, because as long as you keep trying, eventually you'll succeed, and that's what really matters. The game acknowledges that humans are deeply flawed creatures, but if we work together, we can counteract each other's flaws and build something greater than ourselves.

This is why I find it depressing that so many people object to Solus' behavior. Surely, no one would actually do that? However, at the beginning of the game, Solus' defining character moment is being bullied and abused by the other owls. His response to this abuse is to become withdrawn, trusting no one, determined to do whatever is necessary without relying on anyone else. This is exactly what happens to people who have been abused or betrayed and develop trust issues. These people will refuse help that they are offered, pushing other people away, often forcing an intervention to try and break through the emotional wall they have built to try and keep themselves from being hurt again. That's why you have to fight Solus first, because he's spent his whole life building a mental block to keep other people out, and Otus has to break through it to force him to finally accept help.

Far from being unbelievable, Owlboy's plot is entirely too real, and like any good work of art, it forces us to confront truths that make us uncomfortable. Owlboy forces us to confront the tragedy of human nature, and deal with the consequences. At the same time, it shows us that, if we persevere, we can overcome our flaws, and build a better world, together.

January 17, 2017

Our Software Is a Beacon of Hope

As I draw closer to releasing the first early alpha of a library I've been working on for several years, I've noticed that it's release time coincides with some rather depressing politics that I wish I could ignore. Unfortunately, it's difficult to ignore politicians who seek to destroy the lives of your closest friends. A common theme I hear from my friends is a feeling of helplessness, as though we all know terrible things are happening, but no one really knows what to do about it.

But we are not helpless. It is so easy to lose ourselves in pessimism and cynicism, to think that everything will continue to be shit simply because it's been shit for so many years. A common refrain I heard while still working at a large corporation was, we know our code is shit, but it's okay, because everyone else's code is shit. This kind of mentality really bothers me, not because it isn't true, but because it seems to reflect a resigned, cynical view of the world, and never strives to do better. Yes, everything might end up being shit in the end, but if we don't even attempt to build something better, we never will. If we hadn't been trying to write better code we never would have invented structured programming, or object oriented programming, or functional programming. Each innovation builds on the last, and each innovation has it's faults, but each brings us one step closer to not being shit.

What disturbs me is that the software industry's inability to strive for better code now mirrors a feeling of political resignation, a malaise that seems to be settling on our polarized political discourse. If democracy isn't working, what else can we do? If the entire system is corrupt, what hope do we have of fixing it? As wealth inequality rises to ever more absurd levels, our society is tumbling towards a breaking point. Just like a badly maintained software stack, our society is threatening to topple over under economic stresses it was never built to handle. How much longer can we keep it running with duct-tape and terrible hacks? It is our duty to push for real change, to build a better foundation for our future using whatever skills or resources we have.

As I prepare my library for its initial release, I'm not just zipping up a bunch of files to upload to GitHub. I'm lighting a fire, a beacon of hope that I will cast out into the endless night, praying that it might lead us to a brighter future. I am joined by millions of other flickering flames, many of which will fade with time. A few, however, will grow brighter and brighter, until they bring forth a new dawn and shepherd us into a bright new future, even if only for a moment, before sinking below the horizon. Our entire society is built on these brief moments of clarity. Some are longer than others, but all are important. Even when the night seems to stretch on without end, the rays of hope from a thousand stars can still point us in the right direction.

We all wish for a brighter future, a future free from disease and poverty and war and hunger. We seek the future that we envision for ourselves in our stories and movies. Sometimes, it seems like society is going backwards, when everything is going wrong and nothing seems to go right for anyone. But it is in these moments that we must not forget the most important thing: we cannot wait for the future to come to us, we must build it ourselves. 2016 was not a very good year for many people. 2017 shows signs of being even worse, but I don't care.

I'm going to make 2017 be a better year. I'm going to make the future I want to live in. I'm going to build the tools I wish I had. We are not pawns in some cosmic game, we are human beings. We are free agents capable of making our own decisions, and I'm making a decision to make the future happen, whether it wants to or not.

I'm not letting this year take me down without a fight.

December 20, 2016

Everyone Does sRGB Wrong Because Everyone Else Does sRGB Wrong

Did you know that CSS3 does all its linear gradients and color interpolation completely wrong? All color values in CSS3 are in the sRGB color space, because that's the color space that gets displayed on our monitor. However, the problem is that the sRGB color space looks like this:

sRGB gamma curve

Trying to do a linear interpolation along a nonlinear curve doesn't work very well. Instead, you're supposed to linearize your color values, transforming the sRGB curve to the linear RGB curve before doing your operation, and then transforming it back into the sRGB curve. This is gamma correction. Here are comparisons between gradients, transitions, alpha blending, and image resizing done directly in sRGB space (assuming your browser complies with the W3C spec) versus in linear RGB:


At this point you've probably seen a bazillion posts about how you're doing color interpolation wrong, or gradients wrong, or alpha blending wrong, and the reason is because... you're doing it wrong. But one can hardly blame you, because everyone is doing it wrong. CSS does it wrong because SVG does it wrong because Photoshop does it wrong because everyone else does it wrong. It's all wrong.

The amazing thing here is that the W3C is entirely aware of how wrong CSS3 linear gradients are, but did it anyway to be consistent with everything else that does them wrong. It's interesting that while SVG is wrong by default, it does provide a way to fix this, via color-interpolation. Of course, CSS doesn't have this property yet, so literally all gradients and transitions on the web are wrong because there is no other choice. Even if CSS provided a way to fix this, it would still have to default to being wrong.

It seems we have reached a point where, after years of doing sRGB interpolation incorrectly, we continue to do it wrong not because we don't know better, but because everyone else is doing it wrong. So everyone's doing it wrong because everyone else is doing it wrong. A single bad choice done long ago has locked us into compatibility hell. We got it wrong the first time so now we have to keep getting it wrong because everyone expects the wrong result.

It doesn't help that we don't always necessarily want the correct interpolation. The reason direct interpolation in sRGB is wrong is because it changes the perceived luminosity. Notice that when going from red to green, the sRGB gradient gets darker in the middle of the transition, while the gamma-corrected one has constant perceived luminosity. However, an artist may prefer the sRGB curve to the linearized one because it puts more emphasis on red and green. This problem gets worse when artists use toolchains inside sRGB and unknowingly compensate for any gamma errors such that the result is roughly what one would expect. This is a particular issue in games, because games really do need gamma-correct lighting pipelines, but the GUIs were often built using incorrect sRGB interpolation, so doing gamma-correct alpha blending gives you the wrong result because the alpha values were already chosen to compensate for incorrect blending.

In short, this is quite a mess we've gotten ourselves into, but I think the most important thing we can do is give people the option of having a gamma correct pipeline. It is not difficult to selectively blend things with proper gamma correction. We need to have things like SVG's color-interpolation property in CSS, and other software needs to provide optional gamma correct pipelines (I'm looking at you, photoshop).

Maybe, eventually, we can dig ourselves out of this sRGB hell we've gotten ourselves into.

November 14, 2016

The Answer Is Not More Censorship, It's More Information

After being accused of censoring conservative news, Facebook fired all it's human editors, which was shortly followed by the algorithm being inundated with fake news. It now appears to be regretting that choice as it came under fire for potentially contributing to the rise of Trump.

This is consistent with a disturbing trend in liberal media, which is to outright censor any speech that is considered even remotely hurtful. Predictably, the definition of "hurtful" speech has gotten wider and wider to the point of absurdity, because that's what happens when you censor things! What kind perversion of progressivism is this? We're supposed to stand for freedom of information, intellectual free thought, and above all, free speech. There is a difference between free speech and harassment. You can ask people to be respectful when debating you, but you cannot ask them to censor their opinions, even if you find those opinions morally reprehensible. If someone wants to respectfully debate the morality of eating babies, you can't censor them for harassment because their only crime is holding an opinion that you disapprove of.

Fake news is yet another problem that is not going to be solved by more censorship. If a platform censors information that people want, no matter how wrong it is, they will simply find another platform that lets them share it. This is why Fox News exists. Instead of outright censoring fake stories, websites need to bring attention to falsehoods. A handy warning message prominently displayed on the article linking to an appropriate Snopes page would be a good step. This way, people are still free to share their fake news stories, but because they're doing it on your platform, you can use this opportunity to give anyone who sees it additional information that allows them to recognize the story as false. If you just censor the story, people will use another platform that you have no control over.

Example of flagged news story

The answer to filter bubbles is not to simply crush the filter bubble you don't agree with. The fact that this notion is even being entertained is frightening. Culture wars are not won by suffocation, they are won by changing minds, and the only way to do that is to respect free speech and be willing to debate people who have views that are wildly and totally incompatible with yours, so long as they are respectful. Otherwise, all we will do is scream at each other, and President Trump may only be the beginning of a catastrophic chasm in America.

You can't change minds by throwing stones. This applies to both sides.