Multithreading

This was a mistake. I have learned several things about multithreading ever since I started attempting to restructure my engine around it. Recently I have just discovered that everything I did with multithreading was wrong. Lockless threading was invented for being applied to realtime applications. In addition, Windows has a serious issue with this kind of stuff, and so the best library out there only works on Mac and Linux. All the windows lockless thread are written in C# because they love the garbage collector.

Thus, even beginning to start with implementing lockless stuff would require me to multithread my memory management, my utilities, my graphics engine, my audio engine, and rewrite pretty much every single line of code I've written over the past 4 years. And then I'd have to test all of it.

So instead of making everything threaded, I have ended up having to remove all threads completely. I know that somewhere, somehow, a bunch of programmers are going to laugh at my poor, single-threaded game, but unless I had like 6 other programmers to help me out here, I have no choice. I've learned a lot about priorities, and I know not to do stupid things simply for the sake of being modern. Luckily, this is a 2D graphics engine, so the vast majority of the stress put on the CPU is the physics. Because of this there may be a way for me to evaluate physics while I'm rendering the graphics and doing game logic. If I can just get the physics into another thread, even if its a stop-and-go kind of thing, it would give me all the performance bonuses I need from the second CPU core without needing super-duper complex lockless architectures and god knows what else. It may also be possible to multithread the network update packets by taking advantage of the update function in the cReal objects and forcing them to wait until they are about to sync all their physics data to actually update the information. By carefully structuring the update functions, it is possible to achieve fairly high performance without building an entire goddamn library.

Note to self: Update each physics step with a delta value from the graphics engine.

Updated To-do-list

- Fix cVectorUnique
- Build basic editor
○ Integrate TinyXML map parser
○ Implement image selection
○ Build image editing GUI
○ Build image editing mouse functions
○ Put in XML modification and saving
○ Modify imagesplitter to take command lines
○ Auto-image-splitting
○ Test for stability
- Now C# interop
○ Engine dependencies first
○ Use Interface technique to bypass multiple inheritance incompatabilities
○ cImage dependency tree
○ Ignore GUI
○ Test for stability
- Build kd-tree implementation.
○ Each layer swaps between x and y and has a value that specifies where to make the split.
○ A balance integer is required so the tree can keep itself roughly balanced, but this should be athreshold difference of about 5 or so to prevent a single object from constantly resizing the tree.
○ Each layer holds a list of renderables of that depth. A renderable's total radius must fit entirely inside the branch in question, or if it has no rotation, do a simple bounds check.
○ Most of the work is done when a renderable is inserted, because this is when the bounds checks get made
○ When a renderable moves, it just checks its nearby nodes for a needed crossover.
○ When a renderable changes dimension, it may need to be moved to a higher level.
§ Note that you may be able to combine the scale and movement checks into a single bounds check
- Remove the radial check from anything using the kd-tree, but cImageZ doesn't use the kd-tree so keep the radial check for that one.
- Because use of the kd-tree is optional, you need a way to standardize its use. Some kind of function somewhere saying "Add to render queue" or "add to kd-tree" or something.
- Swap the render buffer to use an additive memory allocator
○ You will need to create a separate render buffer to maintain a list of renderables that don't use the kd-tree.
○ To do this properly you'll want to create a tree merge function with a mass allocation, which will allow for the transfer of memory in one batch per-frame, which should be crazy fast.
○ Do this first, then start adding in stuff from the kd-tree.
- Do stress testing on the kd-tree
○ Make line renders for the tree (that'll be a lot of fun to watch)
○ Ensure optimal performance on low and high density images.
§ Also check the resulting performance hit on a single image

- Implement multiple passes
○ Abstract out all rendering code
○ Move camera attachment to pass
○ Implement placeholder rendering order functionality

- Go back to implementing the lighting system
○ Ensure blending functions correctly
○ Implement shadows (No penumbra! but still use the same circle calculation so your getting the correct umbra)
○ Calculate soft shadow points
○ Build soft shadow triangles
○ put in option to have object either be affected by light or not (Do it as a flag)
○ Put in coronas
○ put in option to have light cast shadows instead of be occluded. This is used for things like the sun, where color is ignored.
○ Optimize
○ Implement arc culling
○ ensure rect culling is working
○ Ensure backup textures allow it to work on the laptop
- Ensure lighting system works well
- Build example of lighting system
- Write up whitepaper for the lighting system using TeX
- Publish via gamedev.net and test reaction

- Implement new text renderer

- Now get back to linux server stuff
○ Transfer postgresql to mysql
○ Build testapp that utilizes that
○ Test on sea's linux box
○ Once stable, build the interface for NAT punchthrough and the game list.
○ Throw it on the webserver and pray to god it works
- Now go back to networked chat
○ Ensure connectivity tests work over a wide range, over internet, on LAN, through firewalls, etc.
○ Ensure chat functions properly
- Integrate lowest level physics
- Network lowest level physics
- Stabilize box2D hack
○ Set up a massive, horrifically nasty stress test for that.
○ Build low level network interpolation equation by defining the required detail level in terms of distance to object between this frame and then next frame.
§ I.E. if our object is moving this fast, it must have highest detail physics information about any object that it could hit by the next network ping, whatever that is.

*skip to main to-do-list*

- Get Brick replica to work
- Define 4 levels of physics serialization
- Finish writing box2D networking interpolation hack
- process packets and implement interpolation on a simple level
- Network physics
- throw bricks
- Implement destructables
- Implement a physics callback system
- Use this to implement impact damage based on relative physics formulas
- Sync destructables using RPC calls
- Put in health bars, network names, and other information
- Sync all this, including rudimentary score information as held by the server
- Build a functional basic shape editor
- Implement protocol buffers
- allow testbed activation on editor using in-game logic
- Build weapon system core
- Implement inventory
- Implement basic grappling gun
- Give GUI basic functionality (Weapon ammo tracking + health, etc.)
- Build options window and ensure most graphics options are functional
- Sync spawned weapon objects
- Build property-based weapon creation system
- Build weapons editor
- Design and implement weapon-centric distribution system
- Design weapon deadliness algorithm
- Implement weapon hashing and self-correcting danger network handling
- Implement anti-cheating weapon designs (weapon combination blacklist too)
- Differentiate between weapon restricted servers and open weapon servers
- Add LUA scripting core
- Integrate into weapons
- Extend weapons editor
- Implement complex object handling system
- Extend physics syncronization to handle complex objects
- Implement 2D nearest neighbor algorithm
- Test interpolation for complex object special cases
- Design complex object animation and syncronization schemes
- Extend weapons to allow for complex objects
- Extend editor to account for complex objects in generic cases
- Extend editor to handle basic animations for complex objects in generic cases
- Implement FX system
- Extend animation editor to handle animations for FX special cases
- Build specialized physics model for client-side FX.
- Integrate FX system into weapon subsystems and physics response system on a generic basis
- Make explosions
- Design hovering situation special-case for physics response system
- Apply this to giant hovering bases
- Ensure large physics object special-case in physics response system is stable
- Adapt 2D nearest neighbor algorithm for 2D lights
- Ensure lights act appropriately in indoor environments
- Implement powerups (including special-case physics response)
- Extend inventory to handle items on an abstract interactive basis
- Extend GUI into final mockup
- Implement unique kill registers for physics callbacks as dependent on weapon type/class/ID, as well as for specific event IDs
- Implement adaptive animation overloading system for complex avatars
- Ensure proper death animation as well as weapon swapping
- Abstract out the entire avatar into a class-system that must adapt for different body shapes.
- Implement class-specific statistics
- Create generic statistic trackers
- Build an interaction response system
- Combine interaction system with complex objects to create a generic vehicle class
- Convert base into a vehicle
- Build vehicles
- Implement vehicle spawn system and vehicle generic handling
- Build adaptive GUI system
- Create specialized vehicle GUI modifications
- Implement Map handling system
- Build map object spawn factory
- Network dynamic map changes
- Integrate LUA core into map scripting
- Compile list of basic map triggers
- Migrate objects over to map object handling
- Allow for multiple situational physics layers on base
- Get that stupid elevator to work
- Implement aircraft as a vehicle subset (this requires a physics response special case)
- Create Resource System
- Modify all spawned upgrades, powerups, vehicles and weapons to have generated resource costs.
- Implement drops
- Implement team resource counter as well as individual resource sharing systems
- Sync these over the network and apply anti-cheating subsystems
- Implement generic multiplayer statistic tracking over the client/server model
- Create the Lobby
- Add rooms
- Build server tracking system using the superserver
- Implement anti-cheating core on superserver and its authorization channels
- Ensure there are sufficient game creation options
- Test initial join and in-game join combinations
- Implement multiplayer statistic tracking over the entire superserver model and website (concept of a 'confirmed kill')
- Website integration

Texas Fucks Everyone

Texas conservatives screw history

Texas Board of Education cuts Thomas Jefferson out of its textbooks.

The Texas Board of Education has been meeting this week to revise its social studies curriculum. During the past three days, “the board’s far-right faction wielded their power to shape lessons on the civil rights movement, the U.S. free enterprise system and hundreds of other topics”:

– To avoid exposing students to “transvestites, transsexuals and who knows what else,” the Board struck the curriculum’s reference to “sex and gender as social constructs.”

– The Board removed Thomas Jefferson from the Texas curriculum, “replacing him with religious right icon John Calvin.”

– The Board refused to require that “students learn that the Constitution prevents the U.S. government from promoting one religion over all others.”

– The Board struck the word “democratic” from the description of the U.S. government, instead terming it a “constitutional republic.”

So now the United States isn't a democratic nation anymore because someone didn't like it. I guess Texans aren't worthy of being called human beings anymore because I don't like them. I guess Fred Meyer shouldn't let conservatives into the store because they're a public nuisance. Why don't we just send all the kids into fucking gas chambers and speed up the process?! How about we just re-name America to "The United States of Idiocy". If anyone so much as says one word in support of Texas, I will BURN YOU ALIVE. I have HAD IT with people amusingly demanding that I be more accepting of other political views.

Oh, I need to be more accepting of other political views. SURE. I have a better idea, why don't you all go fuck yourselves because I don't care. So far I have met about 150 people who have qualified themselves as decent human beings. About 40 of them don't have self-esteem issues.

Society is destroying the only human beings worthy of being called "human." People who are sensible. People who listen to opposing evidence. People who are actually attempting to do something useful. People who understand what is important, people who know that its our intelligence, our creativity, and our dreams that are worth fighting for. People who don't understand why anyone would consider them worthy of such praise without realizing that it is this very trait that puts them far above the average. It's people who know they have to earn their life.

Fucking HDR

I really, really have to avoid thinking about HDR, because once I start working on one thing, I become completely and utterly obsessed with it. I'm talking an obsession more extreme then my love of bunnies. I'm talking an obsession that would have kept me up all night if I had let it. I've tried approximately 240 different equations in my head and none of them work. The approaches don't work. Nothing seems to work other then a few key things:

1. Bloom is subtle
2. Bloom is only visible under high contrast environments

Problem: How the fuck does one determine what exactly high contrast is?

3. IRL bloom behavior is a certain threshold below the brightness in question, and then anything getting too close to the original HDR brightness is completely eliminated.
4. It almost seems like you could just subtract the original source from the bloom, then somehow add this back into the calculation and do the HDR threshold calculation. This doesn't make sense mathematically though.

5. The IRL bloom tends to extend extremely far, but at the very edges its incredibly faint and just barely noticeable. This is stupidly hard to implement in a graphics pipeline because when you get things that faint they tend to just drop out completely instead of being subtle. if you try to use the inverse square law, squaring the values just removes the lower ones completely. I have been unable to find (for LDR anyway) an equation that preserves the subtle low values while throttling the high values. It might be that working with values of 0 to 1 is flawed and I though instead add 1 so that the lowest possible value is 1, thereby enabling division that makes sense. I implemented a version of this that did a really nice job of throttling the HDR values, but it still didn't seem to work very well for bloom.

6. I'm fairly confidant the key here lies in operating under HDR conditions by throwing out the assumptions normal bloom shaders use that are often used just to compensate for woefully inadequate spectrum analysis.

7. One thing that's really annoying is that I'm not entirely sure what I'm looking for each time I attempt to implement this. Furthermore most HDR pictures are corrupted by natural bloom which fucks with my own bloom if I try to use them. For an ideal testing scenario a full HDR pipeline must be implemented with proper HDR spectrum analysis for any of this to be effective. Attempting to put proper bloom on a traditional HDR crush algorithm will look stupid because the image has already been irreparably destroyed by the flawed HDR processing.

8. Note that the hilariously bad HDR example in the directX SDK has a natural bloom technique that, when not being grossly overused, is actually reasonably accurate in that it is circular and has a linear falloff. Ideally the falloff would be more exponential but that's proven to be stupendously difficult, so a subtler version of this would work really nicely.

9. hidden

I need to shut down my train of thought here before it overrides everything else (it's finals week for crying out loud), but hopefully the next time I become obsessed with this, it will be next year, about this time, when I have the proper environment to work in. When that happens I need to do some research on photographic HDR analysis, since I'm pretty sure they've figured out equations for a lot of the spectrum analysis.

Holy crap

I must have accidentally imbued my to-do-list with rabbits or something, because it just gets bigger and bigger.

To-Do
- _bucketsort cRBT_list isn't working because the getnear function doesn't differentiate between a "before" and an "after."
○ Implement checking for this so it can fail appropariately
○ Figure out if it needs to be a before or an after for the actual getnear call in the alloc function
- Do stress tests on the memory allocator and make sure it doesn't fail under any circumstances
- Do a test on the string pool allocator being inside the DLL and try to figure out why the hell the allocator gets initialized like 6 times.
- Finalize the string pool for usage
- Finish the last bit of the string table

- Build kd-tree implementation.
○ Each layer swaps between x and y and has a value that specifies where to make the split.
○ A balance integer is required so the tree can keep itself roughly balanced, but this should be athreshold difference of about 5 or so to prevent a single object from constantly resizing the tree.
○ Each layer holds a list of renderables of that depth. A renderable's total radius must fit entirely inside the branch in question, or if it has no rotation, do a simple bounds check.
○ Most of the work is done when a renderable is inserted, because this is when the bounds checks get made
○ When a renderable moves, it just checks its nearby nodes for a needed crossover.
○ When a renderable changes dimension, it may need to be moved to a higher level.
§ Note that you may be able to combine the scale and movement checks into a single bounds check
- Remove the radial check from anything using the kd-tree, but cImageZ doesn't use the kd-tree so keep the radial check for that one.
- Because use of the kd-tree is optional, you need a way to standardize its use. Some kind of function somewhere saying "Add to render queue" or "add to kd-tree" or something.
- Swap the render buffer to use an additive memory allocator
○ You will need to create a separate render buffer to maintain a list of renderables that don't use the kd-tree.
○ To do this properly you'll want to create a tree merge function with a mass allocation, which will allow for the transfer of memory in one batch per-frame, which should be crazy fast.
○ Do this first, then start adding in stuff from the kd-tree.
- Do stress testing on the kd-tree
○ Make line renders for the tree (that'll be a lot of fun to watch)
○ Ensure optimal performance on low and high density images.
§ Also check the resulting performance hit on a single image

Implement front-to-back transparency blending
- First, undo all that stuff you just did.
- The first thing that needs to be working is the multi-texture technique.
- Then, you need to make sure the blending is there, and that the backbuffer has an alpha (Should be done already)
- At the end of the render, reset the alpha channel to opaque
- Now you should be able to see the results. Ensure the blending functions properly under high tranparency complexities and adjust it as necessary
- Once the blending is working perfectly, enable the stencil buffer and do a write when the pixel is opaque (or in the case of this algorithm, completely transparent.
- Now extend this to multi-texture scenarios
- Then, ensure that it still works with the lighting technique.
○ Later on you can figure out if the alternate lighting method is still viable
- Done properly, the normalmaps should automatically use this technique as well. You may need to tweak that a bit though

- Go back to implementing the lighting system
○ Ensure blending functions correctly
○ Implement shadows (No penumbra! but still use the same circle calculation so your getting the correct umbra)
○ Calculate soft shadow points
○ Build soft shadow triangles
○ put in option to have object either be affected by light or not (Do it as a flag)
○ Put in coronas
○ put in option to have light cast shadows instead of be occluded. This is used for things like the sun, where color is ignored.
○ Optimize
○ Implement arc culling
○ ensure rect culling is working
○ Ensure backup textures allow it to work on the laptop

- Implement new text renderer
- Do C# interop