Coder Corner

To iterate is human, to recurse is divine, and the Big Bang was a stack overflow.

—-

I should have a patent on patents. People would pay me recursively and I would get a cash overflow!

—-

The true meaning of LHS is “Little Hidden Stall”.

—-

A stupid magician is a Garcimoron!  (*)

(*) I realize only 1% of the audience will get that one

Simulationism is basically the belief that we actually live in a computer simulation. While not seriously believing in it, I found it interesting from a programmer’s point of view. Bizarrely a lot of things “make sense” from this perspective:

- We live in a giant, cosmic version of a computer simulation. It started with a Big Bang, equivalent of a Big Boot. It might end up with a reboot. The universe is simply the sandbox in which the simulation runs.

- The entity that created this sandbox can be seen as the First Scientist, or the First Programmer, or whatever you want to call it. It is a being outside of the universe, i.e. outside of the simulation. We might have been created in its image, or not. It depends on what the simulation is trying to prove or achieve, which is something we may never know.

- The laws of physics are simply the rules that have been hardcoded to make the simulation work. There might be no particular reason why the rules are what they are. Maybe they just make for an interesting simulation, in the same way totally artificial and arbitrary game rules create interesting gameplay.

- Fundamental physical constants appear meticulously tweaked because they actually are. Every programmer knows about those “magic values” that most of us used at one point or another to make everything work well. The First Programmer may have tweaked the gravity force, etc, with a cosmic slider just like we would tweak the lacunarity or the fractal increment when generating a procedural landscape.

- We are probably not the first simulation run. We are just one particular run, and if those constants all work out ok this is simply because a large number of failures might have preceded us. Things might not be perfect yet because we are a work in progress, in a way similar to what Teilhard De Chardin was writing (e.g. about the Omega Point). In this iteration humans may not be very wise yet, but we may do better in the next run.

- As a consequence, there might also be no reason for the existence of some things in our universe, in the same way there is often “dead code” remaining in a codebase. Introns in our DNA might be just that. As programmers know, there is no reason to optimize or even clean up your code before it even works. In other words we’re still prototype code, not production code.

-  We have no way to know what the outside world, where the First Programmer lives, looks like. We are quite simply “evolved virtual creatures” similar to the ones Karl Sims created a while back. Only much, much more involved, to the point that our conscience emerged. But then again, conscience might just be a convenient way to label what appears, to us, like unimaginable complexity (which reminds me of what Jean Guitton was saying about randomness: there’s no randomness, it just appears random to us because the forces that acted to produce a given event are beyond our analytical capabilities). So while conscience might appear like a miraculous trait to us, it might just be because we lack the proper caps to grasp it. In the same way a blind person can not really understand what the color blue or red is, we might lack the sense to properly understand conscience. But it might end up being a simple thing to program for a higher being like our First Programmer, in the same way programming “eyes” for a game AI is relatively easy in our own computer simulations. The bottom line anyway, is that we can not imagine the world outside of our universe, no more than a game AI can imagine our “real” world beyond the walls of the computer memory.

- We only see our world through imperfect sensors (our eyes do not see UVs or infrareds, our ears can not hear infra or ultra-sounds, our sense of touch is good for our fingertips but lousy in our back, etc, etc, all our sensors are pretty limited). In the same say a game AI sees the computer world through imperfect sensors like raycasts, sound volumes, collision detection checks, etc. Our limited sensors can not even sample the world inside our universe accurately (our brain does its best to construct something from the limited-accuracy inputs), so they are totally inadequate to figure out the real world of the First Programmer, outside the universe. Similarly, a game AI only “sees” and “hears” and “feels” what its limited sensors have been programmed to feed it with. Those sensors only let the game AI capture a small part of the program they live in. If we would succeed in creating a real AI whose consciousness would emerge, it would first discover the concept of a computer, i.e. the universe beyond their game world. But it still would not be able to imagine our world behind, in the same way our limited sensors can not tell us much about the world of the First Programmer.

- “Rien ne se perd, rien ne se crée, tout se transforme”. This is because memory is limited, really. Atoms or sub-particles are counterparts of bits in the computer memory. When an object is deleted and a different object gets created at the same memory location, it is the same as when our bodies die, decompose, and go back to cosmic dust. Our giant cosmic simulation has a finite amount of resources, in the same way a computer has a finite amount of memory.

- The First Programmer does not intervene in human affairs, does not answer prayers, does not perform miracles, does not spy on each simulated entity, simply because this is not how simulations work. You usually do not mess with a simulation while it is running. You tweak the settings, run the simulation to the end, check the output, adjust the parameters and run another one, until you get the desired results.

- A special note must be written about time, which is a very relative concept. We already know from Einstein, Langevin and others, that time slows down when you go faster (see e.g. the twins paradox, etc). To the limit, when you reach the speed of light, time does not flow anymore, it stops. The photon knows its complete history from birth to death in an instant, etc. For our computer simulations, time passes a lot quicker than for us. A lot of things happen inside a computer simulation in a few nanoseconds, a lot of history. The same might be true for the First Programmer and his simulation, i.e. us. Many centuries for us might pass in the blink of the First Programmer’s eye. This does not favorize interventions or reactions to events happening in our simulated world, simply because it is too quick for the First Programmer to react. For example the whole modern civilization might be simulated in one frame of the First Programmer’s game, so there is not much he can do about punishing sins or rewarding good deeds (if he even cares, after all we are only game AIs in this). The only thing he can do is record the simulation results and analyze them later when the simulation has ended, i.e. after the end of our world and before it is reborn/rebooted for a new iteration.

Another idea for optimizing the SAP when we have a small amount of dynamic objects moving amongst a large amount of static objects:

Put the dynamics in their own vanilla SAP. Get the dynamic-vs-dynamic overlaps from there, nothing new.

Put the dynamics & statics in a special SAP (“Static SAP”, or “Bipartite SAP”) modified to only report dynamic-vs-static overlaps:

• The statics are stored in the SAP as usual (same box/endpoint structures as before)
• The dynamics however are stored in a separate array, using a new box structure.
• When the dynamic objects are inserted in the SAP, their endpoints are not inserted into the same array as for statics. They are stored directly in the new box structure, i.e. in the (dynamic) box class itself. They link to the static arrays of endpoints (their virtual position within those arrays, if they would have been inserted there as usual)
• When running “updateObject” on those dynamic objects, we move their endpoints virtually instead of for real. That is, we still look for the new sorted positions within the arrays of endpoints, but we don’t actually shift the buffers anymore. The update is a const function for the main SAP data-structure, the only thing that gets modified is the data within the dynamic box we’re updating. In other words we don’t have real “swaps” anymore. No more memmoves, thus reduced number of LHS, etc.

The drawback is that we update dynamic objects twice, once in the “static SAP”, once in the normal SAP.

Didn’t try it, don’t know about the perf, but might be interesting.

The comments have been re-activated, after a few people complained. Oh well.

It seems there is one in every team. He is living by the book, following the rules to the letter. He considers himself bright and smart because he always knows the latest trends, the latest official Right Way to write things according to the C++ Standard. He follows it religiously, even the new rules not implemented yet by any compiler. And he looks down on you if you do not write “proper” code. He is the prophet. The Book is right. You must follow the rules.

His zealotry has no limits. He will conscientiously rewrite your “illegal” C++ when you are not looking. For your own good of course.

He will go and meticulously replace all your “i++” with “++i” in your vanilla integer for-loops.

He will go through incredible hoops to get rid of a lonely “goto”, used to skip a large block of code and jump to your function’s epilogue.

He will use unreadable, cryptic, unbelievable templates to replace your simple define, because “define is bad”.

He will tell you with a straight face that he went ahead and replaced all the “NULL” in the codebase with “0″ or “nullptr”, because “NULL is bad C++”.

He filled his head with many of those mantras, and he is obsessed with them. They are the rules. They must be followed.

Well, my dear prophet programmer, I have news for you: you are not bright. You are not smart. You are not clever. You’re a fucking robot. It does not take a genius to blindly follow recipes from your cookbook. You are a brain-washed moron doing a machine’s job. If you blindly follow the Standard, you end up with standard code, which by definition anybody can write.

The best programmers are not the ones blindly following anything. They are exactly the opposite of you. The best programmers are the ones who know when rules should be bent, when boundaries should be broken, and when envelopes should be pushed. The best programmers are the ones who, constantly, on a case by case basis, hundred times a day, stop for a moment and think about how to best solve a problem. They are not the ones turning off their brain to follow a recipe. They are not the ones trying to fit a preconceived solution (design pattern?) to everything. If a preconceived solution solves your problem, it was probably not really a problem worth solving - that is, it is such a common and tired issue that anybody can look up a standard answer in a book. How does solving such a thing in such a way makes you “smart” ?

The best programmers are creative. They have a big imagination, and they are not afraid to use it. They borrow techniques from one field and apply them successfully to an apparently unrelated field, discovering subtle links and connections between them in the process. They are never satisfied with the status quo.

The best programmers, the heroes, the top coders, like Nick of TCB did with the sync-scrolling eons ago, are the ones who invent new techniques to solve problems that nobody solved before them. By definition they are not standard. They are the very opposite of what you preach.

Some years ago I wrote a bit about the Multi-SAP approach in my SAP document. I think I also said I would release some code, but never did. Erwin Coumans tried to implement one version in Bullet, but it’s still marked as “experimental” some years later, and it looks a bit like everybody gave up on that technique - including us in PhysX 3.0.

Well I didn’t.

I still think it’s a very nice method, and very easy to implement once you have a regular sweep-and-prune working. So I’m restarting this effort, refreshing my old prototype, adding some new tricks learnt since last time, etc.

As a teaser, and to show non-believers that yes, it rocks, I quickly added it to my old CDTestFramework, as found in Bullet 2.78. I’ll let you guys test it on your respective machines, but at least here at home, it beats the alternative algorithms in Bullet’s default scene.

msap_demo

Stay tuned…

The typical STL-vector-style dynamic array usually has 2 functions to “clear the array”. One releases objects and deallocates the memory (size and capacity become 0), one simply releases objects and sets the size to 0 without actually deallocating the memory (capacity remains the same).

Let’s pretend the function names are “clear” and “reset”, and let’s pretend that “reset” is the version deallocating the memory, while “clear” does not.

“reset” is good to save memory and make sure you don’t get arrays that grow out of control, never actually releasing huge amount of memory that they may not need anymore.

“clear” is good to limit the number of array resizes when the same array is reused over and over, say from one frame to the next.

But of course the truth is that both are actually bad. “reset” is bad because it makes you resize and reallocate stuff over and over again. “clear” is bad because it will waste memory at some point (Murphy’s Law being what it is, one of those never-deallocated arrays WILL grow out of control even if you think “it never happens”).

However a lot of programmers still continue to use dynamic arrays, and still continue to use either “clear” or “reset” depending on what they think is the best option in each particular case.

I would argue that in most cases except maybe the most trivial ones, one can’t really tell ahead of time whether “reset” or “clear” should be used. And thus, maybe we should not.

Instead, just create a small wrapper doing the proper thing depending on how many objects are actually contained in the array:

void Array::resetOrClear()
{
const int s = size();
const int c = capacity();
if(s>c/2)
clear();
else
reset();
}

In other words:

• if we used more than half the capacity, it’s likely that the memory is not wasted and we may need a similar amount next frame => keep the memory => “clear”
• if we used less than half the capacity, maybe we’re an array that grew for some time during a spike but now the memory isn’t used anymore => discard the memory => “reset”

You don’t need to pause and think for each array whether to call “clear” or “reset”, just call “resetOrClear” all the time, and it will likely make the right decision, limiting both the number of resizes and the amount of wasted memory.

Or better, of course: don’t use bloody dynamic arrays.

It happened in 1994, in Paris.

1994, fucking hell. 17 years ago already?!

I received a tape in the mail, a cassette with copied songs on it, from one of my contacts. I had a lot of contacts all over the country. We were all music junkies. Every week, if not every day, we all sent tapes and songs to each other.

Oh don’t give me that look: I know, it’s bad to copy stuff, whatever. It’s not that, as students, most of us did not have money to spend on CDs. We did. The little money I had, I blew it on CDs, buying cheap junk food for lunch just to afford that rare EP found at Jussieu Music, O’CD, Gibert Joseph, or any of the numerous music shops on my way to school. I visited those shops pretty much every day for three years, and certainly spent a significant amount of cash there.

But it was never enough. I had discovered indie music in general, and indie rock in particular just the year before. And it had blown my mind away. There is little I can report to faithfully explain the sad state of my musical education up to that point. Suffice to say that discovering that new, weird, outlaw, fascinating musical landscape was a milestone in my life. And thus, eager to get up to speed, I traded, listened, swapped, listened, copied, listened – nothing could quench my thirst.

Most of my contacts had been found thanks to the Minitel, which was this old, French, very slow ersatz to Internet, at a time when almost nobody had Internet in France. We were all conscientiously reading “Les Inrockuptibles“, and listening to Bernard Lenoir on the radio every evening (the French counterpart to John Peel). The lucky ones living in Paris attended his famous Black Sessions regularly. For the most part though, we had never met each other. But a shared love of alternative music made us all blood brothers. I spent countless hours creating carefully crafted compilations, noting down the lyrics in hand-written letters, along with annotations, remarks, questions, for people who were effectively total strangers.

One of those strangers had sent me a tape that morning, whose A-side was labeled “Juliana Hatfield - Become What You Are“. I had never heard of her. But then again, at the time I had never heard of many bands. Total rookie.

So I lied on my bed, closed my eyes, and played it once.

And then I played it again.

And once more.

And once more.

That’s how I am, can’t do things “a little bit”. At the time I had a room-mate, JC, who couldn’t listen to the same song just twice in a row, claiming he got “bored”. I was the opposite: each time a song caught my ears I would just stop the CD (even when listening to it for the very first time), and I would replay the same catchy song a good ten times in a row before listening to the next. At the end of the day, I had played that album for hours and hours, and JC was absolutely sick of it. Me, on the other hand, I was delighted. It was perfect. The catchy tunes had captured me. The melodies had grown on me. I loved the voice. I loved the lyrics. I was hooked.

In the following months, I became semi-obsessed with Juliana Hatfield. Oh, it happened all the time. I was equally obsessed with Jeff Buckley, Björk, PJ Harvey, or even Elastica. But Juliana had something that the others lacked: a connection with the audience, a proximity, a humanity that I didn’t feel, or not as much, in the others. Take Björk for example. I loved the music. Hell, I was part of “Etoile Polaire”, a magazine dedicated to Iceland & Björk. I sold Björk records at the Route du Rock in St Malo. I talked about Björk on a radio show. I was a huge fan. But as much as I enjoyed her music, I was rarely able to truly connect to Björk, to put myself in her shoes, to see the world like she did, to make her lyrics mine. Björk was simply too alien, too out there for me to fully understand her. On the other end of the spectrum, a band like Elastica rocked, was great fun, and easy to connect to. But their lyrics lacked depth: right, sure, “waking up and getting up has never been easy”, I get that, I know what you mean. But that’s a bit meh, isn’t it?

Juliana Hatfield however, a bit like Kristin Hersh, had lyrics that went right through your heart. “Ugly“, “Addicted“, “I got no idols“, and many others: those songs hit you like a punch in the guts, and they felt real. They felt like they were written by a real person, a flesh-and-blood human suffering, experiencing the same problems as the rest of us. In her book years later, she wrote that she never felt like a rock star. I think that’s exactly what made her stand out, why we loved her: we simply felt close to her. We fell for the flaws. It was very easy to feel empathy for the fragile girl who “sold her soul for rock and roll and never ever got a kiss“. It was a lot more difficult to feel anything for one more famous yet tedious rock star, like, I don’t know, David-fucking-Lee-Roth.

This all goes back to my favorite Buckley quote, really, from an interview in Les Inrockuptibles:

« Voilà l’héritage des punks: il n’existe plus de héros mais juste des êtres humains. »

That’s punk legacy, there are no more heroes, only humans. No more heroes, as the single from The Stranglers that Elastica (surprise?) so carelessly sampled. We’re running in circles here. Maybe Juliana tried too hard to become something we wouldn’t have cared about anyway. Maybe it’s better indeed, to just become what you are. To be yourself.

So what was she? What is she? Genuine rock star, or fraud? Or simple “music worker”? Nobody cares. Only uninspired journalists and academics care about labels. I hate people who put a sticker on others, reducing them to a single little concept they can easily manipulate and dismiss. I’m a coder. I’m a writer. On the ST I was drawing graphics. Now I do some 3DS Max for KP. Once again, we’re just builders. We create stuff to feel happy, doesn’t matter what it is or in what category the janitors want to put it. As far as Juliana was concerned, what I loved most is that she could also rock the house, with titles like “What a life“, “OK OK“, or even B-sides like “Rider“. Those weren’t from a shy bird in need of protection. Those were from the powerfrau, the huntress out of her lair to kick some male asses. And I loved that mix, that blend of strength and fragility. It felt honest. It felt… human.

Time passed. Over the years, I made mine many of her lyrics. Some of them, sometimes out-of-context, became my mottos. “I got no idol” has long been my answer, my weapon against a lot of so-called heroes, self-appointed experts or prima donnas I was confronted with. It was especially clear when making demos: there was no way I’d have bowed towards anybody. You can still find this quote in some of my old angry texts against the world, from that period. Others like “a heart that hurts is a heart that works” were used as a shield against failed, doomed relationships. But my most cherished line came from one of my favorite Juliana songs ever (a B-side?! What the fuck!): “You showed me it’s a crime to never even try.” How many times did I listen to that! I played that song, “Where would I be without you?“, countless times, always eagerly waiting for the last part when she utters that line. I used to be a pathologically shy boy, easily paralyzed by shyness in front of a benign-yet-scary-for-me situation. But then sometimes, out of the blue, “it’s a crime to never even try” would pop up in my head, repeated like a mantra, and it would push me over the edge, forcing me to “go for it”. The last time it happened, it made me kiss a girl, who is now the mother of my first child. Where would I be without you indeed?

It’s been 17 years since I first discovered Juliana Hatfield. To this date, “Become What You Are” is probably the record I played the most in my life after “Grace” from Jeff Buckley. That’s hundreds and hundreds of time, easily. It’s one of those rare albums where every song, one after the other, became at a time my favorite – no disk filler, nothing to leave out. My old tape died from exhaustion a long time ago, and got replaced by a brand new CD, then later by MP3s. I’m still playing them 17 years later, to create a wall of sound around me at work. It’s a cocoon I never got tired of. I have more than a thousand CDs in my collection; very few passed the test of time as well as this one.

It’s been a few weeks since I discovered Juliana’s new stuff, totally by accident. The book, the story behind God’s Foot, the latest albums. I bought “How To Walk Away”, and played “This Lonely Love” all day long. For a week. I heard “Fade Away” for the first time yesterday. And I can’t stop listening to it. I’m listening to it now, writing this post. This is insanity. Nothing changed. The old magic never vanished. I’m hooked again. I’m addicted again. Hopefully for 17 more years!

I was playing Red Faction Armageddon this week-end when my Xbox suddenly froze. Apparently I’m not the only one having this issue, they released that game with a big “physics bug” in it:

http://www.redfaction.com/forums/topic/14022

(Havok, tsss, tsss!)

Now the funny thing is the workaround that players quickly found: if you destroy everything in the area before pulling the switch (a 100% crash repro otherwise), the bug vanishes I guess I’ll have to try that.

Still very disappointing. So far RF Guerilla was 10 times better.

After maybe a decade using Win2K, I finally joined the modern world and bought a new PC, which came with a pre-installed, 64-bit version of Win7.

Man, what a disaster.

After just a few hours I suddenly remembered why I had not upgraded the machine for all those years.

• The search function (you know, the basic stuff to find files) doesn’t work anymore. Ridiculous.
• The machine only has two times 289 Gb, even though it’s advertized as “640 Gb” (there’s even a sticker on the side claiming this).
• Where the fuck are my Windows CDs or DVD?! You mean I can’t reinstall it when needed? You think it will never get corrupted or something?
• The guarantee is invalidated if I open the machine, LOL. Of course opening the PC was the first thing I did, breaking some seal in the process that was there to prevent me from doing so.  Day 1, guarantee is already dead, yay.
• 3DS MAX refuses to install on this OS. Bloody marvellous. It’s a legal version and everything, but no, of course, doesn’t work. Yes it’s 3DS MAX 6, so fucking what? I’m a programmer, I don’t need all the fancy stuff from new versions, and all those plug-ins don’t work in newer versions of MAX anyway. Right now I’m totally screwed, I can’t create new levels for Konoko Payne. Don’t fucking tell me it’s hard to make something work on both old Windows and new Windows, my own engine does that perfectly, it’s just fucking laziness or stupidity if you can’t even do that.
• Don’t get me started on the new retarded features that keep getting in the way. Give me a bloody search feature that works, then we’ll talk about the fancy bullshit.

I’m very tempted to just format the whole thing and re-install Win2K over the Win7 crap.