The Linux Kernel

Aaah. Another day, another discussion about how badly C sucks. Today I'm going to counter one of the most often-used arguments in the field, which is amazing because of how poor it is.

"But the Linux kernel was written in C!".

This is a complete fallacy- appeal to authority, specifically.

In order for this to be true, you would have to prove firstly, the Linux kernel is of significant quality. I mean, it's assumed in the argument that the Linux kernel is something you'd want to emulate, but I've seen no evidence that it's actually of noteworthy quality.

Secondly, that the Linux kernel was developed with a reduced cost (bearing in mind it's quality), because after all, anything could be good if you plough in a billion man hours.

And thirdly, that this was because it was written in C. If you happen to hire a bunch of incredible engineers, just for example, then they probably could build a quality work for little, even in a bad language.

As there are few, if any, major kernels written in any other language except C, there's basically no possible comparison, which eliminates the idea of there being an actual argument to be found here.

Of course, at the time, there was hardly anything *else* to use, even for general-purpose programming. This makes it exceedingly irrelevant that the Linux kernel is written in C. Not to mention the fact that kernel mode is definitely a distinct area of programming, like embedded, where the rules are different to that of general-purpose programming. As a niche, what's useful or advisable here is not very applicable to the general world.

JIT integration

Here's a fun puzzle.

Let's say that I want to integrate a JIT into WideC, which is fine and implementable for me because I have LLVM to rock with which supports JIT capabilities.

So let's say that I have an implementation that supports this. How does it look like?

Main() {
    somevalue := Standard.IO.Input.Read(int)();
    Standard.Containers.Array(float, somevalue) variable; // OK, JITs the type
}

This is all well and fine- nice and smooth.

But where does the JIT's memory go? If you don't have enforced-GC in this situation, then you can't really guarantee that the memory is cleaned up correctly. If you do, then those people who do need the performance might get flying fucked.

Maybe I'll just have to live with "less-smooth" if you want to have non-GC. Let's theorize about how this might go.

Main() {
    do_stuff = Standard.JIT.Create(function(int somevalue)() { Standard.Containers.Array(float somevalue) variable; });
    do_stuff(); // do_stuff is an RAII object like std::function
}

This might work. It might not even require being a special language feature.

Problems

Let's get to the meat of the problems that I'm currently facing.

1. Bison blows. No, really. I'm tired of not being able to return non-POD data or pass arguments forward. In addition, I generally find it's interface poor- for example not being able to specify arbitrary ICEs as the token values is going to be the maintenance death of me next time I add a new token. In addition, the code is unreadable, and it uses unnecessary memory allocation. I figure that creating an LR parser shouldn't be so hard, and I don't see why it can't be implemented more simply.

2. The lexer could really use being predictive. :(

3. The AST needs to have a bunch of LLVM stuff in it. This is unhappy faces for me, because I want to have separate code for dealing with LLVM. I don't want to start including all the LLVM stuff in the parser header. Been thinking of PIMPL for this purpose, but man, that shit is ugly.

4. I have seriously GOT TO write a specification. I keep forgetting half my damn semantics. I need to re-host my site, and put up a specification that clearly explains everything.

So much to do

1. Change the parser so that it properly sends location information back through so the semantic analyser can give meaningful errors. Right now, only modules and some variables properly give location information. This is a source of nasty bugs because the analyser will try to access NULL pointers when trying to decide the location information to dish out to the user when someshit goes wrong.

2. Update the collater so that it can collate more than just namespaces and variables.

3. Update the lexer to be predictive instead of backtracking.

4. Work more on the website so that it contains some actual content.

Never overestimate the difficulty of concurrent code

Whenever anything goes wrong in concurrent code, people assume it's because of the concurrency. Especially if it's an apparently-random error, e.g., memory corruption.

But that doesn't actually mean the concurrency is the cause.

Turns out that I read the MSDN page wrong- it was false for "insertion did not occur", not false for "value did not already exist". Whoopsie.

Concurrency fun

Concurrency is no fun. Let's review.

1. The implementation is buggy as fuck. I mean, stupid, obvious bugs.
2. There's nobody around to help. If I post i++ + ++i, then it gets answered by everyone, and if my concurrent code fails, then nobody even looks.

Concurrency in the compiler

So right now, we've identified five stages in compilation:

Lex
Parse
Collate
Compile
Codegen

So which, if any, of these can we run in parallel?

Lex- yes. The lexer simply operates on a series of files. These files could be each opened in parallel.
Parse- yes. The parser simply operates on the tokens produced by the lexer, so each file can be parsed in paralle.
Collate- yes. Thanks to the PPL, a concurrent_unordered_map ought to do just fine here to allow us to collate concurrently.

Compile - Unknown. As this stage currently has no implementation, there's no way to know if I can parallelise that implementation. However, right now, I think the answer may yet be "yes". For example, given a simple binary expression of the form "expr + expr", then logically, each expression could be constructed concurrently. In addition, any statement that does not change the symbol table could be evaluated concurrently.

Codegen - Unlikely. LLVM does not appear to offer much concurrent compilation functionality, so this could get rather messy if concurrency is aimed for.