sizeoff(), ABI, and code duplication

I'm looking at implementing more ABI stuff. Right now, I have the vtable layout stuff fixed, I hope- the vtable layout can now contain things other than virtual functions, like virtual destructors, deleting destructors, offsets (glory) and RTTI pointers.

What I'm really thinking about right now though is layout.

In order for Wide and C++ to communicate using a type, they have to agree on it's layout. You could not have a type where Wide thinks one subobject is in a different place. The problem with this is that Clang cannot lay out arbitrary types in the way that Wide can- it can only lay out Clang types, and it can only do so in the context of a particular translation unit.

The real problem is sizeof(). Since sizeof() is a constant, I have to know when you request the size how to lay out the class. If I lay it out in one way, and then Clang lays it out in another, I can't simply drop my own layout. I have to know beforehand. This means either strictly laying out all classes in the same way as Clang (sucks), and duplicating their layout code, or, change sizeof().

I've been thinking about introducing a new class of value- a semi-constant, you could call it. The value would not be a constant (since it's only semi) but not vary at run-time. There would be some language features that could accept semi-constants instead of constants- say, array size.

Another advantage of this would be that strictly speaking, the code would be more platform-independent. One of the reasons that C++ is not platform independent right now is that when you use sizeof(), it has to tell you the size. You can't port the IR output of Clang from x86 to x64 because the sizeof()s will be incorrect.

But in principle, a hypothetical Wide VM could use the same LLVM IR across multiple platforms. There's already work in this direction with PNaCL.

Quick edit: First, LLVM's array types take only integer values, not constantexprs, so that would be fun. Second, turns out that Itanium ABI specifies a bunch of secondary virtual tables which are complete duplicates of the primary ones for ... some reason. I didn't have these secondary tables. This just shows that I really, really need more tests. But onwards and upwards, as they say. I will stop implementing new features when I have Itanium-compatible EH. And my code count is now 18,500. Feel the growth.

Thirdly, my laparoscopy is scheduled for Monday. If the surgeon gives me the all clear then I'm done, done, done.

Implemented basic constructor exporting: check.
Implemented basic destructor exporting: check
Fixed Coliru: check

TODO tomorrow: Virtual Itanium ABI destructors and make a start on RTTI, video maybe, definitely a lot of slacking and eating, dog cuddling and walking, you know. I've found that somehow life's more satisfying when you do stuff instead of cry about stuff.

What I really ought to do is UNINSTALL GAMES.

ABI support

I need to seriously consider how dependent I am on a particular ABI. I've been looking into adding RTTI and several parts of my code could not possibly handle another ABI. I've certainly been thinking about supporting Microsoft ABI as well as Itanium. Some ABI details Clang very neatly abstracts away from us. Some it does not.

Mangled names are one example of an ABI detail that I need virtually never concern myself with. Clang has a simple function to mangle the name, I use it, I'm done. The mangled name does not concern me in the slightest. There are a few ABI details for which this does concern me but they're quite limited and easily handled.

Class layout is (will be) another. Soon I can unify AggregateType and ClangType, and allow Clang to perform all layout for Clang types. This will simply be a question of setting the appropriate ABI and letting Clang handle the rest.

Vtable layout is something I do myself, which will require adjustment. Currently, based on some Clang APIs, for Itanium ABI I can perform a compliant vtable layout. For Microsoft ABI I'd have to rework this code.

Calling convention. Part of calling convention is handled by LLVM but another part is handled by Clang. I'm not quite sure why non-complex types are not handled entirely at the LLVM level but that's another question. I will probably have to duplicate Clang's code here (it's quite short) to determine the correct calling convention for C++ functions. For Wide functions I can use whatever calling convention I like.

RTTI will be completely ABI-dependent, as will EH. Clang contains some support routines for RTTI for Itanium, I'm not sure how solid they are for Microsoft ABI as their support for that is still under construction.

Just for reference, Itanium ABI is the one followed by GCC and Clang on nearly all platforms, optionally including Windows. ARM ABI is used on ARM processors and is a close derivative. As far as I'm aware, Microsoft are pretty much the only ones who don't follow Itanium ABI, on any platform.

I've been wondering about how to architect support for various ABIs. FunctionType, my class that handles calling functions, will probably need re-working to handle calling functions of differing ABIs, and thunk-handling code will have to be able to generate thunks for more than one ABI. For stuff like vtables, a single class can only have one vtable layout, but I figure that the base classes can have vtables in any ABI.

Currently, Wide does not take advantage of ODR- every TU's copy of a given type representation is a distinct Wide type. This is something I'd like to change but for sure, every ABI's copy of a given type is distinct.

The next thing I need to do in terms of ABI support is exporting constructors and destructors, and support virtual destructors. When this is done, I can move to RTTI and then EH.

For search paths on Coliru, I have decided to simply hardcode them into the Wide shell script. That will solve the immediate problem of not being able to use it as a demo.

Clang- not designed as a library

It's becoming all too clear to me that Clang was not, in fact, designed as a library, except for some uses supporting Intellisense and such. Here's an unfortunate and simple example. Clang has acres of code (it's really quite a lot) to handle finding G++ include paths. But it's impossible to re-use this code in Wide because their structure talks in terms of Clang driver command-line arguments. So now I can't deploy Wide to Unix systems because I can't find the G++ include paths, which vary a lot more than you'd expect from system to system (why? who knows). Clang can find them, but good luck actually getting that to function when you're using Clang as a library.

This is a prime example of what I want to avoid with Wide.

Lines of code

On a more personal note, I love watching the lines. I run the command to check how many lines Wide is nearly every commit. It's not that I feel that this is some empirical evidence of quality. We all know that adding LoC means little. But when the lines of code grows a lot, I feel like I'm making progress. Just for reference, the entire LLVM Project (including clang, and some other subprojects) has 860,000 lines of code right now, and 18,000 tests. I have 18,000 lines and 140 tests. I guess this always leaves me feeling like the small fish in the pond (also that LLVM has nearly triple the number of tests that I do when accounting for codebase size).

Obviously I like to feel like my code is high quality. I don't mind making changes that reduce the LoC and I know there are plenty of good changes that decrease it and bad changes that increase it. Tests are included in my measurement so the more tests I have, the higher that value should be. But ultimately, as an entirely subjective feeling, I feel like I should be adding to the codebase's size.

I've been sitting pretty at about 17-18k for a while now. I guess it's a good thing that I've implemented many new features like inheritance without substantially increasing the size of the codebase, and since I've introduced automated testing (with many more tests to come, hopefully) the reliability is a lot higher. And now that I'm not horribly, horribly sick, I'm much more available.

What I really need to do is ensure that I spend less of my time chatting in the Lounge, shooting people in the face, flying spacecraft, or lynching people for being the Mafia, and more time working. Also job-hunting. That would be good too. Maybe I should ask Daisy to help me, she's always happy to make sure that my left hand isn't good for much.

Time to write tests for all those new features I implemented. And devise a test driver for warnings.

Fixing up some real-world issues.

Yesterday I took a big crack at fixing up the meagre stdlib and associated. Turns out there were a few issues that I don't currently test for that were missing. Here's the commit. Notable is that I have no tests for any of these fixes. Also notable is my lack of tests for warnings, my driver won't support them so more fun there, no tests for the lib itself, etc. Plus I need to look into exception handling (the joy!). I also need to fix things like overload set exposure to C++. And I also need to look into more ABI support- particularly for Microsoft, but also better for Itanium, including RTTI and EH.

Once I have RTTI and EH across all platforms, I can bootstrap Wide and that shall be a glorious day. Let's face it, right now the language definitely feels like C with Classes with a few nice extras on top like lambdas. Sometimes it's hard not to try and rush directly for these features. Rushing for features in the past has been a bad move for me, though. It's clear that I still don't understand some features I want, like incremental re-analysis, and the new semantic error handling model is a mile away.

On the upside the non-void falloff warning means computing the CFG, which can resolve a few issues. LLVM is amazingly finnicky about when it will and will not accept code. For example, consider the following program:

    f() {
        return true;
        return true;
    }


It's pretty clear that the second return will never be executed but you would think that it should be legal. Perhaps I will consider explicitly rejecting such code in the future. However, naively generating both returns to LLVM IR will result in an assertion failure, because LLVM will not handle more than one control flow statement per basic block. Using a CFG can avoid such problems because we can eliminate statements without any predecessors. We can also issue a warning.

This leads us to the next question. Imagine something like

    f(arg) {
        if (decltype(arg).size > 5)
            return true;
    }


It seems obvious to all that this can fall off the end of a non-void function- for some instances. Other instances cannot. We could discriminate at compile-time, but should we? Right now, the compiler will warn for this function for all instances. When I implement some constant folding (low priority) it will stop warning for instances where it's statically provable.

In short, it's pretty obvious that I need more manpower. There's just so many tests to write, so many new features to implement, and I need people to bounce ideas off. Monologuing into a blog only serves this purpose to some degree, and the LLVM chat only really suffices for lower-level code-generation stuff (thanks for the help on that stuff, btw). Plus, I don't get Cool Internet Points for working in silence in a corner.

I've been considering putting together a YouTube video or two about Wide. I don't know shit about animating or anything, but when I have another Unix build and upload it, I have my online compiler back again, which should make life a lot easier w.r.t. advertising the language. Just go here and play with this sample, and you can see how easy it is. Fixing up my VS addin (I doubt it needs much work) would also help in this regard.

Uses and analyzer design

I believe I've come to the next stage of analyzer design. It occurred to me that many of the problems I'm looking at have already been solved, by LLVM. Simply stealing their design would seem to be an appropriate solution here.

The problem I've been considering is thteefold. One, exceptions. Currently, I can only determine which destructors need calling at the Statement level. However, I need to be able to determine which destructors need calling at the Expression level in order to implement appropriate EH. Secondly, I've been considering the uses problem. For example, given an ImplicitTemporaryExpression, which is stored to, and I'm trying to load from, is it safe to elide the temporary and just take the value that was stored to it? Only if I'm the only user. This suggests that I need to be able to track who uses what expressions. Thirdly, I've been considering the problem of incremental re-analysis and such further. I've come to the conclusion that there are two different types of Expression. The first cannot change- it is an implementation. The second can and it would be a function. The key insight here is that first, I can represent these as different types in my analyzer. In addition, if the second is viewed as a function, then the arguments are all "metaexpressions" that are arguments- including their types, which are meta-expressions.

First, we observe that all expression dependencies form a DAG, or should do. Second, we maintain a list of those uses. When an expression's use count drops to zero, we destroy it. This gives us several things. first, the ability to find and enumerate all uses of an expression. Second, we can eliminate all those annoying ExpressionReference things. 

Right now, I am looking at issuing warnings through control flow analysis- e.g. flow may reach end of non-void function. But after that, it's time for another analysis overhaul... great.

I've had one last-ditch thought about the syntax, and I may just introduce attribute syntax from C#- say something like

    [export := "name"]
    [export := cpp("main.cpp").print]
    [return := blah]
    f() { return "hello"; }

I also have failed to consider exporting functions or dynamic functions where their return types or arguments are is-a matches but not an exact match. I need to unify my thunk-generating code to handle these issues.

Finally, I also need to add one of the features I really needed from this analyzer design- MultiTypeDependency. This will essentially tell the analyzer which expressions/statements hold dependencies on arguments of variable type.