I personally believe these tools can be a net add (with lots of straight jackets and caveats), but I also think we’ve reached a point where I can’t really be sure whether the incoming PR requests are from people using AI tooling to help them, or just free roaming OpenClaw instances that are masking their AI nature very well, and so it feels like I’d be firmly swimming against the current as a very part-time open source contributor to try and police this.

To be an open and clear about my use of AI I’ve cut pre-ai releases for all my libraries so you can get access to the code from the point before I personally started to use AI tooling (I 100% cannot confirm whether any of the contributions from others to the libraries used AI or not, but I can say with certainty that I did not use AI before these points).

The pre-ai tags are:

• utest.h
• ubench.h
• utf8.h
• json.h
• subprocess.h
• hashmap.h

I’ve also added a disclaimer to the README.md of each of the repositories that states:

AI Usage

AI tool use is explicitly permitted in commits to this repository. There is a tagged release pre-ai that denotes the last release where AI tooling was not used.

And each of the PRs authored by my pi.dev agent has the footer:

🧙 Conjured by AI via pi.dev using gpt-5.5

As does each commit the AI authored too.

Over December I noticed an uptick in people that I highly respect using Claude Code 4.5 in particular, and this got me curious. I knew the AI was rubbish. These are people I have a huge respect for (I worked with a fair few of them and knew first hand how good at programming they were). What the heck were they seeing in the AI that I wasn’t?. So I got the cheapest Claude Code plan and started messing around.

I wanted to evaluate it against something that wasn’t on the internet already that it could find or have seen, and something that I wrote and knew intimately. That way I could judge its ability and compare what it produced against what I had. I have a little Rust project that scrapes a bunch of financial data on the various stocks I’m invested in, and gives me fundamentals so that I know at a glance that the companies are still in as good as a shape as I expected.

I explained to Claude Code what I wanted at a high level (write it in Rust, get the data from here, calculate these metrics). I was pleasantly surprised when it came back to me asking me follow-up questions to check it understood what I wanted (remember - I hadn’t used Claude Code before nor seen it in use, so I was discovering how it worked as I went!). It went off and burned a bunch of tokens from my plan and then it was done. I ran the program, compared its output to my own. It matched. Huh I thought. I looked at the code. Well commented, laid out nicely, it had even optimized a few of the async http request queries in a way I hadn’t thought to do. Huh.

Then Claude Code 4.6 came out with the new models. And oh boy they are better. I pointed it at my previous blog post I [[musttail]] You About a Tokenizer and asked it to flesh that out into a fully fledged programming language with a few tips here and there of how I wanted it to look. And it just smashed through it, writing lots of unit tests on its way. Huh. I upgraded to the Max plan.

I’ve since recommended to many people, who were of varying degrees AI-sceptical, that they try Claude Code. The conversation has roughly went like this:

• Me: “You should try Claude Code.”
• Them: “Nah AI is rubbish.”
• Me: “No seriously give it a go and tell me what you think.”
• Them: tries Claude Code “Ok its pretty good.”
• Me: “Isn’t it?”
• Them: “I’ve just upgraded to the Max plan.”

This has happened multiple times now, with a similar flow.

My best description of what is happening to our industry now is that we’re collectively going through the Five Stages of Grief for the craft we all love:

• Denial: I was telling everyone how crap the AI tooling was.
• Anger: I was getting angry at all the AI lovers telling me how good the models were.
• Bargaining: I was realising the AIs could do something but not my job.
• Depression: I got really down that the thing I’ve made a huge part of my identity, coding, was being affected.
• Acceptance: I’m now realising that I can use these tools to magnify what I want my code to do by using all the knowledge and taste I’ve accumulated to craft more software of the ilk I desire.

That’s my journey with this, but I’ve seen many many people across the industry who seem to me to be in the various stages. It sucks to be a craft furniture maker when IKEA just showed up in town.

Now don’t get me wrong - there are problems with the models. They are not perfect. They are like a super eager beaver intern and need a good set of rules to follow to make software you like. But once you start using them you realise how you can craft these rules so that the models produce quality code that meets your quality bar.

But it’s starting to feel essential to my future employment, perhaps all future employment in our industry, that I learn how to do this scaffolding.

int foo(float x);

int bar(float x) {
  x += 42.0f;
  return foo(x);
}

In the above example, foo is a tail call within the function bar. Compilers can take advantage of this by doing something called tail-call optimization, which allows the compiler to not push a new stack frame, and also to change the call into a jump. This means you can have infinitely deep callstacks that won’t cause stack overflows for instance.

Clang has a cool compiler attribute [[clang::musttail]] that you can place on a return statement to force a tail call. EG. it becomes a compiler error if the compiler could not perform a tail call. For example functions with different arguments would not work.

int foo(float x);

int bar(float x) {
  x += 42.0f;
  [[clang::musttail]] return foo(x);
}

Another fun little thing I’ve been toying with is using lookup tables - in particular jump tables. Jump tables are arrays of function pointers such that the index into the array will result in you jumping to some other function to execute.

So an idea came to me - could I use tail calls in combination with jump tables to create a tokenizer for a toy language? A tokenizer where there were zero branches?

Turns out - yes you can! I’ve put up a gist of it here.

An example program in the toy language would be:

# Some cool comment!
some_func := (a : i32, b : f32) : f64 {
  x := a + 42;
  x += 13;
  r := 0;
  for i in range(0, x) {
    if b > 0.0e0 {
      r *= b;
    }
  }
  return r;
}

The tokenizer is basically a number of different jump tables all 256 sized (the number of unique values in an 8-bit ASCII value), and works by passing each 8-bit value as the index to the jump table and then doing a tail call on that function pointer - thus not creating a call frame. So the code goes into the main jump table, and from there will filter into sub jump tables whether the ASCII character is a digit, a symbol, a comment, etc. Lets look at parsing the comment from the example program above:

# Some cool comment!

Firstly - the main_jump_table_s class contains, well, the main jump table. And the # index into that jump table takes you to a function:

static int comment(tokenizer_s *const tokenizer, token_t *out_token) {
  *out_token = {token_type_comment, tokenizer->current, 1, tokenizer->line,
                tokenizer->column};
  tokenizer->skip_current();
  [[clang::musttail]] return comment_jump_table_s::singleton()
      .jt[tokenizer->data[tokenizer->current]](tokenizer, out_token);
}

This function:

• Sets the token type to be a comment.
• Stores the locations of the start of the token.
• Bumps the tokenizer beyond the # character.
• And then does a tail call into the comment_jump_table_s, using the next character in the input data stream for the index.

struct comment_jump_table_s final
    : public tokenizer_jump_table_s<comment_jump_table_s> {
  static int newline(tokenizer_s *const tokenizer, token_t *out_token) {
    tokenizer->current += 1;
    tokenizer->line += 1;
    tokenizer->column = 0;
    return 0;
  }

  static int keep_going(tokenizer_s *const tokenizer, token_t *out_token) {
    tokenizer->skip_current();
    out_token->length += 1;
    [[clang::musttail]] return comment_jump_table_s::singleton()
        .jt[tokenizer->data[tokenizer->current]](tokenizer, out_token);
  }

  constexpr comment_jump_table_s() : Super(&keep_going) {
    jt['\n'] = &newline;
    jt['\0'] = &return_zero;
  }
};

Then in the comment_jump_table_s we initialize the jump table by default to keep_going (see the Super constructor), and specify that for the two exit conditions we should call their respective functions (either newline, or end of string).

Through these various jump tables we can build up complex tokenization patterns without having a complex soup of branching, switches, loops. I’m actually quite impressed at how elegant this tokenizer turned out to be just 742 lines of code.

And what does the codegen look like too?

<_ZN17main_jump_table_s7commentEP11tokenizer_sP7token_s>:
;     *out_token = {token_type_comment, tokenizer->current, 1, tokenizer->line,
mov eax, dword ptr [rdi + 0x8]
mov dword ptr [rsi], 0x1
mov rcx, qword ptr [rdi + 0xc]
mov dword ptr [rsi + 0x4], eax
mov dword ptr [rsi + 0x8], 0x1
mov qword ptr [rsi + 0xc], rcx
;     current += 1;
mov eax, dword ptr [rdi + 0x8]
inc eax
mov dword ptr [rdi + 0x8], eax
;     column += 1;
inc dword ptr [rdi + 0x10]
;         .jt[tokenizer->data[tokenizer->current]](tokenizer, out_token);
mov rcx, qword ptr [rdi]
movzx eax, byte ptr [rcx + rax]
;     [[clang::musttail]] return comment_jump_table_s::singleton()
lea rcx, [rip]              # 0x34 <_ZN17main_jump_table_s7commentEP11tokenizer_sP7token_s+0x34>
jmp qword ptr [rcx + 8*rax]

And that’s overall not too shabby. I could probably change the function signature to pass bits of the tokenizer and the output token down individually rather than as pointer arguments, thus avoiding having to do indirect loads and stores through their pointers. But overall I’m quite happy that the whole thing worked out as nice as it did.

My motivation in allowing asserts to be runtime togglable is that we tried with the custom clang compiler we are working on for Verse here at Epic Games to just always enable asserts in clang, and kept running into upstream clang bugs with asserts enabled. We decided to compile asserts off (like mostly everyone does), but it always annoys me that clang could benefit from these found bugs being reported and fixed and that we are now flying blind (like mostly everyone does!) in the face of the upstream bugs.

I deferred to their wisdom - but wondered if there was an easy way to check that their assertions (ba-dum-tish) were correct?

So I hacked the LLVM CMake to add:

add_definitions(-include AssertInjectoroo.h)

To let me include a header upfront in every source file. Then in AssertInjectoroo.h I did:

#undef NDEBUG

To make sure the extra LLVM checks were always compiled in.

#include <assert.h>

Included assert.h so that if any LLVM file included it, it wouldn’t be included again.

#undef assert

I undefined the original C assert because I was going to provide my own.

#if defined(__cplusplus)
extern "C"
#endif
int ExposeAssertsEnabledToC(void) __attribute__((weak));

#define assert(x) if (__builtin_expect(ExposeAssertsEnabledToC(), false)) { if (!(x)) { llvm_unreachable(#x); } }

#if defined(__cplusplus) && __cplusplus >= 201703L
#include "llvm/Support/CommandLine.h"

#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wglobal-constructors"

__attribute__ ((weak)) llvm::cl::opt<bool> AssertsEnabled("enable-asserts", llvm::cl::desc("enable asserts"), llvm::cl::init(false));

#pragma clang diagnostic pop

extern "C" int ExposeAssertsEnabledToC(void)
{
  return AssertsEnabled;
}

#endif

Then I used this hack to provide my own assert definition. Because I wanted to control whether the assert triggered with an LLVM option (so that the built clang would automagically be able to read my command line option and set this flag), and the LLVM option stuff is C++ metasoup code, I had to provide a hack to get at the definition from C files with the weak symbol.

The new assert itself just checks whether the flag has been set, before doing the underlying asserting check and calling llvm_unreachable if the assert fails.

So first off - executable size differences. My injected assert made the binary 1.85x bigger on a release build with clang. Ouch! If I compare my build to just compiling with LLVM_ENABLE_ASSERTIONS on in the stock LLVM build my injected assert only made the binary 1.005x bigger though - so its just that asserts in LLVM are costly for binary size.

Next up - runtime cost. I used a bootstrap build of clang to test the runtime performance (thanks to Chandler Carruth for this suggestion), building my compiler and stock LLVM and then using these built compilers to rebuild clang. My injected assert that was toggled off at runtime made compilation time 14.00% slower than stock LLVM. If I had the injected asserts toggled on compilation time was 20.61% slower than stock LLVM.

So what did we learn? Doing the calls to assert themselves results in a 6% performance hit. Processing all the other data structures that are hidden behind NDEBUG checks is the main cost with a 14% performance hit attributed to that. This matches with what Renato Golin and Reid Kleckner said on the thread - that LLVM is just not setup to have runtime togglable asserts the way it is, and it’d be a monumental effort to make that work.

__m128 test(const __m128 vec)
{
    return _mm_sqrt_ps(vec);
}

Clang would compile it correctly without fast-math:

test:                                   # @test
        sqrtps  xmm0, xmm0
        ret

And create this monstrosity with -ffast-math:

.LCPI0_0:
        .long   0xbf000000                      # float -0.5
        .long   0xbf000000                      # float -0.5
        .long   0xbf000000                      # float -0.5
        .long   0xbf000000                      # float -0.5
.LCPI0_1:
        .long   0xc0400000                      # float -3
        .long   0xc0400000                      # float -3
        .long   0xc0400000                      # float -3
        .long   0xc0400000                      # float -3
test:
        rsqrtps xmm1, xmm0
        movaps  xmm2, xmm0
        mulps   xmm2, xmm1
        movaps  xmm3, xmmword ptr [rip + .LCPI0_0] # xmm3 = [-5.0E-1,-5.0E-1,-5.0E-1,-5.0E-1]
        mulps   xmm3, xmm2
        mulps   xmm2, xmm1
        addps   xmm2, xmmword ptr [rip + .LCPI0_1]
        mulps   xmm2, xmm3
        xorps   xmm1, xmm1
        cmpneqps        xmm0, xmm1
        andps   xmm0, xmm2
        ret

The optimization flow here in LLVM is:

• That under fast-math conditions, sqrt(x) == x * rsqrt(x), so it uses rsqrtps instead.
• But that has precision issues between Intel and AMD due to a high ULP tolerance for the rsqrtps instruction.
• So LLVM does two newton-raphson iterations anytime it calls rsqrtps to correct the precision between the CPU implementations.

The ‘fix’ here is just to use inline assembly to guarantee you’ll get the instruction selection you want always:

__m128 test(__m128 vec)
{
    __asm__ ("sqrtps %1, %0" : "=x"(vec) : "x"(vec));
    return vec;
}

But there is one additional thing I’d advocate you do if you need to use inline assembly - write your own constant folding.

See the one downside to the inline assembly above is that if test is inlined and vec was a constant, it wouldn’t constant fold it. For example:

__attribute__((always_inline)) __m128 test(__m128 vec)
{
    __asm__ ("sqrtps %1, %0" : "=x"(vec) : "x"(vec));
    return vec;
}

__m128 call_test()
{
    return test(_mm_setr_ps(1.f, 2.f, 3.f, 4.f));
}

Will produce:

test:
        sqrtps  xmm0, xmm0
        ret
.LCPI1_0:
        .long   0x3f800000                      # float 1
        .long   0x40000000                      # float 2
        .long   0x40400000                      # float 3
        .long   0x40800000                      # float 4
call_test:
        movaps  xmm0, xmmword ptr [rip + .LCPI1_0] # xmm0 = [1.0E+0,2.0E+0,3.0E+0,4.0E+0]
        sqrtps  xmm0, xmm0
        ret

So that even under inlining, when we could have constant folded it away entirely, we are still calling sqrtps when we don’t have to. So what is the fix?

LLVM has an intrinsic is_constant which can be got at via the Clang-supported GCC extension __builtin_constant_p. If we extend our test above to check when vec is constant, we can call _mm_sqrt_ps when it is constant, and benefit from the constant folder doing its thing and removing the call entirely. So our code becomes:

__attribute__((always_inline)) __m128 test(__m128 vec)
{
    if (__builtin_constant_p(vec))
    {
        return _mm_sqrt_ps(vec);
    }

    __asm__ ("sqrtps %1, %0" : "=x"(vec) : "x"(vec));
    return vec;
}

__m128 call_test()
{
    return test(_mm_setr_ps(1.f, 2.f, 3.f, 4.f));
}

And we get:

call_test:
        movaps  xmm0, xmmword ptr [rip + .LCPI11_0] # xmm0 = [1.0E+0,2.0E+0,3.0E+0,4.0E+0]
        sqrtps  xmm0, xmm0
        ret

What the heck?! It hasn’t constant folded! Turns out GCC is a bit picky with this builtin, and it looks like LLVM has inherited that funky behaviour. You cannot use it with a vector - even though LLVM happily has the support in the IR for it. But there is a workaround, an ugly one:

__attribute__((always_inline)) __m128 test(__m128 vec)
{
    if (__builtin_constant_p(vec[0]) &&
      __builtin_constant_p(vec[1]) &&
      __builtin_constant_p(vec[2]) &&
      __builtin_constant_p(vec[3]))
    {
        return _mm_sqrt_ps(vec);
    }

    __asm__ ("sqrtps %1, %0" : "=x"(vec) : "x"(vec));
    return vec;
}

__m128 call_test()
{
    return test(_mm_setr_ps(1.f, 2.f, 3.f, 4.f));
}

Will produce:

.LCPI15_0:
        .long   0x3f800000                      # float 1
        .long   0x3fb504f3                      # float 1.41421354
        .long   0x3fddb3d7                      # float 1.73205078
        .long   0x40000000                      # float 2
call_test:
        movaps  xmm0, xmmword ptr [rip + .LCPI15_0] # xmm0 = [1.0E+0,1.41421354E+0,1.73205078E+0,2.0E+0]
        ret

Nice! We’ve got the constant folding we want. And also nicely, if we mark test as noinline instead, the code for test is:

test:
        sqrtps  xmm0, xmm0
        ret

Meaning the branch is folded away. In both cases we now get the behaviour we want. We’ve wrote our own constant folder. Nice! You can see the full example on godbolt.

It’d be nice if we could just use the vector in __builtin_constant_p, but I think the LLVM folks have purposefully tried to match what GCC would do. I’d personally advocate for a loosening of the builtin, and I might file a GitHub issue about just that.

When I was an undergrad my honours project was writing a Software Transactional Memory (STM) system for the PlayStation 3. I loved the PlayStation 3’s architecture and I knew nothing about STMs before I started. This project ended up being awarded the honours project of the year and I had found STMs so interesting a concept. I even toyed with going to EPFL in Switzerland to do a PhD in STMs.

Cut back to 15 years later. Verse needed an STM, it needed C++ and Verse to behave correctly while sharing the same STM, and it would need to be fast.

Well that’s what I, and a great team here at Epic, have been working on - bringing an STM to C++.

I’m not going to revisit the contents of the post (please read it!), but suffice to say the last 18 months working on this has been a blast. We’ve got a ways to go, and I’m looking forward to the challenge.

So how hard is it? Simple! You just add macos-14 to the build -> strategy -> matrix. I took the opportunity to drop macos-latest (which is still set to macos-13, the last x86 runner) and explicity use the oldest supported macos-11 instead.

The new Apple Silicon runner is roughly 2x faster than the x86 one too - nice!

I’m going to do the same steps:

• Compile LLVM using the default clang got via apt-get.
• Compile LLVM again using the clang we just built.
• And compile it a third time with the clang we built with our own clang (this step should be the most accurate picture of the difference in performance between the 4 and 5, because it should be the same binary compiling the same project).

I’ve used the same LLVM version as before just to make the comparisons accurate. There are a also few discrepencies on the image that I should call out - the Raspberry PI 4 image had Clang 11 as the default, whereas the Raspberry PI 5 has Clang 14. So I expect there to be a performance difference for the initial compile, but by the third go through we should be able to derive a good idea of the actual raw performance between the two boards. I’ve used the exact same MicroSD card, have the boards in the same location, and both are using ethernet (not that that will matter as I’m not timing the LLVM download from the interwebs, but consistency is key!).

I’m blown away by this. It is 2.82x faster to compile on the Raspberry PI 5 than the Raspberry PI 4. That’s amazing! I was expecting maybe a 50% improvement at 1.5x, but not nearly three times faster.

So if you have any performance sensitive workloads I highly recommend you upgrade to the new board. I’m glad I did!

For my initial exploration I decided to test just three things:

1. Using the stock Clang compiler I could get at via apt get to compile LLVM. This is Clang 11 at the time of writing this post.
2. Use the Clang 17 compiler built above to compile LLVM itself again. This stage will mean that the LLVM code itself is being built with the latest compiler.
3. Then use the stage 2 compiler to compile LLVM a third time. This will test whether the Clang 17 compiler built by the Clang 17 compiler is more optimal.

I decided to just build with CMAKE_BUILD_TYPE=RELEASE, and I decided to build the Bolt, Clang, LLD, and compiler-rt subprojects. Just to keep it simple but interesting!

So what are the results?

First lets look at the time taken:

As can be seen in the chart it takes a cool 9.52 hours to build with the stock Clang 11 compiler. First observation - wow that’s a long time! But the Clang 17 compiler we built with Clang 11 goes a cool 12.9% faster, coming in at 8.42 hours. The Clang 17 compiler built with Clang 17 would reliably run a smidge faster at 13.3% faster than Clang 11, coming in at 8.40 hours.

And now the binary size of Clang too:

The binary size of the compiler built with Clang 17 was identical in both cases, which is as expected (it’d only differ if there was a compiler bug or some sort of non-deterministic behaviour in the compiler). The binary built with Clang 17 is 15.3% larger than the binary built with Clang 11.

Overall this result shows that Clang 17 is faster than Clang 11, but it is achieving this in part with a sizable increase in binary size.

It’ll be interesting to see how the Raspberry Pi 5 compares in comparison once I can test it!

Let me get all the caveats out the way - I worked in an office full time for the first five years of my career, and have now been working remotely for nine years. You could argue that there will be a bias in my viewpoint because remote work is so intrinsic to my ability to live and work on the Isle of Skye where I’ve made my home. But I think I’ve managed to have a nuanced take on this subject. Feel free to judge though.

Remote work is here to stay. Infact I’d go as far to argue that most people in my industry are already working remotely, but with a caveat I will cover later. The pandemic years were obviously a great accelerator for remote work, but I think the direction of travel was already clear long before the pandemic - it’s just that the pandemic exposed more people to the opportunities that remote work can provide for them and their families.

But I acknowledge that in-person teams are more effective. That ability to collaborate in a really unstructured way is something you just can’t fully replicate in a remote setting. You can’t bond with your colleagues and truly understand how they work, and how to socialise with them without being in-person. It is also harder for young people to enter the workforce remotely as they don’t generally have the money to have a good home working environment - at least as they start their careers. Junior hires also benefit the most from being in an environment where they can absorb all the knowledge floating about a well oiled office.

The problem with the return-to-office hawks is that I fear they don’t really understand what modern office work is like. While there are a few companies that will geographically gate teams to a specific location, nearly all of the big companies I know will have teams spread across multiple locations. When I worked for Unity, most people would work in an office, but the teams would be split across say five offices. This meant that to collaborate with your team you were using Zoom, and fighting for shared meeting rooms or even annoying your open-office colleagues because you were taking a call at your desk.

Long gone are the days where a team is in a single office.

The reason companies do this is because hiring people with the skills you need for a specific team is incredibly hard if you restrict them to a single location. It used to be everyone would relocate to Silicon Valley in search of their fortune. But nowadays no-one I know wants to move there. You have the inflated house prices, the polarised political situation in the USA, the visa pain, the guns - the list goes on. There is also powerful tax benefits available to companies if they set up offices in other states or countries, and so companies have taken advantage of this.

If I was to give an ordering from best-to-worst for the various working arrangements for teams, it’d be:

1. A geographically-locked team where everyone on the team commutes into the same office.
2. A remote team of remote workers.
3. A remote team where everyone commutes into the different offices.

But with the caveat that for 1. you are going to suffer from a much much smaller potential hiring pool of candidates that are even vaguely suitable for the job. This can be mitigated if you have local universities and a team that prioritises hiring junior talent and training them up (a good thing!), but you will have some amount of attrition of people coming through that end up being poached by other companies.

So if you work for a company where your team is only accessible via Zoom anyway, mandating a return to office means you have all of the negatives of remote work and all of the negatives of office work. The worst of all worlds.

For those that don’t know - modern hardware often includes a way to set the rounding mode that floating-point operations use when there is an imprecision in the result of a floating-point operation that cannot be encoded in the format. There are four alternatives that I am aware of:

• Round to nearest - behaves like how math works, will round up or down to the nearest number that can fit in the representation. Often the default rounding mode on CPUs.
• Round to zero - all ties go towards 0. Is often the default rounding mode on GPUs.
• Round to positive infinity.
• Round to negative infinity.

On GPUs, in Vulkan for instance you can use VK_KHR_shader_float_controls to specify the floating point mode for an entire shader execution.

But on CPUs there is often a hardware register that controls the floating-point rounding - on x86 it is the mxcsr register and on 64-bit Arm it is the fpcr register.

C lets you get and set the current rounding mode by using the fenv.h functions fesetround and fegetround. So how costly is it to use these operations?

I did a few tests on an Arm-based Macbook Air M1:

And an x86-based Threadripper Pro:

The tests were as follows:

• Set the rounding mode to round-to-nearest.
• Run 1024 floating-point adds in a benchmark.
• Then run with each call changing the rounding mode to round-to-zero before the add and then resetting it after to the default.
• Then I ran with doing 4, 8, and 16 operations and amortizing any cost when changing the rounding mode.
• And lastly I wanted to know if checking whether the rounding mode matched before changing it could save any cycles - so I tested getting the rounding mode, if it matches I don’t set it, otherwise if it doesn’t I have to set it, and I tested this when it does match and doesn’t.
• And did the whole thing above with calls to tanf too.

So what did we find?

On Arm it is 30x slower when you set the rounding mode. On x86 it is 69x slower. I’d guess that setting these registers does a full pipeline stall in the hardware (the CPU won’t predict ahead), but I don’t know for sure.

On Arm it is only 1.9x slower if the rounding mode you want is actually already being used on the CPU (and so you don’t have to reset it). On x86 it is 8.7x slower. Here’s the code to do this check:

int old_mode = fegetround();
if (old_mode != FE_TOWARDZERO) {
  fesetround(FE_TOWARDZERO);
  // Do your operation here!
  fesetround(old_mode);        
} else {
  // Do your operation here!
}

I’m not sure if the difference in performance between Arm and x86 is some cost in querying the register, or just the different hardware costs for doing a branch like this.

The tan data didn’t show anything hugely interesting beyond the add - but it at least matched the performance characteristics of the floating-point add.

So perhaps unsurprisingly - its slow! Probably about as slow as I would have expected if you were flipping the rounding mode regularly. This is obviously not how these APIs are meant to be used, but you could see a situation where you call some foreign code and you want to ensure it hasn’t messed with the control registers and would want to guard against that - and this gives you some idea of the cost of doing just that.

I remember maybe two or three years into Twitter trying to change my handle over to be my name, and it was already taken. It always irked me that I couldn’t just use my name, but it wasn’t to be. Now with the birdshite being a husk of what it was, and Mastodon being my primary means to engage with people in the industry, I thought this might be the opportunity to just use my actual unique ID (my name) everywhere, including for my website. Anyone who was following me on the old domain’s RSS - sorry! I didn’t realise when I made the migration that the RSS would ping every article like it was new again. Good thing I follow myself on RSS (just to check the feed works - I promise!) and at least noticed that that wasn’t the best.

So anyway - you’ll find new content on these two new links, but with the wonder of Mastodon all my followers have automagically been pulled over to my new handle.

Then friend of the blog Nathan Reed made me aware of Fibonacci Hashing, which is utterly wonderful.

Before my hasher looked like:

unsigned get_bucket(const void* const data, const unsigned length) {
  unsigned hash = crc32_hasher(data, length);
  return hash & (capacity - 1);
}

And now with the fibonacci hash:

unsigned get_bucket(const void* const data, const unsigned length) {
  unsigned hash = crc32_hasher(data, length);
  return (hash * 2654435769u) >> (32 - integer_log2(capacity));
}

2654435769 is chosen by doing pow(2, 32) / fibonaccis_golden_ratio and rounding towards the nearest odd integer (so that the bottom bit is set).

One of the claims of fibonacci hashing is that consecutive inputs will produce a good spread of outputs, meaning you don’t hit problems when multiple things with similar hashes get inserted next to each other. So how does it perform?

The four lines on the chart as as follows:

• avalanche - the percentage of locations that have at least half of their active bits flipped between consecutive inputs. The closer to 100% this is the better a hashing approach is regarded to be.
• average difference - how different two consecutive inputs are to each other by using a delta between them, as a percentage of the range. So an average difference of 50% with a bucket size 8 means that for any two consecutive inputs we’d expect them to have a difference of 4.
• min difference - for two consecutive inputs in the integer range that pass through the fibonacci hash, what is the minimum delta difference between them. So a bucket size of 16 with a minimum difference of 25% would mean that every consecutive input has a difference of at least 4.
• max difference - the opposite of the min difference!

Notice:

• From a bucket size of 8 and up (highly likely with most hashmaps in my experience!) the min, average, and max differences are all stable. I’m especially happy with the min difference - as this means that we can rely on the fibonacci hash putting a 38% delta difference between consecutive inputs.
• We average about a 47% difference between any two consecutive inputs.
• The avalanche percentage is quite lumpy - this is because any log2(bucket_size) that is odd will result in us under or over rounding when we check if at least half the bits flipped. I’ve leaned on under rounding any ties.
• Overall we’re seeing about a 50% probability for avalanching to have happened between consecutive inputs. But remember - this fibonacci hash isn’t our actual hash! It’s what we apply after we’ve already hashed the key. So this will compound with the hash I actually use on the input keys, meaning this just helps us avalanche better than before. Nice!

I’m pretty happy overall with this - the hashing performance has no effective regressions in my benchmarks, but the hashing is much better as a result. I’ve landed this code in GitHub already so anyone who uses my hashmap.h library will already benefit from this.

One thing I used previously was adding a special case when people were using the library with SSE4.2 enabled:

for (; i < len; i++) {
  crc32val = _mm_crc32_u8(crc32val, HASHMAP_CAST(unsigned char, s[i]));
}

This SSE4.2 path was 2.5x faster than the fallback (a lookup table).

I use a M1 Macbook Air, which as everyone knows uses the Arm architecture. There is no SSE4.2 on Arm, so can I make this faster on Arm too?

Well turns out I can! There is an extension for CRC32 on Arm too, that I can use.

for (; i < len; i++) {
  crc32val = __crc32b(crc32val, HASHMAP_CAST(unsigned char, s[i]));
}

And running this on my M1 Macbook Air this goes 2.5x too! I patted myself on the back and went to do a PR on GitHub. Then I realised - why am I only doing my CRC calculations on a byte-by-byte basis?

Turns out I just didn’t know how the CRC intrinsics actually worked. And there are CRC intrinsics that let us process multiple bytes at a time:

for (; (i + sizeof(uint64_t)) < len; i += sizeof(uint64_t)) {
  uint64_t next;
  memcpy(&next, &s[i], sizeof(next));
  crc32val = __crc32d(crc32val, next);
}

This will process eight bytes at a time instead of one, so how much faster does it go? 5.2x faster than the single-byte CRC intrinsic, and a whopping 13.3x faster than the original fallback code.

And the SSE4.2 code can process four bytes at a time with the intrinsics there too:

for (; (i + sizeof(unsigned)) < len; i += sizeof(unsigned)) {
  unsigned next;
  memcpy(&next, &s[i], sizeof(next));
  crc32val = _mm_crc32_u32(crc32val, next);
}

Adding this makes the X86 code go 3.1x faster than the single-byte CRC intrinsic, and a nice 7.7x than the original fallback code.

I also noticed that I only have power of 2 size for my hashmap, but I wasn’t relying on this when doing the hashmap calculations (I was doing an integer modulus, where I could just use an and-mask instead!). Doing this made the hashing calculations 1.01x faster, but every little helps.

All the code has now landed in the library and any users will be able to make use of it already!

For those that do not know I have six main libraries that I maintain that are single header and work across C/C++:

• 🧪 utest.h - a single header unit testing framework that supports C and C++.
• ⏱️ ubench.h - a single header unit benchmarking framework that supports C and C++.
• 📚 utf8.h - a single header UTF-8 string implementation supporting header for C and C++.
• 🗄️ json.h - a single header JSON parsing and writing header that supports JSON5 and simplified-JSON, and works with C and C++.
• 🐜 subprocess.h - a single header process spawning, joining, and interacting with library for C and C++.
• #️⃣ hashmap.h - a single header hashmap implementation for C and C++.

So what have I done in this new year cleanup?

utest.h

In my unit testing framework utest.h I’ve:

• Added support for building with MinGW. A user of my other library subprocess.h submitted a PR that adds MinGW support, and in that PR they had modified utest.h to also support MinGW. But with that code not merged into the main utest.h repository, any updates to the library would override the changes. So I added testing and support for MinGW in utest.h.
• Added Clang sanitizer support for utest.h to ensure I’m being as safe as I can be.
• A user had previously added support for exception handling in tests and I suddenly realised that this support assumed all users would be compiling with exceptions enabled in their code. So I made sure my library would work if the user had disabled exceptions explicitly.
• I also added support for comparing enum classes for C++11 and onwards. I used this as an opportunity to add explicit testing for C++14 and C++17 too.

ubench.h

In my benchmarking framework ubench.h I’ve added sanitizers. No bugs in the code were found, but its nice to have extra checks for future changes.

utf8.h

In my UTF-8 string header utf8.h I’ve added sanitizers. I found one wee bug in my unit tests where a constexpr test was going out of bounds.

hashmap.h

In my single header hashmap library hashmap.h I’ve also added sanitizers. No bugs in the code were found.

Let’s be honest - Twitter wasn’t exactly ideal for a pretty long time. The sheer amount of ads that were pushed into the feed even long before Space Karen bought the birdshite wasn’t great. I had already blocked thousands of accounts that were pushing promoted tweets into my feed, and had to fight the UI that would constantly try and move me back to the never-ending algorithmic feed that all these advert-centric businesses so love as it eeks out some percentage more time where you eyeballs are glued to their content.

Third-party clients always had an air of Wile E Coyote about them - they were off a cliff but hadn’t realised that gravity was going to kick in any second now. This is no slight against all the awesome developers of third-party birdshite clients - but it always felt like the birdshite were two steps away from yanking the ground from under the developers feet. The Talk Show podcast had a great dive into this that I recommend about Twitterific’s journey in this regard, worth a listen.

But the real kicker is just the fragmentation and loss of community on there. The birdshite was for a long time my view into the games industry. Hell I can probably say with all honesty that my career so far is a direct result of my ability to reach a community of fellow bit-twiddlers, mathooligans, and compiler tinkerers. Who would know a wee plonker like myself who lives as remote as I do in the Isle of Skye without some sort of social place that people like me could coinhabit and share ideas.

With the fragmentation of the community on the birdshite something has been broken there. Even with all the will in the world I do not think there is a way to kintsugi the birdshite back into what it was. Space Karen made a bunch of people jump ship. Third-party clients being banned has made the ship sink. I logged into the birdshite and within five clicks around I got asked to join the subscription service twice. They ‘forgot’ that I didn’t want the algorithmic feed once already. At most every five tweets there is an ad. It’s hellish.

Like a lot of people from my community I’ve landed on Mastodon. It’s quieter, more peaceful, and more pleasant. Not everyone has made it over there, and that obviously makes me sad for the loss of insight and interaction with my friends lives. But Mastodon gives me what I got from the birdshite all those years ago without all the garbage fires that now pollute the place.

So you can find me at https://mastodon.gamedev.place/@neilhenning, where I’ll be posting the usual mix of dog photos, coding funtimes, and the occassional link out to my blog.

I won’t be deleting my birdshite account - not for any loyalty or desire to return, but the advice from people leaving is that if you leave someone or some bot will hoover up your account handle and likely impersonate you. So I’ll just let the account lie in limbo.

Today was my first day, so obviously this is the day that I feel the most like an impostor! I’ve done this a few times now (jumping contract to contract when I worked at Codeplay, and then moving to AMD, Unity, and now Epic), and the feeling always passes once you become familiar with the way things are done at the new gig. The team have all been amazingly nice at answering my dumb questions so far, and I’m excited to get my teeth into solving some problems soon.

There was three different approaches that I looked at with this chart.

zfp

One thing that was suggested was zfp which claims to compress floating-points really well. I really disliked the API to get this to work - really hard to use. Also it doesn’t have a streaming API, so you need to declare up front how many floats are in it. Given that I’m compressing 16GB worth of floating point numbers, I didn’t want to load up all 16GB of them at one go! So the approach I used was to batch them into 128MB regions, and compress these. Overall zfp achieves a 1.8x compression ratio against the raw data - which is 4x worse than the best compression we’ve achieved.

If we look at the compression ratio of each individual chunk in the chart below, we can see where we are losing out:

As we can see in the chart, the compression ratio of zfp is really good for the densely populated [0..1] range of floating-point numbers - getting a 6x compression ratio. It’s pretty terrible for the higher numbers though, hovering around 1.03x on average. Meaning basically no compression here at all. The higher inputs to cos will result in vastly different outputs, so its no suprising that the compression is worse up there.

fpzip

Another was fpzip. Similarly to zfp above, there is no streaming API and I thus need to batch the input data. I choose the same chunk size of 128MB.

Firstly - note the log scale for the chart. That’s because fpzip achieves a much much higher compression ratio for the numbers in the [0..1] range of floating point (almost a 14000x compression ratio, nice!). But it still suffers badly when the outputs from cos get wildly different - hovering around 1.09x on average. The compression ratio is better than zfp, with a 2.43x compression ratio overall compared to the raw data size, but it is still 2.98x worse than the best compression we have achieved.

Per-Bit Compression

The best approach suggested was what if instead of doing a single zstd for the entire data stream, what if we do 32 zstd compressions, one for each bit of each float in the input stream? The general idea from people who proposed this approach was that so many of the bits would be consistent across subsequent output numbers, zstd would be much better at considering each bit separately.

This turned out to be the best approach I tested - zstd at compression level 10 was 2.9x more compressed doing per-bit compressed streams, and at compression level 21 it is 1.33x more compressed. Pretty cool!

Strangely xor’ing the previous with the current and then doing the per-bit calculation resulted in a worse zstd compression, but delta-encoding the previous with the current gave us a slightly higher compression ratio.

Also of note - doing delta-encoding with the per-bit zfp compression at zstd level 10 was 1.06x more compressed than doing the next best full-stream compression of xor’ing and zstd at compression level 21. This is interesting because compressing at level 10 was many many times faster to compute than using level 21.

Other Approaches

There are for sure other approaches we could use here. Using the cos identity cos(x) = cos(-x) would cut the encoding range in half (as long as we correctly sign any NaNs produced). We could use a bit precise Payne-Hanek reduction so that we store only the [0..2π]range in the table. Food for thought (but I think I’ve scratched my itch with this problem for now!).

I fulfilled a lifelong ambition to work closely with some heroes of mine like Andreas Fredriksson, Daniel Collin, Cort Stratton, and many others, and I met some truly amazing people at Unity who I didn’t previously know (Lee Hammerton, Sebastian Schöner, Alex Thibodeau, Tim Jones, Alexandre Mutel, among many others).

Achievements of my Time at Unity

It’s quite hard to realise how much of a game changer Burst and HPC# has been for Unity and it’s users. 8x performance wins were considered low when using Burst - I frequently remember people being amazed when they refactored for the constraints of HPC# that they would see 40x performance gains.

When I was interviewing for Unity the pitch for Burst was that maybe a hundred jobs would be written in HPC# and compiled with Burst - the absolute core logic of the game that needed to run as fast as possible. Burst was so successful and provided so much performance that I’ve seen shipping titles with over six thousand Bursted methods - games that could never ship without Burst.

After shipping six versions of Burst during my tenure here, I can really break down my achievements at Unity into three parts: performance, compile time, and startup time.

Performance

My main job at Unity was to extract every iota of performance out of our LLVM codepath with Burst. I have eleven years of experience with LLVM as a technology, so I brought all that knowledge to Burst.

• I added a custom alias-analysis that utilized all the knowledge from Unity’s job system to make code run fast
• I completely reworked our LLVM pass pipeline to make it 2x faster
• I reworked how we vectorize to ensure that vectorization succeeded in many many more cases than with stock LLVM
• I added over 15 custom LLVM passes for missing optimizations specific to Burst
• I reduced the final game executable size by 43% on average

I also added lots of ways users could tell the compiler about performance sensitive things:

• [NoAlias] / [AssumeRange] attributes
• Assume / Likely / Unlikely / Pause / Prefetch / umul128 / InterlockedAnd / InterlockedOr intrinsics
• helper intrinsics to query if the compiler performed optimizations like IsConstantExpression / ExpectAliased / ExpectNotAliased
• DisableSafetyChecks = true to let users bless certain Burst jobs as being safe, with a global option to ForceOn safety checks even for these jobs
• OptimizeFor for jobs to say what the optimization target was for the code (size, performance, just compile it fast)
• per assembly [BurstCompile] attributes that lets you specify the default options for an entire assembly
• F16C and FMA x86 intrinsics

Compile Time

When I started we thought 100 jobs might be Bursted, and as I said before I’ve seen titles with 6000 jobs today. That’s a 60x increase in content Burst has to compile, and so compile time became a real focus during my time here.

• cummulatively I improved the compile time with Burst by 26x in my time here - so while the amount of content we are dealing with has grown significantly, the compile time hasn’t grown linearly with it
• I spent a lot of time with big users of Burst ensuring that code compiled fast and was highly optimized. For instance with NetCode I added Burst compiler optimizations that made compile time faster by 39x

Startup Time

One often overlooked thing with optimizations is that the feel of the product is oftentimes more important than the raw power. Burst has required over the years a lot of domain-reload time hooks to compile things, setup shared-statics, make direct call work - all these things would result in a cost when the editor starts, and when code changes are made by the user.

• I made direct call 33x faster to process during domain reload and startup
• I made shared statics 13.3x faster when entering playmode
• I made Burst load 60x faster during initial startup
• And code changes 2x faster during domain reload

Overall these changes didn’t make the core work of Burst any faster (compilers still gotta compile), but the feeling of the editor being usable while Burst is background compiling is so important for the productivity of Unity’s users.

Highlights of my Time at Unity

Working with the Burst team, and a bunch of the people across DOTS more broadly was a highlight here. I learned a lot from a lot of people, and I hope I helped others too.

Being able to give people what they need from the compiler was so powerful. Hashing is slow? Here have a umul128 intrinsic to make that fast. You don’t understand what the compiler is doing? Here have an ExpectAliased intrinsic so you can compile-time enforce things that you - the user - knows , but want to be sure the compiler knows too.

But my biggest highlight is how closely we got to work with Unity’s customers, and help them succeed. I was routinely told that Burst was amazing to deal with from a customer perspective because we were super responsive on the forums and on bugs, and we fixed our user issues in a fast and timely manner.

Why Am I Moving On?

It was time for a change, for a new challenge.

This great talk by Brian Karis had a line from Brian which I’ll badly paraphrase to:

What are the biggest challenges in your field, and why are you not working on them?

This really hit home for me. I made Burst significantly better during my tenure at Unity, but I no longer felt that what I was doing day-in day-out was really working on the big compiler-focused challenges for the next ten years.

What Now?

I’ve got some time off before my next gig begins and so I have the time to breathe out and relax for a bit.

I’m super excited with what comes next - stay tuned for the details!

Having implemented my fair share of drivers, and worked on a few languages too, one thing that has always interested me is how can we check that math functions are correct. OpenCL C has an entire reference implementation of the math functions, but if I recall on platforms without 80-bit floats, testing that double precision results matched meant adding 0.5 ULP to ensure your result matched. Not ideal!

But taking all that aside, if you have reference floating-point functions that you want to compare against that do use the floating point units on your machine, there are some fun things that can cause discrepencies. Like if you happen to have any library that uses fast-math, it can set the floating point control registers.

The way I’ve seen math function checking done correctly is by comparing the results against a soft-float library that has predictable results on all platforms. But how can we be sure that two platforms with the same reference code produce the same results? Ideally we’d have to test two different platforms and compare each result matches. To do this without having to have two machines running concurrently, I wondered if it was possible to use MPFR to generate every possible result of a math function where we can test the entire range before the heat death of the universe? A 32-bit single input function like cos would be possible to test, but every input to a 64-bit cos would not, or a two input 32-bit function like pow would also not be fully testable.

The Raw Data

So each result value from cos for a 32-bit input is 32-bits and there are 4,294,967,296 resulting values - meaning the raw uncompressed data would be 16GB. Not ideal for sending between machines! This data is the output from every representable floating-point number, starting at 0x00000000 (positive zero), all the way to 0xffffffff (a negative NaN).

ZSTD To the Rescue?

My first thought was to use ZSTD on a high compression level to compress the results, which resulted in a 2.9GB result file, which is 5.5x smaller. So obviously better, but still a lot of data to ship around.

Ok, so compression alone is not enough to fix this, can we use some knowledge of the resulting data to help the compression achieve a better result?

Some Data Gathering

So lets look at the graph of cos:

We can see that in the range [-1..1] each next value should be close to the previous one. And if you didn’t know, over half the entire range of floating-point numbers lie in that range. So for half the input data the resulting values should be close, and my guess was these would be 1 or 2 bits out on average. So the next thing I did was gather some stats on the entire range of cos:

• The sign of the output flips between positive and negative in 20.36% of the consecutive outputs. Given that the high exponent inputs will result in wildly irregular outputs, my guess is these are primarily in these high exponent cases.
• The average exponent difference between outputs is 28.01. Again, I would expect this to be the high outputs that causes this.
• If we delta encode the mantissa (by subtracting the previous mantissa from the current one), we need 2.25 bytes on average to encode it (versus 3 bytes required in the floating-point standard).
• And if we xor the mantissa with the previous one, we need 2.30 bytes on average.

Xoring

So my first attempt was to just store the previous output from cos, and xor the current one with the previous:

let next = current ^ previous;

The result? 2.6GB when I xor then ZSTD the result, a 0.3GB saving against raw ZSTDing. Better but not good enough.

Delta Encoding

Next attempt was to just delta encode the entire number. Delta encoding just means we subtract the previous from the current, and store the ‘delta’ between them instead of the original.

let next = current - previous;

This took 3.8GB - ouch! My guess is that with the sign flipping 20% of the time, in those cases the delta will be huge between the two numbers and we’ll lose any benefit from the runs being close.

Varint Encoding the Mantissa

At this point - I was running out of ideas. But I remembered that varint encoding could be used to reduce the storage space for integers if the number is small enough. And given that for most of our range we should have relatively small differences in the mantissa between each encoded result and that we could encode 3 bytes down to on average 2.2 bytes based on the stats I collected earlier, would that help? I used the integer-encoding crate which automatically does zig-zag encoding for signed integers, and used that on the difference of the mantissas only (xor’ing everything else).

3.6GB of space taken. Damn!

Conclusion

I’m no compression expert, so there is probably some way to compress this better that I don’t know about. But even so - my original idea of just shipping these across platforms was a bad one it turns out. It’d have to get to below 500MB or thereabouts to make it feasible to ship this to multiple platforms (and you’d have to do it for every math function you wanted to test too!).

Editors suffix: The next thing I would try is based on this tweet, and use fpzip to try compress the numbers, thanks to Robin Green’s suggestion!

I’ve used a big internal project that amounts to about 2.5 million lines of LLVM IR after optimizations as the input for this test, and forced all LLVM versions from LLVM 10 (the oldest supported) to LLVM 15 (the latest supported).

As you can see from the chart - performance improvements in LLVM were going great until LLVM 13, and then LLVM 14 and now LLVM 15 have taken a pretty bad leap back towards the performance of LLVM 10. We use the new pass manager on LLVM versions 12 and above, so that is not the cause of this regression. We also do not use opaque pointers (our codebase relies on typed pointers for now, won’t be easy to unpick that!).

I’ve also heard people say that using clang-cl.exe will result in up to 30% better performance as compared to using the cl.exe shipped with Visual Studio.

And yup - its true. 13% improved performance just for using the clang-based compiler. I’ll have to do a full performance rundown and check that debuggability is still just as good with clang-cl.exe, but for that level of performance improvement it looks like it is worth moving to.

EDIT: I really should have provided the runtime performance across versions too:

As can be seen - its mostly in the noise, things got a little bit faster on LLVM 14, regressed slightly on LLVM 15, but overall not much between the versions for the significant differences in compile time.

To support this I’ve submitted a PR that adds a UTEST_SKIP, which can be used like:

UTEST(foo, bar) {
  UTEST_SKIP("Need to implement this test!");
}

To let you skip the remainder of the test. Then when executing the test will output:

[ RUN      ] foo.bar
   Skipped : 'Need to implement this test!'
[  SKIPPED ] foo.bar (39ns)

And also reports at the end of execution before any failing tests:

[==========] 860 test cases ran.
[  PASSED  ] 859 tests.
[  SKIPPED ] 1 tests, listed below:
[  SKIPPED ] foo.bar

As a reminder you can get my library at https://github.com/sheredom/utest.h, it works with mixed C/C++ content, and is blazingly fast.

TL;DR running less passes makes things compile faster.

The longer version requires a bit more nuance though.

LLVM’s PassManagerBuilder

LLVM has a default pass optimization pipeline that you create with the PassManagerBuilder class:

llvm::legacy::PassManager passManager;

llvm::PassManagerBuilder passManagerBuilder;

// We can set the optimization level (-O0 .. -O3)
passManagerBuilder.OptLevel = 3;

// Or the size level (-Os -Oz)
passManagerBuilder.SizeLevel = 0;

// We can add passes at certain extension points.
passManagerBuilder.addExtension(
  llvm::PassManagerBuilder::EP_Peephole,
  [](const llvm::PassManagerBuilder& passManagerBuilder,
     llvm::legacy::PassManagerBase& passManager)
  {
    passManager.add(burst_CreatePeepholePass());
  });

passManagerBuilder.populateModulePassManager(passManager);
passManager.run(*module);

The pros for using the PassManagerBuilder:

• The best way to start running optimizations on your code
• Can specify opt/size levels
• Allows you to inject your own passes via extensions
• Any LLVM update will have new passes ran at the correct times

And the cons:

• Pass pipeline optimized for C/C++ content (it’s used by Clang)
• Some passes will probably do nothing as a result for your language
• TBAA was unconditionally added to the pass pipeline - even though HPC# doesn’t have typed-based aliasiing guarantees

Overall I’d argue that using the default pass pipeline is what you should do, unless you are willing to sink a decent chunk of time with every major LLVM version bump to check all the new passes and work out if/where they should fit in your custom pass pipeline.

Reducing The Number of Passes

We wanted to be able to use most of what the PassManagerBuilder provided, but cut out the passes that did nothing on Burst.

To do this we extended the PassManager class and added our own custom add function that would let us do something before actually adding the pass into the pass pipeline.

struct MyPassManager final : public llvm::legacy::PassManager {
  void add(llvm::Pass* const pass) override {
    const char* name = pass->getPassName();
    llvm::errs() << count << “ ” << name << “\n”;
    count++;
    llvm::legacy::PassManager::add(pass);
  }

  unsigned count = 0;
} passManager;

llvm::PassManagerBuilder passManagerBuilder;
passManagerBuilder.populateModulePassManager(passManager);
printf(“Total %u\n”, passManager.count);

Which at the time I last ran this (in 2020, whatever LLVM version was active then!) would produce:

0 Target Transform Information
1 Target Pass Configuration
2 Simplify the CFG
...
Total: 84

So we had 84 passes being run in the pass pipeline.

Now with Burst we use golden file generation to ensure we do not regress on the optimizations that the compiler performs. This means that for every test in our test suite, we generate the assembly for it and compare it against the last known good assembly. We then verify any changes to this on each commit, to ensure we maintain good code quality always. There are over 100,000 tests that produce assembly into these gold files, which raised an interesting idea - I could selectively disable each pass, run all the tests, and if there are no gold file changes then the pass did nothing and could safely be removed.

To do this I used an environment variable DISABLE_PASS_N and skipped the passes by:

struct MyPassManager final : public llvm::legacy::PassManager {
  void add(llvm::Pass* const pass) override {
    if (count == skip) {
      count++;
      delete pass;
      return;
    }

    count++;
    llvm::legacy::PassManager::add(pass);
  }

  MyPassManager() {
    skip = atoi(getenv(“DISABLE_PASS_N”));
  }

  unsigned count = 0, skip = 0;
} passManager;

llvm::PassManagerBuilder passManagerBuilder;
passManagerBuilder.populateModulePassManager(passManager);

I could then re-run the test suite in a loop, running DISABLE_PASS_N=0 ./run_tests for each of the 84 passes, and if the test suite passed then the gold files weren’t changed.

How Many Could We Disable?

Turns out that 13 passes did nothing on Burst. And so to disable them we used this lovely hackeroo:

struct MyPassManager final : public llvm::legacy::PassManager {
  llvm::SmallDenseSet<llvm::AnalysisID, 16> skipSet;

  void add(llvm::Pass* const pass) override {
    if (skipSet.count(pass->getPassID()) != 0) {
      delete pass;
      return;
    }

    llvm::legacy::PassManager::add(pass);
  }

  MyPassManager() {
    // We have to create all the passes so we can get their IDs
    const llvm::Pass* const passes[] = {
      llvm::createTypeBasedAAWrapperPass(),
      llvm::createForceFunctionAttrsLegacyPass(),
      llvm::createInferFunctionAttrsLegacyPass(),
      llvm::createCallSiteSplittingPass(),
      llvm::createCalledValuePropagationPass(),
      llvm::createPruneEHPass(),
      llvm::createSpeculativeExecutionPass(),
      llvm::createLibCallsShrinkWrapPass(),
      llvm::createPGOMemOPSizeOptLegacyPass(),
      llvm::createEliminateAvailableExternallyPass(),
      llvm::createGlobalDCEPass(),
      llvm::createAlignmentFromAssumptionsPass(),
      llvm::createStripDeadPrototypesPass(),
    };
    
    for (const llvm::Pass* const pass : passes) {
      // Insert the ID into the skip set, then delete the pass!
      skipSet.insert(pass->getPassID());
      delete pass;
    }
  }
} passManager;

Note: Burst doesn’t have global variable support - you have to use SharedStatic’s to get access to global data that is shared between managed C# code and HPC# with Burst. So a bunch of the global optimization passes are expensive and useless for us.

How Much Faster Was Compilation

In the end this change made our test suite 1.64x faster, from 391s to 237s. This translates into happier Burst developers (we’re waiting on testing less), happier CI (it takes less time to run the tests on PRs), and happier Unity users (Burst compiled their code faster).

The Future

Since the fun filled days pre pandemic when I wrote this into the talk we’ve moved away from the above approach in Burst. This was because:

• We’ve moved to the new pass manager for nearly all of our compilations now, and the new pass manager doesn’t have the same fudgeability as the old one
• Even with the above reduction in passes, we decided that the cost to the team to maintain a custom pass pipeline was worth it to improve compile time even more. Our test suite has grown in size by 2.5x and we’ve reduced the run time of the test suite down to 150s - overall the compiler is about 4x faster than what I was going to present about in 2020

Even though this is soon to be dead tech with the death of the legacy LLVM pass manager, I quite liked that this tiny code change could result in such a big win for us and our users.

TL;DR - I’ve switched the library to use POSIX spawn.

There are various reasons fork can be slow, so how does my library fare? This is running on my MacBook Air (M1, 2020):

So it is faster, albeit not by a huge amount. Note this is a really synthetic example where my parent process doesn’t have a huge amount of state, but since using POSIX spawn is already faster I pulled the trigger and merged it.

So I’ve attempted (and most likely failed) to switch over to a light mode theme, used the lovely Alegreya font as the post font, and moved from the purplish theming to a more neutral bronzy/gold look. Reader mode should now also work.

Hopefully a bit better overall for my readers!

On the other hand Zig, with a strong ethos guided by Andrew Kelley, has this guiding light that there should be one way to do something in the language, and that is something that I really appreciate in language design.

Last year I attempted to do Advent of Code 2020 in Zig, but the language was just a little too fresh for me to get into. The documentation was basically non-existent, and even getting the tools and working how to use them was too confusing for me. On day one I gave up and switched to Rust instead. This year though I was determined to try the whole challenge in Zig, and what a difference a year has made to the language! The community is now massive, there are GitHub templates for Advent of Code to just get you coding, and the Zig documentation is so rich and detailed that I could pick up some of the basic concepts quite quickly.

So now that I’ve completed Advent of Code 2021, I thought I’d share the good and the bad about Zig, and some summary thoughts on the language.

Note: I’m assuming a base level of understanding about what Zig is here, there are plenty of guides on the language available elsewhere!

The Good

The best thing about Zig is that the language is small. There isn’t even a foreach for like structure and Andrew has stated ‘While loops work, why add another way?’ and I really appreciate this approach. It means I am not wondering about what tool to reach for when I want to do something, there is a single tool with a single use. Especially when learning a language (for myself and for anyone else that would want to pick it up) - brevity is key. I think Rust got lost in trying to nicely provide so much of what C++ badly provides that random users of the language looking at any arbitrary code written in Rust suffer for the sheer breadth of the language. Zig’s approach here meant I could read the standard library code and understand what it was doing (even with all the comptime type fun!).

Nullability is fun in Zig - the fact that optionality is built into the language with the ? prefix on types (so ?i32 is maybe-a-32-bit-integer) and that they have combined this with pointers so that you can assume that any pointer (*i32 for instance) isn’t null. This is great for the compiler, great for the optimizer, and I think also great for the user.

How things are brought in from the standard library or general foreign code is interesting:

const std = @import("std");
const print = std.debug.print;

There is a builtin compiler marco @import that does the heavy lifting of pulling in the code, and then you assign this into a const some_var variable. This is really neat because you could call that whatever you wanted (to avoid naming conflicts). Also when you want to pull in definitions from within an imported package you just use the same mechanism of assigning the package.with.a.thing.in-it into a constant variable. Most other languages have a using foo::bar::haz::baz; type mechanism for this, but having it use the same mechanism for a bunch of different things means that you don’t have to switch in your head to another tool. I hadn’t considered this language concept before using Zig, and its a very good idea!

The fact all containers take an allocator on intialization, and you can only get a heap pointer via an allocator is genius in Zig. Memory isn’t free, and allocations are not cheap, and so making getting at heap allocations harder by explicitly getting them through an allocator is a great thing.

Also the error mechanism in Zig is wonderful. Zig has this special prefix for a type (for example !u32 means ‘an error or a u32’) and you can cascade errors from deep in Zig code with the try statement. So var x = try foo(); means x is equal to the result of foo() unless there was an error in the result. If there was an error, return from the function with the error now. This meant that you don’t have the messy littering of if conditionals after every function that you typically get in C, but you also don’t have the complete disaster that is exceptions in C++/C#. Rust has a similar mechanism to this, but they use the clunkier Result<T, E>. While Zig has effectively added another thing for the frontend to handle by adding in a ! prefix on the types, the language is certainly nicer for it.

The Bad

There are a collection of things in Zig that I didn’t like. All languages have things that any random subset of users won’t like, so I am not saying Zig should change any of these or anything like that.

Initializing arrays is weird in Zig. Lets say you want to have a 0 initialized array, you declare it like [_]u8{0} ** 4 which means I want an array, of type u8, that is initialized to 0 and is 4 elements long. You get used to the syntax, but it’s not intuitive.

For loops are a bit strange too - you write for (items) |item| {}, which means you specify the container before the per-element variable. Mentally I think of for as for something in many_things {} and so in Zig I constantly had to write it wrong and then rewrite. Also you use the | character in Zig quite a lot, and while this may just be a problem with Apple UK keyboards, actually getting to the | character on my laptop was uncomfortable. When doing C/C++ or Rust, you use the | character much less and so the pain of writing the character was something I never noticed before. Jonathan Blow has gone on the record to say that with his language, Jai, he spent a lot of time working out how easy it would be to type common things, such that the more common an operation in the language, the easier it would be to type. That seems to be missing here (well at least for Apple UK keyboard layouts, I’d need to write Zig extensively on another layout to know whether this was a universal thing!).

Switch statements where you want to have multiple arguments resolve to the same code I wrote as a | b, whereas in Zig it is a, b. Nothing major with this, but I constantly tripped up on this.

Zig test was a bit clunky - you have to specify the file you want to test. So to test src/foo.zig, you’d do zig test src/foo.zig. I wanted something more like Rust’s cargo test that’d find all tests and run them. Maybe Zig does have this but I just didn’t find it?

And how you declare functions is a little strange. Like a function in a struct would be:

const foo = struct {
  pub fn bar() {}
};

Everything in Zig is const x = blah;, so why are functions not const bar = function() {};?

The Ugly

Zig is still a little raw in a few areas. Some compile errors are less than useful. For instance if you forgot to put !T on a return type, but were using try in the body of the function, the compiler error was very confusing. This is only really an issue for new Zig users (like I was when I first hit this), because you quickly learn that when the compiler spits out something less than useful and you are using try, check the return type first. Occassionally Zig would spit out 100’s of lines of notes after an error, giving me flashbacks to the C++ template mess errors you’d get.

The builtin compiler macros (that start with @) are a bit confusing. Some of them have a leading uppercase, others a lowercase, and I never did work out any pattern to them. Is it @as or @As? I still couldn’t tell you without looking at the manual.

The type system in Zig is loose in some ways and tight in others. If Zig can detect the type of the right hand side of a variable declaration, you don’t need an explicit type. But if you had something like var x = 0; you have to specify a type. It’d be nice for users (but obviously harder for the compiler team!) if the compiler would be able to deduce these types too.

But the worst bit about Zig at present is the standard library documentation is broken and non-existent. This is probably the one reason I wouldn’t recommend Zig more generally at present, because I resorted to looking at the source files of the standard library on GitHub to work out what I could do with what provided stuff in the standard library. I know there is a plan that with the new compiler frontend (written in Zig!) to fix this, so its just a time problem.

Conclusion

Overall my gut feeling is that Zig is about ready for developing with for people like myself (coders that don’t mind a bit of pain to a lot of benefit), but it is not quite ready for more general usage. Fixing the standard library documentation would be my biggest priority if I worked on Zig, because I think that is the only thing holding back general usage of the toolchain.

One nugget of knowledge I’ve worked out though - Zig is not a replacement for C. It is another replacement for C++. comptime which while amazingly powerful, already has echoes of the hard to reason about C++ template code or Rust generic mess, and there are still quite a few bits of syntatic sugar hiding the real cost of certain operations (like the try error handling, there is implicit branches everywhere when you use that).

This isn’t to say Zig is any lesser by being a much better C++ replacement rather than a C replacement in my estimation, infact I’d argue that aslong as Zig doesn’t fall into Rust’s trap of constantly adding yet more ways to do the same damn thing and making the language that little bit harder for new people to onboard with, then Zig once it hits a stable language around 1.0 will be my recommended tool going forward.

I really enjoyed doing Advent of Code in Zig, and I think I’ll be writing more software in Zig going forward. I’d highly recommend you check out the language and the community around the language are a great group of people that have been super helpful with my dumb onboarding questions.

But what about the opposite of hacking - what about spending some time consolidating what is currently there and working on the things that you really think will benefit the product over all, but haven’t had the time to work on. We did exactly that last week.

One thing that has been bugging me since LLVM 12 landed was that we were still using the legacy pass manager, which was the default optimization pipeline for LLVM for every release prior to LLVM 12. But in LLVM 12 they finally, after about 7 years since I first heard about the new pass manager, pulled the trigger and switched the default pipeline over. Since the legacy pass manager is still in the codebase (it is still actually being used at present to power the backend infrastructure), we didn’t have to use the new one with Burst. But the writing is clearly on the wall for the legacy pass manager, and I’ve heard and read claims that the new pass manager does some things better than the legacy one, maybe meaning faster compilation or better codegen.

So TL;DR I’ve added the new pass manager support to Burst, and here is a brain dump of how I went about it incase there is anyone else wanting to attempt the feat - hopefully you can avoid a few of the landmines I trod on!

Porting Over a Pass

The first thing is - how do you port a pass over from the legacy pass manager to the new one. Keep in mind that our code with Burst has to support LLVM 8..12 at present (for various platform reasons), so all my passes must work with the legacy pass manager and the new one concurrently, in the same codebase. I’d argue that you should do this approach anyway even if you want to abandon the legacy pass manager because:

• While porting both the legacy and the new pass manager code should work in the same codebase, letting you A/B test the changes for any regressions.
• Also lets you do performance profiling between the pass managers too.
• Even after landing you might yet find some corner case in the new pass manager that is broken and need to switch back to the old approach.

So let’s take one of our custom passes in Burst - a pass that does a much more extensive job of removing dead loops from the code. We need this pass because we found a bunch of cases where vectorization was blocked by a dead inner loop. The pass structure looked like:

struct DeadLoopRemovalPass final : public llvm::FunctionPass
{
    DeadLoopRemovalPass() : llvm::FunctionPass(ID) {}

    bool runOnFunction(llvm::Function& function) override;

    void getAnalysisUsage(llvm::AnalysisUsage& analysisUsage) const override;

    llvm::StringRef getPassName() const override { return "Burst Dead Loop Removal"; }

    static char ID;
};

char DeadLoopRemovalPass::ID;

So I first looked at how LLVM did the dual support for legacy/new pass managers. Their general overall approach is one of two ways:

• Either they have a wrapper around the new pass manager structure, providing the same code to the legacy pass manager.
• Or they have a shared class that both the legacy/new pass managers will use.

I decided to use the latter approach, for no reason other than it seemed easier to port over to.

The Implementation Struct

To do this I first removed the dependency of the struct above on the legacy llvm::FunctionPass, and made it a standalone struct:

struct DeadLoopRemoval final
{
    explicit DeadLoopRemoval() {}

    bool run(
        llvm::Function& function,
        std::function<llvm::LoopInfo& ()>,
        std::function<llvm::DominatorTree* ()>,
        std::function<llvm::PostDominatorTree* ()>) const;

    static llvm::StringRef getPassName() { return "Burst Dead Loop Removal"; }
};

Which entailed:

• Removing the inherit from llvm::FunctionPass.
• Removing the static char ID.
• Morphing runOnFunction -> run, and passing in callbacks to retrieve the analysis information I required to implement the pass. I changed the name from runOnFunction -> run just so that it gave me compile errors at the callsite as I was porting so I could find everything. Also note the override disappears from the function because we have no abstract virtual function to overload.
• Changed getPassName() to be a static function and removed the override. The new pass manager uses a static pass name retriever whereas the old used a virtual member function - so this allows us to support both.

The Legacy Wrapping Pass

Then before I even touched the new pass manager, I implemented the original pass for the legacy pass manager again, but making it just wrap the logic of the pass from the new DeadLoopRemoval struct:

struct DeadLoopRemovalLegacyPass final : public llvm::FunctionPass
{
    DeadLoopRemovalLegacyPass() : llvm::FunctionPass(ID) {}

    bool runOnFunction(llvm::Function& function) override
    {
        if (skipFunction(function))
        {
            return false;
        }

        auto getLI = [&]() -> llvm::LoopInfo&
        {
            return getAnalysis<llvm::LoopInfoWrapperPass>().getLoopInfo();
        };

        auto getDT = [&]() -> llvm::DominatorTree*
        {
            llvm::DominatorTreeWrapperPass* const dominatorTreeWrapperPass =
                getAnalysisIfAvailable<llvm::DominatorTreeWrapperPass>();

            return dominatorTreeWrapperPass ?
                &dominatorTreeWrapperPass->getDomTree() :
                nullptr;
        };

        auto getPDT = [&]() -> llvm::PostDominatorTree*
        {
            llvm::PostDominatorTreeWrapperPass* const postDominatorTreeWrapperPass =
                getAnalysisIfAvailable<llvm::PostDominatorTreeWrapperPass>();

            return postDominatorTreeWrapperPass ?
                &postDominatorTreeWrapperPass->getPostDomTree() :
                nullptr;
        };

        return pass.run(function, getLI, getDT, getPDT);
    }

    void getAnalysisUsage(llvm::AnalysisUsage& analysisUsage) const override
    {
        // We need loop analysis to replace our intrinsics.
        analysisUsage.addRequired<llvm::LoopInfoWrapperPass>();
        analysisUsage.addPreserved<llvm::LoopInfoWrapperPass>();

        // We will use the dominator tree, and preserve it during this pass. We
        // do this because recalculating the dominator tree after this pass was
        // much more expensive than just preserving it.
        analysisUsage.addPreserved<llvm::DominatorTreeWrapperPass>();
        analysisUsage.addPreserved<llvm::PostDominatorTreeWrapperPass>();
    }

    llvm::StringRef getPassName() const override { return DeadLoopRemoval::getPassName(); }

    static char ID;

private:
    DeadLoopRemoval pass;
};

char DeadLoopRemovalLegacyPass::ID;

So this code is pretty straight-forward, I started with a copy-paste of the original pass and then started hacking it to use the new struct underneath:

• I first added the DeadLoopRemoval pass member to the struct.
• Then I forwarded the getPassName() member virtual function to the static method in the implementing struct.
• For the runOnFunction body I had to query the analysis passes I required in the lambdas, and pass these to the pass.run method.

At this point I can test the ported pass with our existing test suite and the legacy pass manager, to ensure that worked before doing anything else. And it did! So next up, lets port it to the new pass manager.

The New Wrapping Pass

The new pass manager uses a mixin approach with the Curiously Recurring Template Pattern (CRTP, we’ll come back to this) to implement the structure. This means no virtuals, no overrides, but there is still a specific function you are expected to provide. For a function pass you implement llvm::PreservedAnalyses run(llvm::Function&, llvm::FunctionAnalysisManager&) for instance. Our new pass manager wrapping pass thus looks like:

struct DeadLoopRemovalPass final : public llvm::PassInfoMixin<DeadLoopRemovalPass>
{
    explicit DeadLoopRemovalPass() : pass() {}

    llvm::PreservedAnalyses run(llvm::Function& function, llvm::FunctionAnalysisManager& analysisManager)
    {
        auto getLI = [&]() -> llvm::LoopInfo&
        {
            return analysisManager.getResult<llvm::LoopAnalysis>(function);
        };

        auto getDT = [&]() -> llvm::DominatorTree*
        {
            return analysisManager.getCachedResult<llvm::DominatorTreeAnalysis>(function);
        };

        auto getPDT = [&]() -> llvm::PostDominatorTree*
        {
            return analysisManager.getCachedResult<llvm::PostDominatorTreeAnalysis>(function);
        };

        if (pass.run(function, getLI, getDT, getPDT))
        {
            llvm::PreservedAnalyses preserved;

            preserved.preserve<llvm::LoopAnalysis>();
            preserved.preserve<llvm::DominatorTreeAnalysis>();
            preserved.preserve<llvm::PostDominatorTreeAnalysis>();

            return preserved;
        }
        else
        {
            return llvm::PreservedAnalyses::all();
        }
    }

    static llvm::StringRef name() { return DeadLoopRemoval::getPassName(); }

private:
    const DeadLoopRemoval pass;
};

Digging into this:

• Our pass derives from llvm::PassInfoMixin<DeadLoopRemovalPass>, taking itself as the template argument (the CRTP paradigm).
• We have an implementation of the run method like I stated above.
• We use the explicit analysisManager to get the analysis passes we require, rather than using the getAnalysis that the old pass manager used to get at these.
• We do not return a true/false value from our run method like the legacy pass manager runOnFunction. True meant ‘something changed, analysis passes might be invalid!’. For the new pass manager we explicitly return an llvm::PreservedAnalyses which we add what passes we preserved into.
• Also note the static llvm::StringRef name() method - this is how you set the pass name of the pass.

And that’s it! We now have our pass working both with the new and legacy pass managers. I had to do this same approach with all 20 of our custom LLVM passes, quite a lot of typing, but meant I could be sure at each step it was doing the right thing.

Porting Our Custom Pass Pipeline

We long since abandoned the default LLVM pass pipeline for a custom one.

As an aside my COVID cancelled LLVM talk was going to explain how we cut the cost of using llvm::PassManagerBuilder’s default optimization pass structure by 40% by cutting out the cruft from it while still using the builder, but even that wasn’t enough control for us to get the abosolute best compile time possible.

So our legacy pass manager usage involved us calling each of the LLVM passes (a mix of the standard ones and our custom ones, hand crafted for best codegen and compile time):

mpm.add(burst_CreateEarlyMemOptsLegacyPass());

if (optLevel >= 1)
{
    mpm.add(llvm::createCFGSimplificationPass());
    mpm.add(llvm::createSROAPass());
    mpm.add(llvm::createEarlyCSEPass());
}

if (optLevel >= 3)
{
    mpm.add(llvm::createIPSCCPPass());
    mpm.add(llvm::createGlobalOptimizerPass());
}

This is just a sample from the pass pipeline, but it gives you an idea I hope.

For the new pass pipeline I did the obvious (and wrong) thing of copying that and changing it to the new pass manager:

burst_AddEarlyMemOptsPass(mpm);

if (optLevel >= 1)
{
    mpm.addPass(llvm::SimplifyCFGPass());
    mpm.addPass(llvm::SROA());
    mpm.addPass(llvm::EarlyCSEPass());
}

if (optLevel == 3)
{
    mpm.addPass(llvm::IPSCCPPass());
    mpm.addPass(llvm::GlobalOptPass());
}

I did this for our entire pass pipeline, Visual Studio showed no errors! I hit compile, and explosions. It took me longer than I’d care to admit to work out what the issue was - the new pass manager requires you to add a pass to the correct pass manager type.

With the legacy pass manager everything derived from llvm::Pass, including llvm::ModulePass and llvm::FunctionPass. Behind the scenes the pass manager would work out that you were using a function or module pass, and behind your back create a manager of the correct type (so if you used a function pass, it’d make a function pass manager for you). This is a bad design, and so with the new pass managers they changed it.

Remember the CRTP thing I mentioned above? This is where it really hurts developers - Visual Studio didn’t provide me with compile errors until I actually did a full compile, because the CRTP is basically an interface that requires all the templates to be instantiated with their implementing types, before it can work out you’ve done the wrong thing. This shows up two bad bits of C++ - no interfaces (C# interfaces are so very nice for exactly this kind of issue), and templates nearly always result in horrific error messages. Very user hostile!

Because its been an easy 6 years since I last looked at the new pass manager (I honestly figured it would just die after such a long stint in the codebase but not being used by Clang), I took a look at the llvm::PassBuilder which helped me work out my mistake:

{
    llvm::FunctionPassManager fpm;

    burst_AddEarlyMemOptsPass(fpm);

    if (optLevel >= 1)
    {
        fpm.addPass(llvm::SimplifyCFGPass());
        fpm.addPass(llvm::SROA());
        fpm.addPass(llvm::EarlyCSEPass());
    }

    mpm.addPass(llvm::createModuleToFunctionPassAdaptor(std::move(fpm)));
}

if (optLevel == 3)
{
    mpm.addPass(llvm::IPSCCPPass());
    mpm.addPass(llvm::GlobalOptPass());
}

So you can see I had to explicitly make an llvm::FunctionPassManager, add the function passes to that, and then call llvm::createModuleToFunctionPassAdaptor to provide this function pass to the surrounding module pass manager. A bit verbose, and horrific template mess aside, a much much better design overall. This makes it really obvious now when you are mixing function pass managers with module passes in a poor way, because you have to re-create the function pass manager each time.

More explicit is always a win in my book.

Another thing that got me was that you have to manually register all the analysis passes you need with the analysis managers explicitly:

llvm::PassBuilder& pb; // an already created pass builder

llvm::LoopAnalysisManager lam;
pb.registerLoopAnalyses(lam);

llvm::FunctionAnalysisManager fam;

fam.registerPass([&] { return llvm::TargetIRAnalysis(targetMachine->LLVM->getTargetIRAnalysis()); });
fam.registerPass([&] { return llvm::TargetLibraryAnalysis(targetLibraryInfo); });

llvm::AAManager aam;

aam.registerFunctionAnalysis<llvm::BasicAA>();

if (optLevel == 3)
{
    aam.registerFunctionAnalysis<llvm::ScopedNoAliasAA>();
}

if (optLevel >= 2)
{
    burst_AddExtraAliasAnalysisPass(fam, aam);
}

fam.registerPass([aam] { return std::move(aam); });

pb.registerFunctionAnalyses(fam);

llvm::CGSCCAnalysisManager cgam;
pb.registerCGSCCAnalyses(cgam);

llvm::ModuleAnalysisManager mam;

pb.registerModuleAnalyses(mam);

pb.crossRegisterProxies(lam, fam, cgam, mam);

Here I created an llvm::PassBuilder not to actually build the passes, but to build the analysis managers so that the analysis passes I required are available in the managers. There are some quirks:

• To provide an analysis to the analysis manager you pass in a callback that creates the pass. Note sure why this is needed (maybe have to delay the initialization?), but it confused me at the beginning!
• To register our custom alias analysis we had to add that to a llvm::AAManager, and then register that with our function analysis manager.
• You need to call crossRegisterProxies on the pass builder to make sure all the analysis managers know about each each (in the case that, say, a function requires a module-level analysis).

The Curious Case of the Optimized optnone Functions

LLVM IR has a function level attribute optnone that disables optimization for a specific function. This is really useful for developers when they suspect there is a bug in a specific function, so they can just disable the optimizations there and work out what is going wrong more easily. When I ran our tests with the new pass manager, I noticed that our optimize none tests were… being optimized?! The functions have optnone on them, how could the pass manager be allowing them to be optimized?

In the legacy pass manager each pass has a skipFunction call on it that tells you if you should skip running on that function. If optnone was set, you’d skip that function. The new pass manager of course doesn’t use this mechanism and does something entirely different. Obviously.

llvm::PassInstrumentationCallbacks passInstrumentationCallbacks;

llvm::OptNoneInstrumentation optNoneInstrumentation(false);

optNoneInstrumentation.registerCallbacks(passInstrumentationCallbacks);

llvm::PassBuilder pb(targetMachine->LLVM, llvm::PipelineTuningOptions(), llvm::None, &passInstrumentationCallbacks);

You have to register an llvm::OptNoneInstrumentation with a llvm::PassInstrumentationCallbacks, which is then registered in the llvm::PassBuilder constructor. I need to lie down. This then ensures that any function with optnone won’t be optimized through some arcane process I don’t really understand.

The Performance

I’ve long been told that the new pass manager will result in better performance because of how analysis passes are preserved better. So I was excited to run our tests for the first time.

First up the compile time performance:

I’ve got to admit - after first running this I was a little disappointed - the new pass manager was meant to magically make everything faster! Then after I calmed down a bit I realised - this is fine. Since we’ve already got our own custom curated set of passes that are ordered specifically to not trash analysis passes unnecessarily, it stands to reason that we wouldn’t see any magical uplift with the new pass manager. The fact we are seeing numbers that are effectively within the signal-to-noise of each other means that when we do switch to the new pass manager by default, things won’t instantly go slower as a result. That’s a win.

Now for the binary size:

Nothing really to write home about here - slightly higher with -O0 and -O1, and slightly lower with -O2 and -O3. I did a diff of the assembly and its just the usual slight reordering of passes - mostly the same instructions just sometimes in a different order. Again, this is good though - no regressions means we can safely make the switch.

One Last Pain Point

One thing I generally like to do on Windows is use a RelWithDebInfo compiled LLVM with a Debug or Release compiled our code. This gets me the best of both worlds in terms of some debuggability while not running dreadfully slow. For some reason while this has always worked with the legacy pass manager, someone in the LLVM community has added a bunch of:

#ifndef NDEBUG
// Some extra struct state!
#endif

To some of the new pass manager structs.

They are doing this so that on debug builds they can carry around some extra state in the structs to aid debuggability, but this means that the struct layout is now different between Debug and Release builds. This is absolutely awful, and means my RelWithDebInfo compiled approach no longer works. This took me 4 hours to narrow down - I just assumed some of the template mess in the new pass manager was somehow going wrong, it was only when I started printing the size of structs that my head hit the desk. I hope someone can remove these - I’d rather carry around the extra state always and not use it, than have structs change size depending on the optimization level.

Conclusion

This work took me a week to do for our reasonably customized LLVM pass pipeline and custom passes. It is a lot of boiler plate for something that effectively does the same as before, but it does set us up more cleanly for the day that LLVM actually removes the legacy pass manager from the codebase entirely, meaning that when that day comes we aren’t floundering. The new pass manager has some nice features, it is just a shame that C++ is so broken as a language that it takes walking through a wall of template errors to work out what went wrong.

And before anyone @’s me - I know concepts will in theory fix this, just colour me highly skeptical on that count.

Until next time!

The general way to do this in Rust is to use an environment variable, and then in a build.rs you can grab all the paths to the LLVM libraries for linking in.

For instance the llvm-sys crate has an optional feature no-llvm-linking that lets you control where LLVM is sourced from. I use this in a project I’m cooking up to control how LLVM is brought into my project. At present my workflow is something like:

LLVM_INSTALL_DIR=llvm-install cargo test
cargo clippy

If you can see the problem already - when I run other commands on auto-pilot like running cargo clippy, and I do not specify the LLVM_INSTALL_DIR, the command fails to run because it requires LLVM_INSTALL_DIR to be specified.

Coming from a primarily CMake based workflow with my day-to-day job working with LLVM, this completely does my head in. CMake has a way to cache variables and then subsequent commands re-use the cached value. So I got thinking - why can’t I do something similar in Rust?

I’d like to introduce a simple, dumb, but useful little crate - envcache. This crate lets you cache any environment variables you want so they can be used across cargo commands. For example, in a build.rs:

extern crate envcache;

use envcache::EnvCache;

fn main() {
    let mut envcache = EnvCache::new();
    envcache.cache("LLVM_INSTALL_DIR");
}

By doing this simple thing above, it’ll let me specify LLVM_INSTALL_DIR once, and then have this cached value used thereafter for subsequent commands.

This works by saving a file in the OUT_DIR that you can write to with build.rs - envcache.config. Crucially this means that if you run cargo clean it’ll wipe out the cache, and you’ll need to re-specify the environment variable. But it does mean you have to specify the environment variable much less in general development which I find to be much nicer.

The source code is available here, and I hope this is useful to some of you.

• Started a job at AMD in the GPU compiler team.
• Left a job at AMD for Unity in the Burst compiler team.
• Wrapped up my involvement as a creator of Vulkan and SPIR-V.
• Learnt the C# and Rust programming languages.

You couple this with the absolute trainwreck that was 2020 (COVID yay) and now it looks like 2021 might be mostly a write-off too, and the fact that my dog Benji has progressively worsening epilepsy that is horrific to witness - it is safe to say that I’m probably at the highest level of stress I’ve ever been under.

I do realise that I’m in a very fortunate position nonetheless though - I have a good and stable job at Unity, I live on a remote island with a big garden and lots of places I can run while maintaining an easy social distancing. I know a lot of people have it much worse - very much a first-world-problem level of stress.

But one thing that has been nagging me for, and I admit with a good dose of shame, this entire time was that the bug count on my open source C libraries had creeped up to a level of real annoyance.

These issues could be broke up into a few categories:

• Later Clang/GCC would add new warnings that my libraries would fail on (I like to ensure that all the libraries can compile with the full shebang of -Werror -Wall -Wextra).
• I didn’t support clang-cl.exe on Windows at all (my libraries assumed that you were using any of the Visual Studio compilers from 2008 to 2019).
• There were a raft of general bugs people had found.
• And a few quality of life feature requests thrown in there too.

So in this early part of 2021 I’ve went on a pretty severe gutting of my libraries to fix the bugs, implement the features, test the compilers that I didn’t previously test, and generally just get my issue list as close to zero as possible.

The libraries that I put all the effort into are the following six:

• 🧪 utest.h - a single header unit testing framework that supports C and C++.
• ⏱️ ubench.h - a single header unit benchmarking framework that supports C and C++.
• 📚 utf8.h - a single header UTF-8 string supporting header for C and C++.
• 🗄️ json.h - a single header JSON parsing and writing header that supports JSON5 and simplified-JSON, and works with C and C++.
• 🐜 subprocess.h - a single header process spawning, joining, and interacting with library for C and C++.
• #️⃣ hashmap.h - a single header hashmap implementation for C and C++.

GitHub Actions

To fix that I wasn’t testing the latest Clang and GCC I decided to finally pull the trigger on using GitHub Actions instead of my old Travis pipelines. Travis was starting to have some integration issues (I don’t know the details, they just suddenly stopped appearing on the PRs when I pushed changes), and since I knew I wanted to end up on GitHub Actions anyway I used this as the reason to just do it.

GitHub Actions was a little daunting at first, but once you get into it the yml format is very powerful. For example here is the workflow of my subprocess.h library.

The most interesting parts are how I upgraded to the latest clang/gcc:

- name: Setup dependencies
  if: startsWith(matrix.os, 'ubuntu')
  run: sudo apt-get install -y gcc-10 g++-10 clang-10

And then how I used these configurations:

- name: Configure CMake with GCC
  shell: bash
  if: matrix.compiler == 'gcc'
  working-directory: ${{github.workspace}}/build
  run: cmake $GITHUB_WORKSPACE/test -DCMAKE_BUILD_TYPE=${{ matrix.type }} -DCMAKE_C_COMPILER=gcc-10 -DCMAKE_CXX_COMPILER=g++-10

- name: Configure CMake with Clang (Ubuntu)
  shell: bash
  if: (matrix.compiler == 'clang') && startsWith(matrix.os, 'ubuntu')
  working-directory: ${{github.workspace}}/build
  run: cmake $GITHUB_WORKSPACE/test -DCMAKE_BUILD_TYPE=${{ matrix.type }} -DCMAKE_C_COMPILER=clang-10 -DCMAKE_CXX_COMPILER=clang++-10

- name: Configure CMake with Clang (Windows)
  shell: bash
  if: (matrix.compiler == 'clang') && startsWith(matrix.os, 'windows')
  working-directory: ${{github.workspace}}/build
  run: cmake $GITHUB_WORKSPACE/test -DCMAKE_BUILD_TYPE=${{ matrix.type }} -T ClangCL

My build matrix is pretty extensive too:

strategy:
  matrix:
    os: [ubuntu-latest, windows-latest, macOS-latest]
    type: [Debug, RelWithDebInfo, MinSizeRel, Release]
    compiler: [default, clang, gcc]
    exclude:
      - {os: "macOS-latest", compiler: "clang"}
      - {os: "windows-latest", compiler: "gcc"}
      - {os: "macOS-latest", compiler: "gcc"}
      - {os: "ubuntu-latest", compiler: "default"}
      - {os: "ubuntu-latest", compiler: "default"}
runs-on: ${{ matrix.os }}

And all in all this lets me build for the three desktop platforms, using five different compiler toolchains.

Even though this let me ditch Travis I couldn’t ditch appveyor in the process. I would have liked to have a single CI integration as that’d have made maintaining everything easier, but GitHub Actions does not allow you to test with older Visual Studio installations.

Quite a few of my users love the older Visual Studio’s and won’t upgrade under any circumstances, and so I’ve kept appveyor around to ensure this path is tested.

utest.h

In utest.h I removed the #include <winbase.h> entirely. This meant that the huge headers that were pulled in as part of including utest.h on windows were no longer required, which sped up compile time on the Windows platform.

utf8.h

In utf8.h I added some new helper functions utf8makevalid and utf8rcodepoint - allowing you to sanitize a potentially invalid UTF-8 string by replacing malformed codepoints with an ASCII replacement, and allowing you to reverse iterate through the string.

I also fixed a long standing bug whereby the Greek Capital Theta symbol ‘ϴ’ does not have a uniform mapping from upper to lower case, or vice-versa. This means that if you did the dance of doing upper -> lower -> upper you could have a different string than you started with (but the fix means I correctly handle that letter’s encoding).

json.h

In json.h I added all the fun JSON tests from the JSONTestSuite project, and fixed any and all the corner case bugs this uncovered. Mostly I was incorrectly finding an end-of-file and assuming the JSON to be valid (without checking that strings/objects/arrays were actually closed correctly). I also took the opportunity to add clang sanitizer tests as since json.h uses a single allocation to handle the entire JSON DOM, there was a greater chance of trashing problems. Luckily there are zero issues with the library under its current testing load - nice!

I also added a new function json_extract_value which lets you pull out a value from a JSON DOM into a new allocation. This is useful if you want to get a bit of a DOM and do something with just that (you could free the original allocation for instance).

subprocess.h

In subprocess.h I fixed two fun issues:

• On Linux if you did not specify the subprocess_option_inherit_environment option then you had to have a qualified path to the executable (either a full path with the starting ‘/’, or a relative path like ‘./’).
• I also added a new entry-point subprocess_create_ex that lets users specify a custom environment to spawn the subprocess with.

hashmap.h

In hashmap.h I fixed a bug where if you had three entries that mapped to the same hash, and therefore used the linear probe part of hashing to find a location. If you removed the middle entry, and then tried to reinsert the third, it’d wrongly make two copies of that entry in the hash table. The test for this shows the problem nicely:

struct hashmap_s hashmap;

int x = 42;
int y = 13;
int z = -53;

ASSERT_EQ(0, hashmap_create(4, &hashmap));

// These all hash to the same value.
ASSERT_EQ(0, hashmap_put(&hashmap, "000", 3, &x));
ASSERT_EQ(0, hashmap_put(&hashmap, "002", 3, &y));
ASSERT_EQ(0, hashmap_put(&hashmap, "007", 3, &z));
ASSERT_EQ(3u, hashmap_num_entries(&hashmap));

// Now we remove the middle value.
ASSERT_EQ(0, hashmap_remove(&hashmap, "002", 3));
ASSERT_EQ(2u, hashmap_num_entries(&hashmap));

// And now attempt to insert the last value again. There was a bug where this
// would insert a new entry incorrectly instead of resolving to the previous
// entry.
ASSERT_EQ(0, hashmap_put(&hashmap, "007", 3, &z));
ASSERT_EQ(2u, hashmap_num_entries(&hashmap));

hashmap_destroy(&hashmap);

I also added a new function hashmap_remove_and_return_key which lets you remove and item and also return the key. Remember that hashmap.h doesn’t copy the memory backing the key, so if you had to allocate memory for the key and wanted to get the key back so that you could free this memory - this is the function for you!

GitHub Sponsorship

First off - I don’t really need the money from GitHub sponsorship. I’ve got a good job at Unity and my personal projects in C/C++ and Rust are simple labours of love.

But a few people have asked if I would add a sponsorship option to my GitHub anyway, because they like my libraries and use them.

So I’ve added a sponsorship option to my GitHub for anyone that wants to help me justify the time to keep my libraries updated, bug free, and great.

• While brushing my teeth twice a day I know I have a full two minutes where I can make use of my spare hand to do things. Like I might pack the dishwasher, lock the doors, turn off lights, put stuff into the recycling - anything that means I’ve double buffered another activity while I’m stuck brushing.
• I got really obsessed with the number of trips from the dishwasher to put things away, versus the time taken to load up on extra dishes for a lower number of trips. I kid you not I’ve experimented and mostly realised that (if optimizing for time taken as your primary metric) the number of trips is much less important than the time taken to load/unload dishes in an individual trip.

One other area of my life where I’ve adopted this philosophy is the structure of my workday. I thought it might be interesting for people to see if there is any tips in how I lay out my day that they could adopt to optimize their own experiences also.

Booting Up

The first step in any day is the process of booting up. I follow the same routine each morning:

• I wake up.
• I use the facilities.
• I weigh myself.
• I take my dog outside so he can relieve himself.
• I put on my running gear.
• I make a coffee. Caffeinated. I only have a single with-caffeine coffee each day as I’ve experimented and found it to be the optimal balance of getting that ‘wake up’ feeling versus the affect it can have on sleeping at the other end of the day.

Once I’ve performed these most basic tasks in a mostly pre-coffee zombie state, I’m ready to load up on the problems of the day.

Dealing with the Urgents

Once I’m at the computer I do a full parse of slack / work email, answering any of the easy questions and dealing with the easier tasks.

Then I tend to filter through to see if there are any urgent bugs that I need to take care of. These bugs are generally of the ‘stop the world’ kind - EG. I’d only drop whatever main feature I’m working on for a bug if it is truly some sort of regression / ship stopper. My number one aim for any of these bugs is to reproduce them (not always easy) and then give the user a temporary workaround.

Because of the natural cadence of software releases it means that even if I rush to fix a users issue, there is still some time lag between me fixing it and them being able to get access to that fix (while everything is tested for release). Workarounds are a key part to unblock a user while the real fix is deployed.

Urgent Bug or Feature

Once I’ve loaded up on a specific urgent bug to fix, or a feature I am developing, I spend thirty minutes or so acclimatizing myself to the issue.

If it’s a bug - can I reproduce it? Can I check our logs? Can I use a debug build of our Burst compiler to give me more information? Generally at this stage I’m only trying to work out what is the cause not what the fix is.

If it’s a feature - I’m just paging back in what I was doing the day before and trying to get started up again. I have long followed a philosophy of lacing #error’s throughout my code before I leave it for the night so that the next day it won’t compile but I’ve left some breadcrumbs for where my thinking was the night before. Again - all I’m trying to do here is page back in all the problems I was trying to solve at this stage, not do any actual work.

Run Run Run

Then I go for a run. I grab my dog (he runs with me every day!) grab my headphones, and listen to any of my favourite podcasts while I’m exercising. I do at least a three mile run every day, and try to run at least thirty miles a week. What is important at this stage is to get into the monotony of the exercise, don’t think about work, enjoy the podcast and get a sweat on.

There is now a pretty hefty set of scientific evidence that states that the right kind of distraction from a problem actually helps you solve it (for instance here is a BBC article about just that. The TL;DR is that as long as you have some monotonous task like sorting things (sorting lego blocks by colour, putting the dishes away, hanging out washing on a washing line, etc) or more broadly where your brain can mostly power off from any in-focus intensive task, your brain kicks off an asynchronous process in your head to actually solve the problem while you are doing the thing. This is why you hear so many stories of people solving problems in the shower, or waking up with a solution - you are using the same technique.

I’ve found myself that by loading up on the problems and then immediately context switching to exercise, I generally solve them better and faster. I could sit at the screen for hours and get no-where, or I could go for an hour run and solve it in the thirty or so minutes after I sit back down. Nearly every time I engage the crunch like behaviour of trying to power through to a solution it sets me back on actually solving it - so this technique is something I’d definitely recommend trying.

The Rest of My Day

I get about four or five good hours in a day of active problem solving coding. It might be more or less for other people, but aslong as I can get some peace, have my favourite music blaring, and can distract myself occassionally to make a decaf coffee or throw the frisbee to Benji, I’ll get a solid block of good work done in an average day.

Since the my team is split mostly between EU / US, I generally have most of my big meetings in my afternoon. So by that time I’m already pretty tired from the solid work block, and ready to engage in the meeting instead.

When I’m done for the night? I’m done. I work my eight hours a day, and walk away from the machine. I like to cook so I generally make dinner as the bridge between work Neil and home Neil (even though they inhabit the same house!).

This year I’ve changed that, and I’ve pushed all my solutions to my AOC 2020 repository on GitHub. I thought that now that I’ve completed the challenge, I’d do a review mostly of how I used Rust to complete the challenges and what I found.

Rust Lessons Learned

Iterators

I really got more comfortable with iterator’s and began to reach for map, fold, filter, etc. the more I went into the puzzles. To begin with I stuck to what effectively was a ‘safe’ C-like coding experience - using mutable variables and loops. The more I went into the challenge I began to use more Rust things.

The one major downside I did find to iterators is that if they go wrong, you start to want to debug the body of them and then the closure can become frustrating. You either introduce a scope into the closure and bung the stuff in there, or have a seperate method instead of a closure to aid in debugging. Even still, I did feel many times that having a dumb C-like loop with mutable variables ended up being a nicer experience when things went wrong.

Shadowing Isn’t Always Evil

I’ve always felt variable shadowing was an evil that should be stopped. Lets take C for example here:

int foo(int a) {
  if (a < 42) {
    int a = 13;
  }

  return a;
}

I really dislike in C that it lets you shadow a local variable with a previously accessible variable - it just introduces the kind of problems like you see above where the user probably wanted to return 13 if a < 42, but they declared a new variable instead. In C-like languages I like to force shadowing to be an error as a result.

I came into Rust with this same view - shadowing is clearly bad language design! Except that Rust is uniquely placed to make shadowing sometimes nice:

fn foo(a: i32) -> i32 {
  let mut a = a;

  if a < 42 {
    a = 13;
  }

  let a = a;

  a
}

In this example we can declare a variable mutable, shadow it with another variable, and then assign the original variable to the new variable. All in a single step! This lets us temporarily make the a we are using mutable to do things with it. What’s even cooler is we can then do the same in reverse after we have decided a should no longer be mutable.

Before the Rust pedants come out the woodwork, the above should really be something more like:

fn foo(a: i32) -> i32 {
  let a = if a < 42 {
    13
  } else {
    a
  }

  a
}

So we can avoid making a mutable at all (but I couldn’t evangelise my point without the much uglier code before!).

It’s Easy to Use Crates (maybe too easy…)

Using crates is ridiculously easy. The number of times I added a crate just to try solving a problem using it, realised I didn’t need it so I then removed it, and moved on with my solution - was insane. I think that coming from C/C++ languages where I am most fluent has made me supremely dependency adverse, but with Rust it’s so trivial to just try a crate that you get sucked in.

I still think its a useful life skill to always attempt to avoid a dependency where you might not need one, but if you are to use dependencies then Rust is definitely what you want to be using for sure.

Crates Used

The exec_time crate

For Day 1: Report Repair, I started by using the exec_time crate, which I use throughout my solutions. The crate just prints the time taken to execute whatever method the included macro is placed on:

#[exec_time]
fn run_something() {}

Which prints the following to the command line:

Time run_something: 0 mills

The crate was a useful enough way to just dump the time taken in the function, but there is some bug with mutable arguments that causes the macro to fail. I worked around this by never passing mutable arguments, but its not a viable crate to use to be honest.

The colour crate

I also used colour crate in my first solution and all the solutions thereafter. It just lets you print to the command line with colours, with an easy API:

red_ln!("RED!");
green_ln!("GREEN!");

I did this solely for festive reasons - Christmas colours are green, red, and white - this meant that I printed each part 1 solution in red, each part 2 solution in green, and the exec_time crate above prints its timing info in white text interpersed throughout - very festive!

The intbits crate

For Day 3: Toboggan Trajectory, I used the intbits as part of this puzzle solution. The crate just lets you access bits in an integer with a nice API:

let mut i = 0;
i.set_bit(4, true);
assert!(i.bit(4));

The bit_set crate

For Day 5: Binary Boarding, I wanted to use an N-wide bitset for this challenge, to keep data used to a minimum again, and I went with the bit_set crate:

let mut seats = BitSet::with_capacity(1000);
seats.insert(4);
assert!(seats.contains(4));

The itertools crate

For Day 7: Handy Haversacks, I used the itertools crate which has a really nice way to get tuples out of an iter:

let (lhs, rhs) = line.splitn(2, "some seperator").next_tuple().unwrap();

I ended up using this crate everywhere - so that I could use the next_tuple method on the result of splitn to get a known tuple from the result. It was very much overkill for what I actually needed, but super useful.

The num-integer crate

For Day 13: Shuttle Search I used the num-integer crate solely to get access to the least common multiplier (lcm) method:

assert_eq!(lcm(3, 5), 15);

This was probably overkill (bringing in the whole crate), but I wanted to use Rust in all its glory as much as possible.

The regex crate

I’ve been told Advent of Code wouldn’t be complete without using some form of regex, and I hadn’t actually used regex at all in Rust before. I used the regex crate to accomplish what I needed, and while I’m no further forward in being better with regex than I was before, I did find the implementation of it using Rust to be pleasant to use (especially when you need to extract multiple bits from some complicated string).

The bit_reverse crate

I used the bit_reverse crate for my 20th days solution - this was the problem I hated solving more than any other in this years Advent of Code, and I used the bit reverse functionality to quickly get the bits in an integer in the reverse order. This was a really messy solution to the problem and honestly I’m not that proud of it, but it did the job, my code ran pretty fast, and it served a purpose.

Conclusion

My major takeaway from Advent of Code is that it’s a really great way to level up your usage of a language you already know a little of.

Originally I was going to attempt to use the Zig language to solve the problems - but I knew nothing of Zig’s syntax and tooling, I found the docs a little confusing to figure out (I was trying to work out how to load a file into a buffer!), and I quickly realised that I just wouldn’t complete the Advent of Code challenge while learning an entire new language and trying to work out the solutions.

With Rust I had been experimenting with things already and knew enough to be dangerous, but not enough to feel comfortable with it. Advent of Code was tremendously useful for reinforcing a thin level of knowledge of a language with a more robust appreciation of its quirks.

With that - I’m calling time on 2020, tha bliadhna ùr mo chàraidean, and I’ll see you in the new year!

This year I’m changing that, and I’ve just pushed my 1st day’s solutions to my AOC 2020 repository on GitHub. Solution parameters aside:

• I’m doing this in Rust. I started writing it out in Zig but honestly the docs are nowhere near as clear as with Rust, and the tooling isn’t there. There is a more general point here about how much effort it takes for languages to achieve that status (and how much we have Mozilla to thank for doing Rust by themselves for so blooming long to get it amazing!).
• I’ve just created a simple command line app that you can do cargo run and it’ll print the solution to the command line.
• I’m pretty sure there will be a more optimal way to do what I’ve done, I’m really using this as a way to polish up my Rust skills some more!

You can check out the https://github.com/sheredom/AOC2020 if you are interested in my progress, check in tomorrow for those solutions. Tioraidh bye!

• Designing and Evaluating Reusable Components from Casey Muratori. This absolutely wonderful talk is the fundamental source for how to design APIs.
• Hourglass Interfaces for C++ APIs from Stefanus Du Toit. This talk discusses having a rich API for users that is backed by a C API into proprietary code.

These two talks give a really good and grounded lesson on fundamental API design - the choices you make and the ramifications they can have. In particular, the design challenge and solution of the hourglass API design is a really nice approach to being able to ship a high functioning API that is backed by some proprietary software that you don’t want to ship. For example:

struct MyDataOpaque;

extern "C" MyDataOpaque* mydata_create(int);
extern "C" void mydata_destroy(MyDataOpaque*);

struct MyData final {
  explicit MyData(int someState) : opaque(mydata_create(someState)) {}

  ~MyData() { mydata_destroy(opaque); }

private:
  MyDataOpaque* const opaque;
};

This is a classic hourglass API in C++ - we’ve got the C API that returns some opaque data, and the hourglass MyData struct lets us expose this to C++ users in a way they are familiar with. In Stefanus’ talk he went into detail about how they also implement the C API in C++ - meaning that both sides of the thin C API are written in the high level language they enjoy - C++.

This all got me thinking - would this approach be possible in some fashion with Rust? First the caveats:

• I suspect, but am not well versed enough with Rust, that using different versions of Rust to build either side of the hourglass API could cause explosions. So I’m going to assume that both sides are built with the same version.
• I really hope that it is safe to use memory allocating functions (like Box or Vec) on both Rust sides of a C API.

These pretty major caveats aside - is it possible?

The Bottom of the Hourglass

First we’ll implement the bottom of the hourglass - this would be the proprietary code that you don’t want to ship (all your secret sauce might be in it!).

#[repr(C)]
pub struct MyDataOpaque {
  some_state: i32,
}

impl MyDataOpaque {
  pub fn new(some_state: i32) -> MyDataOpaque {
    MyDataOpaque { some_state }
  }
}

#[no_mangle]
pub extern fn mydata_create(some_state: i32) -> *mut MyDataOpaque {
  Box::into_raw(Box::new(MyDataOpaque::new(some_state)))
}

#[no_mangle]
pub extern fn mydata_destroy(d : *mut MyDataOpaque) {
  // This causes the Drop to be called on the box, freeing everything.
  let _ = unsafe{ Box::from_raw(d) };
}

It’s a quite simple bit of code (for our simple example), but the crux of it is just some opaque struct, and some unmangled public extern symbols for the exported C API.

Building the Two Crate Solution

I wanted a way to build a Rust crate for the above, then build with cargo on a second crate that uses it. I did not want to have a single instantiation of cargo because I wanted to be 100% sure that I was exercising the export and import paths of Rust. To do this I used a build.rs like:

use std::env;
use std::process::Command;

fn main() {
    let out_dir = env::var_os("OUT_DIR").unwrap();
    let working_dir = env::var_os("CARGO_MANIFEST_DIR").unwrap();

    Command::new("cargo")
        .arg("build")
        .arg("--manifest-path")
        .arg(&format!(
            "{}/MyDataOpaque/Cargo.toml",
            working_dir.to_str().unwrap()
        ))
        .arg("--target-dir")
        .arg(&format!("{}", out_dir.to_str().unwrap()))
        .arg("--release")
        .status()
        .unwrap();

    println!(
        "cargo:rustc-link-search=native={}/release",
        out_dir.to_str().unwrap()
    );
}

This build script calls cargo in a sub-folder called MyDataOpaque, builds the crate, and then adds the built library as a link requirement to our crate.

Building the top of the Hourglass

So now we’ve got a Rust library, exposed via a C API, that we want to write Rust bindings for.

Just to prove it worked I first tried using cbindgen to generate a C API for the bottom of the hourglass, and then bindgen to take this C API and generate FFI bindings for it in the top of the hourglass, but these have a ton of dependencies (like libclang) that I wasn’t happy about. So I instead decided to just manually write the C API for the top of the hourglass.

Aside: assuming this approach was viable you could forsee an hourglass_bindgen crate that did something similar to what cbindgen does, but just writes out the Rust FFI to the C API instead.

#[repr(C)]
pub struct MyDataOpaque {
    _private: [u8; 0],
}

#[link(name = "MyDataOpaque")]
extern "C" {
    fn mydata_create(some_state: i32) -> *mut MyDataOpaque;
    fn mydata_destroy(opaque: *mut MyDataOpaque);
}

pub struct MyData {
    opaque: *mut MyDataOpaque,
}

impl MyData {
    pub fn new(some_state: i32) -> MyData {
        MyData {
            opaque: unsafe { mydata_create(some_state) },
        }
    }
}

impl Drop for MyData {
    fn drop(&mut self) {
        unsafe {
            mydata_destroy(self.opaque);
        }
    }
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn create_destroy() {
        let _ = MyData::new();
    }
}

As we can see this code is very similar to the C++ example I showed at the beginning of the post - it wraps a C API in a Rust struct and then can use the existing safe mechanisms of Rust (like the Drop trait) to ensure safety of the data used.

And if I run it?

running 1 test
test tests::create_destroy ... ok

It works, nice!

Conclusion

Hourglass APIs appear to work in Rust (minus the caveats above that I’ll have to explore further). I do think that maybe using no_std in the bottom of the hourglass could mitigate even issues with multiple versions of Rust being used, but again I’ll have to verify it.

You could even forsee of an approach where in the build.rs instead of building some other crate (whose source you don’t want to ship), you could use the TARGET to fetch the correctly pre-built version from a web service, or maybe have a bunch of the library versions resident in the crate (obviously the total crate size limit might become an issue though), and pick between them.

It’s pretty awesome to me that this approach looks feasible though!

I didn’t get any responses on this so I commented with a follow-up that I had grabbed some code written a few years back by the awesome Pete Warden of Google fame, and morphed it into what I required:

Authors note: this part used to contain a tweet, But Hellish Tusk / Space Karen / Elon Musk butchered the platform so it is now gone.

Much to my surprise Pete was happy not only for me to do these modifications, but also since he was no longer maintaining the hashmap code he’d happily redirect users to any effort I put together:

Authors note: this part used to contain a tweet, But Hellish Tusk / Space Karen / Elon Musk butchered the platform so it is now gone.

So I’ve done the work and I’m now introducing my latest library, nearly entirely not written by me, hashmap.h!

Null-Terminated to Slices

So Pete’s code was pretty solid as is. The main difference was that it relied on null-terminated strings as the key, whereas I wanted to use string slices. My first modification was to change the entry points that used a key to instead take a key and a length. So hashmap_put went from:

extern int hashmap_put(map_t in, char* key, any_t value);

To:

HASHMAP_WEAK int hashmap_put(struct hashmap_s *const hashmap,
                             const char *const key,
                             const unsigned len,
                             void *const value);

You’ll notice that I also went const mad (and again wished that const was the default and mutable or mut was required on variables - sigh!), and removed the typedef for any_t. The last point is just a general stylistic thing I have for my libraries - I really dislike that APIs like Windows.h abstract you so far away from the underlying types with all the SHOUTY CASE LDPWORD’s and such, that I generally try and have no typedef’s if I can get away with it.

Supports UTF-8 Keys

Pete’s code also used the string.h ASCII-string functions of C to compare whether the key ever matched. Since I wanted to use this hashmap in conjunction with UTF-8 strings (using my utf8.h library) I instead used memcmp. Now that I have an explicit length for the string slice this became possible.

Single Header

The last major change I made was to smush the hashmap.c and hashmap.h files together into a single header. I am pretty obsessed with single headers as a way to get round the botched nature of C and C++’s package story (or lack thereof). This meant leveraging some exists hacks to stop the compiler complaining about multiple function definitions (by using weak function references instead).

The License

Pete’s code was already marked explicitly public domain - do what you want with it. I’ve found over the years that while public domain is all fine and well, having an explicit license like the unlicense or the CC0 can make lawyers happy because there is at least some legal text to reference. It also makes GitHub’s license scraping happier because these licenses are already ones that it knows about.

So I’ve licensed this header under the unlicense - it matches what my existing single-header projects use and is something that my users already favour.

Hashmap All The Things

So I’m pretty happy with the code I mostly did not write - and I hope that my packaging and testing of the header allows it to be more widely useful to some of you fine folks out there.

A big thanks again to Pete Warden for writing this code and being so gracious about me making these changes. I hope this proves useful to some of you out there too.

C FILE’s and Asynchronous Reading

I use a really clever hack to give my users a standard C FILE handle with which they can read the standard output and error, and write to the standard input of any processes they spawn. This is immensely powerful because it means anyone using my API can use an already familiar paradigm to interact with the processes they spawn. Users could fprintf to the standard input of a spawned process, or call fgets to read from the standard output or error. While I love this feature - it causes a complication when trying to support asynchronous reading.

For the normal process spawning on Windows I was using CreatePipe to map a HANDLE on the parent process to HANDLE that’ll be used for the standard pipes on the spawned process. A CreatePipe is also known as an anonymous pipe in Windows lingo - and it turns out you cannot sync from this pipe until the spawned process has complete. But there is another mechanism called a named pipe using the CreateNamedPipe call that can help us here.

Named pipes support an additional flag on creation FILE_FLAG_OVERLAPPED that lets you get at the data from the pipe before the other end of the pipe has been closed. I thought ‘Great! I’ll use this et voila! Everything will work!’. It didn’t.

The problem is that if you use the overlapped creation flags to a named pipe you cannot then do the clever hack anymore to get a C FILE that reads from this. It’ll just hang forever.

So instead I’ve introduced some new helpers methods subprocess_read_stdout and subprocess_read_stderr that use the correct APIs on each operating system to allow for asynchronous reading from a pipe.

Note that while the main problems for this feature have been with Windows support and the example below is Windows specific - the feature supports all three major desktop platforms. Worry not!

Example

As an example of how we can use this I’ve wrote up the following little example that works on Windows (no reason why it couldn’t work on other operating systems, I just used Windows for the test):

int main() {
  const char *const commandLine[] = {"ping", "-t", "www.bbc.net.uk", 0};
  struct subprocess_s process;
  unsigned bytes_read;

  if (0 != subprocess_create(
    commandLine,
    subprocess_option_inherit_environment | subprocess_option_enable_async,
    &process)) {
    fprintf(stderr, "subprocess_create failed!");
  }

  do {
    char buffer[1024] = {0};
    bytes_read = subprocess_read_stdout(&process, buffer, sizeof(buffer));
    printf("Read %u bytes - '%s'\n", bytes_read, buffer);
  } while (bytes_read != 0);
}

This will loop infinitely reading from the ping command with the -t option specified. It’ll read the output from ping as it is flushed and spit out what it read and the number of bytes.

Read 42 bytes - '
Pinging www.bbc.net.uk [212.58.237.253] '
Read 24 bytes - 'with 32 bytes of data:
'
Read 27 bytes - 'Reply from 212.58.237.253: '
Read 27 bytes - 'bytes=32 time=22ms TTL=53
'
Read 27 bytes - 'Reply from 212.58.237.253: '
Read 27 bytes - 'bytes=32 time=22ms TTL=53
'

A few things:

• Note the new subprocess_create option subprocess_option_enable_async. This is needed to setup the pipe for being able to read from it asynchronously. If you use this option you should refrain from using subprocess_stdout or subprocess_stderr to read from the spawned process - they will not work.
• Note the new function subprocess_read_stdout to read from the standard output of the spawned process. It’ll return non-zero while there is any data still to be read. When the child process terminates then it’ll return 0 signalling the end.

One additional thing to bear in mind is that the subprocess_read_stdout and subprocess_read_stderr functions are blocking. If you want to read from both of these simultaneously then it is recommended that you either:

• Spawn a thread for one or both to read from it. It is thread safe to read from both pipes of a spawned process at the same time.
• Or use the subprocess_option_combined_stdout_stderr option if you just want the full contents of both pipes both don’t care to differentiate from either pipes output.

I hope you find this new functionality useful.

An additional gentle reminder that you can stay tuned for future updates by using the RSS feed.

So what are fixtures and why should you use them?

What are fixtures?

Fixtures are a way to setup and teardown state that doesn’t contribute to the timing of the actual benchmark itself:

// First you declare a struct that contains the state you need.
struct foo {
  unsigned size;
  char* foo;
};

// Then you setup the benchmark state - using the ubench_fixture implicit field.
UBENCH_F_SETUP(foo) {
  ubench_fixture->size = 1024 * 1024 * 128;
  ubench_fixture->foo = (char*)malloc(ubench_fixture->size);
}

// In the teardown you can free up anything you allocated.
UBENCH_F_TEARDOWN(foo) {
  free(ubench_fixture->foo);
}

// And then declare the benchmarks that use the state.
UBENCH_F(foo, bar) {
  UBENCH_DO_NOTHING(ubench_fixture->foo);
  memset(ubench_fixture->foo, 0, ubench_fixture->size);
  UBENCH_DO_NOTHING(ubench_fixture->foo);
}

// You can declare multiple benchmarks that use the same fixture too.
UBENCH_F(foo, haz) {
}

As you can see setting up and declaring a fixture isn’t difficult.

Why you should use them?

Let’s take the counter example to the fixture above - using the non-fixture approach:

UBENCH(foo, bar) {
  const unsigned size = 1024 * 1024 * 128;
  char* foo = (char*)malloc(size);
  UBENCH_DO_NOTHING(foo);
  memset(foo, 0, size);
  UBENCH_DO_NOTHING(foo);
  free(foo);
}

And if we look at the results of both these tests we can see that the non-fixture one runs like:

[==========] Running 1 benchmarks.
[ RUN      ] foo.bar
[       OK ] foo.bar (mean 15.506ms, confidence interval +- 1.553863%)
[==========] 1 benchmarks ran.
[  PASSED  ] 1 benchmarks.

Whereas the fixture one runs like:

[==========] Running 1 benchmarks.
[ RUN      ] foo.bar
[       OK ] foo.bar (mean 4.125ms, confidence interval +- 0.844092%)
[==========] 1 benchmarks ran.
[  PASSED  ] 1 benchmarks.

So we can see that the average time for the fixtured one is about 4x faster than the non-fixtured one - because it hasn’t timed the malloc/free cost per benchmark. This means that if you really only care about what memset costs you’ve now got a much more accurate way to get that.

This is where fixtures are really great - hide the cost of setting up the data for your benchmark from the thing you actually want to benchmark.

One last thing…

One thing that might not be obvious is that fixtured tests can have a wall clock time that is longer than the non-fixtured variants. For the examples above the non-fixture one runs in:

real 0m0.592s
user 0m0.368s
sys  0m0.177s

Whereas the fixture benchmark runs in:

real 0m3.631s
user 0m3.508s
sys  0m0.071s

So why is this? In general the longer running a benchmark is the more accurate the result will be - so because we’re measuring less things in the actual benchmark samples the framework is having to run more of them to get an accurate result.

Just thought you might appreciate the heads up!

It’s funny how as soon as you push something public you realise all the flaws in it - but that’s fine! Software (like people) should grow and develop as it gets older.

The way I was trying to gauge the accuracy of the result from multiple benchmark runs was somewhat… broken. In this PR I’ve switched to using a confidence interval, and I’ll now explain why.

The Broken Approach

So I’ve never studied statistics in any capacity. I’d always followed the believe that statisticians were like demi-lawyers - good at manipulating the same set of ‘facts’ to tell whatever story they want.

But ever since I watched the rather awesome Elan Ruskin talk about statistical tests for games I realised I needed to level up.

So the first thing I learned about was standard deviation - well really I had to re-learn this because I’m pretty certain I got taught it in high school and immediately forgot it… For those uninitiated the standard deviation gives you a measure of how variable the average of a set of data is. Let’s say you’ve got 10 recordings of something - the average is just the sum of all items divided by the number of recordings, otherwise know as the mean.

You then use this mean, subtract it from each of the original values, square them, divide by the number of iterations and then take the sqrt of that:

ubench_int64_t ns[iterations];
ubench_int64_t avg_ns = 0;

for (i = 0; i < iterations; i++) {
  avg_ns += ns[i];
}

avg_ns /= iterations;

for (i = 0; i < iterations; i++) {
  const double v = ns[i] - avg_ns;
  deviation += v * v;
}

deviation = sqrt(deviation / iterations);

Once I had this I then used the standard deviation to measure how variable the result was. A high standard deviation means that you don’t have confidence that the result you’ve recorded is stable enough to be meaningful. Conversely a low standard deviation makes you confident. Previously I was aiming for a standard deviation below 2.5% as a gauge of how good the result was.

The problem is that getting a standard deviation below a low margin across many different types of machines we test on (laptops, desktops, mobiles, consoles, etc) can be quite difficult. Pre-emption of the running thread might cause a temporary spike in a given tests result, or you might get a very fast run that hits no speed bumps and thus runs really fast. These outlier results can really affect the standard deviation of the result.

Since I didn’t know much about stats I wrongly thought that the best way to cut out the occassional high variance results was to just sort all the runs and then slice off the top and bottom N items:

ubench_int64_t ns[iterations];
ubench_int64_t avg_ns = 0;

// Sort the results.
qsort(ns, iterations, sizeof(ubench_int64_t), ubench_sort_func);

for (i = 1; i < (iterations - 1); i++) {
  avg_ns += ns[i];
}

avg_ns /= (iterations - 2);

for (i = 1; i < (iterations - 1); i++) {
  const double v = ns[i] - avg_ns;
  deviation += v * v;
}

deviation = sqrt(deviation / (iterations - 2));

This would cut off the top and bottom worse skewed results. The problem is that this didn’t really reduce the standard deviation that much in the first place, and secondly it discounted valuable collected data.

Another problem with the standard deviation is that its not necessarily true that collecting more data will reduce the deviation itself - just because you are collecting more data doesn’t meant that the data will be any less deviant.

So I needed another approach.

I’m 99% Confident with my Interval

If we look at the typical spread of data for any measurement, the data tends to centre around the average or mean value. This is called the normal distribution or a bell curve, and generally looks like:

In this chart, x = 0 is the the average, and y is the number of samples that matched the given value of x. This is what data tends to look like from benchmarks. We can see that most of the samples (larger y) occurs closer to the mean, and there is less and less the further we go away. Given I want to cut out any outliers large or small, what we really want is a way to say ‘Is most of the data close enough to the average?’.

Luckily some people smarter than me have a way to do this - and it is called a confidence interval. A confidence interval is a way to say ‘If we only care about X% of the data, how close is that to the average?’. For my use case I’m using a 99% confidence interval - EG. I’m asking the stats to give me a number for how close the 99% of the samples that are closest to the average actually are:

We can see in this chart the two extra lines - this is my confidence interval, I’m only wanting to know how close the X% of the values that are occuring within these bounds are to the average.

Calculating a confidence interval is pretty easy too:

for (i = 0; i < iterations; i++) {
  avg_ns += ns[i];
}

avg_ns /= iterations;

for (i = 0; i < iterations; i++) {
  const double v = ns[i] - avg_ns;
  deviation += v * v;
}

deviation = sqrt(deviation / iterations);

// Confidence is the 99% confidence index - whose magic value is 2.576.
confidence = 2.576 * deviation / sqrt(iterations);

Now there is this magic constant 2.576 - I genuinely have no idea where it comes from or why it is what it is. But some smart stats people has done some math stats stuff and… this is the value I need.

Now instead of aiming for a 2.5% range with my botched standard deviation, I instead look for a 2.5% range with my 99% confidence interval instead. The great thing about the confidence interval is the more data you throw at it the lower it goes. I think this is because you cut out more outliers (the 1%) the more samples you have, but whatever the reason it means that if I haven’t achieved the 2.5% range I want I can just re-run it with more samples and that generally does what I want. Nice!

The output from ubench.h has now changed to display the confidence interval too, and I added a request from @zeuxcg to change the displayed timing in ms/s/us where appropriate - which makes it much more legible:

[ RUN      ] cpp_my_fixture.strchr
[       OK ] cpp_my_fixture.strchr (mean 4.115ms, confidence interval +- 2.039480%)
[ RUN      ] cpp_my_fixture.strrchr
[       OK ] cpp_my_fixture.strrchr (mean 57.057ms, confidence interval +- 0.445810%)

Conclusion

So I’ve made ubench.h more consistent at profiling and actually used real mathematically sound statistics rather than my bodged attempt at it - nice! For my users it’ll mean more consistent data and more confidence (har har) that it is doing the right thing.

And I’m sitting here feeling guilty as all heck. Why should I, as a peely wally white guy, benefit yet again from black voices rightfully pointing out the utterly unjust situation that we (white folks) have put them through for at least four hundred years. Shamefully I didn’t even know what Juneteenth was until they announced it was a holiday, and I quickly scuttled over to wikipedia to read what actually happened on that day.

To try and shake just a sliver of my guilt I’ve donated my wages for today to the Stephen Lawrence Charitable Trust to in some way help towards all the good they do.

I also want to talk a bit about white guilt - why we should feel guilty, and especially the role that Scotland played in and profited from the horrors of the slave trade, and the ongoing problems with racism and discrimination that still affect Scotland today.

White Guilt

But the one thing the American dream fails to mention, Is I was many steps ahead to begin with - White Privilege II by Macklemore & Ryan Lewis

The line above from the great “White Privilege II” song by Macklemore & Ryan Lewis was in reference to America but it applies equally to white men everywhere. We’ve had a head start on nearly ever situation we’ve been involved in and didn’t realise because that’s just how things went for us. We’d been born white and thus benefitted from a racist system. We’d been born male which meant we benefitted from a sexist system. We’ve had job offers, higher wages, promotions, lenient police intervention, and the list goes on.

If we are assertive and egotistical we’re praised as leaders while the system has actively punished black people and woman for showing the same traits that are so valued in white men.

And the worst bit about all of this for me personally is that because this discrimination didn’t affect me personally in any way I was a passive ally to a broken system.

So I’ve got white guilt - specifically white man guilt - and it is right that I do. I’ve implicitly benefitted from a system that was rigged in my favour and I was too wrapped up in my own ego to realise.

Scotland and Slavery

David Hayman made a fantastic program for BBC Scotland - Slavery: Scotland’s Hidden Shame. In it, David goes into depth about all the horrible ways that Scotland directly benefitted from slavery (lots of people here owned slaves as a ‘good investment’), and also how Scots themselves disproportionately made up the staff employed in managing the slave trade (from the physicians on the boats to those doing the paperwork).

We Scots have to acknowledge the part our country played in this through education and introspection. In high school history, the lessons I was taught consisted of a walk through all the times we won against the English in war, how France is the Auld Ally and our friend, and then it skipped forward to the industrial revolution and the slum-like conditions the families lived in. The period of failed colonisation (see the hilariously badly planned Panama colony - including taking winter kilts to the tropics), the reasons we went into union with England (Scotland was so poorly managed we went bankrupt), and the subsequent involvement of Scots in the slave trade were all massive missing parts of my education.

Another fact that people aren’t taught here is that the racist and facist KKK (who truly should be considered a terrorist organisation I don’t even understand how they can exist) was founded by Scots.

It’s important that we get taught these lessons.

Scotland Today

Scotland today is a country that is trying to be an outward looking and welcoming country. We talk about welcoming refugees and being open to people from all backgrounds and faiths. And yet two days ago our First Minister had to condemn racist thugs on the streets of our biggest city Glasgow.

Scotland has a long history of discrimination. At least where I’m from south of Glasgow, there have been multiple waves of immigrants in the past few hundred years that have caused discrimination to ramp up. My own family has heavy Irish roots - having fled either during the potato famine or when the republican movement flared up in Northern Ireland. There was a huge movement of Italians post World War II that settled in Glasgow and the surrounding areas. There are decent communities of incomers from Bangladesh, India, Pakistan, and Turkey too.

And each of these groups faced persecution. The Irish, being predominantly catholic incomers, met with hostility of the Orange Order and had derogatory phrases like “Fenian Scum!” hurled at them, the Italians were called “Tallies” and faced an anti-Italian fervour, and anyone who looked even slightly like they were from the Middle East or East Asia was called a “Paki”.

I was from a catholic family in a very protestant town, and before the age of 16 I was:

• Petrol bombed.
• Chased with Samurai swords.
• Had bricks thrown at me.
• Involved in many altercations of the fisticuff kind.

Every walk home from school was a gauntlet of ‘what would happen today’. If I wore the colour green on a day where the Orange Order were doing one of their nonsense ‘parades’ (they march and play flutes, followed by a rowdy crowd of drunk racists looking for a fight) I’d have to run - fast.

I have never once witnessed overt racism against black people here, but I’ve seen plenty of people who have had “Paki” thrown in their face in my time. I thought things here had got better, or at least I had hoped. But when you hear the things that the racist thugs above said it is clear we’ve still got a ways to go.

What Can White People Do

There are some things we white men can do to help correct the historial wrongs we’ve inadvertently took part in:

• Make space for black and other under-represented groups. We need to put our elbows out and use our privilege as a way to pull others up.
• Push for diversity in hiring. This requires having hard conversations with your direct management, but also using company wide meetings that allow questions of the high hiedyins as an opportunity to ask about diversity.
• Try to seek out people who don’t look like you and engage with them. My twitter and social circles are predominantly white and male, and for too long I put this down to “Well these are just the people in the industry I know so…”. This was wrong of me and I need to do better.
• Speak up when you see injustice in the world and your workplace. The white man privilege we’ve always had gives us a shield against most of the repercussions that under-represented groups would face if they speak out.
• Where possible donate to groups that help the black community.
• Arrest Brett Hankison, Jonathan Mattingly, and Myles Cosgrove for the murder of Breonna Taylor.

Let us try and make a better world for everyone, together. We’re all the better when we’re in it together.

• It isn’t a single header library.
• It doesn’t support C benchmarks.
• It is huge.
• It has a build process that results in a non trivial integration process.

While none of these are a death knell for an otherwise incredibly useful library, it just didn’t meet enough of the requirements for my own uses.

So I’ve wrote my own - ubench.h.

Introducing ubench.h

So given all the infrastructure I had already worked out to get my utest.h library, I could re-use a significant amount of the code. I started by copying the code from utest.h and doing a rename of UTEST -> UBENCH and utest -> ubench. Next up, I removed all the code that wasn’t required from testing - all the asserts and expects that make up a unit testing framework. I also removed fixture and indexed testcases because I wanted to push for a minimally viable product to begin with.

UBENCH

To declare a benchmark you use the UBENCH macro:

#include "ubench.h"

UBENCH(foo, bar) {
  usleep(100 * 1000);
}

The macro takes two parameters - the set of the benchmark, and the name of the individual benchmark. In the body of the benchmark you specify the code you want to profile - in this case I’m just sleeping for a certain amount of time as a good no-op example.

In one file you then need to specify the UBENCH_MAIN() - this macro will define the main function of your executable, define all the global state we require, and call into ubench.h to run the benchmarks:

#include "ubench.h"

UBENCH_MAIN()

You can alternatively call the ubench_main function yourself if you want to use your own main function instead. First you need to specify the state for ubench.h:

UBENCH_STATE();

And then when you are ready to call into the ubench.h framework do:

int main(int argc, const char *const argv[]) {
  // do your own thing
  return ubench_main(argc, argv);
}

Note - you need to specify UBENCH_MAIN or UBENCH_STATE in a single source file only.

Command Line Output

The command line output for the library is similar to utest.h and thus also similar to googletest’s output:

[==========] Running 1 benchmarks.
[ RUN      ] foo.bar
[       OK ] foo.bar (102289906ns +- 1.751264%)
[==========] 1 benchmarks ran.
[  PASSED  ] 1 benchmarks

The one notable difference is how tests pass/fail, and the additional of +- 1.751264% to the timing information. Benchmarks are only useful if the repeated runs of them result in similar numbers. You need to run a benchmark a number of times and then get an average of the result. The problem is that the average can be vastly affected by huge outliers in the data set (EG. was the test ran from a cold start with dirty caches?) which can drastically affect the reliability of your results. To combat this, the benchmarks are recorded multiple times in multiple passes. Only when the standard deviation of the test is lower than 2.5 percent (a good initial value for a reliable and reproducible result) can the benchmark be seen as being useful. The +- 1.751264% is the standard deviation that was recorded for that example. If after a number of runs the variance of the runs does not conform to a small enough deviation then the benchmark is regarded to have failed, and is reported as such.

UBENCH_DO_NOTHING

Benchmarks are best when they are simple enough to record a single simple thing, that gives good reproducible results. One thing that defeats this is that compilers are so handy now at removing dead code - optimizing away the very thing you want to measure! To get around this I’ve included the UBENCH_DO_NOTHING macro, which takes a pointer argument and looks, to the compiler, like it will trash the memory within (read and write it). This way the compiler shouldn’t optimize out the data you are using and thus the code that you are using to produce it:

UBENCH(do, nothing) {
  static char a[8 * 1024 * 1024];
  static char b[8 * 1024 * 1024];
  UBENCH_DO_NOTHING(a);
  memcpy(b, a, sizeof(a));
  UBENCH_DO_NOTHING(b);
}

The above makes the compiler think that a is getting some data placed within it, and then after the memcpy that something is being done with b. The end result is we can get a meaningful performance number for how expensive the memcpy is.

Summary

The library is already simple yet useful for measuring some stuff - but I obviously want to do more. Fixtures are the obvious addition - you may want some state to be initialized once and then we profile the modifications of the data on its own rather than alongside the cost of initialization / destruction. I’d also like to give the user more options for controlling how many times the benchmark is run to get a low enough standard deviation.

But I think as a programmer our number one job is to get something simple done and then use it in production. Production always helps shuffle out the actual missing things you need from the code rather than in the abstract up front trying to work out the full set of requirements. I hope the library proves useful to some of you out there, and you can keep track of the progress on GitHub at https://github.com/sheredom/ubench.h.

Since the thing I’m working on requires a command line application I thought ‘Why not write my own?’ and herein lies the fun little trip I took.

The Requirements

So I want something that was:

• Written in C entirely.
• Does not dynamically allocate any memory to store the arguments.
• Can automatically generate the --help output for the supported options.

To keep things simple I decided that short form options and long form options would have a defined structure:

• They can take no args and are of the form of -h and --help.
• Or if they take an argument they are structured like -o <arg> or --output=<arg>.
• They can only be specified zero or one times.

I wasn’t aiming to make some library that I could parachute into any project, more that I just wanted something simple that I could own myself.

Abusing Include Files

I really dislike anything that causes me to copy and paste code around. My distaste stems from the fact that you regularly want to then update all N places where you copied something, and it becomes onerous to keep these in check. Concretely for doing a command line parser, I didn’t want to copy and paste code any time I added a new command line option.

Instead I wanted some way I could define all my options in a standard format and the rest of the tooling would pick this up and turn it into what I wanted. The only real way to do this in C is with the preprocessor. Since I knew that I’d be wanting to use the same list of options for multiple purposes (EG. I’d want to use them to parser argv but also print out their descriptions for --help), I decided to use an include file - a .inc. In general the notion for .inc files is that they have some preprocessor macro in their body that the includer defines before including the .inc file. Then they can include the .inc file multiple times and change what the macro does. This means that unlike most includes, .inc files do not have include guards / pragma once to stop multiple includes of them - because including it multiple times is exactly what we want to do.

#ifndef OPTION
#error OPTION(short-name, long-name, number-of-trailing-args, description) was not defined!
#endif

OPTION(o, output, 1, "the output file to write to")
OPTION(h, help, 0, "print all the options and exit")

#undef OPTION

Here is my really simple options.inc that I used. It firstly includes a little if check to ensure that the required OPTION macro was specified, with a description of how the macro is intended to be used. Then it contains the list of the options that I have. At present just two options - one to specify where the output of the tool will go, and the other to display the help message and exit. Note the minor quality of life thing whereby the .inc file undefined the OPTION macro - this just saves you either doing it yourself after each include of the .inc file, or having the compiler warn about multiply defines macros!

Now that I have this I can use it to actually implement the parsing.

Parsing Argv

So the first thing I wanted to ensure was that my OPTION macros were not malformed - specifically that the ’number-of-trailing-args’ was either 0 or 1. Even though I’m the only one working on this and I hopefully shouldn’t be dumb, I’ve found over the years that any notion that you won’t be dumb in some way is generally disproven. We are all stupid meatbags at times and so while you are hyper aware of the constraints it is best to be as defensive a coder as possible. Your future self will thank you!

#define OPTION(s, l, n, d)                                                     \
  bootstrap_assert(n <= 1,                                                     \
                   "option" #s "/" #l " must have 0 or 1 trailing arguments");
#include "options.inc"

So you can see that I just #define the OPTION macro and in this instance make it call my custom assert. Then I just have to include the .inc file and it’ll do this for every option I’ve defined.

  const char *const arg_is_set = "set";
#define OPTION(s, l, n, d) const char *arg_##s = 0;
#include "options.inc"

Next I needed a location to store the results of each of my variable. Using the same pattern again I can define variables of the form arg_h for the help argument, arg_o for the output, etc. For arguments that do not have any arguments I added a single arg_is_set here too - this we will use later to check whether the argument was set or not.

Now to the parser. This one is a bit of a mouthful I’ll just warn you up front - it actually shows one of the big downsides to code generation with the preprocessor in C/C++ in that code in preprocessors is very difficult to debug within both compiler errors and when using a debugger. So remember that I didn’t want any runtime memory allocation. So the one place you generally would use this is for positional arguments. Think of things like the list of files you’d pass to a compiler, you can have N of these and they don’t require options to be passed in. To get this to work I’m going to reuse argv to store the list of the positional arguments. Since each positional argument always arrived in argv in the first place, it stands to reason that they can definitely fit!

  int positional_arguments = 0;

  for (int i = 1; i < argc; i++) {
#define OPTION(s, l, n, d)                                                     \
  {                                                                            \
    if (0 == strcmp("-" #s, argv[i])) {                                        \
      bootstrap_assert(arg_##s == 0, "option" #s "/" #l                        \
                                     " cannot be specified multiple times");   \
      if (n == 0) {                                                            \
        arg_##s = arg_is_set;                                                  \
      } else {                                                                 \
        bootstrap_assert((i + 1) < argc, "trailing argument for option" #s     \
                                         "/" #l " was not provided");          \
        arg_##s = argv[i + 1];                                                 \
        i++;                                                                   \
        continue;                                                              \
      }                                                                        \
    } else if (0 == strncmp((n == 0) ? "--" #l : "--" #l "=", argv[i],         \
                            strlen("--" #l "="))) {                            \
      bootstrap_assert(arg_##s == 0, "option" #s "/" #l                        \
                                     " cannot be specified multiple times");   \
      if (n == 0) {                                                            \
        arg_##s = arg_is_set;                                                  \
      } else {                                                                 \
        arg_##s = argv[i] + strlen("--" #l "=");                               \
        continue;                                                              \
      }                                                                        \
    }                                                                          \
  }
#include "options.inc"

    // Otherwise if we didn't find our option then it is a position argument!
    argv[positional_arguments++] = argv[i];
  }

So let’s break down the above:

• First I do the check for the small option form s, by string comparing "-" #s against the argument.
  • I then check if the option has been specified before which I’m not handling.
  • If the option has zero trailing arguments, then just set the global variable for arg_##s to the arg_is_set helper I defined before.
  • If the option has one trailing argument, then I check there is definitely been a provided argument (EG. someone didn’t have -o at the end of the argument list!).
  • And store the value in argv[i + 1] into the variable for the option.
  • I need to bump i to skip the next argument because we’ve consumed it as the trailing argument of this one.
  • And then continue, this just gives us a nice easy way to skip the backup code to store the argument as a positional one.
• Next I do the check for the long form l - which requires a slightly different string compare. Because long form options that take an argument come like --output=<arg> I need to check if the option had a trailing argument, and then use a strncmp for this.
  • The only other difference is when there is a trailing argument I need to not bump i this time - because the argument is included in the option. To get at the argument we just get the pointer to the first character after the --output= - by using strlen.
• And then the fallback case is to just store the argument into the start of argv as a positional argument.

Using the Parsed Options

So the first obvious way to use the options is to print the help:

  if (arg_h) {
    printf("usage bootstrap");
#define OPTION(s, l, n, d)                                                     \
  if (n == 0) {                                                                \
    printf(" [ %s | %s ]", "-" #s, "--" #l);                                   \
  } else {                                                                     \
    printf(" [ %s <arg> | %s=<arg> ]", "-" #s, "--" #l);                       \
  }
#include "options.inc"
    printf(" <args>\n\n");

#define OPTION(s, l, n, d)                                                     \
  if (n == 0) {                                                                \
    printf(" %s | %s %s\n", "-" #s, "--" #l, d);                               \
  } else {                                                                     \
    printf(" %s <arg> | %s=<arg> %s\n", "-" #s, "--" #l, d);                   \
  }
#include "options.inc"

    return 0;
  }

Breaking this down I:

• Check that the arg_h variable is non-null - which means the user requested help.
• Use an OPTION macro to print the short usage guide - something like usage bootstrap [ -o <arg> | --output=<arg> ] [ -h | --help ] <args>.
• Then I use a second macro to print the longer usage guide with the descriptions.

In the end this produces a help like:

usage bootstrap [ -o <arg> | --output=<arg> ] [ -h | --help ] <args>

 -o <arg> | --output=<arg> the output file to write to
 -h | --help print all the options and exit

The only thing that’d make this nicer would be to align the descriptions. But for now its good enough for me.

Just to test that all the options were set and the positional arguments were stored like I expected, I added a second macro during debugging:

#define OPTION(s, l, n, d)                                                     \
  fprintf(stderr, "%s = %s\n", #s, (arg_##s != 0) ? arg_##s : "not set");
#include "options.inc"

  fprintf(stderr, "Positional arguments:\n");
  for (int i = 0; i < positional_arguments; i++) {
    fprintf(stderr, "%s\n", argv[i]);
  }

All this does is print whether each option was set or not, and then the list of positional arguments we recovered from the command line.

In total this is around 100 lines of C and does what I want in a very simple and easily controlled manner. I’m very happy to be able to go back to basics and write some fun C code like this from time to time!

But this got me thinking - there is no reason why my library shouldn’t work with whole program optimization enabled, so why was this behaving badly?

Visual Studio’s Whole Program Optimization

To enable whole program optimization in Visual Studio requires the /GL option on the cl.exe compiler, and /LTCG on the link.exe. What this does is effectively defer some of the optimizations and optimization ability to link time. At link time it can then perform optimizations across multiple individual compilation units to (in theory!) result in a more optimal resulting executable.

In CMake you can enable this for an executable like so:

if("${CMAKE_CXX_COMPILER_ID}" STREQUAL "MSVC")
  target_compile_options(utest_test_wpo PRIVATE "$<$<CONFIG:RELEASE>:/GL>")
  target_link_options(utest_test_wpo PRIVATE "$<$<CONFIG:RELEASE>:/LTCG>")
endif()

What the above does is turn on the option only for release builds, for the given target. The first thing I did when trying to work out this issue was to duplicate the existing tests I had for utest, and build a separate target utest_test_wpo. I used the above CMake to turn on the whole program optimization, configured CMake to use release and got the following output:

[==========] Running 0 test cases.
[==========] 0 test cases ran.
[ PASSED ] 0 tests.

So this matched what the user found at least!

The Problem

So the issue boils down to a clever little trick I use in utest.h - how it discovers test. Now for those that aren’t aware my utest.h library looks for tests in a specific format:

UTEST(from, c) {
  ASSERT_TRUE(1);
}

And allows you to run them. This works across C or C++ files - if you have T tests spread across F files, when you run the produced executable it’ll run all T tests - even though they started in separate files. I don’t use any custom magic to make this work - so how can I cross compilation units to work out what to run?

I use an idiom that is awful and awesome in equal measures - global constructors. For those used to C++ you’ll be aware that global variables can have constructors to initialize their state. It is general wisdom among those of us burned by the feature to either a) not use global variable constructors at all, or b) if you do make them dumb as possible. This is because the order that all global variables will be initialized is undefined - and thus any interdependency between these variables could cause problems.

What is less well known is that the three major C compilers have ways to add global constructors in C code.

Clang/GCC both support the following:

static void f(void) __attribute__((constructor));

You just tag a function as being a constructor using the attribute mechanism, and it’ll be run during the globals initialization phase.

MSVC (Visual Studio) is a little more complicated, but you can achieve the same thing with:

static void __cdecl f(void);
__declspec(allocate(".CRT$XCU")) void(__cdecl * f_)(void) = f;

What this does is add a function pointer to a special section .CRT$XCU - where all the global initializers are placed. On startup the compiler will run through this list of function pointers to initialize all the global state. It’s aesthetically uglier than the Clang/GCC approach, but it works!

With this key piece of technology we can then use global constructors for each of the UTEST tests to register themselves with a global registry of ’tests that can be run’, effectively just appending themselves to a malloc’ed region. And this is fast - on my 2017 MacBook Pro it takes around 50ms to initialize 502 tests for running, around 99us per test. As an aside, I did an expose of the performance of utest.h and TL;DR it’s blindingly fast.

Back to the problem - remember the problem? MSVC with whole program optimization was not running any tests? So for all intents and purposes these global constructors, the function pointer f_ above, are not actually used in my code. What I mean is that I don’t ever manually call f_ myself, it is just registered and shoved in the global initializer section. So under whole program optimization the compiler comes along and says ‘Oh hey! All these variables are doing stuff and we don’t use them? Get rid!’, meaning that none of the global constructors are then ran, and as a result none of the tests will register themselves with the global test registry and so it looks like there is nothing to run.

Pretty smart of the compiler in some respects, but now we need to defeat it.

The Solution

I was recently made aware of a funky feature of the MSVC compiler thanks to an investigation I did into using rpmalloc with LLVM to improve compile times on multithreaded workloads on Windows. @maniccoder, who knows a thing or two about allocators, gave me this gist on how to do it. The part that was new to me though was:

// Make sure symbols are not purged in linker
#pragma comment(linker, "/include:_rpmllinit_")

So you can use a special MSVC specific pragma to pass options to the linker, and those options can include a special option to always preserve a symbol even if the compiler thinks it is dead. Seems relevant! Now since my UTEST test fixtures use the preprocessor, I cannot use the #pragma variant because it would be invoked at the wrong place. But I knew I could use the __pragma() variant to do the same thing within the preprocessed macro’s body.

So my global test initializer before this change was:

#define UTEST_INITIALIZER(f)                                                   \
  static void __cdecl f(void);                                                 \
  __declspec(allocate(".CRT$XCU")) void(__cdecl * f##_)(void) = f;             \
  static void __cdecl f(void)

And so I just added the __pragma() line to preserve the symbol:

#define UTEST_INITIALIZER(f)                                                   \
  static void __cdecl f(void);                                                 \
  __pragma(comment(linker, "/include:" #f "_"));                               \
  __declspec(allocate(".CRT$XCU")) void(__cdecl * f##_)(void) = f;             \
  static void __cdecl f(void)

And it worked - kinda. So the above worked for all global initializers from tests in a C function compiled for Win64 - 64-bit. It failed for 32-bit executable builds, and for C++ code.

So starting with the C++ tests I realised that even though I’ve compiled the f_ function pointer with the __cdecl calling convention, I hadn’t actually stopped the compiler mangling the variable name which would change the symbol name.

So to fix this I added another macro:

#if defined(__cplusplus)
#define UTEST_C_FUNC extern "C"
#else
#define UTEST_C_FUNC
#endif

That uses extern "C" when C++ is being used, and then modified the initializer to:

#define UTEST_INITIALIZER(f)                                                   \
  static void __cdecl f(void);                                                 \
  __pragma(comment(linker, "/include:" #f "_"));                               \
  UTEST_C_FUNC __declspec(allocate(".CRT$XCU")) void(__cdecl * f##_)(void) = f;\
  static void __cdecl f(void)

Note that it doesn’t matter what the signature of the f function itself is, only that the function pointer global variable does not end up with a mangled name. This fixed the C++ specific link failure.

For the 32-bit windows it was a little funky - but I had a vague memory in my head of dealing with this in a previous life. Basically 32-bit symbols are prepended with an extra underscore _ on the symbol name. I don’t actually know why this is the case to be honest, but I’ve hit this issue before. So to fix that I added:

#if defined(_WIN64)
#define UTEST_SYMBOL_PREFIX
#else
#define UTEST_SYMBOL_PREFIX "_"
#endif

Which will have an additional underscore only on 32-bit builds using MSVC, and then changed the initializer to:

#define UTEST_INITIALIZER(f)                                                   \
  static void __cdecl f(void);                                                 \
  __pragma(comment(linker, "/include:" UTEST_SYMBOL_PREFIX #f "_"));           \
  UTEST_C_FUNC __declspec(allocate(".CRT$XCU")) void(__cdecl * f##_)(void) = f;\
  static void __cdecl f(void)

Which fixed all my build problems. Now utest.h supports whole program optimization correctly.

[==========] Running 502 test cases.
[ RUN      ] utest_cmdline.filter_with_list
[       OK ] utest_cmdline.filter_with_list (4155002ns)
[ RUN      ] c.ASSERT_TRUE
[       OK ] c.ASSERT_TRUE (77ns)
[ RUN      ] c.ASSERT_FALSE
[       OK ] c.ASSERT_FALSE (53ns)

...

[==========] 502 test cases ran.
[  PASSED  ] 502 tests.

use std::process::Command;  // Run programs
use assert_cmd::prelude::*; // Add methods on commands
use predicates::prelude::*; // Used for writing assertions

#[test]
fn file_doesnt_exist() -> Result<(), Box<std::error::Error>> {
    let mut cmd = Command::cargo_bin("grrs")?;
    cmd.arg("foobar")
        .arg("test/file/doesnt/exist");
    cmd.assert()
        .failure()
        .stderr(predicate::str::contains("No such file or directory"));

    Ok(())
}

Basically you use a #[test] inside an integration test (in the tests folder) which uses a std::process to spawn the command line tool you’ve developed. This works, but it has two failings in my opinion:

1. If you want to test the same command line tool with multiple inputs (maybe you want to test that it fails in a specific way?) you’re probably going to add a bunch of tests and include all the boiler plate code to call them.
2. There isn’t a really clean way to check that complex output from your application appears as expected.

I’m coming from the background of writing compilers, where the state of the art for testing is LLVM’s Lit - a tester that combined with the LLVM tool FileCheck lets you do awesome things like:

; RUN: opt < %s -constprop -S | FileCheck %s

; CHECK-LABEL: shift_undef_64
define void @shift_undef_64(i64* %p) {
  %r1 = lshr i64 -1, 4294967296 ; 2^32
  ; CHECK: store i64 undef
  store i64 %r1, i64* %p

  %r2 = ashr i64 -1, 4294967297 ; 2^32 + 1
  ; CHECK: store i64 undef
  store i64 %r2, i64* %p

  %r3 = shl i64 -1, 4294967298 ; 2^32 + 2
  ; CHECK: store i64 undef
  store i64 %r3, i64* %p

  ret void
}

Basically it is:

• Running a program of your choosing with the options that you want.
• The current file is fed as input to the program.
• Then the output is parsed through FileCheck which also takes the same input.
• And it uses special CHECK: statements hidden in comments to assert that the output matches some of the stuff that we expect.

This lets us perform some pretty impressive checks that the output matches what we want, and crucially lets us not check some things, which is pretty useful for the things we don’t care about.

I’m currently working on a compiler framework written in Rust for the craic. I’m not yet at the point where I want to link y’all to the bodgey code and it is in no way usable in any way, shape, or form yet. But one thing I wanted was to be able to unit test the compiler framework by using the command line tools similar to how LLVM did it above. This will let me generate a huge number of tests in a really easy to extend format - I know this because I’ve used LLVM’s tooling for years and can attest it is very powerful.

Attempt 1 - Use LLVM’s Lit?

So my first though was just to use LLVM’s lit tool directly. This quickly fell apart as an idea for a number of reasons:

• The actual lit tooling is a bunch of python scripts, and I don’t want to require anything else beyond Rust.
• The tool that actually does the heavy lifting in lit is the FileCheck tool, which is an executable shipped with LLVM.
• The FileCheck executable is only included in an LLVM you build yourself - you can’t just get it with an LLVM install.

So all in all this didn’t seem like a good way to integrate this sort of testing with Rust. So I was about to try and roll my own when I noticed a really awesome crate lit, so I decided to investigate using it.

The lit crate

So first of all, we need to use the crate. And since we only require the crate for testing we do:

[dev-dependencies]
lit = "0.3.4"

Next up we want to add an integration test that uses this. An integration test can depend on the entire main crate you are developing. In my case I’ve got a library (yair-io) and two executables (yair-as and yair-dis) that use it, and I want to drive these from the integration test. The way you do it is like so:

extern crate lit;

#[cfg(test)]
mod tests {
    use std::env;
    use std::env::consts;
    use std::path::PathBuf;

    fn bin_dir() -> PathBuf {
        env::current_exe()
            .ok()
            .map(|mut path| {
                path.pop();
                path.pop();
                path
            })
            .unwrap()
    }

    fn yair_as_exe() -> String {
        bin_dir()
            .join(format!("yair-as{}", env::consts::EXE_SUFFIX))
            .to_str()
            .unwrap()
            .to_string()
    }

    fn yair_dis_exe() -> String {
        bin_dir()
            .join(format!("yair-dis{}", env::consts::EXE_SUFFIX))
            .to_str()
            .unwrap()
            .to_string()
    }

    #[test]
    fn lit() {
        lit::run::tests(|config| {
            config.add_search_path("tests/lit");
            config.add_extension("ya");

            config.constants.insert("yair_as".to_owned(), yair_as_exe());
            config.constants.insert("yair_dis".to_owned(), yair_dis_exe());
        })
        .expect("Lit tests failed");
    }
}

So lets break this down:

• We use the same #[cfg(test)] that we’d expect from a Rust test.
• And we have a single #[test] tagged function that’ll run our lit tests.
• The lit() function calls into the lit crate to run the tests (lit::run::tests).
  • We need to tell it where our test cases live by using add_search_path to point it at the folder containing your text files to run.
  • And also what file extensions we want to consider ok for testing by using add_extension. In my case my human-readable IR files are like foo.ya so the extension is just ya.
  • And then we need to create some variables that map to where the executables built by rust are located. I add two constants for yair_as and yair_dis to map to where the actual executables exist on disk. To get these locations I use a helper function bin_dir to work out where these executables will be placed on the file-system based on where the integration test is on the file-system.

Note you can add any other constants you want here too for use in the lit scripts. So if you wanted to get the OS being used in the scripts you could add config.constants.insert("os".to_owned(), consts::OS.to_owned()); for instance.

Now we’ve got this you can run cargo test and it’ll have a new integration test that’ll run your lit tests! But since you don’t have a single actual test yet, nothing will actually run yet.

So let’s look at an example of how to write a test case for lit that’ll be picked up by the above integration test.

// RUN: @yair_as @file | @yair_dis -

// CHECK: mod "😀" {
mod "😀" {
  // CHECK: fn foo(a : u8, b : <u8, 2>) : <u8, 2> {
  fn foo(a: u8, b: <u8, 2>) : <u8, 2> {
    // CHECK: b0(v2 : u8, v3 : <u8, 2>):
    bar(a: u8, b: <u8, 2>):
      // CHECK: [[name:\w+]] = insert v3, v2, 1
      c = insert b, a, 1
      // CHECK: ret $$name
      ret c
  }
}

So let’s break down what I’ve done piece by piece:

// RUN: @yair_as @file | @yair_dis -

The first line tells the lit crate how to run this test case by using the RUN: statement. To access constants in lit you use the @foo syntax, so in our case to access the yair-as tool that I setup previously to map to the constant yair_as I use @yair_as to run it. There is a special constant that represents the path to the current file - @file. My yair-as assembler tool takes a positional argument for the input file, so what we are doing here is assembling the current file using my tool, and spitting out the output to the default which is stdout. This produces the binary representation of my intermediate representation, which is kinda hard to test! So what I’m doing is roundtripping it back out via the dissassembler yair-dis (mapped to the @yair_dis constant) and then we can check that the output of the disassembler, which is human readable, matches what the input was.

You’ll note that there is a pipe operator between the two tools, passing the stdout from one into the stdin of the other. Also, the stdout from the total run is what is fed into the lit checks to follow. Also - we do not have a separate FileCheck tool like LLVM uses - which is awesome! It just works, which I really appreciate.

// CHECK: mod "😀" {
mod "😀" {

Next up I have a module 😀 being declared, and I make sure that this is output correctly. You’ll notice that the rest of the file follows a similar pattern - some statements in the original IR matched with some CHECK:’s that ensure that the output was correct. You don’t have to check every line, you could check no lines if you wanted to - but this sparse matching of the output to ensure that some parts of it match what you want is so powerful.

// CHECK: [[name:\w+]] = insert v3, v2, 1
c = insert b, a, 1
// CHECK: ret $$name
ret c

You can also use some more powerful features of checks - variables. Basically you can check that something you’ve matched on earlier appears later in the file. So in the above example I am storing the name of the result from the insert instruction into the variable name, and then getting that name in the subsequent check using $$name. This means that if I changed the naming convention at a later date everything would keep working with the test case - nice!

Lastly - the output from cargo test of lit is as follows:

running 1 test
PASS :: tests/lit/var/basic.ya
PASS :: tests/lit/var/array.ya
PASS :: tests/lit/var/struct.ya
PASS :: tests/lit/var/vector.ya
PASS :: tests/lit/var/export.ya
PASS :: tests/lit/fn/one_arg.ya
PASS :: tests/lit/fn/bad_export.ya
PASS :: tests/lit/fn/one_arg_body.ya
PASS :: tests/lit/fn/export.ya
PASS :: tests/lit/fn/no_args.ya
PASS :: tests/lit/mod/multiple_modules.ya
PASS :: tests/lit/mod/bad_fn_or_var.ya
PASS :: tests/lit/mod/simple.ya
PASS :: tests/lit/mod/missing_closing_brace.ya
PASS :: tests/lit/mod/bad_quoted.ya
PASS :: tests/lit/mod/unicode.ya
PASS :: tests/lit/mod/empty.ya
PASS :: tests/lit/mod/missing_opening_brace.ya
PASS :: tests/lit/mod/missing_module_name.ya
PASS :: tests/lit/mod/bad_simple.ya
PASS :: tests/lit/insts/insert.ya
test tests::lit ... ok

You can see that it appears like a single running test (because like I said previously we are adding a single integration test that powers the lit runs), but the output from that test is a bunch of runs telling you what passes and what fails. If I manually make a test case fail, the output is like so:

FAIL :: tests/lit/insts/insert.ya
tests/lit/insts/insert.ya:15: could not find match: 'something not there'
next line: '}'

Telling me the line with the CHECK that it could not find.

Conclusion

I really love this crate - it is super useful for testing these kind of applications. Full kudos to Dylan McKay for writing such an awesome crate, and I hope I’ve convinced a few more people to try it out.

• Only the summaries of my posts were fed via RSS - not the full contents. I do not do any sort of content tracking on my site because I don’t want to foist vaporware like Google Analytics on my readers, so it really doesn’t matter to me whether people read my content from the RSS feed or from the main site - that they read it is enough!
• The links from the RSS feed were broken.
• I didn’t have an explicit link to signify that my site even supported RSS on the main page.

Let’s look at these in-order.

Full Contents Please!

So I wanted full content rather than summaries of my posts on the RSS feed. Luckily Hugo provides a way to intercept and replace the default RSS template with your own custom one. I followed this guide to try and get it working - but in my RSS reader of choice while the full contents were now being included in the final RSS feed, the HTML tags were being displayed wrongly. I scratched my head for a long while over this before I realised that with RSS 2.0 there are two ways to encode HTML data within an RSS description - and my reader only liked the CDATA approach.

What I ended up with was the following:

<!--https://www.godo.dev/tutorials/hugo-full-text-rss/-->
<!--https://raw.githubusercontent.com/gohugoio/hugo/master/tpl/tplimpl/embedded/templates/_default/rss.xml-->

{{- $pctx := . -}}
{{- if .IsHome -}}{{ $pctx = .Site }}{{- end -}}
{{- $pages := slice -}}
{{- if or $.IsHome $.IsSection -}}
{{- $pages = $pctx.RegularPages -}}
{{- else -}}
{{- $pages = $pctx.Pages -}}
{{- end -}}
{{- $limit := .Site.Config.Services.RSS.Limit -}}
{{- if ge $limit 1 -}}
{{- $pages = $pages | first $limit -}}
{{- end -}}
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>{{ if eq  .Title  .Site.Title }}{{ .Site.Title }}{{ else }}{{ with .Title }}{{.}} on {{ end }}{{ .Site.Title }}{{ end }}</title>
    <link>{{ .Permalink }}</link>
    <description>Recent content {{ if ne  .Title  .Site.Title }}{{ with .Title }}in {{.}} {{ end }}{{ end }}on {{ .Site.Title }}</description>
    <generator>Hugo -- gohugo.io</generator>{{ with .Site.LanguageCode }}
    <language>{{.}}</language>{{end}}{{ with .Site.Author.email }}
    <managingEditor>{{.}}{{ with $.Site.Author.name }} ({{.}}){{end}}</managingEditor>{{end}}{{ with .Site.Author.email }}
    <webMaster>{{.}}{{ with $.Site.Author.name }} ({{.}}){{end}}</webMaster>{{end}}{{ with .Site.Copyright }}
    <copyright>{{.}}</copyright>{{end}}{{ if not .Date.IsZero }}
    <lastBuildDate>{{ .Date.Format "Mon, 02 Jan 2006 15:04:05 -0700" | safeHTML }}</lastBuildDate>{{ end }}
    {{ with .OutputFormats.Get "RSS" }}
      {{ printf "<atom:link href=%q rel=\"self\" type=%q />" .Permalink .MediaType | safeHTML }}
    {{ end }}
    {{ range $pages }}
    <item>
      <title>{{ .Title }}</title>
      <link>{{ .Permalink }}</link>
      <pubDate>{{ .Date.Format "Mon, 02 Jan 2006 15:04:05 -0700" | safeHTML }}</pubDate>
      {{ with .Site.Author.email }}<author>{{.}}{{ with $.Site.Author.name }} ({{.}}){{end}}</author>{{end}}
      <guid>{{ .Permalink }}</guid>
      <description>{{ safeHTML "<![CDATA[" }}{{- .Content | safeHTML -}}{{ safeHTML "]]>" }}</description>
    </item>
    {{ end }}
  </channel>
</rss>

With the only thing I changed was the line:

<description>{{ safeHTML "<![CDATA[" }}{{- .Content | safeHTML -}}{{ safeHTML "]]>" }}</description>

What this does is dump the original HTML content as is (that’s what the safeHTML command does) and wrap it in the <![CDATA[ and ]]> tags. This meant that my content was being displayed correctly.

I then just had to place the above in <root>/layouts/_default/rss.xml et voila, it worked!

Links Should Really Work

All my links in the RSS were broken - the links to my main site and also the links to image content on my main blog. I don’t quite know how I forgot to do this (or how my site actually worked for as long as it did), but I had forgotten to specify the baseurl in the config.toml. Just pointing this at my site fixed the issue.

Advertising RSS Support

Although my site always (poorly) supported RSS, I never advertised it. To solve this I’ve just added an RSS badge to the footer of the page to help people find the feed that want it. Nothing fancy, just an image and a link.

Wrapping It All Up

Now I have fully working RSS - and thanks to a very helpful push from an avid reader (thanks Augustin!) I’ve also enabled SSL/https on my site - and made it the primary way to view my content. My hosting provider Gandi has offered simple SSL integration for a while now, but I had to upgrade my small instance to small+SSL, and I just hadn’t gotten round to it. But no more - I now serve secure content by default!

Sorry for any RSS fans that’ve been maligned for a while now, but hopefully this post will get my RSS subscriptions to rise now that my content is displayed in all its glory.

You can find the RSS feed at this link https://www.neilhenning.dev/index.xml.

Tioraidh a chàirdean!

A ton of people worldwide have just plunged into the icy water of working from home, and since I’ve been doing this for six years I have some advice from the good and the bad of what I learned over the time.

In this post I’ll attempt to cover some of these lessons to try and help y’all who’ve had the cold water treatment learn how to swim.

Don’t Work In Your Pyjamas

The number one mistake people make when working from home is to pour yourself out of bed and immediately sit at your new desk and begin work. Don’t do this!

I personally like to get a sweat on every day either out running with my dog or if I’m housebound then on my exercise bike. So my routine is that I get up, change into my workout gear, take the dog outside for his morning duties, and make a coffee. This routine takes me around twenty minutes or so and gives me some distance from sleeping Neil to working Neil.

If you aren’t an exercise-aholic like myself then I recommend you shower and get changed into ‘work gear’ before sitting at your new desk.

Take Lots of Regular Breaks

The number one mistake I made when I first started working remotely was to not take enough breaks. If you think back to any office environment you have worked in, I’d take a guess that around twenty-five minutes out of every hour has some non-screen interaction. People coming and going as they get a drink / use the toilet / talk to random people about random things. Even if you are one of those introverted headphone wearing office dwellers (like I was) you are still generally getting a ton of visual distractions as people move around you that give you a break from the screen.

When people switch from this to working remotely there is two things that bite - a fear of missing out on interactions that have now moved to chat apps or calls, and a worry that people are convinced you aren’t actually working if you don’t respond instantly to their direct interactions with you. These two things combined can let you almost become magnetized to your chair.

Don’t do this! Think about the office environment - how many times have you looked for a colleague at their desk and they aren’t there. You don’t immediately think ‘This bampot is skiving!’. The same is true for remote working, so give yourself a break.

Trust Your Staff

I’ve already seen a bunch of leads worried that they won’t know what their staff are up to, worried they won’t work on the right things, worried they will go off into the weeds when the grass just needs cut. I’ve seen pitches from extrovert control freaks that all staff should be forced into a continuously running group call for the entire working day to simulate the working environment. What this all boils down to is that you don’t trust the people working for you - which honestly is a much deeper problem than them being at home. This screams to me of a failure of hiring the wrong person or a failure of not training the person.

When I was at AMD I learned that having an asynchronous way for people to know what you are working on is key. The entire game engineering part of the org I was part of would keep weekly logs on confluence of what we were doing, which let people across the org and actually across the company see what we were doing. This is a great way to keep people updated on what you are doing and also letting you develop side interactions about your work.

When I joined Unity I brought this with me, and keep a running yearly log on all the things I’ve been up to. It provides a great way for remote people to feel connected to what other people are doing and have side conversations on the work.

Stop Working

The thing I still struggle with the most isn’t a lack of working, it is working too much. When you were in the office and had to leave at five to have dinner with the family, that forced a physical break between you and your ability to work. When the computer is in the same dwelling as you the temptation to say ‘I’ll just finish this one other thing that has been annoying me’ is a fight. Especially when it is a bug you have caused. Just this week at Unity I found two bugs in some complication optimizations I’ve been working on, and I found myself itching to keep working on them late into the night because I felt guilty that I was holding up a release of Burst.

Don’t do this! I need to learn this lesson fully myself, but since we all know crunching is bad, crunching in the home can’t be any better.

Don’t Conference Call During Your Lunch Break

One thing I definitely advocate is to take your lunch break away from your desk or computer. Get some fresh air, do something else round the house, socialize with your equally quarantined partner / cat / Rocket League friends.

One thing I think you should never never never do is have some form of ‘Let’s all eat lunch together on a conference call to scratch the social itch.’. It is totally ok to have a social-only set of interactions with your co-workers, but that should be on the companies time. Think back to when you are in the office - how much time in a day do you spend during company time interacting with your co-workers about non-company things? Think of all the watercooler or coffee-making moments where you have a five minute stop and chat. You don’t clock out to do this, so why should you sacrifice your contractually allowed lunch break to recreate the same just because you are remote?

Like I said before, I use my lunch break to get out of the house. I walk the dog, go for a run, exercise on my bike. Hell I’ve even taken a bath in my lunch hour. Do anything you can to get away from the desk for an hour.

Lean Into Your New Super Power

I found working from home to be utterly liberating. I’m a highly socially capable introvert - anyone who has met me will know I am life-and-soul of any party, but I naturally want to be away in the wilds with my dog away from people. I need a huge period of time to recharge from social interactions - a week away for work would need two months for me to recover.

For me the ability to go full focus when I started working from home was so empowering. I could go into a zen like zone that I’d never been able to in the office. I honestly reckon that I was operating about 10% of my peak performance when I was in the office. My productivity and happiness at how I’m getting to really think through the problems I get to work on has been super charged since I’ve been remote. Remote working is my super power - it’s how I get to be as good at what I do as I am.

Don’t Let Real Life Invade Your Work

It’s so easy for real life to invade your work when you are at home. This was a hard adjustment for me and my partner. ‘Oh you are at home so you can do X around the house!’ - no, no you can’t. It’s very easy to see that because the person is at home they can do the dishes, clean the home, look after the kids, whatever. It’s super key that you resist this urge otherwise you won’t succeed at home. It is better to just have a blanket ban on doing anything around the house during work hours that you wouldn’t have been able to do in the office, at least to start with. Have a seperation of work-you and home-you - even though they are sharing the same location!

Take Advantage of the Lack of Commute

If you are like the average office worker you’ll have somewhere around an hour commute built into your day. Now you’ve just got that time back to do what you want with. Read books, learn a language, play Witcher III some more, build Lego with the kids - whatever you always wanted to do but didn’t have time? Now is the time to do it. So liberating to get that dead time back!

Summary

This isn’t an exhaustive list of all the things that’ll make remote working work for you - just a meandering list of some of the things I’ve learned along the way. For me working from home has been a game changer in my life. I’m happier, healthier, I got a dog that I get to spend every minute of the day with, I bought a house in the stunning Isle of Skye with views to die for, and live in a close knit community. I’ve built social interactions outwith my work colleagues into my life which let me broaden my view points.

If you are in an enforced Coronavirus-derived work from home then I encourage you to lean into it and see if its for you. I realise that some extroverted people will miss the chaos of the office, but if you’ve been one of those introverts that has been forced to live in the extroverted world this is the time to see if working from home would allow you to be happier, healthier, and honestly a better worker. It worked for me at least!

So I wondered - is there anyway to do something similar for my single-header libraries? I decided to try with json.h.

The Code Samples

I want to have more and more code samples to show how easy the library is to use, but the big one there currently is:

 const char json[] = "{\"a\" : true, \"b\" : [false, null, \"foo\"]}";
 struct json_value_s* root = json_parse(json, strlen(json));
 assert(root->type == json_type_object);

 struct json_object_s* object = (struct json_object_s*)root->payload;
 assert(object->length == 2);

 struct json_object_element_s* a = object->start;

 struct json_string_s* a_name = a->name;
 assert(0 == strcmp(a_name->string, "a"));
 assert(a_name->string_size == strlen("a"));

 struct json_value_s* a_value = a->value;
 assert(a_value->type == json_type_true);
 assert(a_value->payload == NULL);

 struct json_object_element_s* b = a->next;
 assert(b->next == NULL);

 struct json_string_s* b_name = b->name;
 assert(0 == strcmp(b_name->string, "b"));
 assert(b_name->string_size == strlen("b"));

 struct json_value_s* b_value = b->value;
 assert(b_value->type == json_type_array);

 struct json_array_s* array = (struct json_array_s*)b_value->payload;
 assert(array->length == 3);

 struct json_array_element_s* b_1st = array->start;

 struct json_value_s* b_1st_value = b_1st->value;
 assert(b_1st_value->type == json_type_false);
 assert(b_1st_value->payload == NULL);

 struct json_array_element_s* b_2nd = b_1st->next;

 struct json_value_s* b_2nd_value = b_2nd->value;
 assert(b_2nd_value->type == json_type_null);
 assert(b_2nd_value->payload == NULL);

 struct json_array_element_s* b_3rd = b_2nd->next;
 assert(b_3rd->next == NULL);

 struct json_value_s* b_3rd_value = b_3rd->value;
 assert(b_3rd_value->type == json_type_string);

 struct json_string_s* string = (struct json_string_s*)b_3rd_value->payload;
 assert(0 == strcmp(string->string, "foo"));
 assert(string->string_size == strlen("foo"));

 /* Don't forget to free the one allocation! */
 free(root);

I want to be able to parse the README.md, extract the code samples, turn them into a test for use with my utest.h library will run. I already use CMake for building just the unit-tests, and so given I already use it and I’m pretty familiar with it (despite its glaring flaws), I wondered if I could use it to do the extraction.

CMake of Horrors

So CMake has built-in regex string support so I thought could I use that to do the extraction? The one big issue is that CMake only supports greedy matching of regex - meaning that I have to be super careful when searching for start/end tokens with which to match.

First of all we need to read the whole file into a CMake variable:

file(READ ${CMAKE_CURRENT_SOURCE_DIR}/../README.md readme_md)

CMake has this wonderfully messed up method for differentiating between strings and lists - where a list in CMake terminology is just a string that has semi-colons within it. The problem is that code samples in languages like C use semi-colons as end of statement terminators - which will cause us issues. The best way I’ve found around this is to change the semi-colons to some symbol that wouldn’t appear in the original source. I used the ‘@’ symbol for this since there isn’t an operator in C for it:

string(REPLACE ";" "@" readme_md "${readme_md}")

Ok now we have the string as a real string (non-list) we can extract the code samples themselves. You’ll notice that in the README.md all code samples begin with “```c” and end with “```”. So we can use this to look for our code.

As I said earlier CMake is greedy when it comes to regex, which means if we used the more natural “.*” we’d match from the very first code sample to the very last in the file. Not ideal. Instead we need to use the more constrained search of “```c[^`]*```” - search for the start pattern, and then all symbols except a “`” until we get to our end. This stores each match as a list entry into the variable snippets - meaning we have introdued some semi-colons into the string too:

string(REGEX MATCHALL "```c[^`]*```" snippets "${readme_md}")

Now to help us be able to test the examples we want to be able to compile each of the code snippets in isolation from each other. I first attempted to create a UTEST(foo, bar) wrapper around each snippet, but I could not figure out how to create these wrappers such that they would be unique. What I mean is that the first snippet would be UTEST(generated, snippet0) the next UTEST(generated, snippet1), etc. For the life of me I couldn’t work out how this was possible. So instead I just wrapped each snippet into its own braced region which guaranteed their isolation.

string(REPLACE "```c" "{" snippets ${snippets})
string(REPLACE "```" "}\n\n" snippets ${snippets})

Now all we need to do is remove the semi-colons that were added for the lists, and then turn all “@” symbols we introduced before back into semi-colons:

string(REPLACE ";" "" snippets "${snippets}")
string(REPLACE "@" ";" snippets "${snippets}")

And then we just need to write out the file into some location for inclusion:

file(WRITE ${CMAKE_CURRENT_BINARY_DIR}/generated.h "${snippets}")

Sample generated.h

For the current master json.h, the generated.h file is:

{
struct json_value_s *json_parse(
    const void *src,
    size_t src_size);
}

{
struct json_value_s {
  void *payload;
  size_t type;
};
}

{
struct json_value_s *json_parse_ex(
    const void *src,
    size_t src_size,
    size_t flags_bitset,
    void*(*alloc_func_ptr)(void *, size_t),
    void *user_data,
    struct json_parse_result_s *result);
}

{
enum json_parse_flags_e {
  json_parse_flags_default = 0,
  json_parse_flags_allow_trailing_comma = 0x1,
  json_parse_flags_allow_unquoted_keys = 0x2,
  json_parse_flags_allow_global_object = 0x4,
  json_parse_flags_allow_equals_in_object = 0x8,
  json_parse_flags_allow_no_commas = 0x10,
  json_parse_flags_allow_c_style_comments = 0x20,
  json_parse_flags_deprecated = 0x40,
  json_parse_flags_allow_location_information = 0x80,
  json_parse_flags_allow_single_quoted_strings = 0x100,
  json_parse_flags_allow_hexadecimal_numbers = 0x200,
  json_parse_flags_allow_leading_plus_sign = 0x400,
  json_parse_flags_allow_leading_or_trailing_decimal_point = 0x800,
  json_parse_flags_allow_inf_and_nan = 0x1000,
  json_parse_flags_allow_multi_line_strings = 0x2000,
  json_parse_flags_allow_simplified_json =
      (json_parse_flags_allow_trailing_comma |
       json_parse_flags_allow_unquoted_keys |
       json_parse_flags_allow_global_object |
       json_parse_flags_allow_equals_in_object |
       json_parse_flags_allow_no_commas),
  json_parse_flags_allow_json5 =
      (json_parse_flags_allow_trailing_comma |
       json_parse_flags_allow_unquoted_keys |
       json_parse_flags_allow_c_style_comments |
       json_parse_flags_allow_single_quoted_strings |
       json_parse_flags_allow_hexadecimal_numbers |
       json_parse_flags_allow_leading_plus_sign |
       json_parse_flags_allow_leading_or_trailing_decimal_point |
       json_parse_flags_allow_inf_and_nan |
       json_parse_flags_allow_multi_line_strings)
};
}

{
const char json[] = "{\"a\" : true, \"b\" : [false, null, \"foo\"]}";
struct json_value_s* root = json_parse(json, strlen(json));
assert(root->type == json_type_object);

struct json_object_s* object = (struct json_object_s*)root->payload;
assert(object->length == 2);

struct json_object_element_s* a = object->start;

struct json_string_s* a_name = a->name;
assert(0 == strcmp(a_name->string, "a"));
assert(a_name->string_size == strlen("a"));

struct json_value_s* a_value = a->value;
assert(a_value->type == json_type_true);
assert(a_value->payload == NULL);

struct json_object_element_s* b = a->next;
assert(b->next == NULL);

struct json_string_s* b_name = b->name;
assert(0 == strcmp(b_name->string, "b"));
assert(b_name->string_size == strlen("b"));

struct json_value_s* b_value = b->value;
assert(b_value->type == json_type_array);

struct json_array_s* array = (struct json_array_s*)b_value->payload;
assert(array->length == 3);

struct json_array_element_s* b_1st = array->start;

struct json_value_s* b_1st_value = b_1st->value;
assert(b_1st_value->type == json_type_false);
assert(b_1st_value->payload == NULL);

struct json_array_element_s* b_2nd = b_1st->next;

struct json_value_s* b_2nd_value = b_2nd->value;
assert(b_2nd_value->type == json_type_null);
assert(b_2nd_value->payload == NULL);

struct json_array_element_s* b_3rd = b_2nd->next;
assert(b_3rd->next == NULL);

struct json_value_s* b_3rd_value = b_3rd->value;
assert(b_3rd_value->type == json_type_string);

struct json_string_s* string = (struct json_string_s*)b_3rd_value->payload;
assert(0 == strcmp(string->string, "foo"));
assert(string->string_size == strlen("foo"));

/* Don't forget to free the one allocation! */
free(root);
}

{
const char json[] = "{\"a\" : true, \"b\" : [false, null, \"foo\"]}";
struct json_value_s* root = json_parse(json, strlen(json));

struct json_object_s* object = json_value_as_object(root);
assert(object != NULL);
assert(object->length == 2);

struct json_object_element_s* a = object->start;

struct json_string_s* a_name = a->name;
assert(0 == strcmp(a_name->string, "a"));
assert(a_name->string_size == strlen("a"));

struct json_value_s* a_value = a->value;
assert(json_value_is_true(a_value));

struct json_object_element_s* b = a->next;
assert(b->next == NULL);

struct json_string_s* b_name = b->name;
assert(0 == strcmp(b_name->string, "b"));
assert(b_name->string_size == strlen("b"));

struct json_array_s* array = json_value_as_array(b->value);
assert(array->length == 3);

struct json_array_element_s* b_1st = array->start;

struct json_value_s* b_1st_value = b_1st->value;
assert(json_value_is_false(b_1st_value));

struct json_array_element_s* b_2nd = b_1st->next;

struct json_value_s* b_2nd_value = b_2nd->value;
assert(json_value_is_null(b_2nd_value));

struct json_array_element_s* b_3rd = b_2nd->next;
assert(b_3rd->next == NULL);

struct json_string_s* string = json_value_as_string(b_3rd->value);
assert(string != NULL);
assert(0 == strcmp(string->string, "foo"));
assert(string->string_size == strlen("foo"));

/* Don't forget to free the one allocation! */
free(root);
}

I wanted to keep normal assert.h assert’s in the sample source, but I also want these to be turned into my utest.h ASSERT_TRUE macros, so I just use the preprocessor to define these over, and include the source into the test:

#define assert(x) ASSERT_TRUE(x)

UTEST(generated, readme) {
#include "generated.h"
}

And the output when I run?

[ RUN      ] generated.readme
[       OK ] generated.readme (9715ns)

A pass!

Conclusion

Ok - it is not as nice as what Rust has built-in, but it works! I can now modify the README.md and be sure that the code compiles correctly. I even found a bug in the sample in the process, so well worth the work. Just a shame I had to invest in proper demonology to support this within the C eco-system.

C89 Support

One requested thing over the years has been C89 support, and I was reluctant to do this mostly out of sheer laziness. I’d have to convert a bunch of comments over and remove my use of long long. This was my entire reasons for not doing it, which was dumb in hindsight. So I’ve got it working now for the people that really really care.

Single Header Support

I always maintained that having one header and one source file was the best design decision for the library. It is a big implementation in the source file, and I mostly have a distaste for having a header file where you have to do some preprocessor define in a single source file.

But single headers are so super useful for users to just copy the one file and ship it.

So I’ve done made the change to bring it into a separate header.

This will definitely result in a slower compile time when bringing in the header, but I realised that in reality you will probably only include json.h in a single file when you are interacting with the JSON. So it seemed fine overall for this as a compromise.

process.h -> subprocess.h

I’ve had a long standing complaint that the name process.h conflicted with a Windows system header. I’ve decided (after a good three years of indecision mind you) to just rename the header and the entry points to subprocess.h instead.

This isn’t a perfect solution, but it avoids me having to namespace my headers using my own name (which as a Scotsman I just can’t quite stomach for some reason).

utest.h

I added the ability to assert/expect on pointers types, which was a longstanding annoyance. So the following now works:

int* a, *b;
ASSERT_EQ(a, b);

I’ve also made it so that on GCC/Clang with C, or with C++11 using auto, you can perform checks on void-pointers, and things of array type:

static int data[42];

void* a;
ASSERT_EQ(a, data);

I also made the testing of the --list-tests option that was contributed by ldrumm (which required spawning a separate process) work on Windows and macOS too by using my subprocess.h.

I also added a product page for utest.h to help me try and advertise it more greatly for people than before.

utf8.h

I fixed a bug with utf8casecmp in my UTF-8 parsing library and added some extra tests to stop the regression happening again.

The ‘bug’ wasn’t really a bug in that the function behaved in a similar fashion to the specification of the GNU extension strcasecmp that it is based on, but it made sense to make utf8casecmp match what strcasecmp does.

• It only works with modern C++.
• It has a complex build system integration, with multiple source files and headers.
• The startup time of using googletest is terrible.

So with all this in mind I wrote my own library - utest.h. The library is:

• 9x faster than googletest.
• Works with C and C++.
• Works with mixing C and C++ in the same testing application (very useful if you want to test a C++ and C API to your code).
• Has test fixtures.
• Allows filtering of test cases.
• Compiles with -Wall -Werror -Weverything.

Lets look at an example application:

In a single file (C or C++, it doesn’t matter) you need to need to call the macro UTEST_MAIN() to create the main method for the application.

#include "utest.h"

UTEST_MAIN()

Note this doesn’t need to be in a separate C/C++ file to the actual tests!

Then we’ll have two files - a C and a C++ file just to show that it works.

#include "utest.h"

UTEST(from, c) {
  ASSERT_TRUE(1);
}

And:

#include "utest.h"

UTEST(from, cpp) {
  ASSERT_EQ(true, false);
}

Now we’ll compile this all together with clang and show the output.

Anyone who is familiar with googletest will notice that I’ve intentionally kept the output similar (even colour highlighted) to stop any mental gymnastics that could be involved.

Lastly the license - the code is as close to public domain as I could get in a license, using the unlicense to let any of you do basically anything you want with it.

This post is the latest in a series about my experimentations with floating-point scalar evolution, you really want to read them in order:

• An Experimental Floating-Point Scalar Evolution
• Using Floating-Point Scalar Evolution to Propagate Fast-Math Flags
• Using Floating-Point Scalar Evolution to Simplify Instructions

The fpscev pass changes discussed in this post is available on github here.

Improving the Range

When I wrote the first draft of the fpscev analysis, any functionality that LLVM’s APFloat did not contain I simply assumed the worst case for the range of the input (-NaN..NaN unless fast-math flags were specified). For instance, the code used to evaluate the range of exp2 was:

FPSCEV fpscev(inst.getType());

// Exp always has a positive result.
const APFloat zero = APFloat::getZero(fpscev.min.getSemantics(), false);
fpscev.min = zero;

const FastMathFlags flags = inst.getFastMathFlags();
fpscev.max = applyFastMathFlags(fpscev.max, flags);

fpse.map[&inst] = fpscev;

I could do no better because APFloat did not contain an exp2 call that I could make use of to evaluate the range. After having some twitter discussions with the rather awesome Marc B. Reynolds, I realised I should revisit my assumption on the ranges of some of these intrinsics to see if I couldn’t make a better guess than what was there.

Exp2

For exponentials with a base of 2 as I’ve already stated there isn’t an LLVM APFloat function that I can just call into. There is something sort of similar that I can use though to estimate the range such that we can return something better than 0..NaN! LLVM APFloat does contain a scalbn definition for APFloat. Scalbn allows you to efficiently multiply a value by 2-to-the-power-of another value - x * 2^y. If x just happened to be 1.0 - then we are effectively calculating 2^y. This just so happens to be exp2! The one downside is that we can only specify a y that is an integer - so we will lose some precision on the range by doing this.

FPSCEV exp2Bounds(FPSCEV fpscev) {
  const fltSemantics &semantics = fpscev.getSemantics();
  const APFloat one = getOne(semantics);

  bool losesInfo;

  if (fpscev.min.isFinite()) {
    fpscev.min.roundToIntegral(APFloat::rmTowardNegative);
    fpscev.min.convert(APFloat::IEEEdouble(), APFloat::rmTowardNegative,
                       &losesInfo);

    const int power = static_cast<int>(fpscev.min.convertToDouble());

    fpscev.min = scalbn(one, power, APFloat::rmTowardNegative);
  } else {
    fpscev.min = APFloat::getInf(semantics, true);
  }

  if (fpscev.max.isFinite()) {
    fpscev.max.roundToIntegral(APFloat::rmTowardPositive);
    fpscev.max.convert(APFloat::IEEEdouble(), APFloat::rmTowardPositive,
                       &losesInfo);

    const int power = static_cast<int>(fpscev.max.convertToDouble());

    fpscev.max = scalbn(one, power, APFloat::rmTowardPositive);
  } else {
    fpscev.max = APFloat::getInf(semantics, false);
  }

  return fpscev;
}

The approach I take is:

1. Round the minimum range value down to the next whole integer, the maximum range up to the next whole integer.
2. Convert them to doubles rounding the values away from zero again (this only really matters in the case where we are evaluating a quad-precision number that isn’t fully representable in 64-bits of precision).
3. Convert these doubles to integer (as scalbn takes an int y). I rely on the fact that compilers generally round an overflowing double to INT_MAX/INT_MIN which will easily create an infinity/-infinity when we raise that value by the power of 2.
4. Then use scalbn like I stated above to create the bounds.
5. Note: I only attempt this approach at all if the min/max value were finite in the first place.

Lets have a look at the range of this exp2 value plotted across some key values to understand how accurate our evaluation is:

The chart above firstly shows that the actual value of exp2 if we could calculate it correctly lies inclusively within the calculated min/max ranges we’ve established using scalbn. So the result is always correct. You can also see that the higher the input value to the exp2 function, the more imprecise our range analysis becomes (as exp2 gets more, well, exponential the higher the input value this is to be expected).

All in all - this is pretty good! Much better than always returning a range of 0..NaN anyway.

Log2

For log2 I explored a similar idea to the solution I used for exp2 above. While LLVM APFloat doesn’t have a suitable log2 equivalent function, it does have ilogb though. The ilogb function effectively extracts the exponent from the floating-point number without fudging it like which happens with frexp (frexp always returns a number between 0.5..1, and so has to fudge the exponent it returns to make sure of that). Because ilogb loses the fractional part of our floating-point number, we can use ilogb of the minimum-range to get the lower bound of our result. For the upper bound, we use ilogb and then add one onto it - this rounds up the estimation to the next highest whole number.

FPSCEV log2Bounds(FPSCEV fpscev) {
  const fltSemantics &semantics = fpscev.getSemantics();

  // If the full range is negative, always returns a NaN.
  if (fpscev.isAllNegative()) {
    APFloat nan = APFloat::getNaN(semantics, false);
    return FPSCEV(nan, nan, false);
  }

  const APInt ilogbMin(32, ilogb(fpscev.min));
  // We do +1 here so we round the range up.
  const APInt ilogbMax(32, ilogb(fpscev.max) + 1);

  APFloat values[2] = {getFromInt(semantics, ilogbMin, true),
                       getFromInt(semantics, ilogbMax, true)};

  // If the input is entirely greater than zero, we can use ilogb to get a much
  // firmer estimate on the log result.
  if (fpscev.isGreaterThan(APFloat::getZero(semantics))) {
    fpscev.min = getMinimum(values);
    fpscev.max = getMaximum(values);
  } else {
    fpscev.min = APFloat::getInf(semantics, true);
    fpscev.max = values[1];
  }

  // Can't be sure about this because we're getting very vague bounds, so wipe
  // it.
  fpscev.isInteger = false;

  return fpscev;
}

Lets have a look at the chart for this to show how good our bounds are:

As can be seen - the actual result is entirely within our calculated min/max range, and because logarithms grow so slowly the range isn’t so bad (as compared to our exponential prediction from before).

Log & Log10

Logarithms have some very fun rules - one of which is the change of base formula. For those unfamiliar, any log can be transfered into any other log calculation by loga(x) = logb(x) / logb(a). This means we can use the calculation of the bounds we already discovered for log2 to calulate a bound for any other log! I use this same trick for log and log10 - so I’ll just cover log here to keep this succinct.

template <>
void FPScalarEvolutionPass::visitIntrinsic<Intrinsic::log>(
    IntrinsicInst &inst) {
  const FastMathFlags fmf = inst.getFastMathFlags();

  FPSCEV fpscev =
      fpse.getFPSCEV(inst.getOperand(0))->cloneWithFastMathFlags(fmf);

  fpscev = log2Bounds(fpscev);

  // We are calculating log(x), but we only have ilogb (which gives us an
  // effective lower bound of log2(x)). To convert this bound into log(x) by
  // multiplying it by log(2).
  APFloat converter(
      0.693147180559945309417232121458176568075500134360255254120);
  bool losesInfo;
  converter.convert(fpscev.getSemantics(), APFloat::rmNearestTiesToEven,
                    &losesInfo);

  fpscev.min.multiply(converter, APFloat::rmTowardNegative);
  fpscev.max.multiply(converter, APFloat::rmTowardPositive);

  fpse.map[&inst] = fpscev.cloneWithFastMathFlags(fmf);
}

I basically calculate the log2 range of the input, and then use the change of base formula for logarithms to turn this into the bounds for log instead. Lets look at the chart:

Pretty good! The bounds follow a similar trajectory to the previous log2 calculation, but in this newer range of log.

Pow

Anyone who has tried to implement pow(x, y) will know that there is a common hack that gets touted as some miracle solution to this calculation - turning pow(x, y) into exp2(y * log2(x)). There are numerous reasons why this isn’t generally a good idea (the precision loss is terrible when you do the y * log2(x), you have to handle a negative x to an integer y that might be odd/even and this affects the sign of the result, etc) - but since we have no alterative and we are ok to lose a bit of precision as long as we lose the precision correctly (EG. as long as we always grow the range through precision loss rather than accidentally shrink it).

Pow has a lot of chances to go awry (fractional bases, fractional powers, negatives) - so I play it very safe:

// For pow we use exp2(y * log2(x)) to get the best bounds we can hope for.
template <>
void FPScalarEvolutionPass::visitIntrinsic<Intrinsic::pow>(
    IntrinsicInst &inst) {
  const FastMathFlags fmf = inst.getFastMathFlags();

  const FPSCEV xFpscev =
      fpse.getFPSCEV(inst.getOperand(0))->cloneWithFastMathFlags(fmf);
  const FPSCEV yFpscev =
      fpse.getFPSCEV(inst.getOperand(1))->cloneWithFastMathFlags(fmf);

  FPSCEV log2Fpscev = log2Bounds(xFpscev);
  FPSCEV fpscevs[4] = {log2Fpscev, log2Fpscev, log2Fpscev, log2Fpscev};

  fpscevs[0].min.multiply(yFpscev.min, APFloat::rmTowardNegative);
  fpscevs[0].max.multiply(yFpscev.min, APFloat::rmTowardNegative);

  fpscevs[1].min.multiply(yFpscev.max, APFloat::rmTowardNegative);
  fpscevs[1].max.multiply(yFpscev.max, APFloat::rmTowardNegative);

  fpscevs[2].min.multiply(yFpscev.min, APFloat::rmTowardPositive);
  fpscevs[2].max.multiply(yFpscev.min, APFloat::rmTowardPositive);

  fpscevs[3].min.multiply(yFpscev.max, APFloat::rmTowardPositive);
  fpscevs[3].max.multiply(yFpscev.max, APFloat::rmTowardPositive);

  fpscevs[0] = exp2Bounds(fpscevs[0]);
  fpscevs[1] = exp2Bounds(fpscevs[1]);
  fpscevs[2] = exp2Bounds(fpscevs[2]);
  fpscevs[3] = exp2Bounds(fpscevs[3]);

  FPSCEV result;

  result.min = getMinimum({fpscevs[0].min, fpscevs[0].max, fpscevs[1].min,
                           fpscevs[1].max, fpscevs[2].min, fpscevs[2].max,
                           fpscevs[3].min, fpscevs[3].max});
  result.max = getMaximum({fpscevs[0].min, fpscevs[0].max, fpscevs[1].min,
                           fpscevs[1].max, fpscevs[2].min, fpscevs[2].max,
                           fpscevs[3].min, fpscevs[3].max});
  result.isInteger = false;

  fpse.map[&inst] = result.cloneWithFastMathFlags(fmf);
}

The approach is basically:

1. Get the log2 bounds of the input x.
2. Multiply this by the bounds for y resulting in 8 multiplications for the whole space of the pairings.
3. Feed these multiplications into the exp2 bounds.
4. Then work out the minimum/maximum of all the resulting values.

This gets us a very imprecise result, as the following chart shows:

It’s really very terribly imprecise - we are losing so much precision by the fact our exp and log are so very imprecise too. While this isn’t great - it is still better than having pow always return -NaN..NaN - at least we have some semblance of a range!

Sqrt

Sqrt is one of the few functions that the C stdlib version is required to have 0.5 ulp (EG. the same precision as basic mathematical operators like + or -). This means, as long as we carefully round the min/max ranges as we translate from the APFloat to double, we can use the C stdlib sqrt implementation for our range calculation - huzzah!

FPSCEV sqrtBounds(FPSCEV fpscev) {
  const fltSemantics &semantics = fpscev.getSemantics();

  // If the full range is negative, always returns a NaN.
  if (fpscev.isAllNegative()) {
    APFloat nan = APFloat::getNaN(semantics, false);
    return FPSCEV(nan, nan, false);
  }

  bool losesInfo;

  if (fpscev.isAllNonNegative()) {
    fpscev.min.convert(APFloat::IEEEdouble(), APFloat::rmTowardNegative,
                       &losesInfo);

    // Get the next number less than the current (unless we are zero).
    if (!fpscev.min.isZero()) {
      fpscev.min.next(true);
    }
    fpscev.min = APFloat(std::sqrt(fpscev.min.convertToDouble()));
    fpscev.min.convert(semantics, APFloat::rmTowardNegative, &losesInfo);
  }

  // If the whole fpscev is not negative, it means max must be positive.
  if (!fpscev.isAllNegative()) {
    fpscev.max.convert(APFloat::IEEEdouble(), APFloat::rmTowardPositive,
                       &losesInfo);
    fpscev.max.next(false);
    fpscev.max = APFloat(std::sqrt(fpscev.max.convertToDouble()));
    fpscev.max.convert(semantics, APFloat::rmTowardPositive, &losesInfo);
  }

  fpscev.isInteger = false;

  return fpscev;
}

The basic approach is that we:

1. Check if the entire fpscev is not negative (meaning that min is definitely going to produce a non-NaN result).
2. If so - convert the minimum value to APFloat’s double representation. This just allows us to convert to double using the convertToDouble method.
3. If the converted value is not 0, then we move it over to the next smallest floating-point value using the next method (akin to nextafter in C).
4. Convert to double, use std::sqrt, then convert back, rounding towards negative as we convert back to the original semantics (ensuring we grow the min range on ties).
5. Check if the fpscev is not all negative (meaning that max is definitely going to produce a non-NaN result).
6. Do the same approach as for min, but use next to move the value up, and convert with rounding towards positive as we convert back (to grow the range).

This means we might get a slightly off result, but the result will be pretty strongly bound to what sqrt would produce.

For instance, for an input between 0..511, we produce an output range of:

min: 0
max: 22.6053104
isInteger: 0

Which is much better than the 0..NaN we have previously.

Conclusion

With this improvement I’ve managed to really tighten up some of the functions I couldn’t get a good handle on previously.

In the next post I’m going to look at if it is possible to make our fpscev’s aware of where they are used, and see if we can’t constrain the range even further. Watch this space!

It is also the end of my five year involvement in Vulkan & SPIR-V - having contributed to both specifications since 2014.

I’ve very much enjoyed my time in this slice of the industry, but it was time for a new challenge.

Farewell AMD

I’ve been at AMD a little over a year now - and it has been an interesting challenge. I was brought in to improve the performance of the LLVM compiler stack (LLPC) as compared to the legacy Shader Compiler (SC). All-in during my time there I improved the LLVM stack by just over 2x performance across a raft of key shaders, which I am pretty proud of.

I also implemented a bunch of fun shader extensions in the stack which was a lot of fun (VK_EXT_buffer_device_address, VK_KHR_vulkan_memory_model, and quite few secret-sauce ones still to come!).

I think the stack is in a good state going forward.

As always - there is always a few things about any employment that aren’t as great. AMD is an old company, 50 years old this year. Some of the internal systems feel about that old too. Doing expenses is a pain - the system is archaic and frustrating, getting new hardware wasn’t always as easy as it could be, and I wasn’t quite prepared for cross-team politics that a big company like AMD is going to have.

The other thing I really wish I had asked about before I start was how code is actually written. Perforce was the main version control system I used while at AMD, and I find perforce to be an unmitigated disaster for software development. I genuinely think I was at about 10% of my peak productivity.

I think coming from a relatively small company like Codeplay (about 85 people when I left) who had much leaner systems and which individuals could easily move the needle left me a little ill-prepared for the reality of a big companies. You live and learn - I am now much wiser.

That being said - I still did some good and enjoyable work, made some positive changes, and made some good friends. AMD has a bunch of really talented staff that move mountains to make good changes.