By Florian Verdonck, Open-Source Software Engineer
Our researchers and engineers use a lot of C# and F# in their models and over time, they became increasingly frustrated with the time it took to update and execute code.
Given the potential impact even the smallest improvement here might yield, this was something that we needed to improve. However, given the sheer size and performance requirements of our platforms and tooling, these fixes to expedite coding and execution were not necessarily something that the wider open source software community would help with.
Enter Project Velocity.
Velocity was a three-month internal effort to investigate what could be done to improve the experience of developing and pushing code into production, and I was excited to become a part of this work.
I joined the open source team full time in April 2022 and my first assignment was to improve the .NET developer experience. G-Research is a special environment, with tight security constraints. So, to experience what the developers were experiencing, I had to go to the office in London.
In May 2022, during my first visit, I was introduced to Janusz Wrobel and Chris Arnott. They had been working on Velocity, an internal project to investigate what could be done to improve the developers’ experience.
Our engineers deal with a large solution, consisting of hundreds of instances, with a mix of C# and F#. This made everything slow: from restoring NuGet packages, to building the code, as well as the editor environment.
My colleague Marcin Krystianc was already investigating the NuGet side of things, and had worked on something to make the static graph evaluation better for NuGet, as well as detecting any regressions. This was important work and provided a useful start point from which we could further accelerate the Velocity project.
During the investigation, we discovered that F# code takes a long time to build and does not seem to use a lot of parallelization. When compiling an F# project, it was also apparent that the type-check phase was the most expensive one. This was something I could focus on. Here was a place where I could bring my expertise to bear.
Though I previously wrote about my work creating this feature in the Microsoft Developer Blog, I’ll use this blog series to tell the behind-the-scenes story of how and why the graph-based type-checking feature in the F# compiler came to be, and to show how it is being used at scale to ensure faster processing for G-Research.
It’s the story of how I advocated for important changes to the open source code of F# that would allow us to push the software beyond what previously seemed like its maximum capacity. It’s the story of how open source innovation doesn’t just happen – it sometimes needs to be midwifed-along, pushed through, cajoled into existence, prodded beyond what seems possible – to get real progress.
