Language-agnostic build tools: how do they stack up?

In the distant past, make was the only build tool worth using. As time went on, we began producing language-specific build tools. But in recent times, language-agnostic tools have had a minor resurgence.

Should you be looking into language-agnostic tools? Let’s take a look!

First, let’s have a glance at a couple of language-specific build systems, then we’ll compare with language-agnostic ones.

Maven / Gradle

Maven and Gradle aren’t really the same build system, but they behave relatively similarly, targeting the JVM and the same repository of libraries. They have several components:

  • Implicit package structure: you’re expected to use directories like src/main/java to organize code. If you stick to this format, you have less to configure.
  • Finding your source code automatically: even if you diverge from that standard, you only have to specify your own root directories for project code, test code, project resources, and test resources.
  • Dependency management: you can specify a set of versioned dependencies to reference from your project.
  • Plugin architecture: you can create plugins to alter how builds happen (for instance, the shadowjar plugin for gradle, which effectively statically links your jar file).

Beyond that, they call javac and stitch stuff together into a jar file. They can also run your unittests.

A quick example of a gradle buildfile:

group 'org.ikeran'
project.version = '1.0.337'
apply plugin: 'java'
apply plugin: 'maven'
dependencies {
    compile 'ch.qos.logback:logback-classic:1.1.7'
    compile 'ch.qos.logback:logback-core:1.1.7'
    compile 'com.google.guava:guava:22.0'
    testCompile group: 'junit', name: 'junit', version: '4.12'
    testCompile group: 'org.hamcrest', name: 'java-hamcrest', version: '2.0.0.0'
}

Dub

Dub is the D programming language’s package manager.

One of the small parts about it is that it offers two different DSLs to define a package: JSON and SDLang. SDLang is generally a lot nicer, supporting comments (among other things), but it’s an extra thing to learn.

name "scid"
description "Command-line spreadsheet program"
authors "Neia Neutuladh"
copyright "Copyright © 2018, Neia Neutuladh"
license "MS-PL"
dependency "pegged" version="~>0.4.3"
libs "ncursesw"

configuration "application" {
    mainSourceFile "source/app.d"
    targetType "executable"
}
configuration "unittest" {
    dependency "unit-threaded" version="~>0.7.45"
    targetType "executable"
    targetName "scid-test-library"
    mainSourceFile "bin/ut.d"
    excludedSourceFiles "source/app.d"
}

This is a bit more verbose than the the gradle file. It’s got a few lines for packaging, which lets you publish code on code.dlang.org, and it explicitly defines two build targets. The “application” target is pretty much redundant, while the “unittest” target adds a dependency and slightly changes the set of source files to build. This would also be redundant, but the unit-threaded library has some extra setup compared to the default.

That would be handled as a plugin, but dub doesn’t have a plugin architecture. It does, however, have a few hooks besides defining these configurations.

One last thing of note: this package depends on a native system library, libncursesw. Dub can’t ensure that the library is there, check its version, or anything like that.

Meson

The next build system of note is Meson. Its claim to fame is the GNOME project, which is pushing it as the primary way to build GNOME and GTK+ official projects. It’s also possibly common in the Vala community, but that community is pretty small.

Meson’s actually a meta-build system, a high-level language that outputs a Ninja build file that you then run.

It’s not exactly language-agnostic, but it does support a range of languages. Let’s try replicating the SCID dub script in Meson.

# You can add other languages after the fact, but you have to
# start with just one.
project('scid', 'd')
curses = dependency('ncursesw')
pegged = dependency('pegged', method: 'dub', version: '~>0.4.3')
main_sources = [
  'source/scid/style.d',
  'source/scid/formats/scid.d',
  'source/scid/ods.d',
  'source/scid/command.d',
  'source/scid/functions.d',
  'source/scid/rational.d',
  'source/scid/ui/keys.d',
  'source/scid/ui/sheets.d',
  'source/scid/ui/curses.d',
  'source/scid/ui/state.d',
  'source/scid/ui/controller.d',
  'source/scid/ui/view.d',
  'source/scid/formula/package.d',
  'source/scid/formula/scopes.d',
  'source/scid/formula/grammar.d',
  'source/scid/formula/val.d',
  'source/scid/formula/context.d',
  'source/scid/cell.d',
  'source/scid/sheet.d',
  'source/scid/util.d'
]
executable(
  'scid',
  ['source/app.d'] + main_sources,
  dependencies: [pegged, curses])
executable(
  'scid-test-library',
  ['bin/ut.d'] + main_sources,
  d_unittest: true,
  dependencies: [pegged, curses])

Headache #1: every single file

The first obvious thing should be the 20 lines listing every file in the project. This is because, we’re told, if you have a very large project with tens of thousands of files, you can’t support an efficient incremental build if your build tool has to re-scan your filesystem each time you build.

Which in turn means you’re supposed to list tens of thousands of source files in your build. Maybe the solution could be to use inotify and a persistent build process instead? And the Meson documentation suggests doing something like that manually.

Headache #2: fetching dependencies

I told Meson that my project depends on the dub package pegged. If you run this on your machine, it will fail here. You don’t have pegged on your system. You have to run dub fetch pegged --version=0.4.3 && dub build pegged first. Meson can find dependencies once dub has fetched and built them.

This sucks. It shouldn’t be this way. Meson knows enough about dub that it should be able to run these commands itself.

Headache #3: which compiler, then?

Once I ran those commands and re-ran meson build, it complained that pegged wasn’t built with dmd. Looking at the meson source code, it seems to be searching in the project directory for the appropriate dub build output, but that is the wrong place for it.

I solved this by adding Pegged as a git submodule and adding a Meson build target for it. As much as it sucks to have to manually ask dub to fetch and build our dependencies, it sucks even more having to write build scripts for all our dependencies.

We basically remove the dub dependency and inject the following into our build file:

peggedlib = shared_library('pegged', [
  'Pegged/pegged/peg.d',
  'Pegged/pegged/grammar.d',
  'Pegged/pegged/parser.d',
  'Pegged/pegged/introspection.d',
  'Pegged/pegged/tohtml.d',
  'Pegged/pegged/dynamic/grammar.d',
  'Pegged/pegged/dynamic/peg.d',
  'Pegged/pegged/tester/grammartester.d',
  'Pegged/pegged/tester/testerparser.d'
])
pegged = declare_dependency(
  link_with: peggedlib,
  include_directories: include_directories(['Pegged']))

Headache #4: is every language C?

You’d expect things to work at this point, right? Haha, to be so optimistic.

The normal thing to do with a set of D source files like this is to pass them all to the compiler and have it build the output all in one go. Meson does it one at a time because it’s intended for very large projects. And Meson was built with C and C++ at the forefront, where there’s a separation between source files and include files.

So I have to tell Meson manually where it can find include directories. That gets tacked on the executable and shared_library rules.

Headache #5: is every compiler GCC?

We declared a dependency on the ncursesw library. This is a native library located with pkg-config. And Meson finds it, like it should, and runs pkg-config, like it should.

Here’s the problem: the D compiler isn’t the GNU C compiler. pkg-config --cflags doesn’t give you command line arguments that make any sense whatsoever with dmd. They’re unlikely to make sense with gdc.

Marching on at all costs, I removed the dependency from the executable targets and instead manually added flags to link to ncursesw. I erred at first by adding this to d_flags instead of link_flags, but that’s not a headache, just a mistake.

(At this point, I also had to do some work to straighten out include directories for my executable target. Just the one; I didn’t have the patience to make another build target for the unit-threaded library, which means I’m not running tests. I’m not sure how, but meson refused for a while to add my project’s source directory to the include paths. This sorted itself through shotgun debugging.)

Final result

At this point, my project takes one minute to build. This is impressive since dub requires 7.07 seconds, and that includes checking out pegged. A single-file-change incremental build takes 4.16 seconds with Meson / Ninja and 5.54 seconds with dub. When two files have changed, Meson / Ninja takes 7.49 seconds, while dub continues to take about 5.6 seconds. So this is quite lackluster in the speed department, and speed is supposed to be Meson’s claim to fame.

Furthermore, this requires us to change directories between updating the Meson build definition and building the code. This gets annoying fast. Much like Meson in general.

It also repeated its full build sometimes for no reason I could discern.

The last, damning problem from Meson was that it didn’t build the whole dependency graph. If you update the definition of a struct and change its size or layout, and that struct’s used in another file, that other file needs to be recompiled. If you change a template and that template’s instantiated in another file, that other file needs to be recompiled. But Meson doesn’t figure out those relationships, and neither does Ninja, so it doesn’t rebuild everything required.

The final build file, for posterity:

project('scid', 'd')
curses = dependency('ncursesw')

peggedlib = shared_library('pegged', [
  'Pegged/pegged/peg.d',
  'Pegged/pegged/grammar.d',
  'Pegged/pegged/parser.d',
  'Pegged/pegged/introspection.d',
  'Pegged/pegged/tohtml.d',
  'Pegged/pegged/dynamic/grammar.d',
  'Pegged/pegged/dynamic/peg.d',
  'Pegged/pegged/tester/grammartester.d',
  'Pegged/pegged/tester/testerparser.d'
], include_directories: include_directories(['Pegged']))
pegged = declare_dependency(
  link_with: peggedlib,
  include_directories: include_directories(['Pegged']))
main_sources = [
  'source/scid/style.d',
  'source/scid/formats/scid.d',
  'source/scid/ods.d',
  'source/scid/command.d',
  'source/scid/functions.d',
  'source/scid/rational.d',
  'source/scid/ui/keys.d',
  'source/scid/ui/sheets.d',
  'source/scid/ui/curses.d',
  'source/scid/ui/state.d',
  'source/scid/ui/controller.d',
  'source/scid/ui/view.d',
  'source/scid/formula/package.d',
  'source/scid/formula/scopes.d',
  'source/scid/formula/grammar.d',
  'source/scid/formula/val.d',
  'source/scid/formula/context.d',
  'source/scid/cell.d',
  'source/scid/sheet.d',
  'source/scid/util.d'
]
executable(
  'scid',
  ['source/app.d'] + main_sources,
  include_directories: [include_directories('source')],
  dependencies: [pegged],
  link_args: '-lncursesw')

Bazel

Bazel is an open source clone of Google’s internal Blaze build tool, which I used to be familiar with. As a result, I expect Bazel will feel somewhat familiar.

Time for some good news / bad news:

  • Bad news: Bazel doesn’t support D natively.
  • Good news: there’s a plugin for that!
  • Bad news: the plugin doesn’t work and there hasn’t been a new release in a while.
  • Good news: there are unreleased updates that fix some problems.
  • Bad news: those updates don’t fix some other problems, so it still fails.
  • Good news: there’s a command line flag to un-break it (though that’s probably going away soon).
  • Bad news: now it’s downloading DMD 2.070, which is very old, and there’s no way to change that.

I ended up forking the project and modifying it, adding a way to specify a version and removing the error with using the wrong HTTP library.

While Bazel doesn’t support downloading dependencies from Dub, it also doesn’t pretend that it can, so that’s +1 for honesty.

Another issue I found later was that import paths were misconfigured. The dlang plugin wrongly assumed that all packages had names, but the root package does not. This produced a commandline like dmd -I/source -I/Pegged, which looked for code to import at the root of the filesystem instead of the current directory. I fixed that as well.

The last wrinkle I encountered was with linking libncursesw: the linkopts field of a dependency seemed to be pretty much ignored. I fixed that with the cc_import rule, which is designed for incorporating a system library into the build system.

One other thing I noticed was the symlinks:

bazel-bin
bazel-genfiles
bazel-out
bazel-scid
bazel-testlogs

Five symlinks in my project’s root. I’d really rather not.

The build file

My project’s BUILD:

package(default_visibility = ["//visibility:public"])
load("@io_bazel_rules_d//d:d.bzl", "d_binary")
load("@io_bazel_rules_d//d:d.bzl", "d_library")

cc_import(
        name = 'ncursesw',
        shared_library = 'libncursesw.so')

d_library(name = 'pegged', srcs = [
  'Pegged/pegged/peg.d',
  'Pegged/pegged/grammar.d',
  'Pegged/pegged/parser.d',
  'Pegged/pegged/introspection.d',
  'Pegged/pegged/tohtml.d',
  'Pegged/pegged/dynamic/grammar.d',
  'Pegged/pegged/dynamic/peg.d',
  'Pegged/pegged/tester/grammartester.d',
  'Pegged/pegged/tester/testerparser.d'
],
  imports = ['Pegged'])

d_binary(
        name = 'scid',
        srcs = glob(['source/**/*.d']),
        deps = [':pegged', ':ncursesw'],
        imports = ['source', 'Pegged'])

This is obviously a lot longer than the dub build file, containing both the Pegged build rule and our own rule. (Pegged has a single source directory containing library code, examples, and more. The dub file also lists out specific modules to include. Yes, it’s hairy.) It’s more verbose to link to a native library. It doesn’t have a way to export the library to anyone else (maybe you’re supposed to use git submodules for this?).

Overall, though, if you’re working with a large internal source tree and have limited external dependencies, this seems like a reasonable option.

There was also a WORKSPACE file:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
git_repository(
    name = "io_bazel_rules_d",
    remote = "https://github.com/dhasenan/rules_d.git",
    commit = "29cceda"
)
load("@io_bazel_rules_d//d:d.bzl", "d_repositories")

d_repositories(version = '2.083.0')

You need one BUILD file per group of related build targets (I should have had one for Pegged and one for SCID), but you only need one WORKSPACE file per source tree.

Again, the tests are missing, and it’s for the same reason: I was kind of low on patience. But the method is straightforward: git submodule, write the build rule, run bazel test :scidtest.

Timings

Speed covers a multitude of sins, right?

A full build (after bazel clean) takes 6.7 seconds, while an incremental build takes 6.1 seconds. This is independent of how many source files change, unlike the Meson / Ninja build. On rare occasions, bazel performs very fast, but it generally has about 1.5 seconds of overhead over dmd.

Verdict

Since some builds seem to take a lot longer than dub and some are faster, I’m going to say that dub still has a small speed advantage. But most importantly, bazel lets me depend on a specific version of DMD. This is pretty huge; incompatible DMD changes have hurt me in the past.

The amount of work required to get started was kind of annoying, but I only had to make a few lines of change to the bazel rules_d project, and now you won’t have to.

Having to write build files for projects that already have their own build files is kind of annoying, though.

All in all, though, I will consider bazel for projects that have few dependencies and that I’m not publishing on dub. The reduction in speed is annoying, as is the fact that you have to manually gather and write build scripts for your dependencies, but the benefit of being able to keep a consistent version of dmd is nontrivial.

General conclusions

Managing other projects’ builds is not the best. Managing dependencies via git submodules is also not the best. However, if you have few enough transitive dependencies, it can be an option, and if your project is polyglot, it might even be a good option.

But it loses to language-specific build tools with integrated package managers. I added one external library and that took only a little bit of work, but I have another project with 22 separate dependencies; it would be a solid day to get them all working together with Bazel.

So, pick your battles, and maybe avoid Meson.