[Lumiera] Build System evaluation -- Autotools / SCons

Ichthyostega prg at ichthyostega.de
Wed Dec 14 07:05:37 CET 2011

Hello Lumiera hackers,

something we might discuss further at the meeting...

It seems overdue that we end the "evaluation of build systems"
and settle the related issues. To me, the result looks outright
clear by now, based on the arguments detailed here.

My judgement is based on the following premise:

- Lumiera is a ``mid scale'' project. It is larger than a single
  person can fully comprehend and it creates a considerable weight
  on its own. It is an Application defining to some degree it's
  own environment / ecosystem. And it can be expected to remain
  in evolution for quite some time to come (yess, we hope so)

  * Lumiera is not that nifty little library with 50 source files
  * Lumiera is not that self-contained tool running virtually everywhere
  * Lumiera will have to deal with seriously complicated and non-standard
    setup- and library and driver choices, once we're getting into the
    actual media and external interface related realms

= Why Autotools isn't the right tool for the task at hand

== Design problems

The lack of design or any kind of consideration regarding requirements for
a build system seems to be the most striking feature of Autotools. It is a
"pragmatic" hack, built on top of a chain of workarounds to circumvent
shortcomings of the underlying base, which is classical Make.

- Make is not really declarative / rules based, because way to much
  focussed on operational semantics
- the make syntax wasn't designed to be readable or to capture knowledge
- instead of creating a clean baseline, it just wraps shell scripts, which
  are again notoriously error prone, unreadable and nonportable
- Make is lacking crucial features, which are added by X derivatives in
  multiple, slightly incompatible ways

Instead of fixing the base (or -- heaven forbid -- stating with a consideration
how a build system should actually work to work well), a second and a third
layer of fixes were added on top. Unfortunately using a text substitution
macro language (M4), which is a perfect misfit for the task at hand.
This language is arcane and unintuitive; text substitution is trivially
to implement yet notoriously hard for humans to judge in its effects,
especially when combined with "wild" (unchecked) interpretation of the
generated results as a shell script. And it lacks the language constructs
required to build a viable abstraction.

In this way, each time exponentiating the problems instead of solving them,
several layers and a whole set of "best" (actually bad) practices where
added. Like obfuscating the build commands and the created build artefacts
by libtool and generated wrapper scripts, or polluting the tree with 10000
lines of generated configure scripts and automagically "repairing" problems
with yet more #defines -- often without real error checks or
usable error diagnostics.

== practical usage problems

- the end user gets to do several steps, slightly different in each project
- problems are hard to diagnose and almost impossible to fix with
  general purpose knowledge, unless the end user or developer is
  willing to acquire specialist knowledge.
- the build scripts themselves are useless for capturing and
  communicating knowledge about the build as such, because the
  "what" is encoded into the "how its done", and this in turn
  is done in un-obvious ways. This creates the liability to
  additionally document the build (which no one does, of course)
- there is an abundance of toggles, options and possible tweaks,
  again generating an exponential number of combinations, which
  no developer ever will be able to test and ensure to work.

== methodical problems

- the fundamentally unsolved problems (dependency handling,
  lack of structuring) poison the everyday practice. People are
  educated to "clean" way to often (because the dependency management
  is unreliable). People are educated to help themselves with yet more
  arcane helper scripts, #defines and environment variables.
  Bad working style is infectious.

- the lower level of project definition (building individual goals) is
  focussed too much on explicitly listing individual files and toggles.

- the higher level (autoconf and friends) is too arcane and esoteric.
  It hides knowledge instead of documenting it. Thus people are driven
  back to the lower level (where you can "see what's going on"), instead
  of working on the appropriate, moderate medium level of abstraction

In addition to these fundamental challenges, the Autotools build system(s)
perpetuate several practices, which were adequate 20 years ago, while
becoming more and more obsolete today

* it's not a build system's job to ensure "that my compiler is sane".
  Because we have (Linux) distributions or software vendors for that purpose

* it's not a build system's job to manage dependencies.
  We have really elaborate package managers for that.

* it's not a build system's job to find other libraries. Rather,
  that's the upstream author's liability, or the packager's. And there are
  meanwhile things like pkg-config, which largely obsoletes ./configure

* it's no longer a build system's job to distribute my software,
  nor is it a build system's job to keep my tree clean. We have
  powerful source code management systems to our disposal todays.

To summarise: meanwhile, we've learned to understand our own practice way
better. Mostly it boils down to social interactions. People need to take on
certain responsibilities (like testing software, caring for compatibility,
reporting problems etc). Any attempt to address or "solve" that kind of stuff
through "clever" tools is bound to fail.

= Some observations in practice

We're using two build systems in parallel now for several years.
I'd like to state some observations.

The fact that different developers have preferences or even just some kind
of focus on one of these systems, tends to create friction in practice.

Developer A writes something new. Everything works with "his" build system.
Developer B perceives the same new feature as a build breakage. Now...

- either he has to reverse-engineer the other guy's work and figure out
  how to add it to the other build system
- or there needs to be additional communication or formalism at places
  where the goal should be exactly *not* to communicate, but to abstract
- or developers need to maintain two systems in parallel.

Actually, there is quite some asymmetry between our Autotools and SCons
build systems regarding that issue. Initially, personally I felt obliged
not to break the Autotools build. Until I realised that I spent a really
considerable amount of time fixing Autotools build problems after the
fact. And I noticed that I got reluctant to change and refactor things.
This is really bad.

So I decided (and announced) that I won't care for Autotools henceforth.

More information about the Lumiera mailing list