[Lumiera] Build System evaluation -- Autotools / SCons

Michael R Fisher mfisher31 at gmail.com
Wed Dec 14 13:14:34 CET 2011

Hash: SHA1

Has anybody ever tried waf?

might discuss further at the meeting...
> It seems overdue that we end the "evaluation of build systems"
> and settle the related issues. To me, the result looks outright
> clear by now, based on the arguments detailed here.
> ?
> My judgement is based on the following premise:
> - Lumiera is a ``mid scale'' project. It is larger than a single
>   person can fully comprehend and it creates a considerable weight
>   on its own. It is an Application defining to some degree it's
>   own environment / ecosystem. And it can be expected to remain
>   in evolution for quite some time to come (yess, we hope so)
>   * Lumiera is not that nifty little library with 50 source files
>   * Lumiera is not that self-contained tool running virtually everywhere
>   * Lumiera will have to deal with seriously complicated and non-standard
>     setup- and library and driver choices, once we're getting into the
>     actual media and external interface related realms
> = Why Autotools isn't the right tool for the task at hand
> == Design problems
> The lack of design or any kind of consideration regarding requirements for
> a build system seems to be the most striking feature of Autotools. It is a
> "pragmatic" hack, built on top of a chain of workarounds to circumvent
> shortcomings of the underlying base, which is classical Make.
> - Make is not really declarative / rules based, because way to much
>   focussed on operational semantics
> - the make syntax wasn't designed to be readable or to capture knowledge
> - instead of creating a clean baseline, it just wraps shell scripts, which
>   are again notoriously error prone, unreadable and nonportable
> - Make is lacking crucial features, which are added by X derivatives in
>   multiple, slightly incompatible ways
> Instead of fixing the base (or -- heaven forbid -- stating with a consideration
> how a build system should actually work to work well), a second and a third
> layer of fixes were added on top. Unfortunately using a text substitution
> macro language (M4), which is a perfect misfit for the task at hand.
> This language is arcane and unintuitive; text substitution is trivially
> to implement yet notoriously hard for humans to judge in its effects,
> especially when combined with "wild" (unchecked) interpretation of the
> generated results as a shell script. And it lacks the language constructs
> required to build a viable abstraction.
> In this way, each time exponentiating the problems instead of solving them,
> several layers and a whole set of "best" (actually bad) practices where
> added. Like obfuscating the build commands and the created build artefacts
> by libtool and generated wrapper scripts, or polluting the tree with 10000
> lines of generated configure scripts and automagically "repairing" problems
> with yet more #defines -- often without real error checks or
> usable error diagnostics.
> == practical usage problems
> - the end user gets to do several steps, slightly different in each project
> - problems are hard to diagnose and almost impossible to fix with
>   general purpose knowledge, unless the end user or developer is
>   willing to acquire specialist knowledge.
> - the build scripts themselves are useless for capturing and
>   communicating knowledge about the build as such, because the
>   "what" is encoded into the "how its done", and this in turn
>   is done in un-obvious ways. This creates the liability to
>   additionally document the build (which no one does, of course)
> - there is an abundance of toggles, options and possible tweaks,
>   again generating an exponential number of combinations, which
>   no developer ever will be able to test and ensure to work.
> == methodical problems
> - the fundamentally unsolved problems (dependency handling,
>   lack of structuring) poison the everyday practice. People are
>   educated to "clean" way to often (because the dependency management
>   is unreliable). People are educated to help themselves with yet more
>   arcane helper scripts, #defines and environment variables.
>   Bad working style is infectious.
> - the lower level of project definition (building individual goals) is
>   focussed too much on explicitly listing individual files and toggles.
> - the higher level (autoconf and friends) is too arcane and esoteric.
>   It hides knowledge instead of documenting it. Thus people are driven
>   back to the lower level (where you can "see what's going on"), instead
>   of working on the appropriate, moderate medium level of abstraction
> In addition to these fundamental challenges, the Autotools build system(s)
> perpetuate several practices, which were adequate 20 years ago, while
> becoming more and more obsolete today
> * it's not a build system's job to ensure "that my compiler is sane".
>   Because we have (Linux) distributions or software vendors for that purpose
> * it's not a build system's job to manage dependencies.
>   We have really elaborate package managers for that.
> * it's not a build system's job to find other libraries. Rather,
>   that's the upstream author's liability, or the packager's. And there are
>   meanwhile things like pkg-config, which largely obsoletes ./configure
> * it's no longer a build system's job to distribute my software,
>   nor is it a build system's job to keep my tree clean. We have
>   powerful source code management systems to our disposal todays.
> To summarise: meanwhile, we've learned to understand our own practice way
> better. Mostly it boils down to social interactions. People need to take on
> certain responsibilities (like testing software, caring for compatibility,
> reporting problems etc). Any attempt to address or "solve" that kind of stuff
> through "clever" tools is bound to fail.
> = Some observations in practice
> We're using two build systems in parallel now for several years.
> I'd like to state some observations.
> The fact that different developers have preferences or even just some kind
> of focus on one of these systems, tends to create friction in practice.
> Developer A writes something new. Everything works with "his" build system.
> Developer B perceives the same new feature as a build breakage. Now...
> - either he has to reverse-engineer the other guy's work and figure out
>   how to add it to the other build system
> - or there needs to be additional communication or formalism at places
>   where the goal should be exactly *not* to communicate, but to abstract
> - or developers need to maintain two systems in parallel.
> Actually, there is quite some asymmetry between our Autotools and SCons
> build systems regarding that issue. Initially, personally I felt obliged
> not to break the Autotools build. Until I realised that I spent a really
> considerable amount of time fixing Autotools build problems after the
> fact. And I noticed that I got reluctant to change and refactor things.
> This is really bad.
> So I decided (and announced) that I won't care for Autotools henceforth.
> From that point on, our Autotools build in Builddrone was "red" for 90%
> of the time. This isn't harmless. This is a Broken Window.
> == disagreement on methodical goals
> On a second thought, for me this issue highlights a more subtle discord
> about what actually is desirable. IMHO *that* is something we really need
> to address and resolve. We seem to go for different goals here.
> The Autotools build seems to be defined deliberately manual and explicit.
> Each file is mentioned explicitly, each file requires one explicit manual
> step to add it to the build, and compile / link flags and depencencies
> seem to be specified deliberately manual.
> I think such would be appropriate for a small project, but is the wrong
> approach for a large project. We're getting close to 1000 files now, and
> this number will increase significantly. Plus there will be plug-ins,
> metadata and interfaces and resources. Rather, we should strive at reducing
> complexity by abstracting things. Instead of looking at 1000++ files,
> we should look at 5 modules or so. Instead of building each test
> a little bit different, we should distil a common pattern for all tests.
> We need to strive at a systematic approach, because we really need to
> focus ourselves at our primary goal.
> Similarly, IMHO there is no point in looking at library dependencies
> with ldd or changing linker flags. Because we *do* have a problem with
> the *concept* of our Lumiera interface library. And the reason is,
> that we're not ready to define an external interface for scripted
> use of Lumiera. Our design simply isn't mature enough for that.
> First things first -- which IMHO means continuing to target
> the design and implementation of a complete render pipeline.
> = Assessment of the Autotools build
> After having listed all those deficiencies of Autotools, I should
> add that under certain circumstances, using Autotools does yield a
> significant benefit, which might outweigh the costs. But I don't
> even remotely see Lumiera to be in that situation. Frankly, I
> see zero benefit from using Autotools for us.
> = Assessment of the SCons build
> The results on the SCons buildsysem are also not entirely favourable,
> but overall I see far less serious problems. Yet still, we should
> noteice and try to judge benefits and problems here as well.
> == benefits
> - the SCons build definition is actually declarative. "What" instead of "how".
> - using Python as the definition language avoids problems with language design.
>   It does have its quirks, but its expressiveness is beyond any doubt.
> - Python has the necessary language constructs to support a clean coding style.
> - the concept of a "build environment" helps with reducing the build into a
>   small number of standard situations plus a local variation here and there.
> - SCons deliberately sanitises the build environment and isolates the build
>   from external influences. The build "sees" exactly those libraries you
>   pick up beforehand, and it effectively prevents any environment variables
>   to "sneak in" and influence the build behind the scenes.
> - SCons is actually based on a design chosen to reflect the reality of
>   the build process. Changes are detected based on actual file content (MD5)
>   plus understanding the dependencies (parsing headers). Additionally, SCons
>   tracks the effect of changed compiler flags and detects changes in library
>   versions due to system upgrade.
> == issues
> Out of the box, SCons is relatively low-level. It lacks some of the ready-made
> solutions we know from Autotools. Basically, SCons is more like a sophisticated
> Make; it's rather just a build tool, not a build system.
> Thus, each SCons based build system is more or less "custom". For example,
> I've written a LumieraEnvironment (sub)class, which embodies project specific
> knowledge how to link and how to build Lumiera plug-ins. (This idea is
> thanks to the build system of Ardour)
> There is no predefined support for libraries at non standard locations.
> SCons either relies on the toolchain, or you have to add an commandline
> option explicitly and feed that manually to the build environment.
> The support for building in a separate build directory is somewhat strange
> and just differs from what Autotools does. For now we build in-tree
> SCons is known to be slow (it's Python, after all). Benchmarks indicate that
> it might degenerate with 20000 or 40000 files. (but stock Autotools similarly
> degenerates on such large projects)
> Prior to SCons 1.0, we had some compatibility problems
> _______________________________________________
> Lumiera mailing list
> Lumiera at lists.lumiera.org
> http://lists.lumiera.org/cgi-bin/mailman/listinfo/lumiera
> http://lumiera.org/donations.html

Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/


More information about the Lumiera mailing list