[Lumiera] Build System evaluation -- Autotools / SCons

Benny Lyons Benny.Lyons at gmx.net
Wed Dec 14 19:40:29 CET 2011

I might be somewhat late to IRC this evening as I'm experiencing considerable latency---larger than usual---in my communication mechanism :=((

I don't think such exegesis by Hermann deserves to go unanswered, especially such a fundamental (for developers) item as the choice of build system to use for Lumiera.

Many of the points you mention are indeed true, indeed using a one-dimensional measuring stick, I'm sure Scons would come out ahead of the autotool chain. However the selection process ought not be so naivly one-dimensional.  Other considerations must be drawn into account.

Technically, there are also a number of downsides to Scons as well: you mention make and its well-known limitations.  Indisputable.  But the autotool chain only __uses__ make: it's like delimitating the limitations of Assembler or Machine Code, just to down C++, which, off course, only uses Machine Code. When you use automake, you never really have to get involved with make itself; you can, but you don't really have to.

Moreover, Scons can become very quickly tricky and intricate just to do rather trivial things.  But, as you mention, it does have an approach to designing the build that is really superior to Automake and co.

One significant clout for the automake tool chain is its wide acceptance and indeed usage within the Open Source community.  Sure Scons is gaining in popularity, but the number of individuals well versed in both Scons itself and Python is certainly an issue that should not be lightly overlooked, especially when Lumiera is still looking for people to join the project.  Will perspective newcommers dwindle away when they hear about Scons, or will the use of Autotools keep this number of potential new contributors constant? I don't know; but I think an important issue in the decision process.  

So many of the points you make are a trueism in themselves: Scons makes one think about the build process itself---how often have I burnt mid-night oil trying to  find some stupid Rule error; has certainly technical superior qualities to the autotool chain; it is time consuming supporting two build systems, (but does have a positive aspect to it);...

Yet, we cannot ignore the popular (just a joke, I mean 'wide-spread' here) use and knowledge of Autotools. I, for one, as you might have gathered, belong to the popular group.  Those python scripts are not trivial and I cannot see myself changing this without investing considerable time in Scons.

There's another dimension, which, infortunately, I don't have time to discuss here, maybe later on.

Hopefully, 'till later

-------- Original-Nachricht --------
> Datum: Wed, 14 Dec 2011 07:05:37 +0100
> Von: Ichthyostega <prg at ichthyostega.de>
> An: LumiList <lumiera at lists.lumiera.org>
> Betreff: [Lumiera] Build System evaluation  -- Autotools / SCons

> Hello Lumiera hackers,
> something we might discuss further at the meeting...
> It seems overdue that we end the "evaluation of build systems"
> and settle the related issues. To me, the result looks outright
> clear by now, based on the arguments detailed here.
> My judgement is based on the following premise:
> - Lumiera is a ``mid scale'' project. It is larger than a single
>   person can fully comprehend and it creates a considerable weight
>   on its own. It is an Application defining to some degree it's
>   own environment / ecosystem. And it can be expected to remain
>   in evolution for quite some time to come (yess, we hope so)
>   * Lumiera is not that nifty little library with 50 source files
>   * Lumiera is not that self-contained tool running virtually everywhere
>   * Lumiera will have to deal with seriously complicated and non-standard
>     setup- and library and driver choices, once we're getting into the
>     actual media and external interface related realms
> = Why Autotools isn't the right tool for the task at hand
> == Design problems
> The lack of design or any kind of consideration regarding requirements for
> a build system seems to be the most striking feature of Autotools. It is a
> "pragmatic" hack, built on top of a chain of workarounds to circumvent
> shortcomings of the underlying base, which is classical Make.
> - Make is not really declarative / rules based, because way to much
>   focussed on operational semantics
> - the make syntax wasn't designed to be readable or to capture knowledge
> - instead of creating a clean baseline, it just wraps shell scripts, which
>   are again notoriously error prone, unreadable and nonportable
> - Make is lacking crucial features, which are added by X derivatives in
>   multiple, slightly incompatible ways
> Instead of fixing the base (or -- heaven forbid -- stating with a
> consideration
> how a build system should actually work to work well), a second and a
> third
> layer of fixes were added on top. Unfortunately using a text substitution
> macro language (M4), which is a perfect misfit for the task at hand.
> This language is arcane and unintuitive; text substitution is trivially
> to implement yet notoriously hard for humans to judge in its effects,
> especially when combined with "wild" (unchecked) interpretation of the
> generated results as a shell script. And it lacks the language constructs
> required to build a viable abstraction.
> In this way, each time exponentiating the problems instead of solving
> them,
> several layers and a whole set of "best" (actually bad) practices where
> added. Like obfuscating the build commands and the created build artefacts
> by libtool and generated wrapper scripts, or polluting the tree with 10000
> lines of generated configure scripts and automagically "repairing"
> problems
> with yet more #defines -- often without real error checks or
> usable error diagnostics.
> == practical usage problems
> - the end user gets to do several steps, slightly different in each
> project
> - problems are hard to diagnose and almost impossible to fix with
>   general purpose knowledge, unless the end user or developer is
>   willing to acquire specialist knowledge.
> - the build scripts themselves are useless for capturing and
>   communicating knowledge about the build as such, because the
>   "what" is encoded into the "how its done", and this in turn
>   is done in un-obvious ways. This creates the liability to
>   additionally document the build (which no one does, of course)
> - there is an abundance of toggles, options and possible tweaks,
>   again generating an exponential number of combinations, which
>   no developer ever will be able to test and ensure to work.
> == methodical problems
> - the fundamentally unsolved problems (dependency handling,
>   lack of structuring) poison the everyday practice. People are
>   educated to "clean" way to often (because the dependency management
>   is unreliable). People are educated to help themselves with yet more
>   arcane helper scripts, #defines and environment variables.
>   Bad working style is infectious.
> - the lower level of project definition (building individual goals) is
>   focussed too much on explicitly listing individual files and toggles.
> - the higher level (autoconf and friends) is too arcane and esoteric.
>   It hides knowledge instead of documenting it. Thus people are driven
>   back to the lower level (where you can "see what's going on"), instead
>   of working on the appropriate, moderate medium level of abstraction
> In addition to these fundamental challenges, the Autotools build system(s)
> perpetuate several practices, which were adequate 20 years ago, while
> becoming more and more obsolete today
> * it's not a build system's job to ensure "that my compiler is sane".
>   Because we have (Linux) distributions or software vendors for that
> purpose
> * it's not a build system's job to manage dependencies.
>   We have really elaborate package managers for that.
> * it's not a build system's job to find other libraries. Rather,
>   that's the upstream author's liability, or the packager's. And there are
>   meanwhile things like pkg-config, which largely obsoletes ./configure
> * it's no longer a build system's job to distribute my software,
>   nor is it a build system's job to keep my tree clean. We have
>   powerful source code management systems to our disposal todays.
> To summarise: meanwhile, we've learned to understand our own practice way
> better. Mostly it boils down to social interactions. People need to take
> on
> certain responsibilities (like testing software, caring for compatibility,
> reporting problems etc). Any attempt to address or "solve" that kind of
> stuff
> through "clever" tools is bound to fail.
> = Some observations in practice
> We're using two build systems in parallel now for several years.
> I'd like to state some observations.
> The fact that different developers have preferences or even just some kind
> of focus on one of these systems, tends to create friction in practice.
> Developer A writes something new. Everything works with "his" build
> system.
> Developer B perceives the same new feature as a build breakage. Now...
> - either he has to reverse-engineer the other guy's work and figure out
>   how to add it to the other build system
> - or there needs to be additional communication or formalism at places
>   where the goal should be exactly *not* to communicate, but to abstract
> - or developers need to maintain two systems in parallel.
> Actually, there is quite some asymmetry between our Autotools and SCons
> build systems regarding that issue. Initially, personally I felt obliged
> not to break the Autotools build. Until I realised that I spent a really
> considerable amount of time fixing Autotools build problems after the
> fact. And I noticed that I got reluctant to change and refactor things.
> This is really bad.
> So I decided (and announced) that I won't care for Autotools henceforth.
> From that point on, our Autotools build in Builddrone was "red" for 90%
> of the time. This isn't harmless. This is a Broken Window.
> == disagreement on methodical goals
> On a second thought, for me this issue highlights a more subtle discord
> about what actually is desirable. IMHO *that* is something we really need
> to address and resolve. We seem to go for different goals here.
> The Autotools build seems to be defined deliberately manual and explicit.
> Each file is mentioned explicitly, each file requires one explicit manual
> step to add it to the build, and compile / link flags and depencencies
> seem to be specified deliberately manual.
> I think such would be appropriate for a small project, but is the wrong
> approach for a large project. We're getting close to 1000 files now, and
> this number will increase significantly. Plus there will be plug-ins,
> metadata and interfaces and resources. Rather, we should strive at
> reducing
> complexity by abstracting things. Instead of looking at 1000++ files,
> we should look at 5 modules or so. Instead of building each test
> a little bit different, we should distil a common pattern for all tests.
> We need to strive at a systematic approach, because we really need to
> focus ourselves at our primary goal.
> Similarly, IMHO there is no point in looking at library dependencies
> with ldd or changing linker flags. Because we *do* have a problem with
> the *concept* of our Lumiera interface library. And the reason is,
> that we're not ready to define an external interface for scripted
> use of Lumiera. Our design simply isn't mature enough for that.
> First things first -- which IMHO means continuing to target
> the design and implementation of a complete render pipeline.
> = Assessment of the Autotools build
> After having listed all those deficiencies of Autotools, I should
> add that under certain circumstances, using Autotools does yield a
> significant benefit, which might outweigh the costs. But I don't
> even remotely see Lumiera to be in that situation. Frankly, I
> see zero benefit from using Autotools for us.
> = Assessment of the SCons build
> The results on the SCons buildsysem are also not entirely favourable,
> but overall I see far less serious problems. Yet still, we should
> noteice and try to judge benefits and problems here as well.
> == benefits
> - the SCons build definition is actually declarative. "What" instead of
> "how".
> - using Python as the definition language avoids problems with language
> design.
>   It does have its quirks, but its expressiveness is beyond any doubt.
> - Python has the necessary language constructs to support a clean coding
> style.
> - the concept of a "build environment" helps with reducing the build into
> a
>   small number of standard situations plus a local variation here and
> there.
> - SCons deliberately sanitises the build environment and isolates the
> build
>   from external influences. The build "sees" exactly those libraries you
>   pick up beforehand, and it effectively prevents any environment
> variables
>   to "sneak in" and influence the build behind the scenes.
> - SCons is actually based on a design chosen to reflect the reality of
>   the build process. Changes are detected based on actual file content
> (MD5)
>   plus understanding the dependencies (parsing headers). Additionally,
> SCons
>   tracks the effect of changed compiler flags and detects changes in
> library
>   versions due to system upgrade.
> == issues
> Out of the box, SCons is relatively low-level. It lacks some of the
> ready-made
> solutions we know from Autotools. Basically, SCons is more like a
> sophisticated
> Make; it's rather just a build tool, not a build system.
> Thus, each SCons based build system is more or less "custom". For example,
> I've written a LumieraEnvironment (sub)class, which embodies project
> specific
> knowledge how to link and how to build Lumiera plug-ins. (This idea is
> thanks to the build system of Ardour)
> There is no predefined support for libraries at non standard locations.
> SCons either relies on the toolchain, or you have to add an commandline
> option explicitly and feed that manually to the build environment.
> The support for building in a separate build directory is somewhat strange
> and just differs from what Autotools does. For now we build in-tree
> SCons is known to be slow (it's Python, after all). Benchmarks indicate
> that
> it might degenerate with 20000 or 40000 files. (but stock Autotools
> similarly
> degenerates on such large projects)
> Prior to SCons 1.0, we had some compatibility problems
> _______________________________________________
> Lumiera mailing list
> Lumiera at lists.lumiera.org
> http://lists.lumiera.org/cgi-bin/mailman/listinfo/lumiera
> http://lumiera.org/donations.html


Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de

More information about the Lumiera mailing list