Git and Autotools – a hate relation?

For a project where I contribute I started to rewrite the build system using autotools (autoconf/automake), since this is one of the most popular build systems as far as I know. The actual conversion turned out to be not too complicated. But it turned out that in modern world of git(hub), it seems that the autotools have lost their place.

Source of all the problems is the dreaded

WARNING: 'aclocal-1.16' is missing on your system.
         You should only need it if you modified 'acinclude.m4' or
         'configure.ac' or m4 files included by 'configure.ac'.

which, when you search for it, appears all over projects hosted on some instance of git. The reason is rather simple: git does not preserve timestamps, and autotools try to rebuild each and everything from the most basic files upward if there is a timestamp squeeze.

Consequences of this PITA is that:

  • Every developer needs to install the whole autotools stack, and before running anything always has to do a autoreconf -i
  • github generated tarballs might have arbitrary time stamps, so release management via git tags is borken, and users who want to install the software package might see the same errors

The best answer I have found to this problem is to use the make dist generated tarballs and replace the github generated ones with the home-generated ones. Not an optimal solution that is purely git based.

Another option that seems to work is mentioned here: enable maintainer mode by adding AM_MAINTAINER_MODE to configure.ac. This is discouraged in the automake manual. But then, the automake manual’s only reference to a VCS is CVS – which is, well, slightly outdated nowadays.

If anyone has a working solution that fixes the above two problems I would be grateful to hear about it.

So where to go from here in the modern world? The answer is (for me) meson and ninja: no intermediate generation of files, no generated files that are necessary for distribution (configure, Makefile.in), no bunch of maintainer mode, distribution mode, extra complicated targets, readable source files, … For the named project I also created a meson/ninja build system and it turned out to be much easier to handle.

The other path I could take (and actually implemented) is dropping automake, and only use autoconf to build Makefile from a pre-made Makefile.in, without the Makefile.am. This is in our case where there is no need for Makefile.am probably the approach best when one wants to use autotools.

Bottomline for me: give up on automake if you want to use git – either meson/ninja or autoconf only.

15 Responses

  1. Well, Meson by itself has problems. Namely, almost always, a too-new version is required. see e.g. this: https://gitlab.freedesktop.org/pulseaudio/pulseaudio/commit/1b7fab22a419ad284a28e84083ba29cae2fb4b2f

    With Autotools and CMake, developers at least have some of the “respect users of old distributions” mindset by default.

  2. Sune says:

    Tried cmake?

    • Sorry no. I always hate it fighting with cmake because I cannot set prefix only for the installation step, and only have to go via DESTDIR. It looks too complicated for me.

      • Roger Leigh says:

        Simply set `CMAKE_INSTALL_PREFIX` which is semantically the same as `–prefix`. Regarding `DESTDIR`, if you use the `Unix Makefiles` generator, it works identically to Automake as a prefix to the installation prefix. If you have some really complex requirements, please do ask on the list.

        I switched away from the Autotools to CMake about 5 years ago for Debian, work and personal projects. After the initial learning curve, I haven’t looked back. It made me realise how limited and poorly maintained the Autotools really were (and I say this as someone who is a former GNU Autotools contributor who went through all the GNU copyright assignment paperwork stuff). They are sorely outdated when it comes to modern portability problems.

        • Thanks Roger for your comment. Concerning the CMAKE_INSTALL_PREFIX: Maybe it was in old times, but in some projects it did trigger a recompilation AFAIR. Since I use graft (similar to stow) to link stuff into /usr/local I don’t want a recompilation, but simply staged install. But that might be project specific.

          Recently I have also heard lots of good things about cmake, and that it has improved considerable from back then when it was extremely hard to grok.

          • Roger Leigh says:

            If you re-run CMake to change the configuration, I think this is entirely expected and an intentional design choice (and would typically affect Autotools as well due to config.h and similar being touched, assuming your build uses generated headers). If you configure a clean build tree with the correct prefix up front, it should just do the one build as expected.

            It’s definitely improved over the last few years. I’ve been using it since 2.8.x, and recent 3.x releases have so many improvements it’s vastly better. Though it still has rough edges and the documentation could be better, there’s still room for improvement. However, it’s a good pragmatic choice, and unlike the Autotools it hasn’t stagnated. There’s a steady stream of improvements and new features, with reasonable care over backward and forward compatibility, and at least there’s just one language to learn instead of five+! I think the lack of copyright assignment has resulted in hundreds of individual contributors, which led to a very active and healthy ecosystem around it. That never really happened for the Autotools; the Autoconf Archive was about it, and it wasn’t very active even at the best of times.

          • Well, what I do is the equivalent of

            ./configure --prefix=/usr/local ...
            make
            make install prefix=/usr/local/pkgs/foobar
            graft -i foobar

            Normally the above install step does not run a recompilation, only adjusts the destination where files are installed.

            Thanks for your comments on cmake history and development!

    • Frans says:

      I’m always fighting CMake. The scripting language is at least as bad as Makefiles and all the simple things I want to do always turn into a painful -DCMAKE_ETC nightmare. I inwardly sigh every time I see it. By contrast, Meson featured none of my issues with CMake in some quick tests, though I’m sure it has its own unique problems when you actually start using it. But the fact that systemd, GTK+ and Norbert Preining use it certainly speaks in Meson’s favor.

      From my perspective, it looks like the main reason CMake took off is because it has better Windows support than Autotools, or perhaps most people’s goals are just very different from my own.

      CMake feels very magical in the rare cases it works, sure, but it’s very hostile to something boring and simple like quickly cross-compiling something or other for my ereader. In CMake that turns into an ordeal, as does slightly tweaking the build more generally. In Autotools/Make it doesn’t. In Meson it doesn’t seem to either.

      Anyway, that’s just my opinion. You can take it with a grain of salt, but if you know how to tame CMake properly, let me know. 😉

      • Alexander Patrakov says:

        And here is my favorite complaint about CMake. They have a tutorial how to use generated files, with generators written in C or C++: https://cmake.org/cmake-tutorial/#s5 – and it does not cross-compile, at all.

        CMake documentation is completely silent on this issue of generators + cross-builds. I was able to find a solution on the mailing lists (it involves separating out the generator into a subfolder and connecting it via ExternalProject_Add, and importing the resulting executable). For new projects I recommend rewriting all the generators in Python.

        On the other hand, with recent meson, such setup is a piece of cake (6 lines of meson.build).

  3. abc says:

    Isn’t possible to make github tag creation start a job which does “make distcheck; make dist” and then automatically publishes tarballs?

  4. Anonymous says:

    Historically, autotools encouraged the concept of “make dist” including all the built makefiles, whereas from day 1 tools like meson said you have to have meson installed and run it to generate makefiles or ninja files.

    Ignore the historical autotools practice, and treat it like a tool you always have to have installed. *Always* run `autoreconf -v -f -i` before building.

  5. Wookey says:

    Installing the autotools stack and running autoreconf is a good thing. This way you are actually bulding from source, not half-building, and taking some random stuff from old versions of autoconf. That model may have made sense in the 1980s and maybe 1990s when computers were slow and there was much more incompatibility between systems, but it certainly doesn’t make sense any more. Just build the whole thing, using current tools and it’s all likely to work much better.
    So I really don’t see a problem here.
    BTW yet another candidate for a largely-declarative build system is QBS from the Qt people. It’s not terrible.

    • I somehow disagree with this: autoconf is too sensitive to changes in the version of the various auto* tools. In the TeX Live sources we often see people doing auotreconf and messing up everything because they use some slightly old auto* system or just one version different, and boom, configuration fails.

      Furthermore, several developers I work with do *not* have auto* installed.

      But the biggest no-go is simply the github issue. One cannot tag a release and let users get the github generated sources. This is 2018 (soon 2019), this is impossible, an absolute no-go. And what does the auto* manuals have to say to this? Nothing. They mention CVS … and nothing else. I mean, that is previous century …

  6. I’ve found autotools tractable, on Linux. On Windows, qmake. Tried cmake, and it seemed just as complex as autotools, and less mature. CMakefile.txt? Gimme a break. Anyway, to me its a non-issue using autotools on Linux (and testing qmake on Linux as well, in preparation for …) using qmake on Windows.

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> <pre lang="" line="" escaped="" cssfile="">