Ramprasad

/home/resources

[Powered by FreeBSD UNIX]

[Powered by Yahoo! Small Business]

Valid HTML 4.01!

Valid CSS!

Anatomy of the GNU build system

Introduction

Following four GNU tools make the GNU build system:

  1. Autoconf - produces a configuration shell script, named `configure', which probes the installer platform for portability related information which is required to customize makefiles, configuration header files, and other application specific files. Then it proceeds to generate customized versions of these files from generic templates. This way, the user will not need to customize these files manually.
  2. Automake - produces makefile templates, `Makefile.in' to be used by Autoconf, from a very high level specification stored in a file called `Makefile.am'. Automake produces makefiles that conform to the GNU makefile standards, taking away the extraordinary effort required to produce them by hand. Automake requires Autoconf in order to be used properly.
  3. Libtool - makes it possible to compile position indepedent code and build shared libraries in a portable manner. It does not require either Autoconf, or Automake and can be used indepedently. Automake however supports libtool and interoperates with it in a seamless manner.
  4. Autotools - helps you develop portable source code that conforms to the GNU coding standards by generating various boilerplate files from which you can plunge into developing your software.

Advantages of the GNU build system:

  1. Building multidirectory software packages. It is much more difficult to use raw make recursively. Having simplified this step, the developer is encouraged to organize per source code in a deep directory tree rather than lump everything under the same directory. Developers that use raw make often can't justify the inconvenience of recursive make and prefer to disorganize their source code. With the GNU tools this is no longer necessary.
  2. Automatic configuration. You will never have to tell your users that they need to edit your Makefile. You yourself will not have to edit your Makefiles as you move new versions of your code back and forth between different machines.
  3. Automatic makefile generation. Writing makefiles involves a lot of repetition, and in large projects very error-prone. Also, certain portions of a good makefile, such as the `install' and `uninstall' targets are very critical because they are run by the superuser. They must be written without any bugs! The GNU build system automates makefile writing. You are only required to write `Makefile.am' files that are much more terse and easy to maintain.
  4. Support for test suites. You can very easily write test suite code, and by adding one extra line in your `Makefile.am' make a check target available such that you can compile and run the entire test suite by running make check.
  5. Automatic distribution building. The GNU build tools are meant to be used in the development of free software, therefore if you have a working build system in place for your programs, you can create a source code distribution out of it by running make distcheck.
  6. Shared libraries. Building shared libraries becomes as easy as building static libraries.

The GNU build system needs to be installed only when you are developing programs that are meant to be distributed. To build a program from distributed source code, the installer only needs a working make utility, a compiler, a shell, and sometimes standard Unix utilities like sed, awk, yacc, lex. The objective is to make software installation as simple and as automatic as possible for the installer. Also, by setting up the GNU build system such that it creates programs that don't require the build system to be present during their installation, it becomes possible to use the build system to bootstrap itself.

In general, one should follow below simple steps to install any gnu software.

  1. Download the required package file
  2. Deflate the package file
  3. cd to the deflated package dir
  4. ./configure
  5. make
  6. make check
  7. make install

Only few people know how to assemble a project into a package that conforms to GNU build system - the above steps which can be followed by any user to install the project package.

Example: Hello world program

#include <stdio.h>
int main() {
  printf("Hello World!\n");
  return 0;
}

Save this into a file `hello.c' and place it under an empty directory. Simple programs like this can be compiled and ran directly with the following commands:

# gcc hello.c -o hello
# ./hello

If you are on a Unix system instead of a GNU system, your compiler might be called `cc' but the usage will be pretty much the same.

Now to do the same thing the `autoconf' and `automake' way create first the following files:

`Makefile.am'

bin_PROGRAMS = hello
hello_SOURCES = hello.c

`configure.in'

AC_INIT(hello.c)
AM_INIT_AUTOMAKE(hello,0.1)
AC_PROG_CC
AC_PROG_INSTALL
AC_OUTPUT(Makefile)

Now run `autoconf':

# aclocal
# autoconf

This will create the shell script `configure'. Next, run `automake':

# automake -a
required file "./install-sh" not found; installing
required file "./mkinstalldirs" not found; installing
required file "./missing" not found; installing
required file "./INSTALL" not found; installing
required file "./NEWS" not found
required file "./README" not found
required file "./COPYING" not found; installing
required file "./AUTHORS" not found
required file "./ChangeLog" not found

The first time you do this, `automake' will complain a couple of things. First it notices that the files `install-sh', `mkinstalldirs' and `missing' are not present, and it installs copies. These files contain boiler-plate shell scripts that are needed by the makefiles that `automake' generates. It also complains that the following files are not around: INSTALL, COPYING, NEWS, README, AUTHORS, ChangeLog

These files are required to be present by the GNU coding standards, and we discuss them in detail in section Maintaining the documentation files. At this point, it is important to at least touch these files, otherwise if you attempt to do a `make distcheck' it will deliberately fail. To make these files exist, type:

# touch NEWS README AUTHORS ChangeLog

and to make Automake aware of the existence of these files, rerun it:

# automake -a

You can assume that the generated `Makefile.in' is correct, only when Automake completes without any error messages.

Now the package is exactly in the state that the end-user will find it when person unpacks it from a source code distribution. For future reference, we will call this state autoconfiscated. Being in an autoconfiscated state means that, you are ready to type:

# ./configure
# make
# ./hello

to compile and run the hello world program. If you really want to install it, go ahead and call the `install' target:

# make install

To undo installation, that is to uninstall the package, do:

# make uninstall

If you didn't use the `--prefix' argument to point to your home directory, or a directory in which you have permissions to write and execute, you may need to be superuser to invoke the install and uninstall commands. If you feel like cutting a source code distribution, type:

# make distcheck

This will create a file called `hello-0.1.tar.gz' in the current working directory that contains the project's source code, and test it out to see whether all the files are actually included and whether the source code passes the regression test suite.

In order to do all of the above, you need to use the GNU `gcc' compiler. Automake depends on `gcc''s ability to compute dependencies. Also, the `distcheck' target requires GNU make and GNU tar.

The GNU build tools assume that there are two types of hats that people like to wear: the developer hat and the installer hat. Developers develop the source code and create the source code distribution. Installers just want to compile and install a source code distribution on their system. In the free software community, the same people get to wear either hat depending on what they want to do. If you are a developer, then you need to install the entire GNU build system, period (see section Installing the GNU build system). If you are an installer, then all you need to compile and install a GNU package is a minimal `make' utility and a minimal shell. Any native Unix shell and `make' will work.

Both Autoconf and Automake take special steps to ensure that packages generated through the `distcheck' target can be easily installed with minimal tools. Autoconf generates `configure' shell scripts that use only portable Bourne shell features. Automake ensures that the source code is in an autoconfiscated state when it is unpacked. It also regenerates the makefiles before adding them to the distribution, such that the installer targets (`all', `install', `uninstall', `check', `clean', `distclean') do not depend on GNU make features. The regenerated makefiles also do not use the `gcc' cruft to compute dependencies. Instead, precomputed dependencies are included in the regenerated makefiles, and the dependencies generation mechanism is disabled. This will allow the end-user to compile the package using a native compiler, if the GNU compiler is not available. For future reference we will call this the installer state.

Now wear your installer hat, and install `hello-0.1.tar.gz':

# gunzip hello-0.1.tar.gz
# tar xf hello-0.1.tar
# cd hello-0.1
# configure
# make
# ./hello

This is the full circle. The distribution compiles, and by typing `make install' it installs. If you need to switch back to the developer hat, then you should rerun `automake' to get regenerate the makefiles.

When you run the `distcheck' target, `make' will create the source code distribution `hello-0.1.tar.gz' and it will pretend that it is an installer and see if it the distribution can be unpacked, configured, compiled and installed. It will also run the test suite, if one is bundled. If you would like to skip these tests, then run the `dist' target instead:

# make dist

Nevertheless, running `distcheck' is extremely helpful in debugging your build cruft. Please never release a distribution without getting it through `distcheck'. If you make daily distributions for off-site backup, please do pass them through `distcheck'. If there are files missing from your distribution, the `distcheck' target will detect them. If you fail to notice such problems, then your backups will be incomplete leading you to a false sense of security.

Explanation

When you made the `hello-0.1.tar.gz' distribution, most of the files were automatically generated. The only files that were actually written by your fingers were:

`hello.c'

#include <stdio.h>
int main() {
  printf("Hello World!\n");
  return 0;
}

`Makefile.am'

bin_PROGRAMS = hello
hello_SOURCES = hello.c

`configure.in'

AC_INIT(hello.cc)
AM_INIT_AUTOMAKE(hello,1.0)
AC_PROG_CC
AC_PROG_INSTALL
AC_OUTPUT(Makefile)

In this section we explain briefly what the files `Makefile.am' and `configure.in' mean.

The language of `Makefile.am' is a logic language. There is no explicit statement of execution. Only a statement of relations from which execution is inferred. On the other hand, the language of `configure.in' is procedural. Each line of `configure.in' is a command that is executed.

Seen in this light, here's what the `configure.in' commands shown do:

The AC_INIT command initializes the configure script. It must be passed as argument the name of one of the source files. Any source file will do. The AM_INIT_AUTOMAKE performs some further initializations that are related to the fact that we are using `automake'. If you are writing your `Makefile.in' by hand, then you don't need to call this command. The two comma-separated arguments are the name of the package and the version number. The AC_PROG_CC checks to see which C compiler you have. The AC_PROG_INSTALL checks to see whether your system has a BSD compatible install utility. If not then it uses `install-sh' which `automake' will install at the root of your package directory if it's not there yet. The AC_OUTPUT tells the configure script to generate `Makefile' from `Makefile.in' The `Makefile.am' is more obvious. The first line specifies the name of the program we are building. The second line specifies the source files that compose the program.

For now, as far as `configure.in' is concerned you need to know the following additional facts:

If you are building a library, then your configure script must determine how to handle `ranlib'. To do that, add the AC_PROG_RANLIB command. If your source code contains C++ files, you need to add the AC_PROG_CXX to your `configure.in'. If your source code contains `yacc' and `lex' files, then you need to add:

AC_PROG_YACC
AC_PROG_LEX

to your `configure.in'. If your source code contains Fortran source code, you need to add `AC_PROG_FC' to your code. If you want to mix C and Fortran, then you need to do a lot more than just that. If you have any makefiles in subdirectories you must also put them in the AC_OUTPUT statement like this:

AC_OUTPUT( Makefile \
  dir1/Makefile \
  dir2/Makefile \
)

Note that the backslashes are not needed if you are using the bash shell. For portability reasons, however, it is a good idea to include them. Make sure that every subdirectory where building takes place, is mentioned! Now consider the commands that are used to build the hello world distribution:

# aclocal
# autoconf
# touch README AUTHORS NEWS ChangeLog
# automake -a
# ./configure
# make

The first three commands bring the package in autoconfiscated state. The remaining two commands do the actual configuration and building. More specifically:

  • The `aclocal' command installs a file called `aclocal.m4'. Normally, in that file you are supposed to place the definitions of any `autoconf' macros that you've written that happen to be in use in `configure.in'. We will teach you how to write `autoconf' macros later. The `automake' utility uses the AM_INIT_AUTOMAKE macro which is not part of the standard `autoconf' macros. For this reason, it's definition needs to be placed in `aclocal.m4'. If you call `aclocal' with no arguments then it will generate the appropriate `aclocal.m4' file. Later we will show you how to use `aclocal' to also install your own `autoconf' macros.
  • The `autoconf' command combines the `aclocal.m4' and `configure.in' files and produces the `configure' script. And now we are in bussiness.
  • The `touch' command makes the files `README' and friends exist. It is important that these files exist before calling Automake, because Automake decides whether to include them in a distribution by checking if they exist at the time that you invoke `automake'. Automake must decide to include these files, because when you type `make distcheck' the presense of these files will be required.
  • The `automake' command compiles a `Makefile.in' file from `Makefile.am' and if absent it installs various files that are required either by the GNU coding standards or by the makefile that will be generated.

The `configure' script probes your platform and generates makefiles that are customized for building the source code on your platform. The specifics of how the probing should be done are programmed in `configure.in'. The generated makefiles are based on templates that appear in `Makefile.in' files. In order for these templates to cooperate with `configure' and produce makefiles that conform to the GNU coding standards they need to contain a tedious amount of boring stuff. This is where Automake comes in. Automakes generates the `Makefile.in' files from the more terse description in `Makefile.am'. As you have seen in the example, `Makefile.am' files can be very simple in simple cases. Once you have customized makefiles, your make utility takes over.

How does `configure' actually convert the template `Makefile.in' to the final makefile? The `configure' script really does two things:

  • It maintains a list of substitutions that it accumulates while probing the installer platform. Each one of these substitutions consists of a symbolic name, and the actual text that we want to substitute. When the `configure' script runs AC_OUTPUT it parses all of the files listed in AC_OUTPUT and every occurance of @FOO@ in these files is substituted with the text that corresponds to FOO. For example, if you add the following lines to `configure.in' you will cause @FOO@ to be substituted with `hello':

    FOO="hello"
    AC_SUBST(FOO)

    This is how `configure' exports compile-time decisions to the makefile, such as what compiler to use, what flags to pass the compilers and so on. Occasionally, you want to use `configure''s substitution capability directly on files that are not makefiles. This is why it is important to be aware of it.
  • It maintains a list of C preprocessor macros with defined values that it also accumulates while probing the installer platforms. Before finishing off, `configure' will either generate a configuration file that defines these C preprocessor macros to the desired values, or set a flag in the generated makefile (through substitution) that will pass `-D' flags to the compiler. We discuss configuration headers in the following section.


Last modified: 10/12/2008