These days, most users don’t need to build open source software from its source code, given the availability of package managers such as apt-get, rpm, etc. But there are many good reasons to want to build software from source. Perhaps the package manager offers an older version and you want the latest, greatest features. Maybe you want to ensure the software is built with specific optimizations or debugging flags. Or, perhaps you need to cross-compile for a different architecture. In any case, building software from source code is usually straightforward.
In this article, I’ll explain what goes on behind this process to help you get the build that best suits your needs. I’ll focus mostly on software written in C/C++ or other compiled languages. Such software is usually distributed with a means to build it flexibly and portably, and there are two predominant build systems today: those based on the GNU Autotools and those based on CMake. Autotools is by far the most commonly encountered method, and it is the official GNU build system.
Autotools
Autotools is a suite of complex tools (autoconf, automake, libtool) used by the developer to package the build. However, users typically are interested only in the end product of applying those tools, which is a Bash shell script conventionally named configure. For users, the build instructions for the software are often as simple as:
- Unwrap the software in a convenient directory.
cdinto that directory.- Run the following three commands:
./configuremakemake install
That’s it—what could be simpler? There is, of course, more to the story, but let’s talk about what happens with the first of those three commands.
Hidden Complexity
The configure command is a large, complex shell script that runs and reports on a lengthy series of tests on your system. It looks in well-known locations for various header files, libraries, APIs, functions, etc. It uses this information to build a Makefile that is tailored to the specific architecture and operating system of your machine. The default target of the Makefile builds the software, and the install target properly installs the software and any supporting files.
Sometimes, there’s an optional step to run a series of tests on the software itself to verify the integrity of the build, which should be run before the installation step:
make check
The complexity implicit behind these simple commands leads to several questions:
- Where will the software be installed?
- What optional features are included in the build?
- What optional third-party packages will be incorporated?
- How can I help
configurefind certain requisite libraries or influence other aspects of the build?
To answer those questions, I recommend that, before attempting a build, you run the command:
./configure --help
This command will produce output similar to the following, which is excerpted from the configure for the libcurl package. I’ve highlighted key bits of the output that align with the four questions above.
`configure' configures curl - to adapt to many kinds of systems.
Usage: ./configure [OPTION]... [VAR=VALUE]...
Defaults for the options are specified in brackets.
Configuration:
-h, --help display this help and exit
--help=short display options specific to this package
--help=recursive display the short help of all the included packages
text clipped . . . . .
Installation directories:
--prefix=PREFIX install architecture-independent files in PREFIX
[/usr/local]
By default, `make install' will install all the files in
`/usr/local/bin', `/usr/local/lib' etc. You can specify
an installation prefix other than `/usr/local' using `--prefix',
for instance `--prefix=$HOME'.
Fine tuning of the installation directories:
--bindir=DIR user executables [EPREFIX/bin]
--sbindir=DIR system admin executables [EPREFIX/sbin]
--libexecdir=DIR program executables [EPREFIX/libexec]
--includedir=DIR C header files [PREFIX/include]
text clipped . . . . .
Optional Features:
--disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no)
--enable-FEATURE[=ARG] include FEATURE [ARG=yes]
--enable-http Enable HTTP support
--disable-http Disable HTTP support
--enable-ldap Enable LDAP support
--disable-ldap Disable LDAP support
--enable-ldaps Enable LDAPS support
text clipped . . . . .
Optional Packages:
--with-PACKAGE[=ARG] use PACKAGE [ARG=yes]
--without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no)
--with-zlib=PATH search for zlib in PATH
--without-zlib disable use of zlib
--with-darwinssl enable Apple OS native SSL/TLS
--without-darwinssl disable Apple OS native SSL/TLS
text clipped . . . . .
Some influential environment variables:
CC C compiler command
CFLAGS C compiler flags
LDFLAGS linker flags, e.g. -L<lib dir> if you have libraries in a
nonstandard directory <lib dir>
LIBS libraries to pass to the linker, e.g. -l<library>
CPPFLAGS (Objective) C/C++ preprocessor flags, e.g. -I<include dir> if
you have headers in a nonstandard directory <include dir>
CPP C preprocessor
text clipped . . . . .
Where Do Things Get Installed?
The first highlighted section answers the question of where things get installed. It tells us that by default, the package will be installed under /usr/local. That means binaries will go into /usr/local/bin, libraries under /usr/local/lib, header files under /usr/local/include, manual pages under /usr/local/man, miscellaneous files under /usr/local/share, etc. (Note that superuser privileges will likely be required to install in that location.)
We can change the root of that directory hierarchy by using the --prefix switch. We also see that there are other more specialized switches beyond --prefix that can be used to highly customize the installation, such that binaries can go in one place (--bindir), libraries in another (--libexecdir), and documentation in yet another (not shown). None of these are required to have a common root prefix. I personally never use those switches, as it does not make much sense to me to scatter package files about the filesystem.
Optional Features
The “Optional Features” section answers our second question. It lists switches that we can use to turn on or off optional behaviors of the software. As seen in the listing, these switches tend to have a consistent form of enable-FEATURE and disable-FEATURE. They often take arguments of the form enable-FEATURE=[no/auto/yes], where enable-FEATURE=no is synonymous with disable-FEATURE, and where auto indicates the feature will be included in the build if configure can find the supporting headers/libraries.
Occasionally, you’ll see the option enable-FEATURE=DIR, where DIR is a directory root where supporting headers/libraries may be found, if they are not installed in well-known locations. It is not uncommon to see dozens of optional features switches; the 2D graphics library cairo, for example, has more than 60 such optional features.
Third-Party Packages
Our third question is addressed by the “Optional Packages” section. These switches also tend to have a consistent form of with-PACKAGE=[yes/no/DIR] and without-PACKAGE, where as before with-PACKAGE=no is identical to without-PACKAGE. There is not a sharp distinction between what is considered an optional feature versus an optional package, but packages tend to imply that some non-standard library is required in order for the software to support a certain feature.
In our example output, the compression library zlib is an optional package, implying that if libcurl is built using that library, network protocols that support such compression will be enabled in the libcurl library. As before, an argument of DIR to the switch specifies the root directory for the headers/libraries of the optional package in cases when they are not installed in well-known directories. It is also not unusual to see large numbers of optional packages; for example, the geospatial processing software Gdal has more than a hundred.
Customize the Build
Finally, the section “Some influential environment variables” describes how to change the configure command’s default assumptions about the build, such as which compiler should be used or what optimization flags should be set, etc. Unlike switches, these variables are specified on the command line as VAR=VALUE pairs. Note that if multiple values are to be passed to a variable, the whole VALUE part should be quoted.
For example, passing options to the C compiler might look like CFLAGS=”-I/opt/include -O3”. The variables shown in the libcurl example are fairly common and generic; however, some packages also define unique and specific variables intended to identify the location of individual optional packages. We’ll see this in the example below.
Real-World Example
The example below shows how I build the Gdal package. Gdal is a large package that supports dozens of geospatial file formats, most of them optionally. In my particular application, I strive for the leanest build possible, so I want to exclude a number of formats that Gdal might find on my system. So, in this case, configure is invoked as:
./configure --prefix=/usr/local --disable-shared \ --with-proj=/usr/local --without-pam --with-png=/usr/local \ --with-gif=internal --with-libtiff=internal --with-geotiff=internal \ --with-sqlite3=no --with-expat=no --with-curl=no --with-hdf5=yes \ --without-grib --with-freexl=no LDFLAGS=-L/uwr/local/lib \ HDF5_CFLAGS=-I/usr/local/include HDF5_LIBS=”-L/usr/local/lib -lhdf5” \ CXX=”g++ -std=c++11”
This example is fairly extreme, but it contains all of the elements discussed above. At least one optional feature is disabled. Several optional packages are included or excluded, and in some cases, directory paths are given to indicate where to find those packages’ header files and libraries. Two environment variables provide directives to the C++ compiler and the linker, and Gdal utilizes special environment variables for locating the HDF5 package. Note that the VALUE parts of several variables are quoted, as there are multiple pieces to the values.
Config.log
Rarely do I end up with a configure command line as complicated as the example above. But there may be times when you need to rebuild a software package and can’t recall the options you used. We noted that the purpose of configure is to generate Makefiles. In most cases, configure will also write two files, config.log and config.status. These are mostly intended for debugging purposes for the developer using Autotools to construct the build system. However, you can also refer to the config.log to see how configure was previously invoked.
Closing Comments
As I’ve shown, configure provides a great deal of flexibility and customization for a build. Understanding how it works and being aware of the options available can be very helpful when building software from source.
In this article, I presented a rather complex example of configure in action, mostly to illustrate its features. However, I frequently build open source software, and in my experience, it is rarely more complicated than simply typing:
./configure
But remember to always do this first:
./configure --help
You might be surprised by your options.
Ready to find a job? Check out the latest job listings at Open Source JobHub.
Comments