These days, most users don’t need to build open source software from its source code, given the availability of package managers such as
rpm, etc. But there are many good reasons to want to build software from source. Perhaps the package manager offers an older version and you want the latest, greatest features. Maybe you want to ensure the software is built with specific optimizations or debugging flags. Or, perhaps you need to cross-compile for a different architecture. In any case, building software from source code is usually straightforward.
In this article, I’ll explain what goes on behind this process to help you get the build that best suits your needs. I’ll focus mostly on software written in C/C++ or other compiled languages. Such software is usually distributed with a means to build it flexibly and portably, and there are two predominant build systems today: those based on the GNU Autotools and those based on CMake. Autotools is by far the most commonly encountered method, and it is the official GNU build system.
Autotools is a suite of complex tools (autoconf, automake, libtool) used by the developer to package the build. However, users typically are interested only in the end product of applying those tools, which is a Bash shell script conventionally named
configure. For users, the build instructions for the software are often as simple as:
- Unwrap the software in a convenient directory.
cdinto that directory.
- Run the following three commands:
That’s it—what could be simpler? There is, of course, more to the story, but let’s talk about what happens with the first of those three commands.
configure command is a large, complex shell script that runs and reports on a lengthy series of tests on your system. It looks in well-known locations for various header files, libraries, APIs, functions, etc. It uses this information to build a Makefile that is tailored to the specific architecture and operating system of your machine. The default target of the Makefile builds the software, and the
install target properly installs the software and any supporting files.
Sometimes, there’s an optional step to run a series of tests on the software itself to verify the integrity of the build, which should be run before the installation step:
The complexity implicit behind these simple commands leads to several questions:
- Where will the software be installed?
- What optional features are included in the build?
- What optional third-party packages will be incorporated?
- How can I help
configurefind certain requisite libraries or influence other aspects of the build?
To answer those questions, I recommend that, before attempting a build, you run the command:
This command will produce output similar to the following, which is excerpted from the
configure for the
libcurl package. I’ve highlighted key bits of the output that align with the four questions above.
`configure' configures curl - to adapt to many kinds of systems. Usage: ./configure [OPTION]... [VAR=VALUE]... Defaults for the options are specified in brackets. Configuration: -h, --help display this help and exit --help=short display options specific to this package --help=recursive display the short help of all the included packages text clipped . . . . . Installation directories: --prefix=PREFIX install architecture-independent files in PREFIX [/usr/local] By default, `make install' will install all the files in `/usr/local/bin', `/usr/local/lib' etc. You can specify an installation prefix other than `/usr/local' using `--prefix', for instance `--prefix=$HOME'. Fine tuning of the installation directories: --bindir=DIR user executables [EPREFIX/bin] --sbindir=DIR system admin executables [EPREFIX/sbin] --libexecdir=DIR program executables [EPREFIX/libexec] --includedir=DIR C header files [PREFIX/include] text clipped . . . . . Optional Features: --disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no) --enable-FEATURE[=ARG] include FEATURE [ARG=yes] --enable-http Enable HTTP support --disable-http Disable HTTP support --enable-ldap Enable LDAP support --disable-ldap Disable LDAP support --enable-ldaps Enable LDAPS support text clipped . . . . . Optional Packages: --with-PACKAGE[=ARG] use PACKAGE [ARG=yes] --without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no) --with-zlib=PATH search for zlib in PATH --without-zlib disable use of zlib --with-darwinssl enable Apple OS native SSL/TLS --without-darwinssl disable Apple OS native SSL/TLS text clipped . . . . . Some influential environment variables: CC C compiler command CFLAGS C compiler flags LDFLAGS linker flags, e.g. -L<lib dir> if you have libraries in a nonstandard directory <lib dir> LIBS libraries to pass to the linker, e.g. -l<library> CPPFLAGS (Objective) C/C++ preprocessor flags, e.g. -I<include dir> if you have headers in a nonstandard directory <include dir> CPP C preprocessor text clipped . . . . .
Where Do Things Get Installed?
The first highlighted section answers the question of where things get installed. It tells us that by default, the package will be installed under
/usr/local. That means binaries will go into
/usr/local/bin, libraries under
/usr/local/lib, header files under
/usr/local/include, manual pages under
/usr/local/man, miscellaneous files under
/usr/local/share, etc. (Note that superuser privileges will likely be required to install in that location.)
We can change the root of that directory hierarchy by using the
--prefix switch. We also see that there are other more specialized switches beyond
--prefix that can be used to highly customize the installation, such that binaries can go in one place (
--bindir), libraries in another (
--libexecdir), and documentation in yet another (not shown). None of these are required to have a common root prefix. I personally never use those switches, as it does not make much sense to me to scatter package files about the filesystem.
The “Optional Features” section answers our second question. It lists switches that we can use to turn on or off optional behaviors of the software. As seen in the listing, these switches tend to have a consistent form of
disable-FEATURE. They often take arguments of the form
enable-FEATURE=no is synonymous with
disable-FEATURE, and where
auto indicates the feature will be included in the build if
configure can find the supporting headers/libraries.
Occasionally, you’ll see the option
DIR is a directory root where supporting headers/libraries may be found, if they are not installed in well-known locations. It is not uncommon to see dozens of optional features switches; the 2D graphics library cairo, for example, has more than 60 such optional features.
Our third question is addressed by the “Optional Packages” section. These switches also tend to have a consistent form of
without-PACKAGE, where as before
with-PACKAGE=no is identical to
without-PACKAGE. There is not a sharp distinction between what is considered an optional feature versus an optional package, but packages tend to imply that some non-standard library is required in order for the software to support a certain feature.
In our example output, the compression library
zlib is an optional package, implying that if
libcurl is built using that library, network protocols that support such compression will be enabled in the
libcurl library. As before, an argument of
DIR to the switch specifies the root directory for the headers/libraries of the optional package in cases when they are not installed in well-known directories. It is also not unusual to see large numbers of optional packages; for example, the geospatial processing software Gdal has more than a hundred.
Customize the Build
Finally, the section “Some influential environment variables” describes how to change the
configure command’s default assumptions about the build, such as which compiler should be used or what optimization flags should be set, etc. Unlike switches, these variables are specified on the command line as
VAR=VALUE pairs. Note that if multiple values are to be passed to a variable, the whole
VALUE part should be quoted.
For example, passing options to the C compiler might look like
CFLAGS=”-I/opt/include -O3”. The variables shown in the
libcurl example are fairly common and generic; however, some packages also define unique and specific variables intended to identify the location of individual optional packages. We’ll see this in the example below.
The example below shows how I build the Gdal package. Gdal is a large package that supports dozens of geospatial file formats, most of them optionally. In my particular application, I strive for the leanest build possible, so I want to exclude a number of formats that Gdal might find on my system. So, in this case,
configure is invoked as:
./configure --prefix=/usr/local --disable-shared \ --with-proj=/usr/local --without-pam --with-png=/usr/local \ --with-gif=internal --with-libtiff=internal --with-geotiff=internal \ --with-sqlite3=no --with-expat=no --with-curl=no --with-hdf5=yes \ --without-grib --with-freexl=no LDFLAGS=-L/uwr/local/lib \ HDF5_CFLAGS=-I/usr/local/include HDF5_LIBS=”-L/usr/local/lib -lhdf5” \ CXX=”g++ -std=c++11”
This example is fairly extreme, but it contains all of the elements discussed above. At least one optional feature is disabled. Several optional packages are included or excluded, and in some cases, directory paths are given to indicate where to find those packages’ header files and libraries. Two environment variables provide directives to the C++ compiler and the linker, and Gdal utilizes special environment variables for locating the HDF5 package. Note that the
VALUE parts of several variables are quoted, as there are multiple pieces to the values.
Rarely do I end up with a
configure command line as complicated as the example above. But there may be times when you need to rebuild a software package and can’t recall the options you used. We noted that the purpose of
configure is to generate Makefiles. In most cases,
configure will also write two files,
config.status. These are mostly intended for debugging purposes for the developer using Autotools to construct the build system. However, you can also refer to the
config.log to see how
configure was previously invoked.
As I’ve shown,
configure provides a great deal of flexibility and customization for a build. Understanding how it works and being aware of the options available can be very helpful when building software from source.
In this article, I presented a rather complex example of
configure in action, mostly to illustrate its features. However, I frequently build open source software, and in my experience, it is rarely more complicated than simply typing:
But remember to always do this first:
You might be surprised by your options.
Ready to find a job? Check out the latest job listings at Open Source JobHub.