The One and the Many

On GNU Make's expansion of prerequisites

I use GNU Make frequently. I use it to build software projects of course, such as any C project I write. I also use it for things where its dependency resolution behaviour fits well. For example, I generate this website with it: Each page on the website depends on a common header and footer.

I've been doing Advent of Code this month. I am solving the problems using C, mainly for practice with it. You can see my solutions on Github. To build these programs I use Make.

I started out with a Makefile I copied from one of my other projects. It is large as I turn on every gcc warning flag that sounds remotely helpful. (I periodically go through the gcc man page and read about each warning flag. If a flag sounds useful, I add it to my list if it is not enabled by another flag.) For each day's problems I have been copying this Makefile into a new directory and modifying it to build the programs for that day. This lead to a lot of duplication. I was okay with this because these are essentially write only, but I began thinking it was silly anyway.

I decided I should split out the common parts of the Makefile so I could share them each day. I thought this could also save time because I could write the Makefile to do more automatically. Ideally I would like to avoid having to edit it each day.

The parts I edited every day look like this:

TARGETS=problemN-1 problemN-2

problemN-1: problemN-1.c
    $(CC) $(CFLAGS) -o $@ problemN-1.c

problemN-2: problemN-2.c
    $(CC) $(CFLAGS) -o $@ problemN-2.c

Where N would vary depending on the day.

The common parts that I never altered look like this:

CC=gcc

CFLAGS= (a lot of flags)

all: $(TARGETS)

clean:
    rm -f $(TARGETS)

Quite basic. I opted to be explicit rather than relying on implicit rules or on patterns. I feel those features can get difficult to understand after a while, but of course there is a balance against how much time they save.

My first thought was to have a common Makefile that would look like this:

CC=gcc

CFLAGS= (a lot of flags)

all: $(TARGETS)

clean:
    rm -f $(TARGETS)

Then for each day I would have a Makefile that would look like this:

include ../Makefile.mk

TARGETS=problemN-1 problemN-2

problemN-1: problemN-1.c
    $(CC) $(CFLAGS) -o $@ problemN-1.c

problemN-2: problemN-2.c
    $(CC) $(CFLAGS) -o $@ problemN-2.c

I was not sure if this would work. In fact, it does not. Running make shows this output:

make: Nothing to be done for 'all'.

I found that if I echoed out the TARGETS variable in the all target's recipe I would see the program names I expected:

all: $(TARGETS)
    @echo $(TARGETS)

This would echo:

problemN-1 problemN-2

I discovered this was because Make expands variables and prerequisites differently. Make expands variables when it uses them (such as in a recipe). It expands prerequisites as it parses the Makefile. (Note that Make also supports alternate ways of using variables where this differs).

This explains why I could echo TARGETS and see the programs I defined in this variable after including the common Makefile: Make looked up TARGETS when running the recipe. It also explains why it saw nothing to do for the all target: When it reached the rule for the all target, TARGETS was not set, so it did not add any prerequisites to all.

The documentation for Make is quite comprehensive. I discovered the above information in a section entitled How make Reads a Makefile. It says this:

We say that expansion is immediate if it happens during the first phase: in this case make will expand any variables or functions in that section of a construct as the makefile is parsed. We say that expansion is deferred if expansion is not performed immediately. Expansion of a deferred construct is not performed until either the construct appears later in an immediate context, or until the second phase.

It states a rule definition expands as follows:

immediate : immediate ; deferred
    deferred

While I could make my initial approach work by setting TARGETS before including the common Makefile, I decided it would be better if I wrote my Makefile so Make could find what programs to build automatically, and put that in the common Makefile. This would solve the second part of what I wanted: Less need to edit the Makefile each day.

I ended up writing the Makefiles as follows:

The common Makefile:

CC=gcc

CFLAGS= (a lot of flags)

# Assume all .c files correspond to a program to build.
TARGETS=$(patsubst %.c,%,$(wildcard *.c))

all: $(TARGETS)

# This is a static pattern rule to build all .c programs (TARGETS).
# Don't define a recipe. Instead, rely on implicit rules to build these. If
# we need to give more specific recipes, we can override the implicit rule
# by writing a recipe for the target.
$(TARGETS): %: %.c

clean:
  rm -f $(TARGETS)

And the Makefile for each day looks like this:

include ../Makefile.mk

One line! In this Makefile I can override the implicit rule if needed, such as if I need to link in additional libraries. Here is an example where I do that (this links in OpenSSL so I can use its MD5 hash function):

include ../Makefile.mk

problem5-1: problem5-1.c
    $(CC) $(CFLAGS) -o $@ problem5-1.c -lssl -lcrypto

problem5-2: problem5-2.c
    $(CC) $(CFLAGS) -o $@ problem5-2.c -lssl -lcrypto

How does the common Makefile work?

I am happy with this solution, and it was fun to figure out. I tend to not use some of Make's more powerful features very often, so it's good to read about it periodically. Making this change definitely met the two goals I had. The commit deleted 533 lines (and added 71)! As well, I will now likely not have to edit the Makefile most days.