Wildcarding sources

tmake attempts to allow both the simplest build description necessary while also being as correct and safe as possible. Consider a simple build description.

$ cat build.spec
Executable prog *.c

For many projects (or directories within a project) this is a perfectly reasonable and correct build description. However many other modern build systems do not support this. Consider the explanation from meson - No wildcards for meson

Meson does not support this syntax and the reason for this is simple. This can not be made both reliable and fast. By reliable we mean that if the user adds a new source file to the subdirectory, Meson should detect that and make it part of the build automatically.

One of the main requirements of Meson is that it must be fast. This means that a no-op build in a tree of 10 000 source files must take no more than a fraction of a second. This is only possible because Meson knows the exact list of files to check. If any target is specified as a wildcard glob, this is no longer possible. Meson would need to re-evaluate the glob every time and compare the list of files produced against the previous list. This means inspecting the entire source tree (because the glob pattern could be src/\*/\*/\*/\*.cpp or something like that). This is impossible to do efficiently.

The main backend of Meson is Ninja, which does not support wildcard matches either, and for the same reasons.

Because of this, all source files must be specified explicitly.

However tmake does not force this either-or choice on you. Instead you can certainly list out all source files explicitly if desired (and this can improve build speed), however wildcards are also supported and they will be expanded during the parse phase with the results available in the build phase. This can be a very useful choice for small to medium size projects

$ ls
build.spec
prog.c
project.spec

$ tmake 
    Cc           prog.o
    Link         prog
Built 2 of 2 target(s) in 0.25 seconds

Now adding a new file is simple.

$ touch add.c
$ tmake 
    Cc           add.o
    Link         prog
Built 2 of 3 target(s) in 0.14 seconds

And removing a file is also simple.

$ rm add.c
$ tmake 
    Clean        removing 1 orphan target(s)
    Link         prog
Built 1 of 2 target(s) in 0.10 seconds

tmake notices that add.c no longer exists, and thereforce add.o is no longer a target. It automatically removes the previous add.o to avoid an inconsistent build.

More sophisticated wildcards may be used with the Glob command.

As a test of wildcarding performance, my Linux VM

gen_src.py.txt -> 1000 files

With wildcarding:

With explicit files: full build: 14.81s, do nothing build: 0.27s With wildcarding: full build: 12.72s, do nothing build: 0.27s

meson/ninja: full build: 6s, do nothing build: 0.01s

On my build systems, it takes around 1-2ms to glob 1000 files (with a hot cache)