Dynamic Dependency Checking

Dynamic dependency checking is supported which allows a commmand (e.g. a file scanner) to automatically determine dependencies.

Consider the following file, a.c

#include <stdio.h>
#include <common.h>
#include "a.h"
...

And a.h

#include "b.h"

The built-in rulebase automatically associates the built-in recursive regex scanner with rules to build object files (.o) from source files (.c) via a rule that looks like:

a.o: a.c
dyndep=header-scan-regexp-recursive $INCPATHS "" $CHDRPATTERN
  var INCPATHS=publish/include .
    run $CCACHE $CC $C_FLAGS $CFLAGS -c $inputs -o $target

Note that the 'dyndep' attribute specifies a command prefix to run that will return a list of dependent targets/files.

The built-in scanner:

• Uses a regular expression to scan for dependencies of the form #include <abc.h> or #include "abc.h"
• Ignores conditional compilation (#ifdef, etc.)
  • And thus is conservative in that it can make create an unnecessary dependency (but not the reverse)
• Is recursive, so that in the example above, a.o depends upon both a.h and b.h
• Knows if a dependency is:
  • a target, and thus must be generated before being scanned recursively
  • an existing source file, and thus creates a time or hash-based dependeny
  • missing, and thus assumed to be a system header, and ignored
• Caches the results of the scan to avoid unnecessary scanning
• Carefully checks to see that the scan would return the same results as the cache to ensure that changes such as creating files in-tree which shadow system headers cause a rebuild

Dynamic dependencies:

• Are cached, along with the command (paths, pattern, ...), so that the (small) cost of scanning is only incurred when the file changes
• The set of dependencies is cached. If this set changes, the target is rebuilt. This can occur if a system header was previously shadowed by a source file, but the build changes so that this no longer occurs, or if a new header file is added in a directory earlier in the set of include paths.

Note that only inputs are passed to the dynamic dependency scanner, not all dependencies. For example, if a rule uses a generator, we typically don't want to scan the generator for dynamic dependencies, only the input files. If the generator has it's own dynamic dependencies, they should be included in the rule to build that generator. (If it is a script generator, PublishBin --copy can be used to add dynamic dependencies).

XXX: Note that "#include INCLUDEFILENAME" is not supported. Suggest workarounds.