It wasn't.

The hard part was deciding when I had enough evidence to safely not run one.

Most C++ projects have a simple answer when code changes: run the test suite. That is safe, but it can get expensive fast. Change one header in a large project and there is a good chance most tests have nothing to do with it.

So I spent the Zero Dependency Hackathon building diff2test, a C++20 CLI that tries to answer a narrower question:

Given these changed files, which CTest tests can I justify running?

The word justify ended up mattering more than I expected.

diff2test reads changed paths and metadata that a CMake/CTest build has already produced. It reconstructs the relationship between files, translation units, targets, executables, and tests. If all the evidence checks out, it emits the affected subset.

If something important is missing or suspicious, the optimization disappears.

That decision shaped almost everything that followed.

I already had the graph. It was just scattered everywhere.

I did not want to predict test impact from filenames.

No parser.cpp probably means ParserTest. No directory heuristics. No fuzzy matching. No history model.

A normal C++ build already knows much more useful information.

GCC and Clang can emit Make-style .d files containing the prerequisites of each compilation.

CMake's File API can describe targets, their sources, their dependencies, and the artifacts they produce.

CTest can export its registered tests and their commands as JSON.

Put those pieces together and the path I wanted looked roughly like this:

changed path
    ↓
compiler .d file
    ↓
translation unit
    ↓
CMake target
    ↓
targets that depend on it
    ↓
executable artifact
    ↓
CTest test

That looked almost suspiciously convenient.

Then I checked the hackathon rules.

The organizer email changed the project

My first design would have been much easier.

diff2test could run git diff, ask CMake for metadata, invoke CTest, maybe use a helper command to discover dependency files, and combine the results.

There was one problem: this was the Zero Dependency Hackathon.

I emailed the organizers and asked specifically whether launching Git, CMake, or CTest from the program would count as depending on separately installed software.

Their answer was yes.

If my executable shelled out to git, then Git was part of its runtime dependency story. Same for CMake and CTest.

They did give me an important escape route: parsing files those tools had already generated was allowed, provided I disclosed that boundary and handled missing metadata gracefully.

So I changed the architecture.

The program would not produce its own evidence. It would consume evidence.

A workflow can still do this:

git diff --name-only HEAD~1 | ./build/diff2test analyze .

but diff2test never launches Git. The shell does that. Git writes paths to stdout and diff2test reads newline-delimited paths from stdin.

The same input could just as easily come from:

printf 'include/alpha.hpp\n' | ./build/diff2test analyze .

or a file:

./build/diff2test analyze . --changed-files changed.txt

CMake, CTest, and the compiler follow the same boundary. They can generate metadata before analysis. The running diff2test process only reads files and stdin.

This distinction became important because calling the project "zero dependency" without explaining where the boundary sits would be misleading. The supported workflow absolutely uses CMake metadata. CMake just is not bundled, linked, or executed by the shipped program.

My first real fixture broke two assumptions quickly

I built a tiny CMake/CTest project early instead of spending the whole hackathon implementing against imaginary metadata.

Good decision.

The first thing I had to correct was CMake target traversal.

CMake naturally tells me that something like a test executable depends on a library:

alpha_test
    ↓
alpha

Impact analysis asks the opposite question.

If alpha was affected, I need to find everything downstream that depends on it.

So I built reverse adjacency:

alpha
    ↓
alpha_test

and propagate impact outward from the target owning the changed translation unit.

That part was straightforward once I saw the real data.

The .d files were more annoying.

My original plan was to recursively scan the build directory and collect anything ending in .d.

Then I found link.d.

Right extension. Wrong meaning.

That was enough to kill recursive discovery.

The final MVP takes an explicit dependency-file list through --dep-list. That list is created outside the process, and diff2test still validates every entry against the CMake targets and compiled sources it already knows about.

It is slightly more manual.

I trust it more.

Then I had to decide what "fallback" actually means

At this point I had the basic graph working, and my mental model had two possible results:

affected subset
or
full suite

Then I started deleting inputs.

Suppose the CTest catalogue is valid and contains:

AlphaTest
BetaTest
CoreTest

but one required dependency file disappears.

I can no longer prove that a narrow result is safe, but I still know exactly what the full test catalogue contains.

So the correct result is:

AlphaTest
BetaTest
CoreTest

That became:

FULL_SUITE_SELECTED
exit 10

Then I deleted the CTest catalogue itself.

Now there was a more basic problem.

I no longer knew what "the full suite" meant.

Printing three remembered test names would be fabricated output. An empty test list could be misinterpreted as "nothing needs testing."

That produced a separate state:

FULL_SUITE_REQUIRED
exit 11

It means the program cannot safely enumerate the suite and the caller should run its normal full-test procedure.

The successful narrow case is:

SUBSET_SELECTED
exit 0

So the three central outcomes became:

complete supported evidence
    → SUBSET_SELECTED
 
trusted test catalogue, unsafe impact evidence
    → FULL_SUITE_SELECTED
 
test catalogue itself cannot be trusted
    → FULL_SUITE_REQUIRED

I made both fallback conditions non-zero on purpose. If this is used in CI, I want degraded analysis to be visible.

That little experiment with deleting ctest-info.json changed the safety model more than any graph algorithm did.

What narrowing looks like when the evidence is good

The repository contains a controlled fixture with three tests.

If I change a header used only by Alpha:

printf 'include/alpha.hpp\n' \
  | ./build/diff2test analyze fixture --format names

the output is:

AlphaTest

There is also a shared header used by Alpha and Beta:

printf 'include/features_shared.hpp\n' \
  | ./build/diff2test analyze fixture --format names

which produces:

AlphaTest
BetaTest

CoreTest stays out.

Remove required dependency evidence and all three known tests come back with exit 10.

Remove the CTest catalogue and no test names are invented at all. The program exits 11.

The changed path can stay identical through all of those runs. What changes is how much of the evidence graph I am willing to trust.

Zero dependencies meant I owned every boring parser

The runtime implementation is a single C++20 source file.

That was one of the hackathon bonus constraints, but the more interesting consequence was that I could not quietly pull in the libraries I would normally use for the tedious parts.

JSON was the obvious one.

CMake File API and CTest both give me JSON, so I needed a parser.

At first this sounds like objects, arrays, strings, numbers, and a recursive-descent function or two.

Then malformed input enters the picture.

The parser ended up handling:

• strict JSON number grammar

• UTF-8 validation

• string escapes

• Unicode escapes and surrogate pairs

• duplicate object keys

• positional error information

• nesting limits

• maximum input size

• maximum string size I also wrote the Make-style dependency parser.

Real .d files can contain line continuations, escaped spaces, escaped characters, comments, CRLF input, repeated prerequisites, and malformed rules. One stress test feeds the parser a rule containing 10,000 prerequisites.

This is probably the part of the hackathon that changed my view of dependencies the most.

In a normal project I would use a mature JSON library. Happily.

Writing my own here was useful because it exposed how much correctness work disappears behind a small include statement.

The standard library gave me the pieces. It did not give me JSON, Make dependency syntax, or the trust policy I needed around either of them.

std::filesystem did not solve path safety for me

Path handling looked less interesting than parsing.

It consumed plenty of time anyway.

A changed file can be deleted, so I cannot assume every path exists on disk and blindly canonicalize it.

I also needed to handle things such as:

/project/foo
/project/foobar

which share a string prefix without one being contained in the other.

There are also .. escapes, project-root boundaries, build-root boundaries, relative metadata paths, and artifact paths coming from CMake.

I ended up using lexical normalization plus explicit containment checks instead of treating filesystem canonicalization as the answer to every path question.

std::filesystem was excellent machinery.

The policy was still mine.

The same source file can mean two different compilations

Another issue showed up once I started thinking about completeness.

Imagine foo.cpp is compiled into two CMake targets.

Those compilations may have different definitions or include paths.

If I see one dependency file for:

foo.cpp

I cannot mark the source globally "covered."

That dependency information belongs to one compilation in one target.

So dependency completeness is tracked per:

(CMake target, compiled source)

rather than only by source path.

It sounds like a small bookkeeping choice. Without it, one valid .d file could accidentally make another compilation of the same file look covered.

I would rather widen than make that assumption.

CTest executable matching also needed a stricter rule

CTest tells me what command a registered test runs.

CMake tells me which executable artifacts targets produce.

The tempting shortcut is to compare executable basenames.

I decided against that.

Two directories can contain executables with the same name. Wrapper commands complicate the relationship further. Multi-configuration builds add another source of ambiguity.

For the supported workflow, a CTest command has to map to exactly one normalized CMake executable artifact.

If it does not, subset selection stops.

That choice narrowed the projects I could support during the hackathon, but it kept the mapping explainable.

Stale evidence was worse than missing evidence

Missing metadata is easy to reason about.

You know it is gone.

Stale metadata is more dangerous because it still looks valid.

A .d file may exist while describing an older compilation. A header might have changed afterward.

For project-local prerequisites, diff2test performs a timestamp check. If a prerequisite is newer than the dependency file that claims to describe it, narrow selection is disabled.

If a timestamp that is required for the check cannot be read safely, the result widens too.

I am deliberately careful about what this proves.

A passing timestamp check does not establish that the metadata cryptographically matches the current source tree. It means I found no detectable staleness under that policy.

There are other inputs that I simply refuse to model narrowly.

If CMakeLists.txt or another .cmake file changes, that can alter source membership, target relationships, definitions, generated files, or test registration. Predicting all of that would mean interpreting CMake itself.

So build-configuration changes fall back.

Unknown changed paths do too. A file outside the dependency graph might be irrelevant, or it might be a generated input, script, resource, or something else the current model cannot see.

I do not classify "unknown" as "unaffected."

Explanations came almost for free

Once I was storing the graph and predecessor relationships, it became possible to show why a test had been selected.

For example:

changed path: include/alpha.hpp
dependency file: CMakeFiles/alpha.dir/src/alpha.cpp.o.d
translation unit: src/alpha.cpp
owning target: alpha
dependent target: alpha_test
registered test: AlphaTest

That turned into --explain.

I like this more than I expected.

If a CI optimization decides that hundreds of tests can be skipped, I want some way to inspect the chain that led to the tests it kept.

It also made debugging much easier during the hackathon. A wrong final set only tells you that something went wrong. A wrong evidence chain often tells you where.

Testing a program whose job is to skip tests felt slightly recursive

By the end, the repository had seven dependency-free C++ test executables covering the JSON parser, .d parser, path handling, CTest metadata, CMake metadata, impact analysis, and additional hardening.

One synthetic target graph deliberately contains a chain, a diamond, a cycle, and an unrelated branch at the same time.

That caught the properties I cared about:

• transitive reverse traversal

• no duplicate propagation through a diamond

• cycle termination

• unrelated tests remaining unrelated I also wanted real generated metadata in CI, not only JSON fixtures I had written myself.

The CI workflow creates a CMake File API query, configures and builds the controlled fixture, collects real compiler .o.d files, exports real CTest JSON, and analyzes those artifacts with diff2test.

The narrow Alpha case, the Alpha/Beta shared-header case, missing dependency evidence, and missing CTest catalogue are all exercised there.

Some failures were less interesting.

At one point a helper function in a test collided with std::quoted.

I renamed it.

That was the fix.

I also tried to attack the zero-dependency claim itself

"Zero runtime dependencies" is easy to put in a README.

I wanted the repository to make the claim inspectable.

CI scans the runtime source for process-spawning APIs.

The Release executable is inspected with ldd. On the Linux CI runner it showed the normal system/toolchain runtime entries:

linux-vdso.so.1
libstdc++.so.6
libgcc_s.so.1
libc.so.6
libm.so.6
/lib64/ld-linux-x86-64.so.2

There is no CMake application library or third-party project library in that list.

A separate development CI job runs the test executables under AddressSanitizer and UndefinedBehaviorSanitizer. Those are verification tools used while building and testing the project; they are not dependencies of the normal shipped executable.

I also tested determinism.

Reversing CMake references, reversing the explicit dependency list, changing CTest catalogue order, and duplicating changed paths should not change user-visible output.

CI captures real CLI stdout and stderr, compares reordered-evidence runs byte-for-byte, and repeats the same real-fixture analysis 20 times.

That may sound excessive for a hackathon project.

The program decides what not to test. I was comfortable being slightly paranoid.

Reproducible builds

The hackathon had a reproducible-build bonus.

I asked the organizers what they considered reproducible in this context. The target was two independent builds using the same environment and toolchain producing identical output, rather than pretending GCC, Clang, MSVC, Linux, and Windows should all produce the same bytes.

CI performs two clean Release builds on the same runner.

The binaries were byte-identical and both produced:

162a6bbf52034f0c468ab2c7c82853a449590530768e9ed6ddd82f1b7aabc903

That is the scope of the claim.

Nothing broader.

What did this make unnecessary?

The hackathon also had a "Package Killer" bonus: identify an installable package or tool whose role your standard-library implementation can remove.

I compared diff2test with RTS++ / Ekstazi++, a C++ regression-test-selection system whose broader approach involves infrastructure including LLVM instrumentation and its own RTS components.

I want to keep this claim narrow because the projects do not have identical feature sets.

For the specific CMake/CTest workflow that diff2test supports, I can perform useful conservative test-impact analysis without adding a dedicated RTS runtime stack.

That works because I am reusing evidence that the existing compiler, CMake, and CTest workflow already generated.

No compiler plugin.

No runtime agent.

No historical coverage database.

No daemon.

No network service.

For a broader production RTS system, some of those techniques may be exactly what you want. For this constrained workflow, I wanted to see how far the existing metadata could take me.

Quite far, as it turned out.

A small performance note

I benchmarked the controlled fixture with 20 warmups and 200 measured full-process invocations.

The result was approximately:

median: 2.106 ms
p95:    2.209 ms

The fixture is tiny, so I am not using that number as evidence that diff2test has solved monorepo-scale performance.

I did not have a verified 100,000-node or million-node benchmark during the hackathon, so I did not invent one.

The useful observation is simply that analysis itself is lightweight once the metadata already exists.

What I would change next

The most obvious rough edge is --dep-list.

Making the caller explicitly provide the dependency files was the safe choice after the link.d discovery, but it makes setup more manual than I would ultimately like.

I would not solve that by going back to recursive extension matching.

I would add proper evidence adapters for additional build shapes and generators.

Ninja should understand Ninja's dependency representation.

MSVC support should understand MSVC's dependency/path semantics.

Other CMake generators should get their own tested mapping rules.

That is slower than adding heuristics, but it preserves the property I care about.

I would also happily delete my JSON parser in a normal production version and use a mature library.

The hackathon was useful precisely because I had to discover what that dependency normally buys me.

Where it ended

diff2test finished the hackathon as one C++20 runtime source file with no third-party runtime code, no runtime subprocess execution, no network requirement, and a deliberately narrow CMake/CTest test-impact model.

The repository verifies the implementation with native C++ tests, real generated metadata, sanitizer runs, deterministic-output checks, dependency inspection, and byte-identical same-runner Release builds.

There are plenty of things it does not support yet: MSVC dependency formats, Ninja's dependency database, arbitrary CMake generator layouts, wrapper/interpreter-style CTest commands, generated custom-command relationships, coverage-guided selection, and historical test mappings.

Those boundaries are written down because guessing around them would defeat the point.

I started the weekend focused on the graph that selects affected tests.

I finished it thinking much more about the checks surrounding that graph.

A fast path is easy to write.

The interesting engineering starts when you have to decide whether you are actually allowed to take it.

For diff2test, that became the lesson I care about most:

safe optimization is mostly precondition engineering.

Links

Repository: https://github.com/ThunderKhan/diff2test

Release: https://github.com/ThunderKhan/diff2test/releases/tag/v0.1.1

Demo: https://www.youtube.com/watch?v=TQp6\_BOJHbw

Built during the Zero Dependency Hackathon 2026 by Hackathon Raptors.