A test isn't finished until it cleans up its mess

To make Lunar Dispatch, a deterministic rover simulation in Unreal Engine, behave the same way on every review, I was testing its keyboard controls and scenario state against fresh local builds; however, the latest successful run left three copies of the same app in the macOS Dock, while a stale crash report remained open beside them.

Nothing made it obvious which build was current, which icon was harmless, or whether the latest run had caused the crash, and that ambiguity was not merely cosmetic: a reviewer could open the wrong copy and report feedback against old code; yesterday's crash dialog could make today's corrected build look unstable; or an aggressive cleanup could close the one healthy instance the user was reviewing.

The assertions were green, but the user's machine had become the cleanup queue, which exposed the missing requirement: a GUI test must leave the computer in a known, understandable state because verifying the feature is only half of the job.

One running process did not mean one clean application

The rover moved as expected, keyboard controls changed the simulation, the Unreal test suite passed, and a fresh build was available for review. The first correction therefore seemed obvious: close the older Unreal Editor before launching another one, but the next launch still looked duplicated.

A process inspection reported only one active editor, but macOS continued to show three application icons because the automation had searched for a generic UnrealEditor process and missed project-specific names. Besides, Unix process inspection could not see stale macOS application registrations that no longer corresponded to a healthy visible instance.

At another point, an older launcher and its active child looked independent; however, terminating the launcher closed the child too, which meant the process tree was technically accurate but operationally misleading.

The report had a result, but the environment did not have a trustworthy story.

Cleanup needs identity, not aggression

The tempting fix was to kill every process related to Unreal before each run, which would make the Dock look cleaner but could also terminate another project, a system crash service, or the current review session.

Creation and cleanup need the same discipline: if launching a build requires knowing its exact executable and project, shutting it down should require knowing exactly which instance the automation owns.

The workflow therefore changed in stages: it inspected both relevant process names and macOS application registrations, identified the existing review instance, requested a graceful shutdown, verified that the visible registration disappeared, and only then launched the exact application executable rather than a wrapper that created another Dock-visible relationship.

1. 01Discover owned processes and visible app registrations
2. 02Identify the exact prior review instance
3. 03Request a graceful application shutdown
4. 04Confirm both process and visible registration are gone
5. 05Launch one exact fresh build
6. 06Run the assertions and human review
7. 07Quit normally and verify the final cleanup

This is more work than calling kill and less work than asking a person to untangle the desktop after every automated run.

The same lifecycle mistake later appeared in reverse while testing Roughwise, an Electron desktop application: two smoke tests exercised the interface successfully but timed out during cleanup because closing the last window on macOS does not necessarily quit an application. The assertions were complete, though the process lifecycle was not, and requesting a normal application quit before closing the automation connection fixed the test.

Different frameworks exposed the same distinction: a window, a process, a launcher, an application registration, and an automation session are related, but they are not interchangeable.

The environment is part of the result

Automated checks usually collect assertions, screenshots, exit codes, and logs; however, desktop automation also creates state outside the application through visible windows, background processes, temporary profiles, ports, lock files, permission prompts, crash reporters, and operating-system registrations.

Those leftovers may not be defects in the product itself, but they are still defects in the workflow when the automation created them and transferred the cost of understanding them to the user.

This is why “the feature passed” and “the test finished” must be separate claims: the first concerns application behavior, while the second also includes restoring the environment, preserving the intended review session, and making the next run trustworthy.

After the cleanup policy changed, Lunar Dispatch's headless verification completed with no Unreal Editor process left running, and its documentation recorded the supported review launch path. As of this Note's publication date, the simulator remains a private portfolio prototype rather than a consumer release.

Environmental hygiene sounds like housekeeping until the wrong build gets reviewed or the right one gets killed, but a test is finished only when its evidence is trustworthy and its mess is gone.