The Go port deliberately dropped the browser UI. The API tester was the awkward exception: not its pages, but the program behind them. It could create a test configuration, publish it, make real API calls, extract identifiers, wait for a result and clean up afterward. Deleting its screen was straightforward. Deleting all of that would have thrown away a useful part of the migration.
Replacing it with a CLI also exposed distinctions the screen could make easy to overlook. An HTTP 200 response could mean a failed test step. A failed run could still be a successfully produced command result. A saved step record didn’t mean its API call would never execute again. Those weren’t reasons to redesign the tester during the port; they were behavior to understand before moving its entry point.
The replacement kept the scenario program and changed its presentation adapter. This is how the actual catalog, state renderer, HTTP client and run store fit together—not a proposal for building a new generic test framework.
The catalog contained programs, not examples to paste
In PHP, a registry supplied manifests. Each manifest had metadata, initial state, execution steps and cleanup steps. The browser could select one and ask an execution service to run it. A step might generate a request document, send an HTTP request, poll a JSON path or purge a test division. Those actions had different owners; they weren’t all disguised calls to one controller.
The Go export path called the existing catalog builder and manifest registry at the pinned PHP revision. It checked the source revision and refused uncommitted executable inputs before bootstrapping the PHP console application. The exporter fixed the clock and passed a namespace placeholder into each manifest’s initial-state method. Its central projection was:
foreach ($registry->all() as $key => $manifest) {
$scenarios[] = [
'key' => $key,
'metadata' => $manifest->metadata(),
'initial_state' => $manifest->initialState('{{namespace}}'),
'steps' => $manifest->steps(),
'cleanup_steps' => $manifest->cleanupSteps(),
];
}The original also applied the permitted product-name substitution around that array; it is omitted here. Two compressed JSON documents carried the selection catalog and scenario definitions into Go. This avoided inventing a second catalog by reading the old screen and copying whichever examples looked important.
The Go loader decompressed those documents, preserved scenario order and built a lookup by key. Empty and duplicate keys were errors. A runtime registry then turned a selected scenario into an application definition and attached its failure-diagnostics implementation. Metadata, state, steps and cleanup steps were decoded separately before the definition validated their shapes.
There was a real trade-off in that export. It captured manifest output at a chosen source revision and clock, with namespace substitution left for the run. It wasn’t executing arbitrary PHP manifest code inside Go. Any source behavior depending on something outside that captured definition needed its own mapping; a matching compressed file alone couldn’t establish it. The test-suite inventory had the same distinction between finding a source artifact and preserving everything that artifact did.
Catalog inspection didn’t need the execution runtime. The CLI entry point
loaded the embedded catalog and handled list or show before requiring a
database connection. Running, resuming, inspecting persisted history or
cleaning up did need the database-backed services. That separation made the
catalog inspectable without pretending execution was an offline operation.
The execution path was separate from catalog inspection:
catalog -> ScenarioDefinition
-> ExecutionService
-> RunStore (run/state)
-> StepExecutor
-> StateRenderer
-> HTTP client
-> public API
-> RunStore (steps)For a handful of sample requests, ordinary Go integration tests would have been less machinery. Here the manifests already described multi-step programs with reusable state and cleanup. Recreating their behavior in unrelated tests would still have required that mapping, while losing the ability to select and inspect the same scenario outside the test suite.
State moved between requests without becoming a string
Each run received a generated namespace and an initial state document. Later steps rendered requests from that state, then extracted values from responses for subsequent steps. The whole-placeholder case and embedded-placeholder case had deliberately different meanings.
With state containing count: 42, disabled: false and a document object,
the actual renderer produced these results in PHP and Go:
"req-{{count}}"became the string"req-42"."{{disabled}}"became the booleanfalse."x{{disabled}}y"became the string"xy".- A request consisting of
"{{document}}"became the document object.
The empty text in "xy" came from PHP’s string conversion of false. A whole
placeholder instead returned the value itself. Rendering every placeholder
through fmt.Sprint would turn those rules into a different request: Go’s
ordinary string representation of false is not PHP’s. An embedded object was
rejected rather than helpfully converted to an invented string.
The PHP renderer made the whole-value distinction before interpolation:
if (preg_match('/\A\{\{(?<key>[^}]+)\}\}\z/', $value, $matches) === 1) {
return Arr::get($state, trim($matches['key']));
}The corresponding Go branch resolved the path and cloned the typed result:
if key, whole := testerWholePlaceholder(text); whole {
resolved, _ := testerArrayGet(state, testerPHPTrim(key))
return cloneTesterValue(resolved)
}The cloned result mattered because later request preparation shouldn’t mutate the stored state through a shared map or slice. But preserving types wasn’t permission to accept every shape everywhere. The exact request-placeholder path allowed an object, string or null at that renderer seam; the later HTTP step required an array-like request or null. Passing one renderer method wasn’t complete admission to an API call.
Extraction worked in the opposite direction. A definition could map
division_id to response path data.id. After the response, that value became
part of the run state. Nested data stayed data, missing paths resolved to null,
and the next step could use the result in its URL or request body. Treating
this as plain text substitution would miss half of the program.
A completed HTTP request could still fail the step
The HTTP client retained method, URL, JSON body, authentication and version headers, raw response text, decoded response and elapsed time. Those details belonged to the tester’s contract even though it was internal tooling. A cleaner client that changed them could make the migration tests exercise different requests.
For example, the source client retried only HTTP 429 responses: at most four attempts, with 250 ms between them. A fourth 429 was returned as the final response. A 503 was returned after one attempt. The Go loop kept that decision; this excerpt preserves its retry and error classification:
for attempt := 1; attempt <= maximumAttempts; attempt++ {
response, err := client.send(ctx, apiKey, method, path, payload)
if err != nil {
if isConnectionFailure(err) {
return client.syntheticResponse(
startedAt, "connection_failed",
"The docs runner could not reach the public API.", err,
), nil
}
return client.syntheticResponse(
startedAt, "request_failed",
"The docs runner could not execute the public API request.", err,
), nil
}
if response.status != http.StatusTooManyRequests ||
attempt == maximumAttempts {
return testerapplication.NewAPIResponse(
response.status, response.body, response.rawBody,
client.durationMS(startedAt),
), nil
}
client.sleep(retryDelay)
}That is a description of this runner, not a general recommendation to retry 429 and never retry 503. Changing the policy while porting would change the number and timing of requests made by a scenario. The client also retained 30-second request and 10-second connection timeouts. Connection failure could produce a synthetic response with status zero so the result could be recorded; that was different from receiving an HTTP error response.
After transport, the step executor decided whether the result counted as
completed. Non-2xx responses failed. So did a 2xx body with boolean false at
data.attributes.succeeded or data.attributes.valid. The latter wasn’t
checked by converting every false-looking string to a boolean.
One source-backed target test exercised an HTTP 200 with succeeded: false,
a failure code and a division identifier in the body. The executor marked the
step failed, extracted the identifier into state and attached the manifest’s
diagnostics using the request identifier. Failure didn’t mean “discard the
response and keep the previous state.” Those returned details were often the
most useful explanation of why the scenario stopped.
A polling step added another layer. It could require a particular HTTP status and an exactly matching value at a JSON path, retrying the observation when the condition wasn’t met. It did not use the ordinary HTTP step’s business-failure classification as a substitute for its own condition. Transport retries and polling attempts therefore needed separate preservation. This runner wasn’t evidence that any repeated business operation was safe.
The run store recorded a state machine, not console output
The database distinguished a parent run from its step slots:
- A run stored its user and saved credential, scenario key, namespace and state snapshot. Execution and cleanup each had a status and timestamps.
- A step stored its parent run, execution/cleanup kind, index and key, request and response snapshots, extracted state, duration and error.
The step table had a unique key on (docs_example_run_id, kind, step_index).
That allowed execution step 1 and cleanup step 1 to coexist, but not two
independent records for the same slot. PHP used updateOrCreate at that key;
Go looked up the slot, inserted when missing, or updated the existing row.
The row was the latest saved result for that slot, not an append-only record
of every attempt. Rerunning a step could send its request again and replace
its saved snapshot. A uniqueness constraint on the result record did not make
the external effect exactly-once.
Full execution walked the manifest in order and passed the updated state from one step to the next. A small orchestration test makes the stop condition particularly clear:
1 generate completed add generated request to state
2 fail failed POST returns 422
3 never not run no request sent
run.status = failed
saved state includes step 1's generated requestThose are the test’s synthetic steps, not an incident from a deployed system. The actual execution service used the returned step status to stop; it didn’t continue just because the executor itself returned no Go error. It saved the failure, last error, current state and completion time before producing the run payload. A successful loop instead saved the final state as completed.
Single-step execution had a different transition: a successful nonfinal step
left the run in_progress; selecting the final step could mark it completed;
a failed step marked it failed without assigning the full-run completion
timestamp. “Execute step” didn’t mean replay every earlier step or verify that
their external effects still existed. That difference had to survive the new
interface too.
Resume used the run’s identity, not new command defaults
On resume, the run lookup was scoped to its ID, user and scenario key. The actual query had this shape:
select * from "docs_example_runs"
where "docs_example_runs"."id" = $1
and "user_id" = $2
and "example_key" = $3
limit 1The execution service then loaded the saved state and resolved the run’s original saved credential record. Supplying another credential to the new command didn’t replace that choice. The target test used a run saved with credential 51 and a new request supplying credential 99; execution still used 51. That tied resumption to the existing run rather than quietly creating a different execution context.
This preserved credential identity, not an immutable copy of its secret for all time. The run’s foreign key could become null if that credential record was deleted. In that case resume refused to continue; it didn’t pick the current default and hope the same permissions applied.
Cleanup used the same saved-credential requirement and ownership check, but
had its own status and timestamps. It ran cleanup definitions in order, saved
each result, and stopped at the first failed cleanup step. A run could have
failed execution and successful cleanup, or completed execution and failed
cleanup. Reducing both to a single success flag would lose the information
needed to understand what remained afterward.
There was one more interface trap. The CLI returned an error exit when its
execution service returned an error, but a serialized run payload was an
ordinary command result. Its output writer returned zero after writing that
payload. Supplying {"run":{"status":"failed"}} to the actual adapter still
produced exit zero and that failed status in JSON. A script checking only the
process exit would miss the scenario outcome. That may be an interface worth
changing later; doing so silently during this port would change the contract.
The focused checks here executed the frozen PHP client and renderer, the Go client against local HTTP servers, the real Go execution/step collaborators with in-memory dependencies, and the CLI adapter with a supplied result. They didn’t run the entire scenario catalog through a database-backed application. Schema and persistence paths above come from the implementations; those checks aren’t a substitute for complete database and backend parity.
For an unused documentation page, deletion would have been the sensible end of the work. This tester was different because its scenario definitions, state transitions and observations were part of how the backend could be checked. Its replacement had to leave the same requests, state and inspectable failure behind even when nobody opened a browser.