A Cache Hit Still Has a Payload

The rules were cached. The expensive graph had already been compiled. A warm request should have been the easy case, yet loading that prepared data was still taking enough work to deserve its own investigation. The cache was working; the representation inside it was the problem.

One attempted fix was to cache a larger, ready-to-use pricing profile and skip reconstruction. That made cache-hit performance worse. Later, compacting rows helped, moving their expansion helped, and then keeping one expensive family expanded helped too. Apparently contradictory results, until you follow where each version made the request do its work.

This is that part of the engine’s development: not another cache layer, but several changes to the bytes and arrays passing through the same read path. The interesting bit is why the final format didn’t apply one optimization uniformly to every kind of row.

Finding the key wasn’t the expensive part we could ignore

The preparation chapter explains how a published ruleset became a durable runtime projection. The indexing chapter explains how a candidate found relevant row references. Neither makes loading the projection free. Before selection can use those references, the process still needs the row map and its index.

The useful breakdown was:

find the active ruleset identity
    -> read the cached projection
    -> decompress and decode it
    -> restore the runtime envelope
    -> select row references
    -> expand selected compact rows
    -> match and materialize charges

The initial identity lookup stays outside the cache-hit restoration work. It determines which published configuration this request should use. The cache then avoids rebuilding that configuration from its authoring graph, not every operation before evaluation.

The read model measured the call to the cache separately from the call to the projection decoder. In condensed PHP, the important boundaries were:

$startedAt = hrtime(true);
$cached = $cache->get($key);
$timings = $timings->withStage(
    'ruleset_cache_get_ms',
    $startedAt,
);

if (is_string($cached)) {
    $startedAt = hrtime(true);
    $data = $codec->decode($cached);
    $timings = $timings->withStage(
        'ruleset_cache_hydrate_ms',
        $startedAt,
    );

    // Only a usable decoded projection counts as a hit.
    if ($data instanceof ActiveRulesetData) {
        return $data;
    }
}

This excerpt leaves out result construction, recording, and fallback reads; it isn’t a replacement read model. The distinction matters because a non-null cache value could still fail restoration. Counting its presence as a successful hit would conceal the work which followed.

Nor was the cache timer a network profiler. It surrounded the cache client’s get, which can include client processing as well as waiting and transfer. The hydration timer surrounded the codec: decompression, deserialization, envelope reconstruction, and any work its profile reader performed. It did not isolate gzip from array creation, and later selected-row expansion could happen outside it. Moving work out of that timer wasn’t enough; the request still had to get faster.

The implementation also recorded a value called ruleset_cache_payload_bytes. That name is more confident than its implementation: it used mb_strlen without an explicit byte encoding. I wouldn’t use that field as an exact binary-byte measurement without checking the runtime encoding. The public example below measures actual encoded byte lengths instead. Instrumentation needs the same scrutiny as the code it’s supposed to explain.

The first format change removed repeated row structure

The projection held indexed artifacts. Each had a rows_by_id map containing row bodies and separate buckets containing references into that map. Before compaction, every body was an associative array. Thousands of rows repeated the same field names, and every metric axis repeated its own names inside the row.

The first compaction change replaced those bodies with positional arrays. The artifact carried one ordered field descriptor instead. Index buckets could continue referring to the same row IDs; they didn’t need another copy of either representation.

A later service-constraint artifact makes the actual format easier to show than a made-up two-column price list:

{
  "runtime_row_encoding": "tuple_v1",
  "runtime_row_fields": [
    "row_code", "sequence", "scope", "decision",
    "service_match", "criteria", "metric_axes",
    "required_geography_codes", "zone_codes",
    "valid_from", "valid_to"
  ],
  "rows_by_id": {
    "constraint-1": [
      "BASE", 10, "shipment", "allow",
      {"option_codes": ["fragile"]}, [],
      [["weight", "kg", 0, 5]],
      [], [], null, null
    ]
  }
}

The values are illustrative; the eleven fields, tuple layout, and nested axis layout come from the PHP compactor. The row map supplies constraint-1 as identity when the row is restored. The tuple doesn’t need another row_id slot. scope is still shipment, the required option is still fragile, and the weight axis still spans zero through five kilograms.

The compactor’s central operation was quite ordinary:

$tuple = array_map(
    fn (string $field): mixed => compactField(
        $field,
        $row[$field] ?? null,
    ),
    $fields,
);

compactField is an abbreviated name for the source helper. Most fields pass through unchanged. For metric_axes, each axis becomes another positional array:

[
    $axis['metric_type'] ?? null,
    $axis['metric_unit'] ?? null,
    $axis['minimum_value'] ?? null,
    $axis['maximum_value'] ?? null,
];

This was a runtime format, not a new authoring schema. The database still held the configuration described in the earlier chapters. Compaction happened after projection, at the storage and transport boundary. Trying to make people author these arrays would have saved the wrong kind of work.

It also wasn’t a lossless encoding of an arbitrary PHP row. Only descriptor fields were transported; an extra field disappeared. Missing and explicitly null values occupied the same null slot. Those choices are acceptable only where the runtime contract permits them. Add a new field to the evaluator but forget the compact writer and a cold projection can appear correct while the cached version silently loses the field. That’s a particularly unpleasant way to discover that the cache format is an interface.

Compression changed the cost again, not by the same ratio

Compaction was followed by gzip-compressed JSON. The historical performance notes describe the large cached ruleset dropping from multi-megabyte scale to a few hundred kilobytes, with a useful reduction in cache retrieval and request time on the generated 15,000-row workload. They identify shrinking the projection as the most important improvement at that stage. They don’t provide an isolated speedup for every format change, so I won’t manufacture one from the sequence of commits.

There are two different savings here. Removing repeated field names reduces the plain representation. Compression can already encode repeated names efficiently, so the same change need not reduce compressed bytes by anything like the same proportion.

The public payload experiment holds 800 synthetic rows constant and compares expanded and tuple JSON at gzip level one:

Those are example byte counts, not the private engine’s payloads. Almost half the plain JSON disappeared; the compressed difference was much smaller. Neither column measures decode time or PHP memory. It would be very easy to turn that first column into an impressive and entirely unsupported latency claim.

Gzip also adds work to both ends. Publication can pay the encoding cost once for many reads, but every cache restoration still decompresses its payload. Smaller transfer is worth that trade when it improves the complete read, not because compression looks efficient in isolation. With a tiny local payload, the simpler uncompressed representation is a credible alternative.

We stopped rebuilding rows before knowing which ones we needed

The first compact format eagerly expanded its rows again on a hit. That kept the evaluator’s familiar associative arrays, but it restored the whole row map before the candidate index discarded most of it. A smaller object in the cache had become a large graph of arrays again on every request.

The next change kept tariff and service-availability tuples compact through cache restoration. Candidate selection looked up references first; only rows the runtime touched were expanded. The difference was the order of work:

Earlier:
decode all -> expand all -> select references -> evaluate

Later:
decode all -> select references -> expand selected -> evaluate

Both paths still decode the complete stored payload. This was not random access into compressed data or selective JSON parsing. It avoided creating expanded rows which evaluation would never use.

The row reader restored identity from the map and paired each descriptor entry with its tuple value. Here is the outer part, shortened from the PHP hydrator:

$row = ['row_id' => $rowId];

foreach ($fields as $index => $field) {
    $value = $reference[$index] ?? null;

    if ($value === null) {
        continue;
    }

    if ($omitEmptyArrays && $value === []) {
        continue;
    }

    if ($field !== 'metric_axes' || !is_array($value)) {
        $row[$field] = $value;
        continue;
    }

    // Expand each axis's four positional values here.
}

The axis branch reconstructs metric_type, metric_unit, minimum_value, and maximum_value. Zero bounds survive. A null outer field is omitted; empty arrays survive unless the caller requests their omission. This is deliberately not array_filter: false, zero, and the empty string are not interchangeable with absence.

The public experiment contrasts 800 eager row expansions with five selected expansions, checking that the selected values agree. That establishes avoided row work, not a 160-fold faster request. It doesn’t include the real candidate mix, index lookup, PHP decoder, or price assembly. In the previous chapter, selection also merged alias buckets, deduplicated rows, and sorted them; deferred expansion didn’t remove those responsibilities.

This is also why selective expansion isn’t an automatic choice. If selection touches most of the map, the request has paid to decode tuples, follow the index, and then rebuild almost everything anyway. The useful quantity isn’t just the total row count. It’s how many rows requests actually touch, how often they revisit them, and what restoration costs compared with carrying the expanded payload.

A codec change and a work change landed together

The selective-expansion change also replaced JSON with PHP serialization. The write path changed from json_encode($payload) followed by gzip to this:

$serialized = serialize($payload);
$compressed = gzencode($serialized, 1);

return is_string($compressed)
    ? $compressed
    : $serialized;

The reader decompressed the string and used:

$decoded = unserialize(
    $payload,
    ['allowed_classes' => false],
);

These excerpts show the format operations, not the whole decoder. The later persisted-projection codec retains this encoding, identifies it as serialized_gzip_v1, and reconstructs the ruleset envelope from the resulting array. Its profile reader doesn’t eagerly expand every tuple family, although it still traverses some artifacts to normalize criteria. There is work after deserialization even in the newer path.

The historical commit describes cheaper decoding and less warm-hit hydration work. It changes two things at once: the decoder and the amount of eager row expansion. It isn’t an experiment which assigns the improvement to serialization alone. A JSON-versus-PHP microbenchmark would answer a narrower question, and even that would need representative arrays and the same downstream work.

There was a compatibility consequence too. The cache namespace changed with the encoding changes, including the JSON-to-serialization switch. New code didn’t simply assume that an old key contained a new format. A fresh namespace forces the cold path to refill it; it doesn’t magically update existing bytes. The durable store’s explicit format and the cache’s versioned identity serve related but different readers. Both belong in the deployment plan.

PHP serialization was an internal implementation choice, not a public API or an attractive cross-language interchange format. A later Go implementation must preserve the computation’s behavior; that doesn’t mean treating a PHP serialized cache entry as its natural native data structure. The migration needs to identify which stored formats are compatibility obligations and which caches can be rebuilt. We’ll return to that distinction when the port gets its own contract.

And allowed_classes => false isn’t a license to deserialize arbitrary input. The PHP manual warns against untrusted input regardless of that option . The writer, storage access, and payload origin still matter. I wouldn’t copy this internal codec into a customer-facing import endpoint.

Then tariff rows stopped being compact

The later history contains a change whose entire point was to keep tariff rows expanded in the published projection. Tariff tuple hydration was still a large cost on the 15,000-row path after selector and hydrator work. Rather than build another orchestration layer around it, the change removed tariff compaction from the writer. The selector could already consume expanded rows.

That left a mixed projection. In the inspected PHP implementation, service availability, service constraints, package breaks, shipment breaks, surcharges, and adjustments have tuple-compaction paths. tariff_grid does not. The compactor also leaves an artifact alone if it doesn’t have a rows_by_id map. There isn’t one universal “compact pricing row” format silently applied to every object.

I like this part of the history because it resists a neat ending. “We made the data smaller” is a satisfying optimization story. It isn’t a requirement to keep doing so after the cost moves. The earlier rejected ready-to-use cache and the later expanded tariff representation are not the same experiment: one enlarged the prepared profile broadly; the other targeted the artifact whose reconstruction remained expensive, after the surrounding path had changed.

The tariff change’s commit records an improvement in direct-app median latency. I don’t have an isolated retained before-and-after run here which supports a numerical factor, so that is the limit of the result I can report. It certainly doesn’t establish that expanded arrays beat tuples generally. It does explain why the later code differs from the earlier performance notes: they describe different points in development.

Mixed formats are manageable when the reader knows which contract it has. The current hydrator accepts associative rows without tuple expansion. For tuple rows, it requires projected field metadata instead of quietly assuming today’s default field order. But that isn’t complete schema validation. A short list can supply missing null values, and surplus positional values can be ignored. The helper doesn’t enforce exact tuple width.

For example, the actual helper produces this result:

fields:  ["row_code", "sequence"]
tuple:   ["BASE"]
result:  {"row_id": "constraint-1", "row_code": "BASE"}

The missing sequence isn’t rejected at that boundary. This doesn’t establish that malformed customer input can reach it, but it rules out claiming that the hydrator itself proves every tuple valid. The public experiment is stricter about width; its rejection checks are teaching safeguards, not a description of the private reader. Preserve that distinction when writing characterization tests. Otherwise the supposed reference implementation quietly becomes the implementation we wish we had written.

A smaller payload was useful, not sacred

The decisions make more sense as a sequence of costs than as a list of techniques: reduce repeated structure, reduce transferred bytes, avoid eager expansion, then stop compacting a family where rebuilding it still cost too much. The unsuccessful larger cache and unhelpful loop-versus-array_map tweaks belong in that sequence too. Neither deserved to survive merely because it sounded plausible.

A process-local cache could move restoration out of more requests. Under persistent workers, though, it would need explicit identity, invalidation, lifetime, and a memory budget. Caching whichever request’s state happened to arrive last is not that design. The performance notes rejected unsafe static state under Octane; they didn’t prove immutable, versioned shared values impossible.

For this engine, I’d compare a representation using the same published graph, candidate workload, cache state, and correct prices, then look at the complete request as well as retrieval, restoration, and selected-row work. If expanded tariffs are cheaper overall, keep them expanded. There’s no prize for the smallest payload when every quote has to spend the saving rebuilding it.