Compile Published Rules Before the Quote Arrives

A cache flush shouldn’t make the next quote rebuild a price book that was already published. The shipment is new. The structure of the tariff isn’t. Yet “load the graph, prepare it, cache it” puts both jobs back on the request as soon as the cache disappears.

That boundary changed during development of my private rating engine. First, published rulesets gained a persisted runtime projection. Later, supporting configuration gained a separate persisted bundle. These changes didn’t precompute shipment prices. They gave repeated preparation somewhere to survive other than a volatile cache.

The distinction sounds small until you follow the read path. It determines which queries still happen, what a warm request decodes, what publication actually prepares, and which failures can return the work to a quote. The excerpts here show the later PHP implementation of those boundaries; the formats and mappings evolved after their initial introduction. Imports, tracing calls, and unrelated fields are omitted where stated. Names and sample identities are anonymized.

A stored tariff graph isn’t the runtime representation

The table-and-row redesign gave recurring price facts explicit homes. That still left work between storage and evaluation: select the published ruleset, load its profile and artifact relationships, normalize their data, map rules, prepare matching structures, and carry the selected identities into the evaluator.

A four-kilogram parcel and a seven-kilogram parcel may select different rows. They don’t need different definitions of the selected price book’s rows, units, service conditions, or profile references. Rebuilding that common structure doesn’t teach the engine anything about either parcel.

The projection compiler made the distinction explicit. Its core was:

$rules = $includeRules
    ? $this->ruleMapper->mapRules($ruleset)
    : [];

$pricingProfile = $ruleset->pricingProfile !== null
    ? $this->pricingProfileMapper->map(
        $ruleset->pricingProfile,
        !$includeRules,
    )
    : null;

return new ActiveRulesetData(
    rulesetId: $this->rulesetId($ruleset),
    divisionId: $ruleset->division_id,
    code: $ruleset->code,
    version: (int) $ruleset->version,
    state: $ruleset->state,
    operationCode: $ruleset->operation_code,
    rules: $rules,
    pricingProfile: $pricingProfile,
    effectiveFrom: $ruleset->effective_from,
    effectiveTo: $ruleset->effective_to,
    publishedAt: $ruleset->published_at,
    rulesetHash: $this->projectionHasher->hashRulesetProjection(
        $ruleset,
        $rules,
        $pricingProfile,
    ),
);

“Compile” meant preparing this data, not generating machine code. The compiler didn’t know the next shipment’s weight or destination. It mapped configuration into a representation the runtime could consume, with an identity and a hash for that representation.

There were two variants. A consumer needing the complete rule graph could request rules. The pricing path requested includeRules: false; in that case the compiler also selected the runtime-optimized pricing-profile mapping. It wasn’t simply serializing the same large object and deleting one top-level field afterward.

The mapped profile carried its profile ID, an explicit-leg requirement, and an artifact collection. That artifact collection could contain prepared row structures, not just Eloquent model dumps. An authoring graph optimized for editing and a runtime graph optimized for matching had different responsibilities. Treating them as one universal representation would make either the authoring model or the quote pay for the other’s needs.

Publication created durable prepared copies

The publication materializer compiled both variants and stored them:

foreach ([false, true] as $includeRules) {
    $projection = $this->runtimeProjectionCompiler->compile(
        $ruleset,
        $includeRules,
    );

    $this->runtimeProjections->upsert(
        $identity,
        $includeRules,
        $this->runtimeProjectionCodec->encode($projection),
        RulesetRuntimeProjectionCodec::FORMAT,
        $projection->rulesetHash,
    );
}

The projection table stored more than the payload. Its columns included owner, ruleset ID, projection mode, profile ID, ruleset code/version/state, operation, publication time, profile update time, format, hash, and projection time. The unique identity was owner plus ruleset plus include_rules. For a given ruleset and mode, this was an updatable prepared record, not an append-only history of every compilation.

The repository used that identity in updateOrCreate. Its payload write included a storage conversion:

RulesetRuntimeProjection::query()->updateOrCreate(
    [
        'division_id' => $identity->divisionId,
        'ruleset_id' => $identity->rulesetId,
        'include_rules' => $includeRules,
    ],
    [
        // Other identity and publication metadata are stored here too.
        'projection_format' => $projectionFormat,
        'projection_payload' => base64_encode($projectionPayload),
        'ruleset_hash' => $rulesetHash,
        'projected_at' => CarbonImmutable::now('UTC'),
    ],
);

The codec produced a binary-capable payload; the database column was text, so the repository stored a base64 representation. The corresponding read decoded that storage layer before handing the bytes to the runtime codec. The cache could hold the codec payload directly. “The cached value” and “the database column” weren’t necessarily the same bytes.

That distinction matters when measuring payload size or diagnosing a decode failure. Base64 is a storage encoding here, not the compiler and not a compression algorithm. Compression and row compaction were separate steps inside the runtime representation. Their performance deserves its own investigation rather than an assumption that every prepared byte is cheap to move.

The repository also forgot both projection cache variants after an upsert. Publishing prepared the durable copies; it didn’t, through that write, fill the ruleset’s volatile cache. A subsequent read could populate the cache from the stored projection. The separately invoked context-bundle prewarm had a different behavior, which matters later.

The reader first asks which ruleset is active

A cache key can’t choose the correct effective-dated ruleset by itself. The reader first resolved a lightweight projection identity for the owner, operation, and requested instant. If no published identity existed, it returned a missing-ruleset failure before looking at any warm payload.

The later cache-key factory included the ruleset version, publication timestamp, profile identity/update timestamp, and projection mode:

return implode(':', [
    'rating',
    'active_ruleset_projection',
    'v8',
    $includeRules ? 'with_rules' : 'pricing_only',
    $identity->divisionId,
    $identity->rulesetId,
    (string) $identity->version,
    $identity->publishedAt?->toIso8601String() ?? 'unpublished',
    $identity->pricingProfileId ?? 'no_profile',
    $identity->pricingProfileUpdatedAt?->toIso8601String()
        ?? 'no_profile_update',
]);

The v8 prefix belonged to the cache-key scheme. It wasn’t the ruleset’s business version, and it wasn’t the codec’s payload-format name. Conflating those versions would make a format change or invalidation rule much harder to reason about.

This left an identity query on an ordinary warm request. The optimization was to avoid loading and mapping the full configuration graph merely to establish which payload to use. It wasn’t a claim that pricing could now skip the database entirely.

Once the identity was selected, the reader tried the cache. A string value was decoded; a successfully restored ActiveRulesetData could return immediately. If that didn’t produce a usable result, the reader tried the persisted projection. This is the stored branch with tracing removed:

$storedProjection = $this->runtimeProjections->findByRulesetId(
    $identity->rulesetId,
    $input->includeRules,
);

if ($storedProjection instanceof RulesetRuntimeProjectionRecordData) {
    $storedData = $this->codec->decode(
        $storedProjection->projectionPayload,
    );

    if ($storedData instanceof ActiveRulesetData) {
        $this->cache->forever(
            $cacheKey,
            $storedProjection->projectionPayload,
        );

        return GetActiveRulesetResultData::success(
            $storedData,
            $this->benchmarkTimings($benchmarkTimings),
        );
    }
}

That branch changed the meaning of a cache flush. Losing the volatile entry could now cost a stored-payload read, decoding, and cache refill instead of loading the authoring graph and compiling it again. Those aren’t free operations, but they are a different amount of work.

A failed restoration could still return compilation to the request

When neither prepared source yielded usable data, the reader selected the appropriate source graph and compiled it. The pricing-only and full-rule paths remained separate. After compilation it encoded, persisted, and cached the new result, then returned it.

active identity
  missing → fail
  selected:
    cache restores → return
    else stored restores
      → refill cache, return
    else load source graph
      → compile
      → persist + cache
      → return

Keeping that final path made the source graph a recovery source. It also kept an expensive operation reachable from a quote. Removing it would make successful preparation a hard availability prerequisite: a missing or unusable projection would fail until another operation repaired it. That can be a sensible latency contract, but it isn’t the fallback contract this implementation chose.

“Unusable” needs a precise meaning too. The projection codec could return null for unrecognized bytes, invalid gzip, missing required values, or a mismatched hash-material version. Those cases let the reader continue to the next branch. It did not catch every possible decoder exception. For example, the codec converted a stored lifecycle string with RulesetState::from($state). An otherwise structured payload containing an unknown enum value could throw instead of returning null.

I checked both behaviors against the actual codec. The fallback explains what happens after a null decode result; it isn’t a promise that arbitrary corruption always repairs itself. A teaching model with try/catch around every decode would quietly make that promise stronger than the source.

The codec’s hash check had a similarly specific role. It checked that the payload used the supported hash-material version and carried a string hash. It didn’t recompute that hash from the restored data and compare the two at this read boundary. The hash field alone wasn’t a payload integrity check.

These limits don’t undo the value of persisted preparation. They make its recovery contract testable. A junk payload that returns null, an unsupported format revision, and a structured payload that triggers an exception are different tests. One successful fallback doesn’t stand in for all three.

The ruleset projection didn’t contain the whole runtime context

The engine still needed override resolution, option relationships, currency metadata, and runtime policy. Reassembling those dependencies separately left repeated work around an already prepared ruleset. The next change gave that supporting combination its own bundle.

The two prepared artifacts had different identities and contents:

ruleset projection
  selected ruleset + mode
  mapped rules + pricing profile/artifacts

supporting context bundle
  owner + selected ruleset
  override policy + option relationships
  currency metadata + runtime policy

The bundle didn’t contain a particular shipment or its calculated result. Nor was it merely the pricing profile under another name. The runtime context resolver fetched the pricing-only ruleset projection, then asked for the supporting bundle using that selected ruleset ID and the same decision instant.

To choose the bundle, its identity resolver selected four active dependency IDs in one SQL statement using scalar subqueries. For example, the override policy selection was shaped like this:

$query = OverrideResolutionPolicy::query()
    ->select('id')
    ->where('division_id', $divisionId)
    ->where('ruleset_id', $rulesetId)
    ->where('state', OverrideResolutionPolicyState::Published);
$this->withinEffectiveWindow($query, $requestedAt);

return $query
    ->orderByDesc('effective_from')
    ->orderByDesc('version')
    ->limit(1);

The window helper allowed an absent start or a start at/before the requested instant, and an absent end or an end strictly after it. These were half-open effective windows. That shouldn’t be confused with the inclusive metric bands in the pricing matcher. The same word “range” can describe different contracts inside one engine.

Selecting four IDs in one statement reduced round trips and avoided loading all four payloads just to identify the combination. It didn’t make four selections disappear inside the database, and it didn’t establish which query plans were cheapest for every data distribution. The source test for this resolver compared its IDs with the individual repository choices and counted one select. That’s the boundary of that query-count claim, not a throughput measurement.

Missing dependencies were checked before cache reuse. The first absent override policy, option set, currency version, or runtime policy produced its corresponding failure. An old warm bundle wasn’t permission to keep using a dependency that no longer resolved for the requested instant.

Bundle identity followed the selected dependency versions

The bundle cache key included its codec format, owner, ruleset, and all four dependency IDs:

return implode(':', [
    'rating',
    'active_runtime_context_bundle',
    'v1',
    RuntimeContextBundleCodec::FORMAT,
    $identity->divisionId,
    $identity->rulesetId,
    $identity->overrideResolutionPolicyId,
    $identity->optionRelationshipSetId,
    $identity->currencyMetadataVersionId,
    $identity->divisionRuntimePolicyId,
]);

The database’s bundle identity used the same owner/ruleset/dependency combination, with a unique constraint across those columns. Its payload, format, and materialization time belonged to that combination. Selecting a new currency version could therefore select a different persisted bundle while leaving the ruleset ID unchanged.

The read order resembled the projection reader: cache, stored bundle, then materialization from the active dependency payloads. Materialization could fail at each dependency query, even after the earlier identity selection succeeded. The read model and materializer kept those failures explicit instead of substituting an unrelated previously warm combination.

This identity scheme depended on a real content/versioning contract. Changing a dependency’s meaning under the same ID would not select a new bundle key. The key wasn’t a digest of every dependency field. Similarly, the ruleset projection’s cache key contained profile metadata, while its persisted lookup selected a record by ruleset and mode; the read branch didn’t compare every stored metadata field with the newly selected identity. The readers relied on controlled writes, replacement, and invalidation.

The integrity chapter explains why that cannot be established by a version-shaped name or a model comment saying “immutable.” Here the practical consequence is prepared data becoming stale. A new ID or an explicit invalidation/rebuild must correspond to a meaningful content change. Otherwise the fast path can be reliably fast at returning yesterday’s meaning.

Prewarming had a time boundary, not knowledge of the future

After persisting ruleset projections, the publication materializer invoked the active-bundle reader using the ruleset’s published_at instant:

($this->runtimeContextBundles)(new GetActiveRuntimeContextBundleInputData(
    divisionId: $ruleset->division_id,
    rulesetId: $rulesetId,
    requestedAt: CarbonImmutable::instance($ruleset->published_at)->utc(),
    traceStages: false,
));

That call could materialize and cache the combination active at publication time. It didn’t enumerate every future effective-dated combination. Suppose a currency policy changes at a later instant while the ruleset remains active. A quote after that change resolves the later dependency ID and therefore another bundle. If nobody has prepared that combination yet, the quote can reach the materializer.

The code also used publication time, not necessarily the start of the ruleset’s future effective window. Publishing now for later doesn’t make the currently active supporting combination the only one future quotes will need. That is a useful limit to know before describing the engine as entirely precompiled.

There was another small but important boundary in this call: the method invoked the bundle reader without inspecting its returned result. The source therefore establishes a prewarm attempt, not a mandatory successful bundle-preparation gate for publication. An exception can still abort the call; a returned failure result wasn’t itself checked here. A stricter design could require successful preparation before making a revision selectable. It would also need to define recovery when a required future combination couldn’t yet be resolved.

Measure the work left on each path

The read models recorded identity query, cache retrieval, cache hydration, persisted read, persisted hydration, source-graph query, compilation/store, and cache-store stages where applicable. A successful cache hit and a persisted-projection hit were distinct observations. The cache payload’s presence and byte length were separate from a successful restoration.

That made the change inspectable. After a cache flush, a test should be able to distinguish “read the prepared database copy” from “recompiled the source graph and happened to return the same price.” After a dependency change, it should distinguish reuse of the old combination from selection of a new one. For simultaneous misses, it would need separate checks before claiming compilation happened only once; a unique storage identity alone doesn’t prevent duplicate preparation attempts.

The small public preparation model demonstrates those broad branch choices with in-memory maps. Its operation counts are not this application’s query counts or latency, and its JSON restoration is deliberately simpler than the actual codecs. The implementation above is what defines the real branch and exception boundaries. I also checked the source codec and key factories directly, including mode changes, profile-update identity, unsupported hash-material versions, and the enum failure described earlier.

Persisting preparation moved work and storage toward publication and first materialization. It added formats to maintain, two projection variants, supporting combinations, invalidation rules, and recovery paths. A small price book with infrequent reads might be better served by lazy loading. A hard latency budget might justify refusing request-time compilation entirely. I chose to preserve a source recovery path while removing repeated graph preparation from the ordinary case.

The next question was no longer whether a prepared copy existed. It was how much work remained inside a warm read: transferring bytes, decoding them, expanding rows, and finding candidates. Persisting a large graph solved its lifetime. It didn’t make that graph small.