A Rating Engine That Outgrew Its First Design

The first version could calculate the prices. That wasn’t the same as having a good representation of the price book, or an affordable way to evaluate it. Those distinctions drove most of the interesting work on this project—well before changing languages became the next decision.

I built a private logistics rating engine which could express different tariffs without adding a new application branch for every combination. It stayed private rather than becoming a commercial product. It was still one of the most satisfying systems I’ve worked on: the complexity largely belonged to the problem itself. I wasn’t first trying to explain years of unrelated exceptions which had somehow become permanent business rules.

There were plenty of legitimate complications instead. Physical weight and chargeable weight differed. Contract prices could replace some charges without replacing others. Rules could apply to a parcel, a leg or a shipment. More flexibility made it easier to express those cases, but it could also make ordinary tariff structure harder to inspect and expensive to execute.

This is a retrospective of the PHP design, its major representation and performance changes, and the later Go port. The chapters describe the code and checks at particular stages; their dates set the reading order, not when those events happened. The example rates and identifiers are fictional. The implementation decisions and their inconvenient boundaries come from the actual system.

The price isn’t the first number in the matching row

Take a two-kilogram parcel measuring 40 by 25 by 24 centimetres. A configured divisor of 5,000 cubic centimetres per kilogram gives a volumetric weight of 4.8 kilograms. If the policy chooses the greater of physical and volumetric weight, then rounds upward to the next half kilogram, the tariff lookup uses 5 kilograms. The scale reading hasn’t changed.

Now suppose the destination is in zone B and the decision falls in a winter season. Generic transport costs €30, handling adds €3 per eligible parcel and a seasonal charge adds €4 per shipment. An account’s negotiated schedule replaces transport with €24. The ordinary account gets €37; the negotiated account gets €31. Adding every matching amount would give the negotiated account €61. Finding all the rows is only part of calculating the price.

Even the apparently obvious total hides decisions. Does a percentage charge include handling? Does it apply within one parcel’s scope or across the shipment? Is a minimum applied before or after competing components are removed? In an actual prepared-configuration test, two charge orders produced the same €35 total but different fuel amounts: €2.70 and €2.40. A minimum top-up concealed the difference. Testing only the total would accept both.

The engine therefore needed to keep the physical input, derived facts, configuration, matching, monetary components and final decision separate. Calling the entire operation a price lookup would have been a fairly effective way to lose the problem before starting to solve it.

The formula system could calculate what it couldn’t describe

The original pricing model was more structured than “put a script in a column.” A ruleset owned component definitions. A component declared its scope, sequence, amount and currency, targets, validity conditions, optional applicability and amount expressions, and ordered amount operations. The expressions were persisted JSON syntax trees interpreted by a dedicated evaluator, not PHP passed to eval.

That design could express a staggered parcel tariff: €20 for the first parcel, €5 for parcels two through ten and €3 for each parcel afterward. Twelve parcels cost 20 + 9 × 5 + 2 × 3 = 71 euros. Each component could inspect the current package position, apply its condition and contribute its amount. Collection operations could bind an item into an alias scope without rewriting the request’s root context.

It was flexible, and it worked as a calculation model. The cost appeared when the rest of the application needed to understand what those programs meant. The middle band was an and of “position greater than one” and “position at most ten.” An importer or validator couldn’t simply ask for that tariff’s explicit bands; it had to deal with programs which happened to encode them. Recurring pricing structure kept arriving as another interpreter input.

The formula language also had real semantics to preserve: missing paths versus present nulls, strict operand types, lazy conditional branches, collection scope and failure results. The evaluator converted numeric values into floats for arithmetic and formatted money later. A decimal database column did not make the whole calculation decimal arithmetic. Neither a language port nor a new table would remove the need to understand those boundaries.

I chose to replace that pricing-authoring path rather than keep an old formula mode beside a new row mode indefinitely. The change was staged: establish the new profile and artifact families, switch the runtime path, then remove the old authoring components. This was private development, so there wasn’t a customer database requiring a zero-downtime conversion. That gave me room to change the model without manufacturing a permanent compatibility mode.

It didn’t mean formulas were a mistake everywhere. Computed facts still needed them. The more specific conclusion was that an ordinary tariff band shouldn’t need to masquerade as a small program just because a general interpreter was available.

Rows gave recurring pricing facts an explicit home

The replacement model bound a pricing profile to explicit artifact families. Tariff grids, contract schedules, geographic classifications, weight policies, seasonal calendars and precedence rules had different responsibilities. They could vary without every change becoming a new generic rule type.

The central relationship was:

ruleset
  -> pricing profile
       -> tariff grid -> tariff rows
       -> contract schedule -> contract rows
       -> geography catalog -> geography rows
       -> weight policy
       -> seasonal calendar -> season rows
       -> precedence policy -> precedence rules

The profile referenced particular artifact records, not a collection of names which each query resolved to “latest.” Roots carried identity, version, state and effective dates. Rows exposed amount, currency, scope, order and composition fields while keeping variable matching dimensions in structured JSON. A row could require both a weight range and a dimensional range; passing one axis didn’t excuse failing the other.

That was a much more useful database boundary. The schema could express parent ownership, unique row codes and unique sequence numbers. For example, these constraints belong to the tariff-row relationship, with ownership and table names shortened:

UNIQUE (tenant_id, tariff_grid_id, row_code),
UNIQUE (tenant_id, tariff_grid_id, sequence),

FOREIGN KEY (tenant_id, tariff_grid_id)
    REFERENCES tariff_grids (tenant_id, id)

An existing grid ID wasn’t enough; it had to belong to the same owner. Those guards applied to imports as well as ordinary writes. Recurring price facts now had explicit identities and relationships instead of being discoverable only by evaluating an expression.

There was no magical jump from relational structure to correct pricing. Foreign keys didn’t decide whether overlapping bands were legitimate. Family signatures, range checks, profile readiness and publication checks still had their own jobs. Version numbers didn’t make every published child row immutable either. The writer and competing updates determined that behavior. The redesign made more questions visible and enforceable; it didn’t make all of them database constraints.

The input model became deliberately narrower too. I dropped one parcel plus a copy count in favour of explicit parcel records. Count-based pricing remained possible, but the engine could retain each parcel’s identity, measurements, items and leg membership. Two shipments with the same total weight and parcel count don’t necessarily have the same oversized parcels or handling charges. Compressing them into a count would throw away facts the evaluator needed.

Computed values remained an ordered program over that input. Later formulas could consume earlier results. Weight and dimension conversions produced physical facts; candidate-specific policies then produced chargeable weights. Some geometry could be reused across services, but the policy-dependent price weight couldn’t simply be cached as a property of the parcel. The new rows needed better inputs, not the abolition of computation.

The next bottleneck was preparing and moving the price book

The table redesign brought substantial gains in my testing. It also made the remaining costs easier to distinguish. Reconstructing an authoring graph, projecting runtime data, fetching supporting configuration, selecting tariff rows, decoding a cached payload and writing a completed result were different operations. Optimizing one didn’t make the others free.

Published rulesets gained persisted runtime projections. A cache could hold the prepared representation, but it no longer held the only prepared copy. After eviction, the system could restore the durable version rather than necessarily assemble the whole authoring graph again. Exceptional misses and invalid prepared data still had recovery or failure paths; publication hadn’t precomputed every possible future request.

Supporting configuration had another lifetime. Currency metadata, runtime policy and option relationships could change active versions independently of the ruleset. A prepared context bundle therefore depended on the selected combination of identities and the decision instant, not merely a ruleset ID. Prewarming one publication-time combination didn’t warm every future effective window. Keeping that distinction prevented a faster cache from quietly selecting the wrong configuration.

The main calculation path became easier to reason about as a sequence:

admitted request and operation identity
  -> configuration for the decision instant
  -> computed facts and physical metrics
  -> candidate services and candidate-specific policies
  -> eligible rows and ordered monetary components
  -> precedence and option totals
  -> durable completed decision and replayable outcome

Candidate indexing addressed repeated work inside that sequence. Scanning the entire tariff separately for every service was wasteful. The index selected general rows, carrier rows, service rows and their combinations, preserving ordering and avoiding duplicates when several identifiers referred to one service. A map which forgot a general surcharge would be quicker at computing the wrong answer. Eligibility checks had to agree before the shorter loop was interesting.

Caching produced a less flattering result. A larger, ready-to-use representation looked attractive because it avoided some restoration work. Warm reads became worse. The request still had to move and decode the payload; caching more work didn’t remove the cost of moving the result of that work. More compact formats and selected-row expansion addressed that cost, with versioned readers and writers that had to agree about tuple widths, nulls and missing fields.

Even compaction wasn’t a rule to apply everywhere. The history includes a later reversal of tariff compaction while other artifact families stayed compact forms. The useful representation depended on the reader’s actual work. I don’t want “we made the payload smaller” to become a substitute for checking whether the consuming path became cheaper or harder to understand.

Persistence had its own boundary. The synchronous path saved a minimal completed decision and replayable idempotency outcome while heavier decision snapshots could be deferred. A successful response didn’t imply that every diagnostic child record had already been written. Moving that work changed the completion contract, so the benchmark and the migration tests had to inspect the records which actually defined completion.

A fast response wasn’t necessarily a new quote

One early benchmark correction made the results slower. The runner had reused operation identities across invocations, allowing the endpoint to return successful stored outcomes instead of computing new quotes. Idempotency was working. The experiment wasn’t measuring what I thought it was measuring.

The corrected runner gave each run a new token, separated warmup from measured requests, and assigned an ordinal to each attempt. It changed the request and idempotency identities, not just a tracing header. Repeating the same shipment could still warm and exercise the same configuration without pre-completing the operation being timed.

Unique inputs weren’t sufficient. Responses had to identify new results rather than replays, and their decision IDs had to match completed decision and idempotency records for the submitted requests. Comparing a global row count would allow unrelated work to conceal a missing result. That reconciliation ran after the timed HTTP phase; putting it inside every request would measure another workload.

The harness also corrected its throughput denominator. Summing individual latencies doesn’t give elapsed batch time when requests overlap. Two requests which each take 50 milliseconds can finish together in 50 milliseconds. Their latency sum is 100 milliseconds, but the batch still completed two requests in 0.05 seconds. Correct arithmetic mattered as much as a faster algorithm.

There are real historical numbers worth keeping, with their setup attached. One direct-application HTTP baseline used 500 services and 15,000 tariff rows, a dense mixed scenario, ten warmups and fifty measured requests at concurrency one, repeated ten times. Its per-run p50 values ranged from 127.1 to 137.3 milliseconds, with a mean of 132.7. The mean of the per-run p95 values was 145.1 milliseconds. Those are historical measurements of that workload, not a newly reproduced capacity result or a PHP-versus-Go comparison.

Later small runs established stronger response and persistence reconciliation. I wouldn’t retroactively certify the older baseline under checks added later, or pretend a seven-request correctness run established sustained throughput. The bounded-concurrency count runner also slowed its arrivals when all client slots were busy. That isn’t the same workload as an independent fixed arrival rate. The harness had to say which question it was answering.

Go replaced runtime mechanics, not the application

After that structural work, I was still paying a request-path cost which didn’t belong to tariff matching. In the setups I tested over weeks, PHP/Laravel spent roughly 50–100 milliseconds around the request before the business processing; Octane reduced that to roughly 30–40 milliseconds. Go removed most of the comparable language/framework overhead and performed better at the computation and rule-evaluation work, particularly on larger datasets.

Those observations drove the decision along with resource consumption and stability. They aren’t a universal Laravel tax, and they don’t establish one clean speedup factor for a complete quote. The historical full-quote benchmark, the minimal-response comparison and the broader overhead observations were different experiments. The earlier representation and cache gains belonged to the PHP work, not to Go.

Octane had already removed repeated full application boot. The remaining path still included routing, input transformation, authentication, admission, dispatch and response work. Moving a few controller lines or shaving another tariff loop wouldn’t remove all of it. At some point the sensible question was whether the runtime remained the right fit, not whether I could find one more way to defend the familiar stack.

The target server used Go’s net/http beneath its router and adapters, with the application graph constructed before serving. It still needed API-version checks, bearer permissions, rate limiting, request limits, input normalization, errors, queues, repositories and durable results. Comparing an empty Go handler with that PHP application would have dodged most of the migration.

The intended boundary was a faithful backend port. Preserve the physical schema, algorithms, useful query behavior, public contracts and workflows; retire the website and administrative UI. The internal API tester was an explicit exception: lose its browser adapter, keep its catalog, HTTP execution, saved runs, resumability and cleanup through a CLI. A binary was a different way to run the program, not permission to discard it.

That boundary needed a correction during AI-assisted planning. I approved a planning package which called for a fresh schema, then asked whether the existing migrations and database design would actually be preserved. They wouldn’t have been. The documents were coherent under a different premise. The approval was mine; the answer was to correct the target across the plan, not turn it into a story about an agent secretly ignoring a clear instruction.

The database represented months of modeling and tuning. Rebuilding it while changing runtime would introduce another design project and muddle the comparison. A schema redesign could be justified separately if the model or database were the problem. “We’re moving to Go” didn’t establish that case.

AI made the old tests useful in a different way

The PHP implementation had a large suite, but a file inventory wasn’t a Go verification plan. Pest datasets expanded individual test bodies into multiple cases. Setup, assertion subjects, boundary surfaces and expected failures mattered. A generated mapping which pointed at a similarly named Go function could still leave the source scenario untested.

The migration work used a frozen PHP revision as the behavioral reference. Tooling inventoried the source, expanded supported datasets, extracted actions and observations, and linked retained cases to executable Go tests. Unknown providers and unresolved setup needed explicit treatment rather than plausible guesses. AI helped with that translation and implementation, but generated tests weren’t an independent oracle merely because they agreed with generated code.

An earlier problem during the formula-to-row cutover already showed why this mattered. A test helper translated a strict volume boundary above 0.5 into an inclusive minimum of 0.500001. The original example still passed, but a volume of 0.5000005 fell into the gap. Keeping a scenario didn’t preserve its meaning if the helper underneath it changed the supported boundary.

Another concrete disagreement appeared in persistence. PHP’s associative model cast could turn a nested empty JSON object into an array during a read-modify-write cycle; a reasonable Go append preserved the object. The price could agree while stored history changed. Copying the broad feature name into a test wouldn’t expose either issue.

Native differential checks supplied a stronger reference where the uncertainty lived in execution. Some exported frozen PHP classes, checked installed framework dependencies against the source lockfile, and ran the same selected inputs in PHP and Go. Database comparisons used isolated migrated PostgreSQL instances and observed ordered projections, hashes, queries and partially loaded relationships, not only final values.

That exposed behavior a hand-populated struct could hide. A deferred lane read could reject an invalid enum state before attempting its rows query. An eager graph reader could reach a missing rows table first instead. Framework exception preparation could also change the failure seen by the application’s resolver. The correct comparison point belonged to the application path, not to whichever unit function was easiest to call.

The checking code deserved the same skepticism. A probe reproduced how decoding JSON into generic Go values could round different large integers into equality. The selected tests passed while that comparator limitation remained. It wasn’t a demonstrated wrong quote, but it ruled out claiming exact integer parity from that comparison. A green test isn’t more authoritative than the observation it actually makes.

“Preserve everything” also needed a limit. The goal was the application’s supported inputs, outputs, stored effects, ordering and failure behavior, not an independent implementation of PHP’s parsers, object model and sorting internals. A runtime difference needed a reachable application consequence before it justified compatibility machinery. Otherwise the migration could spend its time faithfully rebuilding work the product didn’t need.

What the project had—and hadn’t—established

During the verification stage described here, the Go port was unfinished. Source cases had registered Go proofs, but some remained unregistered, and registration wasn’t the same as fresh execution or complete parity. Its validator checked references, fixture and test-file hashes, declared assertion dimensions and test boundaries. That helps catch stale or incomplete mappings; it doesn’t establish that a declared test faithfully exercises every scenario. Completion also required the exact tests to execute successfully; a package passing while a conditional test skipped wasn’t enough.

The useful outcomes were more concrete than “a rewrite made it fast.” Recurring tariff structure had explicit representations. Preparation had a durable lifetime. Candidate retrieval avoided repeated whole-book work. Cache design accounted for transfer and decoding. Benchmarks distinguished new calculations from replays. The port had executable ways to challenge assumptions about the source, including assumptions made by its own tests.

There were costs attached. More explicit artifact families meant more schema, writers and publication rules. Prepared formats needed compatible readers and versioning. A faithful port meant preserving awkward application behavior until a separate change could justify correcting it. AI made it possible to delegate more implementation work, but didn’t make approval, interpretation or the reference behavior somebody else’s responsibility.

For a small, fixed price list, direct code and a few well-chosen tables could be easier to understand and cheaper to maintain than this engine. Here the dimensions genuinely needed to vary independently. The project became better when those recurring dimensions had clear homes, and when “faster” had to mean the same useful work completed correctly.

Development is finished now. I’m polishing the Go release, then I’ll archive the project. It stayed private, so this remains a development case study—not evidence of product-market fit, customer savings or production reliability. Finishing the project doesn’t turn the separate historical measurements into a matched PHP-versus-Go benchmark.

Follow the implementation

The accompanying Thoughts develop the actual interpreter, connected schema, evaluator, preparation, indexes, cache formats, benchmarks, runtime choice and migration checks. Their source excerpts explain the system; the smaller public reproductions isolate selected mechanisms with synthetic data. The chapter list below follows the retrospective’s January–November 2026 reading order and includes only chapters whose publication dates have arrived.

1. How a Shipping Quote Gets Its Price — Inside a logistics rating engine: the connected database model, request pipeline, multi-axis matching, charge composition, and durable result behind one quote.
2. Not Every Pricing Rule Belongs in a Formula — The original pricing interpreter, the table-and-row replacement, and the test translation that could hide a boundary error during the cutover.
3. A Valid Pricing Row Can Still Be Wrong — Inside the pricing schema, range validator, draft writer, and publication path: what each integrity check protects, and where its guarantee stops.
4. The Engine Prices Parcels, Not Copies — Why the engine requires explicit package records, and how that decision appears in request validation, package/item indexes, leg selection, allocations, and shipment aggregates.
5. One Parcel Doesn't Have One Pricing Weight — Inside the computed-fact and measurement pipeline: ordered formulas, value envelopes, unit conversion, candidate-specific weight policies, package rounding, and geometry reuse.
6. The Order of Charges Changes the Price — Follow a quote through contract replacement, scoped components, percentage bases, currency conversion, rounding, minimum top-ups and final precedence in the actual pricing implementation.
7. Compile Published Rules Before the Quote Arrives — How persisted ruleset projections and supporting-context bundles moved preparation off the ordinary quote path, including active identity, storage, codecs, fallback, and prewarming limits.
8. Index Tariffs Without Losing Charges — The actual PHP tariff projector and selector: moved predicates, wildcard buckets, service aliases, deduplication, row hydration, ordering, and a compound-key limit.
9. A Cache Hit Still Has a Payload — The rating engine cached its compiled rules, then paid to move and rebuild them. Follow the tuple format, codec changes, selective expansion, and the decision to leave tariff rows expanded again.
10. Is Your Benchmark Doing New Work? — The first fast results included idempotent replays. Follow the real runner from fresh request identities through response checks, exact persisted results, wall-clock throughput, and the limits of the retained measurements.
11. When the Framework Sets the Latency Floor — Why the pricing backend moved to Go after substantial PHP optimization, what the timing observations meant, and which framework responsibilities still had to exist in the new server.
12. The Migration Plan Was Solving the Wrong Problem — An AI-assisted port first drifted toward database redesign, then needed a clearer boundary between application compatibility and PHP emulation. The corrections mattered more than a larger plan.
13. A PHP Test File Isn't One Migration Case — Follow the existing PHP suite through pinned source files, Pest dataset expansion, extracted assertions and matching Go subtests without confusing an inventory with a verified migration.
14. A Rewrite Can Change Data Without Changing the Price — Follow a real port mismatch through an Eloquent getter, PHP array append, JSON reassignment, PostgreSQL persistence, and the Go correction. Correct totals were nowhere near enough.
15. Retire the Browser, Keep the Test Runner — Follow a PHP browser tester into a Go CLI while preserving its scenario catalog, typed state, HTTP behavior, saved credentials and failure history.
16. Testing the Go Port Against PHP — How the PHP-to-Go port compared native exception handling, ordered database projections and failure traces—and where its own JSON comparison could hide a difference.

Use the model through the API

The development chapters explain the implementation decisions. The companion API series starts with an empty configuration boundary and follows complete requests through publication, rating and response interpretation. It covers chargeable weight and goods, monetary bases, postcode boundaries, package scope, allocations, offers, load transitions, route eligibility and candidate comparison. The examples use synthetic prices and run through the Go HTTP application with PostgreSQL, rather than a simplified replacement calculator.

All twelve sets of full request bodies, the runner and an API reference are available in the source pack . Execution requires the private engine and a disposable authorized installation; the pack doesn’t include a server or credentials. All twelve walkthroughs publish on 1 November 2026 with the closing development chapter. Their series index keeps the teaching order separate from that shared publication date.