DistributedByte

Testing behavior, not implementation

2026-10-14T13:00:00+00:00

A test suite earns its keep when it lets you change the code with confidence. A suite that fails every time you refactor, even though nothing a user could notice has changed, does the opposite: it teaches the team to distrust red builds and to avoid improving the design.

The usual cause is not too many tests, but tests that are coupled to how the code works instead of what it promises.

Ask what a caller can observe

Before writing a test, finish this sentence: “When a caller does X, they can see Y.” The answer is the behavior worth pinning down. Return values, persisted state, messages sent to other systems, and error responses all qualify. The order of private method calls, the number of times a helper ran, and the shape of an internal data structure do not.

// Coupled to implementation: breaks if the discount logic is reorganized.
verify(priceCalculator).applyDiscount(order);
verify(taxCalculator).addTax(order);

// Coupled to behavior: survives any internal restructuring.
assertEquals(Money.of("108.00"), checkout.totalFor(order));

Be careful with mocks

Mocks are useful at real boundaries: a payment gateway, a message broker, a clock. Mocking your own collaborators inside the same module records the current wiring and little else. When you notice a test that is mostly when(...) and verify(...), consider using the real objects, or a small in-memory fake, and asserting on the outcome.

Prefer fewer, wider tests over many narrow ones

Test through a public entry point, such as a service method or an HTTP handler, and let the internals vary freely. Reserve tiny unit tests for logic that is genuinely intricate, like parsing, calculations, and state machines. Wider tests are slower to write the first time, but they are cheaper to maintain because they only change when the behavior does.

Treat a painful test as design feedback

If a test needs a dozen collaborators to be constructed, the unit probably has too many responsibilities. If a behavior cannot be observed without reaching into private state, the interface may be hiding something callers actually need. Listen to that friction before reaching for a more powerful mocking tool.

Good tests document intent, fail for the right reasons, and make refactoring boring. Aim for that, and the coverage number will mostly take care of itself.

Designing reliable event-driven systems

2026-10-12T13:00:00+00:00

Event-driven architecture can loosen the dependencies between services, but it does not make complexity disappear. It moves some of that complexity into message contracts, delivery guarantees, and the operational life of events.

The most reliable designs begin with a clear answer to one question: what fact has happened that other parts of the system may care about?

Model facts, not commands

An event describes something that already happened: OrderPlaced, PaymentCaptured, or ShipmentDispatched. A command asks a particular service to do something. Keeping those meanings distinct helps consumers react independently without pretending they own the producer’s workflow.

{
  "eventId": "evt_01J9...",
  "type": "OrderPlaced",
  "occurredAt": "2026-10-12T13:00:00Z",
  "data": { "orderId": "ord_4821", "customerId": "cus_193" }
}

Include stable identifiers and a timestamp, and make the payload explicit. Consumers should not need to infer a business fact from a database row or an internal implementation detail.

Assume delivery can repeat

Most practical brokers provide at-least-once delivery. A consumer can finish its work and fail before acknowledging the message, so the broker may deliver it again. Make handlers idempotent: processing the same event twice should have the same effect as processing it once.

For example, record the event ID alongside a consumer’s durable result. A uniqueness constraint can turn a repeated delivery into a safe no-op.

Keep the database and event in step

Writing a database change and publishing an event are two separate operations. If the process stops between them, the system can end up with a changed record and no event. The transactional outbox pattern stores the business change and an outgoing event in the same database transaction; a separate publisher forwards pending events and marks them delivered.

This does not remove retries. It makes the source of truth recoverable and the failure mode visible.

Make operations part of the design

Track consumer lag, processing failures, retry counts, and the age of the oldest unprocessed message. Define where poison messages go and how a human can inspect and replay them. A queue that accepts events but silently accumulates work is not a reliable system.

Good event-driven architecture is less about adding a broker and more about making ownership, failure, and recovery explicit. Start with a small set of meaningful events, document their contracts, and let real boundaries—not fashion—determine where asynchronous communication helps.

A practical guide to data pipeline reliability

2026-10-08T13:00:00+00:00

A pipeline is reliable when its users can tell what data is ready, how fresh it is, and what happens when a run fails. Successful jobs alone do not prove that the output is complete or correct.

State the guarantee

Start by defining the contract: expected freshness, completeness, and acceptable delay. “Runs every hour” describes a schedule. “The dashboard reflects all events received by 10:00 within 15 minutes” describes a user-visible guarantee.

Make retries safe

Transient failures are normal. Prefer deterministic transformations and writes that can be repeated without duplicating records. For batch jobs, a partition can often be recomputed and replaced atomically. For streaming jobs, use stable event keys and explicit handling for late or malformed data.

Measure the data, too

Monitor volume, freshness, schema changes, and key business invariants—not just process health. A job can finish successfully while producing an empty or unexpectedly small dataset. A few targeted checks often catch more meaningful failures than a long list of generic alerts.

Design the recovery path

Document how to identify affected partitions, replay input, and validate corrected output. Keep enough source history to support that recovery, and make ownership clear so an alert leads to action.

Reliable data platforms make their guarantees visible and their failures repairable. That is a stronger foundation than assuming the pipeline will never break.

Evaluating retrieval-augmented generation

2026-10-04T13:00:00+00:00

When a retrieval-augmented generation system gives a poor answer, the cause may be missing evidence, weak ranking, or a model that fails to use the evidence it received. Looking only at the final response makes those problems hard to distinguish.

Build a representative question set

Collect real questions users ask, including ambiguous questions, questions with no answer in the corpus, and questions whose answer depends on a particular source. Keep the expected evidence and a short reference answer for each one.

Score retrieval and generation separately

For retrieval, measure whether the needed passages appear near the top of the results. For generation, check whether the answer is supported by those passages, answers the question, and abstains when evidence is absent. Human review remains useful for judging whether a response is genuinely helpful.

Evaluate changes as a system

Compare chunking, metadata filters, embedding models, rerankers, prompts, and model versions against the same evaluation set. Track latency and cost alongside quality. A small improvement in answer quality may not justify a large increase in latency for every use case.

The goal is not one universal RAG score. It is a repeatable way to see which stage improved, which regressed, and whether the change matters to the people using the system.