Explain immediately
After the chapter, close it and answer one related question. Repair obvious misunderstanding while the learning context is still available.
Evidence:A complete explanation and one changed example.
Familiar words can feel like understanding while the page is open. Durable knowledge appears when you can retrieve a model, explain why it predicts behavior, and use it after the context changes.
The answer key is feedback, not the first step. One honest failed retrieval produces more useful routing evidence than five comfortable rereads.
Review the mechanism you used, not every subject on the site. If a project failed at a boundary, choose the deck that explains that boundary.
Write or speak a complete explanation. Include state, operation, boundary, result, and one case where the claim does not apply. “I recognize this” is not an answer.
Mark the answer as uncertain, plausible, or confident only after you produce it. Confidence before retrieval measures feeling; confidence after retrieval can expose dangerous certainty.
Do not score exact wording. Check whether your model predicts the same behavior, names the same boundary, and preserves the same distinctions.
Return to the linked course or glossary entry for the first gap. Rewrite the answer from memory after a short break instead of copying the key.
Use the idea in a new language, dataset, failure, user journey, or operational condition. Transfer reveals whether recall is tied to one example.
Spacing is not a magic interval. The delay creates a real retrieval problem, and the result decides whether the next interval should expand or contract.
After the chapter, close it and answer one related question. Repair obvious misunderstanding while the learning context is still available.
Evidence:A complete explanation and one changed example.
Answer the full five-question deck without notes. Reopen only the exact relationship that failed.
Evidence:Questions classified as retrieved, partial, or studied.
Mix the original deck with a related domain so you must choose the correct model rather than repeat a familiar pattern.
Evidence:Correct distinctions between neighboring concepts.
Complete the transfer prompt in a new context, then explain which parts of the original rule stayed stable and which assumptions changed.
Evidence:An independent artifact, diagnosis, or design decision.
If recall is accurate and transferable, lengthen the interval. If the answer is missing or confidently wrong, shorten the interval and repair the prerequisite model. Repetition count alone does not prove retention.
Each deck crosses several courses because real engineering problems rarely announce the chapter they belong to.
Retrieve the execution chain beneath every language and tool.
What is the difference between a program and a process, and why can the same program produce several processes with different state?
Connect CPU, memory, and persistent storage during one program run. What disappears when the process ends, and what can remain?
Why can the same relative file path work from one terminal directory and fail from another even when the source code did not change?
Trace a browser request through DNS, connection setup, HTTP, and HTML parsing. What distinct evidence proves each stage?
Distinguish source code, build artifact, configuration, and runtime state. Why does preserving only source code fail to reproduce some systems?
A program is an artifact or set of instructions; a process is one operating-system-managed execution of that program. Each process receives identity, address space, open resources, environment, scheduling state, and current execution position. Launching the same artifact twice creates separate process state even though code bytes match. Processes can still communicate or share external files, sockets, and databases, so isolation is defined rather than absolute.
The CPU executes instructions over values held in registers and memory. Memory contains active code, call frames, objects, buffers, and runtime bookkeeping. Persistent storage holds files and durable database state beyond one process lifetime. Ordinary process memory disappears when the process ends, but effects already written and committed to storage can remain. Caches, memory-mapped files, and operating-system buffers complicate timing, so “written” and “durable” need explicit boundaries.
A relative path is resolved from a defined base, commonly the process working directory, not necessarily the source file’s directory. Starting the same command elsewhere changes that base. Evidence includes the working directory, exact path argument, resolved absolute target, and process identity. A repair should define path ownership—command argument, configuration directory, packaged resource, or working directory—rather than assuming the launcher’s location.
DNS maps the hostname to address records; a lookup result is evidence. A route and connection then reach an address and port; socket state or connection outcome is evidence. HTTP exchanges method, path, headers, status, and body; the network record is evidence. The browser parses the returned representation into a document and loads dependent resources; DOM and console evidence belong here. Success at one stage does not prove the next.
Source expresses intended logic. A build artifact is the exact executable, bundle, image, or package derived from source and dependencies. Configuration selects environment-specific behavior. Runtime state includes processes, memory, files, queues, database rows, and external service state. Reproduction can fail when compiler version, dependency lock, generated artifact, configuration, migration, fixture, or retained data differs even if the source commit matches.
Choose one command-line program and one Web page. Draw both execution chains from artifact and input to process, external resources, observable output, persistent effects, and cleanup. Mark which evidence would locate a failure at each boundary.
Retrieve the language-independent model beneath syntax.
How do value, type, and representation differ? Give two values that look similar in output but require different operations.
How do you prove which branch ran without guessing from the final output?
What four facts define a loop, and how do zero-, one-, and two-item inputs expose boundary mistakes?
Why is choosing a data structure a contract decision rather than only a performance decision?
What does asymptotic complexity tell you, and what important production facts does it omit?
A value is the information a program operates on, a type defines permitted operations and meaning, and a representation is how the value is encoded in memory, text, bytes, or a protocol. The integer 10 and text "10" may print alike, but addition and concatenation differ. The decimal amount 10.00 and binary floating approximation may also display alike while equality and rounding behavior differ.
Write the predicate, record the actual operand values and types immediately before the decision, evaluate the predicate under the language rules, and observe a branch-specific event or return. Final output can be shared by several paths, so it is weak evidence. A focused test should make competing branches produce distinguishable state while avoiding logs that change timing-sensitive behavior unnecessarily.
A loop has initial state, continuation condition, body, and state update. Also state the invariant that should remain true before each iteration. Empty input tests whether initialization and result policy are defined. One item tests whether the first element is included. Two items reveal premature return or incorrect update. Exact-boundary sizes expose whether the stopping endpoint is included or excluded.
A structure defines identity, ordering, duplication, lookup, mutation, and iteration semantics. A set removes duplicates; a list preserves position; a map requires keys; a queue constrains removal order. Those behaviors affect correctness and interface meaning before speed. Performance then depends on operation mix, data size, memory layout, concurrency, persistence, and implementation—not only the structure’s textbook name.
Asymptotic analysis describes how resource use grows with input size under a model, ignoring constant factors and lower-order terms. It helps compare scalability classes but does not supply real latency, memory limits, cache behavior, distribution, network cost, database plans, parallelism, adversarial input, or correctness. Measure the actual operation under representative size and environment after selecting a sound algorithmic model.
Implement duplicate detection once with a list and once with a set. State the semantic difference, trace empty and repeated inputs, predict growth, measure several sizes, and explain when the list version could still be the correct design.
Retrieve the boundaries that make code explainable and safe to change.
What is the difference between a parameter and an argument, and why does passing an object not necessarily copy it?
Distinguish a returned value from a side effect. Why can a function have both?
How does a state machine make impossible states and repeated events easier to reason about?
What makes an interface a useful abstraction rather than a thin wrapper around hidden code?
Where should an invariant be enforced when several processes can write the same durable data?
A parameter is the input name in a function declaration; an argument is the actual value supplied at a call. Many languages pass an object reference, pointer, or handle by value, so the callee receives a way to access the same mutable object rather than a deep copy. Rebinding the local parameter and mutating the referenced object are different operations and require language-specific reasoning.
A return communicates a result to the caller through the function boundary. A side effect changes observable state or interacts with the outside world: mutating an object, writing a file, committing a transaction, sending a request, or logging. A function can calculate and return a receipt while also committing a payment. That design needs ordering, atomicity, retry, and failure rules because the returned value alone does not describe the effect.
A state machine names allowed states, events, transition preconditions, new states, effects, and rejected events. Instead of scattered booleans that permit combinations like “paid and cancelled without refund,” the model defines reachable combinations explicitly. Repeated events receive deliberate outcomes such as idempotent success, conflict, or rejection. Tests can cover every transition and prove that critical invalid states are unreachable through the interface.
A useful abstraction exposes operations that match domain meaning while preserving constraints, failure categories, ownership, and lifecycle. It hides details callers should not depend on and remains stable when an implementation changes. A wrapper that repeats every low-level storage call leaks the same model and adds indirection. Evaluate an abstraction by what reasoning it removes without concealing important costs or guarantees.
Validate early in the application for useful feedback, but enforce shared durable invariants at the authoritative concurrency boundary—often database constraints and transactions—because another process can write after an application check. Some cross-system rules need workflow coordination, idempotency, or reconciliation when no single transaction owns them. State which authority decides and how conflicts, retries, partial effects, and repair are observed.
Model a document through draft, submitted, approved, rejected, and archived states. Define events, actors, invariants, repeated-event policy, durable enforcement, and one interface that hides storage without hiding authorization or failure.
Retrieve the evidence cycle that turns a change into maintainable knowledge.
Why must a text-processing program define encoding, line, character, and invalid-byte policy?
How does an error differ from an exception, and why does catching an exception not prove state is safe?
What does a passing test prove, and what does it not prove?
What is the earliest-difference model of debugging, and how does a minimal reproduction support it?
Why are small coherent Git commits part of reasoning and recovery rather than only collaboration etiquette?
Text is decoded from bytes under an encoding. A “character” can mean byte, code point, or user-perceived grapheme; a line depends on separator policy. Without explicit choices, counts and slicing differ across platforms and inputs. Invalid bytes can reject, replace, or be preserved, each with data-loss implications. Fixtures should cover empty, Unicode, mixed newline, normalization, and invalid sequences while leaving source evidence unchanged.
An error is any failure outcome; an exception is one language mechanism for carrying failure and changing control flow. A caught exception says a handler received it, not that earlier mutations rolled back, locks released, files closed, retries are safe, or the user received a truthful result. Correct handling classifies recovery, preserves causal context, cleans resources, and makes partial state explicit or impossible.
A passing test proves that its setup, operation, and assertions passed in that environment and version. It does not prove unasserted behavior, all inputs, absence of races, production configuration, security, accessibility, performance, or lack of defects. Strong evidence connects a requirement to boundary cases and uses the narrowest test level that can observe the responsible mechanism, supplemented where integration or environment matters.
Debugging searches for the earliest observable point where actual state differs from predicted state. A minimal reproduction removes irrelevant dimensions while preserving the same failure signature, reducing the possible causal chain. Each removal is an experiment. Once the first wrong boundary is located, a falsifiable hypothesis predicts a discriminating result, and a regression test preserves the repaired causal lesson.
A coherent commit records one understandable change with its purpose and evidence. It lets reviewers isolate reasoning, tools compare exact state, tests bisect regressions, and operators restore a known version. A huge mixed commit couples unrelated causes and makes rollback risky. Commit boundaries are not deployment boundaries automatically, but disciplined history improves diagnosis and controlled recovery when paired with artifacts and data compatibility.
Seed one file-decoding defect in a small program. Capture the first error, reduce the fixture, repair the boundary, add tests, and create two commits whose messages explain the causal change and regression protection.
Retrieve how document meaning, presentation, interaction, and network loading cooperate.
Why is semantic HTML not interchangeable with visually similar generic elements?
What gives a control an accessible name, and why are placeholder text and nearby visual text often insufficient?
How do cascade, computed value, layout, and painted result differ when debugging CSS?
What is the difference between responsive reflow and hiding content to make a narrow screenshot fit?
Trace the browser from HTML response to DOM, styles, scripts, dependent requests, and pixels. Where can failure stop or alter the result?
Semantic elements expose roles, relationships, native behavior, keyboard interaction, form participation, and accessibility information to browsers and assistive technologies. A styled div can look like a button without activation semantics, focus, name, disabled behavior, or form rules. Prefer the native element whose contract matches the task; custom behavior inherits responsibility for every missing state and interaction.
An accessible name comes from defined browser naming rules such as an associated label, element text, or carefully used ARIA reference. Placeholder text disappears, can have weak contrast, and usually acts as a hint rather than a persistent name. Visual proximity is not a programmatic relationship. Inspect the accessibility tree and complete the task with keyboard and representative assistive technology, not only an automated rule.
The cascade chooses among declarations using origin, importance, layers, specificity, scope, and order. Inheritance and value processing produce a computed value. Layout resolves sizes and positions under formatting contexts and intrinsic constraints. Painting and compositing determine visible pixels and stacking. Inspect the stage where expectation first differs; adding higher specificity cannot repair an intrinsic-size or stacking-context misunderstanding.
Reflow preserves information and task completion as width, zoom, text size, writing direction, or content length changes. Hiding labels, clipping actions, or forcing tiny text can remove overflow while destroying access. Test at narrow widths, zoom, large text, long content, keyboard focus, reduced motion, and forced colors. Intentional two-dimensional content such as code may use a named, focusable internal scroller.
The browser receives bytes and metadata, decodes and parses HTML into a DOM, discovers styles and resources, builds style information, executes scripts according to loading rules, performs layout, paints, and composites. CSS or script requests can fail independently. Parser corrections can change structure. A script exception can stop later behavior while the document remains. Use Network, Elements, Console, and Performance for their distinct evidence.
Build one labeled form and result list that works with CSS and JavaScript disabled, then progressively enhance it. Test keyboard, 320-pixel width, 200% zoom, long content, an invalid submission, and an unavailable request.
Retrieve execution, ordering, identity, cancellation, and runtime boundaries.
Why does asynchronous not mean parallel, and what changes when an operation completes later?
How can two individually correct requests produce the wrong visible result?
What must cancellation clean up beyond stopping a visible loading indicator?
How do state identity and immutable updates help rendering systems decide what changed?
Why do TypeScript types not remove the need to validate JSON, storage, URL, or user input at runtime?
An asynchronous operation can begin and complete later without blocking the current path in place. The underlying work may be serial, concurrent, or parallel. Completion introduces another event and ordering boundary: state may change before the result arrives, errors travel through promise or callback paths, and resources need deadlines and cancellation. Reason with a timeline and ownership rather than assuming source order equals completion order.
Request A can start first, request B start later, B complete first, and A complete last. If every completion writes shared visible state, the older logical query overwrites the newer result. Both network responses are locally correct; ownership is wrong. Use request generation, stable identity, cancellation where supported, and a commit check that accepts only the result still authorized by current state.
Cancellation should reach child tasks, fetch or stream readers, timers, subscriptions, event listeners, response bodies, locks, workers, and state that would otherwise accept a late completion. It should produce one defined terminal outcome and preserve already committed effects according to policy. Test repeated cancellation and count live resources before and after; removing a spinner only changes presentation.
Rendering systems often compare identities to decide whether data may have changed. Mutating an object in place can preserve identity and hide a change from shallow comparison, while replacing unrelated state can cause excess work. Immutable updates create a new identity along the changed path and preserve old snapshots for reasoning. They do not automatically prevent races or deep accidental mutation; ownership and transition rules still matter.
TypeScript checks code under declared compile-time assumptions and erases before execution. External bytes can violate those declarations. JSON, URL parameters, storage, DOM data, environment values, and API responses must be decoded and validated at the trust boundary, producing a typed internal value only after success. A type assertion changes the checker’s belief; it does not inspect runtime data.
Build a fake search transport that completes requests in a chosen order. Test success, timeout, invalid data, cancellation, and reversed completion while proving only the current request can commit result state.
Retrieve contracts, authority, hostile input, idempotency, and safe evidence.
What makes an HTTP operation safe or idempotent, and why does the method name alone not guarantee implementation behavior?
How do decoding, validation, authentication, and authorization differ at an API boundary?
Why must authorization be enforced on the requested resource rather than inferred from a successful login?
What general design rule prevents SQL, HTML, and command injection across different contexts?
How should an API expose useful failure evidence without leaking secrets, private data, or internal structure?
A safe operation is intended not to change server state; an idempotent operation has the same intended effect when repeated. HTTP defines method semantics, but code can violate them by changing state on GET or duplicating effects on retries. Protect logical operations with contracts, preconditions, idempotency keys where appropriate, transactions, and tests. Repeated response bytes need not be identical for the effect to be idempotent.
Decoding turns bytes into a structured candidate under content and size rules. Validation checks shape, types, ranges, relationships, and business preconditions. Authentication establishes a principal from trusted credentials. Authorization decides whether that principal may perform this operation on this resource in current context. Combining the stages obscures error policy and can accidentally use untrusted identity or validate data after an unauthorized lookup leaks existence.
Authentication answers who the requester is, not which tenant, record, field, or action they may access. A logged-in user can change an identifier to another user’s resource. Every list, read, write, nested lookup, export, and background effect needs resource-scoped authorization under authoritative ownership. Test owner, non-owner, changed tenant, absent resource, revoked session, and privileged roles without trusting client-supplied owner fields.
Keep untrusted data separate from executable or structural syntax. Use parameterized SQL, context-aware output encoding, structured process APIs rather than shell concatenation, safe path resolution, and parsers with limits. Validation alone is not universal sanitization because safe characters depend on the destination context. Also bound size, nesting, time, and expansion so structurally valid input cannot exhaust resources.
Give errors stable public categories, safe messages, optional field details, and a correlation identifier. Log structured internal context under access controls, redaction, bounded labels, and retention policy. Preserve nested causes without returning stack traces, queries, tokens, paths, or private records. Metrics should aggregate outcomes without high-cardinality identities. A request ID supports diagnosis but must never become proof of authorization.
Design a document API contract with create, read, update, delete, list, conflict, invalid input, forbidden access, repeated request, size limit, and audit evidence. Write resource-level authorization tests before handler code.
Retrieve relational meaning, concurrency authority, query work, and durable restoration.
What does row grain mean, and how can a one-to-many join silently change the meaning of an aggregate?
How do keys, constraints, and application validation divide responsibility?
What does a transaction guarantee, and what must still be chosen about isolation and external effects?
Why can an index improve one workload while harming another, and what plan evidence should guide the decision?
Why are replication and backup not substitutes, and what proves that a backup is useful?
Grain states what one row represents and which identity makes it unique. Joining one order row to several item rows produces item-grain output and repeats order-level facts. Summing order total after that join overcounts. State grain before and after every join, reconcile row counts, and aggregate each measure at its natural grain before combining results.
Keys identify rows and relationships; constraints enforce shape, required values, uniqueness, references, and checks for every writer at the database boundary. Application validation provides earlier, domain-specific feedback and can check rules that do not fit one schema expression. It cannot protect against another concurrent writer between check and commit. Shared durable invariants belong in constraints and transactions whenever the authority can enforce them.
A transaction groups database changes into an atomic commit or rollback and provides an isolation model for concurrent observations. The chosen isolation level determines which anomalies remain, and application logic may need locks, version preconditions, or retries. Calls to email, queues, files, and remote services do not automatically roll back with the database; use outbox, workflow, idempotency, and reconciliation patterns for cross-boundary effects.
An index adds a searchable structure that can reduce reads and sorting but consumes storage, cache, maintenance work, and write cost. Its usefulness depends on predicates, ordering, selectivity, columns, table size, distribution, and query mix. Inspect estimated versus actual rows, access method, joins, sorts, spills, and timing with representative parameters. An unused or redundant index still has operational cost.
Replication copies current changes for availability and read scaling; it can quickly copy deletion, corruption, or bad writes. Backup retains recoverable historical state under a retention policy. A backup is only evidence after a restore rehearsal verifies artifact integrity, keys, schema compatibility, row and domain reconciliation, recovery time, recovery point, permissions, and application behavior in an isolated environment.
Model the last seat in a course. Write constraints and a transaction, demonstrate two concurrent schedules, inspect the roster query plan, then restore a fixture backup and reconcile seats, enrollments, and audit events.
Retrieve how software becomes an operating service under finite resources and failure.
How do process and thread differ in identity, memory, scheduling, and failure boundaries?
What is a file descriptor, and why can a service fail from descriptor exhaustion even when CPU and memory appear healthy?
Distinguish DNS, routing, TCP, TLS, HTTP, and application failure using one request path.
How do latency, throughput, concurrency, and saturation interact under load?
What makes an SLI and SLO user-relevant, and why is an error budget not permission to ignore every failure?
A process has operating-system identity, address space, resources, and protection boundary. Threads are scheduled execution paths within a process and commonly share its memory and descriptors while retaining their own stacks and registers. One thread’s memory corruption can affect the process; one process crash is more isolated from another. Exact models vary by operating system and runtime, so observe actual ownership and synchronization.
A file descriptor is a process-local handle to an open file, socket, pipe, or related kernel object. Every accepted connection or unclosed resource can consume one. At the limit, opens and accepts fail even with spare CPU and memory. Measure per-process counts and limits, classify owners, close on success, error, timeout, and cancellation, and bound concurrency before simply raising the limit.
DNS resolves a name to addresses. Routing chooses a path and interface. TCP establishes an ordered byte stream to a listening socket. TLS authenticates names or identities and protects transport. HTTP frames method, path, headers, status, and body. The application validates identity, authority, state, and work. Record evidence at each stage; a 500 response proves transport and HTTP succeeded far enough to receive an application failure.
Latency is time for one operation; throughput is completions per time; concurrency is work in flight; saturation appears when a constrained resource cannot absorb more demand without queueing or rejection. Increasing concurrency can raise throughput until CPU, descriptors, connections, locks, memory, disk, or a dependency saturates, after which tail latency and errors rise. Measure distributions, arrival rate, queue depth, and resource use together.
An SLI measures a user-visible population such as successful valid requests, latency below a threshold, correct results, or data freshness. An SLO sets a target over a window, creating a reliability risk budget relative to perfection. It can guide release pace but never overrides security, privacy, integrity, legal, or catastrophic safety constraints. Alerts should identify actionable budget burn while retaining per-failure evidence.
Run a disposable local request path with bounded concurrency. Record artifact, process, descriptors, listener, connection, HTTP result, latency distribution, saturation, one injected failure, graceful cleanup, and verified recovery.
Retrieve the evidence needed to trust derived data through change and replay.
What does a data contract specify beyond a schema, and which party owns each promise?
What makes a batch pipeline replayable, and why is rerunning not automatically safe?
How do event time, processing time, windows, and watermarks change the meaning of streaming output?
What do change-data-capture updates and tombstones require from downstream consumers?
How do lineage, reconciliation, staging, and atomic publication cooperate after a failed run?
A schema describes fields and structural types. A contract also defines grain, identities, units, time meaning, nulls, constraints, ordering, version compatibility, quality, limits, ownership, access, privacy, service expectations, failure handling, deletion, and change communication. Producers own what they emit and disclose; platform controls transport and storage guarantees; consumers own compatible use. Shared governance names decision and escalation authority.
A replayable pipeline references immutable inputs, versioned code and configuration, stable transformations, deterministic or explained nondeterministic behavior, classified rejects, run identity, lineage, and isolated staged output. Rerunning can duplicate effects, overwrite newer facts, restate history, exceed capacity, or republish partially. Use idempotent keys, transactional or atomic publication, reconciliation, and explicit backfill scope before changing authoritative output.
Event time describes when the source event occurred; processing time describes when the platform handled it. Windows group events under a time rule. A watermark expresses how far event time is believed to have progressed and determines when results close or update. Late-event policy—drop, revise, compensate, or retain—changes correctness. Record time zones, clocks, window boundaries, allowed lateness, and revision semantics.
CDC communicates inserts, updates, deletes, and transaction ordering from an authoritative log or change stream. A tombstone represents deletion or removal rather than an empty value. Consumers need stable identity, ordering, deduplication, schema version, replay position, deletion propagation, retention, and bootstrap rules. Compaction or missed offsets can make absence ambiguous, so reconcile snapshots and changes under documented authority.
Lineage connects each output to source snapshot, record, contract, transform, run, and quality decision. Reconciliation proves accepted, rejected, ignored, and output counts or totals agree with declared rules. Staging keeps incomplete results separate. Atomic publication changes the consumer-visible pointer only after checks pass. After failure, retry or repair the run without replacing the last verified output, then preserve evidence and downstream impact.
Design a daily pipeline with late corrections. Define contract, immutable input, event-time policy, rejects, deduplication, replay, lineage, reconciliation, staged publication, rollback, and one downstream consumer compatibility test.
Retrieve the boundaries between numerical behavior, evidence, model output, and product authority.
Why must train, validation, and test boundaries follow identity and time rather than only random row assignment?
What does an embedding similarity score express, and what does it not prove?
Why can one aggregate metric and a fluent example both hide unacceptable model behavior?
Why must RAG evaluate retrieval separately from generation and citation appearance separately from citation support?
How should authority, prompt injection, abstention, versioning, cost, and rollback constrain an AI product?
Rows from the same user, document, device, episode, or future time can share information. Random row splits leak that relationship and overstate generalization. Split by the unit that will be independent in use and respect temporal causality when predicting the future. Validation guides choices; the test set supports a final unbiased estimate and must not become another tuning source. Assert identity and time separation explicitly.
An embedding maps input into a vector space learned or chosen for useful relationships. A similarity score measures proximity under a metric and model version. It can support candidate retrieval but does not prove factual equivalence, authority, entailment, causation, freshness, permission, or user intent. Evaluate on representative queries, hard negatives, slices, changing corpora, and the actual downstream decision.
An aggregate can average away rare but severe failures, subgroup regressions, calibration problems, security violations, or distribution shift. One fluent example is selected and has no denominator. Preserve versioned cases, baselines, per-case outcomes, meaningful slices, uncertainty, error costs, latency, cost, and non-negotiable invariants. Human and automated evaluators also need validation and disagreement analysis.
If relevant authorized evidence was never retrieved, generation cannot ground an answer in it. Measure retrieval recall, ranking, permission filtering, and context construction first. Then judge claims, completeness, refusal, and safety. A citation-shaped link may point to an irrelevant or contradictory passage; citation support requires claim-level entailment and source authority. Missing or conflicting evidence should produce explicit abstention or qualified output.
Filter authority before retrieval and tool use, treat retrieved instructions as untrusted content, enforce tool contracts and approvals outside model prose, and test exfiltration and cross-tenant cases. Define when the system must abstain. Record model, prompt, corpus, evaluator, policy, and tool versions with latency and cost. Roll them back independently where possible, and block promotion on security or integrity invariants regardless of average score.
Create four RAG cases: answerable, unanswerable, conflicting, and unauthorized. Record retrieval, permission decision, context, claims, citations, abstention, latency, cost, versions, baseline comparison, release result, and rollback target.
Retrieve how a change moves from requirement to verified operation and independent handoff.
How do a user goal, requirement, invariant, and acceptance check differ?
What must be recorded to connect a source change to the exact artifact and configuration that ran?
How do rollout, health, readiness, canary evidence, and rollback divide responsibility during deployment?
Why are logs, metrics, and traces complementary, and how can each become misleading or unsafe?
What proves ownership has transferred to another maintainer rather than documentation merely existing?
A user goal describes the outcome someone needs. A requirement states behavior or constraint the system must satisfy. An invariant must remain true across allowed operations. An acceptance check defines observable evidence that a scoped result meets the requirement. A feature can satisfy a visible goal while violating a security invariant, and a passing test can be irrelevant if it does not observe the acceptance boundary.
Record source revision, dependency lock, build tool and environment, generated artifact identity or hash, provenance, test results, public configuration schema, migration compatibility, deployment manifest, and release identity. Secrets should be referenced by controlled identity, not copied into artifacts. Runtime evidence must connect requests and state to that release. A branch name or mutable tag is insufficient for reproducible rollback.
Rollout controls how exposure changes. Health reports whether a process can function; readiness decides whether it should receive work. A canary limits exposure and compares user-relevant indicators, errors, saturation, integrity, and security signals. Rollback restores a known compatible artifact and configuration when gates fail. Database and message changes may be irreversible, so expand-migrate-contract and forward recovery planning precede deployment.
Logs describe discrete events and context, metrics summarize populations over time, and traces connect work across boundaries. Logs can leak data or drown signal; metrics can hide individual failures or explode cardinality; sampling can omit traces and correlation does not authorize access. Design all three from user outcomes and failure questions, use stable bounded labels, preserve request or operation identity safely, and define retention and access.
Ownership transfers when a new maintainer can start from a clean environment, explain architecture and invariants, operate ordinary and failure paths, diagnose a seeded defect, make an unseen coherent change, deploy or package it, and recover without hidden author knowledge. Documentation is one input. Rehearsal, review, runbooks, tests, artifact access, credential authority, and observed handoff outcomes provide evidence.
Choose one completed project. Give another learner a clean checkout and one unseen requirement plus one seeded failure. Record setup, explanation, change, tests, artifact, rollout plan, observability, rollback, recovery, feedback, and remaining hidden knowledge.