Mathematics for AI
Why does an AI model turn words, images, or records into numbers, and how do those numbers lead to one decision? Begin with coordinates, distances, weighted sums, and small tables that can be drawn and calculated by hand. Then introduce matrices as many linear rules at once, derivatives as local change, probability as explicit uncertainty, and statistics as evidence from samples. Information and numerical accuracy appear when the model must compare predictions and run on a finite computer. The final project rebuilds one small learning system from scratch and labels every assumption and limitation.
Vectors, geometry, and representations
Objective Interpret vectors as both geometric objects and structured representations used by AI systems.
Core explanation
A vector is an ordered collection of numbers, but its meaning comes from the space and coordinate system around it. A two-dimensional vector can describe a direction on a page; a vector with hundreds of components can represent an image, document, customer, or model state. Addition combines displacements, scalar multiplication changes magnitude, and a norm measures size. Distance is useful only when the coordinates have compatible scales and the chosen geometry matches the task. AI systems often transform raw observations into feature vectors, then compare or combine them. This makes representation design as important as the later algorithm. Always write down what each coordinate means, its units, allowed range, missing-value policy, and whether normalization changes the interpretation. A mathematically valid calculation can still be a poor model of the real-world object.
Define the objects and assumptions in Vectors, geometry, and representations
A vector is an element of a specified vector space, while a coordinate list is its description in a chosen basis. Addition and scalar multiplication combine directions; norms and metrics measure size and separation only after their axioms, units, and domain are declared. Features therefore encode modeling choices rather than neutral facts. For a scale-aware learner representation and neighbor explorer, begin by naming the mathematical objects, their domains and codomains, the population or data source, and which equalities are definitions, derived results, approximations, or empirical claims. Translate every symbol into a sentence and every sentence back into a typed quantity so a beginner can check that the problem being solved is the problem that was asked.
Name every coordinate, unit, valid range, missing-value rule, population, and fitted transformation. Separate a raw observation x from a transformed feature phi(x). Changing basis preserves an abstract vector when the change is invertible, but normalization, clipping, hashing, aggregation, and embedding can discard information and alter which objects appear near. Create a notation and shape table before calculating. Include units, support, coordinate or basis choices, conditioning information, fitted state, and invalid inputs. Give one ordinary example, one boundary example, and one counterexample. This prevents syntactically legal arithmetic from silently answering a different question.
Derive and interpret the central result for Vectors, geometry, and representations
Derive Euclidean distance from the norm of a difference and compare it with Manhattan, weighted, and cosine geometry. Show how a diagonal scaling matrix converts unit-dependent coordinates into dimensionless ones, and prove that multiplying one raw feature by a unit conversion can arbitrarily change unscaled nearest neighbors. Write the derivation line by line, cite the definition or theorem used at each transition, and check dimensions, signs, normalization constants, and limiting behavior. A derivation is complete only when the learner can explain why each transformation is allowed and what the final expression says about the original problem.
Build two hand-checkable instances of a scale-aware learner representation and neighbor explorer: one satisfying the premises and one changing exactly one premise. The chapter-specific interpretation is this: Name every coordinate, unit, valid range, missing-value rule, population, and fitted transformation. Separate a raw observation x from a transformed feature phi(x). Changing basis preserves an abstract vector when the change is invertible, but normalization, clipping, hashing, aggregation, and embedding can discard information and alter which objects appear near. Substitute small values, calculate intermediate quantities, and explain why the two outcomes differ before using software; the contrast separates mathematical necessity from representation, data, approximation, and product policy.
Compute and diagnose a scale-aware learner representation and neighbor explorer
Fit scaling parameters on training data only, preserve them with the artifact, calculate distances by hand for a small table, then reproduce them with NumPy. Test zero-variance columns, missing values, extreme outliers, duplicated observations, unseen categories, changed units, and a shifted deployment population. Preserve the input fixture, code version, dtype, random generator state, fitted transformations, expected invariants, and a reference calculation. Inspect intermediate shapes and values instead of accepting a plausible final scalar. Complexity, memory, convergence, and numerical range belong to the algorithm contract just as much as the formula.
The diagnostic mutation is: A mathematically valid nearest-neighbor result becomes misleading when minutes dominate ratios, a fitted scaler sees evaluation rows, a zero norm is divided, or representation distance is asserted to equal semantic or causal similarity. Reproduce it with the smallest example, locate the first violated definition, assumption, shape, independence, numerical, or evaluation invariant, and repair that boundary. Re-run ordinary, boundary, adversarial, shifted, and repeated cases so the repair does not merely hide the final symptom.
Laboratory: build a scale-aware learner representation and neighbor explorer
First write a one-page specification from these controls: A vector is an element of a specified vector space, while a coordinate list is its description in a chosen basis. Addition and scalar multiplication combine directions; norms and metrics measure size and separation only after their axioms, units, and domain are declared. Features therefore encode modeling choices rather than neutral facts. Name every coordinate, unit, valid range, missing-value rule, population, and fitted transformation. Separate a raw observation x from a transformed feature phi(x). Changing basis preserves an abstract vector when the change is invertible, but normalization, clipping, hashing, aggregation, and embedding can discard information and alter which objects appear near. Then derive the expected result for a tiny fixture, implement the simplest transparent version, compare it with an independent calculation or trusted library, and visualize the geometry, distribution, optimization path, graph, or error curve that carries the chapter’s meaning. Record failures and uncertainty rather than deleting inconvenient trials.
Acceptance requires the learner to explain the derivation without the code, reproduce and repair “A mathematically valid nearest-neighbor result becomes misleading when minutes dominate ratios, a fitted scaler sees evaluation rows, a zero norm is divided, or representation distance is asserted to equal semantic or causal similarity.”, and then add a new feature with documented units and compare neighbor stability without leaking evaluation data. Deliver formulas with assumptions, annotated computations, tests and counterexamples, numerical and statistical tolerances, plots with labeled axes, exact environment and artifact identity, limitations, and a short teach-back that predicts an unseen case.