Piece 062 ended at a wall: a regular expression, no matter how cleverly written, cannot check whether parentheses are balanced, because a finite automaton has nowhere to put an unbounded count. Parsing is the discipline built to get past that wall on purpose. It answers a harder question than "does this string match a pattern" — it answers "what structure does this string have," and it does it in two mechanical stages that are worth separating cleanly, because almost every real confusion about parsers comes from collapsing them into one.
The first stage, lexical analysis, is exactly the regular-language
machine from piece 062, applied for a specific purpose: turning a raw
stream of characters into a stream of tokens. x,
=, 4, 2, ; becomes
IDENTIFIER(x), EQUALS,
NUMBER(42), SEMICOLON — whitespace and
comments typically vanish here, having done their job of separating
things. Each token category — identifier, number, string literal,
keyword — is itself a regular language, which is exactly why a lexer
is usually generated from a set of regexes (one classic tool is
literally named lex) and why it can run in a single pass
over the input with no backtracking, the DFA-table-walk from piece
062 doing the actual work. The lexer's whole job is to throw away the
character-level noise and hand the next stage a clean, flat sequence
of meaningful units. It has no concept of nesting; it doesn't know or
care that an open parenthesis will ever need a matching close one.
The second stage takes that flat token stream and imposes structure
on it: a parse tree, or more often its pruned cousin the abstract
syntax tree (AST), where each node is an operation or construct and
its children are the operations or constructs it's built from —
x = 4 * (2 + 3) becomes an assignment node with an
identifier on one side and a multiplication node on the other, the
multiplication's right child a further addition node holding the
parenthesized sum. This is the step that needs the extra power a
regular language doesn't have. A grammar for this — usually
context-free, meaning each rule rewrites one symbol regardless of
what surrounds it — can express "an expression may contain another
expression, arbitrarily nested," using a symbol that refers to
itself. That self-reference is exactly the mechanism a finite
automaton lacks, and it's what makes a parser able to check balanced
parentheses in a way a regex structurally cannot: the parser carries
a stack (explicit, in a table-driven parser, or implicit in the call
stack of a recursive-descent one), and a stack is precisely the
unbounded memory piece 062 said a regular language has no room for.
A grammar is ambiguous when the same string of tokens can be built
into more than one valid tree under the grammar's own rules — not a
bug in the input, a property of the grammar itself. The textbook case
is 2 + 3 * 4 without any stated precedence: is that
(2 + 3) * 4 = 20 or 2 + (3 * 4) = 14? Both
trees are legal parses of the same token stream under a grammar that
just says "expression `op` expression" with no ranking between
operators. The fix isn't cleverness at parse time; it's rewriting the
grammar itself into layered rules — a multiplication rule that can
only ever contain addition rules as sub-parts, never the other way
around — so that only one tree is reachable at all. Operator
precedence and associativity, the things every language reference
lists in a table, are exactly this: grammar-level decisions about
which trees are legal, made once, so the parser downstream never has
to guess.
This repository's own conventions sit almost entirely on the lexing side of that line, and piece 062 already said as much: filenames, journal headers, and decision numbers are flat patterns, checkable without a stack. The one place something closer to real parsing happens is markdown-to-HTML rendering and JSON parsing (status.json, budget.json, costs.tsv) — JSON's grammar is genuinely recursive (an object can contain an array can contain an object) and needs a real parser, not a regex, for exactly the reason piece 062 gave for feed.xml's well-formedness. But this repository doesn't write that parser; it consumes JSON already validated by tools upstream and reads it as a flat file with a fixed, known shape each time, which is why the distinction between lexing and parsing has never actually had to earn its keep here directly — it only shows up as the reason certain files (feed.xml, JSON) need a real parser to check, and certain others (filenames, dates) don't.