Unit 2, Part 2 of 2 · CEUC302 Theory of Computation
The theory that builds a real compiler stage
Chapter 2 built machines and proved one non-regularity result the Myhill–Nerode way. This chapter finishes the toolkit — how to combine regular languages without starting from scratch (closure properties), and the official, examinable way to prove a language isn't regular (the Pumping Lemma) — then spends it immediately on the reason this unit exists at all: turning raw characters into the token stream a parser expects. Leven and Lends-ab make one last appearance each; a small toy scanner carries the second half of the chapter.
Closure properties
No new machine built from nothing
If you already have a machine for each piece, ∩, ∪, and complement are mechanical.
Section 3.1 builds the actual product machine for Leven ∩ Lends-ab — "aab" sits in the shaded overlap because it has an even a-count and ends in ab.
Before you start
This chapter assumes Chapter 2 throughout: Meven's states and transitions, the Lends-ab NFA and its subset-construction DFA, and the general idea that a DFA's δ is always total. Section 3.2 also leans on Chapter 1's pigeonhole principle and Chapter 2's Myhill–Nerode proof for Leq — not to repeat them, but to contrast a third method against the same target.
Combining machines you already have
∩, ∪, and complement never need a new machine built from scratch — they're mechanical operations on the machines you already built.
Chapter 1 previewed closure as a definition, with finite languages and union as the toy example. Now that real machines exist — Meven and the Lends-ab DFA — the question gets teeth: given two regular languages, each with its own machine, can you build a machine for their intersection without redesigning anything?
The intuition, no symbols yet
Run both machines at once, side by side, feeding each the same input. At every moment, the pair of current states — one from each machine — tells you everything both machines currently know. A pair is like a single combined state; there are only (states in M1) × (states in M2) possible pairs, still a finite number. Whether that pair counts as "accepting" is just a choice you make afterward: both halves accepting means the pair is in for intersection; either half accepting means the pair is in for union.
The formal treatment
Regular languages are closed under union, intersection, complement, concatenation, and star — meaning each operation, applied to regular language(s), always produces another regular language. Each closure comes with an explicit construction:
| operation | construction |
|---|---|
| complement (L̄) | Take a DFA for L. Swap F to Q−F. Nothing else changes. |
| intersection (L1∩L2) | Product construction: states Q1×Q2, δ((p,q),a)=(δ1(p,a),δ2(q,a)), accept iff both p∈F1 and q∈F2. |
| union (L1∪L2) | Identical product construction; accept iff either p∈F1 or q∈F2. |
| concatenation (L1L2) | NFA gluing: every accepting state of M1 gets an ε-edge into M2's start (Section 2.4's construction, one operator at a time). |
| star (L1*) | NFA gluing: ε-edges loop each accepting state back to the start, plus a bypass so zero repetitions is allowed. |
Don't confuse these
- Complement needs a total DFA, not an NFA. Flipping accept/non-accept on an NFA does not give the complement — a string with an empty active set is already "not accepted," and flipping wouldn't magically accept it (there's no state left to flip into acceptance). Complement only works cleanly on a DFA, and only because δ is guaranteed total (Section 2.1): every string leads somewhere, so swapping F is a genuine, complete flip of the verdict.
- Product construction can look bigger than it needs to be. |Q1|×|Q2| states are built, but many might be unreachable or later merge under minimization (Section 2.2). Build first, minimize after, if a smaller machine matters.
Worked example · Leven ∩ Lends-ab
Worked example
product construction, 2 × 3 = 6 statesSame six states, three different languages
flip the accepting rule, get ∩, ∪, or the complement of ∩
fig 3.1 · interactivePractice · 3.1
Seven problems
direct → variation → interpretation → synthesis3.1.1 · From (q1,D2), what state does the product machine move to on symbol b?
Show solution
δeven(q1,b)=q1; δendsab(D2,b)=D0. New state: (q1,D0).
3.1.2 · Complement Meven directly (state which states become accepting). What language results?
Show solution
Swap F from {q0} to {q1}. The resulting language is exactly Lodd — the complement of "even a-count" is "odd a-count" (over the full alphabet {a,b}, with no other strings excluded).
3.1.3 · Variation: build (on paper) the accept-state rule for Leven ∪ Lends-ab, using the same six product states.
Show solution
Accept iff q-part is q0 OR D-part is D2: (q0,D0), (q0,D1), (q0,D2), (q1,D2) — four of the six states, matching fig 3.1's "∪" toggle.
3.1.4 · Variation: is Leven − Lends-ab (strings in Leven but not Lends-ab) regular? If so, describe its accept rule on the same six states without building anything new.
Show solution
Yes — set difference is intersection with a complement, Leven ∩ (Lends-ab)̄, and both operations preserve regularity, so the combination does too. Accept rule: q-part is q0 AND D-part is not D2: (q0,D0), (q0,D1).
3.1.5 · Interpretation: a classmate complements the Lends-ab NFA (Chapter 2) by flipping which of p0,p1,p2 count as accepting, and claims the result recognises "doesn't end in ab." Check a string that breaks this.
Show solution
Try "ba": the NFA's active set ends at {p0,p1} wait — check directly: read b: {p0}; read a: {p0,p1}. Flipping accepting states to {p0,p1} (everything except the old p2) makes this flipped-NFA accept "ba" via p0 or p1 being active — correctly, since "ba" doesn't end in ab. Now try "ab" itself: active set ends at {p0,p2}; under the flipped rule only p0 or p1 count, and p0 IS active, so the flipped machine also accepts "ab" — incorrectly, since "ab" does end in ab and should be rejected by a true complement. The flip breaks precisely because "some thread is in a flipped-accepting state" isn't the same condition as "every thread failed the original acceptance" — confirming the trap box: complement needs a DFA.
3.1.6 · Interpretation: why doesn't building the 6-state product machine ever risk landing outside the regular languages, no matter which two DFAs you started from?
Show solution
The construction only ever produces a finite state set (bounded by |Q1|×|Q2|) with a fully total δ built from two already-total functions — which is, by definition, exactly another DFA. Since every DFA's language is regular, the result is automatically regular; there's no step in the recipe that could produce anything else.
3.1.7 · Synthesis (challenge) · Using closure under complement and intersection, express Leven ∪ Lends-ab using only ∩ and complement (De Morgan's law), and say why this proves union doesn't need its own separate closure proof.
Show solution
Leven ∪ Lends-ab = complement of ( (Leven)̄ ∩ (Lends-ab)̄ ) — De Morgan's law translated directly into languages. Since complement and intersection are each already proven closed, chaining them (complement, then intersection, then complement again) never leaves the regular languages at any step — union closure is technically a free corollary of the other two, though this chapter still gave it its own direct product-construction proof since that's more useful for actually building the machine.
The proof examiners actually want to see
Chapter 1 sketched why Leq can't be regular. Chapter 2 proved it, formally, via Myhill–Nerode. This section proves it a third way — the way most syllabi mean when they just say "use the Pumping Lemma."
Every regular language has a machine with some fixed number of states, say p. Feed that machine a string longer than p symbols, and by the pigeonhole principle (Chapter 1, Section 1.4) it must revisit some state while reading it. Whatever the machine read between the two visits to that repeated state forms a loop — and loops can be gone around any number of times, including zero, without changing where the machine ends up. That's the entire engine behind the Pumping Lemma; everything below just makes it precise enough to write a proof with.
The intuition, no symbols yet
If a long-enough string is accepted, some middle chunk of it must be "loop-shaped" — cut it out, or paste in a second copy, and the machine, which only ever remembers its current state, cannot tell the difference. It will make the exact same decision either way. If you can find even one way of looping that breaks membership, the assumption that a machine existed at all must have been wrong.
The formal treatment
Used directly, this only ever confirms something about a language already known to be regular. Used as a contrapositive, it becomes a disproof machine: to show L is not regular, show that for every candidate p, some string w∈L with |w|≥p has the property that every valid split (satisfying (1) and (2)) fails condition (3) for at least one i. One surviving split anywhere ruins the proof; the argument needs to close off every possible split at once.
Worked example · Leq is not regular, formally
Worked example
the canonical proof, for general pthe split, made visible
fig 3.2Don't confuse these
- One split surviving ruins the whole proof. The disproof needs every valid (x,y,z) to fail — checking a single convenient split and finding a contradiction proves nothing, since a different split might not break.
- Forgetting |xy|≤p loses the entire argument. Without this constraint, y could be chosen anywhere in w, including straddling the a's and b's boundary — and such a y might pump safely. The constraint is what forces y into "all a's" territory for Leq's proof to work at all.
- Surviving the Pumping Lemma never proves regularity. The lemma is a necessary condition of being regular, not sufficient — some non-regular languages can still admit a pumping length that happens not to produce a contradiction through this particular test. A clean pumping-lemma failure proves non-regularity; a clean pumping-lemma "pass" proves nothing on its own.
Practice · 3.2
Seven problems
direct → variation → interpretation → synthesis3.2.1 · State, from memory, the three conditions a split w=xyz must satisfy for the Pumping Lemma to apply.
Show solution
(1) |y| > 0 (never empty), (2) |xy| ≤ p (split happens early), (3) xyiz ∈ L for every i ≥ 0 (pump any number of times, including zero, and stay in the language).
3.2.2 · For p=4, w=a4b4, pick one valid split (x,y,z) with y=aa and show pumping to i=2 breaks Leq membership.
Show solution
x=aa, y=aa, z=b4 (check: |xy|=4≤p=4 ✓, |y|=2>0 ✓). At i=2: xy2z = aa·aaaa·bbbb = a6b4 — 6≠4, not in Leq. Contradiction confirmed for this split (and, per the worked example, for every other valid split too).
3.2.3 · Variation: adapt the worked proof to show L3eq = {anbncn} is not regular via the Pumping Lemma (Chapter 2 did this via Myhill–Nerode; now do it this way).
Show solution
Pick w=apbpcp. Any split with |xy|≤p lands entirely in the leading a's, so y=ak, k≥1. Pumping to i=0 gives ap−kbpcp, with a-count ≠ b-count — not in L3eq. Every split fails identically. Not regular. (Numerically checked at p=3: all 6 splits fail.)
3.2.4 · Variation: use the Pumping Lemma to show L> = {aibj : i>j} is not regular.
Show solution
Pick w=ap+1bp (i=p+1>j=p, so w∈L>). Any split with |xy|≤p lands in the leading a's (there are p+1 of them), so y=ak, k≥1. Pumping to i=0: ap+1−kbp. Since k≥1, p+1−k≤p, so the a-count no longer exceeds the b-count — not in L>. Not regular. (Verified at p=4: all 10 splits fail.)
3.2.5 · Interpretation: a proof picks w=apbp, chooses the specific split x=ε, y=ap, z=bp, shows pumping breaks it, and declares Leq not regular. What's missing?
Show solution
The proof only checked one split (admittedly, per condition (2), it's the only shape a split can take here since the first p symbols are all a's — but the write-up needs to say that every valid split has y consisting only of a's, and argue that generally, rather than silently picking one convenient case and hoping the grader doesn't notice the gap).
3.2.6 · Interpretation: does finding some string and split that survives pumping (stays in L for every i) prove L is regular?
Show solution
No. The Pumping Lemma is a necessary condition of regularity (every regular language satisfies it), not a sufficient one. A non-regular language could still happen to have some strings admit a pumping length and a split that survives — passing this one check is weak evidence at best, never a proof. Proving regularity means exhibiting an actual DFA, NFA, regular grammar, or RE (Chapters 1–2), not surviving a pumping attempt.
3.2.7 · Synthesis (challenge) · Chapters 1–3 have now shown Leq is not regular three ways: an informal pigeonhole sketch, a formal Myhill–Nerode argument, and this section's Pumping Lemma proof. What's the one idea all three genuinely share, and what's actually different between the Myhill–Nerode and Pumping Lemma versions specifically?
Show solution
Shared idea: a machine with finitely many states can't keep an unboundedly large count distinct forever — something must eventually repeat or collide. The difference between the two formal versions is where the argument starts: Myhill–Nerode works directly on the language (comparing strings ai, aj for future-behaviour differences, no machine needed at all), while the Pumping Lemma works by first assuming a machine/pumping-length exists and deriving a contradiction from how it would have to behave. Same destination, opposite starting assumptions.
Where all of this was always heading
A compiler's first stage turns raw characters into a stream of meaningful chunks. Every idea from Chapters 1–3 was quietly built for exactly this.
A token is a category — IDENTIFIER, NUMBER, KEYWORD. A lexeme is the actual text matched — "counter", "42", "if". A pattern is the rule deciding which lexemes belong to which token, and by now patterns are old news: they're regular expressions, exactly as defined in Section 2.4. The whole first stage of a compiler, called the lexical analyser or scanner, is a machine built from those patterns exactly the way Chapters 1–3 already know how to build machines.
The intuition, no symbols yet
Line up every token's pattern side by side and run them all at once against the input, the same parallel-threads idea Section 2.3 used for a single NFA. Whichever pattern (or patterns) are still alive tells you what the text read so far could still turn out to be. The one rule that decides when to stop reading and commit to a token: always take the longest lexeme some pattern will still accept, never the first one that happens to match.
The formal treatment
A lexical specification is a list of (token name, pattern) pairs. Building the scanner: take the RE→FA construction (Section 2.4) for each pattern separately, union them all under one shared start state exactly as Kleene's theorem's union case describes, then run subset construction and minimize (Sections 2.3, 2.2) — producing one DFA whose accepting states are each tagged with which token they complete.
Character classes keep the machine small
Real alphabets are large (26 letters, 10 digits, punctuation) but most patterns treat whole groups of characters identically. Section 3.4's machine uses just two abstract symbols, L (any letter) and D (any digit), standing in for 36 real characters — exactly what a real lexer generator does internally, not a simplification made only for this book.
Worked example · why the scanner doesn't stop at "if"
Worked example
maximal munch, one decision point| rule | result on "if12" | correct? |
|---|---|---|
| stop at first accepting state | IDENTIFIER "if", then restart at "12" → NUMBER "12" | wrong |
| longest match (maximal munch) | IDENTIFIER "if12", one token | correct |
Keywords: a table lookup, not a separate machine path
"if" matches IDENTIFIER's pattern just fine — i and f are both letters. Rather than building competing literal-string machinery inside the DFA, real scanners let the DFA match "if" as an ordinary IDENTIFIER lexeme, then check the result against a small fixed table of reserved words. Only "if," "then," "else," and friends get relabelled KEYWORD after the fact; every other identifier passes through unchanged. Simpler than it sounds, and exactly how production lexers do it.
Don't confuse these
- An accepting state mid-scan is not a stop signal. It only means "a valid token ends here if nothing longer is available" — keep extending while any transition still exists.
- Ties between different token types need a priority rule, separately from length. If two different patterns could both match the exact same longest lexeme, list order (or an explicit priority) breaks the tie — length alone doesn't always settle it.
- Whitespace and comments are usually tokens too, just discarded ones. They get matched and recognised by the scanner like anything else; they're simply never handed to the parser afterward.
Practice · 3.3
Seven problems
direct → variation → interpretation → synthesis3.3.1 · Using IDENTIFIER = L(L+D)*, is "9x" a valid IDENTIFIER lexeme? Why or why not?
Show solution
No. The pattern requires the first symbol to be L (a letter); "9x" starts with a digit. "9x" would instead need to be scanned as NUMBER "9" followed by a separate IDENTIFIER "x" (two tokens).
3.3.2 · What is the longest-match token for the input "abc123"?
Show solution
IDENTIFIER "abc123" — one token. The pattern L(L+D)* happily consumes letters and digits in any mix after the first letter, and there's no character afterward to force a stop.
3.3.3 · Variation: add a token BOOL with pattern (true+false). Does this create any longest-match conflict with IDENTIFIER for the lexeme "true"?
Show solution
Both patterns match "true" at exactly the same length (4 characters) — a genuine tie, not resolved by "longest wins" since they're equally long. This needs an explicit priority rule (list BOOL's pattern before IDENTIFIER's, so BOOL wins ties) — the same style of fix as the keyword-table approach, just applied inside the DFA's tagging instead of after it.
3.3.4 · Variation: scan "12if" against the running token spec (IDENTIFIER, NUMBER, LT, LE). What token stream results?
Show solution
NUMBER's pattern D+ matches "12" but has no transition on a letter, so it stops there: NUMBER "12", then restart at "if": IDENTIFIER "if" (later relabelled KEYWORD by the reserved-word table). Two tokens, in that order.
3.3.5 · Interpretation: a classmate builds a separate 2-state machine just for recognising the literal word "if," wired in parallel with the IDENTIFIER machine. Is this wrong?
Show solution
Not wrong exactly, but needlessly complicated — and it reintroduces the BOOL-style tie problem from 3.3.3 for every keyword, needing an explicit priority rule for each one. The table-lookup approach sidesteps this entirely: let IDENTIFIER's machine match "if" like any other identifier, and relabel afterward. Fewer machine states, one small table, same result.
3.3.6 · Interpretation: why does the longest-match rule specifically need the machine to keep track of "the last position where I was in an accepting state," rather than just the current state?
Show solution
Because the machine might walk past an accepting state into further non-accepting territory that still has live transitions (as "if" → "if1" does, briefly passing through non-obviously-different accepting states) before eventually dying. Only by remembering the last accepting position can the scanner correctly "back off" to the longest valid lexeme once it finally gets stuck, rather than being stuck with only whatever the current (possibly non-accepting, dead) state says.
3.3.7 · Synthesis (challenge) · Explain, using Section 2.4's Kleene's-theorem construction by name, why a lexer generator can accept a plain-text file of (name, regex) pairs and mechanically produce a working scanner with no further human design work.
Show solution
Each regex, by Kleene's theorem, has a mechanically constructible equivalent NFA (RE→FA, Section 2.4). Unioning them under one shared start — itself just the union construction from that same theorem — produces a single NFA for "any of these patterns." Subset construction (Section 2.3) turns it into a DFA, minimization (Section 2.2) shrinks it, and tagging each accepting state with its originating token name finishes the job. Every step is an algorithm already covered, chained together; nothing requires a human to think up new automaton structure by hand, which is exactly why tools like flex can generate a scanner in milliseconds from a spec file.
Building the machine and running it
One combined DFA, one scanning algorithm, one full trace — everything from Chapters 1–3 cashed in at once.
Section 3.3 named the pieces. Here they become one concrete machine: a single DFA covering IDENTIFIER, NUMBER, LT, and LE at once, each accepting state tagged with which token it completes, plus the exact algorithm that runs it correctly.
The intuition, no symbols yet
Walk forward through the input, one character at a time, always taking a live transition when one exists. Every time you land on an accepting state, jot down "a valid token could end right here" without stopping. The moment no transition exists for the next character, stop walking, rewind to the last spot you jotted down, and cut the token there. Then start over from that cut point for the next token.
The formal treatment · the combined machine
one DFA, four token types (dead-state arrows omitted for clarity)
fig 3.4Worked example · the full trace
Worked example
scanning "if12<=x" end to endRun your own input
the scanner, one character at a time
fig 3.4b · interactiveDon't confuse these
- Getting stuck is not the same as an error. Getting stuck with a recorded lastAcceptPos just means "back off and emit" — perfectly normal, happens after every single token. A real lexical error only occurs when the machine gets stuck with no accepting state ever reached since the current token's start.
- The position after backing off is not where the machine got stuck. It's the last place the machine was accepting, which is usually earlier. Reporting the stuck position as the token boundary is a common, specific mistake.
Practice · 3.4
Seven problems
direct → variation → interpretation → synthesis3.4.1 · Trace the algorithm on "x<y" and report the token stream.
Show solution
S→xI (accept). No transition on '<' from I — emit IDENTIFIER "x", pos→1. S→<LT (accept). No transition on 'y' from LT — emit LT "<", pos→2. S→yI (accept), end of input — emit IDENTIFIER "y". Stream: IDENTIFIER "x", LT "<", IDENTIFIER "y".
3.4.2 · Trace the algorithm on "12ab" and report the token stream.
Show solution
S→1N→2N (accept both steps). No transition on 'a' from N — emit NUMBER "12", pos→2. S→aI→bI, end of input — emit IDENTIFIER "ab". Stream: NUMBER "12", IDENTIFIER "ab".
3.4.3 · Variation: what happens on input "12#34"? Report the exact point of failure.
Show solution
NUMBER "12" emits normally (pos→2). Starting fresh at '#': S has no transition on '#' at all (not L, D, <, or =), so lastAcceptPos is never set even once — LEXICAL ERROR at position 2, the '#'. (Verified by direct simulation.)
3.4.4 · Variation: add a token ASSIGN with pattern = (a single equals sign) to the spec. Which existing state's transitions need to change, and how?
Show solution
Only S needs a new transition: currently δ(S,'=')=DEAD; change it to a new accepting state ASSIGN with no further outgoing transitions of its own (mirroring how LT was built before LE existed). Nothing about I, N, LT, or LE changes — adding a token only ever adds machinery reachable from S, consistent with how the union construction (Section 2.4) works.
3.4.5 · Interpretation: a student's trace on "if12<=x" reports the first token as IDENTIFIER "if" instead of "if12." What mistake did they make?
Show solution
They stopped extending at the first accepting state instead of continuing while transitions remained live — the first-match mistake Section 3.3 warned about, not the correct longest-match/maximal-munch behaviour the algorithm actually specifies.
3.4.6 · Interpretation: why does the algorithm reset to state S (not wherever it got stuck) at the start of every new token?
Show solution
Each token is scanned independently — the machine has no memory of "IDENTIFIER" vs "NUMBER" context once a token boundary is cut; every new attempt starts from the same blank slate S, exactly as if scanning a brand-new, shorter input starting at that position. This is what makes the combined DFA reusable token after token without rebuilding anything.
3.4.7 · Synthesis (challenge) · The combined DFA has 5 visible states (S, I, N, LT, LE) plus an implicit dead state. Relate this state count to the closure-property product construction of Section 3.1 — is this DFA "the union of four machines" in the same product-construction sense?
Show solution
No — this is the union-via-NFA-gluing-then-subset-construction route (Section 2.4's Kleene construction), not the product construction. Product construction (Section 3.1) tracks all component machines' states simultaneously as a tuple, growing multiplicatively (|Q1|×|Q2|×…); this scanner instead merges the token patterns into one shared exploration from a single start, growing more like a sum than a product, precisely because at most one "kind" of token is ever being attempted at a time from S. Both routes are valid closure-preserving constructions for union; they just answer different design needs.
Cheat sheet
One line per idea. If a line doesn't ring a bell, that section needs a re-read before Chapter 4.
Closure properties · 3.1
complement — flip F on a total DFA only, never an NFA
product construction — Q1×Q2, move both halves at once
intersection = AND both accept; union = OR either accepts
concatenation, star — NFA gluing via ε-edges (Section 2.4)
Pumping Lemma · 3.2
w=xyz, |y|>0, |xy|≤p, xyiz∈L for all i≥0
disproof: some w, every valid split fails for some i
necessary, not sufficient — passing proves nothing about regularity
one surviving split kills the whole disproof attempt
Lexical analysis · 3.3
token = category, lexeme = matched text, pattern = regex
longest match / maximal munch — never stop at the first accepting state
keywords: match as IDENTIFIER, relabel via reserved-word table lookup
character classes (L, D) keep the machine small
DFA-based scanners · 3.4
remember lastAcceptPos while extending; back off to it when stuck
no lastAcceptPos ever set ⇒ LEXICAL ERROR, not a normal token boundary
restart from the backed-off position, fresh, at state S every time
whole pipeline = RE→NFA (2.4) → DFA (2.3) → minimize (2.2) → tag & scan
Ten questions, no section labels
Exams don't tell you which lecture a question came from. Neither does this.
ST1 · A product construction combines a 3-state DFA and a 4-state DFA. How many product states result?
Show solution
12 (3×4) — possibly fewer reachable in practice, but the construction itself always builds the full |Q1|×|Q2| set before any pruning.
ST2 · True or false: flipping which states are accepting on an NFA produces the complement of its language.
Show solution
False. Complement requires a total DFA; an NFA's "some thread accepts" condition doesn't invert cleanly into "some thread accepts" for the complement (Section 3.1's worked trap on "ab").
ST3 · Why does the Pumping Lemma require |xy| ≤ p specifically?
Show solution
It forces the split to happen within the first p symbols of w — which is exactly what pins y down to a predictable shape (e.g., "all a's" for w=apbp) and makes the contradiction constructible. Without it, y could be chosen anywhere, including a mix that might survive pumping harmlessly.
ST4 · If a string and split survive every pumping attempt, is the language proven regular?
Show solution
No. The lemma is necessary, not sufficient. Surviving pumping on some strings is consistent with regularity but doesn't establish it — only an explicit DFA/NFA/RE/regular grammar does that.
ST5 · Match: "counter", IDENTIFIER, and L(L+D)* to token, lexeme, and pattern.
Show solution
"counter" = lexeme (the actual text). IDENTIFIER = token (the category). L(L+D)* = pattern (the rule deciding membership).
ST6 · Using IDENTIFIER = L(L+D)* and NUMBER = D+, what's the token for "a1"?
Show solution
IDENTIFIER "a1", one token — 'a' starts it as an identifier, and the digit that follows is allowed to continue it (L(L+D)* permits digits after the first letter).
ST7 · In the scanning algorithm, what happens when lastAcceptPos is never set during a token attempt?
Show solution
LEXICAL ERROR is reported at the token's start position — there is no valid token boundary to back off to, which is different from the routine "got stuck after a valid accept" case.
ST8 · L1 is regular, L2 is not regular. Is L1 ∪ L2 necessarily regular, necessarily not regular, or neither?
Show solution
Neither is guaranteed. The union-closure theorem requires both inputs regular to promise anything; with one input non-regular, the union could go either way (e.g. L1=Σ* makes the union Σ*, regular, regardless of what L2 is) — no general conclusion follows from the theorem alone.
ST9 · A machine has 5 states. What's the shortest input length that pigeonhole guarantees forces a repeated state?
Show solution
5 symbols — reading 5 symbols means 6 state-visits total (including the start), which by pigeonhole must repeat some state among only 5 available. This is exactly the pumping length threshold |w|≥p with p=5.
ST10 · Scanning begins on input starting with '#', a character no pattern recognises. What's the very first thing that happens?
Show solution
S has no transition defined for '#' — the inner loop of the algorithm never executes even once, lastAcceptPos stays unset, and a LEXICAL ERROR is reported immediately at position 0, before any other token is attempted.
Where this goes next
- Unit 3 moves from recognising to generating: context-free grammars, formally, with the ambiguity groundwork from Chapter 1 finally paid off in full.
- The "guess and confirm" NFA design instinct from Chapter 2 reappears as the intuition behind pushdown automata (Unit 4) adding a stack to the same machine idea.
- Leq, provably not regular three ways now, becomes the first language this whole course can finally build a machine for — once that machine has a stack.
Further reading
- Sipser, Introduction to the Theory of Computation, 3rd ed. — the regular-languages chapter's closure and pumping-lemma sections match this chapter's order and notation closely.
- Aho, Lam, Sethi, Ullman, Compilers: Principles, Techniques, and Tools. — the standard reference for lexical analysis and scanner construction in practice, considerably beyond this chapter's toy token set; good for seeing the real scale of a production lexer specification.
Next: Chapter 4 · CFG I — Grammars, Derivations & Ambiguity