Words Into Numbers

Introduction

S7 found a word's place in the table. But a word's place is a poor thing to build a game on: put a new word near the top of the table and every number below it changes. And GET and TAKE are two places that should mean one thing. The answer is a second column beside the words, saying what each word means: its token.

A Token For Every Word

type
  Verb = (Nothing, Go, Take, Drop,
          Look, Inv, Quit);
const
  Words: array[1..Count] of Str12 =
    ('GO', 'WALK', 'TAKE', 'GET', ...);
  Means: array[1..Count] of Verb =
    (Go, Go, Take, Take, ...);

Words[3] is TAKE and Means[3] is Take. Words[4] is GET, and Means[4] is also Take. Two words, one meaning: a synonym costs one entry in each list and nothing anywhere else.

Find looks up the typed word and returns its token, or Nothing. From then on the program deals only in tokens, and a case acts on them:

case V of
  Go:   ...
  Take: ...

Turbo will not case on a string (Error 46), so a parser that kept its words as strings would be a long chain of ifs, each comparing a string letter by letter. A token is compared as a single small number.

An enumerated value cannot be written out (Writeln(V) is Error 66), but Ord(V) gives its number: Nothing is 0, Go 1, and so on in the order they are declared.

How Alice Did It

Alice and the March Hare kept exactly this table, in its data file, one entry after another: each word, a $ to end it, and then one byte, its token.

Synonyms share a token, and the one-letter words are simply more synonyms. In play, KILL and STAB got the same answer.

The Code

program Tokens;
type
  Str12 = string[12];
  Verb = (Nothing, Go, Take, Drop,
          Look, Inv, Quit);
const
  Count = 11;
  Words: array[1..Count] of Str12 =
    ('GO', 'WALK', 'TAKE', 'GET',
     'DROP', 'LOOK', 'L', 'INVENTORY',
     'I', 'QUIT', 'STOP');
  Means: array[1..Count] of Verb =
    (Go, Go, Take, Take, Drop, Look,
     Look, Inv, Inv, Quit, Quit);
var
  W: Str12;
  V: Verb;

function Find(W: Str12): Verb;
var
  I: Integer;
begin
  Find := Nothing;
  for I := 1 to Count do
    if Words[I] = W then
      Find := Means[I];
end;

begin
  repeat
    Write('> ');
    Readln(W);
    V := Find(W);
    Write('Token ', Ord(V), ': ');
    case V of
      Go:   Writeln('You walk.');
      Take: Writeln('Taken.');
      Drop: Writeln('Dropped.');
      Look: Writeln('You look.');
      Inv:  Writeln('Empty hands.');
      Quit: Writeln('Goodbye.');
      Nothing:
        Writeln('I don''t know ', W);
    end;
  until V = Quit;
end.

Starting from

A new work file TOKENS.

What you should see

> GET
Token 2: Taken.
> TAKE
Token 2: Taken.
> L
Token 4: You look.
> I
Token 5: Empty hands.
> JUMP
Token 0: I don't know JUMP
> STOP
Token 6: Goodbye.

Change One Thing

• In Means, change the fourth value (the one for GET) from Take to Drop. What does GET do now? How many other lines had to change?
• Add Writeln(V); after V := Find(W);. What does the compiler say?
• Change Count = 11 to Count = 10, and change nothing else. What does the compiler say, and why?

Exercises

8.1 Add a new verb, Exam, with two words for it, EXAMINE and X.

8.2 Using an array[Verb] of Integer, count how many words there are for each verb, and print the counts when the program starts.

8.3 Add a typed constant Name: array[Verb] of Str12 holding a name for each verb, and print GET means TAKE instead of the token's number.

Worked solutions are in Appendix II.

When It Goes Wrong

Summary

A second table gives every word a token, and synonyms are just two words with the same token. After Find, the program works in tokens: small numbers, or values of an enumerated type, that a case can act on and that cost nothing to compare. Alice and the March Hare stored its word list this way, a word and a one-byte token at a time.

Next

S9, Finding A Word Faster: a long table, read from the disc, and two ways to search it.