Introduction

It is a basic tenet of the TEX faith that TEX is Turing-equivalent and that we can write any program in TEX. It is also widely held that TEX is “the most idiosyncratic language known to us.”^{1} This project is an attempt to provide a simple programming front end to TEX.

Basic was selected because it is a widely used interpreted language. It also features an infix syntax not found in Lisp or PostScript. This makes it a more difficult but more general problem than either of these others.

The speed of the BASIX interpreter is not impressive. It is not meant to be. The purpose of this interpreter is not to serve as the Basic implementation of choice. Its purpose is to display useful paradigms of input parsing and advanced TEX programming.

Interaction with TEX

Associative arrays. Using \csname it is possible to implement associative arrays in TEX. Associative arrays are arrays whose index is not necessarily a number. As an example, if \student has the name of a student, we might look up the student’s grade with

\csname grade.\student\endcsname

which would be \grade.Greene in my case. (In the case of \csname, all characters up to the \endcsname are used in the command sequence regardless of their category code.)

1 Ward, Stephen A. and Robert H. Halstead, Jr., Computation Structures, MIT Press, 1990

BASIX — An Interpreter Written in TEX

Andrew Marc Greene

MIT Project Athena, E40-342, 77 Massachusetts Avenue, Cambridge, MA 02139

617-253-7788. Internet: amgreene@mit.edu

Abstract

An interpreter for the Basic language is developed entirely in TEX. The interpreter presents techniques of scanning and parsing that are useful in many contexts where data not containing formatting directives are to be formatted by TEX. TEX’s expansion rules are exploited to provide a macro that reads in the rest of the input file as arguments and which never stops expanding.

BASIX makes extensive use of these arrays. Commands are begun with C; functions with F; variables with V; program lines with /; and the linked list of lines with L. This makes it easy for the interpreter to look up the value of any of these things, given the name as perceived by the user.

One benefit of \csname… \endcsname is that if the resultant command sequence is undefined, TEX replaces it with \relax. This allows us to check, using \ifx, whether the user has specified a non-existent identifier. This trick is used in exercise 7.7 in The TEXbook. We use it in BASIX to check for syntax errors and uninitialized variables.

Token streams. The BASIX interpreter was designed to be run interactively. It is called by typing tex basix; the file ends waiting for the first line of Basic to be entered at TEX’s * prompt. This also allows other files to \input basix and immediately follow it with Basic code.

We cannot have the scanner read an entire line at once, since if the last line of basix.tex were a macro that reads a line as a parameter,

we’d get a “File ended while scanning use of

\getline” error. Instead, we use a method which at first blush seems more convoluted but which is actually simpler.

We note that TEX does not make any distinction between the tokens that make up our interpreter and the tokens that form the Basic code. The BASIX interpreter is carefully constructed so that each macro ends by calling another macro (which may read parameters). Thus, expansion is never completed, but the interpreter can continue to absorb individual characters that follow it. These characters affect the direction of the expansion; it is this behaviour that allows us to implement a Basic interpreter in TEX.

Category codes. Normally, TEX distinguishes between sixteen categories of characters. To avoid unwanted side effects, the BASIX interpreter reassigns category codes of all non-letters to category 12, “other”. One undocumented feature of TEX is that it will never let you strand yourself without an “active” character (which is normally the backslash); if you try to reassign the category of your only active character (with, e.g., \catcode`\\=12), it will fail silently. To allow us to use every typeable character in BASIX, we first make \catcode31=0, which then allows us to reassign the catcode of the backslash without stranding ourselves. (Of course, the BASIX interpreter provides an escape back to TEX which restores the normal category codes.)

We also change the category code of ^^M, the end-of-line character, to “active”. This lets us detect end-of-line errors that may crop up.

etc..

Semantics

Words. A word is a collection of one or more characters that meet requirements based on the first character. The following table describes these rules using regular expression notation.^{2} These rules are:

First  Regular
Character Expression  Meaning

[A-Z,a-z]  [A-Z,a-z][A-Z,a-z,0-9]*$? Identifier
[0-9]      [0-9]+  Integer
"          "[^"]*[",^^M]  String

Other  .  Symbol

An end-of-line at the end of a string literal is converted to a ".

Everything in BASIX is one of these four types. Line numbers are integer literals, and both variables and commands are identifiers.

The \scan macro is used in BASIX to read the next word. It uses \futurelet to look at the next token and determine whether it should be part of the current word; i.e., whether it matches the regular expression of the current type. If so, then it is read in and appended; otherwise \scan returns,

2 In this notation, [A-Z,a-z] means “any character falling between A and Z or between a and z, inclusive.” An asterisk means “repeat the preceding specification as many times as needed, or never.” A plus means “repeat the preceeding specification as many times as needed, at least once.” A question mark means “repeat the preceeding specification zero or one times.” A dot means “any character.”

Andrew Marc Greene

leaving this next token in the input stream. The word is returned in the macro \word.

The peculiar way \scan operates gives rise to new problems, however. We can’t say

    \def\Cgoto{\scan\lineno=\word}

because \scan looks at the tokens which follow it, which in this case are \lineno=\word. We need some way to define the goto command so that the \scan is at the end of the macro; this will take the next tokens from the input stream. We therefore have \scan “return” to its caller by breaking the caller into two parts; the first part ends with \scan and the second part contains the code which should follow. The second macro is stored

in \afterscan, and \scan ends with \afterscan.

As syntactic sugar, \after has been defined as

\let\afterscan. This allows \Cgoto to be coded

as

    \def\Cgoto{\after\gotoPartTwo\scan}
    \def\gotoPartTwo{\lineno=\word}

(Actually, the goto code is slightly more complicated than this; but the scanner is the important point here.) This trick is used throughout the BASIX interpreter to read in the next tokens from the user’s input without interrupting the expansion of the TEX macros that comprise the interpreter.

Expressions. An expression is a sequence of words that, roughly speaking, alternates between values and operators. A value is either a literal, an identifier, or a parenthesized expression. An operator is one of less-than, more-than, equality, addition, subtraction, multiplication, division, or reference. (Reference is an implicit operator that is inserted between a function identifier and its parameters.)

Expressions are evaluated in an approach similar to that used in the scanner. A word is scanned using \scan and its type is determined. “Left-hand” values are stored for relatives, additives, multiplicatives, and references. Using TEX’s grouping operations the evaluator is reentrant, permitting parenthesized expressions to be recursively evaluated.

In order to achieve a functionality similar to that of \futurelet, we exit the evaluator by

\expandafter\aftereval\word, where the macro
\aftereval is analogous to \afterscan. Since

\word will contain neither macros nor tokens whose category codes need changing, this is as good as

\futurelet.

Structure of the Interpreter

The file basix.tex, which appears as an appendix to this paper, defines all the macros that are needed to run the interpreter. The last line of this file is \endeval, which is usually called when the interpreter has finished evaluating a line. In this case, it calls \enduserline, which, in turn, calls

\parseline.

The \parseline macro is part of the Program Parser section of the interpreter. It starts by calling \scan to get the first word of the next line. If \word is an integer literal, it is treated as a line number and the rest of the input line is stored in the appropriate variable without interpretation. If \word is not an integer literal, it is treated as a command.

Each command is treated with an ad hoc routine near the bottom of basix.tex. However, most of them call on a set of utilities that appear earlier in the file.

Character-string calls. There is a simple library of macros that convert between ASCII codes and character tokens, test for string equality, take subsections of strings, and deal with concatenation.

Debugging definitions. The macro \diw is a debugging-mode-only \immediate\write16 (hence the name \diw). It is toggled by the user commands

debug and nodebug.

Expression evaluation. The expression evaluator has a calling structure similar to that of the scanner. Calling routines are split in twain, with

    \def\foo{\after\fooPartTwo\expression}
    \def\fooPartTwo{\etc}

being a prototype of the calling convention. The evaluator will \scan as many words as it can that make sense; in contrast to the scanner, however, it evaluates each instead of merely accumulating them. This process is described above.

Linked list. The BASIX interpreter maintains a linked list of line numbers. The macro

\csname L⟨current line number⟩\endcsname contains the next line number. These macros will follow the linked list (for the list command); they also can insert a new line or return the number of the next line.

Program parser. This section contains a number of critical routines. \evalline is the macro that does the dispatching based on the user’s command. \mandatory specifies what the next character must be (for example, the character after the identifier

in a let statement must be =). \parseline has already been described.

Syntactic scanner. This is the section containing \scan and its support macros, which are described above.

Type tests. These routines take an argument and determine whether it is an identifier, a string variable, a string literal, an integer literal, a macro, or a digit. The normal way of calling these routines is

                                                  \expandafter\if\stringP\word

These predicates expand into either tt (true) or tf (false). Syntactic sugar is provided in the form of

                                              \itstrue and \itsfalse. \ifstring\word doesn’t

work because of the way TEX matches \if and \fi tokens — only \if-style primitives are recognized.

User Utilities. This is the section of the interpreter in which most of the user commands are defined. Commands are preceded by \C (e.g., \Clist is the macro called when the user types list). Functions are preceded by \F.

Limitations of this implementation

BASIX is a minimal Basic interpreter. There are enough pieces to show how things work, but not enough to do anything practical. Here is a description of the capabilities of this interpreter, so that the reader can play with it. Error recovery is virtually non-existant, so getting the syntax right and not calling non-existent functions is critical.

Entering programs. Lines beginning with an integer literal are stored verbatim. Lines are stored in ascending order, and if two or more lines are entered with the same number, only the last is retained.

Immediate commands. Lines not beginning with an integer are executed immediately. Colons are not supported, so only one command may appear on a line. (When a program line is executed, its line number is stripped and the remainder is executed as though it were an immediate command.)

Commands. The following commands are implemented in some form: goto, run, list, print, let,

                                              if, debug, nodebug, rem, system, exit, and stop

(but not cont). The interpreter is case sensitive (although with an appropriate application of \uppercase it needn’t be; I was lazy), so these must be entered in lowercase.

The following tables list the commands with no parameters, the commands that take one parameter, and the two commands (let and if) that take special forms.

The commands with no parameters are:

run  Starts execution at the lowest line number.

Does not clear variables.

stop  Stops execution immediately.
list  Lists all lines in order.
debug Useful information sent to terminal.
nodebug Stop debugging mode.
rem  Rest of line is ignored.
system Exit TEX.
exit  Exit BASIX to TEX.

The commands with one parameter are:

goto Starts execution at the given line number.
list Lists the given line.
print Displays the given argument.

(Any of these arguments may be an expression.)

The let and if commands take special forms. Variable assignments require an explicit let command:

let ⟨identifier⟩ = ⟨expression⟩ Conditionals do not have an else clause, and goto is not implied by then:

if ⟨expression⟩ then ⟨new command⟩ The new command is treated as its own line.

Expressions. Expressions are defined explicitly above. The operators are +, -, *, /, <, =, and >. Parentheses may be used for grouping. Variables may not be referenced before being set. (Unlike in traditional Basic, variables are not assumed to be 0 if never referenced, and they aren’t cleared when run is encountered).

Functions are invoked with ⟨function name⟩(⟨param⟩,⟨param⟩,...)

The parameters are implicitly-delimited expressions that are passed to \matheval (which is simply called \eval in the table below to save space). The following functions are defined:

len(string)  Returns number of characters

in the string.

chr$(expr)  Returns the character with the

given ascii value.

inc(expr)  Returns \eval(expr) + 1.
min(expr1,expr2) Returns the lesser of two \evals.

Generalization

The BASIX interpreter can easily be generalized to serve other needs. These other needs might be to interpret Lisp or PostScript code [Anyone want to write a PostScript interpreter in TEX?];

or to take source code in a given language and pretty-print it.

The definition of a “word” can be changed by modifying the \scan macro. It selects a definition for \scantest based on the first character; scantest is what determines if a given token matches the selected regular expression. The \scantest macro is allowed to redefine itself.

The evaluation of expressions can be extended or changed by modifying the \math code. Floatingpoint (or even fixed-point) numbers could be dealt with, although the period would need to pass the \digitP test in some cases and not in others.

The method of dealing with newlines is easily removed for languages such as PostScript or Lisp for which all whitespace is the same.

Obtaining copies of basix.tex

The source code to this paper and the BASIX interpreter are available by anonymous ftp from gevalt.mit.edu, which is at IP address 18.72.1.4. I will also mail out copies to anyone without ftp abilities.

Summary

Using a number of TEX tricks, some more devious than others, a Basic interpreter can be written in TEX. While TEX macros will often be less efficient than, for example, awk paired with TEX, solutions using only TEX will be more portable. A less general macro package than BASIX could be written that uses these routines as paradigms and that is very efficient at parsing a specific input format.

                           Listing of basix.tex

1. %

 2. %   BaSiX (with the emphasis on SICK!) by Andrew Marc Greene

3. %

 4. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

5. %

 6. %  Andrew's Affiliations

7. %

 8. %   Copyright (C) 1990 by Andrew Marc Greene
 9. %   <amgreene@mit.edu>
10. %   MIT Project Athena
11. %   Student Information Processing Board
12. %   All rights reserved.

13. %

14. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

15. %

16. %  BaSiX's Beginnings

17. %

18. \def\flageol{\catcode13=13}
19. \def\endflageol{\catcode13=5}
20. \def\struncat{\catcode`\$=12}
21. \def\strcat{\catcode`\$=11}
22. \flageol\let\eol
23. \endflageol
24. \newif\ifresult
25. %\newcount\xa\newcount\xb
26. \def\iw{\immediate\write16}
27. \def\empty{}
28. \def\gobble#1{}
29. \def\spc{ }
30. \def\itstrue{tt}
31. \def\itsfalse{tf}
32. \def\isnull#1{\resultfalse
33. \expandafter\ifx\csname empty#1\endcsname\empty\resulttrue\fi}
34. \newcount\matha\newcount\mathb

35. %

36. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

37. %

38. %  Character-string Calls

39. %

40. \newcount\strtmp
41. \def\ascii#1{\strtmp`#1}
42. \def\chr#1{\begingroup\uccode65=#1\uppercase{\gdef\tmp{A}}\endgroup}
43. \def\strlen#1{\strtmp-2% don't count " " \iw tokens
44. \expandafter\if\stringP #1\let\next\strIter\strIter #1\iw\fi}
45. \def\strIter#1{\ifx\iw#1\let\next\relax\else\advance\strtmp by 1\relax
46. \fi\next}
47. \def\Flen{\expandafter\strlen\expandafter{\Pa}\return{\number\strtmp}}
48. \strcat
49. \def\F$chr${\expandafter\chr\expandafter{\Pa}\return{\tmp}}
50. \struncat
51. % first char only:
52. % \def\Fasc{\expandafter\asc\expandafter{\Pa}\return{\number\strtmp}}

53. %

54. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

55. %

56. %  Debugging Definitions

57. %

58. \def\debug{\tracingmacros=2}
59. \def\diw#1{}
60. \def\Cdebug{\let\diw\iw\tracingmacros=2 \endeval}
61. \def\Cnodebug{\def\diw##1{}\endeval}

62. %

63. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

64. %

65. %  Expression Evaluation

66. %

67. \def\expression{\let\afterexpression\afterscan\math}

68. %

69. %  (math is a misnomer and should -> expr)

70. %

71. \newcount\parens\newcount\mathParams
72. \def\math{\parens=0\mathParams96\mathInit\matheval}
73. \def\mathRecurse{\advance\parens by 1\relax\mathParams96\mathInit\matheval}
74. \def\mathInit{\begingroup
75. \let\mathAcc\empty
76. \let\mathOpRel\empty
77. \let\mathOpAdd\empty
78. \let\mathOpMul\empty
79. \let\mathlhRef\empty}

80. %

81. \def\matheval{\after\mathbranch\scan}

82. %

83. \def\mathbranch{\diw{EXPRESS:\expandafter\noexpand\word:}
84. \let\next\matherr
85. \ifx\empty\word\let\next\mathHardEnd\else                   % Expr. end?
86. \expandafter\if\numberP\word\let\next\mathLiteral\diw{@@}\fi  % Number?
87. \expandafter\if\stringP\word\let\next\mathLiteral\fi        % String literal?
88. \expandafter\if\identifierP\word\let\next\mathIdentifier\fi % Identifier?
89. \expandafter\if\stringvarP\word\let\next\mathIdentifier\fi  % :-(
90. \expandafter\if\macroP\word\let\next\mathMacro\fi           % Macro?
91. \ifx\word\Oleft\let\next\mathRecurse\fi                     % Open paren?
92. \ifx\word\Oright\let\next\mathEndRecurse\fi                 % Close paren?
93. \ifx\word\Ocomma\let\next\mathComma\fi                      % Comma?

94. %

95. %  Operator?

96. %

97. \ifx\word\Oplus\let\next\mathOp\diw{!+}\fi
98. \ifx\word\Ominus\let\next\mathOp\diw{!-}\fi
99. \ifx\word\Otimes\let\next\mathOp\diw{!*}\fi
100. \ifx\word\Odiv\let\next\mathOp\diw{!/}\fi
101. \ifx\word\Olt\let\next\mathOp\diw{!<}\fi
102. \ifx\word\Oeq\let\next\mathOp\diw{!=}\fi
103. \ifx\word\Ogt\let\next\mathOp\diw{!>}\fi

104. %

105. \fi\next}

106. %

107. \def\Oleft{(}\def\Oright{)}\def\Ocomma{,}
108. \def\Oplus{+}\def\Ominus{-}\def\Otimes{*}\def\Odiv{/}
109. \def\Olt{<}\def\Oeq{=}\def\Ogt{>}

110. %

111. % There's got to be a better way to do the above....

112. %

113. \def\mathLiteral{\diw{MLIT}\ifx\empty\mathAcc\diw{ACCUMUL:\word:}
114. \expandafter\def\expandafter\mathAcc\expandafter
115. {\expandafter\expandafter\expandafter\empty\word}
116. \else
117. \diw{ACC has :\mathAcc: and word is :\word:}
118. \errmessage{Syntax Error: Two values with no operator}\fi\matheval}

119. %

120. %  Operator stuff:  (Need to add string support / error checking)

121. %

122. \def\mathAdd{\advance\matha by \mathb}
123. \def\mathSub{\advance\matha by -\mathb}
124. \def\mathMul{\multiply\matha by \mathb}
125. \def\mathDiv{\divide\matha by \mathb}
126. \def\mathEQ{\ifnum\matha=\mathb\matha-1\else\matha0\fi}
127. \def\mathGT{\ifnum\matha>\mathb\matha-1\else\matha0\fi}

128. \def\mathLT{\ifnum\matha<\mathb\matha-1\else\matha0\fi}

129. %

130. \def\mathFlushRel{\mathFlushAdd\ifx\empty\mathOpRel\else
131. \matha=\mathlhRel\relax\mathb=\mathAcc\relax\mathOpRel
132. \edef\mathAcc{\number\matha}\let\mathOpRel\empty\fi}

133. %

134. \def\mathFlushAdd{\mathFlushMul\ifx\empty\mathOpAdd\else
135. \matha=\mathlhAdd\relax\mathb=\mathAcc\relax\mathOpAdd
136. \edef\mathAcc{\number\matha}\let\mathOpAdd\empty\fi}

137. %

138. \def\mathFlushMul{\mathFlushRef\ifx\empty\mathOpMul\else
139. \matha=\mathlhMul\relax\mathb=\mathAcc\relax\mathOpMul
140. \edef\mathAcc{\number\matha}\let\mathOpMul\empty\fi}

141. %

142. \def\mathFlushRef{\ifx\empty\mathlhRef\else
143. \mathParam
144. \mathlhRef\let\mathlhRef\empty\fi}

145. %

146. \def\mathOp{%
147. \if\word+
148. \mathFlushAdd\let\mathlhAdd\mathAcc\let\mathOpAdd\mathAdd\fi
149. \if\word-
150. \mathFlushAdd\let\mathlhAdd\mathAcc\let\mathOpAdd\mathSub\fi
151. \if\word*
152. \mathFlushMul\let\mathlhMul\mathAcc\let\mathOpMul\mathMul\fi
153. \if\word/
154. \mathFlushMul\let\mathlhMul\mathAcc\let\mathOpMul\mathDiv\fi
155. \if\word=
156. \mathFlushRel\let\mathlhRel\mathAcc\let\mathOpRel\mathEQ\fi
157. \if\word>
158. \mathFlushRel\let\mathlhRel\mathAcc\let\mathOpRel\mathGT\fi
159. \if\word<
160. \mathFlushRel\let\mathlhRel\mathAcc\let\mathOpRel\mathLT\fi
161. \let\mathAcc\empty
162. \matheval}

163. %

164. \def\mathIdentifier{%
165. \expandafter\ifx\csname C\word\endcsname\relax
166. \expandafter\ifx\csname F\word\endcsname\relax
167. \expandafter\ifx\csname V\word\endcsname\relax
168. \let\next\matherr\diw{LOSING:\word:}
169. \else\let\next\mathVariable\fi
170. \else\let\next\mathFunction\fi
171. \else\let\next\mathCommand\fi\next}

172. %

173. \def\mathVariable{\expandafter\edef\expandafter\word\expandafter
174. {\csname V\word\endcsname}\mathbranch}
175. \def\mathCommand{\expandafter\mathHardEnd\word}
176. \def\mathFunction{\expandafter\let\expandafter\mathlhRef
177. \csname F\word\endcsname\matheval}

178. %

179. \def\mathParam{\advance\mathParams by 1\relax\chr\mathParams
180. \diw{PARAM:\tmp:\mathAcc:}
181. \expandafter\edef\csname P\tmp\endcsname{\mathAcc}}
182. \def\mathComma{\mathEnd\mathParam\mathInit\matheval}
183. \def\mathEndRecurse{\mathEnd\advance\parens by -1\matheval}
184. \def\mathEnd{\diw{MATHEND: ACC=\mathAcc:}\mathFlushRel
185. \xdef\mathtemp{\mathAcc}\endgroup\edef\mathAcc{\mathtemp}}
186. \def\mathHardEnd{\ifnum\parens>0\errmessage{Insufficient closeparens.}\relax
187. \let\next\endeval\else\let\next\mathFinal\fi\next}
188. \def\mathFinal{\mathEnd\let\value\mathAcc\endexpression}
189. \def\matherr{\errmessage{Syntax error: Unknown symbol \word}}
190. \def\endexpression{\afterexpression}

191. %

192. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

193. %

194. %  Linked List

195. %

196. \def\gotofirstline{\edef\lpointer{\csname L0\endcsname}}
197. \def\foreachline#1{\ifnum\lpointer<99999\edef\word{\lpointer}#1%
198. \edef\lpointer{\csname L\word\endcsname}\foreachline{#1}\fi}

199. %

200. %  gotopast{#1}  where #1 is a line number, will set \lpointer to
201. %                the least value such that L(lpointer)>#1

202. %

203. \def\gotopast#1{\def\lpointer{0}\def\target{#1}\gotopastloop}

204. %

205. \def\gotopastloop{\edef\tmp{\csname L\lpointer\endcsname}%
206. \ifnum\tmp<\target%
207. \edef\lpointer{\csname L\lpointer\endcsname}%
208. \let\next=\gotopastloop\else\let\next=\relax\fi
209. \next}

210. %

211. \flageol\def\addLineToLinkedList#1#2
212. {\def#1{#2}\diw{Just stored #2 in \noexpand #1}%
213. % now put it into linked list...
214. \expandafter\ifx\csname L\word\endcsname\relax% if it isn't already there,
215. \gotopast{\word}% \def\lpointer{what-should-point-to-word}
216. \expandafter\edef\csname L\word\endcsname{\csname L\lpointer\endcsname}%
217. \expandafter\edef\csname L\lpointer\endcsname{\word}%
218. \fi\endeval
219. }\endflageol
220. \expandafter\def\csname L0\endcsname{99999}

221. %

222. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

223. %

224. %  Program Parser

225. %

226. \def\evalline{%\iw{EVALLINE :\word:}%
227. \csname C\word\endcsname}  %error-checking?  :-)
228. \def\evalerror{\errmessage{Unkonwn command.  Sorry.}}

229. %

230. %  \mandatory takes one argument and checks to see if the next
231. %  non-whitespace token matches it.  If not, an error is generated.

232. %

233. \def\mandatory#1{\def\tmp{#1}\mandatest}
234. \def\mandatest#1{\def\tmpp{#1}\ifx\tmp\tmpp\let\next\afterscan\else
235. \let\next\manderror\fi\next}
236. \def\manderror{\errmessage{\tmpp\spc read when \tmp\spc expected.}%
237. \afterscan}

238. %

239. %  \parseline gets the first WORD of the next line.  If it's a line
240. %  number, \scanandstoreline is called; otherwise the line is executed.

241. %

242. \def\parseline{\after\firsttest\scan}
243. \def\firsttest{\expandafter\if\numberP\word
244. \let\next\grabandstoreline\else\let\next\evalline\fi\next}
245. \def\grabandstoreline{\diw{Grabbing line \word.}%
246. \expandafter\addLineToLinkedList\csname/\word\endcsname}

247. %

248. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

249. %

250. %  Syntactic Scanner

251. %

252. %  The \scan routine reads the next WORD and then calls \afterscan.

253. %

254. %  As syntactic sugar, one can write \after\foo to set \afterscan to
255. %  \foo.

256. %

257. %  Here are the rules governing WORD.  Initial whitespace is
258. %  discarded.  The word is the next single character, unless that
259. %  character is one of the following:

260. %

261. %   A-Z or a-z:  [A-Z,a-z][A-Z,a-z,0-9]*\$?
262. %   0-9:         [0-9]+
263. %   ":           "[^"]*"
264. %   <,=,>:       [<=>][<=>]?  (one or two; not the same if two)

265. %

266. %   Note that the string literal ignores spaces but may be abnormally
267. %   terminated by an end-of-line.  (I wasn't sure how to express that
268. %   as a regexp).

269. %

270. %

271. \newif\ifscan  %  shall we continue scanning?

272. %

273. \def\scan{\def\word{}\futurelet\q\scanFirst}

274. %

275. \def\scanFirst{%  Checks the first character to determine type.
276. \let\next\scanIter
277. \expandafter\if\spc\noexpand\q         %  Space -- ignore it
278. \let\next\scanSpace\else
279. \if\eol\noexpand\q                     %  End of line -- no word here
280. \let\next\scanEnd\else
281. \ifcat A\noexpand\q                    %  Then we have an identifier
282. \let\scanTest\scanIdentifier\else
283. \expandafter\if\digit\q
284. \let\scanTest\scanNumericConstant\else
285. \if"\noexpand\q
286. \let\scanTest\scanStringConstant\else
287. \expandafter\if\relationP\q
288. \let\scanTest\scanRelation
289. \else
290. \let\scanTest\scanfalse
291. \fi\fi\fi\fi\fi\fi\next}

292. %

293. \def\scanIter#1{\expandafter\def\expandafter\word\expandafter{\word #1}
294. \futurelet\q\scanContinueP}
295. \def\scanContinueP{\scanTest\ifscan\let\next\scanIter
296. \else\let\next\scanEnd\fi\next}

297. %

298. \def\scanSpace#1{\scan}% If the first char is a space, gobble it and try again.
299. \def\scanIdentifier{\ifcat A\noexpand\q\scantrue\else
300. \expandafter\if\digit\q\scantrue
301. \else\if$\noexpand\q\scantrue
302. \expandafter\def\expandafter\word\expandafter{\expandafter$\word}
303. \let\scanTest\scanfalse\else
304. \scanfalse\fi\fi\fi}
305. \def\scanEndString{\scanfalse}
306. \def\scanNumericConstant{\expandafter\if\digit\q\scantrue\else\scanfalse\fi}
307. \def\scanStringConstant{\scantrue\if"\q\let\scanTest\scanfalse\fi}
308. \def\scanRelation{\if<\q\scantrue\else\if>\q\scantrue\else\if=\q\scantrue
309. \else\scanfalse\fi\fi\fi}

310. %

311. \def\scanEnd#1{\relax\diw{SCANNED:\word:}
312. \afterscan #1}% dumps trailing spaces.
313. \def\after{\let\afterscan}%

314. %

315. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

316. %

317. %  Type Tests (Predicates for type determination)

318. %

319. \def\relationP#1{tf}  % for now, only single-char relations

320. \def\identifierP#1{\expandafter\identifierTest #1\\}
321. \def\identifierTest#1#2\\{\ifcat A#1\itstrue\else\itsfalse\fi}
322. \def\stringvarP#1{\expandafter\stringvarTest #1\\}
323. \def\stringvarTest#1#2\\{\if$#1\itstrue\else\itsfalse\fi}
324. \def\stringP#1{\expandafter\stringTest #1\\}
325. \def\stringTest#1#2\\{\if #1"\itstrue\else\itsfalse\fi}
326. \def\numberP#1{\expandafter\numberTest #1\\}
327. \def\numberTest#1#2\\{\expandafter\if\digit #1\itstrue\else\itsfalse\fi}
328. \def\macroP#1{\expandafter\macroTest #1\\}
329. \def\macroTest#1#2\\{\expandafter\ifx #1\relax\itstrue\else\itsfalse\fi}

330. %

331. %  \digit tests its single-token argument and returns tt if true,
332. %  tf otherwise.

333. %

334. %

335. \def\digit#1{%
336. \if0\noexpand#1\itstrue\else
337. \if1\noexpand#1\itstrue\else
338. \if2\noexpand#1\itstrue\else
339. \if3\noexpand#1\itstrue\else
340. \if4\noexpand#1\itstrue\else
341. \if5\noexpand#1\itstrue\else
342. \if6\noexpand#1\itstrue\else
343. \if7\noexpand#1\itstrue\else
344. \if8\noexpand#1\itstrue\else
345. \if9\noexpand#1\itstrue\else\itsfalse
346. \fi\fi\fi\fi\fi\fi\fi\fi\fi\fi
347. % 9  8  7  6  5  4  3  2  1  0

348. }

349. %

350. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

351. %

352. %  User Utilities.  These are the commands that are called by the
353. %  user.  We could really use a better section name.  :-)

354. %

355. %  List (one line or all lines, for now)

356. %

357. \def\Clist{\after\listmain\scan}
358. \def\listmain{\isnull{\word}\ifresult\let\next\listalllines
359. \else\let\next\listoneline\fi\next}
360. \def\listline{\iw{\word\spc\csname/\word\endcsname}}
361. \def\listalllines{\gotofirstline\foreachline{\listline}\endeval}
362. \def\listoneline{\listline\endeval}

363. %

364. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

365. %

366. %  Different degrees of ``stop execution''

367. %

368. \def\Csystem{\end}       % exits to the system
369. \def\Cexit{}% \endflageol} % exits to TeX
370. \flageol%
371. \def\Cstop#1
372. {\iw{Stopped in \lineno.}\cleanstop}%
373. \def\cleanstop{\diw{CLEANSTOP}\let\endeval\enduserline\endeval
374. }\endflageol

375. %

376. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

377. %

378. %  The command ``rem'' introduces a remark

379. %

380. \def\Crem{\endeval}%

381. %

382. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

383. %

384. %  The ``let'' command allows variable assignments

385. %

386. \def\Clet{\after\letgetequals\scan}
387. \def\letgetequals{\after\letgetvalue\mandatory{=}}
388. \def\letgetvalue{\after\letdoit\expression}
389. \def\letdoit{\expandafter\edef\csname V\word\endcsname{\value}%
390. \endeval}

391. %

392. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

393. %

394. %  The ``print'' command takes a [list of] expression[s] and displays
395. %  it [them].

396. %

397. \def\Cprint{\after\printit\expression}
398. \def\printit{\iw{\value}\endeval}

399. %

400. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

401. %

402. %  The ``if'' command takes an expression, the word ``then,'' and
403. %  another command.  If the expression is non-zero, the command is
404. %  executed; otherwise it is ignored.

405. %

406. \def\Cif{\after\getift\expression}
407. \def\Cthen{\errmessage{Syntax error: THEN without IF}}
408. \def\getift{\after\getifh\mandatory t}
409. \def\getifh{\after\getife\mandatory h}
410. \def\getife{\after\getifn\mandatory e}
411. \def\getifn{\after\consequent\mandatory n}
412. \def\consequent{\ifnum\value=0\let\next=\endeval\else\let\next=\evalconsq\fi
413. \next}
414. \def\evalconsq{\after\evalline\scan}

415. %

416. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

417. %

418. %  Functions

419. %

420. %  Functions may read the counter \mathParams to find out the number
421. %  of the top parameter.  Parameters are in Pa Pb Pc etc.

422. %

423. \def\return{\expandafter\def\expandafter\mathAcc\expandafter}

424. %

425. \def\Finc{\matha=\Pa \advance\matha by 1
426. \return{\number\matha}}

427. %

428. \def\Fmin{\ifnum\Pa<\Pb\return{\Pa}\else\return{\Pb}\fi}

429. %

430. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

431. %

432. %  Program execution control

433. %

434. \def\Crun{\let\endeval\endincrline\def\lineno{0}\endeval}
435. \def\Cgoto{\let\endeval\endgotoline\after\gotomain\scan}
436. \def\gotomain{\edef\lineno{\word}\endeval}

437. %

438. \flageol%
439. \def\execline{%\message{Executing line \lineno...}%
440. \edef\theline{\csname/\lineno\endcsname}%
441. %\message{THE LINE\theline}%
442. \let\endeval\endincrline\after\evalline\expandafter\scan\theline
443. }\endflageol

444. %

445. %  Different varieties of what to do at the end of a command:

446. %

447. %  get new line from user  (enduserline)

448. %  get next line in order  (endincrline)
449. %  get line in \lineno     (endgotoline)
450. %  keep parsing current line (endcolonline tbi)

451. %

452. \flageol%
453. \def\endincrline#1
454. {\diw{ENDINCRLINE}\edef\lineno{\csname L\lineno\endcsname}\execnextline}

455. %

456. \def\endgotoline#1
457. {\diw{ENDGOTOLINE}\let\endeval\execincrline\execnextline}%

458. %

459. \def\execnextline{\diw{Ready to execute line \lineno...}%
460. \ifnum\lineno<99999\let\next\execline\else\let\next\cleanstop\fi\next}%

461. %

462. \def\enduserline #1
463. {\diw{ENDUSERLINE}\parseline}\endflageol

464. %

465. \let\endeval\enduserline

466. %

467. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

468. %

469. %   Start your engines!

470. %

471. \iw{This is BaSiX, v0.3, emphasis on the SICK!  by amgreene@mit.edu}
472. \flageol
473. \catcode32=12
474. \endeval