472  PART 53: EXTENSIONS  TEX82 §1340

1340. Extensions. The program above includes a bunch of “hooks” that allow further capabilities to be added without upsetting TEX’s basic structure. Most of these hooks are concerned with “whatsit” nodes, which are intended to be used for special purposes; whenever a new extension to TEX involves a new kind of whatsit node, a corresponding change needs to be made to the routines below that deal with such nodes, but it will usually be unnecessary to make many changes to the other parts of this program.

In order to demonstrate how extensions can be made, we shall treat ‘  ’, ‘  ’, ‘  ’,
‘  ’, ‘  ’, and ‘  ’ as if they were extensions. These commands are actually

primitives of TEX, and they should appear in all implementations of the system; but let’s try to imagine that they aren’t. Then the program below illustrates how a person could add them.

Sometimes, of course, an extension will require changes to TEX itself; no system of hooks could be complete enough for all conceivable extensions. The features associated with ‘ ’ are almost all confined to the following paragraphs, but there are small parts of the print ln and print char procedures that were introduced specifically to characters. Furthermore one of the token lists recognized by the scanner is a write text ; and there are a few other miscellaneous places where we have already provided for some aspect of . The goal of a TEX extender should be to minimize alterations to the standard parts of the program, and to avoid them completely if possible. He or she should also be quite sure that there’s no easy way to accomplish the desired goals with the standard features that TEX already has. “Think thrice before extending,” because that may save a lot of work, and it will also keep incompatible extensions of TEX from proliferating.

1341. First let’s consider the format of whatsit nodes that are used to represent the data associated with and its relatives. Recall that a whatsit has type = whatsit node, and the subtype is supposed to distinguish different kinds of whatsits. Each node occupies two or more words; the exact number is immaterial, as long as it is readily determined from the subtype or other data.

We shall introduce five subtype values here, corresponding to the control sequences  ,  ,
,  , and  . The second word of I/O whatsits has a write stream field that

identifies the write-stream number (0 to 15, or 16 for out-of-range and positive, or 17 for out-of-range and negative). In the case of and , there is also a field that points to the reference count of a token list that should be sent. In the case of , we need three words and three auxiliary subfields to hold the string numbers for name, area, and extension.

define write node size = 2 { number of words in a write/whatsit node }

define open node size = 3 { number of words in an open/whatsit node }

define open node = 0 { subtype in whatsits that represent files to }

define write node = 1 { subtype in whatsits that represent things to }

define close node = 2 { subtype in whatsits that represent streams to }

define special node = 3 { subtype in whatsits that represent things }

define language node = 4 { subtype in whatsits that change the current language }

define what lang (#) ≡ link (# + 1) { language number, in the range 0 . . 255 }

define what lhm(#) ≡ type(# + 1) { minimum left fragment, in the range 1 . . 63 }

define what rhm(#) ≡ subtype(# + 1) { minimum right fragment, in the range 1 . . 63 }

define write tokens (#) ≡ link (# + 1) { reference count of token list to write }

define write stream(#) ≡ info(# + 1) { stream number (0 to 17) }

define open name(#) ≡ link (# + 1) { string number of file name to open }

define open area(#) ≡ info(# + 2) { string number of file area for open name }

§1342  TEX82  PART 53: EXTENSIONS 473

define open ext (#) ≡ link (# + 2) { string number of file extension for open name } 1342. The sixteen possible streams are represented by the write file array. The jth file is open if and only if write open[j] = true. The last two streams are special; write open[16] represents a stream number greater than 15, while write open[17] represents a negative stream number, and both of these variables are always false. ⟨ Global variables 13 ⟩ +≡ write file: array [0 . . 15] of alpha file; write open: array [0 . . 17] of boolean;

1343. ⟨ Set initial values of key variables 21 ⟩ +≡ for k ← 0 to 17 do write open[k] ← false;

1344. Extensions might introduce new command codes; but it’s best to use extension with a modifier, whenever possible, so that main control stays the same.

define immediate code = 4 { command modifier for } define set language code = 5 { command modifier for }

⟨ Put each of TEX’s primitives into the hash table 226 ⟩ +≡ primitive( , extension, open node); primitive( , extension, write node); write loc ← cur val ; primitive( , extension, close node); primitive( , extension, special node); primitive( , extension, immediate code); primitive( , extension, set language code);

1345. The variable write loc just introduced is used to provide an appropriate error message in case of “runaway” write texts. ⟨ Global variables 13 ⟩ +≡ write loc: pointer ; { eqtb address of }

1346. ⟨ Cases of print cmd chr for symbolic printing of primitives 227 ⟩ +≡ extension: case chr code of

open node: print esc( );

write node: print esc( );

close node: print esc( );

special node: print esc( );

immediate code: print esc( );

set language code: print esc( );

othercases print ( ␣ )

endcases;

1347. When an extension command occurs in main control , in any mode, the do extension routine is called. ⟨ Cases of main control that are for extensions to TEX 1347 ⟩ ≡ any mode(extension): do extension;

This code is used in section 1045.

1348. ⟨ Declare action procedures for use by main control 1043 ⟩ +≡ ⟨ Declare procedures needed in do extension 1349 ⟩ procedure do extension;

var i, j, k: integer ; { all-purpose integers }

p, q, r: pointer ; { all-purpose pointers }

begin case cur chr of

open node: ⟨ Implement 1351 ⟩; write node: ⟨ Implement 1352 ⟩; close node: ⟨ Implement 1353 ⟩; special node: ⟨ Implement 1354 ⟩; immediate code: ⟨ Implement 1375 ⟩; set language code: ⟨ Implement 1377 ⟩; othercases confusion( ) endcases; end;

1349. Here is a subroutine that creates a whatsit node having a given subtype and a given number of words. It initializes only the first word of the whatsit, and appends it to the current list. ⟨ Declare procedures needed in do extension 1349 ⟩ ≡ procedure new whatsit (s : small number ; w : small number );

var p: pointer ; { the new node } begin p ← get node(w); type(p) ← whatsit node; subtype(p) ← s; link (tail ) ← p; tail ← p; end;

See also section 1350. This code is used in section 1348.

1350. The next subroutine uses cur chr to decide what sort of whatsit is involved, and also inserts a write stream number. ⟨ Declare procedures needed in do extension 1349 ⟩ +≡ procedure new write whatsit (w : small number );

begin new whatsit (cur chr , w); if w ≠ write node size then scan four bit int else begin scan int ;

if cur val < 0 then cur val ← 17

else if cur val > 15 then cur val ← 16; end;

write stream(tail ) ← cur val ; end;

1351. ⟨ Implement 1351 ⟩ ≡ begin new write whatsit (open node size); scan optional equals ; scan file name; open name(tail ) ← cur name; open area(tail ) ← cur area; open ext (tail ) ← cur ext ; end

This code is used in section 1348.

1352. When ‘  ’ appears, we scan the token list ‘  ’ without expanding its macros; the

macros will be expanded later when this token list is rescanned. ⟨ Implement 1352 ⟩ ≡

begin k ← cur cs ; new write whatsit (write node size); cur cs ← k; p ← scan toks (false, false); write tokens (tail ) ← def ref ; end

This code is used in section 1348.

1353. ⟨ Implement 1353 ⟩ ≡ begin new write whatsit (write node size); write tokens (tail ) ← null ; end

This code is used in section 1348.

1354. When ‘  ’ appears, we expand the macros in the token list as in  and  .

⟨ Implement 1354 ⟩ ≡ begin new whatsit (special node, write node size); write stream(tail ) ← null ; p ← scan toks (false, true); write tokens (tail ) ← def ref ; end

This code is used in section 1348.

1355. Each new type of node that appears in our data structure must be capable of being displayed, copied, destroyed, and so on. The routines that we need for write-oriented whatsits are somewhat like those for mark nodes; other extensions might, of course, involve more subtlety here. ⟨ Basic printing procedures 57 ⟩ +≡ procedure print write whatsit (s : str number ; p : pointer );

begin print esc(s); if write stream(p) < 16 then print int (write stream(p)) else if write stream(p) = 16 then print char ( )

else print char ( ); end;

1356. ⟨ Display the whatsit node p 1356 ⟩ ≡ case subtype(p) of open node: begin print write whatsit ( , p); print char ( );

print file name(open name(p), open area(p), open ext (p));

end; write node: begin print write whatsit ( , p); print mark (write tokens (p));

end; close node: print write whatsit ( , p); special node: begin print esc( ); print mark (write tokens (p));

end; language node: begin print esc( ); print int (what lang (p)); print ( ␣ ␣ );

print int (what lhm(p)); print char ( ); print int (what rhm(p)); print char ( ); end;

othercases print ( ) endcases

This code is used in section 183.

1357. ⟨ Make a partial copy of the whatsit node p and make r point to it; set words to the number of

initial words not yet copied 1357 ⟩ ≡ case subtype(p) of open node: begin r ← get node(open node size); words ← open node size;

end; write node, special node: begin r ← get node(write node size); add token ref (write tokens (p));

words ← write node size; end;

close node, language node: begin r ← get node(small node size); words ← small node size;

end; othercases confusion( ) endcases

This code is used in section 206.

1358. ⟨ Wipe out the whatsit node p and goto done 1358 ⟩ ≡ begin case subtype(p) of open node: free node(p, open node size); write node, special node: begin delete token ref (write tokens (p)); free node(p, write node size);

goto done;

end; close node, language node: free node(p, small node size); othercases confusion( ) endcases; goto done; end

This code is used in section 202.

1359. ⟨ Incorporate a whatsit node into a vbox 1359 ⟩ ≡

do nothing

This code is used in section 669.

1360. ⟨ Incorporate a whatsit node into an hbox 1360 ⟩ ≡

do nothing

This code is used in section 651.

1361. ⟨ Let d be the width of the whatsit p 1361 ⟩ ≡

d ← 0

This code is used in section 1147.

1362. define adv past (#) ≡ if subtype(#) = language node then

begin cur lang ← what lang (#); l hyf ← what lhm(#); r hyf ← what rhm(#); end

⟨ Advance past a whatsit node in the line break loop 1362 ⟩ ≡ adv past (cur p)

This code is used in section 866.

1363. ⟨ Advance past a whatsit node in the pre-hyphenation loop 1363 ⟩ ≡ adv past (s)

This code is used in section 896.

1364. ⟨ Prepare to move whatsit p to the current page, then goto contribute 1364 ⟩ ≡

goto contribute

This code is used in section 1000.

1365. ⟨ Process whatsit p in vert break loop, goto not found 1365 ⟩ ≡

goto not found

This code is used in section 973.

1366. ⟨ Output the whatsit node p in a vlist 1366 ⟩ ≡

out what (p)

This code is used in section 631.

1367. ⟨ Output the whatsit node p in an hlist 1367 ⟩ ≡

out what (p)

This code is used in section 622.

1368. After all this preliminary shuffling, we come finally to the routines that actually send out the requested data. Let’s do first (it’s easier). ⟨ Declare procedures needed in hlist out , vlist out 1368 ⟩ ≡ procedure special out (p : pointer );

var old setting : 0 . . max selector ; { holds print selector }

k: pool pointer ; { index into str pool } begin synch h; synch v ; old setting ← selector ; selector ← new string ; show token list (link (write tokens (p)), null , pool size − pool ptr ); selector ← old setting ; str room(1); if cur length < 256 then

begin dvi out (xxx1 ); dvi out (cur length); end

else begin dvi out (xxx4 ); dvi four (cur length);

end; for k ← str start [str ptr ] to pool ptr − 1 do dvi out (so(str pool [k])); pool ptr ← str start [str ptr ]; { erase the string } end;

See also sections 1370 and 1373. This code is used in section 619.

1369. To write a token list, we must run it through TEX’s scanner, expanding macros and and , etc. This might cause runaways, if a delimited macro parameter isn’t matched, and runaways would be extremely confusing since we are calling on TEX’s scanner in the middle of a command. Therefore we will put a dummy control sequence as a “stopper,” right after the token list. This control sequence is artificially defined to be . ⟨ Initialize table entries (done by only) 164 ⟩ +≡

text (end write) ← ; eq level (end write) ← level one; eq type(end write) ← outer call ; equiv (end write) ← null ; 1370. ⟨ Declare procedures needed in hlist out , vlist out 1368 ⟩ +≡ procedure write out (p : pointer );

var old setting : 0 . . max selector ; { holds print selector } old mode: integer ; { saved mode } j: small number ; { write stream number } q, r: pointer ; { temporary variables for list manipulation }

begin ⟨ Expand macros in the token list and make link (def ref ) point to the result 1371 ⟩; old setting ← selector ; j ← write stream(p); if write open[j] then selector ← j else begin { write to the terminal if file isn’t open }

if (j = 17) ∧ (selector = term and log ) then selector ← log only ; print nl ( ); end;

token show (def ref ); print ln; flush list (def ref ); selector ← old setting ; end;

1371. The final line of this routine is slightly subtle; at least, the author didn’t think about it until getting burnt! There is a used-up token list on the stack, namely the one that contained end write token. (We insert this artificial ‘ ’ to prevent runaways, as explained above.) If it were not removed, and if there were numerous writes on a single page, the stack would overflow.

define end write token ≡ cs token flag + end write ⟨ Expand macros in the token list and make link (def ref ) point to the result 1371 ⟩ ≡

q ← get avail ; info(q) ← right brace token + ; r ← get avail ; link (q) ← r; info(r) ← end write token; ins list (q); begin token list (write tokens (p), write text ); q ← get avail ; info(q) ← left brace token + ; ins list (q);

{ now we’re ready to scan ‘ ⟨ token list ⟩ ’ } old mode ← mode; mode ← 0; { disable , , , } cur cs ← write loc; q ← scan toks (false, true); { expand macros, etc. } get token; if cur tok ≠ end write token then ⟨ Recover from an unbalanced write command 1372 ⟩; mode ← old mode; end token list { conserve stack space }

This code is used in section 1370.

1372. ⟨ Recover from an unbalanced write command 1372 ⟩ ≡

begin print err (  ␣     ␣  );

help2 ( ␣    ␣    ␣     ´ ␣ ␣      ␣    ␣     ␣    ␣ ´ ␣    ␣ ´  )

( ␣ ´ ␣ ␣ ␣ ␣ ␣ ␣ ); error ; repeat get token; until cur tok = end write token; end

This code is used in section 1371.

1373. The out what procedure takes care of outputting whatsit nodes for vlist out and hlist out. ⟨ Declare procedures needed in hlist out , vlist out 1368 ⟩ +≡ procedure out what (p : pointer );

var j: small number ; { write stream number } begin case subtype(p) of open node, write node, close node: ⟨ Do some work that has been queued up for 1374 ⟩; special node: special out (p); language node: do nothing ; othercases confusion( ) endcases; end;

1374. We don’t implement inside of leaders. (The reason is that the number of times a leader box appears might be different in different implementations, due to machine-dependent rounding in the glue calculations.) ⟨ Do some work that has been queued up for 1374 ⟩ ≡

if ¬doing leaders then begin j ← write stream(p); if subtype(p) = write node then write out (p) else begin if write open[j] then a close(write file[j]);

if subtype(p) = close node then write open[j] ← false else if j < 16 then

begin cur name ← open name(p); cur area ← open area(p); cur ext ← open ext (p); if cur ext = then cur ext ← ; pack cur name; while ¬a open out (write file[j]) do prompt file name( ␣ ␣ , ); write open[j] ← true; end;

end; end

This code is used in section 1373.

1375. The presence of ‘ ’ causes the do extension procedure to descend to one level of recursion. Nothing happens unless is followed by ‘ ’, ‘ ’, or ‘ ’. ⟨ Implement 1375 ⟩ ≡

begin get x token; if (cur cmd = extension) ∧ (cur chr ≤ close node) then

begin p ← tail ; do extension; { append a whatsit node } out what (tail ); { do the action immediately } flush node list (tail ); tail ← p; link (p) ← null ; end

else back input ; end

This code is used in section 1348.

1376. The extension is somewhat different. We need a subroutine that comes into play when a character of a non-clang language is being appended to the current paragraph. ⟨ Declare action procedures for use by main control 1043 ⟩ +≡ procedure fix language;

var l: ASCII code; { the new current language } begin if language ≤ 0 then l ← 0 else if language > 255 then l ← 0

else l ← language; if l ≠ clang then

begin new whatsit (language node, small node size); what lang (tail ) ← l; clang ← l; what lhm(tail ) ← norm min(left hyphen min); what rhm(tail ) ← norm min(right hyphen min); end;

end;

1377. ⟨ Implement 1377 ⟩ ≡ if abs (mode) ≠ hmode then report illegal case else begin new whatsit (language node, small node size); scan int ;

if cur val ≤ 0 then clang ← 0 else if cur val > 255 then clang ← 0

else clang ← cur val ; what lang (tail ) ← clang ; what lhm(tail ) ← norm min(left hyphen min); what rhm(tail ) ← norm min(right hyphen min); end

This code is used in section 1348.

1378. ⟨ Finish the extensions 1378 ⟩ ≡

for k ← 0 to 15 do

if write open[k] then a close(write file[k])

This code is used in section 1333.