Ethan Dicks wrote: > > On Wed, Sep 30, 2026 at 11:10 AM Paul Koning via cctalk > <[email protected]> wrote: >> > On Sep 30, 2026, at 9:30 AM, ben via cctalk <[email protected]> wrote: >> > >> > On 2026-09-29 5:57 p.m., Jim Davis via cctalk wrote: >> >> And all our daily driver languages look like Algol or FORTRAN, >> >> excepting Forth and APL, both written by mad scientists, >> > >> > And token parsing still is from the 1950's for many languages >> > where blank spaces are ignored and a end of line/card or semicolon >> > is needed to separate id's. >> >> Huh? The only language I can think of that ignores whitespace is TECO. >> Just about everything else treats whitespace as a token separator. > > Microsoft 8-bit BASIC tokenizes when you enter a new line of BASIC but > whitespace is not required between tokens. It can get confused in very > specific edge cases but mostly, it works. When executing tokenized > code, outside of quotes, the actual next-char fetch routine in low mem > (CHRGET) explicitly skips whitespace, so the token evaluator never > sees it. > > The stream of tokens and untokenized args presented to the BASIC > statement evaluator sees exactly the same stream from the following > two lines... > > 10FORX=1TO100:PRINTX:NEXTX > > and > > 10 FOR X = 1 TO 100 : PRINT X : NEXT X > > Those spaces are stored in RAM but CHRGET skips them on each fetch. > > -ethan
My recollection is that BBC Basic (on the BBC Micro) similarly ignores space characters but not tabs or other characters that might traditionally be regarded as whitespace. I don't recall any edge cases that cause it to be confused but IIRC pseudovariable tokens (like TIME for example) need a space or other separator character after them to avoid difficulties accessing actual variables that begin with the names of pseudovariables (such as TIMER for example). I've tried the above two lines in BBC Basic in my BBC Micro emulator and they both behave the same. One slightly interesting anomaly in BBC Basic is that the REPEAT token does not even require a colon following it. I don't know if this is by accident or by design but the following works: 10X=1:REPEATPRINTX:X=X+1:UNTILX>100 Regards, Peter Coghlan.
