Re: BASIC (Was: Reading HP2000 tapes
On Jul 18, 2018, at 10:21 AM, Paul Berger via cctalk wrote: > I would think that any interpreted BASIC would do this or for that matter any > interpreted language except maybe for APL You'd think so but many BASIC dialects don't fully tokenize. Many leave in all the extra spaces the user types in, such as the Microsoft BASIC on various Commodores. Some BASICs even require the spaces in order to properly parse during tokenization. There's a page here which includes a field for machines which do full tokenization: https://en.wikipedia.org/wiki/List_of_computers_with_on-board_BASIC -- Follow me on twitter: @FozzTexx Check out my blog: http://insentricity.com
Re: BASIC (Was: Reading HP2000 tapes
> On Jul 18, 2018, at 1:21 PM, Paul Berger via cctalk > wrote: > > I would think that any interpreted BASIC would do this or for that matter any > interpreted language except maybe for APL which is pretty much written with > tokens anyway. One other exception I can think of is perl which is stored > as source text. Saving in tokenized form was good for to reasons, it saved > storage space, both in memory and on mass storage and when you loaded the > program it was ready to go. > > Paul... >> > I think it was called a "decompiler" though. Seemed like magic at the > time. > > Googling reveals "You may be remembering the BASIC PLUS > decompiler under RSTS. RSTS BASIC PLUS was interpreted from "push-pop" > code. > The symbol table was available in the compiled file, and the > correspondence > between push-pop operations and BASIC PLUS source was very close, so you > could get back very reasonable code." > > And our previous discussion of it a decade ago: > > https://marc.info/?l=classiccmp=121804804023540=2 I would not say "written with tokens". Basic-PLUS essentially used a stack machine code, easy to generate and pretty efficient. It wasn't designed to be reversible, but since the symbol table was saved as well (had to be, to allow for incremental editing and interactive debugging) you could reverse pretty easily. This sort of thing has a long history. UCSD Pascal used something similar, which it called "P-code". The TUTOR language of the U of Illinois PLATO system did as well, except for expressions which were compiled into actual machine code. That sort of mixed encoding was used a decade earlier in the first ALGOL compiler, by Dijkstra and Zonneveld, 1961, for the EL-X1. And yes, it's still done a lot, I believe Python is a good example. A somewhat different approach is found in RT-11 BASIC, a somewhat simpler language than BASIC-PLUS and an unrelated implementation. That one does convert the text into tokens, it doesn't generate a stack language transformation as B+ did. And the token encoding is explicitly designed to be reversible: when you use the LIST command the token stream is converted back to source text. That means, for example, that comments are included in the token stream (unlike B+). paul
Re: BASIC (Was: Reading HP2000 tapes
I would think that any interpreted BASIC would do this or for that matter any interpreted language except maybe for APL which is pretty much written with tokens anyway. One other exception I can think of is perl which is stored as source text. Saving in tokenized form was good for to reasons, it saved storage space, both in memory and on mass storage and when you loaded the program it was ready to go. Paul. On 2018-07-18 2:00 PM, Curious Marc via cctalk wrote: And so does the HP 85. Marc On Jul 17, 2018, at 1:50 PM, Brent Hilpert via cctalk wrote: The HP9830 (1972) with it's ROM'ed BASIC works this way. LIST produces a 'cleaned up' version of the source code. On 2018-Jul-17, at 1:21 PM, Guy Sotomayor Jr via cctalk wrote: I should also mention that for the IBM S/23, once the BASIC program is entered, the original source is discarded and only the tokenized code remains (comments are retained as-is). The LIST command runs a de-tokenizer and reconstructs the original source (well close to it anyway). TTFN - Guy On Jul 17, 2018, at 12:33 PM, John Foust via cctalk wrote: At 03:53 PM 7/14/2018, Fred Cisin via cctalk wrote: On Sat, 14 Jul 2018, Ed Sharpe via cctalk wrote: isn't the basic programs also stored in tokinized forms!?!? Yes. And the tokens are not the same between different brand implementations, or even between different versions, such as MBASIC 4 and MBASIC 5. http://fileformats.archiveteam.org/wiki/Tokenized_BASIC I remember a detokenizer for RSTS BASIC-PLUS that's not on that list. I think it was called a "decompiler" though. Seemed like magic at the time. Googling reveals "You may be remembering the BASIC PLUS decompiler under RSTS. RSTS BASIC PLUS was interpreted from "push-pop" code. The symbol table was available in the compiled file, and the correspondence between push-pop operations and BASIC PLUS source was very close, so you could get back very reasonable code." And our previous discussion of it a decade ago: https://marc.info/?l=classiccmp=121804804023540=2 - John
Re: BASIC (Was: Reading HP2000 tapes
And so does the HP 85. Marc > On Jul 17, 2018, at 1:50 PM, Brent Hilpert via cctalk > wrote: > > The HP9830 (1972) with it's ROM'ed BASIC works this way. > LIST produces a 'cleaned up' version of the source code. > > > >> On 2018-Jul-17, at 1:21 PM, Guy Sotomayor Jr via cctalk wrote: >> >> I should also mention that for the IBM S/23, once the BASIC program is >> entered, the original >> source is discarded and only the tokenized code remains (comments are >> retained as-is). The >> LIST command runs a de-tokenizer and reconstructs the original source (well >> close to it anyway). >> >> TTFN - Guy >> >>> On Jul 17, 2018, at 12:33 PM, John Foust via cctalk >>> wrote: >>> >>> At 03:53 PM 7/14/2018, Fred Cisin via cctalk wrote: > On Sat, 14 Jul 2018, Ed Sharpe via cctalk wrote: > isn't the basic programs also stored in tokinized forms!?!? Yes. And the tokens are not the same between different brand implementations, or even between different versions, such as MBASIC 4 and MBASIC 5. http://fileformats.archiveteam.org/wiki/Tokenized_BASIC >>> >>> I remember a detokenizer for RSTS BASIC-PLUS that's not on that list. >>> >>> I think it was called a "decompiler" though. Seemed like magic at the time. >>> >>> Googling reveals "You may be remembering the BASIC PLUS >>> decompiler under RSTS. RSTS BASIC PLUS was interpreted from "push-pop" >>> code. >>> The symbol table was available in the compiled file, and the correspondence >>> between push-pop operations and BASIC PLUS source was very close, so you >>> could get back very reasonable code." >>> >>> And our previous discussion of it a decade ago: >>> >>> https://marc.info/?l=classiccmp=121804804023540=2 >>> >>> - John >>> >> >
Re: BASIC (Was: Reading HP2000 tapes
The HP9830 (1972) with it's ROM'ed BASIC works this way. LIST produces a 'cleaned up' version of the source code. On 2018-Jul-17, at 1:21 PM, Guy Sotomayor Jr via cctalk wrote: > I should also mention that for the IBM S/23, once the BASIC program is > entered, the original > source is discarded and only the tokenized code remains (comments are > retained as-is). The > LIST command runs a de-tokenizer and reconstructs the original source (well > close to it anyway). > > TTFN - Guy > >> On Jul 17, 2018, at 12:33 PM, John Foust via cctalk >> wrote: >> >> At 03:53 PM 7/14/2018, Fred Cisin via cctalk wrote: >>> On Sat, 14 Jul 2018, Ed Sharpe via cctalk wrote: isn't the basic programs also stored in tokinized forms!?!? >>> >>> Yes. >>> And the tokens are not the same between different brand implementations, or >>> even between different versions, such as MBASIC 4 and MBASIC 5. >>> http://fileformats.archiveteam.org/wiki/Tokenized_BASIC >> >> I remember a detokenizer for RSTS BASIC-PLUS that's not on that list. >> >> I think it was called a "decompiler" though. Seemed like magic at the time. >> >> Googling reveals "You may be remembering the BASIC PLUS >> decompiler under RSTS. RSTS BASIC PLUS was interpreted from "push-pop" code. >> The symbol table was available in the compiled file, and the correspondence >> between push-pop operations and BASIC PLUS source was very close, so you >> could get back very reasonable code." >> >> And our previous discussion of it a decade ago: >> >> https://marc.info/?l=classiccmp=121804804023540=2 >> >> - John >> >
Re: BASIC (Was: Reading HP2000 tapes
I should also mention that for the IBM S/23, once the BASIC program is entered, the original source is discarded and only the tokenized code remains (comments are retained as-is). The LIST command runs a de-tokenizer and reconstructs the original source (well close to it anyway). TTFN - Guy > On Jul 17, 2018, at 12:33 PM, John Foust via cctalk > wrote: > > At 03:53 PM 7/14/2018, Fred Cisin via cctalk wrote: >> On Sat, 14 Jul 2018, Ed Sharpe via cctalk wrote: >>> isn't the basic programs also stored in tokinized forms!?!? >> >> Yes. >> And the tokens are not the same between different brand implementations, or >> even between different versions, such as MBASIC 4 and MBASIC 5. >> http://fileformats.archiveteam.org/wiki/Tokenized_BASIC > > I remember a detokenizer for RSTS BASIC-PLUS that's not on that list. > > I think it was called a "decompiler" though. Seemed like magic at the time. > > Googling reveals "You may be remembering the BASIC PLUS > decompiler under RSTS. RSTS BASIC PLUS was interpreted from "push-pop" code. > The symbol table was available in the compiled file, and the correspondence > between push-pop operations and BASIC PLUS source was very close, so you > could get back very reasonable code." > > And our previous discussion of it a decade ago: > > https://marc.info/?l=classiccmp=121804804023540=2 > > - John >
Re: BASIC (Was: Reading HP2000 tapes
At 03:53 PM 7/14/2018, Fred Cisin via cctalk wrote: >On Sat, 14 Jul 2018, Ed Sharpe via cctalk wrote: >>isn't the basic programs also stored in tokinized forms!?!? > >Yes. >And the tokens are not the same between different brand implementations, or >even between different versions, such as MBASIC 4 and MBASIC 5. >http://fileformats.archiveteam.org/wiki/Tokenized_BASIC I remember a detokenizer for RSTS BASIC-PLUS that's not on that list. I think it was called a "decompiler" though. Seemed like magic at the time. Googling reveals "You may be remembering the BASIC PLUS decompiler under RSTS. RSTS BASIC PLUS was interpreted from "push-pop" code. The symbol table was available in the compiled file, and the correspondence between push-pop operations and BASIC PLUS source was very close, so you could get back very reasonable code." And our previous discussion of it a decade ago: https://marc.info/?l=classiccmp=121804804023540=2 - John
Re: BASIC (Was: Reading HP2000 tapes
On 2018-07-14 5:10 PM, Fred Cisin via cctalk wrote: > Apparently Eric Smith has already done so. > http://www.brouhaha.com/~eric/software/tsbutils/ > > > You can still send me back to the sixties. > Hey, form an orderly queue, please!
Re: BASIC (Was: Reading HP2000 tapes
Apparently Eric Smith has already done so. http://www.brouhaha.com/~eric/software/tsbutils/ You can still send me back to the sixties.
