Re: BASIC (Was: Reading HP2000 tapes
On Jul 18, 2018, at 10:21 AM, Paul Berger via cctalk wrote: > I would think that any interpreted BASIC would do this or for that matter any > interpreted language except maybe for APL You'd think so but many BASIC dialects don't fully tokenize. Many leave in all the extra spaces the user types in, such as the Microsoft BASIC on various Commodores. Some BASICs even require the spaces in order to properly parse during tokenization. There's a page here which includes a field for machines which do full tokenization: https://en.wikipedia.org/wiki/List_of_computers_with_on-board_BASIC -- Follow me on twitter: @FozzTexx Check out my blog: http://insentricity.com
Re: BASIC (Was: Reading HP2000 tapes
> On Jul 18, 2018, at 1:21 PM, Paul Berger via cctalk > wrote: > > I would think that any interpreted BASIC would do this or for that matter any > interpreted language except maybe for APL which is pretty much written with > tokens anyway. One other exception I can think of is perl which is stored > as source text. Saving in tokenized form was good for to reasons, it saved > storage space, both in memory and on mass storage and when you loaded the > program it was ready to go. > > Paul... >> > I think it was called a "decompiler" though. Seemed like magic at the > time. > > Googling reveals "You may be remembering the BASIC PLUS > decompiler under RSTS. RSTS BASIC PLUS was interpreted from "push-pop" > code. > The symbol table was available in the compiled file, and the > correspondence > between push-pop operations and BASIC PLUS source was very close, so you > could get back very reasonable code." > > And our previous discussion of it a decade ago: > > https://marc.info/?l=classiccmp=121804804023540=2 I would not say "written with tokens". Basic-PLUS essentially used a stack machine code, easy to generate and pretty efficient. It wasn't designed to be reversible, but since the symbol table was saved as well (had to be, to allow for incremental editing and interactive debugging) you could reverse pretty easily. This sort of thing has a long history. UCSD Pascal used something similar, which it called "P-code". The TUTOR language of the U of Illinois PLATO system did as well, except for expressions which were compiled into actual machine code. That sort of mixed encoding was used a decade earlier in the first ALGOL compiler, by Dijkstra and Zonneveld, 1961, for the EL-X1. And yes, it's still done a lot, I believe Python is a good example. A somewhat different approach is found in RT-11 BASIC, a somewhat simpler language than BASIC-PLUS and an unrelated implementation. That one does convert the text into tokens, it doesn't generate a stack language transformation as B+ did. And the token encoding is explicitly designed to be reversible: when you use the LIST command the token stream is converted back to source text. That means, for example, that comments are included in the token stream (unlike B+). paul
Re: BASIC (Was: Reading HP2000 tapes
I would think that any interpreted BASIC would do this or for that matter any interpreted language except maybe for APL which is pretty much written with tokens anyway. One other exception I can think of is perl which is stored as source text. Saving in tokenized form was good for to reasons, it saved storage space, both in memory and on mass storage and when you loaded the program it was ready to go. Paul. On 2018-07-18 2:00 PM, Curious Marc via cctalk wrote: And so does the HP 85. Marc On Jul 17, 2018, at 1:50 PM, Brent Hilpert via cctalk wrote: The HP9830 (1972) with it's ROM'ed BASIC works this way. LIST produces a 'cleaned up' version of the source code. On 2018-Jul-17, at 1:21 PM, Guy Sotomayor Jr via cctalk wrote: I should also mention that for the IBM S/23, once the BASIC program is entered, the original source is discarded and only the tokenized code remains (comments are retained as-is). The LIST command runs a de-tokenizer and reconstructs the original source (well close to it anyway). TTFN - Guy On Jul 17, 2018, at 12:33 PM, John Foust via cctalk wrote: At 03:53 PM 7/14/2018, Fred Cisin via cctalk wrote: On Sat, 14 Jul 2018, Ed Sharpe via cctalk wrote: isn't the basic programs also stored in tokinized forms!?!? Yes. And the tokens are not the same between different brand implementations, or even between different versions, such as MBASIC 4 and MBASIC 5. http://fileformats.archiveteam.org/wiki/Tokenized_BASIC I remember a detokenizer for RSTS BASIC-PLUS that's not on that list. I think it was called a "decompiler" though. Seemed like magic at the time. Googling reveals "You may be remembering the BASIC PLUS decompiler under RSTS. RSTS BASIC PLUS was interpreted from "push-pop" code. The symbol table was available in the compiled file, and the correspondence between push-pop operations and BASIC PLUS source was very close, so you could get back very reasonable code." And our previous discussion of it a decade ago: https://marc.info/?l=classiccmp=121804804023540=2 - John
Re: BASIC (Was: Reading HP2000 tapes
And so does the HP 85. Marc > On Jul 17, 2018, at 1:50 PM, Brent Hilpert via cctalk > wrote: > > The HP9830 (1972) with it's ROM'ed BASIC works this way. > LIST produces a 'cleaned up' version of the source code. > > > >> On 2018-Jul-17, at 1:21 PM, Guy Sotomayor Jr via cctalk wrote: >> >> I should also mention that for the IBM S/23, once the BASIC program is >> entered, the original >> source is discarded and only the tokenized code remains (comments are >> retained as-is). The >> LIST command runs a de-tokenizer and reconstructs the original source (well >> close to it anyway). >> >> TTFN - Guy >> >>> On Jul 17, 2018, at 12:33 PM, John Foust via cctalk >>> wrote: >>> >>> At 03:53 PM 7/14/2018, Fred Cisin via cctalk wrote: > On Sat, 14 Jul 2018, Ed Sharpe via cctalk wrote: > isn't the basic programs also stored in tokinized forms!?!? Yes. And the tokens are not the same between different brand implementations, or even between different versions, such as MBASIC 4 and MBASIC 5. http://fileformats.archiveteam.org/wiki/Tokenized_BASIC >>> >>> I remember a detokenizer for RSTS BASIC-PLUS that's not on that list. >>> >>> I think it was called a "decompiler" though. Seemed like magic at the time. >>> >>> Googling reveals "You may be remembering the BASIC PLUS >>> decompiler under RSTS. RSTS BASIC PLUS was interpreted from "push-pop" >>> code. >>> The symbol table was available in the compiled file, and the correspondence >>> between push-pop operations and BASIC PLUS source was very close, so you >>> could get back very reasonable code." >>> >>> And our previous discussion of it a decade ago: >>> >>> https://marc.info/?l=classiccmp=121804804023540=2 >>> >>> - John >>> >> >
Re: BASIC (Was: Reading HP2000 tapes
The HP9830 (1972) with it's ROM'ed BASIC works this way. LIST produces a 'cleaned up' version of the source code. On 2018-Jul-17, at 1:21 PM, Guy Sotomayor Jr via cctalk wrote: > I should also mention that for the IBM S/23, once the BASIC program is > entered, the original > source is discarded and only the tokenized code remains (comments are > retained as-is). The > LIST command runs a de-tokenizer and reconstructs the original source (well > close to it anyway). > > TTFN - Guy > >> On Jul 17, 2018, at 12:33 PM, John Foust via cctalk >> wrote: >> >> At 03:53 PM 7/14/2018, Fred Cisin via cctalk wrote: >>> On Sat, 14 Jul 2018, Ed Sharpe via cctalk wrote: isn't the basic programs also stored in tokinized forms!?!? >>> >>> Yes. >>> And the tokens are not the same between different brand implementations, or >>> even between different versions, such as MBASIC 4 and MBASIC 5. >>> http://fileformats.archiveteam.org/wiki/Tokenized_BASIC >> >> I remember a detokenizer for RSTS BASIC-PLUS that's not on that list. >> >> I think it was called a "decompiler" though. Seemed like magic at the time. >> >> Googling reveals "You may be remembering the BASIC PLUS >> decompiler under RSTS. RSTS BASIC PLUS was interpreted from "push-pop" code. >> The symbol table was available in the compiled file, and the correspondence >> between push-pop operations and BASIC PLUS source was very close, so you >> could get back very reasonable code." >> >> And our previous discussion of it a decade ago: >> >> https://marc.info/?l=classiccmp=121804804023540=2 >> >> - John >> >
Re: BASIC (Was: Reading HP2000 tapes
I should also mention that for the IBM S/23, once the BASIC program is entered, the original source is discarded and only the tokenized code remains (comments are retained as-is). The LIST command runs a de-tokenizer and reconstructs the original source (well close to it anyway). TTFN - Guy > On Jul 17, 2018, at 12:33 PM, John Foust via cctalk > wrote: > > At 03:53 PM 7/14/2018, Fred Cisin via cctalk wrote: >> On Sat, 14 Jul 2018, Ed Sharpe via cctalk wrote: >>> isn't the basic programs also stored in tokinized forms!?!? >> >> Yes. >> And the tokens are not the same between different brand implementations, or >> even between different versions, such as MBASIC 4 and MBASIC 5. >> http://fileformats.archiveteam.org/wiki/Tokenized_BASIC > > I remember a detokenizer for RSTS BASIC-PLUS that's not on that list. > > I think it was called a "decompiler" though. Seemed like magic at the time. > > Googling reveals "You may be remembering the BASIC PLUS > decompiler under RSTS. RSTS BASIC PLUS was interpreted from "push-pop" code. > The symbol table was available in the compiled file, and the correspondence > between push-pop operations and BASIC PLUS source was very close, so you > could get back very reasonable code." > > And our previous discussion of it a decade ago: > > https://marc.info/?l=classiccmp=121804804023540=2 > > - John >
Re: BASIC (Was: Reading HP2000 tapes
At 03:53 PM 7/14/2018, Fred Cisin via cctalk wrote: >On Sat, 14 Jul 2018, Ed Sharpe via cctalk wrote: >>isn't the basic programs also stored in tokinized forms!?!? > >Yes. >And the tokens are not the same between different brand implementations, or >even between different versions, such as MBASIC 4 and MBASIC 5. >http://fileformats.archiveteam.org/wiki/Tokenized_BASIC I remember a detokenizer for RSTS BASIC-PLUS that's not on that list. I think it was called a "decompiler" though. Seemed like magic at the time. Googling reveals "You may be remembering the BASIC PLUS decompiler under RSTS. RSTS BASIC PLUS was interpreted from "push-pop" code. The symbol table was available in the compiled file, and the correspondence between push-pop operations and BASIC PLUS source was very close, so you could get back very reasonable code." And our previous discussion of it a decade ago: https://marc.info/?l=classiccmp=121804804023540=2 - John
Re: BASIC (Was: Reading HP2000 tapes
On 2018-07-14 5:10 PM, Fred Cisin via cctalk wrote: > Apparently Eric Smith has already done so. > http://www.brouhaha.com/~eric/software/tsbutils/ > > > You can still send me back to the sixties. > Hey, form an orderly queue, please!
Re: BASIC (Was: Reading HP2000 tapes
Apparently Eric Smith has already done so. http://www.brouhaha.com/~eric/software/tsbutils/ You can still send me back to the sixties.
BASIC (Was: Reading HP2000 tapes
On Sat, 14 Jul 2018, Ed Sharpe via cctalk wrote: isn't the basic programs also stored in tokinized forms!?!? Yes. And the tokens are not the same between different brand implementations, or even between different versions, such as MBASIC 4 and MBASIC 5. http://fileformats.archiveteam.org/wiki/Tokenized_BASIC I don't know the token list for it. You MIGHT be able to find that in one of the early manuals for it (1969), possibly as an appendix. newline is 0D,0A Saving a file in UN-tokenized form is done with "CSAVE". (similar to the "SAVE xxx,A in Microsoft BASIC) If writing a conversion program for it is an unsurmountable obstacle, then all that you need to do is to send me back in time to 1960. That will give me almost a decade to get established and get the money together, so that when the machine is first released, I will get hold of one, and write a program in BASIC to detokenize a stored BASIC file, and save it in untokenized form. I will then put a copy of that BASIC utility program onto the tape that you will end up with. To leave enough space on the tape for the other stuff, I will store that program in tokenized form. Optionally, I can set up a portfolio for you then, to fund the time machine. NOTE: I have made a similar open offer to John Titor. So do it quickly to be first! Offer is for a ONE-WAY trip. Round-trip is not acceptable. NOTE: the language BASIC is an acronym for "Beginner's All-purpose Symbolic Instruction Code."ALL CAPS ! The basic programs and utilities that come with the machine that are not written in BASIC are not tokenized. The language was developed by Thomas E. Kurtz and John G. Kemeny at Dartmouth College in 1963. One of their "basic" oremises was "a number is a number", and that people using the language shouldn't have to understand the difference between integer and floating point. They abandoned their offspring for decades, and it wasn't until mid 1980s that they ever even looked at the BASIC that was present, even built-in, on most personal computers! They finally noticed how the adult had grown up, and were SHOCKED at "street BASIC", and how people such as billg (MICROS~1) and Gordon Eubanks (CBASIC) had corrupted it. So, they created and marketed "TRUE BASIC" in 1985, which was a structured compiled language, that returned to the true faith. "I think I'll pass up the opportunity to become a born-again True BASIC believer. I'll enjoy my Microsoft and CBASIC heresies." - Jerry Pournelle -- Grumpy Ol' Fred ci...@xenosoft.com
