Re: [Oorexx-devel] open() option APPEND
On 2021-02-10 15:21:18 CVBruce wrote: > Eric, > > To your point #2. In looking at other implementations of Rexx, I found > that some implementations note that if you move the write pointer to > someplace in the existing file, and then write, the file will be truncated > to that written data. This led me to the realization that the write > pointer is also the “working” end of file pointer. If you move the write > pointer to the middle of the file what will the result be, especially if > the data written is greater than the original data in the position, wiping > out the record separator (CRLF) in the process. > > I would lean toward not being able to move the write pointer, and use > sequential file update processes that are old and well defined. Ah. That's right, I forgot about that truncation issue. > > I know that update in place is a common technique on the mainframe, but > mainframes also support record structures, and record i/o, Rexx for the > most part doesn’t. There is not a data structure that is tied to the file > in away that prohibits writing records that don’t agree with the structure > of the file. On the mainframe (where Rexx was first implemented), update-in-place only worked for fixed-length record files; until the Byte File System in VM and Unix System Services in MVS were implemented, variable-length record files had a record-length prefix on each record; there was no concept of "stream-of-bytes with imbedded linend characters". > > To your point #1, I think replace is more common than default of append. > In most programming environments I’ve used you must explicitly specify > append. > > There have been times in Rexx, where to bullet proof a program, I’ve had to > issue a close then open against a file because the program in development > never reached the file close logic from the last execution, the result > being an append rather than the desired replace. > > Bruce Leslie -- ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] open() option APPEND
Eric, To your point #2. In looking at other implementations of Rexx, I found that some implementations note that if you move the write pointer to someplace in the existing file, and then write, the file will be truncated to that written data. This led me to the realization that the write pointer is also the “working” end of file pointer. If you move the write pointer to the middle of the file what will the result be, especially if the data written is greater than the original data in the position, wiping out the record separator (CRLF) in the process. I would lean toward not being able to move the write pointer, and use sequential file update processes that are old and well defined. I know that update in place is a common technique on the mainframe, but mainframes also support record structures, and record i/o, Rexx for the most part doesn’t. There is not a data structure that is tied to the file in away that prohibits writing records that don’t agree with the structure of the file. To your point #1, I think replace is more common than default of append. In most programming environments I’ve used you must explicitly specify append. There have been times in Rexx, where to bullet proof a program, I’ve had to issue a close then open against a file because the program in development never reached the file close logic from the last execution, the result being an append rather than the desired replace. Bruce > On Feb 10, 2021, at 11:21 AM, Erich Steinböck > wrote: > > Thanks for clarifying the timeline :-) > > Now back to the question: > s~open('append'); s~lineOut('xxx', 1) ignores the line number argument for > lineOut and unconditionally appends > s~open; s~lineOut('xxx', 1) overwrites the first line > > This is pretty crazy, as > 1) "append" should be default (or not?) for any write operation, and > 2) if "append" prohibits overwriting existing lines, we can never overwrite > parts of an existing file at all > > I'd propose to remove the rewrite restriction and make "append" default, but > then: we also have this rexxref paragraph "The write pointer cannot be moved > anywhere within the extent of the file as it existed when the file was > opened." > > So many errors in one place? > Thus my question: anyone remembers what the intention of all this was? > > > > ___ > Oorexx-devel mailing list > Oorexx-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/oorexx-devel signature.asc Description: Message signed with OpenPGP ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] open() option APPEND
Greetings Erich, Erich Steinböck wrote: but then: we also have this rexxref paragraph "The write pointer cannot be moved anywhere within the extent of the file as it existed when the file was opened." So many errors in one place? Thus my question: anyone remembers what the intention of all this was? I believe opening a file in "append" mode is special in that it protects the file content already on disk with a sort of virtual partition. That pre-existing part you may read, just not make changes to. Anything appended to the file while the file is open in append mode is far game for write/read/remove. The program is free to move around within the file reading... just the virtual partition comes firmly into place for write/delete operations. I do not know history on this "append" mode topic. I think it is a solid functionality to maintain in the ooRexx language. Is it causing you some sort of technical issues? I am thankful, -- Michael Lueck Lueck Data Systems http://www.lueckdatasystems.com/ ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] open() option APPEND
From my collection of old manuals, the 1982 version of Rexx (3.10) recommended EXECIO or IOX (which it notes was "specially written for Rexx") for file I/O. Stream/linein/lineout, etc. aren't there. The Stream/linein/lineout,etc. are not in the 1986 VM/SP Release 5 System Product Interpreter manual either. There was a 1992 package for AIX from Endicott that includes Stream, Linein, lineout, etc. including the replace & append options Stream, Linein, Lineout, charin, charout, etc. appear to be fully documented in the VM/ESA 2.1 manual from 1993. However, unlike the 1992 AIX package "append" is not documented as an option on the stream open command. Only read/write/new/replace are listed as "intents". The summary of changes at the front of the manual indicates that the I/O functions were added to VM/ESA 2.1 in June, 1993 and existed in a previous version of Rexx. The wording in the AIX description of append is similar to the description in rexxref. It says the write pointer is positioned at the end of the stream and you cannot position the write pointer to any position that already existed when the file was opened. It doesn't say that the write location operand of charout or lineout is ignored. That sounds to me like it might have allowed you to re-write any new lines you appended but not any old, existing lines. It also doesn't say what happens if you try to re-write an old location i.e. does it append anyway? or does it return a non-zero result indicating the chars/line was not written? Chuck Berghorn oorexx-devel@lists.sourceforge.net On 2/10/21 12:56 PM, Mike Cowlishaw wrote: The Stream functions were added to Rexx in 1982 (I think) but didn't make the cut for the first VM/CMS release (SP3, 1983). They are in the first edition of The Rexx Language (1985). It took a frustratingly long time for them to be added -- no quite sure why that was. Mike -Original Message- From: J Leslie Turriff [mailto:jlturr...@mail.com] Sent: 10 February 2021 16:36 To: Open Object Rexx Developer Mailing List Subject: Re: [Oorexx-devel] open() option APPEND You may be right; I was unable to find a copy of the pertinent Rexx User's Guide from that period, though http://www.rexxinfo.org/All%20IBM%20Rexx%20manuals/ALLIBM_1.HTM purports to provide such. Leslie On 2021-02-10 04:30:20 René Jansen wrote: Hi Leslie, I am sure you mean 1984. And then there was still only EXECIO. I have to look up when VM/CMS got stream; TSO never had it until now, unless we installed a library. I will try to find out the timeline. best regards, René. ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] open() option APPEND
Thanks for clarifying the timeline :-) Now back to the question: s~open('append'); s~lineOut('xxx', 1) ignores the line number argument for lineOut and unconditionally appends s~open; s~lineOut('xxx', 1) overwrites the first line This is pretty crazy, as 1) "append" should be default (or not?) for any write operation, and 2) if "append" prohibits overwriting existing lines, we can never overwrite parts of an existing file at all I'd propose to remove the rewrite restriction and make "append" default, but then: we also have this rexxref paragraph "The write pointer cannot be moved anywhere within the extent of the file as it existed when the file was opened." So many errors in one place? Thus my question: anyone remembers what the intention of all this was? ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] open() option APPEND
The Stream functions were added to Rexx in 1982 (I think) but didn't make the cut for the first VM/CMS release (SP3, 1983). They are in the first edition of The Rexx Language (1985). It took a frustratingly long time for them to be added -- no quite sure why that was. Mike > -Original Message- > From: J Leslie Turriff [mailto:jlturr...@mail.com] > Sent: 10 February 2021 16:36 > To: Open Object Rexx Developer Mailing List > Subject: Re: [Oorexx-devel] open() option APPEND > > You may be right; I was unable to find a copy of the > pertinent Rexx User's Guide from that period, though > http://www.rexxinfo.org/All%20IBM%20Rexx%20manuals/ALLIBM_1.HTM > purports to provide such. > > Leslie > > On 2021-02-10 04:30:20 René Jansen wrote: > > Hi Leslie, > > > > I am sure you mean 1984. And then there was still only > EXECIO. I have > > to look up when VM/CMS got stream; TSO never had it until > now, unless > > we installed a library. I will try to find out the timeline. > > > > best regards, > > > > René. ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] open() option APPEND
You may be right; I was unable to find a copy of the pertinent Rexx User's Guide from that period, though http://www.rexxinfo.org/All%20IBM%20Rexx%20manuals/ALLIBM_1.HTM purports to provide such. Leslie On 2021-02-10 04:30:20 René Jansen wrote: > Hi Leslie, > > I am sure you mean 1984. And then there was still only EXECIO. I have to > look up when VM/CMS got stream; TSO never had it until now, unless we > installed a library. I will try to find out the timeline. > > best regards, > > René. > > > On 10 Feb 2021, at 05:45, J Leslie Turriff wrote: > > > > On 2021-02-09 19:02:12 Michael Lueck wrote: > >>> Any ideas on how this was meant to be when it was designed? > >> > >> I never claim to be the best expert at REXX history, however... > >> > >> On Mainframe, I believe either the Stream concept came much later or > >> still is not there at all. > > > > The file I/O functions [stream(), charin(), charout(), chars(), > > linein(), lineout(), lines()] have been in the language since at least > > when I first used VM/CMS in ~1974. They could operate on CMS minidisk, > > Shared File System*, or Byte File System* files; sockets capabilities > > came later when a TCP/IP stack was added to VM. In MVS Rexx could use > > these functions to do I/O to QSAM, Open Edition or Unix System Services > > files. See https://www.vm.ibm.com/library/640pdfs/64622201.pdf Chapter 9: > > Input and Output, for example. > > > > Leslie > > > > > > -- > > > > > > ___ > > Oorexx-devel mailing list > > Oorexx-devel@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/oorexx-devel > > ___ > Oorexx-devel mailing list > Oorexx-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/oorexx-devel -- ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] open() option APPEND
Greetings René, René Jansen wrote: I am sure you mean 1984. And then there was still only EXECIO. Excellent correlation! Indeed EXECIO was the mainframe I/O alternative I could not recall the name of. I am thankful, -- Michael Lueck Lueck Data Systems http://www.lueckdatasystems.com/ ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] open() option APPEND
On Wed, Feb 10, 2021 at 5:31 AM René Jansen wrote: > Hi Leslie, > > I am sure you mean 1984. And then there was still only EXECIO. I have to > look up when VM/CMS got stream; TSO never had it until now, unless we > installed a library. > I will try to find out the timeline. > VM/CMS got it 1988, at the earliest. Rick > > best regards, > > René. > > > > On 10 Feb 2021, at 05:45, J Leslie Turriff wrote: > > > > On 2021-02-09 19:02:12 Michael Lueck wrote: > >>> Any ideas on how this was meant to be when it was designed? > >> > >> I never claim to be the best expert at REXX history, however... > >> > >> On Mainframe, I believe either the Stream concept came much later or > still > >> is not there at all. > > > > The file I/O functions [stream(), charin(), charout(), chars(), > linein(), lineout(), > > lines()] have been in the language since at least when I first used > VM/CMS in ~1974. > > They could operate on CMS minidisk, Shared File System*, or Byte File > System* files; > > sockets capabilities came later when a TCP/IP stack was added to VM. In > MVS Rexx could > > use these functions to do I/O to QSAM, Open Edition or Unix System > Services files. > > See https://www.vm.ibm.com/library/640pdfs/64622201.pdf Chapter > 9: Input and Output, for > > example. > > > > Leslie > > > > > > -- > > > > > > ___ > > Oorexx-devel mailing list > > Oorexx-devel@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/oorexx-devel > > > > ___ > Oorexx-devel mailing list > Oorexx-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/oorexx-devel > ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] open() option APPEND
Hi Leslie, I am sure you mean 1984. And then there was still only EXECIO. I have to look up when VM/CMS got stream; TSO never had it until now, unless we installed a library. I will try to find out the timeline. best regards, René. > On 10 Feb 2021, at 05:45, J Leslie Turriff wrote: > > On 2021-02-09 19:02:12 Michael Lueck wrote: >>> Any ideas on how this was meant to be when it was designed? >> >> I never claim to be the best expert at REXX history, however... >> >> On Mainframe, I believe either the Stream concept came much later or still >> is not there at all. > > The file I/O functions [stream(), charin(), charout(), chars(), > linein(), lineout(), > lines()] have been in the language since at least when I first used VM/CMS in > ~1974. > They could operate on CMS minidisk, Shared File System*, or Byte File System* > files; > sockets capabilities came later when a TCP/IP stack was added to VM. In MVS > Rexx could > use these functions to do I/O to QSAM, Open Edition or Unix System Services > files. > See https://www.vm.ibm.com/library/640pdfs/64622201.pdf Chapter 9: > Input and Output, for > example. > > Leslie > > > -- > > > ___ > Oorexx-devel mailing list > Oorexx-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/oorexx-devel ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] open() option APPEND
On 2021-02-09 19:02:12 Michael Lueck wrote: > > Any ideas on how this was meant to be when it was designed? > > I never claim to be the best expert at REXX history, however... > > On Mainframe, I believe either the Stream concept came much later or still > is not there at all. The file I/O functions [stream(), charin(), charout(), chars(), linein(), lineout(), lines()] have been in the language since at least when I first used VM/CMS in ~1974. They could operate on CMS minidisk, Shared File System*, or Byte File System* files; sockets capabilities came later when a TCP/IP stack was added to VM. In MVS Rexx could use these functions to do I/O to QSAM, Open Edition or Unix System Services files. See https://www.vm.ibm.com/library/640pdfs/64622201.pdf Chapter 9: Input and Output, for example. Leslie -- ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] open() option APPEND
Greetings Erich, Erich Steinböck wrote: Any ideas on how this was meant to be when it was designed? I never claim to be the best expert at REXX history, however... On Mainframe, I believe either the Stream concept came much later or still is not there at all. On OS/2 / Windows / Linux... Object Rexx... there is support for both Classic Rexx File I/O functions, and the Object Rexx class based File I/O. The Object based File I/O quite well encapsulates the former Classic Rexx File I/O functions. I am not certain how all the legacy support (Classic Rexx running under ooRexx) has been mapped. Back in the Object Rexx v2.x days, there were some attempts to improve the performance of Object Rexx running Classic Rexx function call based File I/O in not having it create and destroy so many Stream Objects behind the scenes. Oh then there was I believe File I/O on the Array class in the v1.0.3.0 days. Later IBM Germany worked to improve the poor performance of that implementation by supporting... /* Check fow how many lines the file has */ FILEsize = file~command('QUERY SIZE') /* Read in the file */ /* This code requires at least ORexx 2.1.1 on Win32 - only tested with >=2.1.2 */ INstr = file~charin(1, FILEsize) FINDarray = INstr~makearray() MakeArray() method off the String class. And for output... /* Write out the file */ /* This code requires at least ORexx 2.1.3 on Win32 */ file~LineOut(FINDarray~MakeString()) So apparently IBM Germany provided one higher performance method in the String Class and the other higher performance method was later added to the Array Class. That pair of methods were added in the ORexx 2.x era, never in the ORexx 1.0.3.0 At RexxLA symposiums, must have been early 2000's, I spoke with the IBM Germany developers about the poor performance performing File I/O via class methods, and they seemed unwilling to touch that part of the ORexx code. So in my mind... "replace (with the above two added API's) rather than touch the ORexx methods that did exist back then." Back then, ORexx 1.0.3.0, File I/O had a recursively growing lag in processing files when the number of lines would grow. Perhaps the code is a bit of a patchwork quilt in that it is expected to execute all possible eras of CRexx / ORexx / ooRexx code? <><><><> So about Rexx and file write in append mode... as far as I know that method of output is true append. So a Windows system opens in append, issues LineOut calls, and results in lines being added with CrLf line endings. A Linux ooRexx program writes more to the same file, opening in append mode, LineOut calls result in lines being added with Lf line endings. I would think if the same were done on Mac, then lines would be added with Cr line endings. So writing to a shared file with ooRexx code running on multiple operating systems could produce a file with mixed line endings. Erich Steinböck wrote: Do we have a third WRITE mode - non-APPEND, non-REPLACE - that needs to be documented? What characteristics would the third WRITE mode have? Ignore "non" capabilities... those show what it lacks. What is left as a capability to yet support? I am thankful, -- Michael Lueck Lueck Data Systems http://www.lueckdatasystems.com/ ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] open() option APPEND
Any ideas on how this was meant to be when it was designed? On Fri, Feb 5, 2021 at 9:14 PM Erich Steinböck wrote: > I understand we can open a file for write access in either REPLACE or > APPEND mode, the latter being the default. Also, rexxpg says "By default, > LINEOUT appends to an existing file." > > But the open() method description in rexxref describes the APPEND option > with "The write pointer cannot be moved anywhere within the extent of the > file as it existed when the file was opened." > > And we have code in place which for s~open('append'); s~lineOut('xxx', 1) > ignores the line number argument for lineOut and unconditionally appends, > whereas for s~open; s~lineOut('xxx', 1) the line overwrites the first > line (as expected). > > Do we have a third WRITE mode - non-APPEND, non-REPLACE - that needs to > be documented? > ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
[Oorexx-devel] open() option APPEND
I understand we can open a file for write access in either REPLACE or APPEND mode, the latter being the default. Also, rexxpg says "By default, LINEOUT appends to an existing file." But the open() method description in rexxref describes the APPEND option with "The write pointer cannot be moved anywhere within the extent of the file as it existed when the file was opened." And we have code in place which for s~open('append'); s~lineOut('xxx', 1) ignores the line number argument for lineOut and unconditionally appends, whereas for s~open; s~lineOut('xxx', 1) the line overwrites the first line (as expected). Do we have a third WRITE mode - non-APPEND, non-REPLACE - that needs to be documented? ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel