Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS?
What is a physical block? Even without taking into accounnt that contemporary DASD subsystems are cached, on old CKD DASD you could read any of count, key and data without reading the others. -- Shmuel (Seymour J.) Metz http://mason.gmu.edu/~smetz3 עַם יִשְׂרָאֵל חַי From: IBM Mainframe Assembler List on behalf of Bob Raicer Sent: Monday, November 13, 2023 5:59 PM To: ASSEMBLER-LIST@LISTSERV.UGA.EDU Subject: Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS? There have been a few posts in this thread that more or less stated that only complete physical blocks could be read into storage from a device. This is not strictly true. There is the obvious case of truncation, where the number of bytes the device has been requested to transfer is less than the actual physical size of the block. For CCW types of I/O operations (as opposed to TCW types) there is also the possibility of exploiting the "Skip" feature of the channel. Bit 35 (format-0 CCWs) or bit 11 (format-1 CCWs) of the CCW is referred to as the Skip Flag Bit. When this bit is one, the channel suppresses the transfer of data into storage. This flag bit is valid ONLY for read, read-backward, sense ID, or sense operations. Note that the storage device still engages with the data transfer to the channel and all of the data error checking functions (such as ECC) still occur. It is the channel that suppresses the storage of data into memory. When combined with Data Chaining this can be used to store subsets of a data block into potentially discontiguous areas of memory. I do not believe that any of the access methods exploit this, however. OPEN/CLOSE/EOV at one time used this feature when processing Standard User Label tapes. The interesting portion of the channel program was a Read with the Skip Bit on, command chained to a Transfer In Channel back to the Read. When no errors occurred, the channel program ended with Unit Exception status when the Tape Mark at the end of the label set was encountered. Bob
Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS?
On 11/13/23 15:59:45, Bob Raicer wrote: There have been a few posts in this thread that more or less stated that only complete physical blocks could be read into storage from a device. This is not strictly true.> We don't know the OP's objective: performance, security, or other. -- gil
Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS?
There have been a few posts in this thread that more or less stated that only complete physical blocks could be read into storage from a device. This is not strictly true. There is the obvious case of truncation, where the number of bytes the device has been requested to transfer is less than the actual physical size of the block. For CCW types of I/O operations (as opposed to TCW types) there is also the possibility of exploiting the "Skip" feature of the channel. Bit 35 (format-0 CCWs) or bit 11 (format-1 CCWs) of the CCW is referred to as the Skip Flag Bit. When this bit is one, the channel suppresses the transfer of data into storage. This flag bit is valid ONLY for read, read-backward, sense ID, or sense operations. Note that the storage device still engages with the data transfer to the channel and all of the data error checking functions (such as ECC) still occur. It is the channel that suppresses the storage of data into memory. When combined with Data Chaining this can be used to store subsets of a data block into potentially discontiguous areas of memory. I do not believe that any of the access methods exploit this, however. OPEN/CLOSE/EOV at one time used this feature when processing Standard User Label tapes. The interesting portion of the channel program was a Read with the Skip Bit on, command chained to a Transfer In Channel back to the Read. When no errors occurred, the channel program ended with Unit Exception status when the Tape Mark at the end of the label set was encountered. Bob
Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS?
On 11/13/2023 7:23 AM, Schmitt, Michael wrote: As long no one ever uses DISP=MOD ...or checkpoints. Checkpoints can cause short blocks to be written. What about CLOSE TYPE=T? Would that flush the current block? -- Phoenix Software International Edward E. Jaffe 831 Parkview Drive North El Segundo, CA 90245 https://www.phoenixsoftware.com/ This e-mail message, including any attachments, appended messages and the information contained therein, is for the sole use of the intended recipient(s). If you are not an intended recipient or have otherwise received this email message in error, any use, dissemination, distribution, review, storage or copying of this e-mail message and the information contained therein is strictly prohibited. If you are not an intended recipient, please contact the sender by reply e-mail and destroy all copies of this email message and do not otherwise utilize or retain this email message or any or all of the information contained therein. Although this email message and any attachments or appended messages are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by the sender for any loss or damage arising in any way from its opening or use.
Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS?
At one time there was a component called Actual Block Processor that was used both by VSAM and paging, and the paging data sets were CI format. I have no idea how much of that, if any, remains in z/OS, but essentially my question was whether pages were now stored in 4 KiB physical records. -- Shmuel (Seymour J.) Metz http://mason.gmu.edu/~smetz3 עַם יִשְׂרָאֵל חַי From: IBM Mainframe Assembler List on behalf of Bob Raicer Sent: Monday, November 13, 2023 2:14 PM To: ASSEMBLER-LIST@LISTSERV.UGA.EDU Subject: Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS? A PDSE is essentially a form of VSAM Linear, where the physical data set is a collection of 4K-byte blocks. It does not have the notion of CA's and CI's like other VSAM data sets. The notion of BLKSIZE is synthesized and mapped onto the internal PDSE structures; the BLKSIZE value is not the physical unit of transfer size as is the typical case for a traditional Physical Sequential data set. Here is a URL for a SHARE presentation that provides a nice explanation of how a PDSE is put together: https://share.confex.com/share/124/webprogram/Handout/Session16956/PDSE%20Nuts%20and%20Bolts.pdf Bob On 2023-11-13 7:48 AM, Seymour J Metz wrote: > > Does anybody know whether CI size = block size for PDSE and zFS (linear) data > sets? VSAM used to use multiple blocks for a CI. > > >
Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS?
A PDSE is essentially a form of VSAM Linear, where the physical data set is a collection of 4K-byte blocks. It does not have the notion of CA's and CI's like other VSAM data sets. The notion of BLKSIZE is synthesized and mapped onto the internal PDSE structures; the BLKSIZE value is not the physical unit of transfer size as is the typical case for a traditional Physical Sequential data set. Here is a URL for a SHARE presentation that provides a nice explanation of how a PDSE is put together: https://share.confex.com/share/124/webprogram/Handout/Session16956/PDSE%20Nuts%20and%20Bolts.pdf Bob On 2023-11-13 7:48 AM, Seymour J Metz wrote: Does anybody know whether CI size = block size for PDSE and zFS (linear) data sets? VSAM used to use multiple blocks for a CI.
Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS?
On 11/13/23 11:54:07, Schmitt, Michael wrote: That I don't know. I've only dealt with GSAM checkpoints (which may not support RECFM=VBS). And one other: checkpoints in z/OS FTP. My *guess* is that since a checkpoint has to flush the buffers to disk before you fill up the block, even VBS would have a short block. I stand corrected. I intended to inquire about RECFM=FBS. -Original Message- From: IBM Mainframe Assembler List On Behalf Of Paul Gilmartin Sent: Monday, November 13, 2023 12:50 PM To: ASSEMBLER-LIST@LISTSERV.UGA.EDU Subject: Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS? On 11/13/23 08:23:44, Schmitt, Michael wrote: As long no one ever uses DISP=MOD ...or checkpoints. Checkpoints can cause short blocks to be written. (My knowledge is with IMS/DB GSAM checkpoints.) How does checkpoint interact with RECFM=VBS? -- gil
Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS?
That I don't know. I've only dealt with GSAM checkpoints (which may not support RECFM=VBS). And one other: checkpoints in z/OS FTP. My *guess* is that since a checkpoint has to flush the buffers to disk before you fill up the block, even VBS would have a short block. -Original Message- From: IBM Mainframe Assembler List On Behalf Of Paul Gilmartin Sent: Monday, November 13, 2023 12:50 PM To: ASSEMBLER-LIST@LISTSERV.UGA.EDU Subject: Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS? On 11/13/23 08:23:44, Schmitt, Michael wrote: >>> As long no one ever uses DISP=MOD > > ...or checkpoints. Checkpoints can cause short blocks to be written. > > (My knowledge is with IMS/DB GSAM checkpoints.) How does checkpoint interact with RECFM=VBS? -- gil
Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS?
Same as above. On Mon, Nov 13, 2023 at 1:50 PM Paul Gilmartin < 0014e0e4a59b-dmarc-requ...@listserv.uga.edu> wrote: > On 11/13/23 08:23:44, Schmitt, Michael wrote: > >>> As long no one ever uses DISP=MOD > > > > ...or checkpoints. Checkpoints can cause short blocks to be written. > > > > (My knowledge is with IMS/DB GSAM checkpoints.) > > How does checkpoint interact with RECFM=VBS? > > -- > gil >
Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS?
While the original question isn't really about assembler, it is at least about programming. This tangent has nothing to do with either... if you want uninformed speculation on random topics, that's what IBM-MAIN is for. sas On Mon, Nov 13, 2023 at 9:48 AM Seymour J Metz wrote: > I believe that the OP wanted a solution for BSAM. If VSAM is an option > then it's a nobrainer. > > Does anybody know whether CI size = block size for PDSE and zFS (linear) > data sets? VSAM used to use multiple blocks for a CI. > >
Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS?
On 11/13/23 08:23:44, Schmitt, Michael wrote: As long no one ever uses DISP=MOD ...or checkpoints. Checkpoints can cause short blocks to be written. (My knowledge is with IMS/DB GSAM checkpoints.) How does checkpoint interact with RECFM=VBS? -- gil
Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS?
>> As long no one ever uses DISP=MOD ...or checkpoints. Checkpoints can cause short blocks to be written. (My knowledge is with IMS/DB GSAM checkpoints.) -Original Message- From: IBM Mainframe Assembler List On Behalf Of Jon Perryman Sent: Sunday, November 12, 2023 5:18 PM To: ASSEMBLER-LIST@LISTSERV.UGA.EDU Subject: Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS? Everyone is making this far more complicated than it needs to be. Positioning a file is about you designing a method that can calculate a position on the disk that meets your requirements. z/OS, Unix, CMS, z/VSE and MS-Windows have file / dataset positioning because it greatly reduces I/O. There are many designs to available to calculate position. Specifically asked was z/OS RECFM=FB. As long no one ever uses DISP=MOD, then the calculation is simple because number of records per block and blocks per track is consistent. You simply divide record number by records per block giving the block number used in the point and remainder is the number of records you must read after the point. Not quite correct but you get the idea. If you need DISP=MOD functionality, then you must create the functionality. All programs needing DISP=MOD must open the file for update, position to the last block and read until EOF. The simpler solution is VSAM RRDS or ESDS which allows you to position to a specific record number without the worry of DISP=MOD problems. As I said before, there are many solutions and your requirements will determine which solution best fits your needs. Suggestions have been made for a USS file. The advantage is that positioning is at the byte level instead of block. The calculation is greatly simplified to records times record size and DISP=MOD is greatly simplified because Unix does an append to the end of the file. The drawback is that the read record size must be handled by the programmer. Most of the suggestions are viable solutions. You could even maintain an index file but then RRDS already does this for you. The important thing is a consistent method for POINT is determining the block and record for positioning. On Sun, 12 Nov 2023 12:17:12 -0700, Paul Gilmartin wrote: >what does the EXECIO linenum option do? > >EXECIO arose on CMS and antedates REXX. CMS files are indexed by >line number. I'd expect EXECIO on CMS to do True Skip-Sequential >Processing. Less likely on z/OS, even for RECFM=FBS. As for CMS files, It's very doubtful they are indexed because there is no real benefit when the majority of files are processed sequentially. Records in a CMS file are probably stored in what z/OS people would think of as RECFM=VBS where blocksize is probably track size. I doubt that CMS files have line number associations. CMS edit has number on/off but it has a different use. More likely is that CMS keeps a counter with the record number.
Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS?
I believe that the OP wanted a solution for BSAM. If VSAM is an option then it's a nobrainer. Does anybody know whether CI size = block size for PDSE and zFS (linear) data sets? VSAM used to use multiple blocks for a CI. -- Shmuel (Seymour J.) Metz http://mason.gmu.edu/~smetz3 עַם יִשְׂרָאֵל חַי From: IBM Mainframe Assembler List on behalf of Jon Perryman Sent: Sunday, November 12, 2023 8:27 PM To: ASSEMBLER-LIST@LISTSERV.UGA.EDU Subject: Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS? On Sun, 12 Nov 2023 17:14:49 -0700, Paul Gilmartin wrote: >RECFM=FBS prevents that. That is certainly a partial solution but the correct solution is VSAM RRDS or ESDS which eliminates the special handling. >That's mostly how the Divide instruction works. Division is not the hard part. Implementation is what's being discussed which involves division.. >... Fill the block with additional records and rewrite in place. What ever floats your boat. If the OP wants to fully implement that logic, it's available. >Why, in this 21st Century hasn't that function been added to the >access methods!? UNIX does it, as you say below. What makes you falsely claim that function doesn't exist. There is no difference between z/OS and Unix files in this respect. Your programs must deal with those extra records by ignoring them. Both must retain a pointer to the first available instead of using EOF. >I think that can be done by the Multiply instruction, provided >by most modern computer hardware. Wow, the computers you use can't multiply? I stated "times" which is another word for multiply. > But fixed-length records are >alien to UNIX -- I think they're a relic of UR-device practice. RECFM=FB & VB are not relics. It's a vital concept used by databases. A max length is required by VARCHAR for a reason. They know it's more efficient to add a number instead of scanning for X'00'. Databases encourage you to supply max length for everything and truly unknown lengths are avoided. Just because a basic concept is not understood by Unix programmers doesn't mean it doesn't have great value.
Re: Is True Skip-Sequential Processing Possible with RECFM=FB,DSORG=PS?
Someone mentioned the CMS file system in a recent post. The CMS file system uses a hierarchical block map system for each file, which gives very efficient direct access by record number (both for fixed-length and varying-length records) avoiding the need to read any previous file data blocks (although it may be necessary to read higher-level map blocks, which are likely to be cached anyway). Jonathan Scott, HLASM IBM Hursley, UK
