Re: Copying sequential files - BSAM and QSAM
You should up your buffers (and NCP). Two will be spending a lot of time waiting. Just remember that after a CHECK shows EOF, you should not expect a valid status from the pending reads. On Fri, 28 Nov 2014 12:20:08 -0500 Sam Golob sbgo...@cbttape.org wrote: :Hi Folks, : : I was just involved in an exercise to improve an old file copying :program from CBT Tape File 316. It is called FFYCOPY, and it was :written in the mid-1970's by Frank Yates (courtesy of Jim Marshall). :This program's copy engine uses BSAM with double buffers, and for its :time, it was very fast, since BSAM (using the READ and WRITE macros) :deals with copying blocks of data instead of records. Of course, QSAM :(using the GET and PUT macros) is geared to copying records. : : Most sequential file copying programs use QSAM, because the :granularity is at the record level and not at the block level. But :if speed is a priority, BSAM, copying equal sized blocks to equal sized :blocks, can be an advantage. : : I'm not telling anybody here to give up the copying programs they :have been using (there are many of them out there), but this is for the :purpose of discussion, and I hope it helps somebody in the process :(maybe even me). : : FFYCOPY, being a very simply written program (except for the skill :in writing the engine), needs to have the output dataset having :identical RECFM, LRECL, and BLKSIZE to those of the input dataset. To :guarantee that, it simply overlays those attibutes in the output dataset :before OPENing it. This may get into problems if you're copying a :sequential dataset to a member of a pds. The copy does not reblock the :dataset (as it uses BSAM very simply), and the output pds can be messed :up. To avoid such problems, I put safeguards into FFYCOPY (see FFYCOPY :in File 316 on the Updates page of www.cbttape.org), so that if RECFM, :LRECL, or BLKSIZE don't match, the copy is not done. I also added a :SYSPRINT dataset to report status of what happened. And to force the :overlay of the attributes in the output dataset (as in the old version), :you can code PARM=O in the EXEC statement. : : I'd be interested in hearing if there are any issues nowadays in :copying BIG files FAST. Of course the world has progressed a lot in 40 :years (snicker), but I'd still like to hear some thoughts on the subject. : : Thanks much. All the best of everything to all of you. : :Sincerely,Sam : :-- :For IBM-MAIN subscribe / signoff / archive access instructions, :send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- Binyamin Dissen bdis...@dissensoftware.com http://www.dissensoftware.com Director, Dissen Software, Bar Grill - Israel Should you use the mailblocks package and expect a response from me, you should preauthorize the dissensoftware.com domain. I very rarely bother responding to challenge/response systems, especially those from irresponsible companies. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Copying sequential files - BSAM and QSAM
Hi Folks, I was just involved in an exercise to improve an old file copying program from CBT Tape File 316. It is called FFYCOPY, and it was written in the mid-1970's by Frank Yates (courtesy of Jim Marshall). This program's copy engine uses BSAM with double buffers, and for its time, it was very fast, since BSAM (using the READ and WRITE macros) deals with copying blocks of data instead of records. Of course, QSAM (using the GET and PUT macros) is geared to copying records. Most sequential file copying programs use QSAM, because the granularity is at the record level and not at the block level. But if speed is a priority, BSAM, copying equal sized blocks to equal sized blocks, can be an advantage. I'm not telling anybody here to give up the copying programs they have been using (there are many of them out there), but this is for the purpose of discussion, and I hope it helps somebody in the process (maybe even me). FFYCOPY, being a very simply written program (except for the skill in writing the engine), needs to have the output dataset having identical RECFM, LRECL, and BLKSIZE to those of the input dataset. To guarantee that, it simply overlays those attibutes in the output dataset before OPENing it. This may get into problems if you're copying a sequential dataset to a member of a pds. The copy does not reblock the dataset (as it uses BSAM very simply), and the output pds can be messed up. To avoid such problems, I put safeguards into FFYCOPY (see FFYCOPY in File 316 on the Updates page of www.cbttape.org), so that if RECFM, LRECL, or BLKSIZE don't match, the copy is not done. I also added a SYSPRINT dataset to report status of what happened. And to force the overlay of the attributes in the output dataset (as in the old version), you can code PARM=O in the EXEC statement. I'd be interested in hearing if there are any issues nowadays in copying BIG files FAST. Of course the world has progressed a lot in 40 years (snicker), but I'd still like to hear some thoughts on the subject. Thanks much. All the best of everything to all of you. Sincerely,Sam -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Copying sequential files - BSAM and QSAM (UNCLASSIFIED)
Classification: UNCLASSIFIED Caveats: NONE Hi Sam, I realize that you're looking at programmatic copying techniques. When we want to copy a big file fast, we tend to utilize dataset FlashCopy. It's much faster than any software solution. The only reason I mention this is that it sometimes gets overlooked as a viable alternative. Cheers, Alan -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Sam Golob Sent: Friday, November 28, 2014 12:20 PM To: IBM-MAIN@LISTSERV.UA.EDU Subject: Copying sequential files - BSAM and QSAM Hi Folks, I was just involved in an exercise to improve an old file copying program from CBT Tape File 316. It is called FFYCOPY, and it was written in the mid-1970's by Frank Yates (courtesy of Jim Marshall). This program's copy engine uses BSAM with double buffers, and for its time, it was very fast, since BSAM (using the READ and WRITE macros) deals with copying blocks of data instead of records. Of course, QSAM (using the GET and PUT macros) is geared to copying records. Most sequential file copying programs use QSAM, because the granularity is at the record level and not at the block level. But if speed is a priority, BSAM, copying equal sized blocks to equal sized blocks, can be an advantage. I'm not telling anybody here to give up the copying programs they have been using (there are many of them out there), but this is for the purpose of discussion, and I hope it helps somebody in the process (maybe even me). FFYCOPY, being a very simply written program (except for the skill in writing the engine), needs to have the output dataset having identical RECFM, LRECL, and BLKSIZE to those of the input dataset. To guarantee that, it simply overlays those attibutes in the output dataset before OPENing it. This may get into problems if you're copying a sequential dataset to a member of a pds. The copy does not reblock the dataset (as it uses BSAM very simply), and the output pds can be messed up. To avoid such problems, I put safeguards into FFYCOPY (see FFYCOPY in File 316 on the Updates page of www.cbttape.org), so that if RECFM, LRECL, or BLKSIZE don't match, the copy is not done. I also added a SYSPRINT dataset to report status of what happened. And to force the overlay of the attributes in the output dataset (as in the old version), you can code PARM=O in the EXEC statement. I'd be interested in hearing if there are any issues nowadays in copying BIG files FAST. Of course the world has progressed a lot in 40 years (snicker), but I'd still like to hear some thoughts on the subject. Thanks much. All the best of everything to all of you. Sincerely,Sam -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN Classification: UNCLASSIFIED Caveats: NONE -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Copying sequential files - BSAM and QSAM
On 11/28/2014 12:20 PM, Sam Golob wrote: Hi Folks, I was just involved in an exercise to improve an old file copying program from CBT Tape File 316. It is called FFYCOPY, and it was written in the mid-1970's by Frank Yates (courtesy of Jim Marshall). This program's copy engine uses BSAM with double buffers, and for its time, it was very fast, since BSAM (using the READ and WRITE macros) deals with copying blocks of data instead of records. Of course, QSAM (using the GET and PUT macros) is geared to copying records. SNIPPAGE You didn't mention it, but, if you will look at the NCPs, you could have as many as 255 I/Os outstanding. As I recall, you will have to use CHECK for this type of processing. The NCP value can be coded in JCL, or it can be inserted into the DCB and be a system controlled value. In using this, you can, as I recall, even handle striped data sets much faster. Just my 2 cents worth -- which may not be worth anything at all. Regards, Steve Thompson -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: Copying sequential files - BSAM and QSAM
In 5478aec8.8020...@cbttape.org, on 11/28/2014 at 12:20 PM, Sam Golob sbgo...@cbttape.org said: uses BSAM with double buffers I would expect QSAM with lots of buffers to outperform that, but it's not that hard to do BSAM with lots of buffers. To avoid such problems, You might also want to add code to avoid clobbering the directory if the output DD specifies a PDS without a member name. See METZCOPY[1] for an example of how to do that. [1] Written in a fit of pique due to IEBGENER misbehaving. -- Shmuel (Seymour J.) Metz, SysProg and JOAT ISO position; see http://patriot.net/~shmuel/resume/brief.html We don't care. We don't have to care, we're Congress. (S877: The Shut up and Eat Your spam act of 2003) -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
