Hi again, John S is correct. And thanks again for all the replies. There may be some scope for app changes. It may yet prove workable, but it's not going to be as clean as I first hoped.
Best regards Peter Peter Bishop HP Enterprise Services Asia Pacific South Mainframe Capability & Engineering +61 2 9012 5147 office | +61 2 9012 6620 fax | [email protected] 36-46 George St | Burwood | NSW 2134 Australia -----Original Message----- From: Linux on 390 Port [mailto:[email protected]] On Behalf Of John Summerfield Sent: Tuesday, 3 November 2009 3:18 AM To: [email protected] Subject: Re: emulating a z/OS DDNAME dataset concatenation in Linux Rob van der Heij wrote: > On Mon, Nov 2, 2009 at 5:02 AM, BISHOP, Peter <[email protected]> wrote: > >> To John M - yes, I was thinking in "z/OS" terms, where a single open of the >> DDNAME is sufficient for all the datasets in that DDNAME. >> >> To David, Ed and John S - the annoying thing about pipes here is that they >> incur extra I/O for the "cat" command, and since these files are very large >> (approx 20GB daily) this extra I/O will be very costly, probably excessively >> so here. >> Looks like I'm out of luck here, more thinking required. > > What extra I/O do you mean? When "cat" has multiple arguments, it will > open one file after the other, read them to end-of-file and write the > output to stdout. You can pipe that into your processing. If it's your > own application, you could make the application process a list of file > names. As far as I know this is similar to how MVS does extents. > You would have extra I/O when you had to compose the input file on > disk by appending all your input files. But in a lot of cases you can > make the program use stdin and take the data from the pipe. > > Rob I expect Peter's referring to writing and reading the pipes. In OS, the data goes from disk to buffers to program. Here, it would be going disk to buffers to cat to buffers(?) to pipe to buffers(?) to program. I think Peter should measure the performance, but I'm inclined to think he's right. When I suggested it, I didn't expect it to be "as good." If there's some useful preprocessing that could be done before the program, he could replace cat with a program that does that processing and reduce the overhead. -- Cheers John -- spambait [email protected] [email protected] -- Advice http://webfoot.com/advice/email.top.php http://www.catb.org/~esr/faqs/smart-questions.html http://support.microsoft.com/kb/555375 You cannot reply off-list:-) ---------------------------------------------------------------------- For LINUX-390 subscribe / signoff / archive access instructions, send email to [email protected] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 ---------------------------------------------------------------------- For LINUX-390 subscribe / signoff / archive access instructions, send email to [email protected] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
