My very first PUTFILES was indeed simple: an ADDPIPE for each fileid
encountered. Worked great until a compile job sent from z/OS contained
quite a lot of files (180 I read in PUTFILES help file). Solved by the
GROUPED or BATCHED options, where PUTFIES then uses CALLPIPE to process one
file per iteration.
Later I added full SFS support with workunits after a problem that I don't
remember at this time.
Kris Buelens,
--- VM/VSE consultant, Belgium ---
-----------------------------------------------------------------------
Op wo 3 mrt. 2021 om 15:41 schreef van Sleeuwen, Berry <
[email protected]>:
> Hi All,
>
> Thanks for the hints.
>
> I don't expect there are too many files. In this case, there are 6 files
> created in this stage. Given this remark, I expect I shouldn't merge all
> streams into one PUTFILES, the number of files will then go above 100. The
> machine where I notice the error currently produces 112 files but in other
> VM's that will be a lot more, those may easily go above 300 or 400 files in
> total.
>
> The PIPELINE has input data that is used to create various files. So the
> data is copied into a number of streams. Then each stream creates
> outputfiles. I don't expect one stream would prefix the data with a
> filename that should be have been used in another stream.
>
> Indeed, BATCHED would be an option. The data should arrive in order so
> when the fileid changes that would normally be the end of the data for that
> file anyway.
>
> An EXEC collects CP MONITOR data that is processed each cycle (1 minute)
> by a REXX stage that contains the PUTFILES stages. So the EXEC is a never
> ending PIPE, the CALLPIPE within the REXX stage ends after each cycle. That
> should close all files after each cycle.
>
> I noticed the error in a file with one single line for each file, but
> other PUTFILE stages have multiple lines for each file. So DISKSLOW might
> be an option for this stage but not for others. I guess BATCHED would be
> the better option for that.
>
> I noticed the error yesterday while testing the process. The output was
> written into a temp FBA vdisk at that time. The monitor machine is an SFS
> based machine, it writes data into the A disk, but that's actually the
> users SFS root directory. So for the collector machine >SFS would indeed be
> an option.
>
>
> Met vriendelijke groet/With kind regards/Mit freundlichen Grüßen,
> Berry van Sleeuwen
> Flight Forum 3000 5657 EW Eindhoven
>
> -----Original Message-----
> From: CMSTSO Pipelines Discussion List <[email protected]> On
> Behalf Of Kris Buelens
> Sent: Wednesday, 3 March 2021 13:30
> To: [email protected]
> Subject: Re: [CMS-PIPELINES] FPL120E in PUTFILES
>
> Caution! External email. Do not open attachments or click links, unless
> this email comes from a known sender and you know the content is safe.
>
> Hello Berry,
>
> Have you looked into my documentation (that is the HELP file).
> I remember having had problems with too many open files and/or ???. I
> think I created two options to avoid problems: BATCHED, that closes the
> file when I detect a new fileID starts being used. Later I had problems
> with commits when the target is SFS.
> But 2010 is long ago for my memory and without day to day work with z/VM.
>
> At the other hand: if you have two or more PUTFILES stages that try to
> write to the same file, sure that you'll encounter this problem. As usual:
> the different stages don't talk to each-other, so one PUTFILES isn't aware
> of what another PUTFILES tries to do. And, if you have a never ending
> PIPE: when will PUTFILES close the files it writes to? Maybe only when
> they the PUTFILES stages reach end-of-file. With BATCHED they get closed
> when the fileid changes. But, if there's no new record arriving, it can
> take ages....
>
> Kris Buelens,
> --- VM/VSE consultant, Belgium ---
> -----------------------------------------------------------------------
>
>
> Op wo 3 mrt. 2021 om 12:43 schreef van Sleeuwen, Berry <
> [email protected]>:
>
> > Hi All,
> >
> > I have a REXX with a number of PUTFILES stages. Each of them will save
> > a report in a group of files. The REXX is executed every minute.
> >
> > Two times today I got an FPL120E in one of the PUTFILES stages. It
> > looks like the error only appears in the GSTVM000 files, or at least
> > the other PUTFILES stages didn't have an error (yet).
> >
> > FPLDSR120E Return code 16 from parameter list Vblockw GSTVM000 TCPIP
> > A1 FPLMSG003I ... Issued from stage 2 of pipeline 2 FPLMSG001I ...
> > Running "> GSTVM000 TCPIP A"
> > FPLDSR120E Return code 16 from parameter list Vblockw GSTVM000 HIDRO
> > A1 FPLMSG003I ... Issued from stage 2 of pipeline 2 FPLMSG001I ...
> > Running "> GSTVM000 HIDRO A"
> > <Repeated 6 times for each file in the stage>
> >
> > According to the docs a return code 16 suggests that PUTFILES attempts
> > to write to the same file. But as far as I can find there shouldn't be
> > a duplicate file. The input for this stage actually contains just one
> > line for each file so PUTFILES only needs to write 1 record to each
> > file in each execution run.
> >
> > Last night I ran the code with traces for a couple of hours, and
> > obviously Murphy made sure there was no error during that time.
> >
> > What could be the reason for this error? Is there another reason other
> > than the "duplicate file" in this case?
> >
> > And generally, I run a number of PUTFILES stages in the REXX for each
> > report. Is that accepted or should I redirect all data to one single
> > PUTFILES stage?
> >
> > Met vriendelijke groet/With kind regards/Mit freundlichen Grüßen,
> > Berry van Sleeuwen Flight Forum 3000 5657 EW Eindhoven This e-mail and
> > the documents attached are confidential and intended solely for the
> > addressee; it may also be privileged. If you receive this e-mail in
> > error, please notify the sender immediately and destroy it. As its
> > integrity cannot be secured on the Internet, Atos' liability cannot be
> > triggered for the message content. Although the sender endeavours to
> > maintain a computer virus-free network, the sender does not warrant
> > that this transmission is virus-free and will not be liable for any
> > damages resulting from any virus transmitted. On all offers and
> > agreements under which Atos Nederland B.V. supplies goods and/or
> > services of whatever nature, the Terms of Delivery from Atos Nederland
> B.V. exclusively apply.
> > The Terms of Delivery shall be promptly submitted to you on your request.
> >
> This e-mail and the documents attached are confidential and intended
> solely for the addressee; it may also be privileged. If you receive this
> e-mail in error, please notify the sender immediately and destroy it. As
> its integrity cannot be secured on the Internet, Atos’ liability cannot be
> triggered for the message content. Although the sender endeavours to
> maintain a computer virus-free network, the sender does not warrant that
> this transmission is virus-free and will not be liable for any damages
> resulting from any virus transmitted. On all offers and agreements under
> which Atos Nederland B.V. supplies goods and/or services of whatever
> nature, the Terms of Delivery from Atos Nederland B.V. exclusively apply.
> The Terms of Delivery shall be promptly submitted to you on your request.
>
