Re: Displaying Quiesced zFS files
I agree and out of pure morbid curiosity, at this point, did you happen to issue a 'D GRS,C'? And if so did Latch Number 87 come back, a quiesce latch for a zFS file system. Just curious if other zFS support may be lacking.. Mark Zelden [EMAIL PROTECTED] wrote: On Tue, 24 Jun 2008 09:41:13 -0500, Ramiro Camposagrado wrote: The following section was added to the DFS/SMB section of the PSP buckets for z/OS 1.4, 1.5, 1.6, and 1.7, as a result of a PMR that I have opened with zFS development in regards to this issue. I guess they are still working on the fix. 06/04/11 If a zfs filesystem is quiesced 04/11/2006? Over 2 years and still no relief? What about the 1.8 and 1.9 buckets (I didn't check)? Considering the push to zFS, this is really bad. Jeers for IBM on this one... Mark -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Displaying Quiesced zFS files
On Wed, 25 Jun 2008 05:55:16 -0700, Patrick Falcone [EMAIL PROTECTED] wrote: I agree and out of pure morbid curiosity, at this point, did you happen to issue a 'D GRS,C'? And if so did Latch Number 87 come back, a quiesce latch for a zFS file system. Just curious if other zFS support may be lacking.. D GRS,C ISG343I 09.15.40 GRS STATUS 670 NO ENQ RESOURCE CONTENTION EXISTS NO LATCH CONTENTION EXISTS D GRS,L,JOBNAME=ZFS ISG343I 09.15.53 GRS STATUS 672 LATCH DISPLAY FOR JOB ZFS NO LATCHES OWNED OR WAITED UPON D GRS,AN,WAITER ISG349I 09.20.55 GRS ANALYSIS 715 LONG WAITER ANALYSIS: ENTIRE SYSPLEX THERE ARE NO WAITING TASKS MATCHING THE INPUT SPECIFICATION D GRS,AN,BLOCKER ISG349I 09.21.03 GRS ANALYSIS 718 LONG BLOCKER ANALYSIS: ENTIRE SYSPLEX THERE ARE NO BLOCKING TASKS MATCHING THE INPUT SPECIFICATION D GRS,AN,DEPEND ISG349I 09.21.09 GRS ANALYSIS 734 DEPENDENCY ANALYSIS: ENTIRE SYSPLEX THERE ARE NO WAITING TASKS MATCHING THE INPUT SPECIFICATION -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Displaying Quiesced zFS files
Thanks Mark. You gotta wonder what happened to this support. It had to be working at some point, you would think. I wonder where it dropped. I'm laughing and I'm not gonna ask if there are any more commands that were issued or need to be for that matter. I promise! Mark Zelden [EMAIL PROTECTED] wrote: On Wed, 25 Jun 2008 05:55:16 -0700, Patrick Falcone wrote: I agree and out of pure morbid curiosity, at this point, did you happen to issue a 'D GRS,C'? And if so did Latch Number 87 come back, a quiesce latch for a zFS file system. Just curious if other zFS support may be lacking.. D GRS,C ISG343I 09.15.40 GRS STATUS 670 NO ENQ RESOURCE CONTENTION EXISTS NO LATCH CONTENTION EXISTS D GRS,L,JOBNAME=ZFS ISG343I 09.15.53 GRS STATUS 672 LATCH DISPLAY FOR JOB ZFS NO LATCHES OWNED OR WAITED UPON D GRS,AN,WAITER ISG349I 09.20.55 GRS ANALYSIS 715 LONG WAITER ANALYSIS: ENTIRE SYSPLEX THERE ARE NO WAITING TASKS MATCHING THE INPUT SPECIFICATION D GRS,AN,BLOCKER ISG349I 09.21.03 GRS ANALYSIS 718 LONG BLOCKER ANALYSIS: ENTIRE SYSPLEX THERE ARE NO BLOCKING TASKS MATCHING THE INPUT SPECIFICATION D GRS,AN,DEPEND ISG349I 09.21.09 GRS ANALYSIS 734 DEPENDENCY ANALYSIS: ENTIRE SYSPLEX THERE ARE NO WAITING TASKS MATCHING THE INPUT SPECIFICATION -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Displaying Quiesced zFS files
On Wed, 25 Jun 2008 08:11:17 -0700, Patrick Falcone [EMAIL PROTECTED] wrote: Thanks Mark. You gotta wonder what happened to this support. It had to be working at some point, you would think. I wonder where it dropped. It worked for HFS, never for zFS apparently (what was IBM thinking?). And since there is no open APAR (that I can find) that references the INCOROUT for D OMVS,F,E, does that mean a requirement is needed to fix what is BAD. I'm laughing and I'm not gonna ask if there are any more commands that were issued or need to be for that matter. I promise! You might not ask... but I was still trying. And I found one that should do the trick! D OMVS,W (D OMVS,WAITERS) D OMVS,W BPXO063I 10.30.08 DISPLAY OMVS 136 OMVS 000F ACTIVE OMVS=(M8) MOUNT LATCH ACTIVITY: NONE FILE SYSTEM LATCH ACTIVITY: NONE OTHER WAITING THREADS: USER ASID TCB PID AGE -- ZELDENM 00B4 0098959833554812 00.00.19 IS DOING: ZFS OpenCall / Osi Wait FILE: ixm (20,5398) FILE SYSTEM: SYS1.OMVS.RESM81.XML.ZFS It doesn't tell you the file system is quiesced, but it is a single operator command that I can communicate to the console operators, my team, and the WebSphere team that should give enough of a hint that a quiesced zFS could be the problem. Easy enough to ask an operator to do over the phone. Explaining zfsadm would be more difficult. I'm not sure the operators even have OMVS segments in some environments. Mark -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Displaying Quiesced zFS files
On Wed, 25 Jun 2008 11:03:07 -0500, Mark Zelden [EMAIL PROTECTED] wrote: You might not ask... but I was still trying. And I found one that should do the trick! D OMVS,W (D OMVS,WAITERS) D OMVS,W BPXO063I 10.30.08 DISPLAY OMVS 136 OMVS 000F ACTIVE OMVS=(M8) MOUNT LATCH ACTIVITY: NONE FILE SYSTEM LATCH ACTIVITY: NONE OTHER WAITING THREADS: USER ASID TCB PID AGE -- ZELDENM 00B4 0098959833554812 00.00.19 IS DOING: ZFS OpenCall / Osi Wait FILE: ixm (20,5398) FILE SYSTEM: SYS1.OMVS.RESM81.XML.ZFS It doesn't tell you the file system is quiesced, but it is a single operator command that I can communicate to the console operators, my team, and the WebSphere team that should give enough of a hint that a quiesced zFS could be the problem. Easy enough to ask an operator to do over the phone. Explaining zfsadm would be more difficult. I'm not sure the operators even have OMVS segments in some environments. Hey that's neat ! Thanks Bruno Sugliani zxnetconsult(at)free(dot)fr -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Displaying Quiesced zFS files
In a message dated 6/25/2008 11:16:09 A.M. Central Daylight Time, [EMAIL PROTECTED] writes: You might not ask... but I was still trying. And I found one that should do the trick! D OMVS,W (D OMVS,WAITERS) Wonder if it could be opened as 'Integrity APAR'? **Gas prices getting you down? Search AOL Autos for fuel-efficient used cars. (http://autos.aol.com/used?ncid=aolaut000507) -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Displaying Quiesced zFS files
On Mon, 23 Jun 2008 15:12:50 -0700, Patrick Falcone [EMAIL PROTECTED] wrote: Hi Mark, I'm not sure if this has been mentioned/considered but did you try 'D OMVS,F'? Yes. Mentioned in my OP (I wrote no OMVS display command, nor any MODIFY ZFS command ). Here is a maintenance zFS I just quiesced: ZFS20 ACTIVE RDWR 06/15/2008 L=32 NAME=SYS1.OMVS.RESM81.XML.ZFS 01.20.14Q=0 PATH=/servz18/usr/lpp/ixm AGGREGATE NAME=SYS1.OMVS.RESM81.XML.ZFS Here is one that is not: ZFS21 ACTIVE RDWR 06/15/2008 L=33 NAME=SYS1.OMVS.RESM81.PERLZFS 01.20.14Q=0 PATH=/servz18/usr/lpp/perl AGGREGATE NAME=SYS1.OMVS.RESM81.PERLZFS -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html Mark Zelden [EMAIL PROTECTED] wrote: Much to my dismay, there is no operator command I can find that shows a quiesced ZFS. No OMVS display command, nor any MODIFY ZFS command. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Displaying Quiesced zFS files
You would think that the returned *status* from 'D OMVS,F' would be 'quiesced' instead of 'active' but then again I just stumbled upon the below. 'D OMVS,F,e' where 'e' is exception. Mark Zelden [EMAIL PROTECTED] wrote: On Mon, 23 Jun 2008 15:12:50 -0700, Patrick Falcone wrote: Hi Mark, I'm not sure if this has been mentioned/considered but did you try 'D OMVS,F'? Yes. Mentioned in my OP (I wrote no OMVS display command, nor any MODIFY ZFS command ). Here is a maintenance zFS I just quiesced: ZFS 20 ACTIVE RDWR 06/15/2008 L=32 NAME=SYS1.OMVS.RESM81.XML.ZFS 01.20.14 Q=0 PATH=/servz18/usr/lpp/ixm AGGREGATE NAME=SYS1.OMVS.RESM81.XML.ZFS Here is one that is not: ZFS 21 ACTIVE RDWR 06/15/2008 L=33 NAME=SYS1.OMVS.RESM81.PERLZFS 01.20.14 Q=0 PATH=/servz18/usr/lpp/perl AGGREGATE NAME=SYS1.OMVS.RESM81.PERLZFS -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html Mark Zelden wrote: Much to my dismay, there is no operator command I can find that shows a quiesced ZFS. No OMVS display command, nor any MODIFY ZFS command. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Displaying Quiesced zFS files
On Tue, 24 Jun 2008 06:55:49 -0700, Patrick Falcone [EMAIL PROTECTED] wrote: You would think that the returned *status* from 'D OMVS,F' would be 'quiesced' instead of 'active' but then again I just stumbled upon the below. 'D OMVS,F,e' where 'e' is exception. Yes I tried that (I did look at the FM before my first post on this subject). Would you expect active to be an exception? :-) Mark -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Displaying Quiesced zFS files
On Tue, 24 Jun 2008 09:06:14 -0500, Mark Zelden [EMAIL PROTECTED] wrote: On Tue, 24 Jun 2008 06:55:49 -0700, Patrick Falcone [EMAIL PROTECTED] wrote: You would think that the returned *status* from 'D OMVS,F' would be 'quiesced' instead of 'active' but then again I just stumbled upon the below. 'D OMVS,F,e' where 'e' is exception. Yes I tried that (I did look at the FM before my first post on this subject). Would you expect active to be an exception? :-) Mark -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html The following section was added to the DFS/SMB section of the PSP buckets for z/OS 1.4, 1.5, 1.6, and 1.7, as a result of a PMR that I have opened with zFS development in regards to this issue. I guess they are still working on the fix. 06/04/11 If a zfs filesystem is quiesced, either as the result of a DFSMS backup operation or explicitly via a zfsadm quiesce command, the state of the filesystem as displayed by the USS 'df' or D OMVS,F commands will NOT show as QUIESCED. This is a known design issue which is between the LFS (USS) and the PFS (zFS) , and the development teams are currently investigating this. However at this time this is working as designed. Users must be aware that to accurately determine the current status of a zFS filesystem, they must use the provided zfs administration commands: zfsadm lsaggr zfsadm aggrinfo status of the zFS filesystems and will show the status of the filesystem as QUIESCED, if in fact the filesystem is currently in a QUIESCED state. This has become a point of contention recently as backups being taken using the DFSMS DUMP/RESTORE utility will QUIESCE zfs filesystems implicitly. If users are not aware of this activity, they could see temporary 'hang' situations and/or USS Latch contention as indicated by the presence of a BPXM056E errors message displayed to the operators console. To determine ifthe current state of the ZFS filesystems, user are requiredto use the zfsadm commands to accurately determine if ZFS filesystems are QUIESCED -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Displaying Quiesced zFS files
I just thought that *maybe* I could catch you not checking that *F*M! :-) If done from the driving system then I would venture this a problem since the *F*M states that the returned status should be quiesced. Mark Zelden [EMAIL PROTECTED] wrote: On Tue, 24 Jun 2008 06:55:49 -0700, Patrick Falcone wrote: You would think that the returned *status* from 'D OMVS,F' would be 'quiesced' instead of 'active' but then again I just stumbled upon the below. 'D OMVS,F,e' where 'e' is exception. Yes I tried that (I did look at the FM before my first post on this subject). Would you expect active to be an exception? :-) Mark -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Displaying Quiesced zFS files
On Tue, 24 Jun 2008 09:41:13 -0500, Ramiro Camposagrado [EMAIL PROTECTED] wrote: The following section was added to the DFS/SMB section of the PSP buckets for z/OS 1.4, 1.5, 1.6, and 1.7, as a result of a PMR that I have opened with zFS development in regards to this issue. I guess they are still working on the fix. 06/04/11 If a zfs filesystem is quiesced snip 04/11/2006? Over 2 years and still no relief? What about the 1.8 and 1.9 buckets (I didn't check)? Considering the push to zFS, this is really bad. Jeers for IBM on this one... Mark -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Displaying Quiesced zFS files
I agree... two years and counting... There is no mention of this update on the z/OS 1.8 or 1.9 buckets either. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Displaying Quiesced zFS files
I was just called about a problem on our WAS development LPAR that turned out to be a quiesced WAS config ZFS. Much to my dismay, there is no operator command I can find that shows a quiesced ZFS. No OMVS display command, nor any MODIFY ZFS command. The only hint at all is this message on the console: IOEZ00581E There are quiesced zFS aggregates This looks like it came in via PTF to help as an aid in diagnosing problems. Searching the syslog I see it in there once. This would have been expected anyway since one of the WAS sysprogs quiesced the zFS. They later unquiesced it also - unfortunately another WAS sysprog also quiesced it and I got involved when things were hung and they couldn't figure out why. So now to my question. Does anyone know a way to figure out how to display what zFS files are quiesced without going into z/OS UNIX and using zfsadm (like from the console)? If not... this looks like a job for... system REXX! Oh wait... can't get there from here (OMVS can't be used from sysrexx, correct?) :-( Mark -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Displaying Quiesced zFS files
On Mon, 23 Jun 2008 11:31:02 -0500, Mark Zelden [EMAIL PROTECTED] wrote: So now to my question. Does anyone know a way to figure out how to display what zFS files are quiesced without going into z/OS UNIX and using zfsadm (like from the console)? If not... this looks like a job for... system REXX! Oh wait... can't get there from here (OMVS can't be used from sysrexx, correct?) :-( Perhaps RXZFS ? described in sg246580 Although never i tried it but then it is rexx :-)) and it is zfsadm Bruno Sugliani zxnetconsult(at)free(dot)fr -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Displaying Quiesced zFS files
o now to my question. Does anyone know a way to figure out how to display what zFS files are quiesced without going into z/OS UNIX and using zfsadm (like from the You can always run the zfsadm command in batch. Jon L. Veilleux [EMAIL PROTECTED] (860) 636-2683 This e-mail may contain confidential or privileged information. If you think you have received this e-mail in error, please advise the sender by reply e-mail and then delete this e-mail immediately. Thank you. Aetna -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Displaying Quiesced zFS files
On Mon, 23 Jun 2008 12:54:17 -0400, Veilleux, Jon L [EMAIL PROTECTED] wrote: o now to my question. Does anyone know a way to figure out how to display what zFS files are quiesced without going into z/OS UNIX and using zfsadm (like from the You can always run the zfsadm command in batch. Yes. Sort of obvious. But I'm looking for an operator method. I could create an STC that runs bpxbtch and writes to /dev/console. Yuck! Might as well train the operator(s) to get into OMVS and issue the command: zfsadm lsaggr | grep QUI Mark -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Displaying Quiesced zFS files
Hi Mark, I'm not sure if this has been mentioned/considered but did you try 'D OMVS,F'? Mark Zelden [EMAIL PROTECTED] wrote: I was just called about a problem on our WAS development LPAR that turned out to be a quiesced WAS config ZFS. Much to my dismay, there is no operator command I can find that shows a quiesced ZFS. No OMVS display command, nor any MODIFY ZFS command. The only hint at all is this message on the console: IOEZ00581E There are quiesced zFS aggregates This looks like it came in via PTF to help as an aid in diagnosing problems. Searching the syslog I see it in there once. This would have been expected anyway since one of the WAS sysprogs quiesced the zFS. They later unquiesced it also - unfortunately another WAS sysprog also quiesced it and I got involved when things were hung and they couldn't figure out why. So now to my question. Does anyone know a way to figure out how to display what zFS files are quiesced without going into z/OS UNIX and using zfsadm (like from the console)? If not... this looks like a job for... system REXX! Oh wait... can't get there from here (OMVS can't be used from sysrexx, correct?) :-( Mark -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
