Re: Dumping SMF directly to TAPE
On Wed, 17 Oct 2007 15:39:26 +0200, R.S. <[EMAIL PROTECTED]> wrote: > Indeed, if you have a lot of SMF data (btw: how much SMF >do you have ? Just curious), I am not going to try and figure it out with 25 production / development LPARs (sandbox LPARs just dumps to DASD GDGs and we keep about 30 with no archiving). Suffices to say it's a very big number. I think most of the SMF configurations have entire 3390-3 or 9 volumes dedicated with a varying number of MANx data sets (3-5) defined and they can fill one up every 5 minutes. You can do the math. Some LPARs do dump to a PS-E (compact) DASD data set in order to keep up (see other thread on SMF logger). Which proves... one size does not fit all, even in our shop. But mostly it's because there is a mixture of environments from consolidations over the years, so different procedures in different sysplexes. > then it sounds reasonable to dump it to >virtual (!) tape directly. In case of temporary problems you have >another SYS1.MANx, in case of longer outages you can lose some data or >quickly change offload job. > Exactly. I think the horse is dead :-) Mark -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
Mark Zelden wrote:
[...]
To your points above... the VSM hardware is just as fully redundant as
DASD (it *is* DASD). The SL8500 library is fully redundant (but even
problems with the back end physical drives don't keep the VSM from
creating new output tapes and reading in tapes from the buffer).
The HSC CDS is a single point of failure... but it is on all that modern
DASD and it also has a secondary backup copy (which should be kept
on a separate controller if available). There is even a 3rd ('standby')
CDS if you are so inclined to run that way.
I'm not talking about physical corruption of CDS. We have mutiple copies
(2 active + 1 standby) to avoid this. I talk about "software reasons".
There are scenarios, which could require to restart all the HSC
instances. It is quite rare, but (again, from my limited experience) it
tend to happen. Indeed, if you have a lot of SMF data (btw: how much SMF
do you have ? Just curious), then it sounds reasonable to dump it to
virtual (!) tape directly. In case of temporary problems you have
another SYS1.MANx, in case of longer outages you can lose some data or
quickly change offload job.
--
Radoslaw Skorupka
Lodz, Poland
--
BRE Bank SA
ul. Senatorska 18
00-950 Warszawa
www.brebank.pl
Sd Rejonowy dla m. st. Warszawy
XII Wydzia Gospodarczy Krajowego Rejestru Sdowego,
nr rejestru przedsibiorców KRS 025237
NIP: 526-021-50-88
Wedug stanu na dzie 01.01.2007 r. kapita zakadowy BRE Banku SA (w caoci
opacony) wynosi 118.064.140 z. W zwizku z realizacj warunkowego
podwyszenia kapitau zakadowego, na podstawie uchwa XVI WZ z dnia 21.05.2003
r., kapita zakadowy BRE Banku SA moe ulec podwyszeniu do kwoty 118.760.528
z. Akcje w podwyszonym kapitale zakadowym bd w caoci opacone.
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
On Wed, 17 Oct 2007 12:51:59 +0200, R.S. <[EMAIL PROTECTED]> wrote:
>In fact, I met STK outages, but not DASD outages. Modern DASD is fully
>redundant, CF outage - that's why we have spare CF. Sysplex is
>redundant, but usually single HSC CDS set serves multiple sysplexes. We
>don't have spare STK HSC(*). (although it is theoretically possible).
I think we are going in circles here. Following your argument... nothing
important (which SMF isn't really IMO... but no one likes to lose it) should
ever go directly to virtual tape. I haven't been convinced by any past
experience to think that way. In the environments that create a huge
amount of SMF data, virtual tape is a convenient and quick way to
manage it. If it was dumped to disk, it still would need to be copied
to tape in a subsequent step / job, so that would be extra processing
that just isn't needed. Even if there was enough DASD set aside to
put it all on disk for a 24 hour period, that would add processing
time at the end of the day to get it back to tape. Now multiply that
by 25 LPARs.
To your points above... the VSM hardware is just as fully redundant as
DASD (it *is* DASD). The SL8500 library is fully redundant (but even
problems with the back end physical drives don't keep the VSM from
creating new output tapes and reading in tapes from the buffer).
The HSC CDS is a single point of failure... but it is on all that modern
DASD and it also has a secondary backup copy (which should be kept
on a separate controller if available). There is even a 3rd ('standby')
CDS if you are so inclined to run that way.
>
> From my *very* limited experience: Usually HSC outages are really short.
As you said.. occasionally HSC/VTCS has hiccuped (STC abended),
but it gets restarted and all is good.
>
>I agree, DB2 logs are much much more important, than SMF. That's why I
>would suggest having archlogs on DASD and HSM (interval) migration to
>tape. Similar process for SMF and other "logs". Asynchronous processes.
>
Nothing wrong with that. Buf for us, some LPARs in some environments
don't even have HSM running - but have lots of DB2 (on our SAP LPARs
z/OS is really just a DB2 back end for SAP on AIX). HSM could be
implemented there (I wish it was)... but never has been by the
DASD Geeks (tm). Other environments never implemented HSM migration
duplexing for DR (in the past) - another problem in that scenario.
OTOH, we have fully duplexed all our virtual tape for DR since day 1 of
using VSM. DR was one of the main factors that led to getting rid of
VTS when we did that (many years ago now).
So fast forward 6 or 7 years to the present... and the landscape is
different and things could change, but since we've never had a reason
to change it.
Mark
--
Mark Zelden
Sr. Software and Systems Architect - z/OS Team Lead
Zurich North America / Farmers Insurance Group - ZFUS G-ITO
mailto:[EMAIL PROTECTED]
z/OS and OS390 expert at http://searchDataCenter.com/ateExperts/
Systems Programming expert at http://expertanswercenter.techtarget.com/
Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
Mark Zelden wrote: [...] If that were to happen, SMF is the least of my problems. DB2 archives go to virtual tape and as soon as all of the alternate logs fill up, DB2 stops. We can create "what if" scenarios all day that will cause problems. By why limit those to virtual tape? We've had DASD controller failures, CF, OS software, ISV software, even entire CEC go belly up. So your not convincing me of anything. In fact, I met STK outages, but not DASD outages. Modern DASD is fully redundant, CF outage - that's why we have spare CF. Sysplex is redundant, but usually single HSC CDS set serves multiple sysplexes. We don't have spare STK HSC(*). (although it is theoretically possible). From my *very* limited experience: Usually HSC outages are really short. I agree, DB2 logs are much much more important, than SMF. That's why I would suggest having archlogs on DASD and HSM (interval) migration to tape. Similar process for SMF and other "logs". Asynchronous processes. (*) I'm aware I can have multiple instances of HSC tasks, but those task share CDSes. I can have more than one CDS, but sometimes there is a need to restart all the HSC address spaces. Just my $0.02 -- Radoslaw Skorupka Lodz, Poland -- BRE Bank SA ul. Senatorska 18 00-950 Warszawa www.brebank.pl Sd Rejonowy dla m. st. Warszawy XII Wydzia Gospodarczy Krajowego Rejestru Sdowego, nr rejestru przedsibiorców KRS 025237 NIP: 526-021-50-88 Wedug stanu na dzie 01.01.2007 r. kapita zakadowy BRE Banku SA (w caoci opacony) wynosi 118.064.140 z. W zwizku z realizacj warunkowego podwyszenia kapitau zakadowego, na podstawie uchwa XVI WZ z dnia 21.05.2003 r., kapita zakadowy BRE Banku SA moe ulec podwyszeniu do kwoty 118.760.528 z. Akcje w podwyszonym kapitale zakadowym bd w caoci opacone. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
On Tue, 16 Oct 2007 21:45:48 +0200, R.S. <[EMAIL PROTECTED]> wrote: >Mark Zelden wrote: >> On Tue, 16 Oct 2007 16:49:29 +0200, R.S. <[EMAIL PROTECTED]> wrote: >>> Can you use tapes when i.e. moving HSC CDS files ? >> >> Why would I need to move them? > >It was only example. Another example would be CDS backup. During the >backup one cannot use HSC functions. So, currently running tape >activities are still running, but no new allocation would occur. Am I >right with that ? > The allocation occurs, just not the virtual tape mount. :-) But that is correct. Our backup job runs in a "hot batch" service class and even with hundreds of thousands of virtual tapes defined it only runs for a couple of minutes. So that is not a problem at all. > >>> You can have multiple boxes, but usually you have only one NCS >>> environment (HSC, VTCS, LS). More precisely: possibly multiple HSC >>> instances sharing single set of CDSes. >>> >> >> I see where you are going... but what does that have to do >> with the price of tea in China (American saying)? If we need to >> reconfigure our DASD, we have an outage also.With the newer >> versions of Sun/STK software you can add / change just about >> anything on the fly (although we do have scheduled outage windows >> so we don't have to use those features). > >Such outage can be caused by variety of reasons. There still are cases >when total HSC shutdown is required, I also could imagine simple plain >errors like software failure, or CDS corruption. This is quite unlikely, >this is rare, but it can happen. > If that were to happen, SMF is the least of my problems. DB2 archives go to virtual tape and as soon as all of the alternate logs fill up, DB2 stops. We can create "what if" scenarios all day that will cause problems. By why limit those to virtual tape? We've had DASD controller failures, CF, OS software, ISV software, even entire CEC go belly up. So your not convincing me of anything. Mark -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS and OS390 expert at http://searchDataCenter.com/ateExperts/ Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
Mark Zelden wrote: On Tue, 16 Oct 2007 16:49:29 +0200, R.S. <[EMAIL PROTECTED]> wrote: Can you use tapes when i.e. moving HSC CDS files ? Why would I need to move them? It was only example. Another example would be CDS backup. During the backup one cannot use HSC functions. So, currently running tape activities are still running, but no new allocation would occur. Am I right with that ? You can have multiple boxes, but usually you have only one NCS environment (HSC, VTCS, LS). More precisely: possibly multiple HSC instances sharing single set of CDSes. I see where you are going... but what does that have to do with the price of tea in China (American saying)? If we need to reconfigure our DASD, we have an outage also.With the newer versions of Sun/STK software you can add / change just about anything on the fly (although we do have scheduled outage windows so we don't have to use those features). Such outage can be caused by variety of reasons. There still are cases when total HSC shutdown is required, I also could imagine simple plain errors like software failure, or CDS corruption. This is quite unlikely, this is rare, but it can happen. -- Radoslaw Skorupka Lodz, Poland -- BRE Bank SA ul. Senatorska 18 00-950 Warszawa www.brebank.pl Sd Rejonowy dla m. st. Warszawy XII Wydzia Gospodarczy Krajowego Rejestru Sdowego, nr rejestru przedsibiorców KRS 025237 NIP: 526-021-50-88 Wedug stanu na dzie 01.01.2007 r. kapita zakadowy BRE Banku SA (w caoci opacony) wynosi 118.064.140 z. W zwizku z realizacj warunkowego podwyszenia kapitau zakadowego, na podstawie uchwa XVI WZ z dnia 21.05.2003 r., kapita zakadowy BRE Banku SA moe ulec podwyszeniu do kwoty 118.760.528 z. Akcje w podwyszonym kapitale zakadowym bd w caoci opacone. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
On Tue, 16 Oct 2007 16:49:29 +0200, R.S. <[EMAIL PROTECTED]> wrote: > >Can you use tapes when i.e. moving HSC CDS files ? Why would I need to move them? >You can have multiple boxes, but usually you have only one NCS >environment (HSC, VTCS, LS). More precisely: possibly multiple HSC >instances sharing single set of CDSes. > I see where you are going... but what does that have to do with the price of tea in China (American saying)? If we need to reconfigure our DASD, we have an outage also.With the newer versions of Sun/STK software you can add / change just about anything on the fly (although we do have scheduled outage windows so we don't have to use those features). Mark -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS and OS390 expert at http://searchDataCenter.com/ateExperts/ Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
Mark Zelden wrote: On Tue, 16 Oct 2007 15:54:50 +0200, R.S. <[EMAIL PROTECTED]> wrote: Mark Zelden wrote: On Mon, 15 Oct 2007 23:37:50 -0500, Joel C. Ewing <[EMAIL PROTECTED]> wrote: Every time you mount a tape there is a small but non-zero probability that the tape will be physically or logically damaged by a drive (or by an Operator or robot mishandling the tape). Not with virtual tape (which is what we offload to on those LPARs that do offload SMF directly to tape). No mount delay either. However, occasional tape sharing allocation problems (MIM/MIA) have held up the SMF offload processing on those LPARs. Unless you have to do some disruptive maintenance on your STK VTCS. With virtually 100% of our tape environment using virtual tape (no pun intended), that type of work is done when systems can be "quiesced". Besides, that we have redundancy in our virtual tape environment. The only thing that uses native tape is HSM and TSM. Can you use tapes when i.e. moving HSC CDS files ? You can have multiple boxes, but usually you have only one NCS environment (HSC, VTCS, LS). More precisely: possibly multiple HSC instances sharing single set of CDSes. -- Radoslaw Skorupka Lodz, Poland -- BRE Bank SA ul. Senatorska 18 00-950 Warszawa www.brebank.pl Sd Rejonowy dla m. st. Warszawy XII Wydzia Gospodarczy Krajowego Rejestru Sdowego, nr rejestru przedsibiorców KRS 025237 NIP: 526-021-50-88 Wedug stanu na dzie 01.01.2007 r. kapita zakadowy BRE Banku SA (w caoci opacony) wynosi 118.064.140 z. W zwizku z realizacj warunkowego podwyszenia kapitau zakadowego, na podstawie uchwa XVI WZ z dnia 21.05.2003 r., kapita zakadowy BRE Banku SA moe ulec podwyszeniu do kwoty 118.760.528 z. Akcje w podwyszonym kapitale zakadowym bd w caoci opacone. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
On Tue, 16 Oct 2007 15:54:50 +0200, R.S. <[EMAIL PROTECTED]> wrote: >Mark Zelden wrote: >> On Mon, 15 Oct 2007 23:37:50 -0500, Joel C. Ewing <[EMAIL PROTECTED]> wrote: >> >>> Every time you mount a tape there is a small but non-zero probability >>> that the tape will be physically or logically damaged by a drive (or by >>> an Operator or robot mishandling the tape). >> >> Not with virtual tape (which is what we offload to on those LPARs that >> do offload SMF directly to tape). No mount delay either. However, >> occasional tape sharing allocation problems (MIM/MIA) have held up >> the SMF offload processing on those LPARs. > >Unless you have to do some disruptive maintenance on your STK VTCS. > With virtually 100% of our tape environment using virtual tape (no pun intended), that type of work is done when systems can be "quiesced". Besides, that we have redundancy in our virtual tape environment. The only thing that uses native tape is HSM and TSM. Mark -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS and OS390 expert at http://searchDataCenter.com/ateExperts/ Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
Mark Zelden wrote: On Mon, 15 Oct 2007 23:37:50 -0500, Joel C. Ewing <[EMAIL PROTECTED]> wrote: Every time you mount a tape there is a small but non-zero probability that the tape will be physically or logically damaged by a drive (or by an Operator or robot mishandling the tape). Not with virtual tape (which is what we offload to on those LPARs that do offload SMF directly to tape). No mount delay either. However, occasional tape sharing allocation problems (MIM/MIA) have held up the SMF offload processing on those LPARs. Unless you have to do some disruptive maintenance on your STK VTCS. -- Radoslaw Skorupka Lodz, Poland -- BRE Bank SA ul. Senatorska 18 00-950 Warszawa www.brebank.pl Sd Rejonowy dla m. st. Warszawy XII Wydzia Gospodarczy Krajowego Rejestru Sdowego, nr rejestru przedsibiorców KRS 025237 NIP: 526-021-50-88 Wedug stanu na dzie 01.01.2007 r. kapita zakadowy BRE Banku SA (w caoci opacony) wynosi 118.064.140 z. W zwizku z realizacj warunkowego podwyszenia kapitau zakadowego, na podstawie uchwa XVI WZ z dnia 21.05.2003 r., kapita zakadowy BRE Banku SA moe ulec podwyszeniu do kwoty 118.760.528 z. Akcje w podwyszonym kapitale zakadowym bd w caoci opacone. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
On Mon, 15 Oct 2007 23:37:50 -0500, Joel C. Ewing <[EMAIL PROTECTED]> wrote: >Every time you mount a tape there is a small but non-zero probability >that the tape will be physically or logically damaged by a drive (or by >an Operator or robot mishandling the tape). Not with virtual tape (which is what we offload to on those LPARs that do offload SMF directly to tape). No mount delay either. However, occasional tape sharing allocation problems (MIM/MIA) have held up the SMF offload processing on those LPARs. Mark -- Mark Zelden Sr. Software and Systems Architect - z/OS Team Lead Zurich North America / Farmers Insurance Group - ZFUS G-ITO mailto:[EMAIL PROTECTED] z/OS and OS390 expert at http://searchDataCenter.com/ateExperts/ Systems Programming expert at http://expertanswercenter.techtarget.com/ Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
Every time you mount a tape there is a small but non-zero probability that the tape will be physically or logically damaged by a drive (or by an Operator or robot mishandling the tape). If that damage occurs near the load point, you could loose everything on the tape. If you have a process that mods repeatedly to the same tape volume over and over, you are raising the probability of that tape being a victim. Since LOGREC data would rarely be considered mission-critical, Murphy's Law has probably seen fit to spare you. In the case of non-critical data, there is nothing inherently wrong with what you are doing, and there will be no problems as long as the tape drive is functioning properly and the media doesn't have any problem that results in incorrect file positioning. With today's tape technology if would be very unlikely that a tape written with no fatal errors would not be readable afterwards - unless some intervening physical damage occurs to the media. The minor inconvenience of an unlikely loss of data that you can live without is not a big deal. For somewhat more important data it is a better practice to keep input and output tapes totally separate (e.g., copy the input tape to a new output tape and append the new data to the output tape, so that the process is repeatable as long as the input tape doesn't become damaged. For really critical tape data, you would also want to write two tapes at the same time, so you could recover from the alternate if at some later time the primary tape were found damaged when used as input. I don't know if these recommendations are formally documented anywhere, or are just one of those things that becomes reasonable after you've been burned once. Although DASD isn't subject to the same physical damage risk as tape, you still have risk of job step failures or termination from other causes, and use of DISP=MOD for DASD can make a failed job step very difficult to rerun. In a production environment one needs to be very cautious about use of MOD for either tape or DASD datasets because of restart issues. George Dranes wrote: In respect to MOD, are you just referring to moding SMF data to tape? I've successfully used mod (with ICEGENER) to tape for our LOGREC files for 17 years and have yet to have an issue with reading the tape. Actually we also have a few other files we mod to tape which have also never failed. I now understand the way to go for SMF is to use IFASMFD rather than ICEGENER to move data around and not to use mod but with respect to using mod to tape for other types of files I have yet to see an issue?? Am I just lucky or are these just extremely isolated situations where mod to tape fails? I could have very easily have missed it but has IBM made recommendations against using mod for tapes? Thanks for all of the help! -- Joel C. Ewing, Fort Smith, AR[EMAIL PROTECTED] -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
> On October 7,2007 12:14 AM George Dranes wrote: > > Actually our SMF dump job dumps to a daily tape which in the next step is > mod on to a weekly tape using IEBGENER (actually ICEGENER in our case). > This weekly tape is then concatenated to a monthly tape once a week. I > keep multiple generations of the daily and weekly files so I can easily > rebuild if > needed. Does this sound ok? > Here I've set up the SMF rollup system to dump to a DASD dataset from the MAN files as they fill up during the day. Then just after midnight the automated ops system issues a final I SMF command. The dump datasets are then rolled up to daily dataset on virtual tape. After the daily run on Sunday morning the dailies are rolled up to weekly dataset on physical tape. We don't roll up to monthly datasets but keep 106 (2 years) of weekly generations. We use IFASMFDP to do our copying from dump to daily to weekly. I've also written a little SAS routine to keep the input generations in ascending numeric order. The first step of the roll up job creates an IDCAMS LISTCAT listing of the input which is read by the SAS program that creates the input DD statements. I've also split the daily and weekly SMF records into multiple groups based on how we process - one tape has CICS (SMF 110), another has DB2 (SMF 100-102), another has WLC required records (SMF 30, 70-79, and 89), and the last one has everything else. * If you wish to communicate securely with Commerce Bank and its affiliates, you must log into your account under Online Services at http://www.commercebank.com or use the Commerce Bank Secure Email Message Center at https://securemail.commercebank.com NOTICE: This electronic mail message and any attached files are confidential. The information is exclusively for the use of the individual or entity intended as the recipient. If you are not the intended recipient, any use, copying, printing, reviewing, retention, disclosure, distribution or forwarding of the message or any attached file is not authorized and is strictly prohibited. If you have received this electronic mail message in error, please advise the sender by reply electronic mail immediately and permanently delete the original transmission, any attachments and any copies of this message from your computer system. * -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
--- Actually our SMF dump job dumps to a daily tape which in the next step is mod on to a weekly tape using IEBGENER (actually ICEGENER in our case). This weekly tape is then concatenated to a monthly tape once a week. I keep multiple generations of the daily and weekly files so I can easily rebuild if needed. Does this sound ok? Sounds reasonable to me, George. Here's what we did at Clearing: 1. Our IEFU29 exit issued a command to start the SMFDUMP proc. This proc used the old IBM "SMFDUMP" program to dump all non-empty SMF datasets to DASD GDS. Second step was to sort the resulting dataset into time/date sequence, using DFSORT and the SORT exits recommended for processing RMF data. Split-outs of certain records for reporting purposes was done in a third step, making COPIES of the required records. These "subset" datasets were deleted when the appropriate reports were generated. 2. Weekly, use DFSORT to MERGE all those GDS's, in time/date sequence, to a single dataset on tape, discarding certain records that we knew positively that we would not be using again, like 14/15 for TEMP datasets. We had a CLIST that we used to generate the actual MERGE JCL, based on the number of catalog entries for individual dump datasets. We chose to use DFSORT as much as possible, because IFASMFDP was so incredibly SLOW!! In spite of advice to the contrary, we never experienced a "broken record", even across a system "crash". We had to make a slight modification to the SMFDUMP program. With multiple images in a sysplex, we occaissionally had SMFDUMP starting on more than one image at once, so we modified the ENQ that it does (to prevent multiple instances running in parallel on one image) to include the image name, extracted from the SMFCA. This is what worked for us. YMMV. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
I am coming into this thread a bit late... I do not recall ever having a problem using DFSort to process SMF. I use DFSort all the time to spin off my CICS, DB2, Security records etc... It works just fine. I even have it where I use DFSort to keep my SMF data in its own file(s) based on the date of the SMF record, (i.e. "today's" data, yesterday, the day before, etc...) and what is "not processed" is kept in the SPIN dataset until the next time. HTH. On Mon Oct 8 8:56 , Martin Packer <[EMAIL PROTECTED]> sent: >BTW my point about our preferring IFASMFDP over DFSORT (perhaps ICEGENER) >was meant to be a little provocative. (And ONE person responded offline, >pointing out the contradiction between my statement just now and what I've >said in my blog.) > >The recommendation to use IFASMFDP rather than ICEGENER came from "our" >accumulated 25 years or so of folklore in my team (aggregated as probably >more than 100 person years). SOMEWHERE in that vast time range someone >must've hit a problem with ICEGENER. Possibly in fact it would've been a >failure to specify VBS. Whatever, our instructions became "don't use >DFSORT or IEBGENER, use IFASMFDP. > >Personally I've never hit the problem and DFSORT Development pointed out >to me a while back they DON'T have a problem with spanned records. And >haven't as far back as they can remember, which I think is a LONG time. >:-) > >So, more pointedly than my previous attempt, has anyone ever had a problem >with ICEGENER / DFSORT copying SMF? And as much to the point what about >IEBGENER? > >Thanks, Martin > >Martin Packer >Performance Consultant >IBM United Kingdom Ltd >+44-20-8832-5167 >+44-7802-245-584 >[EMAIL PROTECTED] > > > > > > > >Unless stated otherwise above: >IBM United Kingdom Limited - Registered in England and Wales with number >741598. >Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > > > > > >-- >For IBM-MAIN subscribe / signoff / archive access instructions, >send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO >Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
On Oct 8, 2007, at 2:56 AM, Martin Packer wrote: BTW my point about our preferring IFASMFDP over DFSORT (perhaps ICEGENER) was meant to be a little provocative. (And ONE person responded offline, pointing out the contradiction between my statement just now and what I've said in my blog.) The recommendation to use IFASMFDP rather than ICEGENER came from "our" accumulated 25 years or so of folklore in my team (aggregated as probably more than 100 person years). SOMEWHERE in that vast time range someone must've hit a problem with ICEGENER. Possibly in fact it would've been a failure to specify VBS. Whatever, our instructions became "don't use DFSORT or IEBGENER, use IFASMFDP. Martin, The one item that I can think of is that the IFASMDP offers to clear the dataset after dumping. I am not sure if you can do that with DFSORT. Ed -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
Tom Schmidt wrote: You mentioned that you dump directly to "a" tape dataset... isn't that putting an awful lot of faith and hope into one measely tape? I would, if I were doing that, be using 2 tapes written concurrently (just in case one of my tapes broke or otherwise failed). Call it "RAIT 0" (a la RAID 0) if you like. I would also be considering making the output tape blksize closer to 256K as you mentioned. RAIT 0 would be something like writing half of the records to one tape and the other half to another one in parallel. If I understand your suggestion correctly, what you suggest is concurrent backup on two tape drives. That should aptly be called "RAIT 1". -- Ulrich Boche SVA GmbH, Germany IBM Premier Business Partner -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
BTW my point about our preferring IFASMFDP over DFSORT (perhaps ICEGENER) was meant to be a little provocative. (And ONE person responded offline, pointing out the contradiction between my statement just now and what I've said in my blog.) The recommendation to use IFASMFDP rather than ICEGENER came from "our" accumulated 25 years or so of folklore in my team (aggregated as probably more than 100 person years). SOMEWHERE in that vast time range someone must've hit a problem with ICEGENER. Possibly in fact it would've been a failure to specify VBS. Whatever, our instructions became "don't use DFSORT or IEBGENER, use IFASMFDP. Personally I've never hit the problem and DFSORT Development pointed out to me a while back they DON'T have a problem with spanned records. And haven't as far back as they can remember, which I think is a LONG time. :-) So, more pointedly than my previous attempt, has anyone ever had a problem with ICEGENER / DFSORT copying SMF? And as much to the point what about IEBGENER? Thanks, Martin Martin Packer Performance Consultant IBM United Kingdom Ltd +44-20-8832-5167 +44-7802-245-584 [EMAIL PROTECTED] Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
George: The SMF dumping process is a little more complicated than most jobs. IMO it needs a little more TLC than any typical job due to the complexity of what things can happen and the "frequency" in which it occurs. On Oct 7, 2007, at 8:46 AM, George Dranes wrote: I have decided to dump the initial live SMF to dasd rather than tape. So only 2 tape mounts would be required in this second step. We do have our job scheduler monitoring these dump jobs so if there is a failure we are paged and as I said before we only dump once a day, easy for the operator to keep an eye on it. Also, as I mentioned earlier, I keep multiple GDSs of the daily (along with HSM backups now) and weekly files so I could easily rebuild due to an error. We also keep monthly tapes which I will also have to convert from IEBGENER. I do keep a backup copy of the monthlies. Does this sound a little cleaner? -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
On Oct 7, 2007, at 8:46 AM, George Dranes wrote: I have decided to dump the initial live SMF to dasd rather than tape. So only 2 tape mounts would be required in this second step. We do have our job scheduler monitoring these dump jobs so if there is a failure we are paged and as I said before we only dump once a day, easy for the operator to keep an eye on it. Also, as I mentioned earlier, I keep multiple GDSs of the daily (along with HSM backups now) and weekly files so I could easily rebuild due to an error. We also keep monthly tapes which I will also have to convert from IEBGENER. I do keep a backup copy of the monthlies. Does this sound a little cleaner? George, Sorry I had a longer reply set up but my poor system crashed. I will shorted my reply to this put a volcount into the output of your run. There is dumpsmfxy program that (was?) supplied by IBM that makes life a little easier (loook on the CBTTAPE.ORG for it in case IBM has dropped it. As other has said it depends on the importance of your SMF data, but remember this if you need it next year to prove you need a faster processor and you don't have it tough luck. Its a reasonable item to keep safe for at least a year (or maybe longer). Ed -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
In respect to MOD, are you just referring to moding SMF data to tape? I've successfully used mod (with ICEGENER) to tape for our LOGREC files for 17 years and have yet to have an issue with reading the tape. Actually we also have a few other files we mod to tape which have also never failed. I now understand the way to go for SMF is to use IFASMFD rather than ICEGENER to move data around and not to use mod but with respect to using mod to tape for other types of files I have yet to see an issue?? Am I just lucky or are these just extremely isolated situations where mod to tape fails? I could have very easily have missed it but has IBM made recommendations against using mod for tapes? Thanks for all of the help! -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
George Dranes wrote: I have our SMF dump jobs dumping directly to a tape dataset with LRECL=32760 and BLKKSIZE=32760,RECFM=VBS. Fortunately we are small enough to dump SMF once a day so there are no speed issues not dumping to DASD (saves a lot of dasd). I was just curious if this a a sound way of handling SMF? I've even considered making the output tape blksize something larger such as 256K but have always just stayed with the safe 32760. Thanks for any help! As far as I understand you don't need any help rather asking for opinions/comments. My $0.02: 1. Single tape is SPOF (Single Point Of Failure). How important are your SMF data ? 2. Appending data on tape, especially MOD to a dataset can result in damage of previously recorded data. Some shops even do not allow to append data on tapes (only within the job). 3. Since you're small shop, it wouldn't be big problem to keep current SMF archives on disk. Then use HSM to backup/migrate it. Such approach gives you flexibility, you have most recent files online, older on tape (or 2 tapes - managed by HSM), the oldest files are scratched according to policy you set up. HTH -- Radoslaw Skorupka Lodz, Poland -- BRE Bank SA ul. Senatorska 18 00-950 Warszawa www.brebank.pl Sd Rejonowy dla m. st. Warszawy XII Wydzia Gospodarczy Krajowego Rejestru Sdowego, nr rejestru przedsibiorców KRS 025237 NIP: 526-021-50-88 Wedug stanu na dzie 01.01.2007 r. kapita zakadowy BRE Banku SA (w caoci opacony) wynosi 118.064.140 z. W zwizku z realizacj warunkowego podwyszenia kapitau zakadowego, na podstawie uchwa XVI WZ z dnia 21.05.2003 r., kapita zakadowy BRE Banku SA moe ulec podwyszeniu do kwoty 118.760.528 z. Akcje w podwyszonym kapitale zakadowym bd w caoci opacone. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
I have decided to dump the initial live SMF to dasd rather than tape. So only 2 tape mounts would be required in this second step. We do have our job scheduler monitoring these dump jobs so if there is a failure we are paged and as I said before we only dump once a day, easy for the operator to keep an eye on it. Also, as I mentioned earlier, I keep multiple GDSs of the daily (along with HSM backups now) and weekly files so I could easily rebuild due to an error. We also keep monthly tapes which I will also have to convert from IEBGENER. I do keep a backup copy of the monthlies. Does this sound a little cleaner? -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
George, While this may work you do realize this would (may) mount 3 tapes. I think that you may find that dumping SMF to a disk dataset would be faster and less prone to hardware errors (drive or media). I would also suggest that j910 file (that you create daily) should be under catalog control (and put a few days buffer in it for long weekends and possibly vacations depending on who is going to be "monitoring" the job as issues seem to pop up on weekends . Ed On Oct 7, 2007, at 8:11 AM, George Dranes wrote: To avoid mod could I concatenate the input DDs including the last weekly GDS first along with the daily dump as follows crating a new weekly GDS? By the way this would be the second step in my live SMF dump job therefore the +1 for my input daily. //DUMPEXEC PGM=IFASMFDP //SYSPRINT DD SYSOUT=* //SYSINDD DUMMY //INDD1DD DSN=TSU.SM.J910.SMFWEEK.DATA(0),DISP=SHR //DD DSN=TSU.SM.J913.SMFDAY.DATA(+1),DISP=SHR //DUMPOUT DD DSN=TSU.SM.J910.SMFWEEK.DATA(+1),DISP=(,CATLG), // LRECL=32760,BLKSIZE=32760,RECFM=VBS,UNIT=ATL //SYSPRINT DD SYSOUT=* //SYSINDD * INDD(INDD1,OPTIONS(DUMP)) /* -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
To avoid mod could I concatenate the input DDs including the last weekly GDS first along with the daily dump as follows crating a new weekly GDS? By the way this would be the second step in my live SMF dump job therefore the +1 for my input daily. //DUMPEXEC PGM=IFASMFDP //SYSPRINT DD SYSOUT=* //SYSINDD DUMMY //INDD1DD DSN=TSU.SM.J910.SMFWEEK.DATA(0),DISP=SHR //DD DSN=TSU.SM.J913.SMFDAY.DATA(+1),DISP=SHR //DUMPOUT DD DSN=TSU.SM.J910.SMFWEEK.DATA(+1),DISP=(,CATLG), // LRECL=32760,BLKSIZE=32760,RECFM=VBS,UNIT=ATL //SYSPRINT DD SYSOUT=* //SYSINDD * INDD(INDD1,OPTIONS(DUMP)) /* -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
George, Personally I would stay away from "MOD". I can't begin to tell you how many hours I have spent trying to recover SMF data from MODed tapes. If SMF is important enough to save then don't trust MOD. Its not that the software doesn't work its the HARDWARE (tape or drive). I would trust making it a GDG and then bringing then individual entries in at ether daily (or weekly or ..) times. The once or twice times that I have had to repair issues because of catalog issues (or side issues) is a real time saver, IMO. Just remember SMF is reasonably important data and to drop a few days or a weeks worth could be a chance for you to have to find a new job opportunity:) Ed On Oct 7, 2007, at 7:37 AM, George Dranes wrote: So would the following work to mod my SMF files to a weekly tape? I'm not really familiar with IFASMFDP bit I assume the INDD can be a dumped dataset instead of a VSAM file? //DUMPEXEC PGM=IFASMFDP //SYSPRINT DD SYSOUT=* //SYSINDD DUMMY //INDD1DD DSN=TSU.SM.J913.SMFDAY.DATA(+1),DISP=SHR //DUMPOUT DD DSN=TSU.SM.J910.SMFWEEK.DATA(0),DISP=(MOD,KEEP), // LRECL=32760,BLKSIZE=32760,RECFM=VBS,UNIT=ATL //SYSPRINT DD SYSOUT=(R,TSU91$R1) //SYSINDD * INDD(INDD1,OPTIONS(ALL)) /* -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
I made a mistake in my SYSIN, it should read INDD(INDD1,OPTIONS(DUMP)). -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
So would the following work to mod my SMF files to a weekly tape? I'm not really familiar with IFASMFDP bit I assume the INDD can be a dumped dataset instead of a VSAM file? //DUMPEXEC PGM=IFASMFDP //SYSPRINT DD SYSOUT=* //SYSINDD DUMMY //INDD1DD DSN=TSU.SM.J913.SMFDAY.DATA(+1),DISP=SHR //DUMPOUT DD DSN=TSU.SM.J910.SMFWEEK.DATA(0),DISP=(MOD,KEEP), // LRECL=32760,BLKSIZE=32760,RECFM=VBS,UNIT=ATL //SYSPRINT DD SYSOUT=(R,TSU91$R1) //SYSINDD * INDD(INDD1,OPTIONS(ALL)) /* -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
George Dranes wrote: > Actually our SMF dump job dumps to a daily tape which in the next step is > mod on to a weekly tape using IEBGENER (actually ICEGENER in our case). > This weekly tape is then concatenated to a monthly tape once a week. I > keep multiple generations of the daily and weekly files so I can easily rebuild if > needed. Does this sound ok? IEBGENER / ICEGENER stands to break records. (No pun intended.) :-) For example the longer Type 30s and 42-6's. We strongly recommended - to OUR customers - not to use it for chucking SMF around (except in highly specialised circumstances). IFASMFDP is the one to use. (Presently I'll be writing up the Logger-based implementation in the SMF Management wiki as well as my developerWorks blog.) Cheers, Martin Martin Packer Performance Consultant IBM United Kingdom Ltd +44-20-8832-5167 +44-7802-245-584 [EMAIL PROTECTED] Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
Actually our SMF dump job dumps to a daily tape which in the next step is mod on to a weekly tape using IEBGENER (actually ICEGENER in our case). This weekly tape is then concatenated to a monthly tape once a week. I keep multiple generations of the daily and weekly files so I can easily rebuild if needed. Does this sound ok? -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Dumping SMF directly to TAPE
On Sat, 6 Oct 2007 16:30:52 -0500, George Dranes wrote: >I have our SMF dump jobs dumping directly to a tape dataset with >LRECL=32760 and BLKKSIZE=32760,RECFM=VBS. Fortunately we are small >enough to dump SMF once a day so there are no speed issues not dumping to >DASD (saves a lot of dasd). I was just curious if this a a sound way of >handling SMF? I've even considered making the output tape blksize something >larger such as 256K but have always just stayed with the safe 32760. You mentioned that you dump directly to "a" tape dataset... isn't that putting an awful lot of faith and hope into one measely tape? I would, if I were doing that, be using 2 tapes written concurrently (just in case one of my tapes broke or otherwise failed). Call it "RAIT 0" (a la RAID 0) if you like. I would also be considering making the output tape blksize closer to 256K as you mentioned. -- Tom Schmidt Madison, WI -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Dumping SMF directly to TAPE
I have our SMF dump jobs dumping directly to a tape dataset with LRECL=32760 and BLKKSIZE=32760,RECFM=VBS. Fortunately we are small enough to dump SMF once a day so there are no speed issues not dumping to DASD (saves a lot of dasd). I was just curious if this a a sound way of handling SMF? I've even considered making the output tape blksize something larger such as 256K but have always just stayed with the safe 32760. Thanks for any help! -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
