On Wed, Jan 30, 2013 at 11:56 PM, Chase, John <[email protected]> wrote: >> -----Original Message----- >> From: Linux on 390 Port On Behalf Of John McKown >> >> Well, I know that downloading the 160 Gig uncompressed data takes about 8 >> hours on the 10 Gig/sec >> Ethernet connection. I then bzip2 compress that to about 50 Meg. Which I >> binary upload back to z/OS >> for safety (since it's setting on my Linux desktop) in just a few minutes. > > Wow! That data must contain at least 100G of blanks! (You're talking about > IRRADU00 output, right?)
Correct. And yes, there tons of blanks. > > Why not bzip2 it on z/OS, then download that bzip2 file? Should save one FTP > "trip". How does the z/OS CPU usage compare between that, and the download / > upload? How much is your "wall clock" time worth? The entire reason to do this on Linux is to minimize the z/OS CPU usage. I can basically do whatever I want on Linux because it is not monitored. Increasing the z/OS CPU usage increases our bill. Real money. My time on this is "free". Especially since I do the download using Co:Z Hybrid batch on z/OS on Sunday (low-use) to run Linux commands, one of which is to do the ftp on Linux to the z/OS ftp server. Weird, but I can't run an ftp server on Linux because it is not approved and I got "tagged" once in a network scan. They don't scan for SSH because they are Windows oriented. "What is SSH????". > >> But bzgrep can scan the compressed file fairly quickly (FSVO "quickly"). So, >> if I use Perl or C to >> read the compressed file, expand it on-the-fly and process the records, then >> recompress the data >> before writing it to the proper result file (which is what I eventually want >> anyway), that should >> greatly reduce the I/O. I need to see how easy it is to do that in Perl vs >> Python vs C. Again, this is >> all for my personal use so that I can answer questions more quickly about >> "who did what to whom and >> how often? <grin/>" according to RACF. > > As far as "itemizing" the data, how would "itemizing" it on z/OS with > DFSORT/ICETOOL (Syncsort/Synctool or other ISV sort/xxxtool) first, then > bzip2-ing and downloading the resulting "itemized" file(s) compare to your > current method? And since your last statement above suggests you're > primarily interested in RACF commands issued (presumably FAR fewer than > logons, access checks, etc.), you could limit your "itemization", compression > and download to just that data. Again, the entire point of my doing this is to get it off of z/OS. Doing a ftp transfer to Linux is not very CPU intensive and so is not noticed in amongst all the other ftp transfers that we do. Using my time is regarded as "free" because I am a fixed cost item. And this information is not used in a production process. I'm basically doing this JUST IN CASE somebody has a question, because it allows me to give an ad-hoc answer much faster than running an ICETOOL against the data on z/OS (which is on virtual tape). IOW, this is a "skunk works" project of mine. Nobody else cares. Until somebody up the food chain wants an answer. And they want it NOW!!!! > > -jc- > > ********************************************************************** > Information contained in this e-mail message and in any attachments thereto > is confidential. If you are not the intended recipient, please destroy this > message, delete any copies held on your systems, notify the sender > immediately, and refrain from using or disclosing all or any part of its > content to any other person. > > ---------------------------------------------------------------------- > For LINUX-390 subscribe / signoff / archive access instructions, > send email to [email protected] with the message: INFO LINUX-390 or visit > http://www.marist.edu/htbin/wlvindex?LINUX-390 > ---------------------------------------------------------------------- > For more information on Linux on System z, visit > http://wiki.linuxvm.org/ -- Maranatha! <>< John McKown ---------------------------------------------------------------------- For LINUX-390 subscribe / signoff / archive access instructions, send email to [email protected] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 ---------------------------------------------------------------------- For more information on Linux on System z, visit http://wiki.linuxvm.org/
