Re: Moving Data from z/OS to a desktop
Where can I find out more about this? Google was pretty vague... On Thu, Feb 18, 2010 at 7:03 PM, Dave Day david...@consolidated.net wrote: snip Dave, Have you looked at DFS/SMB share from z/OS? Then you can access that share from windows like any network shares. Natarajan snip Yes, that is the direction I'm heading with this now, thanks to Steve Thompson's post earlier, and a follow up ph conversation he and I had. It never crossed my mindI guess I have this ancient mindset that the two platforms are separate and difficult to get data from one to the other. Going to try creating the data in an HFS/ZFS file, then have the Windows platform read from it as a network drive. Should be an interesting project. Thanks. --Dave -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Moving Data from z/OS to a desktop
Try Distributed File Service SMB Administration (SC24-5918-05) and I also have a copy of pages from chap 8 of DFS/SMB for OS/390 -Original Message- From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of Donald Johnson Sent: Friday, February 19, 2010 10:40 To: IBM-MAIN@bama.ua.edu Subject: Re: Moving Data from z/OS to a desktop Where can I find out more about this? Google was pretty vague... On Thu, Feb 18, 2010 at 7:03 PM, Dave Day david...@consolidated.net wrote: snip Dave, Have you looked at DFS/SMB share from z/OS? Then you can access that share from windows like any network shares. Natarajan snip Yes, that is the direction I'm heading with this now, thanks to Steve Thompson's post earlier, and a follow up ph conversation he and I had. It never crossed my mindI guess I have this ancient mindset that the two platforms are separate and difficult to get data from one to the other. Going to try creating the data in an HFS/ZFS file, then have the Windows platform read from it as a network drive. Should be an interesting project. Thanks. --Dave -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Moving data from z/OS to a desktop
Or you could just present the information to the user, directly. (Insert words like dashboard and/or portal here.) With beautiful charts, tabular data, and other current information that everybody (with security access) can share and view at the same time, without the who's got the current numbers? problems that are inevitable otherwise. Of course, if the user insists (and has security access), he/she could download a snapshot of data from the portal in a Web-friendly format. Wouldn't that setup be vastly more business-useful? - - - - - Timothy Sipples IBM Consulting Enterprise Software Architect E-Mail: timothy.sipp...@us.ibm.com -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Moving Data from z/OS to a desktop
Great, Thanks! Don On Fri, Feb 19, 2010 at 11:50 AM, Field, Alan C. alan.c.fi...@supervalu.com wrote: Try Distributed File Service SMB Administration (SC24-5918-05) and I also have a copy of pages from chap 8 of DFS/SMB for OS/390 -Original Message- From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of Donald Johnson Sent: Friday, February 19, 2010 10:40 To: IBM-MAIN@bama.ua.edu Subject: Re: Moving Data from z/OS to a desktop Where can I find out more about this? Google was pretty vague... On Thu, Feb 18, 2010 at 7:03 PM, Dave Day david...@consolidated.net wrote: snip Dave, Have you looked at DFS/SMB share from z/OS? Then you can access that share from windows like any network shares. Natarajan snip Yes, that is the direction I'm heading with this now, thanks to Steve Thompson's post earlier, and a follow up ph conversation he and I had. It never crossed my mindI guess I have this ancient mindset that the two platforms are separate and difficult to get data from one to the other. Going to try creating the data in an HFS/ZFS file, then have the Windows platform read from it as a network drive. Should be an interesting project. Thanks. --Dave -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Moving Data from z/OS to a desktop
Dave, The z/OS 1.9 Information center has books classified under Distributed File Service, SMB, and zFS http://publib.boulder.ibm.com/infocenter/zos/v1r9/index.jsp?topic=/com.ibm.zos.r9.bpxb200/xterm.htm DFS Customization could be found at http://publibz.boulder.ibm.com/epubs/pdf/fcxd2a00.pdf Thanks Natarajan On 02/19/2010 08:40 AM, Donald Johnson wrote: Where can I find out more about this? Google was pretty vague... -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Moving data from z/OS to a desktop
Have a couple of questions that possibly someone who has been down this road before might be able to help with. Have an application that has both a mainframe and a desktop component. The z/OS side is the data generating side that creates some statistical data. The desktop is used for a GUI to display the data. The current state of the application is that the z/OS side creates some summary data in CSV format, and the desktop picks it up using FTP. I want to get away from summary data and FTP, and give the desktop application a 'closer look' at the data before it is summarized. The z/OS component already produces all of the data the desktop would need, but it is in character format. I could just open a pipe between the desktop and z/OS, and shove the data down the pipe. Desktop application is smart enough to parse out the data it needs from the records. But, the volume of records could get fairly large in some instances. Since it is character data, it probably lends itself well to some type of compression. Does anyone know of a compression product that runs on both platforms? 2nd question. The z/OS application could compute the amount of data to be transmitted over the pipe prior to creating the data. What affects the speed of transfer besides the size of the pipe? How important is the size of the data block on each PUT? Thanks in advance for any help on this. --Dave Day -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Moving data from z/OS to a desktop
What affects the speed of transfer besides the size of the pipe? How important is the size of the data block on each PUT? 1. What is the connectivity? VPN over satellite has one set of tradeoffs; direct Ethernet another. 2. Is there really *that* much data? A megabyte or two every couple of hours is not going to be a problem no matter how you do it (within reason). The best form of compression might be to restrict the data to only what the recipient needs, and to transmit it in a more compact form than character. But maintainability/debugability is often more important than absolute speed. Charles -Original Message- From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of Dave Day Sent: Thursday, February 18, 2010 8:02 AM To: IBM-MAIN@bama.ua.edu Subject: Moving data from z/OS to a desktop Have a couple of questions that possibly someone who has been down this road before might be able to help with. Have an application that has both a mainframe and a desktop component. The z/OS side is the data generating side that creates some statistical data. The desktop is used for a GUI to display the data. The current state of the application is that the z/OS side creates some summary data in CSV format, and the desktop picks it up using FTP. I want to get away from summary data and FTP, and give the desktop application a 'closer look' at the data before it is summarized. The z/OS component already produces all of the data the desktop would need, but it is in character format. I could just open a pipe between the desktop and z/OS, and shove the data down the pipe. Desktop application is smart enough to parse out the data it needs from the records. But, the volume of records could get fairly large in some instances. Since it is character data, it probably lends itself well to some type of compression. Does anyone know of a compression product that runs on both platforms? 2nd question. The z/OS application could compute the amount of data to be transmitted over the pipe prior to creating the data. What affects the speed of transfer besides the size of the pipe? How important is the size of the data block on each PUT? Thanks in advance for any help on this. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Moving data from z/OS to a desktop
Oh! Also, to answer the question you asked g. The zip format is quite universal. Windows contains free tools for zipping and unzipping and there are also several paid vendors, of which WinZip comes first to mind. Data 21 (and perhaps others) markets a compatible product for z/OS and VSE: http://www.data21.com/products/zip/compression.asp There is also the tar format which is more UNIX-ey and open-sourcey but I am less familiar with it. (However, the Google knows it well.) You could also ask John McKown for a copy of his RLE compression. (Mostly kidding.) Compression is going to complicate your life somewhat, and has its own tradeoffs of both elapsed and CPU time. Charles -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Moving data from z/OS to a desktop
There are also some inconsistencies in the various compression products across systems (e.g. zip format may not be decompressable by all products that claim zip compatibility and there may be differences in the implementation on different hosts). Of course there may also be translation issues to work out too. -Original Message- Charles Mills Oh! Also, to answer the question you asked g. The zip format is quite universal. Windows contains free tools for zipping and unzipping and there are also several paid vendors, of which WinZip comes first to mind. Data 21 (and perhaps others) markets a compatible product for z/OS and VSE: http://www.data21.com/products/zip/compression.asp There is also the tar format which is more UNIX-ey and open-sourcey but I am less familiar with it. (However, the Google knows it well.) You could also ask John McKown for a copy of his RLE compression. (Mostly kidding.) Compression is going to complicate your life somewhat, and has its own tradeoffs of both elapsed and CPU time. Charles -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Moving data from z/OS to a desktop
-Original Message- From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of Charles Mills Sent: Thursday, February 18, 2010 1:24 PM To: IBM-MAIN@bama.ua.edu Subject: Re: Moving data from z/OS to a desktop Oh! Also, to answer the question you asked g. The zip format is quite universal. Windows contains free tools for zipping and unzipping and there are also several paid vendors, of which WinZip comes first to mind. Data 21 (and perhaps others) markets a compatible product for z/OS and VSE: http://www.data21.com/products/zip/compression.asp There is also the tar format which is more UNIX-ey and open-sourcey but I am less familiar with it. (However, the Google knows it well.) You could also ask John McKown for a copy of his RLE compression. (Mostly kidding.) Well, that __would__ be nasty grin. The code is, at best, tacky. Compression is going to complicate your life somewhat, and has its own tradeoffs of both elapsed and CPU time. Charles Iffen it were me. I'd likely use gzip. Or even the very old UNIX compress. zip is not really designed for on the fly compression and decompression. compress and gzip are for stream compression. I sometimes use them myself like: gzip file | ssh u...@remote 'gunzip file' when I run over a slow link. That compresses the source file, piping the data, as it is being created, into ssh, which establishes a connection to remote as user, then pipes the data coming over into gunzip which write it to file, via redirection. In any of these cases, something need to do the translation from EBCDIC to ASCII. Either before the compression or after the decompress, but before writing. Like: gzip ebcdic.file | ssh u...@remote gunzip | iconv -f ibm-1047 -t iso-8859-1 |tr '\025' '\n' Note in the above, that the iconv messes up in that z/OS UNIX delimits with an NEL (0x15) instead of an LF. So on the ascii side, the 0x15 needs to be translated into an LF (\n). The ASCII file transfer does this for you. But a binary, followed by a normal iconv, does not. -- John McKown Systems Engineer IV IT Administrative Services Group HealthMarkets(r) 9151 Boulevard 26 * N. Richland Hills * TX 76010 (817) 255-3225 phone * (817)-961-6183 cell john.mck...@healthmarkets.com * www.HealthMarkets.com Confidentiality Notice: This e-mail message may contain confidential or proprietary information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. HealthMarkets(r) is the brand name for products underwritten and issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance Company(r), Mid-West National Life Insurance Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Moving data from z/OS to a desktop
zip format may not be decompressable by all products that claim zip compatibility and there may be differences in the implementation on different hosts). Of course there may also be translation issues to work out too. May. May! May? Do you have any specifics? I've worked at many shops using variouse ZIP products. There were few, if any, issues. You test things. You don't rely on 'may'. - Too busy driving to stop for gas! -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Moving data from z/OS to a desktop
What I was getting at was that you do need to test thoroughly and not rely on a vendors promise that it can be done. Preferably use a single vendor solution to cut down on pointing fingers if an issue does arise and as always be aware of mismatched levels of a product.. I had an incident recently where an outside client would zip a file, ftp it to our Mainframe (binary), I in turn ftp it to one of our local UNIX boxen (binary) where it gets unzipped. This process had been working fine for a year or so then suddenly starts to gag on a file. Ftp it from the Mainframe to my workstation and winzip unzip and it works fine. The UNIX gang eventually got a copy of PKWARE zip and all works fine now. I do not know what they were using to unzip before but it was not from PKWARE. I also do not know what the client was using. Unfortunately I don't have the specifics. Like most of us old timers when stuff starts going south and stays that way for awhile (in this case 2 weeks) they start calling in everyone to help debug and as I was the Mainframe guy I got pulled in to debug something I didn't even know existed. From a translation standpoint the usual A-z and 0-9 is safe but some special characters (brackets and braces in particular) have caused me some grief in the past. There were few, if any, issues. It only takes one. -Original Message- Ted MacNEIL zip format may not be decompressable by all products that claim zip compatibility and there may be differences in the implementation on different hosts). Of course there may also be translation issues to work out too. May. May! May? Do you have any specifics? I've worked at many shops using variouse ZIP products. There were few, if any, issues. You test things. You don't rely on 'may'. - Too busy driving to stop for gas! -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Moving data from z/OS to a desktop
What I was getting at was that you do need to test thoroughly and not rely on a vendors promise that it can be done. That's a different statement than 'may'. And, unless I missed it, you said nothing about testing. - Too busy driving to stop for gas! -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Moving data from z/OS to a desktop
OpenSSH will also do compression itself (using zlib's LZ algorighm): cat file | ssh -oCompression=yes u...@remote 'cat file' You can also set this option on when using sftp. Kirk Wolf Dovetailed Technologies http://dovetail.com On Thu, Feb 18, 2010 at 1:51 PM, McKown, John john.mck...@healthmarkets.com wrote: snip gzip file | ssh u...@remote 'gunzip file' when I run over a slow link. That compresses the source file, piping the data, as it is being created, into ssh, which establishes a connection to remote as user, then pipes the data coming over into gunzip which write it to file, via redirection. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Moving data from z/OS to a desktop
On Thu, 18 Feb 2010 20:08:22 +, Ted MacNEIL wrote: zip format may not be decompressable by all products that claim zip compatibility and there may be differences in the implementation on different hosts). Of course there may also be translation issues to work out too. May. May! May? Do you have any specifics? I've worked at many shops using variouse ZIP products. There were few, if any, issues. You test things. You don't rely on 'may'. Lately, zipping tools have mostly settled on the Deflate codec as a de-facto standard. Even gunzip will extract zip archives containing exactly one file and using Deflate. Older tools may use older techniques, and even mix techniques depending on data statistics (most frequently, in the archive TOC, you see a mixture of deflated and stored). So, even testing doesn't protect absolutely against that idiosyncratic data set that triggers a different codec. I know no zip tool that lets the programmer direct the selection of codec. -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Moving data from z/OS to a desktop
On Thu, 18 Feb 2010 13:51:13 -0600, McKown, John wrote: In any of these cases, something need to do the translation from EBCDIC to ASCII. Either before the compression or after the decompress, but before writing. Like: gzip ebcdic.file | ssh u...@remote gunzip | iconv -f ibm-1047 -t iso-8859-1 |tr '\025' '\n' Note in the above, that the iconv messes up in that z/OS UNIX delimits with an NEL (0x15) instead of an LF. So on the ascii side, the 0x15 needs to be translated into an LF (\n). The ASCII file transfer does this for you. But a binary, followed by a normal iconv, does not. In fact, z/OS Unix iconv undoes the LF - NEL (EBCDIC calls it NL) perversion. I consider this a bug. IBM should define a new code page with NEL at 0x25 and LF at 0x15 (say IBM-1047-2), and use that in the documentation, eschewing mention of the problematic IBM-1047 and its problematic footnote. (The experts tell me this would be unwise because it would not work correctly on the 3215 Selectric printer-keyboard. Bad History. OTOH, on occasion a Linux expert codes a translation table by the book, overlooking the footnote, and is dismayed when it doesn't work.) The reality should be documented in a code point matrix for a suitable code page, not merely in a footnote. VM/CMS PIPE TRANSLATE works (is broken) the other way for the (almost) equivalent pair of code pages, omitting the LF - NL perversion. One of them must be broken; neither will own up to being wrong. What does iconv -f IBM-1047 -t UTF-8 do with LF and NL? Again, Bad History. -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Moving data from z/OS to a desktop
True, I didn't explicitly state that you need to test. My humble apologies if I assume that the mindset of Mainframers is to ALWAYS TEST regardless of what a vendor states is compatible (or even functional). My bad. One of the posters implied that zip (among others) was a universal format with several vendors offering products. I merely wanted to point out that there may be inconsistencies in the various implementations so buyer beware. It is of course possible that I'm the only one who has ever come across any issues ;-). -Original Message- Ted MacNEIL What I was getting at was that you do need to test thoroughly and not rely on a vendors promise that it can be done. That's a different statement than 'may'. And, unless I missed it, you said nothing about testing. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Moving data from z/OS to a desktop
Sarcasm on Sarcasm off I assume testing, as well. I try not to make 'may' statements. I'm not trying to tell you how to respond to posts, but maybe; There are format issues that need to be tested, rather than the vague: there may be ,,,. --Original Message-- From: Ken Porowski Sender: IBM Mainframe Discussion List To: IBM Mainframe Discussion List ReplyTo: IBM Mainframe Discussion List Sent: Feb 18, 2010 16:46 Subject: Re: Moving data from z/OS to a desktop True, I didn't explicitly state that you need to test. My humble apologies if I assume that the mindset of Mainframers is to ALWAYS TEST regardless of what a vendor states is compatible (or even functional). My bad. One of the posters implied that zip (among others) was a universal format with several vendors offering products. I merely wanted to point out that there may be inconsistencies in the various implementations so buyer beware. It is of course possible that I'm the only one who has ever come across any issues ;-). -Original Message- Ted MacNEIL What I was getting at was that you do need to test thoroughly and not rely on a vendors promise that it can be done. That's a different statement than 'may'. And, unless I missed it, you said nothing about testing. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html - Too busy driving to stop for gas! -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Moving data from z/OS to a desktop
Dave, Have you looked at DFS/SMB share from z/OS? Then you can access that share from windows like any network shares. Natarajan NOTICE OF CONFIDENTIALITY The information contained in this communication, including but not limited to any accompanying document(s) and/or attachment(s), is privileged and confidential and is intended solely for the above-named individual(s). If you are not the intended recipient, please be advised that any distribution, copying, disclosure, and/or use of the information contained herein is strictly prohibited. If you received this communication in error, please destroy all copies of the communication, whether in electronic or hard copy format, and immediately contact the Security Office at EdFund at (916) 526-7539 or securityoff...@edfund.org. Thank you. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Moving Data from z/OS to a desktop
snip Dave, Have you looked at DFS/SMB share from z/OS? Then you can access that share from windows like any network shares. Natarajan snip Yes, that is the direction I'm heading with this now, thanks to Steve Thompson's post earlier, and a follow up ph conversation he and I had. It never crossed my mindI guess I have this ancient mindset that the two platforms are separate and difficult to get data from one to the other. Going to try creating the data in an HFS/ZFS file, then have the Windows platform read from it as a network drive. Should be an interesting project. Thanks. --Dave -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
