In view of the responses so far, it looks like the only 'reasonable'
(immediate, cheap, self-realizable) solution is to transfer fixed length
records. To summarize suggested options:
-- IBM's TERSE/UNTERSE program, which has 'always' been freely available
via download to all mainframe customers but is now incorporated into z/OS,
which means full support and automatic maintenance
-- TSO XMIT/RECEIVE, which has 'always' been incorporated in z/OS and its
ancestors
The first yields a compressed file at the expense of perhaps considerable
CPU cycles on both ends, which may or may not be worth the cost given
today's network speeds.
The second involves little overhead and yields a NETDATA file that could
even be used on VM/CMS , which may or may not ever be useful.
Unless you're shipping gigantic files that would benefit from compression,
I would go with XMIT/RECEIVE because the actual transmitted file can be
'browsed' to determine original file name and attributes. A tersed file
looks like garbage until it's untersed.
Either method supports both FB and VB customer data, so a single process
would work with any type of file, reducing complexity and training needs.
As a far-out bonus, you could even transfer IDCAMS unloaded VSAM
files--including catalogs if you were so motivated.
Trying to get FTP to handle other than FB files in the realistic future
seems far more trouble the effort is worth.
.
.
JO.Skip Robinson
Southern California Edison Company
Electric Dragon Team Paddler
SHARE MVS Program Co-Manager
626-302-7535 Office
323-715-0595 Mobile
[EMAIL PROTECTED]
Bruce Baxter
<[EMAIL PROTECTED]
.STATE.NY.US> To
Sent by: IBM [email protected]
Mainframe cc
Discussion List
<[EMAIL PROTECTED] Subject
.EDU> Re: File Transfer conundrum
01/09/2008 08:13
PM
Please respond to
IBM Mainframe
Discussion List
<[EMAIL PROTECTED]
.EDU>
ftp directly between the two systems, quite simply isn't an option. Our
business partner chose the mechanism in palce to deal with multiple other
shops, and they're not likely to want to do this sort of one-off thing for
us.
I also took some of the previous comments regarding the RDW site command
(which I'd seen before) and did some testing. As stated by another poster,
there doesn't seem to be any way to get it to honor this and strip it off
on
the final FTP.
I also have to deal with the issue that EITHER the EBCDIC code pages on
either end don't match (less likely) or the ASCII code pages don't match
(more
likely) since the end user has subsequently reported to me that they've
seen
some data corruption due to the translation. I guess they were expecting a
cent sign character in someof their data.
On Wed, 9 Jan 2008 21:51:23 -0500, John S. Giltner, Jr.
<[EMAIL PROTECTED]> wrote:
>Any reason why you can't ftp directly between the two z/OS system?
>
>If security is an issue you could use either IPSec tunnels between the
>two systems or setup IBM SecureFTP server (SSL'ed FTP).
>
>
>
>Bruce Baxter wrote:
>> We've routinely exhanged files with business partners running on z/OS
>> machines using tape for years.
>>
>> We're now in the process of converting a number of these to electronic
>> means, using in part FTP. This is being done at the behest of one of
our
>> business partners, who (IMHO) hasn't thought through all the issues that
the
>> use of FTP introduces in this process. The central issue as I see it is
that
the
>> mainframes at either end of the pipeline are both EBCDIC and record
oriented,
>> and the servers and ftp processes that lie between them to facilitate
these
>> transfers do not have any inherent concept of record oriented files like
the
>> mainframe.
>>
>> I'm going to treat FIXED BLOCK data separately from VARIABLE BLOCKED
data
>> separately.
>>
>> The first files that we received were FIXED BLOCK, and had been
translated
>> from EBCDIC to ASCII, most likely at the first transfer of the file from
z/OS
to
>> an ASCII based server platform (either Windows or AIX). When they
arrived
>> on our z/OS system, we had issues of data corruption because the data
>> contained zoned decimal data. After some discussion, we agreed that
we'd
>> transfer these files in BINARY mode at all steps along the way. Thus,
all we
>> had to do was ensure that the LRECL used for the destination dataset on
>> z/OS was the same as the source dataset. This seems to be working OK.
>>
>> Most recently, we've been having problems with other files that are
VARIABLE
>> BLOCKED. We received the first of these files last week, transmitted
from
end-
>> to-end in BINARY mode. What we got was not at all what we expected.
>> We've discovered that the initial FTP from z/OS to the server stripped
off
all
>> information regarding record length and thus record delineation.
Because
>> there aren't any RDWs in front of every record, ftp doesn't know how
long
the
>> records are and just plunks the data into the destination dataset in
chunks
of
>> LRECL-4. I did a bunch of research on z/OS FTP and there doesn't appear
to
>> be any way to make it convey record length/delineation information to
and
>> ASCII platform other than to use ASCII mode. z/OS FTP appears to have
>> mechanisms for conveying this between two z/OS FTP systems, but that's
not
>> possible here. For the present time, we've had the file shipped with
the
initiatl
>> movement translating the data from EBCDIC to ASCII and all subsequent
>> transfers until the last one back to z/OS in BINARY mode. However, I'm
>> concerned about the possibility of data corruption if the translate
tables
used
>> in the first step and the last step of this files travel aren't exact
inversions
of
>> each other. This would certainly be possible of the initial ASCII
transfer
were
>> done to a Windows Code Page 1252 system and the last transfer (having
>> CP1252 data) were translating between UTF-8 and CP037.
>>
>> I'm interested in other folks war stories and what they've implemented
for
best
>> practices. I've made clear to our developers and end-users that ftp is
>> certainly not a direct replacement for tape transfers. It would appear
that
we
>> need lots of information about all the systems and transformations done
in
>> moving the file from one system to another. FTP doesn't convey this
sort
of
>> information in any way shape or form.
>>
>> What sort of options are there?
>>
>> - Transmit/Receive would certainly be one, but would add a lot of
overhead to
>> the process.
>> - Removal of all non-display data from the files and subjecting them to
ASCII
>> translation at every step would also be an option, but that would likely
be
>> rejected by our business partner as too much work.
>> - Are there any options to z/OS FTP that would allow record formatting
>> information to be conveyed in the file, if we presume that we'd transfer
it in
>> binary mode at every step.
>>
>> Has anyone come across any clear helpful best practice type information
or
>> sites? I'd be interested in anything anyone has.
>>
>> Regards,
>> Bruce Baxter
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html
