[
https://issues.apache.org/jira/browse/SQOOP-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Teoh updated SQOOP-3326:
------------------------------
Description:
The FTP listing will automatically assume the first file in the listing is the
most recent GDG file. This is a problem when there are mixed datasets in the
listing that the FTP listing doesn't filter these out.
GDG base is : HLQ.ABC.DEF.AB15HUP
The sequential dataset in the middle of the GDG member listing is :
HLQ.ABC.DEF.AB15HUP.DATA
The pattern for listing GDG members should be : <<GDG base name>>.G\d\{4}V\d\{2}
Sample below:-
{{ Menu Options View Utilities Compilers Help
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
DSLIST - Data Sets Matching HLQ.ABC.DEF.GDGBASE Row 1 of 8
Command ===> Scroll ===> PAGE
Command - Enter "/" to select action Message Volume
-------------------------------------------------------------------------------
HLQ.ABC.DEF.GDGBASE ??????
HLQ.ABC.DEF.GDGBASE.DUMMY SHT331
HLQ.ABC.DEF.GDGBASE.G0034V00 H19761
HLQ.ABC.DEF.GDGBASE.G0035V00 H81751
HLQ.ABC.DEF.GDGBASE.G0035V00.COPY SHT337
HLQ.ABC.DEF.GDGBASE.G0036V00 H73545
HLQ.ABC.DEF.GDGBASE.G0037V00 G10987
HLQ.ABC.DEF.GDGBASE.HELLO SHT33A
***************************** End of Data Set list ****************************
ftp> open some.machine.network.zxc.au
Connected to some.machine.network.zxc.au (11.22.33.44).
220-TCPFTP01 IBM FTP CS V2R1 at some.machine.network.zxc.au, 00:12:29 on
2018-05-29.
220 Connection will close if idle for more than 10 minutes.
Name (some.machine.network.zxc.au:someuser):
331 Send password please.
Password:
230 someuser is logged on. Working directory is "someuser.".
Remote system type is MVS.
ftp> cd 'HLQ.ABC.DEF.GDGBASE'
250 "HLQ.ABC.DEF.GDGBASE." is the working directory name prefix.
ftp> dir
227 Entering Passive Mode (11,22,33,44,55,66)
125 List started OK
Volume Unit Referred Ext Used Recfm Lrecl BlkSz Dsorg Dsname
H19761 Tape G0034V00
H81751 Tape G0035V00
H73545 Tape G0036V00
G10987 Tape G0037V00
SHT331 3390 **NONE** 1 15 VB 114 27998 PS DUMMY
SHT337 3390 **NONE** 1 15 VB 114 27998 PS G0035V00.COPY
SHT33A 3390 **NONE** 1 15 VB 114 27998 PS HELLO
250 List completed successfully.
ftp>}}
was:
The FTP listing will automatically assume the first file in the listing is the
most recent GDG file. This is a problem when there are mixed datasets in the
listing that the FTP listing doesn't filter these out.
GDG base is : HLQ.ABC.DEF.AB15HUP
The sequential dataset in the middle of the GDG member listing is :
HLQ.ABC.DEF.AB15HUP.DATA
The pattern for listing GDG members should be : <<GDG base name>>.G\d\{4}V\d\{2}
Sample below:-
Menu Options View Utilities Compilers Help
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
DSLIST - Data Sets Matching HLQ.ABC.DEF.GDGBASE Row 1 of 8
Command ===> Scroll ===> PAGE
Command - Enter "/" to select action Message Volume
-------------------------------------------------------------------------------
HLQ.ABC.DEF.GDGBASE ??????
HLQ.ABC.DEF.GDGBASE.DUMMY SHT331
HLQ.ABC.DEF.GDGBASE.G0034V00 H19761
HLQ.ABC.DEF.GDGBASE.G0035V00 H81751
HLQ.ABC.DEF.GDGBASE.G0035V00.COPY SHT337
HLQ.ABC.DEF.GDGBASE.G0036V00 H73545
HLQ.ABC.DEF.GDGBASE.G0037V00 G10987
HLQ.ABC.DEF.GDGBASE.HELLO SHT33A
***************************** End of Data Set list ****************************
ftp> open some.machine.network.zxc.au
Connected to some.machine.network.zxc.au (11.22.33.44).
220-TCPFTP01 IBM FTP CS V2R1 at some.machine.network.zxc.au, 00:12:29 on
2018-05-29.
220 Connection will close if idle for more than 10 minutes.
Name (some.machine.network.zxc.au:someuser):
331 Send password please.
Password:
230 someuser is logged on. Working directory is "someuser.".
Remote system type is MVS.
ftp> cd 'HLQ.ABC.DEF.GDGBASE'
250 "HLQ.ABC.DEF.GDGBASE." is the working directory name prefix.
ftp> dir
227 Entering Passive Mode (11,22,33,44,55,66)
125 List started OK
Volume Unit Referred Ext Used Recfm Lrecl BlkSz Dsorg Dsname
H19761 Tape G0034V00
H81751 Tape G0035V00
H73545 Tape G0036V00
G10987 Tape G0037V00
SHT331 3390 **NONE** 1 15 VB 114 27998 PS DUMMY
SHT337 3390 **NONE** 1 15 VB 114 27998 PS G0035V00.COPY
SHT33A 3390 **NONE** 1 15 VB 114 27998 PS HELLO
250 List completed successfully.
ftp>
> Mainframe FTP listing for GDG should filter out non-GDG datasets in a
> heterogeneous listing
> -------------------------------------------------------------------------------------------
>
> Key: SQOOP-3326
> URL: https://issues.apache.org/jira/browse/SQOOP-3326
> Project: Sqoop
> Issue Type: Improvement
> Reporter: Chris Teoh
> Assignee: Chris Teoh
> Priority: Minor
>
> The FTP listing will automatically assume the first file in the listing is
> the most recent GDG file. This is a problem when there are mixed datasets in
> the listing that the FTP listing doesn't filter these out.
>
> GDG base is : HLQ.ABC.DEF.AB15HUP
>
> The sequential dataset in the middle of the GDG member listing is :
> HLQ.ABC.DEF.AB15HUP.DATA
>
> The pattern for listing GDG members should be : <<GDG base
> name>>.G\d\{4}V\d\{2}
>
> Sample below:-
> {{ Menu Options View Utilities Compilers Help
>
>
> ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
> DSLIST - Data Sets Matching HLQ.ABC.DEF.GDGBASE Row 1 of 8
> Command ===> Scroll ===>
> PAGE
>
>
> Command - Enter "/" to select action Message
> Volume
>
> -------------------------------------------------------------------------------
> HLQ.ABC.DEF.GDGBASE ??????
> HLQ.ABC.DEF.GDGBASE.DUMMY SHT331
> HLQ.ABC.DEF.GDGBASE.G0034V00 H19761
> HLQ.ABC.DEF.GDGBASE.G0035V00 H81751
> HLQ.ABC.DEF.GDGBASE.G0035V00.COPY SHT337
> HLQ.ABC.DEF.GDGBASE.G0036V00 H73545
> HLQ.ABC.DEF.GDGBASE.G0037V00 G10987
> HLQ.ABC.DEF.GDGBASE.HELLO SHT33A
> ***************************** End of Data Set list
> ****************************
> ftp> open some.machine.network.zxc.au
> Connected to some.machine.network.zxc.au (11.22.33.44).
> 220-TCPFTP01 IBM FTP CS V2R1 at some.machine.network.zxc.au, 00:12:29 on
> 2018-05-29.
> 220 Connection will close if idle for more than 10 minutes.
> Name (some.machine.network.zxc.au:someuser):
> 331 Send password please.
> Password:
> 230 someuser is logged on. Working directory is "someuser.".
> Remote system type is MVS.
> ftp> cd 'HLQ.ABC.DEF.GDGBASE'
> 250 "HLQ.ABC.DEF.GDGBASE." is the working directory name prefix.
> ftp> dir
> 227 Entering Passive Mode (11,22,33,44,55,66)
> 125 List started OK
> Volume Unit Referred Ext Used Recfm Lrecl BlkSz Dsorg Dsname
> H19761 Tape G0034V00
> H81751 Tape G0035V00
> H73545 Tape G0036V00
> G10987 Tape G0037V00
> SHT331 3390 **NONE** 1 15 VB 114 27998 PS DUMMY
> SHT337 3390 **NONE** 1 15 VB 114 27998 PS G0035V00.COPY
> SHT33A 3390 **NONE** 1 15 VB 114 27998 PS HELLO
> 250 List completed successfully.
> ftp>}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
