[jira] [Commented] (SQOOP-3224) Mainframe FTP transfer should have an option to use binary mode for transfer
[ https://issues.apache.org/jira/browse/SQOOP-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405022#comment-16405022 ] ASF GitHub Bot commented on SQOOP-3224: --- Github user fszabo2 commented on a diff in the pull request: https://github.com/apache/sqoop/pull/44#discussion_r175478607 --- Diff: src/java/org/apache/sqoop/mapreduce/mainframe/MainframeConfiguration.java --- @@ -33,4 +33,13 @@ public static final String MAINFRAME_INPUT_DATASET_TAPE = "mainframe.input.dataset.tape"; public static final String MAINFRAME_FTP_FILE_ENTRY_PARSER_CLASSNAME = "org.apache.sqoop.mapreduce.mainframe.MainframeFTPFileEntryParser"; + + public static final String MAINFRAME_FTP_TRANSFER_MODE = "mainframe.ftp.transfermode"; + + public static final String MAINFRAME_FTP_TRANSFER_MODE_ASCII = "ascii"; + + public static final String MAINFRAME_FTP_TRANSFER_MODE_BINARY = "binary"; + + // this is the buffer size used when doing binary ftp transfers from mainframe + public static final Integer MAINFRAME_FTP_TRANSFER_BINARY_BUFFER = 32760; --- End diff -- Is this value always a good choice? I would make it configurable (if it isn't already), and use this value as default if said configuration is not present, so the user has a choice here. > Mainframe FTP transfer should have an option to use binary mode for transfer > > > Key: SQOOP-3224 > URL: https://issues.apache.org/jira/browse/SQOOP-3224 > Project: Sqoop > Issue Type: Improvement >Reporter: Chris Teoh >Assignee: Chris Teoh >Priority: Minor > > Currently the mainframe FTP module is hard coded to use ascii transfer mode. > Propose a mainframe module flag to be able to change modes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SQOOP-3224) Mainframe FTP transfer should have an option to use binary mode for transfer
[
https://issues.apache.org/jira/browse/SQOOP-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405021#comment-16405021
]
ASF GitHub Bot commented on SQOOP-3224:
---
Github user fszabo2 commented on a diff in the pull request:
https://github.com/apache/sqoop/pull/44#discussion_r175484508
--- Diff: src/test/org/apache/sqoop/util/TestMainframeFTPClientUtils.java
---
@@ -362,4 +363,29 @@ public void
testPartitionedDatasetsShouldReturnAllFiles() {
Assert.fail(ioeString);
}
}
+ @Test
+ public void testBinaryTransferMode() throws IOException {
+final String EXPECTED_RESPONSE = "200 Representation type is Image";
+final int EXPECTED_RESPONSE_CODE = 200;
+when(mockFTPClient.login("user", "pssword")).thenReturn(true);
--- End diff --
is the typo "pssword" intentional? Though won't cause any trouble I guess...
> Mainframe FTP transfer should have an option to use binary mode for transfer
>
>
> Key: SQOOP-3224
> URL: https://issues.apache.org/jira/browse/SQOOP-3224
> Project: Sqoop
> Issue Type: Improvement
>Reporter: Chris Teoh
>Assignee: Chris Teoh
>Priority: Minor
>
> Currently the mainframe FTP module is hard coded to use ascii transfer mode.
> Propose a mainframe module flag to be able to change modes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SQOOP-3224) Mainframe FTP transfer should have an option to use binary mode for transfer
[
https://issues.apache.org/jira/browse/SQOOP-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405020#comment-16405020
]
ASF GitHub Bot commented on SQOOP-3224:
---
Github user fszabo2 commented on a diff in the pull request:
https://github.com/apache/sqoop/pull/44#discussion_r175483531
--- Diff: src/java/org/apache/sqoop/util/MainframeFTPClientUtils.java ---
@@ -207,8 +208,18 @@ public static FTPClient getFTPConnection(Configuration
conf)
throw new IOException("Could not login to server " + server
+ ":" + ftp.getReplyString());
}
- // set ASCII transfer mode
- ftp.setFileType(FTP.ASCII_FILE_TYPE);
+ // set transfer mode
+ String transferMode =
conf.get(MainframeConfiguration.MAINFRAME_FTP_TRANSFER_MODE);
--- End diff --
The whole getFTPConnection method is already very long and does a bunch of
steps. You could consider refactoring it into smaller methods for readability.
> Mainframe FTP transfer should have an option to use binary mode for transfer
>
>
> Key: SQOOP-3224
> URL: https://issues.apache.org/jira/browse/SQOOP-3224
> Project: Sqoop
> Issue Type: Improvement
>Reporter: Chris Teoh
>Assignee: Chris Teoh
>Priority: Minor
>
> Currently the mainframe FTP module is hard coded to use ascii transfer mode.
> Propose a mainframe module flag to be able to change modes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] sqoop pull request #44: [SQOOP-3224] Binary transfer mode
Github user fszabo2 commented on a diff in the pull request:
https://github.com/apache/sqoop/pull/44#discussion_r175484508
--- Diff: src/test/org/apache/sqoop/util/TestMainframeFTPClientUtils.java
---
@@ -362,4 +363,29 @@ public void
testPartitionedDatasetsShouldReturnAllFiles() {
Assert.fail(ioeString);
}
}
+ @Test
+ public void testBinaryTransferMode() throws IOException {
+final String EXPECTED_RESPONSE = "200 Representation type is Image";
+final int EXPECTED_RESPONSE_CODE = 200;
+when(mockFTPClient.login("user", "pssword")).thenReturn(true);
--- End diff --
is the typo "pssword" intentional? Though won't cause any trouble I guess...
---
[GitHub] sqoop pull request #44: [SQOOP-3224] Binary transfer mode
Github user fszabo2 commented on a diff in the pull request:
https://github.com/apache/sqoop/pull/44#discussion_r175483531
--- Diff: src/java/org/apache/sqoop/util/MainframeFTPClientUtils.java ---
@@ -207,8 +208,18 @@ public static FTPClient getFTPConnection(Configuration
conf)
throw new IOException("Could not login to server " + server
+ ":" + ftp.getReplyString());
}
- // set ASCII transfer mode
- ftp.setFileType(FTP.ASCII_FILE_TYPE);
+ // set transfer mode
+ String transferMode =
conf.get(MainframeConfiguration.MAINFRAME_FTP_TRANSFER_MODE);
--- End diff --
The whole getFTPConnection method is already very long and does a bunch of
steps. You could consider refactoring it into smaller methods for readability.
---
[GitHub] sqoop pull request #44: [SQOOP-3224] Binary transfer mode
Github user fszabo2 commented on a diff in the pull request: https://github.com/apache/sqoop/pull/44#discussion_r175478607 --- Diff: src/java/org/apache/sqoop/mapreduce/mainframe/MainframeConfiguration.java --- @@ -33,4 +33,13 @@ public static final String MAINFRAME_INPUT_DATASET_TAPE = "mainframe.input.dataset.tape"; public static final String MAINFRAME_FTP_FILE_ENTRY_PARSER_CLASSNAME = "org.apache.sqoop.mapreduce.mainframe.MainframeFTPFileEntryParser"; + + public static final String MAINFRAME_FTP_TRANSFER_MODE = "mainframe.ftp.transfermode"; + + public static final String MAINFRAME_FTP_TRANSFER_MODE_ASCII = "ascii"; + + public static final String MAINFRAME_FTP_TRANSFER_MODE_BINARY = "binary"; + + // this is the buffer size used when doing binary ftp transfers from mainframe + public static final Integer MAINFRAME_FTP_TRANSFER_BINARY_BUFFER = 32760; --- End diff -- Is this value always a good choice? I would make it configurable (if it isn't already), and use this value as default if said configuration is not present, so the user has a choice here. ---
[jira] [Commented] (SQOOP-3297) Unrecognized argument: --schema for sqoop job execution
[
https://issues.apache.org/jira/browse/SQOOP-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404859#comment-16404859
]
Fero Szabo commented on SQOOP-3297:
---
[~bbusi],
I don't exactly know what went south here, but recreating the job might help.
If that fails, or isn't an option, folks at Cloudera support should be able to
help you out (I noticed you are using CDH).
> Unrecognized argument: --schema for sqoop job execution
> ---
>
> Key: SQOOP-3297
> URL: https://issues.apache.org/jira/browse/SQOOP-3297
> Project: Sqoop
> Issue Type: Bug
>Reporter: Bhanu Busi
>Priority: Minor
>
> Hi,
>
> previously I have used CDH-5.10 and sqoop 1.4.6. For this combination - -
> schema argument working fine.
> Now I have upgraded cloudera manager to CDH-5.13.1 and sqoop 1.4.6. For this
> combination - - schema argument showing bellow error
> while executing the sqoop job I have faced the following issue.
> {noformat}
> sqoop job --exec X
> (256, 'Warning:
> /cloudera/opt/cloudera/parcels/CDH-5.13.1-1.cdh5.13.1.p0.2/bin/../lib/sqoop/../accumulo
> does not exist! Accumulo imports will fail.\nPlease set $ACCUMULO_HOME to
> the root of your Accumulo installation.\n18/03/19 05:05:40 INFO sqoop.Sqoop:
> Running Sqoop version: 1.4.6-cdh5.13.1\n18/03/19 05:05:41 ERROR
> tool.BaseSqoopTool: Error parsing arguments for import:\n18/03/19 05:05:41
> ERROR tool.BaseSqoopTool: Unrecognized argument: --schema\n18/03/19 05:05:41
> ERROR tool.BaseSqoopTool: Unrecognized argument: public\n\nTry --help for
> usage instructions.'){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SQOOP-3052) Introduce Maven/Gradle/etc. based build for Sqoop to make it more developer friendly / open
[ https://issues.apache.org/jira/browse/SQOOP-3052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Szonyi updated SQOOP-3052: --- Attachment: SQOOP-3052.patch > Introduce Maven/Gradle/etc. based build for Sqoop to make it more developer > friendly / open > --- > > Key: SQOOP-3052 > URL: https://issues.apache.org/jira/browse/SQOOP-3052 > Project: Sqoop > Issue Type: Improvement >Reporter: Attila Szabo >Assignee: Anna Szonyi >Priority: Major > Fix For: 1.5.0 > > Attachments: SQOOP-3052.patch > > > The current trunk version can only be build with Ant/Ivy combination, which > has some painful limitations (resolve is slow / needs to be tweaked to use > only caches, the current profile / variable based settings are not working in > IDEs out of the box, the current solution does not download the related > sources, etc.) > It would be nice to provide a solution, which would give the possibility for > the developers to choose between the nowadays well used build infrsturctures > (e.g. Maven, Gradle, etc.). For this solution it would be also essential to > keep the different build files (if there is more then one) synchronized > easily, and the configuration wouldn't diverege by time. Test execution has > to be solved also, and should cover all the available test cases. > In this scenario: > If we can provide one good working solution is much better, then provide > three different ones which become out of sync easily. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Review Request 66067: SQOOP-3052: Introduce gradle-based build for Sqoop to make it more developer friendly / open
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66067/ --- Review request for Sqoop. Bugs: Sqoop-3052 https://issues.apache.org/jira/browse/Sqoop-3052 Repository: sqoop-trunk Description --- SQOOP-3052: Introduce gradle based build for Sqoop to make it more developer friendly / open Diffs - .gitignore 68cbe28 COMPILING.txt 86be509 build.gradle PRE-CREATION buildSrc/customUnixStartScript.txt PRE-CREATION buildSrc/customWindowsStartScript.txt PRE-CREATION buildSrc/sqoop-package.gradle PRE-CREATION buildSrc/sqoop-version-gen.gradle PRE-CREATION config/checkstyle/checkstyle-java-header.txt PRE-CREATION config/checkstyle/checkstyle-noframes.xsl PRE-CREATION config/checkstyle/checkstyle.xml PRE-CREATION gradle.properties PRE-CREATION gradlew PRE-CREATION gradlew.bat PRE-CREATION settings.gradle PRE-CREATION Diff: https://reviews.apache.org/r/66067/diff/1/ Testing --- ran all new tasks, except for internal maven publishing Notes: - To try it out you can call ./gradlew tasks --all to see all the tasks and compare them to current tasks/artifacts. - Replaced cobertura with jacoco, as it's easier/cleaner to configure, easier to combine all test results into a single report. - Generated pom.xml now has correct dependencies/versions - Script generation is currently hardcoded and not based on sqoop help, as previously - though added the possiblity of hooking it in later Thanks, Anna Szonyi
[jira] [Updated] (SQOOP-3052) Introduce Maven/Gradle/etc. based build for Sqoop to make it more developer friendly / open
[ https://issues.apache.org/jira/browse/SQOOP-3052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Szonyi updated SQOOP-3052: --- Attachment: (was: SQOOP-3052.patch) > Introduce Maven/Gradle/etc. based build for Sqoop to make it more developer > friendly / open > --- > > Key: SQOOP-3052 > URL: https://issues.apache.org/jira/browse/SQOOP-3052 > Project: Sqoop > Issue Type: Improvement >Reporter: Attila Szabo >Assignee: Anna Szonyi >Priority: Major > Fix For: 1.5.0 > > > The current trunk version can only be build with Ant/Ivy combination, which > has some painful limitations (resolve is slow / needs to be tweaked to use > only caches, the current profile / variable based settings are not working in > IDEs out of the box, the current solution does not download the related > sources, etc.) > It would be nice to provide a solution, which would give the possibility for > the developers to choose between the nowadays well used build infrsturctures > (e.g. Maven, Gradle, etc.). For this solution it would be also essential to > keep the different build files (if there is more then one) synchronized > easily, and the configuration wouldn't diverege by time. Test execution has > to be solved also, and should cover all the available test cases. > In this scenario: > If we can provide one good working solution is much better, then provide > three different ones which become out of sync easily. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SQOOP-3052) Introduce Maven/Gradle/etc. based build for Sqoop to make it more developer friendly / open
[ https://issues.apache.org/jira/browse/SQOOP-3052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Szonyi updated SQOOP-3052: --- Attachment: SQOOP-3052.patch > Introduce Maven/Gradle/etc. based build for Sqoop to make it more developer > friendly / open > --- > > Key: SQOOP-3052 > URL: https://issues.apache.org/jira/browse/SQOOP-3052 > Project: Sqoop > Issue Type: Improvement >Reporter: Attila Szabo >Assignee: Anna Szonyi >Priority: Major > Fix For: 1.5.0 > > Attachments: SQOOP-3052.patch > > > The current trunk version can only be build with Ant/Ivy combination, which > has some painful limitations (resolve is slow / needs to be tweaked to use > only caches, the current profile / variable based settings are not working in > IDEs out of the box, the current solution does not download the related > sources, etc.) > It would be nice to provide a solution, which would give the possibility for > the developers to choose between the nowadays well used build infrsturctures > (e.g. Maven, Gradle, etc.). For this solution it would be also essential to > keep the different build files (if there is more then one) synchronized > easily, and the configuration wouldn't diverege by time. Test execution has > to be solved also, and should cover all the available test cases. > In this scenario: > If we can provide one good working solution is much better, then provide > three different ones which become out of sync easily. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SQOOP-3052) Introduce Maven/Gradle/etc. based build for Sqoop to make it more developer friendly / open
[ https://issues.apache.org/jira/browse/SQOOP-3052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Szonyi updated SQOOP-3052: --- Attachment: (was: SQOOP-3052.patch) > Introduce Maven/Gradle/etc. based build for Sqoop to make it more developer > friendly / open > --- > > Key: SQOOP-3052 > URL: https://issues.apache.org/jira/browse/SQOOP-3052 > Project: Sqoop > Issue Type: Improvement >Reporter: Attila Szabo >Assignee: Anna Szonyi >Priority: Major > Fix For: 1.5.0 > > > The current trunk version can only be build with Ant/Ivy combination, which > has some painful limitations (resolve is slow / needs to be tweaked to use > only caches, the current profile / variable based settings are not working in > IDEs out of the box, the current solution does not download the related > sources, etc.) > It would be nice to provide a solution, which would give the possibility for > the developers to choose between the nowadays well used build infrsturctures > (e.g. Maven, Gradle, etc.). For this solution it would be also essential to > keep the different build files (if there is more then one) synchronized > easily, and the configuration wouldn't diverege by time. Test execution has > to be solved also, and should cover all the available test cases. > In this scenario: > If we can provide one good working solution is much better, then provide > three different ones which become out of sync easily. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SQOOP-3052) Introduce Maven/Gradle/etc. based build for Sqoop to make it more developer friendly / open
[ https://issues.apache.org/jira/browse/SQOOP-3052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anna Szonyi updated SQOOP-3052: --- Attachment: (was: SQOOP-3052.patch) > Introduce Maven/Gradle/etc. based build for Sqoop to make it more developer > friendly / open > --- > > Key: SQOOP-3052 > URL: https://issues.apache.org/jira/browse/SQOOP-3052 > Project: Sqoop > Issue Type: Improvement >Reporter: Attila Szabo >Assignee: Anna Szonyi >Priority: Major > Fix For: 1.5.0 > > > The current trunk version can only be build with Ant/Ivy combination, which > has some painful limitations (resolve is slow / needs to be tweaked to use > only caches, the current profile / variable based settings are not working in > IDEs out of the box, the current solution does not download the related > sources, etc.) > It would be nice to provide a solution, which would give the possibility for > the developers to choose between the nowadays well used build infrsturctures > (e.g. Maven, Gradle, etc.). For this solution it would be also essential to > keep the different build files (if there is more then one) synchronized > easily, and the configuration wouldn't diverege by time. Test execution has > to be solved also, and should cover all the available test cases. > In this scenario: > If we can provide one good working solution is much better, then provide > three different ones which become out of sync easily. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SQOOP-3052) Introduce Maven/Gradle/etc. based build for Sqoop to make it more developer friendly / open
[ https://issues.apache.org/jira/browse/SQOOP-3052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404790#comment-16404790 ] Anna Szonyi commented on SQOOP-3052: Hi [~maugli], It seems I've missed your comment - apologies! The patches I attached were in a mostly complete state, I attached it as I got some requests around trying it out, but there were some files missing (script templates, etc.) and some further polishing to be done. I'll attach the finished version. Best, Anna > Introduce Maven/Gradle/etc. based build for Sqoop to make it more developer > friendly / open > --- > > Key: SQOOP-3052 > URL: https://issues.apache.org/jira/browse/SQOOP-3052 > Project: Sqoop > Issue Type: Improvement >Reporter: Attila Szabo >Assignee: Anna Szonyi >Priority: Major > Fix For: 1.5.0 > > Attachments: SQOOP-3052.patch, SQOOP-3052.patch > > > The current trunk version can only be build with Ant/Ivy combination, which > has some painful limitations (resolve is slow / needs to be tweaked to use > only caches, the current profile / variable based settings are not working in > IDEs out of the box, the current solution does not download the related > sources, etc.) > It would be nice to provide a solution, which would give the possibility for > the developers to choose between the nowadays well used build infrsturctures > (e.g. Maven, Gradle, etc.). For this solution it would be also essential to > keep the different build files (if there is more then one) synchronized > easily, and the configuration wouldn't diverege by time. Test execution has > to be solved also, and should cover all the available test cases. > In this scenario: > If we can provide one good working solution is much better, then provide > three different ones which become out of sync easily. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SQOOP-3297) Unrecognized argument: --schema for sqoop job execution
[
https://issues.apache.org/jira/browse/SQOOP-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404775#comment-16404775
]
Bhanu Busi commented on SQOOP-3297:
---
Hi [~fero],
Thanks for reply. Please find this
{noformat}
Warning:
/cloudera/opt/cloudera/parcels/CDH-5.13.1-1.cdh5.13.1.p0.2/bin/../lib/sqoop/../accumulo
does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
18/03/19 06:00:22 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.1
Job:
Tool: import
Options:
verbose = false
hcatalog.drop.and.create.table = false
incremental.last.value = 2018-02-15 00:28:35.821569
db.connect.string =
codegen.output.delimiters.escape = 0
codegen.output.delimiters.enclose.required = false
codegen.input.delimiters.field = 0
mainframe.input.dataset.type = p
hbase.create.table = false
split.limit = null
null.string = \\N
db.require.password = false
hdfs.append.dir = true
codegen.input.delimiters.escape = 0
db.password =
accumulo.create.table = false
import.fetch.size = null
codegen.input.delimiters.enclose.required = false
db.username =
reset.onemapper = false
codegen.output.delimiters.record = 10
import.max.inline.lob.size = 16777216
sqoop.throwOnError = false
hbase.bulk.load.enabled = false
hcatalog.create.table = false
db.clear.staging.table = false
incremental.col = db_modified_ts
codegen.input.delimiters.record = 0
enable.compression = false
hive.overwrite.table = false
hive.import = false
codegen.input.delimiters.enclose = 0
hive.table.name =
accumulo.batch.size = 1024
hive.drop.delims = false
customtool.options.jsonmap = {}
null.non-string = \\N
codegen.output.delimiters.enclose = 0
hdfs.delete-target.dir = false
hive.delims.replacement = anything
codegen.output.dir = .
codegen.auto.compile.dir = true
db.query =
relaxed.isolation = false
mapreduce.num.mappers = 1
accumulo.max.latency = 5000
import.direct.split.size = 0
sqlconnection.metadata.transaction.isolation.level = 2
codegen.output.delimiters.field = 9
export.new.update = UpdateOnly
incremental.mode = AppendRows
hdfs.file.format = TextFile
sqoop.oracle.escaping.disabled = true
codegen.compile.dir =
/tmp/sqoop-hdpprod/compile/67a4c3d8cdaaa676248dcbee44aba0f7
direct.import = false
temporary.dirRoot = _sqoop
hdfs.target.dir =
hive.fail.table.exists = false
tool.arguments.1 = public
tool.arguments.0 = --schema
db.batch = false{noformat}
> Unrecognized argument: --schema for sqoop job execution
> ---
>
> Key: SQOOP-3297
> URL: https://issues.apache.org/jira/browse/SQOOP-3297
> Project: Sqoop
> Issue Type: Bug
>Reporter: Bhanu Busi
>Priority: Minor
>
> Hi,
>
> previously I have used CDH-5.10 and sqoop 1.4.6. For this combination - -
> schema argument working fine.
> Now I have upgraded cloudera manager to CDH-5.13.1 and sqoop 1.4.6. For this
> combination - - schema argument showing bellow error
> while executing the sqoop job I have faced the following issue.
> {noformat}
> sqoop job --exec X
> (256, 'Warning:
> /cloudera/opt/cloudera/parcels/CDH-5.13.1-1.cdh5.13.1.p0.2/bin/../lib/sqoop/../accumulo
> does not exist! Accumulo imports will fail.\nPlease set $ACCUMULO_HOME to
> the root of your Accumulo installation.\n18/03/19 05:05:40 INFO sqoop.Sqoop:
> Running Sqoop version: 1.4.6-cdh5.13.1\n18/03/19 05:05:41 ERROR
> tool.BaseSqoopTool: Error parsing arguments for import:\n18/03/19 05:05:41
> ERROR tool.BaseSqoopTool: Unrecognized argument: --schema\n18/03/19 05:05:41
> ERROR tool.BaseSqoopTool: Unrecognized argument: public\n\nTry --help for
> usage instructions.'){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SQOOP-3297) Unrecognized argument: --schema for sqoop job execution
[
https://issues.apache.org/jira/browse/SQOOP-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404758#comment-16404758
]
Fero Szabo commented on SQOOP-3297:
---
Hi [~bbusi],
Could you paste the output to the following command?
{noformat}
sqoop job --show myjob{noformat}
Thanks!
Fero
> Unrecognized argument: --schema for sqoop job execution
> ---
>
> Key: SQOOP-3297
> URL: https://issues.apache.org/jira/browse/SQOOP-3297
> Project: Sqoop
> Issue Type: Bug
>Reporter: Bhanu Busi
>Priority: Minor
>
> Hi,
>
> previously I have used CDH-5.10 and sqoop 1.4.6. For this combination - -
> schema argument working fine.
> Now I have upgraded cloudera manager to CDH-5.13.1 and sqoop 1.4.6. For this
> combination - - schema argument showing bellow error
> while executing the sqoop job I have faced the following issue.
> {noformat}
> sqoop job --exec X
> (256, 'Warning:
> /cloudera/opt/cloudera/parcels/CDH-5.13.1-1.cdh5.13.1.p0.2/bin/../lib/sqoop/../accumulo
> does not exist! Accumulo imports will fail.\nPlease set $ACCUMULO_HOME to
> the root of your Accumulo installation.\n18/03/19 05:05:40 INFO sqoop.Sqoop:
> Running Sqoop version: 1.4.6-cdh5.13.1\n18/03/19 05:05:41 ERROR
> tool.BaseSqoopTool: Error parsing arguments for import:\n18/03/19 05:05:41
> ERROR tool.BaseSqoopTool: Unrecognized argument: --schema\n18/03/19 05:05:41
> ERROR tool.BaseSqoopTool: Unrecognized argument: public\n\nTry --help for
> usage instructions.'){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (SQOOP-3297) Unrecognized argument: --schema for sqoop job execution
Bhanu Busi created SQOOP-3297:
-
Summary: Unrecognized argument: --schema for sqoop job execution
Key: SQOOP-3297
URL: https://issues.apache.org/jira/browse/SQOOP-3297
Project: Sqoop
Issue Type: Bug
Reporter: Bhanu Busi
Hi,
previously I have used CDH-5.10 and sqoop 1.4.6. For this combination - -
schema argument working fine.
Now I have upgraded cloudera manager to CDH-5.13.1 and sqoop 1.4.6. For this
combination - - schema argument showing bellow error
while executing the sqoop job I have faced the following issue.
{noformat}
sqoop job --exec X
(256, 'Warning:
/cloudera/opt/cloudera/parcels/CDH-5.13.1-1.cdh5.13.1.p0.2/bin/../lib/sqoop/../accumulo
does not exist! Accumulo imports will fail.\nPlease set $ACCUMULO_HOME to the
root of your Accumulo installation.\n18/03/19 05:05:40 INFO sqoop.Sqoop:
Running Sqoop version: 1.4.6-cdh5.13.1\n18/03/19 05:05:41 ERROR
tool.BaseSqoopTool: Error parsing arguments for import:\n18/03/19 05:05:41
ERROR tool.BaseSqoopTool: Unrecognized argument: --schema\n18/03/19 05:05:41
ERROR tool.BaseSqoopTool: Unrecognized argument: public\n\nTry --help for usage
instructions.'){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (SQOOP-3224) Mainframe FTP transfer should have an option to use binary mode for transfer
[ https://issues.apache.org/jira/browse/SQOOP-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404590#comment-16404590 ] ASF GitHub Bot commented on SQOOP-3224: --- GitHub user christeoh opened a pull request: https://github.com/apache/sqoop/pull/44 [SQOOP-3224] Binary transfer mode You can merge this pull request into a Git repository by running: $ git pull https://github.com/christeoh/sqoop 3224-2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/sqoop/pull/44.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #44 > Mainframe FTP transfer should have an option to use binary mode for transfer > > > Key: SQOOP-3224 > URL: https://issues.apache.org/jira/browse/SQOOP-3224 > Project: Sqoop > Issue Type: Improvement >Reporter: Chris Teoh >Assignee: Chris Teoh >Priority: Minor > > Currently the mainframe FTP module is hard coded to use ascii transfer mode. > Propose a mainframe module flag to be able to change modes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] sqoop pull request #44: [SQOOP-3224] Binary transfer mode
GitHub user christeoh opened a pull request: https://github.com/apache/sqoop/pull/44 [SQOOP-3224] Binary transfer mode You can merge this pull request into a Git repository by running: $ git pull https://github.com/christeoh/sqoop 3224-2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/sqoop/pull/44.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #44 ---
[jira] [Created] (SQOOP-3296) 'Duplicate Colum identifier' for DB2.
Ganesh Rajan Thiyagarajan created SQOOP-3296: Summary: 'Duplicate Colum identifier' for DB2. Key: SQOOP-3296 URL: https://issues.apache.org/jira/browse/SQOOP-3296 Project: Sqoop Issue Type: Task Components: sqoop2-api Environment: Cloudera with Sqoop Reporter: Ganesh Rajan Thiyagarajan I am facing issues with 'Duplicate Colum identifier' for DB2. Other database has no issues as their column names are unique, but it is bit different for the DB. Is there any way for me to give manual column name instead of picking up from automatically? I have tried to give column name in the query (ie selelct PAGRP as groupid from TB) but the query doesn't seem to work. The query works in the actual database. It may work for mysql but not for DB2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
