[jira] [Commented] (SQOOP-3135) Not enough error message for debugging when parameters missing
[
https://issues.apache.org/jira/browse/SQOOP-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868912#comment-15868912
]
Eric Lin commented on SQOOP-3135:
-
Review requested: https://reviews.apache.org/r/56737/
> Not enough error message for debugging when parameters missing
> --
>
> Key: SQOOP-3135
> URL: https://issues.apache.org/jira/browse/SQOOP-3135
> Project: Sqoop
> Issue Type: Improvement
>Affects Versions: 1.4.6
>Reporter: Eric Lin
>Assignee: Eric Lin
> Attachments: SQOOP-3135.patch
>
>
> Run the following sqoop command:
> {code}
> sqoop job --create test -- import --connect jdbc:mysql://localhost/test
> --username root --password $pass --target-dir /tmp/test10 -m 1 --driver
> com.mysql.jdbc.Driver --table test
> {code}
> Due to $pass is not set, command will fail with the following error:
> {code}
> 16/12/21 05:48:32 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7-SNAPSHOT
> 16/12/21 05:48:33 ERROR tool.JobTool: Error parsing arguments to the
> job-specific tool.
> 16/12/21 05:48:33 ERROR tool.JobTool: See 'sqoop help ' for usage.
> {code}
> This is not informative, by checking the code in JobTool class:
> {code}
> // Now feed the arguments into the tool itself.
> try {
> childOptions = childTool.parseArguments(parseableChildArgv,
> null, childOptions, false);
> childTool.appendArgs(extraChildArgv);
> childTool.validateOptions(childOptions);
> } catch (ParseException pe) {
> LOG.error("Error parsing arguments to the job-specific tool.");
> LOG.error("See 'sqoop help ' for usage.");
> return 1;
> } catch (SqoopOptions.InvalidOptionsException e) {
> System.err.println(e.getMessage());
> return 1;
> }
> {code}
> The ParseException pe's message has been dropped, we should print out the
> message in the exception so that more meaningful message will be printed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
Review Request 56737: SQOOP-3135 - Not enough error message for debugging when parameters missing
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56737/
---
Review request for Sqoop and Attila Szabo.
Bugs: SQOOP-3135
https://issues.apache.org/jira/browse/SQOOP-3135
Repository: sqoop-trunk
Description
---
Run the following sqoop command:
sqoop job --create test -- import --connect jdbc:mysql://localhost/test
--username root --password $pass --target-dir /tmp/test10 -m 1 --driver
com.mysql.jdbc.Driver --table test
Due to $pass is not set, command will fail with the following error:
16/12/21 05:48:32 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7-SNAPSHOT
16/12/21 05:48:33 ERROR tool.JobTool: Error parsing arguments to the
job-specific tool.
16/12/21 05:48:33 ERROR tool.JobTool: See 'sqoop help ' for usage.
This is not informative, by checking the code in JobTool class:
// Now feed the arguments into the tool itself.
try {
childOptions = childTool.parseArguments(parseableChildArgv,
null, childOptions, false);
childTool.appendArgs(extraChildArgv);
childTool.validateOptions(childOptions);
} catch (ParseException pe) {
LOG.error("Error parsing arguments to the job-specific tool.");
LOG.error("See 'sqoop help ' for usage.");
return 1;
} catch (SqoopOptions.InvalidOptionsException e) {
System.err.println(e.getMessage());
return 1;
}
The ParseException pe's message has been dropped, we should print out the
message in the exception so that more meaningful message will be printed.
Diffs
-
src/java/org/apache/sqoop/tool/JobTool.java 8e1669c
Diff: https://reviews.apache.org/r/56737/diff/
Testing
---
Manual test and confirmed the output.
Thanks,
Eric Lin
[jira] [Commented] (SQOOP-3135) Not enough error message for debugging when parameters missing
[
https://issues.apache.org/jira/browse/SQOOP-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868904#comment-15868904
]
Eric Lin commented on SQOOP-3135:
-
Provided patch so that the same command will print out the following:
{code}
16/12/21 05:57:29 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7-SNAPSHOT
16/12/21 05:57:29 ERROR tool.JobTool: Error parsing arguments to the
job-specific tool: org.apache.commons.cli.MissingArgumentException: Missing
argument for option: password
16/12/21 05:57:29 ERROR tool.JobTool: See 'sqoop help ' for usage.
{code}
which tells us exactly that password option was missing.
> Not enough error message for debugging when parameters missing
> --
>
> Key: SQOOP-3135
> URL: https://issues.apache.org/jira/browse/SQOOP-3135
> Project: Sqoop
> Issue Type: Improvement
>Affects Versions: 1.4.6
>Reporter: Eric Lin
>Assignee: Eric Lin
> Attachments: SQOOP-3135.patch
>
>
> Run the following sqoop command:
> {code}
> sqoop job --create test -- import --connect jdbc:mysql://localhost/test
> --username root --password $pass --target-dir /tmp/test10 -m 1 --driver
> com.mysql.jdbc.Driver --table test
> {code}
> Due to $pass is not set, command will fail with the following error:
> {code}
> 16/12/21 05:48:32 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7-SNAPSHOT
> 16/12/21 05:48:33 ERROR tool.JobTool: Error parsing arguments to the
> job-specific tool.
> 16/12/21 05:48:33 ERROR tool.JobTool: See 'sqoop help ' for usage.
> {code}
> This is not informative, by checking the code in JobTool class:
> {code}
> // Now feed the arguments into the tool itself.
> try {
> childOptions = childTool.parseArguments(parseableChildArgv,
> null, childOptions, false);
> childTool.appendArgs(extraChildArgv);
> childTool.validateOptions(childOptions);
> } catch (ParseException pe) {
> LOG.error("Error parsing arguments to the job-specific tool.");
> LOG.error("See 'sqoop help ' for usage.");
> return 1;
> } catch (SqoopOptions.InvalidOptionsException e) {
> System.err.println(e.getMessage());
> return 1;
> }
> {code}
> The ParseException pe's message has been dropped, we should print out the
> message in the exception so that more meaningful message will be printed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (SQOOP-3135) Not enough error message for debugging when parameters missing
[
https://issues.apache.org/jira/browse/SQOOP-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Lin updated SQOOP-3135:
Attachment: SQOOP-3135.patch
> Not enough error message for debugging when parameters missing
> --
>
> Key: SQOOP-3135
> URL: https://issues.apache.org/jira/browse/SQOOP-3135
> Project: Sqoop
> Issue Type: Improvement
>Affects Versions: 1.4.6
>Reporter: Eric Lin
>Assignee: Eric Lin
> Attachments: SQOOP-3135.patch
>
>
> Run the following sqoop command:
> {code}
> sqoop job --create test -- import --connect jdbc:mysql://localhost/test
> --username root --password $pass --target-dir /tmp/test10 -m 1 --driver
> com.mysql.jdbc.Driver --table test
> {code}
> Due to $pass is not set, command will fail with the following error:
> {code}
> 16/12/21 05:48:32 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7-SNAPSHOT
> 16/12/21 05:48:33 ERROR tool.JobTool: Error parsing arguments to the
> job-specific tool.
> 16/12/21 05:48:33 ERROR tool.JobTool: See 'sqoop help ' for usage.
> {code}
> This is not informative, by checking the code in JobTool class:
> {code}
> // Now feed the arguments into the tool itself.
> try {
> childOptions = childTool.parseArguments(parseableChildArgv,
> null, childOptions, false);
> childTool.appendArgs(extraChildArgv);
> childTool.validateOptions(childOptions);
> } catch (ParseException pe) {
> LOG.error("Error parsing arguments to the job-specific tool.");
> LOG.error("See 'sqoop help ' for usage.");
> return 1;
> } catch (SqoopOptions.InvalidOptionsException e) {
> System.err.println(e.getMessage());
> return 1;
> }
> {code}
> The ParseException pe's message has been dropped, we should print out the
> message in the exception so that more meaningful message will be printed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (SQOOP-3135) Not enough error message for debugging when parameters missing
Eric Lin created SQOOP-3135:
---
Summary: Not enough error message for debugging when parameters
missing
Key: SQOOP-3135
URL: https://issues.apache.org/jira/browse/SQOOP-3135
Project: Sqoop
Issue Type: Improvement
Affects Versions: 1.4.6
Reporter: Eric Lin
Assignee: Eric Lin
Run the following sqoop command:
{code}
sqoop job --create test -- import --connect jdbc:mysql://localhost/test
--username root --password $pass --target-dir /tmp/test10 -m 1 --driver
com.mysql.jdbc.Driver --table test
{code}
Due to $pass is not set, command will fail with the following error:
{code}
16/12/21 05:48:32 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7-SNAPSHOT
16/12/21 05:48:33 ERROR tool.JobTool: Error parsing arguments to the
job-specific tool.
16/12/21 05:48:33 ERROR tool.JobTool: See 'sqoop help ' for usage.
{code}
This is not informative, by checking the code in JobTool class:
{code}
// Now feed the arguments into the tool itself.
try {
childOptions = childTool.parseArguments(parseableChildArgv,
null, childOptions, false);
childTool.appendArgs(extraChildArgv);
childTool.validateOptions(childOptions);
} catch (ParseException pe) {
LOG.error("Error parsing arguments to the job-specific tool.");
LOG.error("See 'sqoop help ' for usage.");
return 1;
} catch (SqoopOptions.InvalidOptionsException e) {
System.err.println(e.getMessage());
return 1;
}
{code}
The ParseException pe's message has been dropped, we should print out the
message in the exception so that more meaningful message will be printed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (SQOOP-3061) Sqoop --options-file failed with error "Malformed option in options file" even though the query is correct
[
https://issues.apache.org/jira/browse/SQOOP-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868800#comment-15868800
]
Eric Lin commented on SQOOP-3061:
-
Hi [~maugli],
Thanks a lot for your help and guidance to get my very first commit. I will try
to contribute more in the near future.
> Sqoop --options-file failed with error "Malformed option in options file"
> even though the query is correct
> --
>
> Key: SQOOP-3061
> URL: https://issues.apache.org/jira/browse/SQOOP-3061
> Project: Sqoop
> Issue Type: Bug
>Affects Versions: 1.4.6
>Reporter: Eric Lin
>Assignee: Eric Lin
>Priority: Minor
> Labels: patch
> Attachments: SQOOP-3061.2.patch, SQOOP-3061.3.patch,
> SQOOP-3061.4.patch, SQOOP-3061.5.patch, SQOOP-3061.patch
>
>
> if you have the following in the options file:
> --query
> SELECT * FROM test WHERE a = 'b'
> and then run
> {code}
> sqoop --options-file
> {code}
> it will fail with the following error:
> {code}
> 16/11/22 16:08:59 ERROR sqoop.Sqoop: Error while expanding arguments
> java.lang.Exception: Malformed option in options
> file(/tmp/sqoop_runner_from_stdin_1112_12354__sqoop_options_file): SELECT *
> FROM test WHERE a = 'b'
> at
> org.apache.sqoop.util.OptionsFileUtil.removeQuoteCharactersIfNecessary(OptionsFileUtil.java:170)
> at
> org.apache.sqoop.util.OptionsFileUtil.removeQuotesEncolosingOption(OptionsFileUtil.java:136)
> at
> org.apache.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:90)
> at
> com.cloudera.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:33)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:199)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> Malformed option in options
> file(/tmp/sqoop_runner_from_stdin_1112_12354__sqoop_options_file): SELECT *
> FROM test WHERE a = 'b'
> {code}
> This is caused by function
> org.apache.sqoop.util.OptionsFileUtil.removeQuoteCharactersIfNecessary only
> checks for starting and ending quotes and will fail if the query does not
> start with a quote but ends with a quote, like the example query above.
> {code}
> private static String removeQuoteCharactersIfNecessary(String fileName,
> String option, char quote) throws Exception {
> boolean startingQuote = (option.charAt(0) == quote);
> boolean endingQuote = (option.charAt(option.length() - 1) == quote);
> if (startingQuote && endingQuote) {
> if (option.length() == 1) {
> throw new Exception("Malformed option in options file("
> + fileName + "): " + option);
> }
> return option.substring(1, option.length() - 1);
> }
> if (startingQuote || endingQuote) {
>throw new Exception("Malformed option in options file("
>+ fileName + "): " + option);
> }
> return option;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (SQOOP-3061) Sqoop --options-file failed with error "Malformed option in options file" even though the query is correct
[
https://issues.apache.org/jira/browse/SQOOP-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867560#comment-15867560
]
Hudson commented on SQOOP-3061:
---
SUCCESS: Integrated in Jenkins build Sqoop-hadoop20 #1089 (See
[https://builds.apache.org/job/Sqoop-hadoop20/1089/])
SQOOP-3061: Improve query validation in connection with --options-file (maugli:
[https://git-wip-us.apache.org/repos/asf?p=sqoop.git=commit=24d08185c502e416265f1fa78300347ad02a4e1f])
* (edit) src/java/org/apache/sqoop/util/OptionsFileUtil.java
* (edit) src/test/com/cloudera/sqoop/util/TestOptionsFileExpansion.java
> Sqoop --options-file failed with error "Malformed option in options file"
> even though the query is correct
> --
>
> Key: SQOOP-3061
> URL: https://issues.apache.org/jira/browse/SQOOP-3061
> Project: Sqoop
> Issue Type: Bug
>Affects Versions: 1.4.6
>Reporter: Eric Lin
>Assignee: Eric Lin
>Priority: Minor
> Labels: patch
> Attachments: SQOOP-3061.2.patch, SQOOP-3061.3.patch,
> SQOOP-3061.4.patch, SQOOP-3061.5.patch, SQOOP-3061.patch
>
>
> if you have the following in the options file:
> --query
> SELECT * FROM test WHERE a = 'b'
> and then run
> {code}
> sqoop --options-file
> {code}
> it will fail with the following error:
> {code}
> 16/11/22 16:08:59 ERROR sqoop.Sqoop: Error while expanding arguments
> java.lang.Exception: Malformed option in options
> file(/tmp/sqoop_runner_from_stdin_1112_12354__sqoop_options_file): SELECT *
> FROM test WHERE a = 'b'
> at
> org.apache.sqoop.util.OptionsFileUtil.removeQuoteCharactersIfNecessary(OptionsFileUtil.java:170)
> at
> org.apache.sqoop.util.OptionsFileUtil.removeQuotesEncolosingOption(OptionsFileUtil.java:136)
> at
> org.apache.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:90)
> at
> com.cloudera.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:33)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:199)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> Malformed option in options
> file(/tmp/sqoop_runner_from_stdin_1112_12354__sqoop_options_file): SELECT *
> FROM test WHERE a = 'b'
> {code}
> This is caused by function
> org.apache.sqoop.util.OptionsFileUtil.removeQuoteCharactersIfNecessary only
> checks for starting and ending quotes and will fail if the query does not
> start with a quote but ends with a quote, like the example query above.
> {code}
> private static String removeQuoteCharactersIfNecessary(String fileName,
> String option, char quote) throws Exception {
> boolean startingQuote = (option.charAt(0) == quote);
> boolean endingQuote = (option.charAt(option.length() - 1) == quote);
> if (startingQuote && endingQuote) {
> if (option.length() == 1) {
> throw new Exception("Malformed option in options file("
> + fileName + "): " + option);
> }
> return option.substring(1, option.length() - 1);
> }
> if (startingQuote || endingQuote) {
>throw new Exception("Malformed option in options file("
>+ fileName + "): " + option);
> }
> return option;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (SQOOP-3061) Sqoop --options-file failed with error "Malformed option in options file" even though the query is correct
[
https://issues.apache.org/jira/browse/SQOOP-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867553#comment-15867553
]
Hudson commented on SQOOP-3061:
---
SUCCESS: Integrated in Jenkins build Sqoop-hadoop23 #1290 (See
[https://builds.apache.org/job/Sqoop-hadoop23/1290/])
SQOOP-3061: Improve query validation in connection with --options-file (maugli:
[https://git-wip-us.apache.org/repos/asf?p=sqoop.git=commit=24d08185c502e416265f1fa78300347ad02a4e1f])
* (edit) src/test/com/cloudera/sqoop/util/TestOptionsFileExpansion.java
* (edit) src/java/org/apache/sqoop/util/OptionsFileUtil.java
> Sqoop --options-file failed with error "Malformed option in options file"
> even though the query is correct
> --
>
> Key: SQOOP-3061
> URL: https://issues.apache.org/jira/browse/SQOOP-3061
> Project: Sqoop
> Issue Type: Bug
>Affects Versions: 1.4.6
>Reporter: Eric Lin
>Assignee: Eric Lin
>Priority: Minor
> Labels: patch
> Attachments: SQOOP-3061.2.patch, SQOOP-3061.3.patch,
> SQOOP-3061.4.patch, SQOOP-3061.5.patch, SQOOP-3061.patch
>
>
> if you have the following in the options file:
> --query
> SELECT * FROM test WHERE a = 'b'
> and then run
> {code}
> sqoop --options-file
> {code}
> it will fail with the following error:
> {code}
> 16/11/22 16:08:59 ERROR sqoop.Sqoop: Error while expanding arguments
> java.lang.Exception: Malformed option in options
> file(/tmp/sqoop_runner_from_stdin_1112_12354__sqoop_options_file): SELECT *
> FROM test WHERE a = 'b'
> at
> org.apache.sqoop.util.OptionsFileUtil.removeQuoteCharactersIfNecessary(OptionsFileUtil.java:170)
> at
> org.apache.sqoop.util.OptionsFileUtil.removeQuotesEncolosingOption(OptionsFileUtil.java:136)
> at
> org.apache.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:90)
> at
> com.cloudera.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:33)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:199)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> Malformed option in options
> file(/tmp/sqoop_runner_from_stdin_1112_12354__sqoop_options_file): SELECT *
> FROM test WHERE a = 'b'
> {code}
> This is caused by function
> org.apache.sqoop.util.OptionsFileUtil.removeQuoteCharactersIfNecessary only
> checks for starting and ending quotes and will fail if the query does not
> start with a quote but ends with a quote, like the example query above.
> {code}
> private static String removeQuoteCharactersIfNecessary(String fileName,
> String option, char quote) throws Exception {
> boolean startingQuote = (option.charAt(0) == quote);
> boolean endingQuote = (option.charAt(option.length() - 1) == quote);
> if (startingQuote && endingQuote) {
> if (option.length() == 1) {
> throw new Exception("Malformed option in options file("
> + fileName + "): " + option);
> }
> return option.substring(1, option.length() - 1);
> }
> if (startingQuote || endingQuote) {
>throw new Exception("Malformed option in options file("
>+ fileName + "): " + option);
> }
> return option;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (SQOOP-3061) Sqoop --options-file failed with error "Malformed option in options file" even though the query is correct
[
https://issues.apache.org/jira/browse/SQOOP-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867523#comment-15867523
]
Hudson commented on SQOOP-3061:
---
SUCCESS: Integrated in Jenkins build Sqoop-hadoop200 #1094 (See
[https://builds.apache.org/job/Sqoop-hadoop200/1094/])
SQOOP-3061: Improve query validation in connection with --options-file (maugli:
[https://git-wip-us.apache.org/repos/asf?p=sqoop.git=commit=24d08185c502e416265f1fa78300347ad02a4e1f])
* (edit) src/test/com/cloudera/sqoop/util/TestOptionsFileExpansion.java
* (edit) src/java/org/apache/sqoop/util/OptionsFileUtil.java
> Sqoop --options-file failed with error "Malformed option in options file"
> even though the query is correct
> --
>
> Key: SQOOP-3061
> URL: https://issues.apache.org/jira/browse/SQOOP-3061
> Project: Sqoop
> Issue Type: Bug
>Affects Versions: 1.4.6
>Reporter: Eric Lin
>Assignee: Eric Lin
>Priority: Minor
> Labels: patch
> Attachments: SQOOP-3061.2.patch, SQOOP-3061.3.patch,
> SQOOP-3061.4.patch, SQOOP-3061.5.patch, SQOOP-3061.patch
>
>
> if you have the following in the options file:
> --query
> SELECT * FROM test WHERE a = 'b'
> and then run
> {code}
> sqoop --options-file
> {code}
> it will fail with the following error:
> {code}
> 16/11/22 16:08:59 ERROR sqoop.Sqoop: Error while expanding arguments
> java.lang.Exception: Malformed option in options
> file(/tmp/sqoop_runner_from_stdin_1112_12354__sqoop_options_file): SELECT *
> FROM test WHERE a = 'b'
> at
> org.apache.sqoop.util.OptionsFileUtil.removeQuoteCharactersIfNecessary(OptionsFileUtil.java:170)
> at
> org.apache.sqoop.util.OptionsFileUtil.removeQuotesEncolosingOption(OptionsFileUtil.java:136)
> at
> org.apache.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:90)
> at
> com.cloudera.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:33)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:199)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> Malformed option in options
> file(/tmp/sqoop_runner_from_stdin_1112_12354__sqoop_options_file): SELECT *
> FROM test WHERE a = 'b'
> {code}
> This is caused by function
> org.apache.sqoop.util.OptionsFileUtil.removeQuoteCharactersIfNecessary only
> checks for starting and ending quotes and will fail if the query does not
> start with a quote but ends with a quote, like the example query above.
> {code}
> private static String removeQuoteCharactersIfNecessary(String fileName,
> String option, char quote) throws Exception {
> boolean startingQuote = (option.charAt(0) == quote);
> boolean endingQuote = (option.charAt(option.length() - 1) == quote);
> if (startingQuote && endingQuote) {
> if (option.length() == 1) {
> throw new Exception("Malformed option in options file("
> + fileName + "): " + option);
> }
> return option.substring(1, option.length() - 1);
> }
> if (startingQuote || endingQuote) {
>throw new Exception("Malformed option in options file("
>+ fileName + "): " + option);
> }
> return option;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (SQOOP-3061) Sqoop --options-file failed with error "Malformed option in options file" even though the query is correct
[
https://issues.apache.org/jira/browse/SQOOP-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867507#comment-15867507
]
Hudson commented on SQOOP-3061:
---
SUCCESS: Integrated in Jenkins build Sqoop-hadoop100 #1053 (See
[https://builds.apache.org/job/Sqoop-hadoop100/1053/])
SQOOP-3061: Improve query validation in connection with --options-file (maugli:
[https://git-wip-us.apache.org/repos/asf?p=sqoop.git=commit=24d08185c502e416265f1fa78300347ad02a4e1f])
* (edit) src/java/org/apache/sqoop/util/OptionsFileUtil.java
* (edit) src/test/com/cloudera/sqoop/util/TestOptionsFileExpansion.java
> Sqoop --options-file failed with error "Malformed option in options file"
> even though the query is correct
> --
>
> Key: SQOOP-3061
> URL: https://issues.apache.org/jira/browse/SQOOP-3061
> Project: Sqoop
> Issue Type: Bug
>Affects Versions: 1.4.6
>Reporter: Eric Lin
>Assignee: Eric Lin
>Priority: Minor
> Labels: patch
> Attachments: SQOOP-3061.2.patch, SQOOP-3061.3.patch,
> SQOOP-3061.4.patch, SQOOP-3061.5.patch, SQOOP-3061.patch
>
>
> if you have the following in the options file:
> --query
> SELECT * FROM test WHERE a = 'b'
> and then run
> {code}
> sqoop --options-file
> {code}
> it will fail with the following error:
> {code}
> 16/11/22 16:08:59 ERROR sqoop.Sqoop: Error while expanding arguments
> java.lang.Exception: Malformed option in options
> file(/tmp/sqoop_runner_from_stdin_1112_12354__sqoop_options_file): SELECT *
> FROM test WHERE a = 'b'
> at
> org.apache.sqoop.util.OptionsFileUtil.removeQuoteCharactersIfNecessary(OptionsFileUtil.java:170)
> at
> org.apache.sqoop.util.OptionsFileUtil.removeQuotesEncolosingOption(OptionsFileUtil.java:136)
> at
> org.apache.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:90)
> at
> com.cloudera.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:33)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:199)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> Malformed option in options
> file(/tmp/sqoop_runner_from_stdin_1112_12354__sqoop_options_file): SELECT *
> FROM test WHERE a = 'b'
> {code}
> This is caused by function
> org.apache.sqoop.util.OptionsFileUtil.removeQuoteCharactersIfNecessary only
> checks for starting and ending quotes and will fail if the query does not
> start with a quote but ends with a quote, like the example query above.
> {code}
> private static String removeQuoteCharactersIfNecessary(String fileName,
> String option, char quote) throws Exception {
> boolean startingQuote = (option.charAt(0) == quote);
> boolean endingQuote = (option.charAt(option.length() - 1) == quote);
> if (startingQuote && endingQuote) {
> if (option.length() == 1) {
> throw new Exception("Malformed option in options file("
> + fileName + "): " + option);
> }
> return option.substring(1, option.length() - 1);
> }
> if (startingQuote || endingQuote) {
>throw new Exception("Malformed option in options file("
>+ fileName + "): " + option);
> }
> return option;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (SQOOP-3061) Sqoop --options-file failed with error "Malformed option in options file" even though the query is correct
[
https://issues.apache.org/jira/browse/SQOOP-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867460#comment-15867460
]
Attila Szabo commented on SQOOP-3061:
-
Hey [~ericlin],
Thanks for your contribution, and let me congratulate for your very first Sqoop
commit.
We hope you enjoyed working with our community and that we could expect more
contributions from you in the future!
Thanks,
[~maugli]
> Sqoop --options-file failed with error "Malformed option in options file"
> even though the query is correct
> --
>
> Key: SQOOP-3061
> URL: https://issues.apache.org/jira/browse/SQOOP-3061
> Project: Sqoop
> Issue Type: Bug
>Affects Versions: 1.4.6
>Reporter: Eric Lin
>Assignee: Eric Lin
>Priority: Minor
> Labels: patch
> Attachments: SQOOP-3061.2.patch, SQOOP-3061.3.patch,
> SQOOP-3061.4.patch, SQOOP-3061.5.patch, SQOOP-3061.patch
>
>
> if you have the following in the options file:
> --query
> SELECT * FROM test WHERE a = 'b'
> and then run
> {code}
> sqoop --options-file
> {code}
> it will fail with the following error:
> {code}
> 16/11/22 16:08:59 ERROR sqoop.Sqoop: Error while expanding arguments
> java.lang.Exception: Malformed option in options
> file(/tmp/sqoop_runner_from_stdin_1112_12354__sqoop_options_file): SELECT *
> FROM test WHERE a = 'b'
> at
> org.apache.sqoop.util.OptionsFileUtil.removeQuoteCharactersIfNecessary(OptionsFileUtil.java:170)
> at
> org.apache.sqoop.util.OptionsFileUtil.removeQuotesEncolosingOption(OptionsFileUtil.java:136)
> at
> org.apache.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:90)
> at
> com.cloudera.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:33)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:199)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> Malformed option in options
> file(/tmp/sqoop_runner_from_stdin_1112_12354__sqoop_options_file): SELECT *
> FROM test WHERE a = 'b'
> {code}
> This is caused by function
> org.apache.sqoop.util.OptionsFileUtil.removeQuoteCharactersIfNecessary only
> checks for starting and ending quotes and will fail if the query does not
> start with a quote but ends with a quote, like the example query above.
> {code}
> private static String removeQuoteCharactersIfNecessary(String fileName,
> String option, char quote) throws Exception {
> boolean startingQuote = (option.charAt(0) == quote);
> boolean endingQuote = (option.charAt(option.length() - 1) == quote);
> if (startingQuote && endingQuote) {
> if (option.length() == 1) {
> throw new Exception("Malformed option in options file("
> + fileName + "): " + option);
> }
> return option.substring(1, option.length() - 1);
> }
> if (startingQuote || endingQuote) {
>throw new Exception("Malformed option in options file("
>+ fileName + "): " + option);
> }
> return option;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (SQOOP-3061) Sqoop --options-file failed with error "Malformed option in options file" even though the query is correct
[
https://issues.apache.org/jira/browse/SQOOP-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867456#comment-15867456
]
ASF subversion and git services commented on SQOOP-3061:
Commit 24d08185c502e416265f1fa78300347ad02a4e1f in sqoop's branch
refs/heads/trunk from [~maugli]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=24d0818 ]
SQOOP-3061: Improve query validation in connection with --options-file
thus correct options/queries like "--query SELECT * FROM test WHERE a = 'b'"
will not fail with "Malformed option in options file" error message
(Eric Lin via Attila Szabo)
> Sqoop --options-file failed with error "Malformed option in options file"
> even though the query is correct
> --
>
> Key: SQOOP-3061
> URL: https://issues.apache.org/jira/browse/SQOOP-3061
> Project: Sqoop
> Issue Type: Bug
>Affects Versions: 1.4.6
>Reporter: Eric Lin
>Assignee: Eric Lin
>Priority: Minor
> Labels: patch
> Attachments: SQOOP-3061.2.patch, SQOOP-3061.3.patch,
> SQOOP-3061.4.patch, SQOOP-3061.5.patch, SQOOP-3061.patch
>
>
> if you have the following in the options file:
> --query
> SELECT * FROM test WHERE a = 'b'
> and then run
> {code}
> sqoop --options-file
> {code}
> it will fail with the following error:
> {code}
> 16/11/22 16:08:59 ERROR sqoop.Sqoop: Error while expanding arguments
> java.lang.Exception: Malformed option in options
> file(/tmp/sqoop_runner_from_stdin_1112_12354__sqoop_options_file): SELECT *
> FROM test WHERE a = 'b'
> at
> org.apache.sqoop.util.OptionsFileUtil.removeQuoteCharactersIfNecessary(OptionsFileUtil.java:170)
> at
> org.apache.sqoop.util.OptionsFileUtil.removeQuotesEncolosingOption(OptionsFileUtil.java:136)
> at
> org.apache.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:90)
> at
> com.cloudera.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:33)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:199)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> Malformed option in options
> file(/tmp/sqoop_runner_from_stdin_1112_12354__sqoop_options_file): SELECT *
> FROM test WHERE a = 'b'
> {code}
> This is caused by function
> org.apache.sqoop.util.OptionsFileUtil.removeQuoteCharactersIfNecessary only
> checks for starting and ending quotes and will fail if the query does not
> start with a quote but ends with a quote, like the example query above.
> {code}
> private static String removeQuoteCharactersIfNecessary(String fileName,
> String option, char quote) throws Exception {
> boolean startingQuote = (option.charAt(0) == quote);
> boolean endingQuote = (option.charAt(option.length() - 1) == quote);
> if (startingQuote && endingQuote) {
> if (option.length() == 1) {
> throw new Exception("Malformed option in options file("
> + fileName + "): " + option);
> }
> return option.substring(1, option.length() - 1);
> }
> if (startingQuote || endingQuote) {
>throw new Exception("Malformed option in options file("
>+ fileName + "): " + option);
> }
> return option;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
Re: Review Request 54251: SQOOP-3061 - Fix a bug when parsing query in sqoop options file
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/54251/#review165664 --- Ship it! Hey Eric, Although the current solutio for validation is not perfect, but as the test/validation is not missleading anymore, I accept the current solution. Many thanks for your contribution! Attila - Attila Szabo On Jan. 19, 2017, 12:17 a.m., Eric Lin wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/54251/ > --- > > (Updated Jan. 19, 2017, 12:17 a.m.) > > > Review request for Sqoop and Szabolcs Vasas. > > > Bugs: SQOOP-3061 > https://issues.apache.org/jira/browse/SQOOP-3061 > > > Repository: sqoop-trunk > > > Description > --- > > SQOOP-3061 - Sqoop --options-file failed with error "Malformed option in > options file" even though the query is correct > > > Diffs > - > > src/java/org/apache/sqoop/util/OptionsFileUtil.java c476e00 > src/test/com/cloudera/sqoop/util/TestOptionsFileExpansion.java 6d3f0f3 > > Diff: https://reviews.apache.org/r/54251/diff/ > > > Testing > --- > > Test case updated and have made sure that existing test cases and new test > cases passed. > > > Thanks, > > Eric Lin > >
