-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50155/#review145463
-----------------------------------------------------------
Import the TST_PRODUCT table which is created by the automated tests:
import --direct --connect jdbc:oracle:thin:@//host:1521/service --username
oraoop --password oraoop --table TST_PRODUCT --target-dir tst_product
Export it back to a new table:
export -Doraoop.template.table=tst_product --direct --connect
jdbc:oracle:thin:@//host:1521/service --username oraoop --password oraoop
--table TST_PRODUCT_TEST --export-dir tst_product
Then do an update export onto the newly created table:
export --direct --connect jdbc:oracle:thin:@//host:1521/service --username
oraoop --password oraoop --table TST_PRODUCT_TEST --export-dir tst_product
--update-key PRODUCT_ID
It doesn't actually update any data due to the following warning:
WARN oracle.OraOopOutputFormatUpdate: 346 records were skipped due to a NULL
value within one of the update-key column(s).
It looks like this is something to do with the quoting of columns but it was
working before this patch. I think the whole table / column quoting is actually
quite broken at the moment so we probably need to do further testing here and
possibly raise another issue.
The new automated tests fail for me as well - the second one looks like it is
caused by this quoting issue. The first one is saying the value is too large -
perhaps I am running a different character set to you or something?
Testsuite: org.apache.sqoop.manager.oracle.OraOopTypesTest
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.532 sec
Testcase: ensureTypesAfterExportMappedAsExpected took 2.519 sec
Caused an ERROR
ORA-12899: value too large for column
"SQOOPTEST"."ORACLE_DATATYPES_TEMPLATE"."C11_CHAR" (actual: 47, maximum: 29)
java.sql.SQLException: ORA-12899: value too large for column
"SQOOPTEST"."ORACLE_DATATYPES_TEMPLATE"."C11_CHAR" (actual: 47, maximum: 29)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:450)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:399)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1059)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:522)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:257)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:587)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:210)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:30)
at oracle.jdbc.driver.T4CStatement.executeForRows(T4CStatement.java:931)
at
oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1150)
at
oracle.jdbc.driver.OracleStatement.executeInternal(OracleStatement.java:1792)
at oracle.jdbc.driver.OracleStatement.execute(OracleStatement.java:1745)
at
oracle.jdbc.driver.OracleStatementWrapper.execute(OracleStatementWrapper.java:334)
at
org.apache.sqoop.manager.oracle.OraOopTypesTest.ensureTypesAfterExportMappedAsExpected(OraOopTypesTest.java:39)
Testsuite: org.apache.sqoop.manager.oracle.OraOopUpdateKeyTest
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 16.195 sec
java.lang.Exception: java.io.IOException: java.sql.SQLSyntaxErrorException:
ORA-00904: "B"."product_id": invalid identifier
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:399)
Caused by: java.io.IOException: java.sql.SQLSyntaxErrorException: ORA-00904:
"B"."product_id": invalid identifier
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:211)
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.close(OraOopOutputFormatBase.java:555)
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatUpdate$OraOopDBRecordWriterUpdate.close(OraOopOutputFormatUpdate.java:113)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:641)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:762)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339)
at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:231)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "B"."product_id":
invalid identifier
- David Robson
On Aug. 9, 2016, 10:50 p.m., Attila Szabo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50155/
> -----------------------------------------------------------
>
> (Updated Aug. 9, 2016, 10:50 p.m.)
>
>
> Review request for Sqoop, David Robson, Jarek Cecho, and Kathleen Ting.
>
>
> Repository: sqoop-trunk
>
>
> Description
> -------
>
> Proposed changes for SQOOP-2983
>
>
> Diffs
> -----
>
> src/java/org/apache/sqoop/manager/oracle/OraOopConnManager.java 216c771
> src/java/org/apache/sqoop/manager/oracle/OraOopOracleQueries.java 6b27bd8
> src/java/org/apache/sqoop/manager/oracle/OraOopOutputFormatBase.java
> 8f94cf8
> src/java/org/apache/sqoop/manager/oracle/OraOopOutputFormatInsert.java
> d5eebf4
> src/java/org/apache/sqoop/manager/oracle/OraOopOutputFormatUpdate.java
> a33768f
> src/java/org/apache/sqoop/manager/oracle/OraOopUtilities.java e81588c
> src/java/org/apache/sqoop/orm/ClassWriter.java 9d91887
> src/test/org/apache/sqoop/manager/oracle/ExportTest.java 991b221
> src/test/org/apache/sqoop/manager/oracle/OraOopTestCase.java 9fe4821
> src/test/org/apache/sqoop/manager/oracle/OraOopTypesTest.java PRE-CREATION
> src/test/org/apache/sqoop/manager/oracle/OraOopUpdateKeyTest.java
> PRE-CREATION
>
> Diff: https://reviews.apache.org/r/50155/diff/
>
>
> Testing
> -------
>
> 800 columns with table
> 100.000 lines (156mb data)
> 1.000.000 lines (1.56 GB data)
> 3.000.000 lines (4.5 GB data)
>
>
> Thanks,
>
> Attila Szabo
>
>
