[jira] [Updated] (SQOOP-3025) Oracle Direct export from HCatalog parquet table gives NPE in kerberized cluster
[
https://issues.apache.org/jira/browse/SQOOP-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruslan Dautkhanov updated SQOOP-3025:
-
Description:
Getting following exception in one of the mappers stack when try to export to
Oracle from Hive Parquet
{noformat}
2016-10-14 23:14:01,027 ERROR [main]
org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert: The following error
occurred during configurePreparedStatement()
java.lang.NullPointerException
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.setBindValueAtName(OraOopOutputFormatBase.java:432)
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.configurePreparedStatementColumns(OraOopOutputFormatBase.java:547)
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.configurePreparedStatement(OraOopOutputFormatInsert.java:246)
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.getPreparedStatement(OraOopOutputFormatBase.java:303)
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:153)
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:194)
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.close(OraOopOutputFormatBase.java:578)
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.close(OraOopOutputFormatInsert.java:107)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:670)
at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:2016)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:797)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
2016-10-14 23:14:01,074 INFO [main]
org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert: Time spent performing
an "exchange subpartition with table": 0.046365 sec.
2016-10-14 23:14:01,074 DEBUG [main]
org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert: Dropping temporary
mapper table "ORAOOP_20161014_231320_3"
2016-10-14 23:14:01,131 INFO [main] org.apache.hadoop.mapred.MapTask: Ignoring
exception during close for
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector@54848551
java.io.IOException: java.sql.SQLException: java.lang.NullPointerException
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:197)
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.close(OraOopOutputFormatBase.java:578)
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.close(OraOopOutputFormatInsert.java:107)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:670)
at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:2016)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:797)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.sql.SQLException: java.lang.NullPointerException
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.configurePreparedStatement(OraOopOutputFormatInsert.java:255)
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.getPreparedStatement(OraOopOutputFormatBase.java:303)
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:153)
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:194)
{noformat}
Will attach full log file as an attachment.
A minimal sqoop export parameters that reproduce this problem:
{quote}
sqoop export -Dmapred.map.max.attempts=1 --connect
"jdbc:oracle:thin:@//somehost:1355/adash_prod_batch" --username
DISCOVER_STG_OWNER --password somePassword --table amf_trans_stg
--hcatalog-database disc_dv --hcatalog-table amf_trans_dv_09142016
--num-mappers 2 --columns "individ,part_code" --direct --verbose
{quote}
was:
Getting following exception in one of the mappers
[jira] [Updated] (SQOOP-3025) Oracle Direct export from HCatalog parquet table gives NPE in kerberized cluster
[
https://issues.apache.org/jira/browse/SQOOP-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruslan Dautkhanov updated SQOOP-3025:
-
Description:
Getting following exception in one of the mappers stack when try to export to
Oracle from Hive Parquet
{noformat}
2016-10-14 23:14:01,027 ERROR [main]
org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert: The following error
occurred during configurePreparedStatement()
java.lang.NullPointerException
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.setBindValueAtName(OraOopOutputFormatBase.java:432)
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.configurePreparedStatementColumns(OraOopOutputFormatBase.java:547)
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.configurePreparedStatement(OraOopOutputFormatInsert.java:246)
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.getPreparedStatement(OraOopOutputFormatBase.java:303)
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:153)
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:194)
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.close(OraOopOutputFormatBase.java:578)
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.close(OraOopOutputFormatInsert.java:107)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:670)
at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:2016)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:797)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
2016-10-14 23:14:01,074 INFO [main]
org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert: Time spent performing
an "exchange subpartition with table": 0.046365 sec.
2016-10-14 23:14:01,074 DEBUG [main]
org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert: Dropping temporary
mapper table "ORAOOP_20161014_231320_3"
2016-10-14 23:14:01,131 INFO [main] org.apache.hadoop.mapred.MapTask: Ignoring
exception during close for
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector@54848551
java.io.IOException: java.sql.SQLException: java.lang.NullPointerException
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:197)
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.close(OraOopOutputFormatBase.java:578)
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.close(OraOopOutputFormatInsert.java:107)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:670)
at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:2016)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:797)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.sql.SQLException: java.lang.NullPointerException
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.configurePreparedStatement(OraOopOutputFormatInsert.java:255)
at
org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.getPreparedStatement(OraOopOutputFormatBase.java:303)
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:153)
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:194)
{noformat}
Will attach full log file as an attachment.
A minimal sqoop export parameters that reproduce this problem:
{noformat}
sqoop export -Dmapred.map.max.attempts=1 --connect
"jdbc:oracle:thin:@//somehost:1355/adash_prod_batch" --username
DISCOVER_STG_OWNER --password somePassword --table amf_trans_stg
--hcatalog-database disc_dv --hcatalog-table amf_trans_dv_09142016
--num-mappers 2 --columns "individ,part_code" --direct --verbose
{noformat}
was:
Getting following exception in one of the
[jira] [Updated] (SQOOP-3025) Oracle Direct export from HCatalog parquet table gives NPE in kerberized cluster
[
https://issues.apache.org/jira/browse/SQOOP-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruslan Dautkhanov updated SQOOP-3025:
-
Attachment: scoop_output.log
> Oracle Direct export from HCatalog parquet table gives NPE in kerberized
> cluster
>
>
> Key: SQOOP-3025
> URL: https://issues.apache.org/jira/browse/SQOOP-3025
> Project: Sqoop
> Issue Type: Bug
> Components: build, codegen, connectors/oracle, hive-integration,
> metastore
>Affects Versions: 1.4.6
> Environment: Hadoop 2.6; Sqoop 1.4.6; Oracle 12c
>Reporter: Ruslan Dautkhanov
> Attachments: scoop_output.log,
> sqoop_OracleDirect_mapper_nullPointer_exception.txt
>
>
> Getting following exception in one of the mappers stack when try to export to
> Oracle from Hive Parquet
> {noformat}
> 2016-10-14 23:14:01,027 ERROR [main]
> org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert: The following error
> occurred during configurePreparedStatement()
> java.lang.NullPointerException
> at
> org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.setBindValueAtName(OraOopOutputFormatBase.java:432)
> at
> org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.configurePreparedStatementColumns(OraOopOutputFormatBase.java:547)
> at
> org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.configurePreparedStatement(OraOopOutputFormatInsert.java:246)
> at
> org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.getPreparedStatement(OraOopOutputFormatBase.java:303)
> at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:153)
> at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:194)
> at
> org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.close(OraOopOutputFormatBase.java:578)
> at
> org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.close(OraOopOutputFormatInsert.java:107)
> at
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:670)
> at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:2016)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:797)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> 2016-10-14 23:14:01,074 INFO [main]
> org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert: Time spent
> performing an "exchange subpartition with table": 0.046365 sec.
> 2016-10-14 23:14:01,074 DEBUG [main]
> org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert: Dropping temporary
> mapper table "ORAOOP_20161014_231320_3"
> 2016-10-14 23:14:01,131 INFO [main] org.apache.hadoop.mapred.MapTask:
> Ignoring exception during close for
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector@54848551
> java.io.IOException: java.sql.SQLException: java.lang.NullPointerException
> at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:197)
> at
> org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.close(OraOopOutputFormatBase.java:578)
> at
> org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.close(OraOopOutputFormatInsert.java:107)
> at
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:670)
> at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:2016)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:797)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.sql.SQLException: java.lang.NullPointerException
> at
> org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.configurePreparedStatement(OraOopOutputFormatInsert.java:255)
> at
>
[jira] [Updated] (SQOOP-3025) Oracle Direct export from HCatalog parquet table gives NPE in kerberized cluster
[
https://issues.apache.org/jira/browse/SQOOP-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruslan Dautkhanov updated SQOOP-3025:
-
Attachment: sqoop_OracleDirect_mapper_nullPointer_exception.txt
full log from failed mapper
> Oracle Direct export from HCatalog parquet table gives NPE in kerberized
> cluster
>
>
> Key: SQOOP-3025
> URL: https://issues.apache.org/jira/browse/SQOOP-3025
> Project: Sqoop
> Issue Type: Bug
> Components: build, codegen, connectors/oracle, hive-integration,
> metastore
>Affects Versions: 1.4.6
> Environment: Hadoop 2.6; Sqoop 1.4.6; Oracle 12c
>Reporter: Ruslan Dautkhanov
> Attachments: sqoop_OracleDirect_mapper_nullPointer_exception.txt
>
>
> Getting following exception in one of the mappers stack when try to export to
> Oracle from Hive Parquet
> {noformat}
> 2016-10-14 23:14:01,027 ERROR [main]
> org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert: The following error
> occurred during configurePreparedStatement()
> java.lang.NullPointerException
> at
> org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.setBindValueAtName(OraOopOutputFormatBase.java:432)
> at
> org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.configurePreparedStatementColumns(OraOopOutputFormatBase.java:547)
> at
> org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.configurePreparedStatement(OraOopOutputFormatInsert.java:246)
> at
> org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.getPreparedStatement(OraOopOutputFormatBase.java:303)
> at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:153)
> at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:194)
> at
> org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.close(OraOopOutputFormatBase.java:578)
> at
> org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.close(OraOopOutputFormatInsert.java:107)
> at
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:670)
> at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:2016)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:797)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> 2016-10-14 23:14:01,074 INFO [main]
> org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert: Time spent
> performing an "exchange subpartition with table": 0.046365 sec.
> 2016-10-14 23:14:01,074 DEBUG [main]
> org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert: Dropping temporary
> mapper table "ORAOOP_20161014_231320_3"
> 2016-10-14 23:14:01,131 INFO [main] org.apache.hadoop.mapred.MapTask:
> Ignoring exception during close for
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector@54848551
> java.io.IOException: java.sql.SQLException: java.lang.NullPointerException
> at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:197)
> at
> org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.close(OraOopOutputFormatBase.java:578)
> at
> org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.close(OraOopOutputFormatInsert.java:107)
> at
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:670)
> at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:2016)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:797)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.sql.SQLException: java.lang.NullPointerException
> at
> org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.configurePreparedStatement(OraOopOutputFormatInsert.java:255)
> at
>
