[ https://issues.apache.org/jira/browse/SQOOP-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15578380#comment-15578380 ]
Ruslan Dautkhanov edited comment on SQOOP-3025 at 10/15/16 4:25 PM: -------------------------------------------------------------------- attached full log from failed mapper was (Author: tagar): full log from failed mapper > Oracle Direct export from HCatalog parquet table gives NPE in kerberized > cluster > -------------------------------------------------------------------------------- > > Key: SQOOP-3025 > URL: https://issues.apache.org/jira/browse/SQOOP-3025 > Project: Sqoop > Issue Type: Bug > Components: build, codegen, connectors/oracle, hive-integration, > metastore > Affects Versions: 1.4.6 > Environment: Hadoop 2.6; Sqoop 1.4.6; Oracle 12c > Reporter: Ruslan Dautkhanov > Attachments: sqoop_OracleDirect_mapper_nullPointer_exception.txt > > > Getting following exception in one of the mappers stack when try to export to > Oracle from Hive Parquet > {noformat} > 2016-10-14 23:14:01,027 ERROR [main] > org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert: The following error > occurred during configurePreparedStatement() > java.lang.NullPointerException > at > org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.setBindValueAtName(OraOopOutputFormatBase.java:432) > at > org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.configurePreparedStatementColumns(OraOopOutputFormatBase.java:547) > at > org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.configurePreparedStatement(OraOopOutputFormatInsert.java:246) > at > org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.getPreparedStatement(OraOopOutputFormatBase.java:303) > at > org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:153) > at > org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:194) > at > org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.close(OraOopOutputFormatBase.java:578) > at > org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.close(OraOopOutputFormatInsert.java:107) > at > org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:670) > at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:2016) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:797) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > 2016-10-14 23:14:01,074 INFO [main] > org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert: Time spent > performing an "exchange subpartition with table": 0.046365 sec. > 2016-10-14 23:14:01,074 DEBUG [main] > org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert: Dropping temporary > mapper table "ORAOOP_20161014_231320_3" > 2016-10-14 23:14:01,131 INFO [main] org.apache.hadoop.mapred.MapTask: > Ignoring exception during close for > org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector@54848551 > java.io.IOException: java.sql.SQLException: java.lang.NullPointerException > at > org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:197) > at > org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.close(OraOopOutputFormatBase.java:578) > at > org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.close(OraOopOutputFormatInsert.java:107) > at > org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:670) > at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:2016) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:797) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > Caused by: java.sql.SQLException: java.lang.NullPointerException > at > org.apache.sqoop.manager.oracle.OraOopOutputFormatInsert$OraOopDBRecordWriterInsert.configurePreparedStatement(OraOopOutputFormatInsert.java:255) > at > org.apache.sqoop.manager.oracle.OraOopOutputFormatBase$OraOopDBRecordWriterBase.getPreparedStatement(OraOopOutputFormatBase.java:303) > at > org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:153) > at > org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:194) > {noformat} > Will attach full log file as an attachment. > Sqoop export options used: > {noformat} > sqoop_params = ['sqoop', 'export' > # marreduce options: > , '-Dmapreduce.map.memory.mb=5000' > , '-Dmapreduce.map.java.opts=-Xmx4000m' > , '-Dmapreduce.job.name=sqooping %s@hadoop -> %s@oracle' % > (inp_table, out_table) > , '-Dmapreduce.map.maxattempts=1' > # Oracle (OraOOP) specific Sqoop options: > , > '-Dsqoop.connection.factories=org.apache.sqoop.manager.oracle.OraOopManagerFactory' > , '-Doraoop.oracle.append.values.hint.usage=ON' > , '-Doraoop.partitioned=true' > , '-Doraoop.nologging=true' > , '-Doraoop.drop.table=true' > , '-Doraoop.template.table=%s_template' % out_table > , '-Doraoop.temporary.table.storage.clause=' + storage_clause > , '-Doraoop.table.storage.clause=' + storage_clause > , '-Doraoop.oracle.rac.service.name=adash_prod_batch' > # generic sqoop options: > , '-Dsqoop.export.records.per.statement=5000' > , '-Dsqoop.export.statements.per.transaction=2000' > , '--direct' #activate Oracle direct mode = 'Data Connector > for Oracle and Hadoop' > , '--connect', "jdbc:oracle:thin:@" + jdbc_tns > , '--username', out_schema, '--password', > jdbc_password.decode('base64') > , '--table', out_table > , '--hcatalog-database', inp_schema > , '--hcatalog-table', inp_table > , '--batch' > , '--num-mappers', str(num_mappers) > ] > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)