[
https://issues.apache.org/jira/browse/SQOOP-3022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15576227#comment-15576227
]
Ruslan Dautkhanov commented on SQOOP-3022:
------------------------------------------
I wasn't able yet to test that..
OraOOP is manager factory loads fine, but then falls into disabled state. See
below.
{quote}
16/10/14 12:39:42 DEBUG sqoop.ConnFactory: sqoop.connection.factories is set;
ignoring managers.d
16/10/14 12:39:42 DEBUG sqoop.ConnFactory: Loaded manager factory:
org.apache.sqoop.manager.oracle.OraOopManagerFactory
16/10/14 12:39:42 DEBUG sqoop.ConnFactory: Trying ManagerFactory:
org.apache.sqoop.manager.oracle.OraOopManagerFactory
16/10/14 12:39:43 DEBUG oracle.OraOopManagerFactory: Data Connector for Oracle
and Hadoop can be called by Sqoop!
16/10/14 12:39:43 INFO oracle.OraOopManagerFactory: Data Connector for Oracle
and Hadoop is disabled.
16/10/14 12:39:43 ERROR tool.BaseSqoopTool: Got error creating database
manager: java.io.IOException: No manager for connect string:
jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=somehost-scan.somedomain.com)(PORT=1355))
{quote}
Any ideas to help resolve this will be very helpful.
Here's list of Sqoop arguments called from a Python script:
{quote}
sqoop_params = ['sqoop', 'export'
# marreduce options:
, '-Dmapreduce.map.memory.mb=5000'
, '-Dmapreduce.map.java.opts=-Xmx4000m'
, '-Dmapred.job.name="sqooping %s@hadoop -> %s@oracle"' %
(inp_table, out_table)
, '-Dmapred.map.max.attempts=1'
# Oracle-specific Sqoop options:
,
'-Dsqoop.connection.factories=org.apache.sqoop.manager.oracle.OraOopManagerFactory'
# fail if OraOOP is unavailable
, '-Doraoop.disabled=false'
, '-Doraoop.oracle.append.values.hint.usage=ON'
, '-Doraoop.partitioned=true'
, '-Doraoop.nologging=true'
, '-Doraoop.template.table=%s_template' % out_table
, '-Doraoop.temporary.table.storage.clause="%s"' % storage_clause
, '-Doraoop.table.storage.clause="%s"' % storage_clause
# generic sqoop options:
, '-Dsqoop.export.records.per.statement=5000'
, '-Dsqoop.export.statements.per.transaction=2000'
, '--connect', "jdbc:oracle:thin:@" + jdbc_tns
, '--username', out_schema, '--password',
jdbc_password.decode('base64')
, '--table', out_table
, '--hcatalog-database', inp_schema
, '--hcatalog-table', inp_table
, '--batch'
, '--num-mappers', str(num_mappers)
]
{quote}
Thanks!
> sqoop export for Oracle generates tremendous amounts of redo logs
> -----------------------------------------------------------------
>
> Key: SQOOP-3022
> URL: https://issues.apache.org/jira/browse/SQOOP-3022
> Project: Sqoop
> Issue Type: Bug
> Components: codegen, connectors, connectors/oracle
> Affects Versions: 1.4.3, 1.4.4, 1.4.5, 1.4.6
> Reporter: Ruslan Dautkhanov
> Labels: export, oracle
>
> Sqoop export for Oracle generates tremendous amounts of redo logs (comparable
> to export size or more).
> We have put target tables in nologgin mode, but Oracle will still generate
> redo logs unless +APPEND Oracle insert hint is used.
> See https://oracle-base.com/articles/misc/append-hint for examples.
> Please add an option for sqoop to generate insert statements in Oracle with
> APPEND statement. Our databases are swamped with redo/archived logs whenever
> we sqoop data to them. This is easily avoidable. And from business
> prospective sqooping to staging tables in nologgin mode is totally fine.
> Thank you.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
