[ 
https://issues.apache.org/jira/browse/SQOOP-3022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15576227#comment-15576227
 ] 

Ruslan Dautkhanov commented on SQOOP-3022:
------------------------------------------

I wasn't able yet to test that.. 

OraOOP is manager factory loads fine, but then falls into disabled state. See 
below.

{quote}
16/10/14 12:39:42 DEBUG sqoop.ConnFactory: sqoop.connection.factories is set; 
ignoring managers.d
16/10/14 12:39:42 DEBUG sqoop.ConnFactory: Loaded manager factory: 
org.apache.sqoop.manager.oracle.OraOopManagerFactory
16/10/14 12:39:42 DEBUG sqoop.ConnFactory: Trying ManagerFactory: 
org.apache.sqoop.manager.oracle.OraOopManagerFactory
16/10/14 12:39:43 DEBUG oracle.OraOopManagerFactory: Data Connector for Oracle 
and Hadoop can be called by Sqoop!
16/10/14 12:39:43 INFO oracle.OraOopManagerFactory: Data Connector for Oracle 
and Hadoop is disabled.
16/10/14 12:39:43 ERROR tool.BaseSqoopTool: Got error creating database 
manager: java.io.IOException: No manager for connect string: 
jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=somehost-scan.somedomain.com)(PORT=1355))
{quote}

Any ideas to help resolve this will be very helpful.

Here's list of Sqoop arguments called from a Python script:

{quote}
    sqoop_params = ['sqoop', 'export'
            # marreduce options:
            , '-Dmapreduce.map.memory.mb=5000'
            , '-Dmapreduce.map.java.opts=-Xmx4000m'
            , '-Dmapred.job.name="sqooping %s@hadoop -> %s@oracle"' % 
(inp_table, out_table)
            , '-Dmapred.map.max.attempts=1'
            # Oracle-specific Sqoop options:
            , 
'-Dsqoop.connection.factories=org.apache.sqoop.manager.oracle.OraOopManagerFactory'
   # fail if OraOOP is unavailable
            , '-Doraoop.disabled=false'
            , '-Doraoop.oracle.append.values.hint.usage=ON'
            , '-Doraoop.partitioned=true'
            , '-Doraoop.nologging=true'
            , '-Doraoop.template.table=%s_template' % out_table
            , '-Doraoop.temporary.table.storage.clause="%s"' % storage_clause
            , '-Doraoop.table.storage.clause="%s"' % storage_clause
            # generic sqoop options:
            , '-Dsqoop.export.records.per.statement=5000'
            , '-Dsqoop.export.statements.per.transaction=2000'
            , '--connect', "jdbc:oracle:thin:@" + jdbc_tns
            , '--username', out_schema, '--password', 
jdbc_password.decode('base64')
            , '--table', out_table
            , '--hcatalog-database', inp_schema
            , '--hcatalog-table', inp_table
            , '--batch'
            , '--num-mappers', str(num_mappers)
            ]
{quote}

Thanks!

> sqoop export for Oracle generates tremendous amounts of redo logs
> -----------------------------------------------------------------
>
>                 Key: SQOOP-3022
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3022
>             Project: Sqoop
>          Issue Type: Bug
>          Components: codegen, connectors, connectors/oracle
>    Affects Versions: 1.4.3, 1.4.4, 1.4.5, 1.4.6
>            Reporter: Ruslan Dautkhanov
>              Labels: export, oracle
>
> Sqoop export for Oracle generates tremendous amounts of redo logs (comparable 
> to export size or more).
> We have put target tables in nologgin mode, but Oracle will still generate 
> redo logs unless +APPEND Oracle insert hint is used.
> See https://oracle-base.com/articles/misc/append-hint for examples.
> Please add an option for sqoop to generate insert statements in Oracle with 
> APPEND statement. Our databases are swamped with redo/archived logs whenever 
> we sqoop data to them. This is easily avoidable. And from business 
> prospective sqooping to staging tables in nologgin mode is totally fine.
> Thank you.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to