[jira] [Issue Comment Deleted] (SQOOP-3022) sqoop export for Oracle generates tremendous amounts of redo logs

[Attila Szabo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SQOOP-3022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Szabo updated SQOOP-3022:

Comment: was deleted

(was: Hi [~Tagar],

I would strongly encourage you to do an exhaustive testing on your side, before 
closing this ticket.

Functionality wise I do think OraOop is feature complete, and you could achieve 
your goals with it (as I've referred to it before, and as [~david.robson] 
confirmed that too).

Although as partitioned/non partitioned versions of the import are very much 
different from performance POV, and as there is an ongoing performance related 
change ( [SQOOP-2983] which might be interesting for you ), I think it would 
make sense to evaluate if the current solution+performance is satisfying for 
you (from the past I do remember you had serious performance related 
constraints in your system/pipline).
)

> sqoop export for Oracle generates tremendous amounts of redo logs
> -
>
> Key: SQOOP-3022
> URL: https://issues.apache.org/jira/browse/SQOOP-3022
> Project: Sqoop
>  Issue Type: Bug
>  Components: codegen, connectors, connectors/oracle
>Affects Versions: 1.4.3, 1.4.4, 1.4.5, 1.4.6
>Reporter: Ruslan Dautkhanov
>  Labels: export, oracle
>
> Sqoop export for Oracle generates tremendous amounts of redo logs (comparable 
> to export size or more).
> We have put target tables in nologgin mode, but Oracle will still generate 
> redo logs unless +APPEND Oracle insert hint is used.
> See https://oracle-base.com/articles/misc/append-hint for examples.
> Please add an option for sqoop to generate insert statements in Oracle with 
> APPEND statement. Our databases are swamped with redo/archived logs whenever 
> we sqoop data to them. This is easily avoidable. And from business 
> prospective sqooping to staging tables in nologgin mode is totally fine.
> Thank you.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (SQOOP-3022) sqoop export for Oracle generates tremendous amounts of redo logs

[Attila Szabo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SQOOP-3022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Szabo updated SQOOP-3022:

Comment: was deleted

(was: Hi [~Tagar],

According to this ( http://docs.oracle.com/cd/E11882_01/server.112/e25494.pdf 
page 604 ) both of them exits, and if I do understand correctly +APPEND is only 
usable when you use it with "INSERT into ... SELECT ..." statement, and 
+APPEND_VALUES when we have an INSERT statement with VALUES close (this is the 
case what makes sense in case of Sqoop IMHO).

According to Tom White +APPEND never worked with values (except one short 
amount of time/version), that was the reason why they added 
(https://asktom.oracle.com/pls/apex/f?p=100:11:0P11_QUESTION_ID:6087912900346548365).

APPEND_VALUES in Sqoop:
org/apache/sqoop/manager/oracle/OraOopOutputFormatInsert.java
org/apache/sqoop/manager/oracle/OraOopOutputFormatUpdate.java
docs/user/connectors.txt
conf/oraoop-site-template.xml

So if I'm not mistaken:
the current usage of hints is appropriate, and with SQOOP-2983 we would also 
have it with a better performance than the current trunk version.

Could you please confirm my findings?

+ [~david.robson]
Dave, as the author of OraOop capabilities of Sqoop, could you please also 
confirm if my findings are valid?

Many thanks,
[~maugli])

> sqoop export for Oracle generates tremendous amounts of redo logs
> -
>
> Key: SQOOP-3022
> URL: https://issues.apache.org/jira/browse/SQOOP-3022
> Project: Sqoop
>  Issue Type: Bug
>  Components: codegen, connectors, connectors/oracle
>Affects Versions: 1.4.3, 1.4.4, 1.4.5, 1.4.6
>Reporter: Ruslan Dautkhanov
>  Labels: export, oracle
>
> Sqoop export for Oracle generates tremendous amounts of redo logs (comparable 
> to export size or more).
> We have put target tables in nologgin mode, but Oracle will still generate 
> redo logs unless +APPEND Oracle insert hint is used.
> See https://oracle-base.com/articles/misc/append-hint for examples.
> Please add an option for sqoop to generate insert statements in Oracle with 
> APPEND statement. Our databases are swamped with redo/archived logs whenever 
> we sqoop data to them. This is easily avoidable. And from business 
> prospective sqooping to staging tables in nologgin mode is totally fine.
> Thank you.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)