[
https://issues.apache.org/jira/browse/SQOOP-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Meng updated SQOOP-3356:
-----------------------------
Description:
Goal: Implementing Sqoop Metastore to run daily incremental ETL.
When executing the following Sqoop job:
{code:java}
sqoop job -Dmapreduce.job.queuename=D_NO_SLA \
--create yle0_gim_interaction_fact - import \
--connect "jdbc:oracle:thin:@//xxxx.xxxx.com:0000/xxxx" \
--username T0V0_DATALAKE_YLE0 \
--password-file /dev/xxxxxxx/YLE0_OAT.password
--outdir ./sqoop/ \
--table RYLEINFOMART_2.INTERACTION_FACT \
--incremental append \
--check-column 'GREATEST(CREATE_AUDIT_KEY, UPDATE_AUDIT_KEY)' \
--last-value '53492217' \
--split-by 'CREATE_AUDIT_KEY' -m 16 \
--target-dir
/dev/T0V0/hivedb/yle0/staging/yle0_gim_interaction_fact_staging/ \
--fields-terminated-by '\001'{code}
Sqoop would automatically run the following query to update the max value:
{code:java}
SELECT MAX("GREATEST(CREATE_AUDIT_KEY, UPDATE_AUDIT_KEY)") FROM
"RYLEINFOMART_2"."INTERACTION_FACT"
{code}
SQL syntax error was given since there shouldn't be double quotes around
GREATEST() function.
The proposal of using free form query is rejected since metastore cannot be
implemented to automatically run the daily incremental job.
was:
Goal: Implementing Sqoop Metastore to run daily incremental ETL.
When executing the following Sqoop job:
{code:java}
sqoop job -Dmapreduce.job.queuename=D_NO_SLA \
--create yle0_gim_interaction_fact - import \
--connect "jdbc:oracle:thin:@//xxxx.xxxx.com:0000/xxxx" \
--username T0V0_DATALAKE_YLE0 \
--password-file /dev/xxxxxxx/YLE0_OAT.password
--outdir ./sqoop/ \
--table RYLEINFOMART_2.INTERACTION_FACT \
--incremental append \
--check-column 'GREATEST(CREATE_AUDIT_KEY, UPDATE_AUDIT_KEY)' \
--last-value '53492217'
--split-by 'CREATE_AUDIT_KEY' -m 16
--target-dir
/dev/T0V0/hivedb/yle0/staging/yle0_gim_interaction_fact_staging/ \
--fields-terminated-by '\001'{code}
Sqoop would automatically run the following query to update the max value:
{code:java}
SELECT MAX("GREATEST(CREATE_AUDIT_KEY, UPDATE_AUDIT_KEY)") FROM
"RYLEINFOMART_2"."INTERACTION_FACT"
{code}
SQL syntax error was given since there shouldn't be double quotes around
GREATEST() function.
The proposal of using free form query is rejected since metastore cannot be
implemented to automatically run the daily incremental job.
> SQL Syntax Error Raised when Double Quotes Added for "check-column"
> -------------------------------------------------------------------
>
> Key: SQOOP-3356
> URL: https://issues.apache.org/jira/browse/SQOOP-3356
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/oracle, metastore
> Affects Versions: 1.4.6
> Reporter: Sean Meng
> Priority: Blocker
>
> Goal: Implementing Sqoop Metastore to run daily incremental ETL.
> When executing the following Sqoop job:
> {code:java}
> sqoop job -Dmapreduce.job.queuename=D_NO_SLA \
> --create yle0_gim_interaction_fact - import \
> --connect "jdbc:oracle:thin:@//xxxx.xxxx.com:0000/xxxx" \
> --username T0V0_DATALAKE_YLE0 \
> --password-file /dev/xxxxxxx/YLE0_OAT.password
> --outdir ./sqoop/ \
> --table RYLEINFOMART_2.INTERACTION_FACT \
> --incremental append \
> --check-column 'GREATEST(CREATE_AUDIT_KEY, UPDATE_AUDIT_KEY)' \
> --last-value '53492217' \
> --split-by 'CREATE_AUDIT_KEY' -m 16 \
> --target-dir
> /dev/T0V0/hivedb/yle0/staging/yle0_gim_interaction_fact_staging/ \
> --fields-terminated-by '\001'{code}
> Sqoop would automatically run the following query to update the max value:
> {code:java}
> SELECT MAX("GREATEST(CREATE_AUDIT_KEY, UPDATE_AUDIT_KEY)") FROM
> "RYLEINFOMART_2"."INTERACTION_FACT"
> {code}
> SQL syntax error was given since there shouldn't be double quotes around
> GREATEST() function.
> The proposal of using free form query is rejected since metastore cannot be
> implemented to automatically run the daily incremental job.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
