[
https://issues.apache.org/jira/browse/AIRFLOW-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16754916#comment-16754916
]
ASF GitHub Bot commented on AIRFLOW-3734:
-----------------------------------------
Fokko commented on pull request #4561: [AIRFLOW-3734] Fix hql not run when
partition is None
URL: https://github.com/apache/airflow/pull/4561
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> function `load_file` will not run if param `partition` is None
> --------------------------------------------------------------
>
> Key: AIRFLOW-3734
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3734
> Project: Apache Airflow
> Issue Type: Bug
> Components: hive_hooks
> Affects Versions: 1.10.1
> Reporter: zhongjiajie
> Assignee: zhongjiajie
> Priority: Major
> Labels: easyfix, patch
>
> the code in master branch in hive_hooks.py in function load_file like below
> {code:java}
> hql = "LOAD DATA LOCAL INPATH '{filepath}' "
> if overwrite:
> hql += "OVERWRITE "
> hql += "INTO TABLE {table} "
> if partition:
> pvals = ", ".join(
> ["{0}='{1}'".format(k, v) for k, v in partition.items()])
> hql += "PARTITION ({pvals});"
> # As a workaround for HIVE-10541, add a newline character
> # at the end of hql (AIRFLOW-2412).
> hql += '\n'{code}
> when param partition is None, the hql will like this
> {code:java}
> LOAD DATA LOCAL INPATH <FILEPATH> OVERWRITE INTO TABLE <TABLE>\n{code}
> without `;` syntax the hql will not run, so I think we should move `;` from
> `hql += "PARTITION (\{pvals});"` to `hql += ';\n'`
>
> so the code will like
> {code:java}
> hql = "LOAD DATA LOCAL INPATH '{filepath}' "
> if overwrite:
> hql += "OVERWRITE "
> hql += "INTO TABLE {table} "
> if partition:
> pvals = ", ".join(
> ["{0}='{1}'".format(k, v) for k, v in partition.items()])
> hql += "PARTITION ({pvals})"
> # As a workaround for HIVE-10541, add a newline character
> # at the end of hql (AIRFLOW-2412).
> hql += ';\n'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
