[
https://issues.apache.org/jira/browse/AIRFLOW-1153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448437#comment-16448437
]
ASF subversion and git services commented on AIRFLOW-1153:
----------------------------------------------------------
Commit 09bbe247728993867c716635951219cc49f65dd1 in incubator-airflow's branch
refs/heads/master from Alan Ma
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=09bbe24 ]
[AIRFLOW-1153] Allow HiveOperators to take hiveconfs
HiveOperator can only replace variables via jinja
and the replacements
are global to the dag through the context and
user_defined_macros.
It would be much more flexible to open up
hive_conf to the HiveOperator
level so hive scripts can be recycled at the task
level, leveraging
HiveHook already existing hive_conf param and
_prepare_hiveconf
function.
Closes #3136 from wolfier/AIRFLOW-1153
> params in HiveOperator constructor can't be passed into Hive execution context
> ------------------------------------------------------------------------------
>
> Key: AIRFLOW-1153
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1153
> Project: Apache Airflow
> Issue Type: Bug
> Components: hive_hooks, operators
> Affects Versions: Airflow 2.0, Airflow 1.8
> Reporter: Xianping lin
> Priority: Critical
> Labels: easyfix, newbie
>
> params parameter in HiveOperator can't be imported into Hive executation
> context.
> so the following centence won't work, because 'mynumber' doesn't work for
> sql sentence.
> test_hiveoperator = HiveOperator(
> task_id='hive_test',
> hiveconf_jinja_translate=True,
> hql = ''' use myDB;
> INSERT OVERWRITE TABLE t2
> select * from t1 where t1.x > ' ${hiveconf:mynumber}'
> ''',
> params={'mynumber': 2},
> dag=dag
> )
> this modification pass the 'params' in HiveOperator construction to Hive
> sql execution context.
> The the variable definition can pass to hive sql
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
