[
https://issues.apache.org/jira/browse/AIRFLOW-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16861310#comment-16861310
]
ROHIT K RAMWAL edited comment on AIRFLOW-4772 at 6/11/19 6:34 PM:
------------------------------------------------------------------
[~ash] I tried to run it. Here is scenario where its failing.
python3 /bin/airflow run hive_test_flow hiveactiontobackuptable
2019-06-11T18:18:43.304403+00:00 --job_id 71 --raw -sd
DAGS_FOLDER/hive_test/test_hive.py --cfg_path /tmp/tmpwz67adw6
The cfg is a showing for user id location inside the tmpwz67adw6 and not the
default.
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 32, in <module>
args.func(args)
File "/usr/local/lib/python3.6/site-packages/airflow/utils/cli.py", line 74,
in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/airflow/bin/cli.py", line 501,
in run
dag = get_dag(args)
File "/usr/local/lib/python3.6/site-packages/airflow/bin/cli.py", line 145,
in get_dag
'parse.'.format(args.dag_id))
airflow.exceptions.AirflowException: dag_id could not be found: hive_test_flow.
Either the dag did not exist or it failed to parse.
was (Author: rramwal):
[~ash] I tried to run it. Here is scenario where its failing.
python3 /bin/airflow run hive_test_flow hiveactiontobackuptable
2019-06-11T18:18:43.304403+00:00 --job_id 71 --raw -sd
DAGS_FOLDER/hive_test/test_hive.py --cfg_path /tmp/tmpwz67adw6
The cfg is a showing for user id location inside the tmpwz67adw6 and not the
default.
> airflow run_as_user creates new config file when trying to execute a task
> -------------------------------------------------------------------------
>
> Key: AIRFLOW-4772
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4772
> Project: Apache Airflow
> Issue Type: Bug
> Components: DAG
> Affects Versions: 1.10.3
> Environment: Linux centos-7.2.1511
> Reporter: ROHIT K RAMWAL
> Priority: Major
>
> I am using celery executor to scale airflow with redis as backed. Workers are
> executed with airflow user and have priviliges to sudo to other users in the
> system.
> In my config i have set the default_impersonation = 'svc*123' which is a
> default load id for all my jobs. When ever i trigger a job the airflow
> creates a new folder under 'svc*123' and generates a default config which
> fails to identify the original configuration.
> Tried the same with run_as_user which also creates new config first time. If
> we use multiple users to administrate our system better airflow keeps on
> generating new config and default database everytime.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
