kshirsagarmahesh edited a comment on issue #10976:
URL: https://github.com/apache/airflow/issues/10976#issuecomment-694448079
Thanks @kaxil -
This works for most situations. However I found a unique situation where
this does not log a error with standard message saying the dag is FAILED.
Scenario: Bash Operator with run_as_user option. The user I am using does
not have sudo permissions. In such case the log remains incomplete:
Here is complete log:
------------------------------------------------------------------------------
[2020-09-17 15:11:37,196] {taskinstance.py:901} INFO - Executing
<Task(BashOperator): sudo_fail_test> on 2020-09-17T19:11:23.017017+00:00
[2020-09-17 15:11:37,196] {base_task_runner.py:131} INFO - Running on host:
devmachine.company.com
[2020-09-17 15:11:37,196] {base_task_runner.py:132} INFO - Running: ['sudo',
'-E', '-H', '-u', 'newuser',
'PYTHONPATH=/xx/rda/airflow/airflow_env/lib/python3.7/site-packages/:/opt/xx/lib/python3.7/site-packages/',
'airflow', 'run', 'ca_exit_code_test', 'sudo_fail_test',
'2020-09-17T19:11:23.017017+00:00', '--job_id', '8332', '--pool',
'default_pool', '--raw', '-sd', 'DAGS_FOLDER/ca_exit_code_test.py',
'--cfg_path', '/tmp/tmpputqd5lz']
[2020-09-17 15:11:37,288] {base_task_runner.py:113} INFO - Job 8332: Subtask
sudo_fail_test testuser is not in the sudoers file. This incident will be
reported.
[2020-09-17 15:11:42,205] {local_task_job.py:102} INFO - Task exited with
return code 1
------------------------------------------------------------------------------
The dag I used had this task:
with models.DAG(
dag_id='ca_exit_code_test',
default_args=default_args,
schedule_interval='0 10 5 * *',
concurrency=3,
max_active_runs=1,
catchup=False
) as dag:
start = DummyOperator(
task_id='start'
)
task_1 = BashOperator(
task_id='sudo_fail_test' ,
bash_command = 'ls' ,
dag = dag,
run_as_user= 'newuser'
)
start >> task_1
As you can see the log is incomplete.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
