blag commented on issue #28280:
URL: https://github.com/apache/airflow/issues/28280#issuecomment-1366352922
Thank you both for the bug fix and the explanations. That explains the
behavior of tasks that succeed and why
`test_bash_operator_backslash_n_not_equals_backslash_r` fails, but I don't
understand how that explains
`test_bash_operator_env_var_from_variable_jinja_interpolation` or
`test_bash_operator_from_variable_jinja_interpolation`. Both of those have
their Unix newlines replaced with Windows newlines, yet Windows is _not_
involved in any part of my environment. It would make more sense to me that the
rendered templates would be rendered with newlines appropriate for the platform
they're being rendered on (Linux). But I'm happy to be wrong and learn more
about this.
I've also noticed that variables set via the web UI are the ones that have
their newlines translated to CRLF:
```python
from datetime import datetime
from airflow import DAG
from airflow.operators.bash import BashOperator
from airflow.operators.smooth import SmoothOperator
with DAG("test_newlines", schedule=None) as dag:
test_bash_operator_env_var_from_variable_webui_jinja_interpolation =
BashOperator(
dag=dag,
task_id="test_bash_operator_env_var_from_variable_webui_jinja_interpolation",
start_date=datetime.now(),
env={
"ENV_VAR_AIRFLOW_VARIABLE_WITH_NEWLINES": "{{
var.value['newlines'] }}",
},
append_env=True,
bash_command='''
diff <(echo "$ENV_VAR_AIRFLOW_VARIABLE_WITH_NEWLINES") <(echo
"$ENV_VAR_AIRFLOW_VARIABLE_WITH_NEWLINES" | dos2unix) || {
echo >&2 "Environment variable contains newlines incorrectly converted
to Windows CRLF"
exit 1
}
''',
)
test_bash_operator_from_variable_webui_jinja_interpolation =
BashOperator(
dag=dag,
task_id="test_bash_operator_from_variable_webui_jinja_interpolation",
start_date=datetime.now(),
bash_command='''
diff <(echo "{{ var.value['newlines'] }}") <(echo "{{ var.value['newlines']
}}" | dos2unix) || {
echo >&2 "Jinja interpolated string contains newlines incorrectly
converted to Windows CRLF"
exit 1
}
''',
)
[
test_bash_operator_env_var_from_variable_webui_jinja_interpolation,
test_bash_operator_from_variable_webui_jinja_interpolation,
] >> failing
```
Is it expected that those tasks would fail? Or should those pass?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
