kaxil opened a new pull request, #74810:
URL: https://github.com/apache/airflow/pull/74810
closes: #74601
`SSHRemoteJobOperator` mangles any `environment` value that contains a space
or a shell metacharacter on POSIX hosts. `build_posix_wrapper_command` escaped
each value for one level of single quotes and wrote `export KEY='value'`, but
those export lines are pasted inside the single-quoted `job_script='...'`
literal. The value's own opening quote therefore closes that literal, and the
value lands unquoted in the wrapper shell. So `{"INPUT_FILE": "two words"}`
fails the submit with `words: command not found`, and `$(...)`, backticks or
`;` in a value are run by the wrapper shell on the remote host instead of
reaching the job.
The fix quotes each value with `shlex.quote` for the job shell, then escapes
the export block once more for the `job_script` literal, which is the treatment
the command already gets a few lines below. Values with no special characters
now render as `export KEY=value` (shlex leaves them unquoted), so the existing
string assertion changed accordingly.
The new test runs the real wrapper under `bash` and checks that the job's
log holds the value verbatim and that no marker file appears, for spaces,
`$(...)`, backticks, `;`, embedded single and double quotes, `$VAR`, a newline
and the empty string. With the source change reverted, 7 of the 8 cases fail
(the empty string passes either way).
The Windows wrapper is not affected by this: its child script is
base64-encoded via `-EncodedCommand`, so env values only pass through one
quoting level. #73352 is rewriting that wrapper separately.
---
* Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information. Note: commit author/co-author name and email in commits
become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment:
`{pr_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
You can add this file in a follow-up commit after the PR is created so you
know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
