nailo2c commented on PR #65991:
URL: https://github.com/apache/airflow/pull/65991#issuecomment-4578565695
Hi, thanks for the reviews, I really appreciated them. All the feedback has
been addressed.
1. Added a new YARN ResourceManager API tracking section to
`providers/apache/spark/docs/operators.rst`.
- It explains what `yarn_track_via_rm_api=True` does.
- It documents the required `yarn_resourcemanager_webapp_address`
connection extra.
- It covers Kerberos usage with requests-kerberos and
`yarn_rm_auth=HTTPKerberosAuth()`.
2. Added eager validation for `yarn_resourcemanager_webapp_address`.
- `_validate_yarn_track_via_rm_api_config()` now resolves the RM base
URL before spark-submit starts.
- This prevents submitting a YARN application and only then failing on
the first RM REST API poll.
3. Updated the missing RM URL regression test.
- The test now calls `hook.submit()`.
- It verifies that a missing `yarn_resourcemanager_webapp_address`
raises before `subprocess.Popen` is called.
4. Combined the auth/no-auth tests into one parameterized test.
- The test now covers both `auth=None` and a provided
`requests.auth.AuthBase` instance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
