baha-bouali opened a new pull request, #71071: URL: https://github.com/apache/airflow/pull/71071
closes: #66466 related: #67600 `azure-batch` 15.x is a ground-up rewrite of Azure's Python SDK (track 1 → track 2); `BatchServiceClient` and the old model classes are gone. #66452 capped `azure-batch<15.0.0` as a stopgap; #66466 tracks the real migration to lift it. ### The migration (#67600, by @arieleli01212) This PR carries forward @arieleli01212's work from [#67600](https://github.com/apache/airflow/pull/67600) as-is: `BatchServiceClient` → `BatchClient`, shared-key auth moved to `AzureNamedKeyCredential`, the renamed model classes (`PoolAddParameter` → `BatchPoolCreateOptions`, `CloudTask` → `BatchTask`, etc.), the move to the new flat client API, dropping `CloudServiceConfiguration`/`os_family`/`os_version`, and the matching test rewrites. ### What this PR adds on top Two commits on top of his 16, kept separate: **1. Rebase onto current `main`.** One real conflict, in `docs/changelog.rst` (an unrelated `14.0.0` entry had landed since). Resolved by keeping it and placing this migration's pending note above it. **2. Fix for the identity-auth path**, addressing [@aaron-y-chen's open review comment](https://github.com/apache/airflow/pull/67600#pullrequestreview-4679686593)(https://github.com/apache/airflow/pull/67600#pullrequestreview-4679686593) on the `# type: ignore[arg-type]` next to the `BatchClient` credential. The identity-based branch of `get_conn()` built an `AzureIdentityCredentialAdapter`, which subclasses msrest's track-1 `BasicTokenAuthentication` and doesn't implement `get_token()` — what azure-core's `TokenCredential` protocol requires, and what `BatchClient` needs to authenticate. `AzureBaseHook.get_token()` in this provider already documents this and raises `AttributeError` for it. So this path passed every mocked test but would fail against a real account. Switched to `get_sync_default_azure_credential()`, already used by `AzureBaseHook` and `asb.py` for this exact purpose, which returns a real `DefaultAzureCredential` that does implement `get_token()`. The `type: igno re` is removed; updated the matching test; added a changelog line. ### Testing `pytest providers/microsoft/azure/tests/unit/microsoft/azure/hooks/test_batch.py` passes. [@eladkal asked](https://github.com/apache/airflow/pull/67600#issuecomment-4850934389) whether this had been tested against a live account. Since the bug above only shows up once the SDK actually tries to authenticate, I validated it two ways: **1. Credential check (no Azure account needed):** ```python hasattr(AzureIdentityCredentialAdapter(None, resource_id="https://batch.core.windows.net/.default";), "get_token") # False hasattr(get_sync_default_azure_credential(), "get_token") # True ``` **2. Live pool lifecycle against a real Azure Batch account** (Azure for Students, Sweden Central), identity-based auth via Azure CLI login, no shared key: ```python from airflow.providers.microsoft.azure.hooks.batch import AzureBatchHook hook = AzureBatchHook(azure_batch_conn_id="test_live_azure_batch") client = hook.get_conn() pool = hook.configure_pool( pool_id="claude-smoketest-pool", vm_publisher="canonical", vm_offer="0001-com-ubuntu-server-jammy", vm_sku="22_04-lts", vm_node_agent_sku_id="batch.node.ubuntu 22.04", vm_size="Standard_A1_v2", target_dedicated_nodes=0, ) client.create_pool(pool) pools = list(client.list_pools()) client.begin_delete_pool("claude-smoketest-pool").result() ``` Real HTTP round trips, no mocking: ``` DefaultAzureCredential acquired a token from AzureCliCredential POST /pools → 201, pool created GET /pools → 200, pool visible DELETE /pools/claude-smoketest-pool → 202, then 404 on follow-up : confirmed deleted ``` No `AttributeError`: the exact failure mode of the pre-fix code. Test resources were deleted after validation. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (Claude Sonnet 5), reviewed and tested by the submitter. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
