SEPURI-SAI-KRISHNA opened a new pull request, #76900: URL: https://github.com/apache/airflow/pull/76900
`DataplexCreateTaskOperator` and `DataplexCreateLakeOperator` document `validate_only` as "Only validate the request, but do not perform mutations. The default is false.", carry it in `template_fields`, and forward it to `DataplexHook.create_task` and `DataplexHook.create_lake`. Both hook methods accept the argument and then leave it out of the request they send, so the service never sees it and the resource is created anyway. Setting `validate_only=True` therefore does not dry run. It creates the task or the lake. This is not a parameter that does nothing, it is a parameter that does the opposite of what it documents, which is why I am fixing it rather than filing it. The field exists on the API: `CreateTaskRequest` and `CreateLakeRequest` both expose `validate_only`. The convention is already established in the same hook, where `create_aspect_type`, `create_entry_type`, `create_entry_group`, `update_aspect_type`, `update_entry_type` and `update_entry_group` all put `validate_only` into the request dict. These two are the only `DataplexHook` methods that take it and drop it. The default is unaffected. Both signatures default to `None`, and proto-plus stores `None` for a `bool` field as `False`, so forwarding it leaves `validate_only` unset exactly as before for anyone who does not pass it. On scope, I checked every `validate_only` site in the repository. 26 functions take the parameter, and these two are the only ones that accept it and never reference it again. The other 24 are 14 in `AlloyDbHook`, the 6 `DataplexHook` catalog methods above, `GoogleSearchAdsReportingHook.search`, and the three operator constructors that store and forward it. No Dataplex zone or asset method takes `validate_only` at all, so nothing else is in scope. It has never worked. Both methods arrived in #20377 with the parameter documented and absent from the request, and it stayed that way through the provider restructure in #46344. **Behaviour change** Anyone who set `validate_only=True` has been getting a real task or lake. After this they get validation only, so a Dag that set it and then relied on the resource existing will start failing on the following step. That is the documented contract being restored rather than a new behaviour, but it is visible, and I am happy to add a note to the provider changelog if you would prefer the change called out there. **One thing I deliberately did not change** With `validate_only=True` now reaching the service, both operators still log "Task ... created successfully" and "Lake ... created successfully" after `wait_for_operation`, because that message is not conditional on `validate_only`. It is misleading. Correcting it means deciding what the operators should log and return for a validation only run, and I could not exercise the live service to see what the long running operation looks like in that case, so I left it rather than guess. Happy to follow up separately if you would like it addressed. **Tests** `test_create_task` and `test_create_lake` already called the hook with `validate_only=None` and asserted the whole request, so they were pinning the bug in place. Both are now parametrized over `None`, `False` and `True` and assert `validate_only` inside the request, which is how `test_create_entry_group` and the other catalog tests already do it. All six cases fail with the two added lines removed and the six sibling methods left intact, so the tests are tied to this change and not to the shared convention. The dataplex hook and operator suites pass together at 80 tests, and `ruff format` and `ruff check` are clean on both changed files. No changelog entry by default: provider release managers regenerate the changelog from `git log`, so per-PR entries are not consumed here. No existing issue; opening the PR directly rather than filing one first. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes - Claude Code (Opus 5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
