aoelvp94 opened a new pull request, #77249:
URL: https://github.com/apache/airflow/pull/77249
`Frame` encodes a `SetXCom` / `SetTaskStateStore` request by running Jackson
`convertValue` on the whole message object. A Scala `Map`, `Seq`, `Option` or
case class (none of which are `java.util.Map` / `java.util.Collection`) was
stored as that object's bean properties, or as `{}` when it had none, and the
task succeeded. `packAny` never saw the original type.
This inspects the original value in `setXCom` and `taskStateStore.set`
before it is placed on the message, and throws `IllegalArgumentException`
naming the type, with a hint to convert Scala collections via
`scala.jdk.CollectionConverters` (`.asJava`) and a case class to a `Map` or
list.
The new check is deliberately narrow, so existing Java and Kotlin callers
keep the previous encoding:
- **Rejected:** Scala types, detected by a `scala.` prefix on the class, any
superclass or any interface (`scala.collection.*`, `scala.Option` / `Some` /
`None`, `scala.Product` for case classes, `scala.Tuple*`). No Scala dependency
is added to the SDK.
- **Rejected:** any other object that the same Jackson mapper would encode
as `{}` (no serializable properties). An empty `Map` is still valid JSON and is
left alone.
- **Unchanged:** JSON scalars, maps and lists; Java POJOs, records and
Kotlin data classes with properties; enums; `java.time` / `Date`; and other
types Jackson already encoded. The original value is passed through, so the
wire shape is the same as before.
- Nested values in a `Map`, `Collection` or array are checked the same way.
Byte arrays keep the existing Base64 error.
The Java SDK guide and `Client` / `TaskStateStore` KDoc now state this
contract.
A follow-up docs PR will document how to use the Java SDK from Scala (sbt
bundle, `TaskInput`, XCom conversions) and describe this error.
related: #66329
No newsfragment is included. `java-sdk/DEVELOPMENT.md` asks for an
`airflow-core/newsfragments/` entry for user-visible changes to `task-sdk/`
(the Python Task SDK). This is a Java SDK behaviour change. Happy to add
`{pr_number}.bugfix.rst` (or `.significant.rst`) if reviewers want one after
the PR number is known.
## How to test
Unit tests (from a checkout with this patch applied; the Scala docs patch
may also be applied — they do not conflict):
```bash
cd java-sdk
./gradlew :sdk:check
```
Expected: 426 tests, 0 failures, including `JsonValueTest` and the updated
`CommTest`, plus ktlint and spotless.
End-to-end against Airflow 3.4.0 with real Scala (a self-contained reviewer
demo; overlay the fix onto official `airflow-sdk` 1.0.0 because 3.4.0 rejects
the `1.1.0-SNAPSHOT` supervisor schema `2027-02-28`):
Demo (step-by-step README, expected baseline vs patched output, exact error
text):
https://github.com/aoelvp94/airflow-scala-spark-demo/tree/setxcom-loud-failure-demo/setxcom-demo
```bash
git clone -b setxcom-loud-failure-demo
https://github.com/aoelvp94/airflow-scala-spark-demo.git && cd
airflow-scala-spark-demo/setxcom-demo
docker build -t airflow-scala-docs-e2e:3.4.0 -f Dockerfile .
export AIRFLOW_SRC=/path/to/this/airflow/checkout
make bundle
make overlay
make baseline # DAG succeeds; Scala types silently stored as bean
properties / {}
make patched # DAG fails; Scala tasks raise IllegalArgumentException and
write no XCom; Java controls unchanged
make evidence
```
See the demo README for the exact error text and the per-task expected
values.
---
##### Was generative AI tooling used to co-author this PR?
- [x] Yes (please specify the tool below)
Generated-by: Grok Bot following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
---
* Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information. Note: commit author/co-author name and email in commits
become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment:
`{pr_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
You can add this file in a follow-up commit after the PR is created so you
know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]