junaiddshaukat opened a new pull request, #39051:
URL: https://github.com/apache/beam/pull/39051
## Summary
A Kafka Serde for `KStreamsPayload` so the envelope can cross a topic
boundary.
Until now it only flowed in-JVM via `ProcessorContext#forward`; GroupByKey
introduces the first real topic (the key-based repartition topic, per the
plan
agreed with @je-ik), which needs the payload serialized. Split out as its own
small PR ahead of GBK.
## Scope
- `KStreamsPayloadSerde<T>`: parameterized by the `Coder<WindowedValue<T>>`
for
the data variant (different topics carry different element types; the
watermark
variant is coder-independent). Wire format: a one-byte discriminator +
body —
data = `[0x00][windowedValueCoder-encoded value]`; watermark =
`[0x01][long millis][int sourcePartition][int totalSourcePartitions]`.
- Unit tests: round-trip of data, watermark, and terminal MAX watermark,
plus an
unknown-tag failure.
## Out of scope
- Wiring the serde into the GBK repartition / watermark fan-out — that's the
GroupByKey PR.
## Notes
- Assumes non-null payloads: the topics it's used on (repartition, watermark
fan-out) are not log-compacted, so no tombstones occur.
## Testing
`./gradlew :runners:kafka-streams:check` green; 4 unit tests.
Closes #39042
Refs #18479
cc @je-ik
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
