wilmerdooley commented on PR #22562: URL: https://github.com/apache/kafka/pull/22562#issuecomment-4698793294
Thanks, that makes sense. I checked the code path, and I don't see the same issue with `.process` / `.processValues`. Those are `KStream` APIs (`streams/src/main/java/org/apache/kafka/streams/kstream/KStream.java` around line 1550 and line 1580), and `KStreamImpl` connects the declared store names directly through `ProcessorToStateConnectorNode` (`streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImpl.java` around line 1306 and line 1355), which then calls `connectProcessorAndStateStores` (`streams/src/main/java/org/apache/kafka/streams/kstream/internals/graph/ProcessorToStateConnectorNode.java` around line 67). The failing path here looks specific to `KTable.transformValues`: downstream KTable joins collect store names from `valueGetterSupplier().storeNames()` (`streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableImpl.java` around line 804), and the unmaterialized transform view goes through `KTableTransformValues.view()` (`streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableTransformValues.java` around line 59). If a processed `KStream` is converted back with `toTable()`, `KStreamImpl.toTable()` creates a materialized table source (`streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImpl.java` around line 593 and line 650), so the downstream join reads that table store instead. Given that `KTable.transformValues` is in the API area being deprecated, how would you prefer we proceed? We could keep this fix on `transformValues` for its remaining lifetime, or move toward a different location/approach if that fits the removal plan better. I also noticed the current `KTable.transformValues` declarations and `ValueTransformerWithKey` interface are not annotated `@Deprecated` yet (`streams/src/main/java/org/apache/kafka/streams/kstream/KTable.java` around line 823, `streams/src/main/java/org/apache/kafka/streams/kstream/ValueTransformerWithKey.java` around line 54). If useful, I would be glad to help with those annotations or follow the relevant KIP. Please let me know which direction you prefer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
