fqshopify commented on code in PR #174:
URL:
https://github.com/apache/flink-connector-kafka/pull/174#discussion_r3197241328
##########
flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/table/KafkaDynamicSource.java:
##########
@@ -72,13 +74,15 @@
import java.util.Properties;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
-import java.util.stream.IntStream;
import java.util.stream.Stream;
/** A version-agnostic Kafka {@link ScanTableSource}. */
@Internal
public class KafkaDynamicSource
- implements ScanTableSource, SupportsReadingMetadata,
SupportsWatermarkPushDown {
+ implements ScanTableSource,
+ SupportsReadingMetadata,
+ SupportsWatermarkPushDown,
+ SupportsProjectionPushDown {
Review Comment:
The interface is always honored. `applyProjection` updates
`producedDataType` and the projection is applied at runtime by the
post-deserialization `Projector` regardless of `*_PROJECTION_PUSHDOWN_LEVEL`.
The option only controls whether projection is *additionally* pushed into
the format (when it implements `ProjectableDecodingFormat`):
- `level=FULL` + `ProjectableDecodingFormat` => projection happens inside
the deserializer (skip unneeded bytes).
- Otherwise (level=`NONE`, non-projectable format, upsert mode) => format
deserializes the full physical row, then `Projector` trims it down before emit.
Either way the source emits the projected row, so downstream operators see
exactly what they would with a post-source Calc - no optimizations are lost.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
