AHeise commented on code in PR #29415:
URL: https://github.com/apache/flink/pull/29415#discussion_r4206635519
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/rules/logical/PushProjectIntoTableSourceScanRule.java:
##########
@@ -348,7 +352,10 @@ private RowType performPushDown(
.map(col ->
col.getMetadataKey().orElse(col.getName()))
.collect(Collectors.toList());
- abilitySpecs.add(new ReadingMetadataSpec(projectedMetadataKeys,
newProducedType));
+ // An earlier ReadingMetadataSpec must be narrowed even if no
metadata is projected.
+ if (!projectedMetadataKeys.isEmpty() ||
hasReadingMetadataSpec(source)) {
+ abilitySpecs.add(new
ReadingMetadataSpec(projectedMetadataKeys, newProducedType));
+ }
Review Comment:
Checked all `SupportsReadingMetadata` table sources in the apache connector
repos (kafka, aws/kinesis, pulsar, cdc ×7, rocketmq ×2, http) plus filesystem
and `ExternalDynamicSource` in core. All but one start with the physical type
and no metadata keys, so a missing call equals an empty one.
The exception is `HttpLookupTableSource`: it sets `producedDataType` only in
`applyReadableMetadata`, and the lookup function drops the row instead of
returning null enrichment columns for a failed call with continue-on-error if
it is unset. It already does that today when no projection is pushed, so I
filed FLINK-40940 to fix the connector rather than keep the planner calling it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
