englefly commented on code in PR #69340:
URL: https://github.com/apache/doris/pull/69340#discussion_r4333869638
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/AccessPathExpressionCollector.java:
##########
@@ -576,11 +576,54 @@ public Void visitArrayMap(ArrayMap arrayMap,
CollectorContext context) {
Expression argument = arrayMap.getArgument(0);
if ((argument instanceof Lambda)) {
- return collectArrayPathInLambda((Lambda) argument, context);
+ Lambda lambda = (Lambda) argument;
+ return collectArrayMapPathInLambda(lambda, context);
}
return visit(arrayMap, context);
}
+ private Void collectArrayMapPathInLambda(Lambda lambda, CollectorContext
context) {
+ if (isRootContainerMetadata(context)) {
+ // A metadata requirement on the ARRAY_MAP result itself (OFFSET
for cardinality(arr),
+ // NULL for arr IS NULL) describes the output ARRAY container, not
the lambda elements.
+ // The output container is shaped by the input arrays: array_map
copies the first
+ // input's offsets and its null map is the union of all inputs'
null maps. Pushing that
+ // context into the lambda body would attach the metadata to the
element layer instead,
+ // producing e.g. `arr.*.OFFSET`, which the BE rejects for a
struct element (OFFSET
+ // metadata is only supported at array/map/string level). Route
the container metadata
+ // to the input arrays, and collect the element paths the lambda
actually reads with a
+ // fresh data context so that the elements stay available to the
lambda body.
+ for (ArrayItemReference lambdaArgument :
lambda.getLambdaArguments()) {
+ continueCollectAccessPath(lambdaArgument.getArrayExpression(),
copyContext(context));
Review Comment:
Fixed in 06d50b36d3e. The finding was correct and reproducible.
`map_values` / `map_keys` produce an array whose container **is the map**:
its length is the map's entry count and its nullness is the map's nullness. So
the root OFFSET/NULL request belongs on the map column, exactly as
`visitCardinality` already normalizes `cardinality(map_values(m))` to `[m,
OFFSET]`.
- Extracted that normalization into `normalizeMapArrayProducer` and applied
it to each ARRAY_MAP input expression in `collectArrayMapPathInLambda`.
- Added unit case `testArrayMapOnMapProducerUsesMapContainerMetadata`
(map_values with a struct value and map_keys shapes) and regression table
`array_map_map_producer_pruning_tbl` with two rows, two result sections and two
explain guards (`notContains "m.VALUES.OFFSET"`, `notContains "m.KEYS.OFFSET"`).
Data used for the regression (`m MAP<STRING, STRUCT<v:INT>>`):
| id | `m` | `map_values(m)` | `array_map(x -> x.v + 1, map_values(m))` |
`cardinality(...)` |
| --- | --- | --- | --- | --- |
| 1 | `{'a': {v:1}, 'b': {v:3}}` | `[{v:1}, {v:3}]` | `[2, 4]` | `2` |
| 2 | `{'c': {v:10}}` | `[{v:10}]` | `[11]` | `1` |
Verification for `cardinality(array_map(x -> x.v + 1, map_values(m)))`:
```
before: all access paths: [m.VALUES.v, m.VALUES.OFFSET] -> [E-3110]
Invalid META access path for column 'VALUES': OFFSET metadata is not supported
at this level
after: all access paths: [m.VALUES.v, m.OFFSET] -> 2 / 1 (same as
nested column pruning disabled)
```
`map_keys` shape: `all access paths: [m.KEYS, m.OFFSET]` -> `2 / 1`.
`PruneNestedColumnTest` passes 80/80, `./build.sh --fe` passes with 0
checkstyle violations, and the whole `nereids_rules_p0/column_pruning`
directory (16 suites) passes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
