tarun11Mavani commented on code in PR #19040:
URL: https://github.com/apache/pinot/pull/19040#discussion_r3682629475
##########
pinot-core/src/main/java/org/apache/pinot/core/plan/AggregationPlanNode.java:
##########
@@ -204,6 +222,55 @@ private boolean isFitForNonScanBasedPlan() {
return true;
}
+ @Nullable
+ private DataSource resolveDataSource(ExpressionContext expression) {
+ return resolveDataSource(expression, _indexSegment,
_queryContext.getSchema());
+ }
+
+ @Nullable
+ static DataSource resolveDataSource(ExpressionContext expression,
IndexSegment segment,
+ @Nullable org.apache.pinot.spi.data.Schema schema) {
+ if (expression.getType() == ExpressionContext.Type.IDENTIFIER) {
+ return segment.getDataSource(expression.getIdentifier(), schema);
+ }
+ if (expression.getType() == ExpressionContext.Type.FUNCTION) {
+ return tryResolveKeyedDataSource(expression, segment, schema);
+ }
+ return null;
+ }
+
+ @Nullable
+ static DataSource tryResolveKeyedDataSource(ExpressionContext expression,
IndexSegment segment,
+ @Nullable org.apache.pinot.spi.data.Schema schema) {
+ FunctionContext function = expression.getFunction();
+ if (function == null
+ ||
!ItemTransformFunction.FUNCTION_NAME.equals(function.getFunctionName())) {
+ return null;
+ }
+ List<ExpressionContext> args = function.getArguments();
+ if (args.size() != 2
+ || args.get(0).getType() != ExpressionContext.Type.IDENTIFIER
+ || args.get(1).getType() != ExpressionContext.Type.LITERAL) {
+ return null;
+ }
+ String columnName = args.get(0).getIdentifier();
+ String key = args.get(1).getLiteral().getStringValue();
+ DataSource columnDs = segment.getDataSource(columnName, schema);
+ if (columnDs instanceof MapDataSource) {
+ DataSource keyDs = ((MapDataSource) columnDs).getDataSource(key);
+ // An absent MAP key yields a NullDataSource whose metadata reports
non-null min/max (the INT
+ // default) and which carries no null vector, so it would wrongly
satisfy the metadata-based
+ // non-scan check and report hasNullValues=false. Fall back to a scan,
mirroring the
+ // OPEN_STRUCT guard below.
+ return keyDs instanceof NullDataSource ? null : keyDs;
+ }
+ if (columnDs instanceof OpenStructDataSource) {
+ OpenStructDataSource osDs = (OpenStructDataSource) columnDs;
+ return osDs.isMaterialized(key) ? osDs.getDataSource(key) : null;
Review Comment:
Confirmed. With the reserved default in the mutable dictionary, a
partially-present key's dictionary now has identical contents to the sealed one
(default + observed values), so the dictionary-based path is correct there.
The one divergent case — key present in every doc with the default never
observed (phantom entry) forces the scan path via a new
`OpenStructDataSource.isKeyDictionaryExact`
The parity test pins MIN/MAX/DISTINCTCOUNT consuming vs sealed with null
handling off.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
