siddharthteotia commented on code in PR #9117:
URL: https://github.com/apache/pinot/pull/9117#discussion_r932918332
##########
pinot-common/src/main/java/org/apache/pinot/common/utils/DataTable.java:
##########
@@ -123,7 +123,12 @@ enum MetadataKey {
NUM_SEGMENTS_PRUNED_BY_LIMIT("numSegmentsPrunedByLimit",
MetadataValueType.INT),
NUM_SEGMENTS_PRUNED_BY_VALUE("numSegmentsPrunedByValue",
MetadataValueType.INT),
EXPLAIN_PLAN_NUM_EMPTY_FILTER_SEGMENTS("explainPlanNumEmptyFilterSegments",
MetadataValueType.INT),
-
EXPLAIN_PLAN_NUM_MATCH_ALL_FILTER_SEGMENTS("explainPlanNumMatchAllFilterSegments",
MetadataValueType.INT);
+
EXPLAIN_PLAN_NUM_MATCH_ALL_FILTER_SEGMENTS("explainPlanNumMatchAllFilterSegments",
MetadataValueType.INT),
+ // TODO: Add a mechanism to pass generic metadata without adding them to
the DataTable's metadata
+ // This metadata field is only required for emitting a metric to identify
whether queries using MV columns (as
+ // identifiers or via transform) are present in the Selection Order-By
queries. This was the only place where
+ // such a metadata could be added in the existing interfaces even though
we don't need this on the Broker side.
+ QUERY_HAS_MV_SELECTION_ORDER_BY("queryHasMVSelectionOrderBy",
MetadataValueType.STRING);
Review Comment:
May be there is no need to log the requestID etc to be able to correlate
with the offending query on broker. All we can do is simply log the ORDER BY
expressions
`"Table <T> has MV column in ORDER BY. Expressions: <ORDER BY mvCol1,
mvCol2>"`
Check table level metric in internal dashboard, in the server log we grep
for "has MV column in ORDER BY", and use the ORDER BY expressions to grep in
broker log and find full offending query.
So, we just need the following change
- Pass ServerMetrics to plan maker
- From PlanMaker pass to SelectionOrderByPlanNode / InstanceResponsePlanNode
- Emit metric
- Log MV ORDER BY expressions and tableName available in QueryContext in
server log for easy grep and mapping back to full query in broker log
- Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
