gortiz commented on code in PR #13628:
URL: https://github.com/apache/pinot/pull/13628#discussion_r1683870200
##########
pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseSingleStageBrokerRequestHandler.java:
##########
@@ -478,8 +481,17 @@ protected BrokerResponse handleRequest(long requestId,
String query, @Nullable S
}
_brokerMetrics.addMeteredTableValue(rawTableName, BrokerMeter.QUERIES,
1);
+ _brokerMetrics.addMeteredGlobalValue(BrokerMeter.QUERIES_GLOBAL, 1);
_brokerMetrics.addValueToTableGauge(rawTableName,
BrokerGauge.REQUEST_SIZE, query.length());
+ if (!pinotQuery.isExplain() && _enableMultistageMigrationMetric) {
Review Comment:
Can we also add a config to sample the queries we are going to compare. I
mean, instead of compiling every query in V2, we can say something like:
compile 1 every X queries or a X% of the queries.
Given the high QPS Pinot usually has, this sampling should be good enough to
provide a good enough overview without having to parse all queries in V2, which
may add some impact.
Alternatively we can delegate this into a background thread that may be
reading from a concurrent queue, so running V1 queries won't be affected by the
time spent in planning V2. The important part here is that the thread adding
into the queue should be using
[Queue.offer](https://docs.oracle.com/javase/8/docs/api/java/util/Queue.html#offer-E-)
so if the queue is full it just continues (the query won't be reported, but at
the same time it won't be blocked).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
