Jackie-Jiang commented on a change in pull request #4714: Remove the segment
missing error message when acquiring segments
URL: https://github.com/apache/incubator-pinot/pull/4714#discussion_r336120908
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/query/executor/ServerQueryExecutorV1Impl.java
##########
@@ -126,8 +126,25 @@ public DataTable processQuery(ServerQueryRequest
queryRequest, ExecutorService e
TableDataManager tableDataManager =
_instanceDataManager.getTableDataManager(tableNameWithType);
Preconditions.checkState(tableDataManager != null, "Failed to find data
manager for table: " + tableNameWithType);
- List<SegmentDataManager> segmentDataManagers =
tableDataManager.acquireSegments(queryRequest.getSegmentsToQuery());
- int numSegmentsQueried = segmentDataManagers.size();
+
+ List<String> segmentsToQuery = queryRequest.getSegmentsToQuery();
+ List<SegmentDataManager> segmentDataManagers =
tableDataManager.acquireSegments(segmentsToQuery);
+
+ // When segment is removed from the IdealState:
+ // 1. Controller schedules a state transition to server to turn segment
OFFLINE
+ // 2. Server gets the state transition, removes the segment data manager
and update its CurrentState
+ // 3. Controller gathers the CurrentState and update the ExternalView
+ // 4. Broker watches ExternalView change and updates the routing table to
stop querying the segment
+ //
+ // After step 2 but before step 4, segment will be missing on server side
+ // TODO: Change broker to watch both IdealState and ExternalView to not
query the removed segments
+ int numSegmentsQueried = segmentsToQuery.size();
+ int numSegmentsAcquired = segmentDataManagers.size();
+ if (numSegmentsQueried > numSegmentsAcquired) {
+ _serverMetrics.addMeteredTableValue(tableNameWithType,
ServerMeter.NUM_MISSING_SEGMENTS,
Review comment:
This is per query basis error message, which can flood the log
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
