Tim Armstrong has posted comments on this change. Change subject: IMPALA-5417: make I/O buffer queue fixed-size ......................................................................
Patch Set 7: I thought about it some more and I'm not sure that adding meaningful timers is simple. We already have TotalStorageWaitTime that measures the time the client spent blocked for a fragment. It might make sense to add a timer per exec node called "IoWaitTime" or "StorageWaitTime" that gives the granular timing. It's harder to produce a meaningful time for how long the queue was full, i.e. we were blocked on compute, because there could be many scan ranges per thread. We could include a counter of the number of times the queue was filled up, e.g. "IoBufferQueueFullCount", but that might not be terribly intuitive. I think that would give us enough information to see if the queue was too small - in that case we'd see significant IoWaitTime and non-zero IoBufferQueueFullCount -- To view, visit http://gerrit.cloudera.org:8080/7408 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: If7cc3f7199f5320db00b7face97a96cdadb6f83f Gerrit-PatchSet: 7 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong <[email protected]> Gerrit-Reviewer: Matthew Jacobs <[email protected]> Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Michael Ho <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-HasComments: No
