homatthew commented on code in PR #3586:
URL: https://github.com/apache/gobblin/pull/3586#discussion_r999770296
##########
gobblin-yarn/src/main/java/org/apache/gobblin/yarn/YarnService.java:
##########
@@ -789,11 +815,17 @@ private ImmutableMap.Builder<String, String>
buildContainerStatusEventMetadata(C
/**
* Get the number of matching container requests for the specified resource
memory and cores.
+ * Due to YARN-1902, this API is not 100% accurate. However, {@link
AMRMClientCallbackHandler#onContainersAllocated(List)}
+ * contains logic for best effort clean up of duplicate requests, and in
practice is pretty accurate.
Review Comment:
This comment is in reference to line `892`. The accuracy of this API request
is actually very subtle due to
[`Yarn-1902`](https://issues.apache.org/jira/browse/YARN-1902). The workaround
is best effort but in my testing was surprisingly accurate.
While testing, I deleted the if statement code block near `892` and the
number of container requests exploded. See the details in the ticket if anyone
is curious
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
