Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19168
Technically speaking, `maxBytesInFlight` controls how many bytes are being
transferred in network, so we should decrease `bytesInFlight` once we get data
from the result queue. The actual memory usage usually beyond
`maxBytesInFlight`, because the network transfer needs memory, and the fetched
data which is being processed also needs memory. This is not a big problem
because the worst case is we consume `maxBytesInFlight * 2` memory.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
