zhztheplayer commented on PR #5674: URL: https://github.com/apache/incubator-gluten/pull/5674#issuecomment-2103715824
> > Thanks for helping investigate this @Yohahaha ! > > What I am curious about is that, since we are on single-threaded execution, each `ListenableArbitrator` should be only used by a single thread. By the PR description did you mean there will be multiple threads calling the arbitrator? > > DirectBufferedInput will allocate memory in each thread of connector IO executor, thus concurrent memory request may occur. > > Velox invoke maybeIncrementReservation when growCapacityCb success before, it's ok for ListenableArbitrator since it does not care pool's reservation incrementation, but after [facebookincubator/velox#9557](https://github.com/facebookincubator/velox/pull/9557), Arbitrator need commit reservation incrementation, so the issue happens. Thank you for the explanation. I think we probably should align up with Velox community for arbitrator API. I feel the API is not constantly stable for `ListenableArbitrator` as we should frequently change our code to match their changes. :( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
