steveloughran commented on pull request #2971: URL: https://github.com/apache/hadoop/pull/2971#issuecomment-933812976
FWIW, I plan to add read and write rate limiters (configurable) for all read ops done by a stage, and all write ops, both configurable, default to be whatever abfs likes. The goal is to throttle back rename in job commit without having to try and do it indirectly through thread pool sizing ... especially as stages biased towards reading (load manifests, create directories) would be happy with large thread pools; it's only rename & delete where we'd really want to throttle back -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
