snvijaya commented on pull request #1898: URL: https://github.com/apache/hadoop/pull/1898#issuecomment-630797418
> looked at, tests look good, little bit of some logic in the production code I'm querying > > Is there any way to avoid a 30s delay on every test run for the timeouts? As that will slow down the tests, and every change like this makes the tests slower and slower, costing us engineers time, our employers money *and reducing the likelihood that people run the tests > > Side issue; do those read buffer manager threads ever get released? And what happens in large JVM processes where you have many abfs fs instances?, eg. hive LLAP, Spark. Does this become a bottleneck as irrespective of the #of FS instances, the buffer size and count is hard coded. > > What I'm wondering here is should the buffer manager actually be something which belongs to a specific FS instance, uses its thread pool, and is released when the FS instance is destroyed. Timeout sleep duration in test have been reduced to 3sec. For the other issues on buffer management in Read buffer manager, will investigate separate and create JIRAs for improvement points. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
