adoroszlai opened a new pull request #1349: HDDS-2026. Overlapping chunk region cannot be read concurrently URL: https://github.com/apache/hadoop/pull/1349 ## What changes were proposed in this pull request? Only allow a single read/write operation for the same path in `ChunkUtils`, to avoid `OverlappingFileLockException` due to concurrent reads. This allows concurrent reads/writes of separate files (as opposed to simply synchronizing the methods). It might be improved later by storing and reusing the file lock. Use plain `FileChannel` instead of `AsynchronousFileChannel` for reading, too, since it was used in synchronous fashion (by calling `.get()`) anyway. https://issues.apache.org/jira/browse/HDDS-2026 ## How was this patch tested? Added unit test. Used improved Freon tool from #1341 to perform read of same key from multiple threads (which revealed the bug in the first place). ``` $ ozone freon ockg -n 1 -p asdf $ ozone sh key list vol1/bucket1 [ { "version" : 0, "size" : 10240, "keyName" : "asdf/0" ... } ] $ ozone freon ocokr -k 'asdf/0' ... mean rate = 164.39 calls/second ... mean = 53.75 milliseconds stddev = 42.11 milliseconds median = 44.05 milliseconds ... Total execution time (sec): 6 Failures: 0 Successful executions: 1000 ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
