elek commented on issue #551: HDDS-2717. Handle chunk increments in datanode URL: https://github.com/apache/hadoop-ozone/pull/551#issuecomment-597565621 I tested with this path with teragen (100GB) and compared the numbers with the current master. 1. I see high variance in the test (same test can provide +/- 10% performance at a different time) 2. During my first test (used an old version of master / old version of this branch) the patch was slower: https://hackmd.io/HbYNWTE2SXyOgZQ6CeXaXg 3. Yesterday I tested the final version and couldn't reproduce any slowness: https://hackmd.io/P3Hkf09fTLS78Ho4W82YbQ * This patch always provided better performance than the master 4. Using load tests (freon) instead of teragen I have seen significant better performance. The current implementation uses a lot of unnecessary file.exists check which makes impossible to get better performance than 1.2 k / sec chunk write. This patch (even with the old layout) can easily provide > 4k without any adjustment.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
