cshannon commented on PR #3640: URL: https://github.com/apache/accumulo/pull/3640#issuecomment-1676419604
@keith-turner - This is probably ready for another review. It still needs more tests and probably some stuff needs to be cleaned up but I am curious what you think after the latest tweaks I made on Friday. I have been testing things against this PR today and on Friday and so far things are looking pretty good. I ran all the ITs that do merges and everything passes. I also modified ScanConsistencyIT (which does random flushes, compactions, merges, etc) to run for 15 minutes and there were no errors. I've also been running some tests using accumulo-testing and Uno. I just ran a test with continuous ingest where I created a table with 20 tablets and ingested about 50 million entries with no deletes (about 1.8 gigs of data). I then created 200 splits and I ran a merge on the table. Using main (so chop compactions were needed) it took ~50 seconds to complete the merge. Running the same test with my no-chop merge prototype and it took ~3 seconds. After I ran verify which completely successfully. So pretty cool to see the time drop so drastically for a merge and that difference will be even greater with more servers and data. Merge should also be even faster with Elasticity and offline tablets when not having to unload things. I just started a full IT to see what the results look like and will report back when it is finished. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
