shenjiayu17 commented on pull request #3986: URL: https://github.com/apache/carbondata/pull/3986#issuecomment-717024597
We tested with 10 segments total 10G and update more than 20 times to see the cost, update row count of each time is 176973. The result and comparison before and after improving is shown below, we can see that time-consuming of UPDATE reduces by about 30% after improving horizontal compaction. | | Time Before Improving | Time after Improving | | ------- | ------------- | ------------- | | Average time <br> from 2nd to 21st UPDATE | **99.19** | **67.55** | | UPDATE | | | | 1st | 38.397 | 71.91 | | 2nd | 74.918 | 67.386 | | 3rd | 43.851 | 46.171 | | 4th | 52.133 | 82.974 | | 5th | 86.021 | 44.499 | | 6th | 62.992 | 39.973 | | 7th | 99.077 | 69.292 | | 8th | 77.815 | 60.416 | | 9th | 74.949 | 50.187 | | 10th | 85.874 | 50.973 | | 11th | 103.902 | 58.005 | | 12th | 77.534 | 86.087 | | 13th | 79.154 | 76.283 | | 14th | 116.117 | 72.029 | | 15th | 112.846 | 74.182 | | 16th | 124.707 | 69.282 | | 17th | 124.677 | 67.546 | | 18th | 147.15 | 66.652 | | 19th | 135.127 | 109.385 | | 20th | 133.94 | 80.849 | | 21st | 171.112 | 78.92 | ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
