[GitHub] [carbondata] Zhangshunyu commented on pull request #3986: [CARBONDATA-4034] Improve the time-consuming of Horizontal Compaction for update
Zhangshunyu commented on pull request #3986: URL: https://github.com/apache/carbondata/pull/3986#issuecomment-717027548 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Zhangshunyu commented on pull request #3986: [CARBONDATA-4034] Improve the time-consuming of Horizontal Compaction for update
Zhangshunyu commented on pull request #3986: URL: https://github.com/apache/carbondata/pull/3986#issuecomment-717025630 > We tested with 10 segments total 10G and update more than 20 times to see the cost, update row count of each time is 176973. > The result and comparison before and after improving is shown below, we can see that time-consuming of UPDATE reduces by about 30% after improving horizontal compaction. > > Time Before Improving Time after Improving > Average time > from 2nd to 21st UPDATE**99.19** **67.55** > UPDATE > 1st38.397 71.91 > 2nd74.918 67.386 > 3rd43.851 46.171 > 4th52.133 82.974 > 5th86.021 44.499 > 6th62.992 39.973 > 7th99.077 69.292 > 8th77.815 60.416 > 9th74.949 50.187 > 10th 85.874 50.973 > 11th 103.902 58.005 > 12th 77.534 86.087 > 13th 79.154 76.283 > 14th 116.117 72.029 > 15th 112.846 74.182 > 16th 124.707 69.282 > 17th 124.677 67.546 > 18th 147.15 66.652 > 19th 135.127 109.385 > 20th 133.94 80.849 > 21st 171.112 78.92 greate! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Zhangshunyu commented on pull request #3986: [CARBONDATA-4034] Improve the time-consuming of Horizontal Compaction for update
Zhangshunyu commented on pull request #3986: URL: https://github.com/apache/carbondata/pull/3986#issuecomment-709698583 Have you ever tested this optimization? Could you pls give a comparison result for this change? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org