[GitHub] [carbondata] Zhangshunyu commented on pull request #3986: [CARBONDATA-4034] Improve the time-consuming of Horizontal Compaction for update

2020-10-27 Thread GitBox 


Zhangshunyu commented on pull request #3986:
URL: https://github.com/apache/carbondata/pull/3986#issuecomment-717027548


   LGTM



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] Zhangshunyu commented on pull request #3986: [CARBONDATA-4034] Improve the time-consuming of Horizontal Compaction for update

2020-10-27 Thread GitBox 


Zhangshunyu commented on pull request #3986:
URL: https://github.com/apache/carbondata/pull/3986#issuecomment-717025630


   > We tested with 10 segments total 10G and update more than 20 times to see 
the cost, update row count of each time is 176973.
   > The result and comparison before and after improving is shown below, we 
can see that time-consuming of UPDATE reduces by about 30% after improving 
horizontal compaction.
   > 
   > Time Before Improving  Time after Improving
   > Average time
   > from 2nd to 21st UPDATE**99.19**   **67.55**
   > UPDATE 
   > 1st38.397  71.91
   > 2nd74.918  67.386
   > 3rd43.851  46.171
   > 4th52.133  82.974
   > 5th86.021  44.499
   > 6th62.992  39.973
   > 7th99.077  69.292
   > 8th77.815  60.416
   > 9th74.949  50.187
   > 10th   85.874  50.973
   > 11th   103.902 58.005
   > 12th   77.534  86.087
   > 13th   79.154  76.283
   > 14th   116.117 72.029
   > 15th   112.846 74.182
   > 16th   124.707 69.282
   > 17th   124.677 67.546
   > 18th   147.15  66.652
   > 19th   135.127 109.385
   > 20th   133.94  80.849
   > 21st   171.112 78.92
   
   greate!



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] Zhangshunyu commented on pull request #3986: [CARBONDATA-4034] Improve the time-consuming of Horizontal Compaction for update

2020-10-15 Thread GitBox 


Zhangshunyu commented on pull request #3986:
URL: https://github.com/apache/carbondata/pull/3986#issuecomment-709698583


   Have you ever tested this optimization? Could you pls give a comparison 
result for this change?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org