Hi shaofeng, Yes, it’s usually smaller then the sum of each segment, but usually a small amount compared with the total size.
But for the statistics estimate, usually result in a N times larger than it actually be, and results in a huge waste of HBase region numbers。 1. Do you have any data about deviation of the two ways in statistics? I mean generally which way will be closer? 2. Is there any improve plan for this in the roadmap? Or some consideration to give more options to user to select their own estimate algo? Thanks Qilong 发件人: ShaoFeng Shi <[email protected]<mailto:[email protected]>> 答复: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> 日期: 2018年1月23日 星期二 09:43 至: user <[email protected]<mailto:[email protected]>> 抄送: 林豪（linhao）-技术产品中心 <[email protected]<mailto:[email protected]>> 主题: Re: segment size estimate when merging Hi Qilong, When merging segments, the dimension-measure values (k-v) will be re-orged and the same key will be merged, so the merged size is not simply a sum of each segment; usually, it is smaller than before. Always using the statistics to estimate the size is for consistency. Of course, there is room to improve the estimation accuracy. 2018-01-22 16:54 GMT+08:00 苏启龙 <[email protected]<mailto:[email protected]>>: Hi， We have some unclear points about the segment size estimate when merging multi-segments. We find that the segment merge job still uses CubeStatsReader::getCuboidSizeMap to estimate the total size of the merged segment. From our understanding, when building a new segment, Kylin uses this way to estimate the total size is OK since no other info we can turn to. But in merging we may sum the table size of the segments to be merged, which should be more accurate. So why for this consideration? Su Qilong -- Best regards, Shaofeng Shi 史少锋
