Hi Joel, I'm also curious about how the new count step took 6 minutes, which is about 1/3 of the total time; if your fact table a Hive View? or it is not partitioned? Usually that step should be much faster than others;
Thanks. 2016-08-02 22:58 GMT+08:00 ShaoFeng Shi <[email protected]>: > Hi Joel, > > I see your point, but I don't have a perfect idea so far; We can discuss > here, any comment is welcomed. > > > > 2016-08-02 18:43 GMT+08:00 Joel Victor <[email protected]>: > >> Is there any way to disable this new step that has been added to the >> build process. https://issues.apache.org/jira/browse/KYLIN-1656 >> >> This adds a new step which counts the number of records at the beginning >> of each build. For my cube builds it does not benefit me much since my >> build latencies have gone up from 11 minutes to 18 minutes where >> approximately 6 minutes is taken up by this new count step. >> >> My extract fact table distinct column used to take ~3 minutes and now >> takes ~2 but at the price of a 6 minute increased latency. >> Going through patch I don't see anyway to disable it. Please let me know >> if there is any. >> >> NOTE: My cube builds are for every 30 minutes of data which is a very >> small interval and also has less amount of data. It seems this count step >> isn't very beneficial when the data is small. >> >> Thanks, >> -Joel >> >> >> > > > -- > Best regards, > > Shaofeng Shi > > -- Best regards, Shaofeng Shi
