There’s no error information. The failure reason is the MR job time exceeds the timeout value, for some mappers retries their jobs. In this kind of situation, the total time of layered cubing may be more that immemory cubing. However, time of each layered job is less than the time of immemory cubing, which will be better for finishing the MR job in cluster.
There’s one case like this: With "-Xmx3600m”, 22 of 27 mappers failed; While with "-Xmx7040m”, 5 of 27 mappers failed; Maybe just increasing the maximum jvm memory is not good. Above all, it may be better to provide the option for clients to manually choose which strategy to use. Best regards, Yanghong Zhong > On Mar 26, 2016, at 4:23 PM, ShaoFeng Shi <[email protected]> wrote: > > Hi Yanghong, > > What's the detail error in such a failed MR job? > > 2016-03-26 7:43 GMT+08:00 Zhong, Yanghong <[email protected]>: > >> For some cube building jobs, kylin chooses im-memory cubing. However, this >> choice is not good for some users due to large memory cost. Customers may >> be able to tolerate with long cube building time, but is not able to >> tolerate with large memory cost, which may lead to cube building MR job >> failure. Therefore, it may be better to provide a parameter for setting >> whether automatically decide the strategy or manually. >> >> Best regards, >> Yanghong Zhong >> [email protected] >> > > > > -- > Best regards, > > Shaofeng Shi
