Yeah, i know that. Here is the jira bug i filed: https://issues.apache.org/jira/browse/KYLIN-619
On Mon, Mar 2, 2015 at 11:47 PM, Luke Han <[email protected]> wrote: > Hi Jakob, did you create JIRA for this already? > > Tez itself should not the problem but just some statistics information > Kylin can't parse as you figure out already. > > Please let me know the JIRA number and we will enhance that to avoid such > issue again. > > Thanks. > Luke > > > Best Regards! > --------------------- > > Luke Han > > 2015-03-03 2:38 GMT+08:00 Jakob Stengård <[email protected]>: > > > I looked at the source code, but the fix wasn't obvious to me. > > > > Maybe, if i have time, i'll look into it further. > > > > /J > > > > > > On Mon, Mar 2, 2015 at 7:13 PM, Adunuthula, Seshu <[email protected]> > > wrote: > > > > > Thanks Jakob, > > > > > > Would be fantastic if you could dig in and submit a patch… > > > > > > > > > On 3/2/15, 10:08 AM, "Jakob Stengård" <[email protected]> > wrote: > > > > > > >I'll file a bug in jira. > > > >For your information i also happen to know that tez boosted map-reduce > > > >doesn't work either, but there, the problem seems to have to do with > the > > > >map reduce jobs them selves failing. > > > > > > > >In this case however, from what i've read on git hub about the "Can't > > get > > > >cube source record size", > > > >error", it could possibly be a problem with that tez doesn't output a > > > >string on the form: > > > >"HDFS Read: (\d+) HDFS Write" > > > > > > > >However, it could be something else, it could also be my logging > > > >configuration for tez. > > > > > > > >/J > > > > > > > > > > > > > > > >On Mon, Mar 2, 2015 at 6:54 PM, Adunuthula, Seshu < > [email protected] > > > > > > >wrote: > > > > > > > >> Did not read your logs fully. The culprit seems to be > > > >> > > > >> > > > >> 1. [QuartzScheduler_Worker-7]:[2015-03-02 > > > >> > > > > > > >>16:09:24,737][ERROR][com.kylinolap.job.flow.JobFlowListener.handleExcepti > > > >>on > > > >> (JobFlowListener.java:312)] - Can't get cube source record size. > > > >> > > > >> source_records_size may have to be computed differently when using > Tez > > > >>as > > > >> the Hive engine. I will let Yang/Luke answer this when they come > back > > > >> onlineŠ > > > >> > > > >> > > > >> > > > >> > > > >> On 3/2/15, 9:30 AM, "Adunuthula, Seshu" <[email protected]> > wrote: > > > >> > > > >> >Jakob, > > > >> > > > > >> >Could you go ahead and file a JIRA with the stack trace at > > > >> >https://issues.apache.org/jira/browse/KYLIN/ > > > >> > > > > >> >Also include the App Master and Map/Reduce Task logs for the Tasks > > that > > > >> >failed. > > > >> > > > > >> >Regards > > > >> >Seshu > > > >> > > > > >> >On 3/2/15, 8:12 AM, "Jakob Stengård" <[email protected]> > > wrote: > > > >> > > > > >> >>Correction: That, isn't the last job, it's step 9. > > > >> >> > > > >> >>/J > > > >> >> > > > >> >>On Mon, Mar 2, 2015 at 5:10 PM, Jakob Stengård > > > >><[email protected] > > > >> > > > > >> >>wrote: > > > >> >> > > > >> >>> Well, this only happens with > > > >> >>> <property> > > > >> >>> <name>hive.execution.engine</name> > > > >> >>> <value>tez</value> > > > >> >>> </property> > > > >> >>> > > > >> >>> If i use mr, it works fine. > > > >> >>> > > > >> >>> > > > >> >>> From what i can see form the job log, none of the previous > > > >>map-reduce > > > >> >>>jobs > > > >> >>> fail, but the build fails at the last step. > > > >> >>> Log from the last mr job for example: > > > >> >>> http://pastebin.com/jNWSRKLi > > > >> >>> > > > >> >>> > > > >> >>> Is tez supposed to work? Because in that case, maybe it's just > my > > > >> >>>setup. > > > >> >>> > > > >> >>> /Jakob > > > >> >>> > > > >> >>> > > > >> >>> > > > >> >>> On Mon, Mar 2, 2015 at 4:34 PM, Santosh Akhilesh < > > > >> >>> [email protected]> wrote: > > > >> >>> > > > >> >>>> This doesn't seems to be complete kylin.log. Do you have any > > > >>exception > > > >> >>>> related to MR counter in map reduce logs? If yes then this > needs > > > >>to be > > > >> >>>> checked because kylin will need the MR counter to calculate the > > > >>cube > > > >> >>>>size. > > > >> >>>> On Mon, 2 Mar 2015 at 8:49 pm, Jakob Stengård > > > >> >>>><[email protected]> > > > >> >>>> wrote: > > > >> >>>> > > > >> >>>> > It breaks on the last step: "Load HFile to HBase Table": > > > >> >>>> > > > > >> >>>> > Kylin.log: > > > >> >>>> > http://pastebin.com/3kAQmJ3u > > > >> >>>> > > > > >> >>>> > On Mon, Mar 2, 2015 at 2:27 PM, Adunuthula, Seshu > > > >> >>>><[email protected] > > > >> >>>> > > > > >> >>>> > wrote: > > > >> >>>> > > > > >> >>>> > > Jakob, > > > >> >>>> > > > > > >> >>>> > > Could you please send out the complete stack trace with Tez > > > >> >>>>enabled? > > > >> >>>> > > > > > >> >>>> > > Regards > > > >> >>>> > > Seshu > > > >> >>>> > > > > > >> >>>> > > On 3/2/15, 4:51 AM, "Jakob Stengård" < > > [email protected] > > > > > > > >> >>>> wrote: > > > >> >>>> > > > > > >> >>>> > > >Hi > > > >> >>>> > > >Will kylin suport hive+tez in v0.7.x? > > > >> >>>> > > > > > > >> >>>> > > >I've tried using tez with hive, but currently the cube > build > > > >> >>>>fails > > > >> >>>> with > > > >> >>>> > > >"Can't get cube source record size", because kylin expects > > > >> >>>>certain > > > >> >>>> > console > > > >> >>>> > > >output from map-reduce, and tez doesn't print this. > > > >> >>>> > > > > > > >> >>>> > > >BG. > > > >> >>>> > > >Jakob Stengard > > > >> >>>> > > > > > >> >>>> > > > > > >> >>>> > > > > >> >>>> > > > >> >>> > > > >> >>> > > > >> > > > > >> > > > >> > > > > > > > > >
