I looked at the source code, but the fix wasn't obvious to me. Maybe, if i have time, i'll look into it further.
/J On Mon, Mar 2, 2015 at 7:13 PM, Adunuthula, Seshu <[email protected]> wrote: > Thanks Jakob, > > Would be fantastic if you could dig in and submit a patch… > > > On 3/2/15, 10:08 AM, "Jakob Stengård" <[email protected]> wrote: > > >I'll file a bug in jira. > >For your information i also happen to know that tez boosted map-reduce > >doesn't work either, but there, the problem seems to have to do with the > >map reduce jobs them selves failing. > > > >In this case however, from what i've read on git hub about the "Can't get > >cube source record size", > >error", it could possibly be a problem with that tez doesn't output a > >string on the form: > >"HDFS Read: (\d+) HDFS Write" > > > >However, it could be something else, it could also be my logging > >configuration for tez. > > > >/J > > > > > > > >On Mon, Mar 2, 2015 at 6:54 PM, Adunuthula, Seshu <[email protected]> > >wrote: > > > >> Did not read your logs fully. The culprit seems to be > >> > >> > >> 1. [QuartzScheduler_Worker-7]:[2015-03-02 > >> > >>16:09:24,737][ERROR][com.kylinolap.job.flow.JobFlowListener.handleExcepti > >>on > >> (JobFlowListener.java:312)] - Can't get cube source record size. > >> > >> source_records_size may have to be computed differently when using Tez > >>as > >> the Hive engine. I will let Yang/Luke answer this when they come back > >> onlineŠ > >> > >> > >> > >> > >> On 3/2/15, 9:30 AM, "Adunuthula, Seshu" <[email protected]> wrote: > >> > >> >Jakob, > >> > > >> >Could you go ahead and file a JIRA with the stack trace at > >> >https://issues.apache.org/jira/browse/KYLIN/ > >> > > >> >Also include the App Master and Map/Reduce Task logs for the Tasks that > >> >failed. > >> > > >> >Regards > >> >Seshu > >> > > >> >On 3/2/15, 8:12 AM, "Jakob Stengård" <[email protected]> wrote: > >> > > >> >>Correction: That, isn't the last job, it's step 9. > >> >> > >> >>/J > >> >> > >> >>On Mon, Mar 2, 2015 at 5:10 PM, Jakob Stengård > >><[email protected] > >> > > >> >>wrote: > >> >> > >> >>> Well, this only happens with > >> >>> <property> > >> >>> <name>hive.execution.engine</name> > >> >>> <value>tez</value> > >> >>> </property> > >> >>> > >> >>> If i use mr, it works fine. > >> >>> > >> >>> > >> >>> From what i can see form the job log, none of the previous > >>map-reduce > >> >>>jobs > >> >>> fail, but the build fails at the last step. > >> >>> Log from the last mr job for example: > >> >>> http://pastebin.com/jNWSRKLi > >> >>> > >> >>> > >> >>> Is tez supposed to work? Because in that case, maybe it's just my > >> >>>setup. > >> >>> > >> >>> /Jakob > >> >>> > >> >>> > >> >>> > >> >>> On Mon, Mar 2, 2015 at 4:34 PM, Santosh Akhilesh < > >> >>> [email protected]> wrote: > >> >>> > >> >>>> This doesn't seems to be complete kylin.log. Do you have any > >>exception > >> >>>> related to MR counter in map reduce logs? If yes then this needs > >>to be > >> >>>> checked because kylin will need the MR counter to calculate the > >>cube > >> >>>>size. > >> >>>> On Mon, 2 Mar 2015 at 8:49 pm, Jakob Stengård > >> >>>><[email protected]> > >> >>>> wrote: > >> >>>> > >> >>>> > It breaks on the last step: "Load HFile to HBase Table": > >> >>>> > > >> >>>> > Kylin.log: > >> >>>> > http://pastebin.com/3kAQmJ3u > >> >>>> > > >> >>>> > On Mon, Mar 2, 2015 at 2:27 PM, Adunuthula, Seshu > >> >>>><[email protected] > >> >>>> > > >> >>>> > wrote: > >> >>>> > > >> >>>> > > Jakob, > >> >>>> > > > >> >>>> > > Could you please send out the complete stack trace with Tez > >> >>>>enabled? > >> >>>> > > > >> >>>> > > Regards > >> >>>> > > Seshu > >> >>>> > > > >> >>>> > > On 3/2/15, 4:51 AM, "Jakob Stengård" <[email protected] > > > >> >>>> wrote: > >> >>>> > > > >> >>>> > > >Hi > >> >>>> > > >Will kylin suport hive+tez in v0.7.x? > >> >>>> > > > > >> >>>> > > >I've tried using tez with hive, but currently the cube build > >> >>>>fails > >> >>>> with > >> >>>> > > >"Can't get cube source record size", because kylin expects > >> >>>>certain > >> >>>> > console > >> >>>> > > >output from map-reduce, and tez doesn't print this. > >> >>>> > > > > >> >>>> > > >BG. > >> >>>> > > >Jakob Stengard > >> >>>> > > > >> >>>> > > > >> >>>> > > >> >>>> > >> >>> > >> >>> > >> > > >> > >> > >
