Thanks Jakob, Would be fantastic if you could dig in and submit a patch…
On 3/2/15, 10:08 AM, "Jakob Stengård" <[email protected]> wrote: >I'll file a bug in jira. >For your information i also happen to know that tez boosted map-reduce >doesn't work either, but there, the problem seems to have to do with the >map reduce jobs them selves failing. > >In this case however, from what i've read on git hub about the "Can't get >cube source record size", >error", it could possibly be a problem with that tez doesn't output a >string on the form: >"HDFS Read: (\d+) HDFS Write" > >However, it could be something else, it could also be my logging >configuration for tez. > >/J > > > >On Mon, Mar 2, 2015 at 6:54 PM, Adunuthula, Seshu <[email protected]> >wrote: > >> Did not read your logs fully. The culprit seems to be >> >> >> 1. [QuartzScheduler_Worker-7]:[2015-03-02 >> >>16:09:24,737][ERROR][com.kylinolap.job.flow.JobFlowListener.handleExcepti >>on >> (JobFlowListener.java:312)] - Can't get cube source record size. >> >> source_records_size may have to be computed differently when using Tez >>as >> the Hive engine. I will let Yang/Luke answer this when they come back >> onlineŠ >> >> >> >> >> On 3/2/15, 9:30 AM, "Adunuthula, Seshu" <[email protected]> wrote: >> >> >Jakob, >> > >> >Could you go ahead and file a JIRA with the stack trace at >> >https://issues.apache.org/jira/browse/KYLIN/ >> > >> >Also include the App Master and Map/Reduce Task logs for the Tasks that >> >failed. >> > >> >Regards >> >Seshu >> > >> >On 3/2/15, 8:12 AM, "Jakob Stengård" <[email protected]> wrote: >> > >> >>Correction: That, isn't the last job, it's step 9. >> >> >> >>/J >> >> >> >>On Mon, Mar 2, 2015 at 5:10 PM, Jakob Stengård >><[email protected] >> > >> >>wrote: >> >> >> >>> Well, this only happens with >> >>> <property> >> >>> <name>hive.execution.engine</name> >> >>> <value>tez</value> >> >>> </property> >> >>> >> >>> If i use mr, it works fine. >> >>> >> >>> >> >>> From what i can see form the job log, none of the previous >>map-reduce >> >>>jobs >> >>> fail, but the build fails at the last step. >> >>> Log from the last mr job for example: >> >>> http://pastebin.com/jNWSRKLi >> >>> >> >>> >> >>> Is tez supposed to work? Because in that case, maybe it's just my >> >>>setup. >> >>> >> >>> /Jakob >> >>> >> >>> >> >>> >> >>> On Mon, Mar 2, 2015 at 4:34 PM, Santosh Akhilesh < >> >>> [email protected]> wrote: >> >>> >> >>>> This doesn't seems to be complete kylin.log. Do you have any >>exception >> >>>> related to MR counter in map reduce logs? If yes then this needs >>to be >> >>>> checked because kylin will need the MR counter to calculate the >>cube >> >>>>size. >> >>>> On Mon, 2 Mar 2015 at 8:49 pm, Jakob Stengård >> >>>><[email protected]> >> >>>> wrote: >> >>>> >> >>>> > It breaks on the last step: "Load HFile to HBase Table": >> >>>> > >> >>>> > Kylin.log: >> >>>> > http://pastebin.com/3kAQmJ3u >> >>>> > >> >>>> > On Mon, Mar 2, 2015 at 2:27 PM, Adunuthula, Seshu >> >>>><[email protected] >> >>>> > >> >>>> > wrote: >> >>>> > >> >>>> > > Jakob, >> >>>> > > >> >>>> > > Could you please send out the complete stack trace with Tez >> >>>>enabled? >> >>>> > > >> >>>> > > Regards >> >>>> > > Seshu >> >>>> > > >> >>>> > > On 3/2/15, 4:51 AM, "Jakob Stengård" <[email protected]> >> >>>> wrote: >> >>>> > > >> >>>> > > >Hi >> >>>> > > >Will kylin suport hive+tez in v0.7.x? >> >>>> > > > >> >>>> > > >I've tried using tez with hive, but currently the cube build >> >>>>fails >> >>>> with >> >>>> > > >"Can't get cube source record size", because kylin expects >> >>>>certain >> >>>> > console >> >>>> > > >output from map-reduce, and tez doesn't print this. >> >>>> > > > >> >>>> > > >BG. >> >>>> > > >Jakob Stengard >> >>>> > > >> >>>> > > >> >>>> > >> >>>> >> >>> >> >>> >> > >> >>
