Re: error execute MapReduceExecutable{id=, name=Extract Fact Table Distinct Columns, state=RUNNING
This is the root cause. The metadata was inconsistent at that point.
> Failed to find 2a3e69cf-7ff1-4c7a-97b2-8909dac4fe51 in cube
CUBE[name=Remain_Cube3
On Wed, Aug 23, 2017 at 11:29 AM, yuyong.zhai <yuyong.z...@ele.me> wrote:
> discard the cube and rebuild success
>
>
>
> 原始邮件
> *发件人:* yuyong.zhai<yuyong.z...@ele.me>
> *收件人:* user@kylin.apache.org<user@kylin.apache.org>
> *发送时间:* 2017年8月23日(周三) 11:26
> *主题:* error execute MapReduceExecutable{id=, name=Extract Fact Table
> Distinct Columns, state=RUNNING
>
> 2017-08-23 11:09:21,414 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
> steps.FactDistinctColumnsJob:106 : Starting: Kylin_Fact_Distinct_Columns_
> Remain_Cube3_Step
>
> 2017-08-23 11:09:21,414 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
> common.AbstractHadoopJob:163 : append job jar:
> xxx/kylin/lib/kylin-job-2.0.0.jar
>
> 2017-08-23 11:09:21,414 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
> common.AbstractHadoopJob:171 : append kylin.hbase.dependency:
> xxx/hbase/lib/hbase-common-0.98.12.1-hadoop2.jar to mapreduce.application.
> classpath
>
> 2017-08-23 11:09:21,414 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
> common.AbstractHadoopJob:176 : Didn't find mapreduce.application.classpath
> in job configuration, will run 'mapred classpath' to get the default value.
>
> 2017-08-23 11:09:21,414 INFO [Job fdeea049-dab0-4f71-96b7-249e4ac03915-65]
> execution.AbstractExecutable:110 : Executing AbstractExecutable (Load
> HFile to HBase Table)
>
> 2017-08-23 11:09:21,415 DEBUG [Job fdeea049-dab0-4f71-96b7-249e4ac03915-65]
> dao.ExecutableDao:217 : updating job output, id: fdeea049-dab0-4f71-96b7-
> 249e4ac03915-17
>
> 2017-08-23 11:09:21,416 INFO [Job fdeea049-dab0-4f71-96b7-249e4ac03915-65]
> execution.ExecutableManager:389 : job
> id:fdeea049-dab0-4f71-96b7-249e4ac03915-17
> from READY to RUNNING
>
> 2017-08-23 11:09:21,432 INFO [Job fdeea049-dab0-4f71-96b7-249e4ac03915-65]
> common.HadoopShellExecutable:58 : parameters of the HadoopShellExecutable:
>
> 2017-08-23 11:09:21,432 INFO [Job fdeea049-dab0-4f71-96b7-249e4ac03915-65]
> common.HadoopShellExecutable:59 : -input xx/kylin_metadata/kylin-
> fdeea049-dab0-4f71-96b7-249e4ac03915/Funnel_Result/hfile -htablename
> KYLIN_2P0N6JLP28 -cubename Funnel_Result
>
> 2017-08-23 11:09:21,623 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
> common.AbstractHadoopJob:202 : Hive Dependencies After
> Filtered:xxx/hive/lib/hive-exec-2.1.1.jar,xxx/hive/lib/
> hive-metastore-2.1.1.jar,xxx/hive/hcatalog/share/hcatalog/
> hive-hcatalog-core-2.1.1.jar
>
> 2017-08-23 11:09:21,623 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
> common.AbstractHadoopJob:230 : Kafka Dependencies:
>
> 2017-08-23 11:09:21,625 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
> common.AbstractHadoopJob:358 : Job 'tmpjars' updated -- file:
>
> 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
> steps.FactDistinctColumnsJob:112 : Failed to find
> 2a3e69cf-7ff1-4c7a-97b2-8909dac4fe51
> in cube CUBE[name=Remain_Cube3]
>
> 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
> steps.FactDistinctColumnsJob:114 : 2017052300_2017060100 with
> status READY
>
> 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
> steps.FactDistinctColumnsJob:114 : 2017060100_2017062700 with
> status READY
>
> 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
> steps.FactDistinctColumnsJob:114 : 2017062700_2017062800 with
> status READY
>
> 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
> steps.FactDistinctColumnsJob:114 : 2017062700_2017062800 with
> status NEW
>
> 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
> steps.FactDistinctColumnsJob:114 : 2017062800_2017062900 with
> status READY
>
> 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
> steps.FactDistinctColumnsJob:114 : 2017062900_2017063000 with
> status READY
>
> 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
> steps.FactDistinctColumnsJob:114 : 2017063000_2017070100 with
> status READY
>
> 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
> steps.FactDistinctColumnsJob:114 : 2017070100_2017070200 with
> status READY
>
> 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
> steps.FactDistinctColumnsJob:114 : 2017070200_2017070300 with
> status READY
>
> 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
&g
回复：error execute MapReduceExecutable{id=, name=Extract Fact Table Distinct Columns, state=RUNNING
discard the cube and rebuild success
原始邮件
发件人: yuyong.zhai<yuyong.z...@ele.me>
收件人: user@kylin.apache.org<user@kylin.apache.org>
发送时间: 2017年8月23日(周三) 11:26
主题: error execute MapReduceExecutable{id=, name=Extract Fact Table Distinct
Columns, state=RUNNING
2017-08-23 11:09:21,414 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
steps.FactDistinctColumnsJob:106 : Starting:
Kylin_Fact_Distinct_Columns_Remain_Cube3_Step
2017-08-23 11:09:21,414 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
common.AbstractHadoopJob:163 : append job jar: xxx/kylin/lib/kylin-job-2.0.0.jar
2017-08-23 11:09:21,414 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
common.AbstractHadoopJob:171 : append kylin.hbase.dependency:
xxx/hbase/lib/hbase-common-0.98.12.1-hadoop2.jar to
mapreduce.application.classpath
2017-08-23 11:09:21,414 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
common.AbstractHadoopJob:176 : Didn't find mapreduce.application.classpath in
job configuration, will run 'mapred classpath' to get the default value.
2017-08-23 11:09:21,414 INFO [Job fdeea049-dab0-4f71-96b7-249e4ac03915-65]
execution.AbstractExecutable:110 : Executing AbstractExecutable (Load HFile to
HBase Table)
2017-08-23 11:09:21,415 DEBUG [Job fdeea049-dab0-4f71-96b7-249e4ac03915-65]
dao.ExecutableDao:217 : updating job output, id:
fdeea049-dab0-4f71-96b7-249e4ac03915-17
2017-08-23 11:09:21,416 INFO [Job fdeea049-dab0-4f71-96b7-249e4ac03915-65]
execution.ExecutableManager:389 : job
id:fdeea049-dab0-4f71-96b7-249e4ac03915-17 from READY to RUNNING
2017-08-23 11:09:21,432 INFO [Job fdeea049-dab0-4f71-96b7-249e4ac03915-65]
common.HadoopShellExecutable:58 : parameters of the HadoopShellExecutable:
2017-08-23 11:09:21,432 INFO [Job fdeea049-dab0-4f71-96b7-249e4ac03915-65]
common.HadoopShellExecutable:59 : -input
xx/kylin_metadata/kylin-fdeea049-dab0-4f71-96b7-249e4ac03915/Funnel_Result/hfile
-htablename KYLIN_2P0N6JLP28 -cubename Funnel_Result
2017-08-23 11:09:21,623 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
common.AbstractHadoopJob:202 : Hive Dependencies After
Filtered:xxx/hive/lib/hive-exec-2.1.1.jar,xxx/hive/lib/hive-metastore-2.1.1.jar,xxx/hive/hcatalog/share/hcatalog/hive-hcatalog-core-2.1.1.jar
2017-08-23 11:09:21,623 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
common.AbstractHadoopJob:230 : Kafka Dependencies:
2017-08-23 11:09:21,625 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
common.AbstractHadoopJob:358 : Job 'tmpjars' updated -- file:
2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
steps.FactDistinctColumnsJob:112 : Failed to find
2a3e69cf-7ff1-4c7a-97b2-8909dac4fe51 in cube CUBE[name=Remain_Cube3]
2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
steps.FactDistinctColumnsJob:114 : 2017052300_2017060100 with status
READY
2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
steps.FactDistinctColumnsJob:114 : 2017060100_2017062700 with status
READY
2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
steps.FactDistinctColumnsJob:114 : 2017062700_2017062800 with status
READY
2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
steps.FactDistinctColumnsJob:114 : 2017062700_2017062800 with status NEW
2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
steps.FactDistinctColumnsJob:114 : 2017062800_2017062900 with status
READY
2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
steps.FactDistinctColumnsJob:114 : 2017062900_2017063000 with status
READY
2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
steps.FactDistinctColumnsJob:114 : 2017063000_2017070100 with status
READY
2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
steps.FactDistinctColumnsJob:114 : 2017070100_2017070200 with status
READY
2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
steps.FactDistinctColumnsJob:114 : 2017070200_2017070300 with status
READY
2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
steps.FactDistinctColumnsJob:114 : 2017070300_2017070400 with status
READY
2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
steps.FactDistinctColumnsJob:114 : 2017070400_2017070500 with status
READY
2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
steps.FactDistinctColumnsJob:114 : 2017070500_2017070600 with status
READY
2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
steps.FactDistinctColumnsJob:114 : 2017070600_2017070700 with status
READY
2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
steps.FactDistinctColumnsJob:114 : 2017070700_2017070800 with s
error execute MapReduceExecutable{id=, name=Extract Fact Table Distinct Columns, state=RUNNING
2017-08-23 11:09:21,414 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] steps.FactDistinctColumnsJob:106 : Starting: Kylin_Fact_Distinct_Columns_Remain_Cube3_Step 2017-08-23 11:09:21,414 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] common.AbstractHadoopJob:163 : append job jar: xxx/kylin/lib/kylin-job-2.0.0.jar 2017-08-23 11:09:21,414 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] common.AbstractHadoopJob:171 : append kylin.hbase.dependency: xxx/hbase/lib/hbase-common-0.98.12.1-hadoop2.jar to mapreduce.application.classpath 2017-08-23 11:09:21,414 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] common.AbstractHadoopJob:176 : Didn't find mapreduce.application.classpath in job configuration, will run 'mapred classpath' to get the default value. 2017-08-23 11:09:21,414 INFO [Job fdeea049-dab0-4f71-96b7-249e4ac03915-65] execution.AbstractExecutable:110 : Executing AbstractExecutable (Load HFile to HBase Table) 2017-08-23 11:09:21,415 DEBUG [Job fdeea049-dab0-4f71-96b7-249e4ac03915-65] dao.ExecutableDao:217 : updating job output, id: fdeea049-dab0-4f71-96b7-249e4ac03915-17 2017-08-23 11:09:21,416 INFO [Job fdeea049-dab0-4f71-96b7-249e4ac03915-65] execution.ExecutableManager:389 : job id:fdeea049-dab0-4f71-96b7-249e4ac03915-17 from READY to RUNNING 2017-08-23 11:09:21,432 INFO [Job fdeea049-dab0-4f71-96b7-249e4ac03915-65] common.HadoopShellExecutable:58 : parameters of the HadoopShellExecutable: 2017-08-23 11:09:21,432 INFO [Job fdeea049-dab0-4f71-96b7-249e4ac03915-65] common.HadoopShellExecutable:59 : -input xx/kylin_metadata/kylin-fdeea049-dab0-4f71-96b7-249e4ac03915/Funnel_Result/hfile -htablename KYLIN_2P0N6JLP28 -cubename Funnel_Result 2017-08-23 11:09:21,623 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] common.AbstractHadoopJob:202 : Hive Dependencies After Filtered:xxx/hive/lib/hive-exec-2.1.1.jar,xxx/hive/lib/hive-metastore-2.1.1.jar,xxx/hive/hcatalog/share/hcatalog/hive-hcatalog-core-2.1.1.jar 2017-08-23 11:09:21,623 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] common.AbstractHadoopJob:230 : Kafka Dependencies: 2017-08-23 11:09:21,625 INFO [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] common.AbstractHadoopJob:358 : Job 'tmpjars' updated -- file: 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] steps.FactDistinctColumnsJob:112 : Failed to find 2a3e69cf-7ff1-4c7a-97b2-8909dac4fe51 in cube CUBE[name=Remain_Cube3] 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] steps.FactDistinctColumnsJob:114 : 2017052300_2017060100 with status READY 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] steps.FactDistinctColumnsJob:114 : 2017060100_2017062700 with status READY 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] steps.FactDistinctColumnsJob:114 : 2017062700_2017062800 with status READY 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] steps.FactDistinctColumnsJob:114 : 2017062700_2017062800 with status NEW 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] steps.FactDistinctColumnsJob:114 : 2017062800_2017062900 with status READY 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] steps.FactDistinctColumnsJob:114 : 2017062900_2017063000 with status READY 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] steps.FactDistinctColumnsJob:114 : 2017063000_2017070100 with status READY 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] steps.FactDistinctColumnsJob:114 : 2017070100_2017070200 with status READY 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] steps.FactDistinctColumnsJob:114 : 2017070200_2017070300 with status READY 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] steps.FactDistinctColumnsJob:114 : 2017070300_2017070400 with status READY 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] steps.FactDistinctColumnsJob:114 : 2017070400_2017070500 with status READY 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] steps.FactDistinctColumnsJob:114 : 2017070500_2017070600 with status READY 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] steps.FactDistinctColumnsJob:114 : 2017070600_2017070700 with status READY 2017-08-23 11:09:21,625 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] steps.FactDistinctColumnsJob:114 : 2017070700_2017070800 with status READY 2017-08-23 11:09:21,626 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71] steps.FactDistinctColumnsJob:114 : 2017070800_2017070900 with status READY 2017-08-23 11:09:21,626 ERROR [Job ab916702-dfe1-4bfa-b976-b1c4c5e8286f-71]
