kangkaisen created KYLIN-2242: --------------------------------- Summary: Directly write hdfs file in reducer is dangerous Key: KYLIN-2242 URL: https://issues.apache.org/jira/browse/KYLIN-2242 Project: Kylin Issue Type: Bug Components: Job Engine Affects Versions: v1.6.0 Reporter: kangkaisen Assignee: Dong Li
currently, Kylin directly write hdfs file in {{FactDistinctColumnsReducer}}, which is dangerous because the MapReduce Speculative Execution will result in more than one reducers write the same hdfs file at the same time. After KYLIN-2217, I think this issue will occur with higher probability. we should output the value by {{context.wirte}} in reducer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)