[ https://issues.apache.org/jira/browse/KYLIN-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15713779#comment-15713779 ]
kangkaisen commented on KYLIN-2242: ----------------------------------- OK, I am willing to do it, but I maybe delay this work. > Directly write hdfs file in reducer is dangerous > ------------------------------------------------ > > Key: KYLIN-2242 > URL: https://issues.apache.org/jira/browse/KYLIN-2242 > Project: Kylin > Issue Type: Bug > Components: Job Engine > Affects Versions: v1.6.0 > Reporter: kangkaisen > Assignee: kangkaisen > > currently, Kylin directly write hdfs file in {{FactDistinctColumnsReducer}}, > which is dangerous because the MapReduce Speculative Execution will result in > more than one reducers write the same hdfs file at the same time. > After KYLIN-2217, I think this issue will occur with higher probability. we > should output the value by {{context.wirte}} in reducer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)