[ https://issues.apache.org/jira/browse/KYLIN-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15833030#comment-15833030 ]
kangkaisen commented on KYLIN-2242: ----------------------------------- commit: 7de8aa1203a72 > Directly write hdfs file in reducer is dangerous > ------------------------------------------------ > > Key: KYLIN-2242 > URL: https://issues.apache.org/jira/browse/KYLIN-2242 > Project: Kylin > Issue Type: Bug > Components: Job Engine > Affects Versions: v1.6.0 > Reporter: kangkaisen > Assignee: kangkaisen > Fix For: v2.0.0 > > Attachments: KYLIN-2242.patch > > > currently, Kylin directly write hdfs file in {{FactDistinctColumnsReducer}}, > which is dangerous because the MapReduce Speculative Execution will result in > more than one reducers write the same hdfs file at the same time. > After KYLIN-2217, I think this issue will occur with higher probability. we > should output the value by {{context.wirte}} in reducer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)