[
https://issues.apache.org/jira/browse/KYLIN-3084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16282609#comment-16282609
]
Vsevolod Ostapenko commented on KYLIN-3084:
-------------------------------------------
It's a tez "feature". In order to instruct tez to coalesce the results from
multiple parallel writers and prevent it from writing into table storage
subfolders set hive.merge.tezfiles to true.
> File not found Exception when processing union-all in TEZ mode
> --------------------------------------------------------------
>
> Key: KYLIN-3084
> URL: https://issues.apache.org/jira/browse/KYLIN-3084
> Project: Kylin
> Issue Type: Bug
> Reporter: Wang Cheng
> Assignee: Wang Cheng
> Priority: Minor
>
> If hive.execution.engine=TEZ and hql contains union all, it causes exception
> like: file not found when materializing the view or redistributing flat hive
> table.
> Here is the reason:
> http://grokbase.com/t/hive/user/162r80a2g9/anyway-to-avoid-creating-subdirectories-by-insert-with-union
> i.e. "The Tez execution of UNION is entirely parallel &
> the task-ids overlaps - so the files created have to have unique names.
> But the total counts for "Map 1" and "Map 2" are only available as the job
> runs, so they write to different dirs."
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
