[
https://issues.apache.org/jira/browse/HIVE-12812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669184#comment-16669184
]
Wang Haihua commented on HIVE-12812:
------------------------------------
Hi [~ychena] , what i meaned is this solution merged doesnot fix mapreduce
application reading data which was generated by tez(with union all semantics)..
you could see the scene in my older comment
{color:#cccccc}_We use hive 1.2.1, just in fact, mapreduce is though old but
always be the most stable engine for Hive. We use hive heavily in production(
expand to 5K nodes+), which consist 60% tez, 35% mapreduce(for corner case tez
cannot handled), and 5% SparkSQL(the future)._ {color}
{color:#cccccc}_So the first priority is stable(include data quality and
consistency, SLA and so on), we put much patch on our version of Hive._{color}
{color:#cccccc}_As tez and mapreduce is mixed, we encountered this issue. And i
suppose this is a common issue on production , for reason that HiveOnSpark,
HiveOnTez is not stable enough._{color}
> Enable mapred.input.dir.recursive by default to support union with aggregate
> function
> -------------------------------------------------------------------------------------
>
> Key: HIVE-12812
> URL: https://issues.apache.org/jira/browse/HIVE-12812
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Affects Versions: 1.2.1, 2.1.0
> Reporter: Chaoyu Tang
> Assignee: Alice Fan
> Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-12812.1.patch, HIVE-12812.2.patch,
> HIVE-12812.patch, HIVE-12812.patch, HIVE-12812.patch
>
>
> When union remove optimization is enabled, union query with aggregate
> function writes its subquery intermediate results to subdirs which needs
> mapred.input.dir.recursive to be enabled in order to be fetched. This
> property is not defined by default in Hive and often ignored by user, which
> causes the query failure and is hard to be debugged.
> So we need set mapred.input.dir.recursive to true whenever union remove
> optimization is enabled.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
