[
https://issues.apache.org/jira/browse/HIVE-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819259#comment-16819259
]
Kshitiz Sethia commented on HIVE-3700:
--------------------------------------
Voting up this feature, it's needed for oozie workflows when waiting for hive
partitions as input. Querying hive catalog returns early, and when not all data
for partition is ready.
> Hive ignores 'mapreduce.fileoutputcommitter.marksuccessfuljobs'
> ---------------------------------------------------------------
>
> Key: HIVE-3700
> URL: https://issues.apache.org/jira/browse/HIVE-3700
> Project: Hive
> Issue Type: Bug
> Components: Shims
> Affects Versions: 0.9.0
> Reporter: Joe Crobak
> Priority: Minor
>
> For downstream computation, it can be useful to have a {{_SUCCESS}} flag
> dropped in the directory to signify that the data is completely written. This
> is typically enabled with the property
> {{mapreduce.fileoutputcommitter.marksuccessfuljobs}}, but hive ignores this.
> I believe it's because the shims layer
> {{src/common-secure/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java}}
> and {{shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java}}
> both supply a NullOutputCommitter that don't respect this property.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
