[
https://issues.apache.org/jira/browse/MAPREDUCE-7060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated MAPREDUCE-7060:
----------------------------------
Description:
It's easier for downstream apps like Spark to pick up the new
PathOutputCommitter superclass if it is there on 2.10+, even if the S3A
committer isn't there.
Adding the interface & binding stuff of MAPREDUCE-6956 allows for third party
committers to be deployed.
I'm not proposing a backport of the HADOOP-13786 committer: that's Java 8,
S3Guard, etc. Too traumatic. All I want here is to allow downstream code to be
able to pick up the new interface and so be able to support it and other store
committers when available
was:
It's easier for downstream apps like Spark to pick up the new
PathOutputCommitter superclass if it is there on 2.10+, even if the S3A
committer isn't there.
Adding the interface & binding stuff of HADOOP-6956 allows for third party
committers to be deployed.
I'm not proposing a backport of the HADOOP-13786 committer: that's Java 8,
S3Guard, etc. Too traumatic. All I want here is to allow downstream code to be
able to pick up the new interface and so be able to support it and other store
committers when available
+1 lgtm.
> Cherry Pick PathOutputCommitter class/factory to branch-3.0 & 2.10
> ------------------------------------------------------------------
>
> Key: MAPREDUCE-7060
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7060
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 3.0.0, 2.10.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Minor
> Attachments: MAPREDUCE-7060-branch-3.0-001.patch,
> MAPREDUCE-7060-branch-3.0-002.patch
>
>
> It's easier for downstream apps like Spark to pick up the new
> PathOutputCommitter superclass if it is there on 2.10+, even if the S3A
> committer isn't there.
> Adding the interface & binding stuff of MAPREDUCE-6956 allows for third party
> committers to be deployed.
> I'm not proposing a backport of the HADOOP-13786 committer: that's Java 8,
> S3Guard, etc. Too traumatic. All I want here is to allow downstream code to
> be able to pick up the new interface and so be able to support it and other
> store committers when available
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
