[
https://issues.apache.org/jira/browse/MAPREDUCE-6988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16312299#comment-16312299
]
Johan Gustavsson commented on MAPREDUCE-6988:
---------------------------------------------
[[email protected]] as I saw you were already listed as watching
this, do you mind taking a look?
> Let JHS support different file systems for intermediate_done and done
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-6988
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6988
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver
> Affects Versions: 2.7.4
> Reporter: Johan Gustavsson
> Priority: Minor
> Attachments: MAPREDUCE-6988.000.patch, MAPREDUCE-6988.001.patch,
> MAPREDUCE-6988.002.patch, MAPREDUCE-6988.003.patch
>
>
> Currently JHS uses filecontext to move files from intermediate_done to done
> folder. Since filecontext limits the use to 1 filesystem it makes it harder
> to use s3 as a storage for jhist files. By moving this to filesystem
> interface we can set hdfs for intermediate storage and s3 as long term
> storage therefore reducing the number of puts to s3 and removing the need for
> all M/R containers to carry a s3 sdk.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
