[
https://issues.apache.org/jira/browse/OOZIE-426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13900590#comment-13900590
]
jay vyas commented on OOZIE-426:
--------------------------------
So, to summarize.
1) A while back, roman noted that s3 was unsupported.
2) Now oozie natively supports s3, and other filesystems can be supported by
simply adding them to the classpath.
3) So.. how do you add them to the classpath in a universal way? I thought
some more about a simple, universal way to deploy any HCFS in oozie and i think
that,
- All WAR files have WEB-INF/lib
- No matter what, the files in WEB-INF/lib will be available to the Oozie
Servlets... thus...
{noformat}
At this point I think in any environment, you should be able to simply copy
your HCFS file system directly into WEB-INF/lib, and that would be a universal
way to add any HCFS implementation to oozie.
{noformat}
Thoughts on this as a universal deployment strategy for any HCFS.... Is it
viable ? Sure would make things easier for the oozie community if "yes" (that
we we dont have to do funny stuff like redeploy the war file at runtime).
> OOZIE-9: Support (Amazon) s3/s3n schemes
> ----------------------------------------
>
> Key: OOZIE-426
> URL: https://issues.apache.org/jira/browse/OOZIE-426
> Project: Oozie
> Issue Type: New Feature
> Reporter: Hadoop QA
> Assignee: Roman Shaposhnik
>
> Currently Oozie supports only the 'hdfs' scheme. For people having their
> hadoop cluster in Amazon EC2 and using S3 for storage, it would be very
> useful if oozie supports the 's3n/s3' scheme(s). The use case I am talking
> about is as follows
> Hadoop cluster in Amazon EC2
> Uses hdfs for intermediate storage
> Uses s3 for getting input for, storing output of map-reduce jobs.
> More details on the above use-case and the exceptions/failures I have seen is
> documented here (http://tech.groups.yahoo.com/group/Oozie-users/message/1138).
> There can be other use cases as well - say use s3 as the DFS instead of HDFS.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
