GitHub user davideanastasia opened a pull request:
https://github.com/apache/oozie/pull/13
Add support for s3/s3n schemes
The idea is to create a new URIHandler, that uses AWS S3 SDK to check
existence of data in S3. Credentials are taken from the IAM Role of the EC2
instance (hence no need of configuration), but I am planning of extending to
support custom AWS credential in the oozie-core.xml file.
This patch is particularly useful with Amazon Elastic MapReduce (EMR),
because no credentials are stored in the core-site.xml in that case (as EMR
will use the custom EmrFileSystem to access S3).
I am a bit unsure about the need of the S3LauncherURIHandler, so I made the
methods throw and UnsupportedOperationException, but I would be happy to fill
the gap here as well.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/davideanastasia/oozie OOZIE-426
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/oozie/pull/13.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13
----
commit 622dcb96505931cfff8c5eabd7b50adeebbbc853
Author: Davide Anastasia <[email protected]>
Date: 2015-07-08T10:45:28Z
Add S3URIHandler to check job dependencies
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
