[jira] [Commented] (HAMA-984) Support AWS S3 schema in Hadoop 2.6+

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAMA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205573#comment-15205573
 ] 

ASF GitHub Bot commented on HAMA-984:
-

Github user asfgit closed the pull request at:

https://github.com/apache/hama/pull/9


> Support AWS S3 schema in Hadoop 2.6+
> 
>
> Key: HAMA-984
> URL: https://issues.apache.org/jira/browse/HAMA-984
> Project: Hama
>  Issue Type: Improvement
>  Components: build 
>Reporter: Cazen Lee
>Assignee: Cazen Lee
>
> Hadoop 2.6+ does not contain AWS S3 related filesystem by default.
> So, IOException(No FileSystem for scheme) occurred while trying to access S3 
> via s3 or s3n schema.
> I know it's not a Hama bug but it will be helpful to Hama users who using AWS 
> S3 because it can be used by previous version(includes 1.x) without manual 
> setting. Of course, we can also guide through the changes, without 
> modification any source code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAMA-984) Support AWS S3 schema in Hadoop 2.6+

[Cazen Lee (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAMA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198566#comment-15198566
 ] 

Cazen Lee commented on HAMA-984:


I've successfully tested on my AWS cluster and mac with this patch. Please 
review and commit it

> Support AWS S3 schema in Hadoop 2.6+
> 
>
> Key: HAMA-984
> URL: https://issues.apache.org/jira/browse/HAMA-984
> Project: Hama
>  Issue Type: Improvement
>  Components: build 
>Reporter: Cazen Lee
>Assignee: Cazen Lee
>
> Hadoop 2.6+ does not contain AWS S3 related filesystem by default.
> So, IOException(No FileSystem for scheme) occurred while trying to access S3 
> via s3 or s3n schema.
> I know it's not a Hama bug but it will be helpful to Hama users who using AWS 
> S3 because it can be used by previous version(includes 1.x) without manual 
> setting. Of course, we can also guide through the changes, without 
> modification any source code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAMA-984) Support AWS S3 schema in Hadoop 2.6+

[Cazen Lee (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAMA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182170#comment-15182170
 ] 

Cazen Lee commented on HAMA-984:


I updated my PR and it works on my local(mac+hdp)

Could you(or any commiters?) please check it?

> Support AWS S3 schema in Hadoop 2.6+
> 
>
> Key: HAMA-984
> URL: https://issues.apache.org/jira/browse/HAMA-984
> Project: Hama
>  Issue Type: Improvement
>  Components: build 
>Reporter: Cazen Lee
>
> Hadoop 2.6+ does not contain AWS S3 related filesystem by default.
> So, IOException(No FileSystem for scheme) occurred while trying to access S3 
> via s3 or s3n schema.
> I know it's not a Hama bug but it will be helpful to Hama users who using AWS 
> S3 because it can be used by previous version(includes 1.x) without manual 
> setting. Of course, we can also guide through the changes, without 
> modification any source code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAMA-984) Support AWS S3 schema in Hadoop 2.6+

[Edward J. Yoon (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAMA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177081#comment-15177081
 ] 

Edward J. Yoon commented on HAMA-984:
-

Hi, the hadoop-auth package is used for security/auth in o.a.h.ipc package and 
YARN module.

> Support AWS S3 schema in Hadoop 2.6+
> 
>
> Key: HAMA-984
> URL: https://issues.apache.org/jira/browse/HAMA-984
> Project: Hama
>  Issue Type: Improvement
>  Components: build 
>Reporter: Cazen Lee
>
> Hadoop 2.6+ does not contain AWS S3 related filesystem by default.
> So, IOException(No FileSystem for scheme) occurred while trying to access S3 
> via s3 or s3n schema.
> I know it's not a Hama bug but it will be helpful to Hama users who using AWS 
> S3 because it can be used by previous version(includes 1.x) without manual 
> setting. Of course, we can also guide through the changes, without 
> modification any source code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAMA-984) Support AWS S3 schema in Hadoop 2.6+

[Cazen Lee (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAMA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175617#comment-15175617
 ] 

Cazen Lee commented on HAMA-984:


Could I ask where is the hadoop-auth dependency in pom.xml used?
And can I remove it if there are no external usage?
(hadoop-common have hadoop-auth dependency already)

> Support AWS S3 schema in Hadoop 2.6+
> 
>
> Key: HAMA-984
> URL: https://issues.apache.org/jira/browse/HAMA-984
> Project: Hama
>  Issue Type: Improvement
>  Components: build 
>Reporter: Cazen Lee
>
> Hadoop 2.6+ does not contain AWS S3 related filesystem by default.
> So, IOException(No FileSystem for scheme) occurred while trying to access S3 
> via s3 or s3n schema.
> I know it's not a Hama bug but it will be helpful to Hama users who using AWS 
> S3 because it can be used by previous version(includes 1.x) without manual 
> setting. Of course, we can also guide through the changes, without 
> modification any source code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAMA-984) Support AWS S3 schema in Hadoop 2.6+

[Cazen Lee (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAMA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15170868#comment-15170868
 ] 

Cazen Lee commented on HAMA-984:


Sorry for inconvenience.
I've noticed that there are something wrong in my PR with S3 connection. 
I'll check it soon and please wait until fix it.

> Support AWS S3 schema in Hadoop 2.6+
> 
>
> Key: HAMA-984
> URL: https://issues.apache.org/jira/browse/HAMA-984
> Project: Hama
>  Issue Type: Improvement
>  Components: build 
>Reporter: Cazen Lee
>
> Hadoop 2.6+ does not contain AWS S3 related filesystem by default.
> So, IOException(No FileSystem for scheme) occurred while trying to access S3 
> via s3 or s3n schema.
> I know it's not a Hama bug but it will be helpful to Hama users who using AWS 
> S3 because it can be used by previous version(includes 1.x) without manual 
> setting. Of course, we can also guide through the changes, without 
> modification any source code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAMA-984) Support AWS S3 schema in Hadoop 2.6+

[Edward J. Yoon (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAMA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15170592#comment-15170592
 ] 

Edward J. Yoon commented on HAMA-984:
-

Thanks for pull request. I can check next week.

> Support AWS S3 schema in Hadoop 2.6+
> 
>
> Key: HAMA-984
> URL: https://issues.apache.org/jira/browse/HAMA-984
> Project: Hama
>  Issue Type: Improvement
>  Components: build 
>Reporter: Cazen Lee
>
> Hadoop 2.6+ does not contain AWS S3 related filesystem by default.
> So, IOException(No FileSystem for scheme) occurred while trying to access S3 
> via s3 or s3n schema.
> I know it's not a Hama bug but it will be helpful to Hama users who using AWS 
> S3 because it can be used by previous version(includes 1.x) without manual 
> setting. Of course, we can also guide through the changes, without 
> modification any source code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAMA-984) Support AWS S3 schema in Hadoop 2.6+

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAMA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15167215#comment-15167215
 ] 

ASF GitHub Bot commented on HAMA-984:
-

GitHub user Cazen opened a pull request:

https://github.com/apache/hama/pull/9

[HAMA-984] Support AWS S3 schema in Hadoop 2.6+



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Cazen/hama HAMA-984

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hama/pull/9.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #9


commit 74b5214848b342a53d7d996c525b16e969850d91
Author: Cazen Lee 
Date:   2016-02-25T13:52:17Z

Add hadoop-aws dependency




> Support AWS S3 schema in Hadoop 2.6+
> 
>
> Key: HAMA-984
> URL: https://issues.apache.org/jira/browse/HAMA-984
> Project: Hama
>  Issue Type: Improvement
>  Components: build 
>Reporter: Cazen Lee
>
> Hadoop 2.6+ does not contain AWS S3 related filesystem by default.
> So, IOException(No FileSystem for scheme) occurred while trying to access S3 
> via s3 or s3n schema.
> I know it's not a Hama bug but it will be helpful to Hama users who using AWS 
> S3 because it can be used by previous version(includes 1.x) without manual 
> setting. Of course, we can also guide through the changes, without 
> modification any source code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)