[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214576#comment-15214576
]
Steve Loughran commented on SPARK-7481:
---
I've created a pull request on this, which has
# a new
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214574#comment-15214574
]
Apache Spark commented on SPARK-7481:
-
User 'steveloughran' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197451#comment-15197451
]
Nicholas Chammas commented on SPARK-7481:
-
(Sorry Steve; can't comment on your proposal since I
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197165#comment-15197165
]
Steve Loughran commented on SPARK-7481:
---
...thinking some more about this
How about
# adding a
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177922#comment-15177922
]
Steve Loughran commented on SPARK-7481:
---
For comparison, the full AWS SDK is 13MB; the s3 SDK 570K,
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177918#comment-15177918
]
Steve Loughran commented on SPARK-7481:
---
Longer term, having spark_home /lib/*.jar is the best,
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177623#comment-15177623
]
Steve Loughran commented on SPARK-7481:
---
Hadoop 2.6 added S3a, which we put into a new
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176559#comment-15176559
]
Nicholas Chammas commented on SPARK-7481:
-
I'm not comfortable working with Maven so I can't
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176551#comment-15176551
]
Nicholas Chammas commented on SPARK-7481:
-
{quote}
One issue here that hadoop 2.6's hadoop-aws
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175664#comment-15175664
]
Steve Loughran commented on SPARK-7481:
---
One issue here that hadoop 2.6's hadoop-aws pulls in the
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15174442#comment-15174442
]
Peng Cheng commented on SPARK-7481:
---
+1 Me four
> Add Hadoop 2.6+ profile to pull in object store FS
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15174438#comment-15174438
]
Nicholas Chammas commented on SPARK-7481:
-
Many people seem to be downgrading to use Spark built
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156101#comment-15156101
]
Yardena commented on SPARK-7481:
+1, please add this.
lib/* approach would be great, or a profile like
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126175#comment-15126175
]
Steve Loughran commented on SPARK-7481:
---
having a lib/* would be fantastic, as it stops spark having
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15125566#comment-15125566
]
Marcelo Vanzin commented on SPARK-7481:
---
It doesn't necessarily affect this proposal. It would make
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15125143#comment-15125143
]
Josh Rosen commented on SPARK-7481:
---
How does this proposal change if we just remove the assembly and
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15085477#comment-15085477
]
Steve Loughran commented on SPARK-7481:
---
Josh,
there is a 2.6 profile —but all it currently does is
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15083646#comment-15083646
]
Josh Rosen commented on SPARK-7481:
---
Hey, is this task done? I see that we have a {{hadoop2.6}} profile
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703025#comment-14703025
]
Thomas Demoor commented on SPARK-7481:
--
[~srowen] and [~pc...@uowmail.edu.au]: with
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703585#comment-14703585
]
Peng Cheng commented on SPARK-7481:
---
Thanks a lot! A long run to the end.
Add Hadoop
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14647406#comment-14647406
]
Thomas Demoor commented on SPARK-7481:
--
Pulled the aws-upgrade out of HADOOP-11684 to
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563033#comment-14563033
]
Thomas Demoor commented on SPARK-7481:
--
[~srowen] and [~ste...@apache.org], for s3a,
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14557549#comment-14557549
]
Peng Cheng commented on SPARK-7481:
---
I've tried to do it but I get a lot of headaches,
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534168#comment-14534168
]
Sean Owen commented on SPARK-7481:
--
Yikes, that seems like a load of stuff to pull in.
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534174#comment-14534174
]
Steve Loughran commented on SPARK-7481:
---
This doesn't contain any endorsement of the
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534210#comment-14534210
]
Sean Owen commented on SPARK-7481:
--
Maybe I'd be less frightened if I knew the size of
[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534622#comment-14534622
]
Steve Loughran commented on SPARK-7481:
---
hadoop openstack 100K +httpclient (400K)
27 matches
Mail list logo
