[
https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176559#comment-15176559
]
Nicholas Chammas commented on SPARK-7481:
-----------------------------------------
I'm not comfortable working with Maven so I can't comment on the details of the
approach we should take, but I will appreciate any progress towards making
Spark built against Hadoop 2.6+ work with S3 out of the box, or as close to out
of the box as possible.
Given Spark's close relation to S3 and EC2 (as far as Spark's user base is
concerned), a good out of the box experience here is critical. Many people just
expect it.
> Add Hadoop 2.6+ profile to pull in object store FS accessors
> ------------------------------------------------------------
>
> Key: SPARK-7481
> URL: https://issues.apache.org/jira/browse/SPARK-7481
> Project: Spark
> Issue Type: Improvement
> Components: Build
> Affects Versions: 1.3.1
> Reporter: Steve Loughran
>
> To keep the s3n classpath right, to add s3a, swift & azure, the dependencies
> of spark in a 2.6+ profile need to add the relevant object store packages
> (hadoop-aws, hadoop-openstack, hadoop-azure)
> this adds more stuff to the client bundle, but will mean a single spark
> package can talk to all of the stores.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
