Github user nchammas commented on the pull request:
https://github.com/apache/spark/pull/5027#issuecomment-93452826
Yeah, I asked about that some time ago, and I believe the concern was about
surprising users (by changing defaults) + the fact that the Hadoop 2 distro
used by
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/5027#issuecomment-93488039
Yeah spark-ec2 does not support Hadoop 2 right now, though there has been a
patch sitting around for a while now
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5027#issuecomment-93284165
How about setting up Hadoop 2 on EC2 by default?
Alternatively, yeah at least you'd want to specify a particular version if
a particular version is needed.
---
If
Github user nchammas commented on the pull request:
https://github.com/apache/spark/pull/5027#issuecomment-93117302
Confirmed. Simply building Spark with the Hadoop version explicitly set to
1.0.4 resolves this issue.
---
If your project is set up for it, you can reply to this email
Github user nchammas commented on the pull request:
https://github.com/apache/spark/pull/5027#issuecomment-93069921
This PR seems to have broken spark-perf. Not sure why, but the executor
stderr logs have the following:
```
15/04/14 19:14:46 INFO
Github user nchammas commented on the pull request:
https://github.com/apache/spark/pull/5027#issuecomment-93076897
Suspicion is it's just a Hadoop 1 vs. 2 issue since spark-ec2 (which we use
for spark-perf testing) launches clusters with Hadoop 1 by default.
Will confirm.
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/5027#issuecomment-84201074
Looks good - thanks for commiting this sean.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/5027
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5027#issuecomment-83586909
I want to double- and triple-check about this. I'm in favor, I think
@pwendell is in favor since it reflects how Spark is already published vs
Hadoop 2.2. It doesn't
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5027#issuecomment-80157961
[Test build #28609 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28609/consoleFull)
for PR 5027 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5027#issuecomment-80253524
[Test build #28609 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28609/consoleFull)
for PR 5027 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5027#issuecomment-80253600
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
GitHub user srowen opened a pull request:
https://github.com/apache/spark/pull/5027
SPARK-5134 [BUILD] Bump default Hadoop version to 2+
Bump default Hadoop version to 2.2.0. (This is already the dependency
version reported by published Maven artifacts.) See JIRA for further
13 matches
Mail list logo
