Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58845062
This looks great; thanks for adding the comments. I'm going to merge this
into master and backport it to `branch-1.0` and `branch-1.1`.
---
If your project is set up
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/2712
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user james64 commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58743254
Can it be that test Flume test failed due to upstream changes? It is
passing for me locally now.
---
If your project is set up for it, you can reply to this email and
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58743306
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21641/consoleFull)
for PR 2712 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58744782
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21641/consoleFull)
for PR 2712 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58744783
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58760768
That particular Flume test is known to be flaky; I think that TD is working
on a rewrite / fix for that test suite.
---
If your project is set up for it, you can
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/2712#discussion_r18743728
--- Diff: core/src/test/scala/org/apache/spark/SparkContextSuite.scala ---
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58760950
It looks like the original implementation of this converter was added in
2604939f643bca125f5e2fb53e3221202996d41b, all the way back in 2011, so I
believe that this
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58761280
Actually, I don't think that this is a bug. Instead, I think that the
behavior that you're seeing could be an instance of
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58761660
Actually, ignore my earlier (deleted) comments; this looks like a valid
issue (see [HADOOP-6298: BytesWritable#getBytes is a bad name that leads to
programming
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/2712#discussion_r18743890
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1409,7 +1410,9 @@ object SparkContext extends Logging {
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58761838
This looks good to me; sorry for my earlier confusion. If you add a
comment and change the name of the test, I'll merge this and cherry-pick it
back into `branch-1.1`
Github user james64 commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58767976
Sorry for the test name. Now it should be all fine including commets.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58768088
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21649/consoleFull)
for PR 2712 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58769371
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21649/consoleFull)
for PR 2712 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58769373
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user dbtsai commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58629065
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58717548
Jenkins, this is ok to test.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58718323
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21602/consoleFull)
for PR 2712 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58718438
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58718433
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21602/consoleFull)
for PR 2712 at commit
Github user sryza commented on a diff in the pull request:
https://github.com/apache/spark/pull/2712#discussion_r18732020
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1409,7 +1411,7 @@ object SparkContext extends Logging {
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58722604
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21603/consoleFull)
for PR 2712 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58726788
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21603/consoleFull)
for PR 2712 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58726794
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user dbtsai commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58732030
It's failing at FlumeStreamSuite.scala:109 which seems to be unrelated to
this patch.
---
If your project is set up for it, you can reply to this email and have your
Github user sryza commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58732168
One more nit: the added java import should go with the other java imports.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user sryza commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58732173
Otherwise, LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user sryza commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58473731
Hmm, yeah, copyBytes is no good if it doesn't appear in Hadoop 1.
My suggestion would be to use from copyOfRange from java.util.Arrays.
---
If your project is set
GitHub user james64 opened a pull request:
https://github.com/apache/spark/pull/2712
[SPARK-3121] Wrong implementation of implicit bytesWritableConverter
val path = ... //path to seq file with BytesWritable as type of both key
and value
val file =
Github user dbtsai commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58361701
Jenkins, please start the test.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58361679
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58386907
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/287/consoleFull)
for PR 2712 at commit
Github user sryza commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58387498
Great catch.
A concern is that calling Array#take requires an implicit conversion, which
has some performance impact that might be unacceptable for this method
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58396199
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/287/consoleFull)
for PR 2712 at commit
Github user james64 commented on the pull request:
https://github.com/apache/spark/pull/2712#issuecomment-58441095
Originaly I wanted to just replace getBytes method with copyBytes. It is
available in newer versions of api but I found an older version is imported in
spark. I am not
37 matches
Mail list logo
