Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22666
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97314/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22666
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22666
**[Test build #97314 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97314/testReport)**
for PR 22666 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20761
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20761
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97320/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20761
**[Test build #97320 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97320/testReport)**
for PR 20761 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20761
**[Test build #97321 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97321/testReport)**
for PR 20761 at commit
Github user vincent-grosbois commented on the issue:
https://github.com/apache/spark/pull/21473
Hello!
Is there anyway this can be backported in 2.3?
---
-
To unsubscribe, e-mail:
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/21157
I mean, we could warn if we are doing the hijacking and not break peoples
pipelines?
---
-
To unsubscribe, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20761
**[Test build #97320 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97320/testReport)**
for PR 20761 at commit
Github user szyszy commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r224914397
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceRequestHelper.scala
---
@@ -0,0 +1,140 @@
+/*
+ * Licensed to
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/21157
But that would break both ipython notebooks and repl right? Pretty
significant breaking change.
---
-
To unsubscribe, e-mail:
Github user szyszy commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r224913985
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceRequestHelper.scala
---
@@ -0,0 +1,140 @@
+/*
+ * Licensed to
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r224913824
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceRequestHelper.scala
---
@@ -0,0 +1,140 @@
+/*
+ * Licensed to
Github user superbobry commented on the issue:
https://github.com/apache/spark/pull/21157
Yes, that is correct. That is why I think hijacking behaviour should be
removed. It silently slows down the job and does not notify the user that a
trivial change such as making the namedtuple
Github user dilipbiswal commented on the issue:
https://github.com/apache/spark/pull/22670
@srowen OK, Let me look.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user szyszy commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r224909836
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceRequestHelper.scala
---
@@ -0,0 +1,140 @@
+/*
+ * Licensed to
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22670
I don't so much mean that much refactoring. I wonder if there are 1-2 other
places where common Kafka params are set in tests that we could add this to for
now, that kind of thing. This change is OK
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/22597
Is it possible to add tests like parquet to remove the filter in Spark SQL
to ensure that the predicate is pushed down to the reader? Thanks.
---
Github user dilipbiswal commented on the issue:
https://github.com/apache/spark/pull/22670
@srowen Thanks. Did you mean, the test cases should extend a shared spark
context (SharedKafkaSparkContext) which would have this property set ?
Actually Sean, there are 3 suites in
Github user superbobry commented on the issue:
https://github.com/apache/spark/pull/21157
Nope, the job I was referring to is not open source; but I guess the
speedup is easy to justify: much less payload and faster deserialization:
```
>>> from collections import
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22379
**[Test build #97319 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97319/testReport)**
for PR 22379 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22666
**[Test build #97318 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97318/testReport)**
for PR 22666 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22379
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22379
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97313/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22379
**[Test build #97313 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97313/testReport)**
for PR 22379 at commit
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/22654
@gatorsmile @srowen Thank you for your work.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user koeninger commented on a diff in the pull request:
https://github.com/apache/spark/pull/22703#discussion_r224899199
--- Diff: docs/streaming-kafka-0-10-integration.md ---
@@ -3,7 +3,11 @@ layout: global
title: Spark Streaming + Kafka Integration Guide (Kafka broker
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21688
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97312/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21688
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21688
**[Test build #97312 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97312/testReport)**
for PR 21688 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22703
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22703
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97307/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22703
**[Test build #97307 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97307/testReport)**
for PR 22703 at commit
Github user dhruve commented on a diff in the pull request:
https://github.com/apache/spark/pull/22288#discussion_r224892495
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/BlacklistTracker.scala ---
@@ -146,21 +146,31 @@ private[scheduler] class BlacklistTracker (
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22593
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22593
Merged to master/2.4
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20999
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20999
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97309/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21710
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22654
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21710
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20999
**[Test build #97309 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97309/testReport)**
for PR 20999 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22593
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22593
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97311/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22593
**[Test build #97311 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97311/testReport)**
for PR 22593 at commit
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22322
Ping @npoberezkin
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22654
LGTM
Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21710
**[Test build #97317 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97317/testReport)**
for PR 21710 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22710
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97315/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22710
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22710
**[Test build #97315 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97315/testReport)**
for PR 22710 at commit
Github user abellina commented on a diff in the pull request:
https://github.com/apache/spark/pull/22288#discussion_r224879925
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/BlacklistTracker.scala ---
@@ -146,21 +146,31 @@ private[scheduler] class BlacklistTracker (
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22670
@dilipbiswal I like this change too. The suite goes from 4:34 to 0:53. I
wonder if we can make this change elsewhere in general Kafka test config? This
kind of setting seems useful everywhere.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22690
**[Test build #97316 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97316/testReport)**
for PR 22690 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22690
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22690
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21322#discussion_r224874828
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -384,15 +385,30 @@ private[spark] class MemoryStore(
}
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21322#discussion_r224875899
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -1930,6 +1930,18 @@ private[spark] object Utils extends Logging {
}
}
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21322#discussion_r224875111
--- Diff:
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
@@ -384,15 +385,30 @@ private[spark] class MemoryStore(
}
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22699
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97308/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22699
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22699
**[Test build #97308 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97308/testReport)**
for PR 22699 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22710
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22710
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22710
**[Test build #97315 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97315/testReport)**
for PR 22710 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22414
**[Test build #4374 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4374/testReport)**
for PR 22414 at commit
GitHub user squito opened a pull request:
https://github.com/apache/spark/pull/22710
DO NOT MERGE
just for testing
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/squito/spark blah
Alternatively you can review and apply these
Github user dhruve commented on a diff in the pull request:
https://github.com/apache/spark/pull/22288#discussion_r224873268
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/BlacklistTracker.scala ---
@@ -146,21 +146,31 @@ private[scheduler] class BlacklistTracker (
Github user JeetKunDoug commented on the issue:
https://github.com/apache/spark/pull/21322
@cloud-fan (Also left on the JIRA ticket) - Sorry this has dropped off my
radar for so long - work + life took me away from it for a while. So looking at
the PR review comments and better
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22689
Thanks a lot @srowen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22703#discussion_r224870517
--- Diff: python/pyspark/streaming/tests.py ---
@@ -1047,259 +1046,6 @@ def check_output(n):
self.ssc.stop(True, True)
-class
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22689
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22689
Merged to master/2.4
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22645
Thanks a lot @srowen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22383#discussion_r224866566
--- Diff: project/MimaExcludes.scala ---
@@ -36,6 +36,8 @@ object MimaExcludes {
// Exclude rules for 3.0.x
lazy val v30excludes =
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/22657
@srowen @HyukjinKwon @cloud-fan Thank you for your review of the PR.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17631
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22700
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22678
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r224865402
--- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala ---
@@ -341,7 +341,9 @@ private class LiveExecutorStageSummary(
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/20503
Gentle ping again to @ashashwat . Also @HyukjinKwon what are your opinions
on the test coverage?
---
-
To unsubscribe, e-mail:
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22700
Merged to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/22664
@peter-toth I assigned to you. Thanks for contribution.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22657
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22678
Merged to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r224864545
--- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala ---
@@ -341,7 +341,9 @@ private class LiveExecutorStageSummary(
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22654
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97306/
Test PASSed.
---
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22645
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/21157
Do you have the code for demonstrating the 2x speed up @superbobry ?
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22654
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/21157
Ok it looks like it was @HyukjinKwon who suggested that we remove this hack
in general rather than the partial work around can I get your thoughts on why?
It seems like the partial work around
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22654
**[Test build #97306 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97306/testReport)**
for PR 22654 at commit
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22657
Merged to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22645
Merged to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/18457
Sounds like we're not going to change this @SoulGuedria but we'd love your
contributions in Spark ML where things are actively being developed.
---
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/22295#discussion_r224860828
--- Diff: python/pyspark/sql/session.py ---
@@ -231,6 +231,7 @@ def __init__(self, sparkContext, jsparkSession=None):
or
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/22295#discussion_r224858616
--- Diff: python/pyspark/sql/session.py ---
@@ -252,6 +253,22 @@ def newSession(self):
"""
return self.__class__(self._sc,
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/22295#discussion_r224858233
--- Diff: python/pyspark/sql/functions.py ---
@@ -2633,6 +2633,23 @@ def sequence(start, stop, step=None):
_to_java_column(start),
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/22295#discussion_r224860350
--- Diff: python/pyspark/sql/tests.py ---
@@ -3654,6 +3654,109 @@ def test_jvm_default_session_already_set(self):
spark.stop()
101 - 200 of 362 matches
Mail list logo
