Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21952
I noticed that the benchmark uses `df.count`, is it possible that column
pruning has some issues in master?
---
-
To
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21948
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/21965
Jenkins, retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21952
The regression happens at writing. Looks like when benchmarking writing
time, we don't use `df.count`?
---
-
To unsubscribe,
Github user hhbyyh commented on the issue:
https://github.com/apache/spark/pull/21981
Thanks for the review @HyukjinKwon.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21952
**[Test build #94104 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94104/testReport)**
for PR 21952 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21952
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21952
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94104/
Test PASSed.
---
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21952
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user mengxr commented on the issue:
https://github.com/apache/spark/pull/21898
Also, if there shouldn't exist two active attempts at the same time for a
barrier stage, maybe we should store attemptId as a state variable. Basically,
if we see a new attempt ID, we should abort
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21952
Ah, finally I can reproduce this. It needs to allocate the array feature
with length 16000. I was reducing it to 1600 and it largely relieve the
regression. `com.databricks.spark.avro` is faster
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21965
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21895
**[Test build #94084 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94084/testReport)**
for PR 21895 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21979
**[Test build #94082 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94082/testReport)**
for PR 21979 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21965
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21965
**[Test build #94107 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94107/testReport)**
for PR 21965 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21948
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21981
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94105/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21981
**[Test build #94105 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94105/testReport)**
for PR 21981 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21981
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21911
**[Test build #94108 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94108/testReport)**
for PR 21911 at commit
Github user hhbyyh commented on the issue:
https://github.com/apache/spark/pull/21981
BTW, @HyukjinKwon, do you know who's still reviewing the ML PRs? I have a
few old PRs and I really want to know which're considered meaningful.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21979
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21895
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94084/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21979
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94082/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21895
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21981
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21981
**[Test build #94105 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94105/testReport)**
for PR 21981 at commit
Github user mengxr commented on the issue:
https://github.com/apache/spark/pull/21898
Here is what I mean:
~~~scala
case class ContextBarrierId(stageId: Int, stageAttemptId: Int)
class ContextBarrierState(val numTasks: Int) {
private var epoch: Int = 0
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21981
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21948
**[Test build #94106 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94106/testReport)**
for PR 21948 at commit
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/21898
> Also, if there shouldn't exist two active attempts at the same time for a
barrier stage, maybe we should store attemptId as a state variable. Basically,
if we see a new attempt ID, we should
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21948
@rdblue I have documented the object reuse behavior and ask data source to
handle it, please take a look, thanks!
---
-
To
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21398
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21699
@HyukjinKwon I revert the last changes. Please, take a look at the PR again.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20345
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/20345
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21979
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user mengxr commented on the issue:
https://github.com/apache/spark/pull/21915
test this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21952
**[Test build #94117 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94117/testReport)**
for PR 21952 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21952
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94117/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21952
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21975
Note that
```
[warn]
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21975
cc @srowen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21982
Isn't this PR related to the Jira ticket
[SPARK-23911](https://issues.apache.org/jira/browse/SPARK-23911)?
---
-
To
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21965#discussion_r207485560
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala
---
@@ -210,3 +219,54 @@ case class
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/21931
cc @maropu
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94109/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21965
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21957
**[Test build #94126 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94126/testReport)**
for PR 21957 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21970
**[Test build #94125 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94125/testReport)**
for PR 21970 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21983
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21955
**[Test build #94086 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94086/testReport)**
for PR 21955 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21981
I am not sure. ML is not my area but I am pretty sure who you know are
basically who I know .. ping me if that's minor or trivial like this. I can
review and merge.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21895
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21895
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21975
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21975
**[Test build #94112 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94112/testReport)**
for PR 21975 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21975
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21965
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21933
**[Test build #94114 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94114/testReport)**
for PR 21933 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21915
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21965
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/21966
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21915
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21975
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21982
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21982
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21975
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21721
**[Test build #94121 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94121/testReport)**
for PR 21721 at commit
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/21895
sure @jerryshao , will do. Thanks for the review.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/21965#discussion_r207479758
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala
---
@@ -210,3 +219,54 @@ case class
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21965#discussion_r207485584
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala
---
@@ -210,3 +219,54 @@ case class
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21981
Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20345
**[Test build #94110 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94110/testReport)**
for PR 20345 at commit
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21439
@gatorsmile @HyukjinKwon May I ask you to look at the PR one more time.
---
-
To unsubscribe, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #94109 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94109/testReport)**
for PR 21699 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20345
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21860#discussion_r207475539
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala
---
@@ -232,6 +232,22 @@ class WholeStageCodegenSuite extends
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #94109 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94109/testReport)**
for PR 21699 at commit
Github user mridulm commented on a diff in the pull request:
https://github.com/apache/spark/pull/21895#discussion_r207493081
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -779,6 +808,8 @@ private[history] class
Github user mridulm commented on a diff in the pull request:
https://github.com/apache/spark/pull/21895#discussion_r207493280
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -161,6 +162,29 @@ private[history] class
GitHub user YuvalItzchakov opened a pull request:
https://github.com/apache/spark/pull/21983
SPARK-24987 - Fix Kafka consumer leak when no new offsets for TopicPartition
## What changes were proposed in this pull request?
This small fix adds a `consumer.release()` call to
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21915
**[Test build #94090 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94090/testReport)**
for PR 21915 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21957
**[Test build #94094 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94094/testReport)**
for PR 21957 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21981
@hhbyyh BTW, there's typo for the JIRA ticket in the title. Not a big deal
though better to match.
---
-
To unsubscribe,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21933
**[Test build #94100 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94100/testReport)**
for PR 21933 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21955
**[Test build #94087 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94087/testReport)**
for PR 21955 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21898
**[Test build #94092 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94092/testReport)**
for PR 21898 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21911
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94108/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21957
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94094/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21975
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21948
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94106/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21933
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21721
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94097/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21586
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94103/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21975
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94096/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21933
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21955
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94086/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21966
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
1 - 100 of 749 matches
Mail list logo
