Github user LuciferYang commented on a diff in the pull request:
https://github.com/apache/spark/pull/23214#discussion_r238558889
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala
---
@@ -398,8 +399,8 @@ private[execution] final class
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23214
**[Test build #99651 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99651/testReport)**
for PR 23214 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22468
**[Test build #99650 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99650/testReport)**
for PR 22468 at commit
GitHub user 10110346 opened a pull request:
https://github.com/apache/spark/pull/23216
[SPARK-26264][CORE]It is better to add @transient to field 'locs' for class
`ResultTask`.
## What changes were proposed in this pull request?
The field 'locs' is only used in driver side for
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/23213
cc: @cloud-fan @mgaido91
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23088
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user LuciferYang commented on the issue:
https://github.com/apache/spark/pull/23214
```
For broadcast hash join, we will copy the broadcasted hash relation to
avoid multi-thread problem, via HashedRelation.asReadOnlyCopy. However, this is
a shallow copy, the
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23194
**[Test build #99656 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99656/testReport)**
for PR 23194 at commit
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/23194
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23215
**[Test build #99657 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99657/testReport)**
for PR 23215 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23195
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99659/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23216
**[Test build #99660 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99660/testReport)**
for PR 23216 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23194
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23214
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23194
**[Test build #99652 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99652/testReport)**
for PR 23194 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/23214
Thanks for doing this. I think we are more close to the root cause.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22468
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23194
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99652/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23214
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99651/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23088
**[Test build #99653 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99653/testReport)**
for PR 23088 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22468
**[Test build #99647 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99647/testReport)**
for PR 22468 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22468
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99650/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22468
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23088
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23088
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99653/
Test FAILed.
---
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/23214
I think there is a problem, but no one found out because it's only about
metrics.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23215
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/23195#discussion_r238596372
--- Diff: docs/structured-streaming-kafka-integration.md ---
@@ -624,3 +624,56 @@ For experimenting on `spark-shell`, you can also use
`--packages`
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/23195#discussion_r238596541
--- Diff: docs/structured-streaming-kafka-integration.md ---
@@ -624,3 +624,56 @@ For experimenting on `spark-shell`, you can also use
`--packages`
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/23195#discussion_r238599018
--- Diff: docs/structured-streaming-kafka-integration.md ---
@@ -624,3 +624,56 @@ For experimenting on `spark-shell`, you can also use
`--packages`
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23216
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23216
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23195
**[Test build #99659 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99659/testReport)**
for PR 23195 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23195
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23194
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/20433
cc: @gatorsmile
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/23213#discussion_r238625915
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala ---
@@ -144,9 +144,10 @@ class SQLQueryTestSuite extends QueryTest with
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23213#discussion_r238625581
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala ---
@@ -144,9 +144,10 @@ class SQLQueryTestSuite extends QueryTest with
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23088
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23088
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22468
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22468
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/22468
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user LuciferYang commented on the issue:
https://github.com/apache/spark/pull/23214
On the other hand, if is only a `multi-thread problem`, may not affect
performance because there is no synchronized code part ...
---
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22957#discussion_r238583330
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala
---
@@ -145,9 +145,14 @@ case class
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23215
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23195
**[Test build #99659 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99659/testReport)**
for PR 23195 at commit
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22957#discussion_r238626367
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala
---
@@ -145,9 +145,14 @@ case class
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22468
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99647/
Test FAILed.
---
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/23207
Can you share some ideas about it? IMO shuffle write metrics is hard, as an
RDD can have shuffle dependencies with multiple upstream RDDs. That said, in
general the shuffle write metrics should
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/23207
Thanks for your reply Wenchen, there's a sketch doc assigned in
JIRA:https://docs.google.com/document/d/1DX0gLkpk_NCE5MwI1_m4gnA2rLdjDkynZ02u2VWDR-8/edit
```
IMO shuffle write
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21919
**[Test build #99658 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99658/testReport)**
for PR 21919 at commit
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/22450
@cloud-fan this has been stuck for a while now. Is there something blocking
this? Is there something I can do? Thanks.
---
-
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/23215
I think this new behavior makes more sense, but we need to add a migration
guide.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23194
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/23215
[SPARK-26263][SQL] Throw exception when Partition column value can't be
converted to user specified type
## What changes were proposed in this pull request?
Currently if user
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/23215
@cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23213#discussion_r238627633
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
---
@@ -2899,6 +2899,144 @@ class SQLQuerySuite extends QueryTest with
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22468
**[Test build #99655 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99655/testReport)**
for PR 22468 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23088
**[Test build #99654 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99654/testReport)**
for PR 23088 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23203
Yea, will update it as well after this one gets merged.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/23195#discussion_r238593413
--- Diff: docs/structured-streaming-kafka-integration.md ---
@@ -624,3 +624,56 @@ For experimenting on `spark-shell`, you can also use
`--packages`
Github user UtkarshMe closed the pull request at:
https://github.com/apache/spark/pull/22822
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/23213
just a question, why didn't we introduce something like what was done in
SPARK-24562? I see that these are configs which are valid for all queries, so
using what was done in SPARK-24562 is not a
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23213#discussion_r238625268
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
---
@@ -2899,6 +2899,144 @@ class SQLQuerySuite extends QueryTest with
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/23213#discussion_r238625477
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
---
@@ -2899,6 +2899,144 @@ class SQLQuerySuite extends QueryTest with
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/23218
Surprisingly, all of three are due to consistent JVM crashes. It seems that
Scala 2.12.8 or Spark has some unstable code somewhere.
-
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/23213
> I personally think its orthogonal to SPARK-24562.
yes I agree. I am just asking if it makes sense to create a framework like
that. Now it is only about codegen, but in the future we may
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22468
thanks, merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/22952
@HeartSaVioR
I've taken a look at the possibilities:
*
GitHub user mgaido91 opened a pull request:
https://github.com/apache/spark/pull/23217
[SPARK-25829][SQL][FOLLOWUP] Refactor MapConcat in order to check properly
the limit size
## What changes were proposed in this pull request?
The PR starts from the
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/23217
cc @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23217
**[Test build #99664 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99664/testReport)**
for PR 23217 at commit
Github user wangyum commented on the issue:
https://github.com/apache/spark/pull/22600
2.12.8 is out. Do we need to upgrade to 2.12.8?
2.12.8 fixes two regressions that appeared in 2.12.7:
```
Don't reject views with result types which are TypeVars (#7295)
Don't emit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23216
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99660/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23216
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23217
**[Test build #99664 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99664/testReport)**
for PR 23217 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23217
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99664/
Test FAILed.
---
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/23098
Note I'm holding on to this PR for a while as I understand it might be
disruptive to downstream builds to remove 2.11 support just now. Will look at
merging it in weeks. Right now it's an FYI.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23098
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23217
**[Test build #99668 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99668/testReport)**
for PR 23217 at commit
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23207#discussion_r238630981
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala ---
@@ -92,6 +92,12 @@ private[spark] class ShuffleMapTask(
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23207#discussion_r238633725
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala ---
@@ -92,6 +92,12 @@ private[spark] class ShuffleMapTask(
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23213
**[Test build #99661 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99661/testReport)**
for PR 23213 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23088
**[Test build #99654 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99654/testReport)**
for PR 23088 at commit
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22957#discussion_r238642801
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala
---
@@ -145,9 +145,14 @@ case class
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23217
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/23194
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21919
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99658/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21919
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23218
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23218
**[Test build #99666 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99666/testReport)**
for PR 23218 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23218
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99666/
Test FAILed.
---
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/22468#discussion_r238683833
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/UnsafeRowConverterSuite.scala
---
@@ -535,4 +535,98 @@ class
Github user TomaszGaweda closed the pull request at:
https://github.com/apache/spark/pull/22249
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user aokolnychyi commented on the issue:
https://github.com/apache/spark/pull/23171
As @rxin said, if we introduce a separate expression for the switch-based
approach, then we will need to modify other places. For example,
`DataSourceStrategy$translateFilter`. So, integrating
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22468
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22957#discussion_r238634730
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala
---
@@ -145,9 +145,14 @@ case class
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22468
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99655/
Test PASSed.
---
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22957#discussion_r238650207
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala
---
@@ -145,9 +145,14 @@ case class
1 - 100 of 468 matches
Mail list logo
