Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3464#issuecomment-68118318
[Test build #24826 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24826/consoleFull)
for PR 3464 at commit
Github user WangTaoTheTonic commented on the pull request:
https://github.com/apache/spark/pull/3607#issuecomment-68118403
Actually we cann't confirm that we will get `--class`(the one indicates
cluster or client mode) before `--driver-memory', can we?
---
If your project is set up
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/3778#discussion_r22277093
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala
---
@@ -112,7 +112,30 @@ case class InSet(value:
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/3778#discussion_r22277105
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala
---
@@ -160,6 +183,49 @@ abstract class
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/3778#discussion_r22277118
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -293,6 +295,380 @@ object OptimizeIn extends
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/3778#discussion_r22277164
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -293,6 +295,380 @@ object OptimizeIn extends
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3795#issuecomment-68118998
[Test build #24825 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24825/consoleFull)
for PR 3795 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3795#issuecomment-68118999
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3466#issuecomment-68119029
[Test build #24827 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24827/consoleFull)
for PR 3466 at commit
Github user jerryshao commented on the pull request:
https://github.com/apache/spark/pull/3466#issuecomment-68119033
Hi TD, thanks a lot for your comments, I just change the code style as you
suggested, also add one more metrics `totalProcessedRecords`, would you mind
reviewing this
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/3778#issuecomment-68119210
This is really a very useful optimization, particularly for those SQLs
generated by machines. And it would make more senses if we add unit test to
reflect the
GitHub user XuTingjun opened a pull request:
https://github.com/apache/spark/pull/3806
specify AM core in yarn-client and yarn-cluster mode
I add some configurations below.
spark.yarn.am.cores/SPARK_MASTER_CORES/SPARK_DRIVER_CORES for yarn-client
mode;
spark.driver.cores
Github user XuTingjun closed the pull request at:
https://github.com/apache/spark/pull/3799
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
GitHub user CodingCat opened a pull request:
https://github.com/apache/spark/pull/3807
SPARK-4971: fix typo in the comment
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/CodingCat/spark new_branch
Alternatively you can review
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3806#issuecomment-68119304
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3807#issuecomment-68119322
[Test build #24828 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24828/consoleFull)
for PR 3807 at commit
Github user scwf commented on a diff in the pull request:
https://github.com/apache/spark/pull/3778#discussion_r22277333
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala
---
@@ -112,7 +112,30 @@ case class InSet(value: Expression,
Github user scwf commented on a diff in the pull request:
https://github.com/apache/spark/pull/3778#discussion_r22277360
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -293,6 +295,380 @@ object OptimizeIn extends
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3800#issuecomment-68120181
[Test build #24829 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24829/consoleFull)
for PR 3800 at commit
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/3778#discussion_r22277573
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala
---
@@ -112,7 +112,30 @@ case class InSet(value:
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3464#issuecomment-68120641
[Test build #24826 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24826/consoleFull)
for PR 3464 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3464#issuecomment-68120643
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/3466#issuecomment-68121187
LGTM. Merging this. Thanks @jerryshao
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/3464#issuecomment-68121248
Merging this. Thanks @zsxwing!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/3466
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/3464
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3466#issuecomment-68121546
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3466#issuecomment-68121543
[Test build #24827 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24827/consoleFull)
for PR 3466 at commit
Github user cloud-fan commented on the pull request:
https://github.com/apache/spark/pull/3796#issuecomment-68121673
Looks like using `AttributeMap` can avoid creating many `AttributeEquals`
during map building and searching. Did I miss something here? I'm not so
familiar with
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3807#issuecomment-68121785
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3807#issuecomment-68121783
[Test build #24828 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24828/consoleFull)
for PR 3807 at commit
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/3778#issuecomment-68121946
@scwf Would you mind to list all the optimizations in the PR description
first? Some more concise examples coupled with each optimization can be really
helpful. Then
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/3780#discussion_r22278028
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala ---
@@ -199,11 +199,15 @@ private[sql] abstract class
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/3661#discussion_r22277996
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/ContextWaiter.scala ---
@@ -17,30 +17,63 @@
package org.apache.spark.streaming
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3800#issuecomment-68122163
[Test build #24829 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24829/consoleFull)
for PR 3800 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3800#issuecomment-68122164
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user sryza commented on the pull request:
https://github.com/apache/spark/pull/3806#issuecomment-68122279
SPARK_MASTER_CORES uses master incorrectly. The only reason we have a
SPARK_MASTER_MEMORY was to preserve backwards capability.
This patch also still appears to
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-68122296
[Test build #24830 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24830/consoleFull)
for PR 3467 at commit
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/3796#issuecomment-68123606
Yes, you're right, thank you. I've updated the code.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/3778#issuecomment-68123618
For numeric comparison optimizations, did some experiments along my former
double interval comparison idea and came up with the following snippet, I
haven't even
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3796#issuecomment-68123625
[Test build #24831 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24831/consoleFull)
for PR 3796 at commit
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/3780#discussion_r22278489
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala ---
@@ -199,11 +199,15 @@ private[sql] abstract class
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-68124725
[Test build #24830 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24830/consoleFull)
for PR 3467 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-68124728
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user XuTingjun commented on the pull request:
https://github.com/apache/spark/pull/3806#issuecomment-68125091
@sryza, I am not agree with you. I only add the below code into cluster
mode. So the --driver-cores will not work in client mode.
OptionAssigner(args.driverCores,
Github user jerryshao commented on the pull request:
https://github.com/apache/spark/pull/3807#issuecomment-68125701
Aha, good catch :).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3796#issuecomment-68126023
[Test build #24831 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24831/consoleFull)
for PR 3796 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3796#issuecomment-68126024
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
101 - 148 of 148 matches
Mail list logo
