Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2538#issuecomment-57758335
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21242/consoleFull)
for PR 2538 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2624#issuecomment-57758615
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21237/consoleFull)
for PR 2624 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2624#issuecomment-57758618
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2432#issuecomment-57760172
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21239/consoleFull)
for PR 2432 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2432#issuecomment-57760182
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user ash211 commented on a diff in the pull request:
https://github.com/apache/spark/pull/2401#discussion_r18382851
--- Diff: docs/configuration.md ---
@@ -253,6 +253,17 @@ Apart from these, the following properties are also
available, and may be useful
Github user ash211 commented on the pull request:
https://github.com/apache/spark/pull/2401#issuecomment-57760828
This looks very reasonable. Counting the executor's bookkeeping core
against the resources also seems much more correct than pretending it doesn't
exist like before.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2538#issuecomment-57761610
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2538#issuecomment-57761606
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21241/consoleFull)
for PR 2538 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2538#issuecomment-57762293
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2538#issuecomment-57762288
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21242/consoleFull)
for PR 2538 at commit
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/2484#issuecomment-57763330
Unfortunately it doesn't merge. How large of a cluster does we need to get
this issue to happen?
---
If your project is set up for it, you can reply to this email and
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/1890#issuecomment-57763605
Yes we will take a look at this for 1.2.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/2563#discussion_r18383911
--- Diff: python/pyspark/sql.py ---
@@ -62,6 +67,17 @@ def __eq__(self, other):
def __ne__(self, other):
return not self.__eq__(other)
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/2563#discussion_r18384024
--- Diff: python/pyspark/sql.py ---
@@ -312,42 +343,24 @@ def __repr__(self):
return (StructType(List(%s)) %
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/2563#discussion_r18384157
--- Diff: python/pyspark/sql.py ---
@@ -385,51 +398,35 @@ def _parse_datatype_string(datatype_string):
check_datatype(complex_maptype)
True
GitHub user tsudukim opened a pull request:
https://github.com/apache/spark/pull/2640
[SPARK-3775] Not suitable error message in spark-shell.cmd
Modified some sentence of error message in bin\*.cmd.
You can merge this pull request into a Git repository by running:
$ git pull
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/2640#issuecomment-57771950
I don't agree that users should be directed to build with SBT. Maven is the
default, but, probably best to not even make a specific recommendation.
---
If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2640#issuecomment-57772040
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user tsudukim commented on the pull request:
https://github.com/apache/spark/pull/2640#issuecomment-57772760
Yes, so I removed the specific recommendation for SBT.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/2595
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/2595#issuecomment-57778527
LGTM. Merged into master. Thanks @chouqin , and @jkbradley and @manishamde
for code review!
Increasing `maxMemoryInMB` also increases the shuffle size. As long as
GitHub user r3natko opened a pull request:
https://github.com/apache/spark/pull/2641
[SQL] SPARK-3776: Wrong conversion to Catalyst for Option[Product]
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/r3natko/spark
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2641#issuecomment-57783977
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/2636#issuecomment-57785126
@mdagost If you convert `(Int, Array[Double])` to a
`java.util.ListObject` (id the first and features the second (without
converting to string)), you should be able to
Github user EugenCepoi commented on the pull request:
https://github.com/apache/spark/pull/2481#issuecomment-57786084
@andrewor14 The trailing % came from a bad copy paste, nice catch :) Double
percent exists but was wrong in this case,
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2481#issuecomment-57786537
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21243/consoleFull)
for PR 2481 at commit
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/1980#issuecomment-57787121
Hi, @JoshRosen, refer to http://wiki.eclipse.org/Jetty/Howto/Configure_SSL
you can generating keys and certificates by openssl or keytool
1 Generating Keys and
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1980#issuecomment-57789135
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21244/consoleFull)
for PR 1980 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2520#issuecomment-57790931
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21245/consoleFull)
for PR 2520 at commit
GitHub user zsxwing opened a pull request:
https://github.com/apache/spark/pull/2642
[SPARK-3777] Display Executor ID for Tasks in Stage page
Now the Stage page only displays Executor(host) for tasks. However, there
may be more than one Executors running in the same host.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/2638#issuecomment-57791903
A particular instance of Spark will be built for a particular version of
Hadoop and/or YARN. It is not at this point a universal binary anyway, and so,
I do not think it
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2642#issuecomment-57792294
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21246/consoleFull)
for PR 2642 at commit
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/2637#issuecomment-57792726
+1, makes more sense for sure. Open a JIRA for this as well?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/2484#issuecomment-57794438
I've seen it with 200 executors.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2481#issuecomment-57795374
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21243/consoleFull)
for PR 2481 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2481#issuecomment-57795383
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1980#issuecomment-57796385
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1980#issuecomment-57796375
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21244/consoleFull)
for PR 1980 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2520#issuecomment-57797668
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21245/consoleFull)
for PR 2520 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2520#issuecomment-57797676
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/2577#discussion_r18395371
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
---
@@ -328,10 +348,18 @@ private[spark] class
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/2577#discussion_r18395494
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
---
@@ -328,10 +348,18 @@ private[spark] class
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2642#issuecomment-57799611
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21246/consoleFull)
for PR 2642 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2642#issuecomment-57799622
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user ueshin commented on the pull request:
https://github.com/apache/spark/pull/2638#issuecomment-57800537
@srowen, Thank you for your comment.
Indeed, when deploy completed apps to Spark cluster, there is a particular
instance of Spark.
But Spark app developers will
Github user WangTaoTheTonic commented on the pull request:
https://github.com/apache/spark/pull/2541#issuecomment-57803237
@JoshRosen I have tested and it worked fine. You can also have a try simply.
---
If your project is set up for it, you can reply to this email and have your
Github user WangTaoTheTonic commented on the pull request:
https://github.com/apache/spark/pull/2541#issuecomment-57803284
@JoshRosen I have tested and it worked fine. You can also have a try simply.
---
If your project is set up for it, you can reply to this email and have your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2561#issuecomment-57803554
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21247/consoleFull)
for PR 2561 at commit
Github user saucam commented on the pull request:
https://github.com/apache/spark/pull/2561#issuecomment-57803869
1. removed unnecessary UnaryMinus case as it is constant-folded away
2. Added unit test in ExpressionEvaluationSuite
3. Added a class OptimizedInSuite which tests
Github user CodingCat commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-57806876
ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user brndnmtthws commented on the pull request:
https://github.com/apache/spark/pull/2401#issuecomment-57806924
Fixed typo's (also switch from `Math.max` to `math.max` because that seems
to be the Scala way).
---
If your project is set up for it, you can reply to this email
Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/2577#issuecomment-57807019
Addressed all the review comments.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
GitHub user shijinkui opened a pull request:
https://github.com/apache/spark/pull/2643
code style format
code style format
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shijinkui/spark master
Alternatively you can review and
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2401#issuecomment-57807681
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21249/consoleFull)
for PR 2401 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2577#issuecomment-57807655
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21248/consoleFull)
for PR 2577 at commit
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/2563#discussion_r18399479
--- Diff: python/pyspark/sql.py ---
@@ -312,42 +343,24 @@ def __repr__(self):
return (StructType(List(%s)) %
Github user shijinkui closed the pull request at:
https://github.com/apache/spark/pull/2643
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2577#issuecomment-57808155
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2577#issuecomment-57808148
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21248/consoleFull)
for PR 2577 at commit
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/2643#issuecomment-57808284
Some changes here make lines longer than 100 characters, which contradicts
the scalastyle rules for the project. I am not sure that moving comments around
to a previous
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/2640#issuecomment-57808436
Oh I'm sorry, my poor brain read the diff the wrong way around. I agree
with the change!
---
If your project is set up for it, you can reply to this email and have your
GitHub user shijinkui opened a pull request:
https://github.com/apache/spark/pull/2644
[SPARK-3781] code Style format
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shijinkui/spark styleFormat
Alternatively you can review and
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2577#issuecomment-57809815
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21250/consoleFull)
for PR 2577 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2644#issuecomment-57810182
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/2636#issuecomment-57813143
@mdagost @mengxr We use Pyrolite to convert Java objects into Python
objects, you can get the type mapping here: https://github.com/irmen/Pyrolite
---
If your project is
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2401#issuecomment-57814712
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21249/consoleFull)
for PR 2401 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2401#issuecomment-57814725
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2561#issuecomment-57814819
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user brndnmtthws commented on the pull request:
https://github.com/apache/spark/pull/2401#issuecomment-57814884
That test failure appears to be unrelated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2561#issuecomment-57814811
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21247/consoleFull)
for PR 2561 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2538#issuecomment-57815905
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21251/consoleFull)
for PR 2538 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2577#issuecomment-57818940
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2577#issuecomment-57818934
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21250/consoleFull)
for PR 2577 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2620#issuecomment-57819162
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21252/consoleFull)
for PR 2620 at commit
Github user ravipesala commented on the pull request:
https://github.com/apache/spark/pull/2620#issuecomment-57819925
You are right, it is not so good to pass resolver in constructor. Instead I
just passed boolean flag.
---
If your project is set up for it, you can reply to this
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/2605#issuecomment-57820191
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2605#issuecomment-57821047
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21253/consoleFull)
for PR 2605 at commit
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/2401#issuecomment-57822162
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2401#issuecomment-57822909
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21254/consoleFull)
for PR 2401 at commit
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/2481#discussion_r18406038
--- Diff: bin/compute-classpath.sh ---
@@ -27,8 +27,14 @@ FWDIR=$(cd `dirname $0`/..; pwd)
. $FWDIR/bin/load-spark-env.sh
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/2481#issuecomment-57823932
LGTM. I'm merging this into master and 1.1. Thanks @EugenCepoi!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/2481
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user staple commented on the pull request:
https://github.com/apache/spark/pull/2491#issuecomment-57824188
@mengxr - Iâm sure you have a lot on your plate right now, but I wanted
to check in on this PR. Overall, how do you feel about this approach for
SPARK-1655? Iâm happy
Github user EugenCepoi commented on the pull request:
https://github.com/apache/spark/pull/2481#issuecomment-57824300
@andrewor14 great! Glad to see it merged into master :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2629#issuecomment-57824840
This looks good to me; thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/2541#discussion_r18407387
--- Diff: sbin/spark-config.sh ---
@@ -33,7 +33,7 @@ this=$config_bin/$script
export SPARK_PREFIX=`dirname $this`/..
export
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/2481#issuecomment-57826573
Ah yes I believe that's for the Spark daemons. We can merge that separately.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/2401#issuecomment-57826312
Hey @brndnmtthws I left a few more minor comments but I think this is
basically ready for merge. Thanks for your changes.
---
If your project is set up for it, you
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/2609#discussion_r18407971
--- Diff: core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala
---
@@ -191,6 +194,8 @@ private[spark] class Worker(
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/2541#issuecomment-57828829
Ok, I'm merging this into master and 1.1. Thanks @WangTaoTheTonic!
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/2541
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2605#issuecomment-57829432
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2605#issuecomment-57829426
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21253/consoleFull)
for PR 2605 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2401#issuecomment-57830054
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/2605#issuecomment-57830437
I see. In your experience how long does it take for the port to change (is
it usually instant)? Is there a relevant Yarn JIRA that describes the issue? If
so we could
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/2605#issuecomment-57830851
It's generally very quick (so quick that I've never seen this failure
locally, although I do see the log message I mention in the code comments with
port 0). I was
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2609#issuecomment-57831124
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21255/consoleFull)
for PR 2609 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2401#issuecomment-57831156
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/2469#discussion_r18409649
--- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala ---
@@ -34,7 +34,7 @@ import org.json4s.JValue
import
1 - 100 of 336 matches
Mail list logo
