Github user markhamstra commented on the pull request:
https://github.com/apache/spark/pull/3550#issuecomment-65194915
The application won't be killed if an executor has been recognized by
master as RUNNING
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3548#issuecomment-65195159
[Test build #24035 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24035/consoleFull)
for PR 3548 at commit
Github user roji commented on the pull request:
https://github.com/apache/spark/pull/1875#issuecomment-65195595
Sorry for dropping out, was involved in other things. @mateiz, I'll take a
look at your suggestions in the coming week.
---
If your project is set up for it, you can reply
GitHub user rxin opened a pull request:
https://github.com/apache/spark/pull/3552
Indent license header properly for interfaces.scala.
A very small nit update.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rxin/spark
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/3544#issuecomment-65196022
cc @ankurdave
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/3540#issuecomment-65196135
@roxchkplusony thanks for submitting this. Wouldn't you be able to achieve
the same thing by setting the compress flag to off?
---
If your project is set up for it, you
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/3530#issuecomment-65196225
Thanks. Merging in master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/3530
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user roxchkplusony closed the pull request at:
https://github.com/apache/spark/pull/3540
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user roxchkplusony commented on the pull request:
https://github.com/apache/spark/pull/3540#issuecomment-65196744
You're absolutely right. I was simply not familiar with
spark.shuffle.compress. I saw spark.broadcast.compress easily enough,
hilariously... This PR and Jira
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3552#issuecomment-65197472
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/3540#issuecomment-65198096
Thanks for doing that!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/3552#issuecomment-65198169
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3552#issuecomment-65198674
[Test build #24037 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24037/consoleFull)
for PR 3552 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3548#issuecomment-65199081
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3548#issuecomment-65199074
[Test build #24031 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24031/consoleFull)
for PR 3548 at commit
Github user zzcclp commented on the pull request:
https://github.com/apache/spark/pull/3530#issuecomment-65199688
Whether this PR will be merged into the branch 1.2?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/3530#issuecomment-65200062
No. This is for SPARK-4397(#3262) and #3262 wasn't merged to branch 1.2.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3550#issuecomment-65200639
[Test build #24033 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24033/consoleFull)
for PR 3550 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3550#issuecomment-65200648
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user roxchkplusony commented on the pull request:
https://github.com/apache/spark/pull/3540#issuecomment-65200691
:)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3551#issuecomment-65200730
[Test build #24039 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24039/consoleFull)
for PR 3551 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3551#issuecomment-65200849
[Test build #24039 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24039/consoleFull)
for PR 3551 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3551#issuecomment-65200852
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3548#issuecomment-65201367
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3548#issuecomment-65201358
[Test build #24034 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24034/consoleFull)
for PR 3548 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3551#issuecomment-65201444
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3548#issuecomment-65202804
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3548#issuecomment-65202792
[Test build #24035 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24035/consoleFull)
for PR 3548 at commit
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/3505#discussion_r21147642
--- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala
---
@@ -481,9 +481,203 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])
*
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3552#issuecomment-65206246
[Test build #24037 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24037/consoleFull)
for PR 3552 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3552#issuecomment-65206253
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
GitHub user maji2014 opened a pull request:
https://github.com/apache/spark/pull/3553
[spark-4691][shuffle]code optimization for judgement
In HashShuffleReader.scala and HashShuffleWriter.scala, no need to judge
dep.aggregator.isEmpty again as this is judged by
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3553#issuecomment-65207158
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
GitHub user Lewuathe opened a pull request:
https://github.com/apache/spark/pull/3554
[SPARK-4685] Update JavaDoc settings to include spark.ml and all spark.m...
...llib subpackages in the right sections
Create subcategory, MLlib and ML for correspond packages.
You can
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3554#issuecomment-65210755
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3551#issuecomment-65213405
[Test build #24040 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24040/consoleFull)
for PR 3551 at commit
GitHub user YanTangZhai opened a pull request:
https://github.com/apache/spark/pull/3555
[SPARK-4692] [SQL] Support ! boolean logic operator like NOT
Support ! boolean logic operator like NOT in sql as follows
select * from for_test where !(col1 col2)
You can merge this pull
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3555#issuecomment-65214491
[Test build #24041 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24041/consoleFull)
for PR 3555 at commit
Github user jacek-lewandowski commented on the pull request:
https://github.com/apache/spark/pull/2739#issuecomment-65216596
Back to working on this... I've rebased against branch-1.2 and then I'll
rebase against master if you want.
---
If your project is set up for it, you can
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3555#issuecomment-65221949
[Test build #24041 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24041/consoleFull)
for PR 3555 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3555#issuecomment-65221953
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3551#issuecomment-65222661
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3551#issuecomment-65222656
[Test build #24040 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24040/consoleFull)
for PR 3551 at commit
Github user akopich commented on the pull request:
https://github.com/apache/spark/pull/1269#issuecomment-65223123
@jkbradley, thank you for you comments!
It seems like we should discuss API for this set of models first.
As far as I can understand, you are not about to
Github user WangTaoTheTonic commented on the pull request:
https://github.com/apache/spark/pull/3551#issuecomment-65223765
I will amend the comments and docs once we can make sure to merge this
commit.
@ScrapCodes I saw you [changed the
GitHub user YanTangZhai opened a pull request:
https://github.com/apache/spark/pull/3556
[SPARK-4693] [SQL] PruningPredicates may be wrong if predicates contains an
empty AttributeSet() references
The sql select * from spark_test::for_test where abs(20141202) is not
null has
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3556#issuecomment-65224550
[Test build #24042 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24042/consoleFull)
for PR 3556 at commit
Github user suyanNone commented on a diff in the pull request:
https://github.com/apache/spark/pull/791#discussion_r21156020
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -837,11 +837,11 @@ private[spark] class BlockManager(
* Drop a
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3547#issuecomment-65231830
[Test build #24043 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24043/consoleFull)
for PR 3547 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3556#issuecomment-65232200
[Test build #24042 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24042/consoleFull)
for PR 3556 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3556#issuecomment-65232207
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3547#issuecomment-65240856
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3547#issuecomment-65240848
[Test build #24043 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24043/consoleFull)
for PR 3547 at commit
Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/3409#issuecomment-65241674
memory should not be specified through the java options (I filed a separate
jira to support that - I would expect to have something like
spark.yarn.am.memory for
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3431#issuecomment-65242931
[Test build #24044 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24044/consoleFull)
for PR 3431 at commit
GitHub user WangTaoTheTonic opened a pull request:
https://github.com/apache/spark/pull/3557
[SPARK-4697]System properties should override environment variables
I found some arguments in yarn module take environment variables before
system properties while the latter override the
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3557#issuecomment-65245329
[Test build #24045 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24045/consoleFull)
for PR 3557 at commit
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/2589#issuecomment-65248499
Thank you @marmbrus , I've updated the code as you suggested.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3429#issuecomment-65248898
[Test build #24046 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24046/consoleFull)
for PR 3429 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2589#issuecomment-65248882
[Test build #24047 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24047/consoleFull)
for PR 2589 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3470#issuecomment-65253492
[Test build #24048 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24048/consoleFull)
for PR 3470 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3499#issuecomment-65254021
[Test build #24050 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24050/consoleFull)
for PR 3499 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3431#issuecomment-65254247
[Test build #24044 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24044/consoleFull)
for PR 3431 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3431#issuecomment-65254255
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3522#issuecomment-65254437
[Test build #24049 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24049/consoleFull)
for PR 3522 at commit
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/3508#issuecomment-65259690
Yeah the performance regression was difficult to reproduce according to
@vanzin. I haven't been able to reproduce it myself but others have. Yes, this
patch should
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3557#issuecomment-65259859
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3557#issuecomment-65259844
[Test build #24045 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24045/consoleFull)
for PR 3557 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3429#issuecomment-65260446
[Test build #24046 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24046/consoleFull)
for PR 3429 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3429#issuecomment-65260456
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user jackylk commented on the pull request:
https://github.com/apache/spark/pull/3133#issuecomment-65260781
had updated according to @marmbrus comment
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2589#issuecomment-65261259
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2589#issuecomment-65261249
[Test build #24047 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24047/consoleFull)
for PR 2589 at commit
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/3550#issuecomment-65264098
Yeah, this seems safe to me. Even if the Master doesn't know that the
driver has exited for some reason (i.e. if the `finishApplication` was somehow
not triggered
Github user avulanov commented on the pull request:
https://github.com/apache/spark/pull/1290#issuecomment-65264353
Just in case, I've tested the ANN without the hidden layer and it seems to
work as multinomial regression, though with (one-half) squared-error cost
function and
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3470#issuecomment-65265127
[Test build #24048 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24048/consoleFull)
for PR 3470 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3470#issuecomment-65265133
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/3542
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3499#issuecomment-65265709
[Test build #24050 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24050/consoleFull)
for PR 3499 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3499#issuecomment-65265713
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3522#issuecomment-65267595
[Test build #24049 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24049/consoleFull)
for PR 3522 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3522#issuecomment-65267607
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user markhamstra commented on the pull request:
https://github.com/apache/spark/pull/3550#issuecomment-65269257
It's worth spending a little time checking that any executors that are
RUNNING for an application definitely will transition to a Finished state and
be removed from
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3558#issuecomment-65270774
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
GitHub user jackylk opened a pull request:
https://github.com/apache/spark/pull/3558
[SPARK-4699][SQL] make caseSensitive configurable in Analyzer.scala
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jackylk/spark case
Github user mridulm commented on the pull request:
https://github.com/apache/spark/pull/3541#issuecomment-65272551
Thx @kayousterhout for the ping !
We are fairly aggressively using blacklisting executors - not hosts.
The assumption that a task failed on an executor in a host
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/2739#issuecomment-65278543
@jacek-lewandowski please work on top of master. We can work on backporting
it to branch-1.2 if there's a strong desire for it, but new features should
always be checked
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/3541#issuecomment-65278813
@mridulm Thanks for the feedback. Could you provide some details about
configurations (such as the number of executors on a host and the number of
hosts)? And in which
Github user sryza commented on the pull request:
https://github.com/apache/spark/pull/3554#issuecomment-65279181
Does this require any corresponding change in the Maven build?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user jacek-lewandowski commented on the pull request:
https://github.com/apache/spark/pull/2739#issuecomment-65279317
@vanzin already did that. Now I'm running tests - is there a new procedure
for testing? Or just sbt clean assembly test?
I can see one test failure
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/2739#issuecomment-65280103
I think `dev/run-tests` will run more comprehensive checks. Also, always a
good idea to try such a thing yourself on real jobs, aside from the unit tests.
:-)
Github user jacek-lewandowski commented on the pull request:
https://github.com/apache/spark/pull/2739#issuecomment-65280862
@vanzin yeah, thats right
I tried running the real jobs before rebasing and they worked :)
---
If your project is set up for it, you can reply to this
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/3558#issuecomment-65281087
Thanks for working on this! What about HiveContext?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/3558#issuecomment-65281098
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3558#issuecomment-65281697
[Test build #24051 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24051/consoleFull)
for PR 3558 at commit
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/3545#issuecomment-65282261
cc @tdas can you take a look at this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/3508#issuecomment-65283129
Just to reinforce, we shouldn't add a security manager without better
understanding what impact it has. Feel free to file a bug if you'd like this
investigated, but I
Github user JoshRosen closed the pull request at:
https://github.com/apache/spark/pull/3548
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/3548#issuecomment-65285340
I'm going to close this for now in favor of Mark's patch (#3550). There
are a couple of ideas here that might be useful for future improvements to this
code
1 - 100 of 310 matches
Mail list logo
