[GitHub] spark pull request: [GraphX] initialmessage for pagerank should be...

2015-02-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/1128#issuecomment-75435408 Is this a WontFix at this stage or needs another look from @ankurdave or @jegonzal ? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-1406] Mllib pmml model export

2015-02-22 Thread selvinsource
Github user selvinsource commented on the pull request: https://github.com/apache/spark/pull/3062#issuecomment-75429812 @jkbradley I could add that too in addition or alternative to the local file. In terms of implementation, I was thinking that the quick way of doing it

[GitHub] spark pull request: SPARK-5819 Backported the fix described in SPA...

2015-02-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4605#issuecomment-75433782 Mind closing this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-5926] [SQL] make DataFrame.explain leve...

2015-02-22 Thread yanboliang
Github user yanboliang commented on the pull request: https://github.com/apache/spark/pull/4707#issuecomment-75437798 test please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-911] allow efficient queries for a rang...

2015-02-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/1381#issuecomment-75434473 Back to @JoshRosen for a final look; it looks good to me eyes. How about another test for a case where the range begins and ends in the middle of a partition? right now

[GitHub] spark pull request: SPARK-5819 Backported the fix described in SPA...

2015-02-22 Thread emres
Github user emres closed the pull request at: https://github.com/apache/spark/pull/4605 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-4475] change localhost to 127.0.0.1...

2015-02-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/3425#issuecomment-75435434 I suggest closing this unless we can point to a clear reason to prefer `127.0.0.1`. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-911] allow efficient queries for a rang...

2015-02-22 Thread aaronjosephs
Github user aaronjosephs commented on the pull request: https://github.com/apache/spark/pull/1381#issuecomment-75443472 @srowen pushed a test with data in the middle of the partitions, retest --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-911] allow efficient queries for a rang...

2015-02-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/1381#issuecomment-75443558 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/731#issuecomment-75443738 [Test build #27836 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27836/consoleFull) for PR 731 at commit

[GitHub] spark pull request: [SPARK-4808] Removing minimum number of elemen...

2015-02-22 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4420#issuecomment-75447262 Hi all, I will revert this in branch 1.3 for now because this was technically merged after the first RC, and people may have already run performance tests on this.

[GitHub] spark pull request: [GraphX] initialmessage for pagerank should be...

2015-02-22 Thread jegonzal
Github user jegonzal commented on the pull request: https://github.com/apache/spark/pull/1128#issuecomment-75451894 Great! I agree with this proposal as well. I apologize for letting it sit so long. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-5926] [SQL] make DataFrame.explain leve...

2015-02-22 Thread yanboliang
Github user yanboliang commented on the pull request: https://github.com/apache/spark/pull/4707#issuecomment-75439264 retest please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-5724] fix the misconfiguration in AkkaU...

2015-02-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4512#discussion_r25133440 --- Diff: docs/configuration.md --- @@ -882,36 +882,24 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark pull request: [SPARK-5724] fix the misconfiguration in AkkaU...

2015-02-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4512#discussion_r25133423 --- Diff: docs/configuration.md --- @@ -882,36 +882,24 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark pull request: [SPARK-3619] Part 2. Upgrade to Mesos 0.21 to ...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4361#issuecomment-75444526 [Test build #27833 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27833/consoleFull) for PR 4361 at commit

[GitHub] spark pull request: [SPARK-3619] Part 2. Upgrade to Mesos 0.21 to ...

2015-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4361#issuecomment-75444531 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: Add a note for context termination for History...

2015-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4721#issuecomment-75452323 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-5174][SPARK-5175] provide more APIs in ...

2015-02-22 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/3984#issuecomment-75441722 ping? @tdas do you mind take a review? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-5724] fix the misconfiguration in AkkaU...

2015-02-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4512#issuecomment-75443002 @ScrapCodes sounds OK with this; I'll wait a short while for confirmation before merging. In the meantime the text can be touched up. --- If your project is set up for

[GitHub] spark pull request: [SPARK-5724] fix the misconfiguration in AkkaU...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4512#issuecomment-75444167 [Test build #27837 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27837/consoleFull) for PR 4512 at commit

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/731#issuecomment-75444168 [Test build #27838 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27838/consoleFull) for PR 731 at commit

[GitHub] spark pull request: [SPARK-3619] Part 2. Upgrade to Mesos 0.21 to ...

2015-02-22 Thread tnachen
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/4361#issuecomment-75448373 I believe spark-env.sh.template in conf folder has a comment about Mesos native library. Can you change that as well? --- If your project is set up for it, you can

[GitHub] spark pull request: [GraphX] initialmessage for pagerank should be...

2015-02-22 Thread ankurdave
Github user ankurdave commented on the pull request: https://github.com/apache/spark/pull/1128#issuecomment-75450827 I talked to @jegonzal and it seems this is the correct thing to do. I'll merge it pending Jenkins. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [GraphX] initialmessage for pagerank should be...

2015-02-22 Thread ankurdave
Github user ankurdave commented on the pull request: https://github.com/apache/spark/pull/1128#issuecomment-75450836 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [GraphX] initialmessage for pagerank should be...

2015-02-22 Thread ankurdave
Github user ankurdave commented on the pull request: https://github.com/apache/spark/pull/1128#issuecomment-75451786 Oh, this PR was against the old version of static PageRank that used Pregel, while we now use a standalone implementation. I think it still has the same problem and

[GitHub] spark pull request: [SPARK-3619] Part 2. Upgrade to Mesos 0.21 to ...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4361#issuecomment-75440593 [Test build #27833 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27833/consoleFull) for PR 4361 at commit

[GitHub] spark pull request: [SPARK-5724] fix the misconfiguration in AkkaU...

2015-02-22 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/4512#issuecomment-75441700 Hi, @ScrapCodes @srowen , this patch is under review for a while, if you think the modification is OK (just remove one line in the code essentially ), it's ready to go

[GitHub] spark pull request: [SPARK-5751] [SQL] Revamped HiveThriftServer2S...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4720#issuecomment-75447519 [Test build #27835 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27835/consoleFull) for PR 4720 at commit

[GitHub] spark pull request: Add a note for context termination for History...

2015-02-22 Thread moutai
GitHub user moutai opened a pull request: https://github.com/apache/spark/pull/4721 Add a note for context termination for History server on Yarn The history server on Yarn only shows completed jobs. This adds a note concerning the needed explicit context termination at the end of

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2015-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/731#issuecomment-75443741 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5724] fix the misconfiguration in AkkaU...

2015-02-22 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/4512#issuecomment-75443974 thanks @srowen , just fixed the docs --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-911] allow efficient queries for a rang...

2015-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1381#issuecomment-75447636 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-911] allow efficient queries for a rang...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1381#issuecomment-75447631 [Test build #27834 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27834/consoleFull) for PR 1381 at commit

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2015-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/731#issuecomment-75448693 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/731#issuecomment-75448691 [Test build #27839 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27839/consoleFull) for PR 731 at commit

[GitHub] spark pull request: [GraphX] initialmessage for pagerank should be...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1128#issuecomment-75451078 [Test build #27840 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27840/consoleFull) for PR 1128 at commit

[GitHub] spark pull request: [SPARK-5751] [SQL] Revamped HiveThriftServer2S...

2015-02-22 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/4720 [SPARK-5751] [SQL] Revamped HiveThriftServer2Suite for robustness `HiveThriftServer2Suite` has been notorious for its flakiness for a while. This was mostly due to spawning and communicate with

[GitHub] spark pull request: [SPARK-911] allow efficient queries for a rang...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1381#issuecomment-75443625 [Test build #27834 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27834/consoleFull) for PR 1381 at commit

[GitHub] spark pull request: [SPARK-5751] [SQL] Revamped HiveThriftServer2S...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4720#issuecomment-75443627 [Test build #27835 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27835/consoleFull) for PR 4720 at commit

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/731#issuecomment-75443622 [Test build #27836 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27836/consoleFull) for PR 731 at commit

[GitHub] spark pull request: [SPARK-911] allow efficient queries for a rang...

2015-02-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/1381#issuecomment-75443581 Looks good to me and I assume it's OK with Josh since it's basically the code he suggested. I'll wait a short while for more comments before merging, if Josh doesn't

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2015-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/731#issuecomment-75444259 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/731#issuecomment-75444258 [Test build #27838 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27838/consoleFull) for PR 731 at commit

[GitHub] spark pull request: [SPARK-5751] [SQL] Revamped HiveThriftServer2S...

2015-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4720#issuecomment-75447521 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5724] fix the misconfiguration in AkkaU...

2015-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4512#issuecomment-75448294 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5724] fix the misconfiguration in AkkaU...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4512#issuecomment-75448291 [Test build #27837 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27837/consoleFull) for PR 4512 at commit

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-02-22 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/3438#issuecomment-75448758 Hi @andrewor14 , I've rebased this code against the latest master, would you please help to review this code, thanks a lot. --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-5943][Streaming] Update the test to use...

2015-02-22 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/4722 [SPARK-5943][Streaming] Update the test to use new API to reduce the warning You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: [SPARK-5942][SQL] DataFrame should not do quer...

2015-02-22 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/4719 [SPARK-5942][SQL] DataFrame should not do query optimization when dataFrameEagerAnalysis is on `DataFrame` will force query optimization to happen right away for the commands and queries with side

[GitHub] spark pull request: [DOCS] Fix typo in API for custom InputFormats...

2015-02-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4718 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [DOCS] Fix typo in API for custom InputFormats...

2015-02-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4718#issuecomment-75426212 Since it fixes just two one-word typos in docs, I think this can go straight in, and to 1.3. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: SPARK-5669 [BUILD] Reverse exclusion of JBLAS ...

2015-02-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4715#issuecomment-75426630 Merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: SPARK-5669 [BUILD] Reverse exclusion of JBLAS ...

2015-02-22 Thread srowen
Github user srowen closed the pull request at: https://github.com/apache/spark/pull/4715 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-5942][SQL] DataFrame should not do quer...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4719#issuecomment-75425031 [Test build #27832 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27832/consoleFull) for PR 4719 at commit

[GitHub] spark pull request: [DataFrame] [Typo] Fix the typo

2015-02-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4717 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [DataFrame] [Typo] Fix the typo

2015-02-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4717#issuecomment-75426304 Looks like another fine tiny doc fix that can go straight in and for 1.3. I checked and there is indeed no `DataFrameImpl` anywhere in the code in master or 1.3. --- If

[GitHub] spark pull request: [SPARK-5942][SQL] DataFrame should not do quer...

2015-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4719#issuecomment-75427089 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5942][SQL] DataFrame should not do quer...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4719#issuecomment-75427085 [Test build #27832 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27832/consoleFull) for PR 4719 at commit

[GitHub] spark pull request: SPARK-4588 [MLLIB] [WIP] Add API for feature a...

2015-02-22 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/4460#issuecomment-75461553 I went to add AttributeGroup, but then I can't figure out how this isn't already covered by Attribute's dimension? It's 1 for a scalar, 1 for a vector-valued feature.

[GitHub] spark pull request: [SPARK-5722] [SQL] [PySpark] infer int as Long...

2015-02-22 Thread dondrake
Github user dondrake commented on the pull request: https://github.com/apache/spark/pull/4681#issuecomment-75469299 @davies @marmbrus Just curious what the status of this PR and JIRA are for 1.2.x? Thanks. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-5641] [EC2] Allow spark_ec2.py to copy ...

2015-02-22 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/4583#issuecomment-75470850 Yeah if we can write up documentation on how having `/` or not having '/' affects things it should be good. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: Add a note for context termination for History...

2015-02-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4721#discussion_r25135717 --- Diff: docs/monitoring.md --- @@ -150,6 +150,8 @@ follows: Note that in all of these UIs, the tables are sortable by clicking their headers,

[GitHub] spark pull request: [GraphX] initialmessage for pagerank should be...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1128#issuecomment-75454845 [Test build #27840 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27840/consoleFull) for PR 1128 at commit

[GitHub] spark pull request: [GraphX] initialmessage for pagerank should be...

2015-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1128#issuecomment-75454846 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-4588 [MLLIB] [WIP] Add API for feature a...

2015-02-22 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/4460#issuecomment-75458264 To me, that means that the idea of a string-valued categorical column still has a place in the representation since it exists at some stage of a pipeline. It's just

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2015-02-22 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/731#issuecomment-75466570 ![image](https://cloud.githubusercontent.com/assets/678008/6320817/b3d4efb2-bab6-11e4-9615-1530d981e286.png)

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/731#issuecomment-75467548 [Test build #27842 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27842/consoleFull) for PR 731 at commit

[GitHub] spark pull request: [SPARK-5095][MESOS] Support capping cores and ...

2015-02-22 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/4027#discussion_r25138364 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala --- @@ -63,20 +64,25 @@ private[spark] class

[GitHub] spark pull request: [MLLIB] SPARK-5912 Programming guide for featu...

2015-02-22 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/4709#issuecomment-75456675 @avulanov Thanks for the updates! Except for those 2 issues, I think this should be ready to go. (I'm testing doc compilation now.) --- If your project is set up

[GitHub] spark pull request: [SPARK-5095][MESOS] Support capping cores and ...

2015-02-22 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/4027#discussion_r25138542 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala --- @@ -204,35 +209,43 @@ private[spark] class

[GitHub] spark pull request: [SPARK-5943][Streaming] Update the test to use...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4722#issuecomment-75453670 [Test build #27841 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27841/consoleFull) for PR 4722 at commit

[GitHub] spark pull request: [MLLIB] SPARK-5912 Programming guide for featu...

2015-02-22 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/4709#discussion_r25136231 --- Diff: docs/mllib-feature-extraction.md --- @@ -375,3 +375,52 @@ data2 = labels.zip(normalizer2.transform(features)) {% endhighlight %} /div

[GitHub] spark pull request: [MLLIB] SPARK-5912 Programming guide for featu...

2015-02-22 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/4709#discussion_r25136229 --- Diff: docs/mllib-feature-extraction.md --- @@ -375,3 +375,52 @@ data2 = labels.zip(normalizer2.transform(features)) {% endhighlight %} /div

[GitHub] spark pull request: [SPARK-5943][Streaming] Update the test to use...

2015-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4722#issuecomment-75457870 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5943][Streaming] Update the test to use...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4722#issuecomment-75457867 [Test build #27841 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27841/consoleFull) for PR 4722 at commit

[GitHub] spark pull request: [MLLIB] SPARK-5912 Programming guide for featu...

2015-02-22 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/4709#issuecomment-75462021 The generated doc seems Ok except for the comments above. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2015-02-22 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/731#issuecomment-75468070 Hi, @andrewor14 , thanks for your review I think I misunderstood your original point. Now, I merged two functions (starting one/more executor(s)) .

[GitHub] spark pull request: [SPARK-5095][MESOS] Support capping cores and ...

2015-02-22 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/4027#discussion_r25138507 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala --- @@ -63,20 +63,25 @@ private[spark] class

[GitHub] spark pull request: [SPARK-5155] [PySpark] [Streaming] Mqtt stream...

2015-02-22 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/4229#issuecomment-75490529 Will do as soon as I get some Spark 1.3 release stuff out of the way. On Fri, Feb 20, 2015 at 10:02 AM, Prabeesh K notificati...@github.com wrote:

[GitHub] spark pull request: [SPARK-2033] Automatically cleanup checkpoint

2015-02-22 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/855#discussion_r25144332 --- Diff: core/src/test/scala/org/apache/spark/ContextCleanerSuite.scala --- @@ -206,6 +206,38 @@ class ContextCleanerSuite extends ContextCleanerSuiteBase {

[GitHub] spark pull request: [SPARK-5641] [EC2] Allow spark_ec2.py to copy ...

2015-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4583#issuecomment-75491462 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5641] [EC2] Allow spark_ec2.py to copy ...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4583#issuecomment-75491457 [Test build #27846 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27846/consoleFull) for PR 4583 at commit

[GitHub] spark pull request: [SPARK-4655] Split Stage into ShuffleMapStage ...

2015-02-22 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4708#issuecomment-75492348 Hmm, it looks like MiMa is complaining about the `Stage` class becoming abstract even though it's `private[spark]`: ``` [info] spark-core: found 1

[GitHub] spark pull request: [SPARK-4655] Split Stage into ShuffleMapStage ...

2015-02-22 Thread ilganeli
Github user ilganeli commented on the pull request: https://github.com/apache/spark/pull/4708#issuecomment-75493744 I've added an exclude for the Spark 1.3 build - hopefully that resolves this issue. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-4924] Add a library for launching Spark...

2015-02-22 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/3916#discussion_r25145503 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java --- @@ -0,0 +1,684 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-4655] Split Stage into ShuffleMapStage ...

2015-02-22 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4708#discussion_r25145518 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -830,39 +836,39 @@ class DAGScheduler( try { // For

[GitHub] spark pull request: [SPARK-4655] Split Stage into ShuffleMapStage ...

2015-02-22 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4708#discussion_r25145581 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -996,50 +1006,52 @@ class DAGScheduler( if

[GitHub] spark pull request: [SPARK-4655] Split Stage into ShuffleMapStage ...

2015-02-22 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4708#discussion_r25145702 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ResultStage.scala --- @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-4655] Split Stage into ShuffleMapStage ...

2015-02-22 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4708#discussion_r25145735 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ResultStage.scala --- @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-5946][Streaming] Add Python API for dir...

2015-02-22 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/4723#issuecomment-75495853 @davies It is a the new experimental API for Kafka, which does not use Receivers at all, and rather treats Kafka as a file system and topics as files. It finds new data in

[GitHub] spark pull request: [SPARK-5946][Streaming] Add Python API for dir...

2015-02-22 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/4723#issuecomment-75496536 @tdas Cool, thanks. Do it mean that it does NOT need WAL to have high durability? This patch looks good to me, just need more docs (somewhere) to tell the

[GitHub] spark pull request: [SPARK-5559] [Streaming] [Test] Remove oppotun...

2015-02-22 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4337#issuecomment-75497641 @sarutak If you've got time, mind addressing Sean's minor comments? Other than those, this LGTM. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-4924] Add a library for launching Spark...

2015-02-22 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3916#issuecomment-75498418 Hey @vanzin okay took another quite long look (sorry, was delayed this week due to strata) and I have embarrassingly few useful comments given how long I looked at it.

[GitHub] spark pull request: [SPARK-4924] Add a library for launching Spark...

2015-02-22 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/3916#discussion_r25147958 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkClassCommandBuilder.java --- @@ -0,0 +1,155 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-5724] fix the misconfiguration in AkkaU...

2015-02-22 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/4512#issuecomment-75489848 Looks good !, haven't tested it though. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-5174][SPARK-5175] provide more APIs in ...

2015-02-22 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/3984#issuecomment-75490230 Will do as soon as I finish some stuff regarding the Spark 1.3 release. :) On Sun, Feb 22, 2015 at 7:55 AM, Nan Zhu notificati...@github.com wrote: ping?

[GitHub] spark pull request: [SPARK-5946][Streaming] Add Python API for dir...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4723#issuecomment-75491038 [Test build #27848 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27848/consoleFull) for PR 4723 at commit

[GitHub] spark pull request: [SPARK-5946][Streaming] Add Python API for dir...

2015-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4723#issuecomment-75491095 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5946][Streaming] Add Python API for dir...

2015-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4723#issuecomment-75491093 [Test build #27848 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27848/consoleFull) for PR 4723 at commit

  1   2   >