[GitHub] spark pull request: [SPARK-3632] ConnectionManager can run out of ...

2014-10-03 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2484#issuecomment-57763330 Unfortunately it doesn't merge. How large of a cluster does we need to get this issue to happen? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-3166]: Allow custom serialiser to be sh...

2014-10-03 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1890#issuecomment-57763605 Yes we will take a look at this for 1.2. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-3713][SQL] Uses JSON to serialize DataT...

2014-10-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/2563#discussion_r18383911 --- Diff: python/pyspark/sql.py --- @@ -62,6 +67,17 @@ def __eq__(self, other): def __ne__(self, other): return not self.__eq__(other)

[GitHub] spark pull request: [SPARK-3713][SQL] Uses JSON to serialize DataT...

2014-10-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/2563#discussion_r18384024 --- Diff: python/pyspark/sql.py --- @@ -312,42 +343,24 @@ def __repr__(self): return ("StructType(List(%s))" % ",".join(str(fi

[GitHub] spark pull request: [SPARK-3713][SQL] Uses JSON to serialize DataT...

2014-10-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/2563#discussion_r18384157 --- Diff: python/pyspark/sql.py --- @@ -385,51 +398,35 @@ def _parse_datatype_string(datatype_string): >>> check_datatype(complex_maptype) Tru

[GitHub] spark pull request: [SPARK-3775] Not suitable error message in spa...

2014-10-03 Thread tsudukim
GitHub user tsudukim opened a pull request: https://github.com/apache/spark/pull/2640 [SPARK-3775] Not suitable error message in spark-shell.cmd Modified some sentence of error message in bin\*.cmd. You can merge this pull request into a Git repository by running: $ git pull ht

[GitHub] spark pull request: [SPARK-3775] Not suitable error message in spa...

2014-10-03 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2640#issuecomment-57771950 I don't agree that users should be directed to build with SBT. Maven is the default, but, probably best to not even make a specific recommendation. --- If your project i

[GitHub] spark pull request: [SPARK-3775] Not suitable error message in spa...

2014-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2640#issuecomment-57772040 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-3775] Not suitable error message in spa...

2014-10-03 Thread tsudukim
Github user tsudukim commented on the pull request: https://github.com/apache/spark/pull/2640#issuecomment-57772760 Yes, so I removed the specific recommendation for SBT. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-3366][MLLIB]Compute best splits distrib...

2014-10-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2595 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-3366][MLLIB]Compute best splits distrib...

2014-10-03 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/2595#issuecomment-57778527 LGTM. Merged into master. Thanks @chouqin , and @jkbradley and @manishamde for code review! Increasing `maxMemoryInMB` also increases the shuffle size. As long as

[GitHub] spark pull request: [SQL] SPARK-3776: Wrong conversion to Catalyst...

2014-10-03 Thread r3natko
GitHub user r3natko opened a pull request: https://github.com/apache/spark/pull/2641 [SQL] SPARK-3776: Wrong conversion to Catalyst for Option[Product] You can merge this pull request into a Git repository by running: $ git pull https://github.com/r3natko/spark feature/catalys

[GitHub] spark pull request: [SQL] SPARK-3776: Wrong conversion to Catalyst...

2014-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2641#issuecomment-57783977 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: SPARK-3770: Make userFeatures accessible from ...

2014-10-03 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/2636#issuecomment-57785126 @mdagost If you convert `(Int, Array[Double])` to a `java.util.List` (id the first and features the second (without converting to string)), you should be able to get the d

[GitHub] spark pull request: SPARK-2058: Overriding SPARK_HOME/conf with SP...

2014-10-03 Thread EugenCepoi
Github user EugenCepoi commented on the pull request: https://github.com/apache/spark/pull/2481#issuecomment-57786084 @andrewor14 The trailing % came from a bad copy paste, nice catch :) Double percent exists but was wrong in this case, http://stackoverflow.com/questions/14509652/what

[GitHub] spark pull request: SPARK-2058: Overriding SPARK_HOME/conf with SP...

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2481#issuecomment-57786537 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21243/consoleFull) for PR 2481 at commit [`0bb32c2`](https://github.com/ap

[GitHub] spark pull request: [SPARK-2750] support https in spark web ui

2014-10-03 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/1980#issuecomment-57787121 Hi, @JoshRosen, refer to http://wiki.eclipse.org/Jetty/Howto/Configure_SSL you can generating keys and certificates by openssl or keytool 1 Generating Keys and Certif

[GitHub] spark pull request: [SPARK-2750] support https in spark web ui

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1980#issuecomment-57789135 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21244/consoleFull) for PR 1980 at commit [`e5c87cb`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3677] [BUILD] [YARN] pom.xml and SparkB...

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2520#issuecomment-57790931 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21245/consoleFull) for PR 2520 at commit [`6a2bfaf`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3777] Display "Executor ID" for Tasks i...

2014-10-03 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/2642 [SPARK-3777] Display "Executor ID" for Tasks in Stage page Now the Stage page only displays "Executor"(host) for tasks. However, there may be more than one Executors running in the same host. Curren

[GitHub] spark pull request: [SPARK-3771][SQL] AppendingParquetOutputFormat...

2014-10-03 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2638#issuecomment-57791903 A particular instance of Spark will be built for a particular version of Hadoop and/or YARN. It is not at this point a universal binary anyway, and so, I do not think it i

[GitHub] spark pull request: [SPARK-3777] Display "Executor ID" for Tasks i...

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2642#issuecomment-57792294 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21246/consoleFull) for PR 2642 at commit [`4bbe2c7`](https://github.com/ap

[GitHub] spark pull request: Rectify gereneric parameter names between Spar...

2014-10-03 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2637#issuecomment-57792726 +1, makes more sense for sure. Open a JIRA for this as well? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-3632] ConnectionManager can run out of ...

2014-10-03 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/2484#issuecomment-57794438 I've seen it with 200 executors. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does n

[GitHub] spark pull request: SPARK-2058: Overriding SPARK_HOME/conf with SP...

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2481#issuecomment-57795374 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21243/consoleFull) for PR 2481 at commit [`0bb32c2`](https://github.com/a

[GitHub] spark pull request: SPARK-2058: Overriding SPARK_HOME/conf with SP...

2014-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2481#issuecomment-57795383 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: [SPARK-2750] support https in spark web ui

2014-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1980#issuecomment-57796385 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: [SPARK-2750] support https in spark web ui

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1980#issuecomment-57796375 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21244/consoleFull) for PR 1980 at commit [`e5c87cb`](https://github.com/a

[GitHub] spark pull request: [SPARK-3677] [BUILD] [YARN] pom.xml and SparkB...

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2520#issuecomment-57797668 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21245/consoleFull) for PR 2520 at commit [`6a2bfaf`](https://github.com/a

[GitHub] spark pull request: [SPARK-3677] [BUILD] [YARN] pom.xml and SparkB...

2014-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2520#issuecomment-57797676 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: [SPARK-3627] - [yarn] - fix exit code and fina...

2014-10-03 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/2577#discussion_r18395371 --- Diff: yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -328,10 +348,18 @@ private[spark] class ApplicationMaster(

[GitHub] spark pull request: [SPARK-3627] - [yarn] - fix exit code and fina...

2014-10-03 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/2577#discussion_r18395494 --- Diff: yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -328,10 +348,18 @@ private[spark] class ApplicationMaster(

[GitHub] spark pull request: [SPARK-3777] Display "Executor ID" for Tasks i...

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2642#issuecomment-57799611 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21246/consoleFull) for PR 2642 at commit [`4bbe2c7`](https://github.com/a

[GitHub] spark pull request: [SPARK-3777] Display "Executor ID" for Tasks i...

2014-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2642#issuecomment-57799622 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: [SPARK-3771][SQL] AppendingParquetOutputFormat...

2014-10-03 Thread ueshin
Github user ueshin commented on the pull request: https://github.com/apache/spark/pull/2638#issuecomment-57800537 @srowen, Thank you for your comment. Indeed, when deploy completed apps to Spark cluster, there is a particular instance of Spark. But Spark app developers will use

[GitHub] spark pull request: [SPARK-3696]Do not override the user-difined c...

2014-10-03 Thread WangTaoTheTonic
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/2541#issuecomment-57803237 @JoshRosen I have tested and it worked fine. You can also have a try simply. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-3696]Do not override the user-difined c...

2014-10-03 Thread WangTaoTheTonic
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/2541#issuecomment-57803284 @JoshRosen I have tested and it worked fine. You can also have a try simply. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: SPARK-3711: Optimize where in clause filter qu...

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2561#issuecomment-57803554 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21247/consoleFull) for PR 2561 at commit [`afedbcd`](https://github.com/ap

[GitHub] spark pull request: SPARK-3711: Optimize where in clause filter qu...

2014-10-03 Thread saucam
Github user saucam commented on the pull request: https://github.com/apache/spark/pull/2561#issuecomment-57803869 1. removed unnecessary UnaryMinus case as it is constant-folded away 2. Added unit test in ExpressionEvaluationSuite 3. Added a class OptimizedInSuite which tests th

[GitHub] spark pull request: [SPARK-732][SPARK-3628][CORE][RESUBMIT] make i...

2014-10-03 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/2524#issuecomment-57806876 ping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-3535][Mesos] Fix resource handling.

2014-10-03 Thread brndnmtthws
Github user brndnmtthws commented on the pull request: https://github.com/apache/spark/pull/2401#issuecomment-57806924 Fixed typo's (also switch from `Math.max` to `math.max` because that seems to be the Scala way). --- If your project is set up for it, you can reply to this email an

[GitHub] spark pull request: [SPARK-3627] - [yarn] - fix exit code and fina...

2014-10-03 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/2577#issuecomment-57807019 Addressed all the review comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: code style format

2014-10-03 Thread shijinkui
GitHub user shijinkui opened a pull request: https://github.com/apache/spark/pull/2643 code style format code style format You can merge this pull request into a Git repository by running: $ git pull https://github.com/shijinkui/spark master Alternatively you can review and ap

[GitHub] spark pull request: [SPARK-3535][Mesos] Fix resource handling.

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2401#issuecomment-57807681 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21249/consoleFull) for PR 2401 at commit [`54a4a06`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3627] - [yarn] - fix exit code and fina...

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2577#issuecomment-57807655 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21248/consoleFull) for PR 2577 at commit [`24c98e3`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3713][SQL] Uses JSON to serialize DataT...

2014-10-03 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2563#discussion_r18399479 --- Diff: python/pyspark/sql.py --- @@ -312,42 +343,24 @@ def __repr__(self): return ("StructType(List(%s))" % ",".join(str

[GitHub] spark pull request: code style format

2014-10-03 Thread shijinkui
Github user shijinkui closed the pull request at: https://github.com/apache/spark/pull/2643 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[GitHub] spark pull request: [SPARK-3627] - [yarn] - fix exit code and fina...

2014-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2577#issuecomment-57808155 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: [SPARK-3627] - [yarn] - fix exit code and fina...

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2577#issuecomment-57808148 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21248/consoleFull) for PR 2577 at commit [`24c98e3`](https://github.com/a

[GitHub] spark pull request: code style format

2014-10-03 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2643#issuecomment-57808284 Some changes here make lines longer than 100 characters, which contradicts the scalastyle rules for the project. I am not sure that moving comments around to a previous li

[GitHub] spark pull request: [SPARK-3775] Not suitable error message in spa...

2014-10-03 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2640#issuecomment-57808436 Oh I'm sorry, my poor brain read the diff the wrong way around. I agree with the change! --- If your project is set up for it, you can reply to this email and have your r

[GitHub] spark pull request: [SPARK-3781] code Style format

2014-10-03 Thread shijinkui
GitHub user shijinkui opened a pull request: https://github.com/apache/spark/pull/2644 [SPARK-3781] code Style format You can merge this pull request into a Git repository by running: $ git pull https://github.com/shijinkui/spark styleFormat Alternatively you can review and a

[GitHub] spark pull request: [SPARK-3627] - [yarn] - fix exit code and fina...

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2577#issuecomment-57809815 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21250/consoleFull) for PR 2577 at commit [`e8cc261`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3781] code Style format

2014-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2644#issuecomment-57810182 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: SPARK-3770: Make userFeatures accessible from ...

2014-10-03 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/2636#issuecomment-57813143 @mdagost @mengxr We use Pyrolite to convert Java objects into Python objects, you can get the type mapping here: https://github.com/irmen/Pyrolite --- If your project is

[GitHub] spark pull request: [SPARK-3535][Mesos] Fix resource handling.

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2401#issuecomment-57814712 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21249/consoleFull) for PR 2401 at commit [`54a4a06`](https://github.com/a

[GitHub] spark pull request: [SPARK-3535][Mesos] Fix resource handling.

2014-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2401#issuecomment-57814725 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: SPARK-3711: Optimize where in clause filter qu...

2014-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2561#issuecomment-57814819 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: [SPARK-3535][Mesos] Fix resource handling.

2014-10-03 Thread brndnmtthws
Github user brndnmtthws commented on the pull request: https://github.com/apache/spark/pull/2401#issuecomment-57814884 That test failure appears to be unrelated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: SPARK-3711: Optimize where in clause filter qu...

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2561#issuecomment-57814811 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21247/consoleFull) for PR 2561 at commit [`afedbcd`](https://github.com/a

[GitHub] spark pull request: [SPARK-2377] Python API for Streaming

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2538#issuecomment-57815905 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21251/consoleFull) for PR 2538 at commit [`52c535b`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3627] - [yarn] - fix exit code and fina...

2014-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2577#issuecomment-57818940 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: [SPARK-3627] - [yarn] - fix exit code and fina...

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2577#issuecomment-57818934 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21250/consoleFull) for PR 2577 at commit [`e8cc261`](https://github.com/a

[GitHub] spark pull request: [SPARK-2693][SQL] Supported for UDAF Hive Aggr...

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2620#issuecomment-57819162 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21252/consoleFull) for PR 2620 at commit [`a8df326`](https://github.com/ap

[GitHub] spark pull request: [SPARK-2693][SQL] Supported for UDAF Hive Aggr...

2014-10-03 Thread ravipesala
Github user ravipesala commented on the pull request: https://github.com/apache/spark/pull/2620#issuecomment-57819925 You are right, it is not so good to pass resolver in constructor. Instead I just passed boolean flag. --- If your project is set up for it, you can reply to this emai

[GitHub] spark pull request: [SPARK-2778] [yarn] Add workaround for race in...

2014-10-03 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/2605#issuecomment-57820191 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2778] [yarn] Add workaround for race in...

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2605#issuecomment-57821047 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21253/consoleFull) for PR 2605 at commit [`8d02ce0`](https://github.com/ap

[GitHub] spark pull request: Rectify gereneric parameter names between Spar...

2014-10-03 Thread nkronenfeld
Github user nkronenfeld commented on the pull request: https://github.com/apache/spark/pull/2637#issuecomment-57821312 https://issues.apache.org/jira/browse/SPARK-3783 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If yo

[GitHub] spark pull request: [SPARK-3535][Mesos] Fix resource handling.

2014-10-03 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/2401#issuecomment-57822162 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this f

[GitHub] spark pull request: [SPARK-3535][Mesos] Fix resource handling.

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2401#issuecomment-57822909 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21254/consoleFull) for PR 2401 at commit [`54a4a06`](https://github.com/ap

[GitHub] spark pull request: SPARK-2058: Overriding SPARK_HOME/conf with SP...

2014-10-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2481#discussion_r18406038 --- Diff: bin/compute-classpath.sh --- @@ -27,8 +27,14 @@ FWDIR="$(cd "`dirname "$0"`"/..; pwd)" . "$FWDIR"/bin/load-spark-env.sh +CLAS

[GitHub] spark pull request: SPARK-2058: Overriding SPARK_HOME/conf with SP...

2014-10-03 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/2481#issuecomment-57823932 LGTM. I'm merging this into master and 1.1. Thanks @EugenCepoi! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: SPARK-2058: Overriding SPARK_HOME/conf with SP...

2014-10-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2481 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-1655][MLLIB] Add option for distributed...

2014-10-03 Thread staple
Github user staple commented on the pull request: https://github.com/apache/spark/pull/2491#issuecomment-57824188 @mengxr - I’m sure you have a lot on your plate right now, but I wanted to check in on this PR. Overall, how do you feel about this approach for SPARK-1655? I’m happy

[GitHub] spark pull request: SPARK-2058: Overriding SPARK_HOME/conf with SP...

2014-10-03 Thread EugenCepoi
Github user EugenCepoi commented on the pull request: https://github.com/apache/spark/pull/2481#issuecomment-57824300 @andrewor14 great! Glad to see it merged into master :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-3765][Doc] add testing with sbt to docs

2014-10-03 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2629#issuecomment-57824840 This looks good to me; thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3696]Do not override the user-difined c...

2014-10-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2541#discussion_r18407387 --- Diff: sbin/spark-config.sh --- @@ -33,7 +33,7 @@ this="$config_bin/$script" export SPARK_PREFIX="`dirname "$this"`"/.. export SPARK_HOME

[GitHub] spark pull request: SPARK-2058: Overriding SPARK_HOME/conf with SP...

2014-10-03 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/2481#issuecomment-57826573 Ah yes I believe that's for the Spark daemons. We can merge that separately. --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark pull request: [SPARK-3535][Mesos] Fix resource handling.

2014-10-03 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/2401#issuecomment-57826312 Hey @brndnmtthws I left a few more minor comments but I think this is basically ready for merge. Thanks for your changes. --- If your project is set up for it, you ca

[GitHub] spark pull request: [SPARK-1860] More conservative app directory c...

2014-10-03 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/2609#discussion_r18407971 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala --- @@ -191,6 +194,8 @@ private[spark] class Worker( changeMaster(masterU

[GitHub] spark pull request: [SPARK-3696]Do not override the user-difined c...

2014-10-03 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/2541#issuecomment-57828829 Ok, I'm merging this into master and 1.1. Thanks @WangTaoTheTonic! --- If your project is set up for it, you can reply to this email and have your reply appear on GitH

[GitHub] spark pull request: [SPARK-3696]Do not override the user-difined c...

2014-10-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2541 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-2778] [yarn] Add workaround for race in...

2014-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2605#issuecomment-57829432 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: [SPARK-2778] [yarn] Add workaround for race in...

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2605#issuecomment-57829426 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21253/consoleFull) for PR 2605 at commit [`8d02ce0`](https://github.com/a

[GitHub] spark pull request: [SPARK-3535][Mesos] Fix resource handling.

2014-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2401#issuecomment-57830054 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21

[GitHub] spark pull request: [SPARK-2778] [yarn] Add workaround for race in...

2014-10-03 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/2605#issuecomment-57830437 I see. In your experience how long does it take for the port to change (is it usually instant)? Is there a relevant Yarn JIRA that describes the issue? If so we could

[GitHub] spark pull request: [SPARK-2778] [yarn] Add workaround for race in...

2014-10-03 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/2605#issuecomment-57830851 It's generally very quick (so quick that I've never seen this failure locally, although I do see the log message I mention in the code comments with port 0). I was thinkin

[GitHub] spark pull request: [SPARK-1860] More conservative app directory c...

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2609#issuecomment-57831124 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21255/consoleFull) for PR 2609 at commit [`87b5d03`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3535][Mesos] Fix resource handling.

2014-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2401#issuecomment-57831156 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/2

[GitHub] spark pull request: [SPARK-3606] [yarn] Correctly configure AmIpFi...

2014-10-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2469#discussion_r18409649 --- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala --- @@ -34,7 +34,7 @@ import org.json4s.JValue import org.json4s.jackson.JsonMethods

[GitHub] spark pull request: [SPARK-3606] [yarn] Correctly configure AmIpFi...

2014-10-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2469#discussion_r18410060 --- Diff: yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/YarnRMClientImpl.scala --- @@ -69,7 +74,28 @@ private class YarnRMClientImpl(args: Appl

[GitHub] spark pull request: [SPARK-3606] [yarn] Correctly configure AmIpFi...

2014-10-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/2469#discussion_r18410229 --- Diff: yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/YarnRMClientImpl.scala --- @@ -69,7 +74,28 @@ private class YarnRMClientImpl(args: Applicat

[GitHub] spark pull request: [SPARK-3212][SQL] Use logical plan matching in...

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2501#issuecomment-57833194 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21257/consoleFull) for PR 2501 at commit [`63fbc2c`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3535][Mesos] Fix resource handling.

2014-10-03 Thread brndnmtthws
Github user brndnmtthws commented on the pull request: https://github.com/apache/spark/pull/2401#issuecomment-57833282 Done as per @andrewor14's suggestions. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-3606] [yarn] Correctly configure AmIpFi...

2014-10-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2469#discussion_r18410472 --- Diff: yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/YarnRMClientImpl.scala --- @@ -69,7 +74,28 @@ private class YarnRMClientImpl(args: Appl

[GitHub] spark pull request: [SPARK-3606] [yarn] Correctly configure AmIpFi...

2014-10-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2469#discussion_r18410539 --- Diff: yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/YarnRMClientImpl.scala --- @@ -69,7 +74,28 @@ private class YarnRMClientImpl(args: Appl

[GitHub] spark pull request: [SPARK-3606] [yarn] Correctly configure AmIpFi...

2014-10-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/2469#discussion_r18410712 --- Diff: yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/YarnRMClientImpl.scala --- @@ -69,7 +74,28 @@ private class YarnRMClientImpl(args: Applicat

[GitHub] spark pull request: [SPARK-3606] [yarn] Correctly configure AmIpFi...

2014-10-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2469#discussion_r18410729 --- Diff: yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/YarnRMClientImpl.scala --- @@ -69,7 +74,28 @@ private class YarnRMClientImpl(args: Appl

[GitHub] spark pull request: [SPARK-3535][Mesos] Fix resource handling.

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2401#issuecomment-57833806 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21258/consoleFull) for PR 2401 at commit [`4abaa5d`](https://github.com/ap

[GitHub] spark pull request: [MLLIB] [WIP] SPARK-1547: Adding Gradient Boos...

2014-10-03 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/2607#issuecomment-57834256 @epahomov If you or your student are able to take a look at this, I'm sure @manishamde would appreciate it. This PR will hopefully be generalized to include Classific

  1   2   3   4   >