[GitHub] spark pull request: [SPARK-5598][MLLIB] model save/load for ALS

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4422#issuecomment-73202406 [Test build #26907 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26907/consoleFull) for PR 4422 at commit

[GitHub] spark pull request: [SPARK-5648][SQL] support alter ... unset tbl...

2015-02-06 Thread DoingDone9
Github user DoingDone9 closed the pull request at: https://github.com/apache/spark/pull/4423 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-5264][SQL] Support `drop temporary tabl...

2015-02-06 Thread OopsOutOfMemory
Github user OopsOutOfMemory commented on the pull request: https://github.com/apache/spark/pull/4060#issuecomment-73202418 yeah, I use a single logical operator here. If handle the type of the table and then delegate. Is that ok to do like below? we need catch exceptions in

[GitHub] spark pull request: [SPARK-5454] [SQL] Fix duplicated exprId for s...

2015-02-06 Thread pierre-borckmans
Github user pierre-borckmans commented on the pull request: https://github.com/apache/spark/pull/4284#issuecomment-73201804 @chenghao-intel No worries, thanks for the update! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-5598][MLLIB] model save/load for ALS

2015-02-06 Thread mengxr
GitHub user mengxr opened a pull request: https://github.com/apache/spark/pull/4422 [SPARK-5598][MLLIB] model save/load for ALS following #4233. @jkbradley You can merge this pull request into a Git repository by running: $ git pull https://github.com/mengxr/spark SPARK-5598

[GitHub] spark pull request: [SPARK-5648][SQL] support alter ... unset tbl...

2015-02-06 Thread DoingDone9
GitHub user DoingDone9 opened a pull request: https://github.com/apache/spark/pull/4423 [SPARK-5648][SQL] support alter ... unset tblproperties(key) make hivecontext support alter ... unset tblproperties(key) like : alter view viewName unset tblproperties(k) alter table

[GitHub] spark pull request: [SPARK-5324][SQL] Results of describe can't be...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4249#issuecomment-73202635 [Test build #26905 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26905/consoleFull) for PR 4249 at commit

[GitHub] spark pull request: [SPARK-5324][SQL] Results of describe can't be...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4249#issuecomment-73202638 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: Spark-2789: Apply names to RDD to create DataF...

2015-02-06 Thread dwmclary
GitHub user dwmclary opened a pull request: https://github.com/apache/spark/pull/4421 Spark-2789: Apply names to RDD to create DataFrame This seemed like a reasonably useful function to add to SparkSQL. However, unlike the [JIRA](https://issues.apache.org/jira/browse/SPARK-2789),

[GitHub] spark pull request: Spark-2789: Apply names to RDD to create DataF...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4421#issuecomment-73201347 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-5648][SQL] support alter ... unset tbl...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4423#issuecomment-73202332 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-5593][Core]Replace BlockManagerListener...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4369#issuecomment-73291233 Thanks merged into master 1.3 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Build] Set all Debian package permissions to ...

2015-02-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4277#issuecomment-73291305 I've also merged this into `branch-1.3` (1.3.0). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-5366][EC2] Check the mode of private ke...

2015-02-06 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/4162#discussion_r24262082 --- Diff: ec2/spark_ec2.py --- @@ -1003,6 +1006,18 @@ def real_main(): DeprecationWarning ) +if

[GitHub] spark pull request: [SPARK-5652][Mllib] Use broadcasted weights in...

2015-02-06 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/4429#issuecomment-73296580 Merged into master and branch-1.3. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-5656] Fail gracefully for large values ...

2015-02-06 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/4433#issuecomment-73296645 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-5601][MLLIB] make streaming linear algo...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4432#issuecomment-73298105 [Test build #26922 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26922/consoleFull) for PR 4432 at commit

[GitHub] spark pull request: [SPARK-5601][MLLIB] make streaming linear algo...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4432#issuecomment-73298114 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4964][Streaming][Kafka] More updates to...

2015-02-06 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4384#discussion_r24267609 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala --- @@ -179,121 +182,190 @@ object KafkaUtils { errs =

[GitHub] spark pull request: [SPARK-5656] Fail gracefully for large values ...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4433#issuecomment-73302682 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5628] Add version option to spark-ec2

2015-02-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4414#issuecomment-73303739 LGTM. I'm going to pull this into `master` (1.4.0) and `branch-1.3` (1.3.0). I'll also commit it to `branch-1.2` (1.2.2), but for that I'll update the version number

[GitHub] spark pull request: [SPARK-5650][SQL] Support optional 'FROM' clau...

2015-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4426 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-5650][SQL] Support optional 'FROM' clau...

2015-02-06 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4426#issuecomment-73304622 Thanks! Merged to master and 1.3 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-4983]insert waiting time before tagging...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3986#issuecomment-73305908 [Test build #26927 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26927/consoleFull) for PR 3986 at commit

[GitHub] spark pull request: [SPARK-4905][STREAMING] FlumeStreamSuite fix.

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4371#issuecomment-73305923 [Test build #26937 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26937/consoleFull) for PR 4371 at commit

[GitHub] spark pull request: [SPARK-4983]insert waiting time before tagging...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3986#issuecomment-73305913 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5324][SQL] Results of describe can't be...

2015-02-06 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4249#issuecomment-73307633 Thanks! Merged to master and 1.3 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-5345][DEPLOY] Fix unstable test case in...

2015-02-06 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/4133#issuecomment-73290752 This PR cannot be pushed as is, since it makes the test test the buggy sort behavior instead of correcting it. Still, if after #4220 these tests still fail, there's

[GitHub] spark pull request: [SPARK-5653][YARN] In ApplicationMaster rename...

2015-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4430 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-5470][Core]use defaultClassLoader to lo...

2015-02-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4258#issuecomment-73291095 I've cherry-picked this into `branch-1.3` (1.3.0) as well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-5396] Syntax error in spark scripts on ...

2015-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4428 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-4983]insert waiting time before tagging...

2015-02-06 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/3986#discussion_r24262711 --- Diff: ec2/spark_ec2.py --- @@ -569,6 +569,9 @@ def launch_cluster(conn, opts, cluster_name): master_nodes = master_res.instances

[GitHub] spark pull request: [SPARK-5619][SQL] Support 'show roles' in Hive...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4397#issuecomment-73295369 ok to test @liancheng --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-5555] Enable UISeleniumSuite tests

2015-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4334 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-5611] [EC2] Allow spark-ec2 repo and br...

2015-02-06 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/4385#discussion_r24264051 --- Diff: ec2/spark_ec2.py --- @@ -1007,6 +1023,11 @@ def real_main(): print stderr, ebs-vol-num cannot be greater than 8

[GitHub] spark pull request: [SPARK-5598][MLLIB] model save/load for ALS

2015-02-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4422#discussion_r24266304 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -136,3 +147,69 @@ class MatrixFactorizationModel(

[GitHub] spark pull request: SPARK-4267 [CORE] Failing to launch jobs on Sp...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4188#issuecomment-73299916 Hey @srowen it seems like this will break existing behavior though. What if I want to run an application with the following arguments ``` a b c d ```

[GitHub] spark pull request: function hasShutdownDeleteTachyonDir should us...

2015-02-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4418#issuecomment-73301472 I think we should create a JIRA for this, if only to help us keep track of where this change is committed. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [Spark-3490] Disable SparkUI for tests (backpo...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3959#issuecomment-73301377 yes, the tests are still not passing I believe, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: SPARK-5613: Catch the ApplicationNotFoundExcep...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4392#issuecomment-73301321 Hey @kasjain can you open this against the master branch next time? It will be easier for us to back port stuff from there --- If your project is set up for it, you

[GitHub] spark pull request: SPARK-4705:Creating different log directories ...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4311#issuecomment-73303253 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-4705:Creating different log directories ...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4311#issuecomment-73303249 [Test build #26935 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26935/consoleFull) for PR 4311 at commit

[GitHub] spark pull request: [SPARK-4874] [CORE] Collect record count metri...

2015-02-06 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/4067#issuecomment-73303194 Jenkins, test this please. This LGTM pending tests. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-5628] Add version option to spark-ec2

2015-02-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4414#issuecomment-73304352 Actually, I'm going to hold off on the `branch-1.2` (1.2.2) commit for now, since there's a bit of divergence in that branch and I don't want to break anything. ---

[GitHub] spark pull request: [SPARK-5366][EC2] Check the mode of private ke...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4162#issuecomment-73304375 [Test build #26924 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26924/consoleFull) for PR 4162 at commit

[GitHub] spark pull request: [SPARK-5628] Add version option to spark-ec2

2015-02-06 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/4414#issuecomment-73307181 Thanks for the review @JoshRosen. I'll tag the JIRA issue for backport. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-5651][SQL] Support 'create db.table' in...

2015-02-06 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4427#issuecomment-73307227 I don't think this is valid. You use backticks to escape cases where you have invalid characters like `.` in your identifiers. --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-5134] Bump default hadoop.version to 2....

2015-02-06 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/3917#issuecomment-73291565 The one negative side-effect I can see is that it will make it easier for people to inadvertently break the hadoop 1.x build. I know because I've done that (since all my

[GitHub] spark pull request: [SPARK-5416] init Executor.threadPool before E...

2015-02-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4212#issuecomment-73291491 Thanks @ryan-williams and @srowen. I've also merged this into `branch-1.3` (1.3.0). If you think it should be cherry-picked to other branches, please edit the JRIA

[GitHub] spark pull request: [SPARK-4983]insert waiting time before tagging...

2015-02-06 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/3986#issuecomment-73292995 Can someone trigger tests for this, by the way? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-4983]insert waiting time before tagging...

2015-02-06 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/3986#issuecomment-73292890 Left 2 minor comments about phrasing, but otherwise this LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-5598][MLLIB] model save/load for ALS

2015-02-06 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/4422#discussion_r24263497 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -136,3 +147,69 @@ class

[GitHub] spark pull request: [SPARK-5598][MLLIB] model save/load for ALS

2015-02-06 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/4422#discussion_r24263522 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -125,6 +130,12 @@ class

[GitHub] spark pull request: [SPARK-5656] Fail gracefully for large values ...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4433#issuecomment-73297460 [Test build #26929 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26929/consoleFull) for PR 4433 at commit

[GitHub] spark pull request: [SPARK-4964][Streaming][Kafka] More updates to...

2015-02-06 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4384#discussion_r24265028 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala --- @@ -36,14 +36,12 @@ import kafka.utils.VerifiableProperties *

[GitHub] spark pull request: SPARK-4267 [CORE] Failing to launch jobs on Sp...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4188#issuecomment-73300317 To fix the YARN issue maybe we should do something specific there, like escaping the double quotes before passing them to YARN? --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-4502] [SQL] Fix reads unnecessary neste...

2015-02-06 Thread cenyuhai
Github user cenyuhai commented on a diff in the pull request: https://github.com/apache/spark/pull/4398#discussion_r24266591 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/dataTypes.scala --- @@ -36,6 +36,8 @@ import org.apache.spark.util.Utils

[GitHub] spark pull request: [SPARK-5396] Syntax error in spark scripts on ...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4428#issuecomment-73301647 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: function hasShutdownDeleteTachyonDir should us...

2015-02-06 Thread haoyuan
Github user haoyuan commented on the pull request: https://github.com/apache/spark/pull/4418#issuecomment-73301858 Thanks @viper-kun Agree w/ @JoshRosen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-4705:Creating different log directories ...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4311#issuecomment-73302969 [Test build #26935 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26935/consoleFull) for PR 4311 at commit

[GitHub] spark pull request: SPARK-5633 pyspark saveAsTextFile support for ...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4403#issuecomment-73303027 [Test build #26934 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26934/consoleFull) for PR 4403 at commit

[GitHub] spark pull request: SPARK-4705:Creating different log directories ...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/4311#discussion_r24267784 --- Diff: yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -88,6 +88,10 @@ private[spark] class

[GitHub] spark pull request: [SPARK-5628] Add version option to spark-ec2

2015-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4414 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-5493] [core] Add option to impersonate ...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4405#issuecomment-73305261 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5493] [core] Add option to impersonate ...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4405#issuecomment-73305254 [Test build #26930 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26930/consoleFull) for PR 4405 at commit

[GitHub] spark pull request: [SPARK-5595][SPARK-5603][SQL] Add a rule to do...

2015-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4373 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-5640] Synchronize ScalaReflection where...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4431#issuecomment-73290734 [Test build #26923 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26923/consoleFull) for PR 4431 at commit

[GitHub] spark pull request: Update ec2-scripts.md

2015-02-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4300#issuecomment-73292485 @srowen you can do `git commit --amend` to edit the commit title yourself before pushing to GitHub. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-5366][EC2] Check the mode of private ke...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4162#issuecomment-73292357 [Test build #26924 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26924/consoleFull) for PR 4162 at commit

[GitHub] spark pull request: [SPARK-4983]insert waiting time before tagging...

2015-02-06 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/3986#discussion_r24262559 --- Diff: ec2/spark_ec2.py --- @@ -569,6 +569,9 @@ def launch_cluster(conn, opts, cluster_name): master_nodes = master_res.instances

[GitHub] spark pull request: [SPARK-5396] Syntax error in spark scripts on ...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4428#issuecomment-73292424 [Test build #26925 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26925/consoleFull) for PR 4428 at commit

[GitHub] spark pull request: [SPARK-5396] Syntax error in spark scripts on ...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4428#issuecomment-73292446 Oops, that is an obvious one. Thanks for fixing this @tsudukim I'm merging this into master and 1.3 --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-4877] Allow user first classes to exten...

2015-02-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3725#issuecomment-73293433 Thanks for the reminder. Given the sign-off from @holdenk and @vanzin, I'm going to merge this into `master` (1.4.0) and `branch-1.3` (1.3.0). --- If your project

[GitHub] spark pull request: Update ec2-scripts.md

2015-02-06 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4300#issuecomment-73293397 @JoshRosen a-ha, so after the script merges locally, go amend it separately, then continue with the script eh? --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-5652][Mllib] Use broadcasted weights in...

2015-02-06 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/4429#issuecomment-73293490 @viirya For the `weightMatrix` - `weights` change. This is inside `predictPoint` and by definition we should use the `weightMatrix` from input arguments. Are we

[GitHub] spark pull request: [SPARK-4877] Allow user first classes to exten...

2015-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3725 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-5600] [core] Clean up FsHistoryProvider...

2015-02-06 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4370#discussion_r24263861 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala --- @@ -211,4 +202,8 @@ class FsHistoryProviderSuite extends

[GitHub] spark pull request: [SPARK-4361][Doc] Add more docs for Hadoop Con...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3225#discussion_r24264861 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -630,7 +634,10 @@ class SparkContext(config: SparkConf) extends SparkStatusAPI

[GitHub] spark pull request: [SPARK-3381] [MLlib] Eliminate bins for unorde...

2015-02-06 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/4231#issuecomment-73300352 ping @jkbradley ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-5644] [Core]Delete tmp dir when sc is s...

2015-02-06 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4412#discussion_r24267365 --- Diff: core/src/main/scala/org/apache/spark/SparkEnv.scala --- @@ -93,6 +93,14 @@ class SparkEnv ( // actorSystem.awaitTermination()

[GitHub] spark pull request: [SPARK-4502] [SQL] Fix reads unnecessary neste...

2015-02-06 Thread cenyuhai
Github user cenyuhai commented on a diff in the pull request: https://github.com/apache/spark/pull/4398#discussion_r24267316 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/BoundAttribute.scala --- @@ -53,7 +53,7 @@ object BindReferences extends

[GitHub] spark pull request: [SPARK-5598][MLLIB] model save/load for ALS

2015-02-06 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/4422#discussion_r24267412 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -136,3 +147,69 @@ class

[GitHub] spark pull request: [SPARK-4874] [CORE] Collect record count metri...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4067#issuecomment-73303658 [Test build #26936 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26936/consoleFull) for PR 4067 at commit

[GitHub] spark pull request: [SPARK-4964][Streaming][Kafka] More updates to...

2015-02-06 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4384#discussion_r24268021 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/OffsetRange.scala --- @@ -19,16 +19,35 @@ package org.apache.spark.streaming.kafka

[GitHub] spark pull request: [SPARK-4964][Streaming][Kafka] More updates to...

2015-02-06 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4384#discussion_r24268035 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala --- @@ -179,121 +182,190 @@ object KafkaUtils { errs

[GitHub] spark pull request: [SPARK-4808] Configurable spillable memory thr...

2015-02-06 Thread mingyukim
Github user mingyukim commented on the pull request: https://github.com/apache/spark/pull/4420#issuecomment-73303667 Can you elaborate on the memory size as an additional heuristic idea? This is consistently causing OOMs in one of our workflows, which is exactly what spilling to disk

[GitHub] spark pull request: [SPARK-5640] Synchronize ScalaReflection where...

2015-02-06 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4431#issuecomment-73304864 Thanks! Merged to master and 1.3 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-5493] [core] Add option to impersonate ...

2015-02-06 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/4405#issuecomment-73305341 In this case, you are only running SparkSubmit as the proxy user. Should we not have the executor code also run as the proxy user, so any writes from the app to

[GitHub] spark pull request: [SPARK-5601][MLLIB] make streaming linear algo...

2015-02-06 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/4432#issuecomment-73305358 Looks good, but not too familiar with this class --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-4905][STREAMING] FlumeStreamSuite fix.

2015-02-06 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/4371#issuecomment-73305408 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-5636] Ramp up faster in dynamic allocat...

2015-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4409 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-4983]insert waiting time before tagging...

2015-02-06 Thread GenTang
Github user GenTang commented on a diff in the pull request: https://github.com/apache/spark/pull/3986#discussion_r24263282 --- Diff: ec2/spark_ec2.py --- @@ -569,6 +569,9 @@ def launch_cluster(conn, opts, cluster_name): master_nodes = master_res.instances

[GitHub] spark pull request: [SPARK-5598][MLLIB] model save/load for ALS

2015-02-06 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/4422#discussion_r24263210 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -136,3 +147,69 @@ class

[GitHub] spark pull request: [SPARK-4808] Configurable spillable memory thr...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4420#issuecomment-73293800 Thanks @mingyukim. On second thought I'm not so sure if we should make these configurable, since doing so kind of pushes the responsibility to the user to experiment

[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/4350#discussion_r24267242 --- Diff: docs/running-on-yarn.md --- @@ -105,6 +105,13 @@ Most of the configs are the same for Spark on YARN as for other deployment modes /td

[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/4350#discussion_r24267256 --- Diff: docs/running-on-yarn.md --- @@ -105,6 +105,13 @@ Most of the configs are the same for Spark on YARN as for other deployment modes /td

[GitHub] spark pull request: [SPARK-5366][EC2] Check the mode of private ke...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4162#issuecomment-73304383 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-4267 [CORE] Failing to launch jobs on Sp...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4188#issuecomment-73305744 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2996] Implement userClassPathFirst for ...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3233#issuecomment-73305760 [Test build #26932 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26932/consoleFull) for PR 3233 at commit

[GitHub] spark pull request: [SPARK-2996] Implement userClassPathFirst for ...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3233#issuecomment-73305769 Test FAILed. Refer to this link for build results (access rights to CI server needed):

  1   2   3   4   >