[GitHub] spark pull request: [SPARK-11029] [ML] Add computeCost to KMeansMo...

2015-10-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/9073#discussion_r42272109 --- Diff: mllib/src/test/scala/org/apache/spark/ml/clustering/KMeansSuite.scala --- @@ -104,5 +104,6 @@ class KMeansSuite extends SparkFunSuite with

[GitHub] spark pull request: [SPARK-9066][SQL] Improve cartesian performanc...

2015-10-16 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/7417#discussion_r42272013 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -274,12 +275,30 @@ private[sql] abstract class

[GitHub] spark pull request: [SPARK-11029] [ML] Add computeCost to KMeansMo...

2015-10-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/9073#discussion_r42272083 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/KMeans.scala --- @@ -24,6 +24,7 @@ import org.apache.spark.ml.util.{Identifiable,

[GitHub] spark pull request: [SPARK-11029] [ML] Add computeCost to KMeansMo...

2015-10-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/9073#discussion_r42272087 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/KMeans.scala --- @@ -117,6 +118,14 @@ class KMeansModel private[ml] (

[GitHub] spark pull request: [SPARK-10484][SQL] Optimize the cartesian join...

2015-10-16 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/8652#discussion_r42272078 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala --- @@ -69,6 +71,58 @@ class HiveQuerySuite extends

[GitHub] spark pull request: [SPARK-11109] [CORE] Move FsHistoryProvider of...

2015-10-16 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9144#issuecomment-148789432 @vanzin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-11129][MESOS] Link Spark WebUI from Mes...

2015-10-16 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9135#issuecomment-148789532 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-11124] JsonParser/Generator should be c...

2015-10-16 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/9130#issuecomment-148790775 Thanks - I've merged this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11109] [CORE] Move FsHistoryProvider of...

2015-10-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9144#issuecomment-148797797 [Test build #43843 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43843/consoleFull) for PR 9144 at commit

[GitHub] spark pull request: [SPARK-11125] [SQL] Unreadable exception when ...

2015-10-16 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9134#issuecomment-148798108 Are we certain that we only get `NoClassDefFoundError` but not also `ClassNotFoundException`? I wonder if we should just check for either of those in the match. Also

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

2015-10-16 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148798288 Actually, this one has not been merged yet (weird thing happened while merging it), @koertkuipers please go ahead to address the comment. --- If your project is set up

[GitHub] spark pull request: [SPARK-10484][SQL] Optimize the cartesian join...

2015-10-16 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/8652#discussion_r42272209 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HivePlanTest.scala --- @@ -17,6 +17,7 @@ package

[GitHub] spark pull request: [SPARK-11029] [ML] Add computeCost to KMeansMo...

2015-10-16 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/9073#issuecomment-148788033 That should be it---thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11109] [CORE] Move FsHistoryProvider of...

2015-10-16 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/9144#issuecomment-148790293 LGTM. ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-9622] [ML] DecisionTreeRegressor: provi...

2015-10-16 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/8866#issuecomment-148793612 Thanks for your updates! Let me know if the proposal sounds workable. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-9622] [ML] DecisionTreeRegressor: provi...

2015-10-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/8866#discussion_r42274055 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/DecisionTreeRegressor.scala --- @@ -39,7 +40,7 @@ import org.apache.spark.sql.DataFrame

[GitHub] spark pull request: [SPARK-11063][Streaming]Change preferredLocati...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9075#issuecomment-148796371 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11149] [SQL] Improve cache performance ...

2015-10-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/9145#discussion_r42278053 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/columnar/GenerateColumnAccessor.scala --- @@ -0,0 +1,149 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...

2015-10-16 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/8884#issuecomment-148769776 @Lewuathe They should be the same, but there is no test proving it. Since in ```scala test("linear regression with weighted samples"){

[GitHub] spark pull request: [SPARK-11131] [core] Fix race in worker regist...

2015-10-16 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/9138#issuecomment-148784377 /cc @andrewor14 @zsxwing I think you're the people most familiar with this code. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-10979] [SparkR] Sparkrmerge: Add merge ...

2015-10-16 Thread NarineK
Github user NarineK commented on the pull request: https://github.com/apache/spark/pull/9012#issuecomment-148787306 Hi @felixcheung, thanks for your comments. 1. I'm a bit confused - why create a seq of indices? instead can't we use lapply on allColNames? yes, that's

[GitHub] spark pull request: [SPARK-11050][MLLIB] PySpark SparseVector can ...

2015-10-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9069#issuecomment-148788943 [Test build #1916 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1916/consoleFull) for PR 9069 at commit

[GitHub] spark pull request: Modified _verify_type() to be more informative...

2015-10-16 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9149#issuecomment-148789247 @lababidi please file a JIRA and add it to the title of the PR. See how other PRs are opened. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-10484][SQL] Optimize the cartesian join...

2015-10-16 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/8652#discussion_r42272725 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala --- @@ -69,6 +71,58 @@ class HiveQuerySuite extends

[GitHub] spark pull request: [SPARK-11109] [CORE] Move FsHistoryProvider of...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9144#issuecomment-148791764 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11109] [CORE] Move FsHistoryProvider of...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9144#issuecomment-148791740 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11126][SQL]Fix a memory leak in SQLList...

2015-10-16 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9132#issuecomment-148783875 LGTM, would be good to backport into 1.5. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11050][MLLIB] PySpark SparseVector can ...

2015-10-16 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/9069#issuecomment-148788590 @bhargav Thanks for the update! LGTM pending tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-11063][Streaming]Change preferredLocati...

2015-10-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9075#issuecomment-148795961 [Test build #43840 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43840/console) for PR 9075 at commit

[GitHub] spark pull request: [SPARK-11050][MLLIB] PySpark SparseVector can ...

2015-10-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9069#issuecomment-148796105 [Test build #1916 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1916/console) for PR 9069 at commit

[GitHub] spark pull request: [SPARK-11063][Streaming]Change preferredLocati...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9075#issuecomment-148796367 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-9963] [ML] RandomForest cleanup: replac...

2015-10-16 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/9044#issuecomment-148805746 I'll take another look at [https://github.com/apache/spark/pull/8609] now. Can you please close this duplicate PR? --- If your project is set up for it, you can

[GitHub] spark pull request: SPARK-10876 - Display total uptime for complet...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9059#issuecomment-148805581 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11114] [PYSPARK] add getOrCreate for Sp...

2015-10-16 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9122#discussion_r42277272 --- Diff: python/pyspark/context.py --- @@ -280,6 +280,18 @@ def __exit__(self, type, value, trace): """ self.stop() +

[GitHub] spark pull request: [SPARK-11149] [SQL] Improve cache performance ...

2015-10-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/9145#discussion_r42277995 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/columnar/GenerateColumnAccessor.scala --- @@ -0,0 +1,149 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-10930] Adds max task duration to all st...

2015-10-16 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/9051#issuecomment-148809195 (incidentally, I just realized this info is totally missing from the json, filed https://issues.apache.org/jira/browse/SPARK-11155) --- If your project is set up for

[GitHub] spark pull request: [SPARK-11043][SQL]BugFix:Set the operator log ...

2015-10-16 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9056#issuecomment-148809251 @yhuai @liancheng --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-10382] [POC] Make example code in user ...

2015-10-16 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/9109#discussion_r42264562 --- Diff: docs/_plugins/include_example.rb --- @@ -0,0 +1,70 @@ +require 'octopress-code-highlighter' +require 'liquid' + +module Octopress

[GitHub] spark pull request: [SPARK-10382] [POC] Make example code in user ...

2015-10-16 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/9109#discussion_r42264578 --- Diff: docs/index.md --- @@ -12,6 +12,8 @@ It also supports a rich set of higher-level tools including [Spark SQL](sql-prog # Downloading

[GitHub] spark pull request: [SPARK-10382] [POC] Make example code in user ...

2015-10-16 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/9109#discussion_r42264565 --- Diff: docs/_plugins/include_example.rb --- @@ -0,0 +1,70 @@ +require 'octopress-code-highlighter' +require 'liquid' + +module Octopress

[GitHub] spark pull request: [SPARK-10382] [POC] Make example code in user ...

2015-10-16 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/9109#discussion_r42264567 --- Diff: docs/_plugins/include_example.rb --- @@ -0,0 +1,70 @@ +require 'octopress-code-highlighter' +require 'liquid' + +module Octopress

[GitHub] spark pull request: [SPARK-10382] [POC] Make example code in user ...

2015-10-16 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/9109#discussion_r42264560 --- Diff: docs/_config.yml --- @@ -2,6 +2,7 @@ highlighter: pygments markdown: kramdown gems: - jekyll-redirect-from + -

[GitHub] spark pull request: [SPARK-11077] [SQL] Join elimination in Cataly...

2015-10-16 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/9089#issuecomment-148786929 @ankurdave Np, thanks for the fix. Btw, should the fix be accompanied by a unit test to catch that issue? --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-11129][MESOS] Link Spark WebUI from Mes...

2015-10-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9135#issuecomment-148792599 [Test build #43842 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43842/consoleFull) for PR 9135 at commit

[GitHub] spark pull request: [SPARK-9622] [ML] DecisionTreeRegressor: provi...

2015-10-16 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/8866#discussion_r42273886 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/shared/SharedParamsCodeGen.scala --- @@ -44,6 +44,8 @@ private[shared] object SharedParamsCodeGen

[GitHub] spark pull request: Packserv

2015-10-16 Thread damnMeddlingKid
GitHub user damnMeddlingKid opened a pull request: https://github.com/apache/spark/pull/9151 Packserv You can merge this pull request into a Git repository by running: $ git pull https://github.com/Shopify/spark packserv Alternatively you can review and apply these changes

[GitHub] spark pull request: SPARK-10876 - Display total uptime for complet...

2015-10-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9059#issuecomment-148807086 [Test build #43845 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43845/consoleFull) for PR 9059 at commit

[GitHub] spark pull request: [SPARK-11126][SQL]Fix a memory leak in SQLList...

2015-10-16 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9132#discussion_r42269957 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLListener.scala --- @@ -126,7 +126,13 @@ private[sql] class SQLListener(conf:

[GitHub] spark pull request: [SPARK-11120] Allow sane default number of exe...

2015-10-16 Thread ryan-williams
Github user ryan-williams commented on a diff in the pull request: https://github.com/apache/spark/pull/9147#discussion_r42271853 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -62,10 +62,23 @@ private[spark] class ApplicationMaster(

[GitHub] spark pull request: [SPARK-10979] [SparkR] Sparkrmerge: Add merge ...

2015-10-16 Thread NarineK
Github user NarineK commented on the pull request: https://github.com/apache/spark/pull/9012#issuecomment-148787814 Hi @felixcheung, thanks for your comments. 1. I'm a bit confused - why create a seq of indices? instead can't we use lapply on allColNames?

[GitHub] spark pull request: [SPARK-11120] Allow sane default number of exe...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9147#issuecomment-148790093 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11129][MESOS] Link Spark WebUI from Mes...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9135#issuecomment-148790133 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11129][MESOS] Link Spark WebUI from Mes...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9135#issuecomment-148790098 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11120] Allow sane default number of exe...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9147#issuecomment-148790122 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [YARN] When driver sends message "GetExecutorL...

2015-10-16 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9129#issuecomment-148792900 @KaiXinXiaoLei can you close this PR? Also next time please open an accompanying JIRA and add it to the title of your PR. --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-11149] [SQL] Improve cache performance ...

2015-10-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/9145#discussion_r42275706 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/columnar/ColumnType.scala --- @@ -28,6 +28,36 @@ import org.apache.spark.sql.types._ import

[GitHub] spark pull request: [SPARK-11121][Core] Correct the TaskLocation t...

2015-10-16 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9096#discussion_r42275658 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskLocation.scala --- @@ -64,10 +64,10 @@ private[spark] object TaskLocation { */

[GitHub] spark pull request: Packserv

2015-10-16 Thread damnMeddlingKid
Github user damnMeddlingKid closed the pull request at: https://github.com/apache/spark/pull/9151 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request: [SPARK-11149] [SQL] Improve cache performance ...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9145#issuecomment-148804511 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark-10994] Add local clustering coefficient...

2015-10-16 Thread SherlockYang
GitHub user SherlockYang opened a pull request: https://github.com/apache/spark/pull/9150 [Spark-10994] Add local clustering coefficient computation in GraphX The local clustering coefficient of a vertex (node) in a graph quantifies how close its neighbours are to being a clique

[GitHub] spark pull request: [Spark-10994] Add local clustering coefficient...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9150#issuecomment-148770205 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11050][MLLIB] PySpark SparseVector can ...

2015-10-16 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/9069#issuecomment-148794016 Btw, it doesn't matter what you call your branch name; that's not a problem. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...

2015-10-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/8416#discussion_r42274201 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala --- @@ -123,6 +124,24 @@ class DataFrameReader private[sql](sqlContext:

[GitHub] spark pull request: [SPARK-11120] Allow sane default number of exe...

2015-10-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9147#issuecomment-148793946 [Test build #43841 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43841/consoleFull) for PR 9147 at commit

[GitHub] spark pull request: Packserv

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9151#issuecomment-148799659 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11149] [SQL] Improve cache performance ...

2015-10-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/9145#discussion_r42275556 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeFormatter.scala --- @@ -44,11 +45,13 @@ private class CodeFormatter

[GitHub] spark pull request: [SPARK-11149] [SQL] Improve cache performance ...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9145#issuecomment-148804541 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11121][Core] Correct the TaskLocation t...

2015-10-16 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9096#issuecomment-148804209 @zhichao-li This looks fine, but are there other implications? In `TaskSetManager` there are entire blocks of code that are not executed before this patch. --- If

[GitHub] spark pull request: [SPARK-10708] [WIP] Consolidate sort shuffle i...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8829#issuecomment-148805580 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-10876 - Display total uptime for complet...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9059#issuecomment-148805547 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10708] [WIP] Consolidate sort shuffle i...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8829#issuecomment-148805552 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11149] [SQL] Improve cache performance ...

2015-10-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/9145#discussion_r42278194 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/columnar/InMemoryColumnarTableScan.scala --- @@ -43,7 +42,7 @@ private[sql] object InMemoryRelation {

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-10-16 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/8984#discussion_r42263878 --- Diff: R/pkg/R/DataFrame.R --- @@ -1880,4 +1880,46 @@ setMethod("as.data.frame", stop(paste("Unused argument(s): ", paste(list(...),

[GitHub] spark pull request: [SPARK-11000][YARN]Bug fix: Derby have booted ...

2015-10-16 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/9026#issuecomment-148775462 Much better. :-) LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-11063][Streaming]Change preferredLocati...

2015-10-16 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/9075#issuecomment-148778827 Tested this patch in a 5 nodes cluster. Each node has an executor, the executor core is 1 and the receiver number is 5. Before this patch, there were several

[GitHub] spark pull request: [SPARK-11000][YARN]Bug fix: Derby have booted ...

2015-10-16 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/9026#issuecomment-148782895 ah, btw, can you fix the pr description to reflect the actual change? thanks! --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: Packserv

2015-10-16 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/9151#issuecomment-148799331 @damnMeddlingKid can you close this PR please? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11149] [SQL] Improve cache performance ...

2015-10-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/9145#discussion_r42277711 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/columnar/GenerateColumnAccessor.scala --- @@ -0,0 +1,149 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-10708] [WIP] Consolidate sort shuffle i...

2015-10-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8829#issuecomment-148806897 [Test build #43846 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43846/consoleFull) for PR 8829 at commit

[GitHub] spark pull request: [SPARK-11149] [SQL] Improve cache performance ...

2015-10-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9145#issuecomment-148806821 [Test build #43844 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43844/consoleFull) for PR 9145 at commit

[GitHub] spark pull request: [SPARK-10930] Adds max task duration to all st...

2015-10-16 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/9051#issuecomment-148808343 Also jumping in late, but I agree with @andrewor14 , I think we should just change duration to (1), that would be the most useful. My vote is for (last task end) -

[GitHub] spark pull request: [SPARK-10599][MLLIB] Lower communication for b...

2015-10-16 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/8757#issuecomment-148817255 @jkbradley According to the time taken it actually did the shuffle. However, I am not sure how useful in practice these block-diagonal matrices. --- If your project

[GitHub] spark pull request: [SPARK-11120] Allow sane default number of exe...

2015-10-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9147#issuecomment-148822901 [Test build #43848 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43848/consoleFull) for PR 9147 at commit

[GitHub] spark pull request: [SPARK-9643] Upgrade pyrolite to 4.9

2015-10-16 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7950#issuecomment-148824693 Hey @davies @angelini, what's the status / final verdict on this PR? Should we still consider bumping Pyrolite for 1.6? --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-10484][SQL] Optimize the cartesian join...

2015-10-16 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/8652#discussion_r42286218 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -268,6 +268,27 @@ private[sql] abstract class SparkStrategies

[GitHub] spark pull request: [SPARK-9818][SQL][WIP]Revert SPARK-6136 to ena...

2015-10-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/8101#discussion_r42286254 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/PostgresIntegrationSuite.scala --- @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [CORE] [SPARK-6593] Provide a HadoopRDD varian...

2015-10-16 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/5368#issuecomment-148825149 Hey @tigerquoll, would you mind closing this PR for now? It sounds like this is going to need some more work or discussion, so I think we should close this until it's

[GitHub] spark pull request: [SPARK-10484][SQL] Optimize the cartesian join...

2015-10-16 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/8652#discussion_r42286246 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -268,6 +268,27 @@ private[sql] abstract class SparkStrategies

[GitHub] spark pull request: [SPARK-11111] [SQL] fast null-safe join

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9120#issuecomment-148825207 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11111] [SQL] fast null-safe join

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9120#issuecomment-148825286 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10666][SPARK-6880][CORE] Use properties...

2015-10-16 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/6291#issuecomment-148827020 Hey @markhamstra, what's the status of this PR? Do you have time to address Kay's latest comment? It's been open for a while and I'd like to get it merged soon if you

[GitHub] spark pull request: [SPARK-11129][MESOS] Link Spark WebUI from Mes...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9135#issuecomment-148839422 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11119][SQL] cleanup for unsafe array an...

2015-10-16 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9131#issuecomment-148839614 LGTM, left one minor comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11129][MESOS] Link Spark WebUI from Mes...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9135#issuecomment-148839421 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-7021 Add JUnit output for Python tests

2015-10-16 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/8323#issuecomment-148840605 @gliptak I had an offline discussion with @davies . Let's do the following: 1. Change L259 in `pyspark/tests.py` to: ~~~python # Regression

[GitHub] spark pull request: [SPARK-10447][WIP][PYSPARK] upgrade pyspark to...

2015-10-16 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/8615#issuecomment-148842506 @kevincox, by no means is this rejecting the idea of upgrading to 0.9; I was only suggesting that we close this PR and re-open once it's ready for further review. If

[GitHub] spark pull request: [SPARK-11120] Allow sane default number of exe...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9147#issuecomment-148842465 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11120] Allow sane default number of exe...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9147#issuecomment-148842488 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [WIP][SPARK-7018][Build]: Refactor dev/run-tes...

2015-10-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7401#issuecomment-148842500 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11050][MLLIB] PySpark SparseVector can ...

2015-10-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9069 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

  1   2   3   4   5   6   7   >