[GitHub] spark pull request: [SPARK-11466] [core] Avoid mockito in multi-th...

2015-11-02 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/9425 [SPARK-11466] [core] Avoid mockito in multi-threaded FsHistoryProviderSuite test. The test functionality should be the same, but without using mockito; logs don't really say anything useful but

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153202265 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153202251 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11329] [SQL] Support star expansion for...

2015-11-02 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9343#discussion_r43703147 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -166,26 +166,63 @@ abstract class Star extends LeafExpress

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9344#discussion_r43703108 --- Diff: core/src/main/java/org/apache/spark/memory/MemoryConsumer.java --- @@ -43,10 +41,14 @@ protected MemoryConsumer(TaskMemoryManager taskMemoryManager)

[GitHub] spark pull request: [SPARK-10622] [core] [yarn] Differentiate dead...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-153201756 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153201802 ping @yhuai @JoshRosen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-11329] [SQL] Support star expansion for...

2015-11-02 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9343#discussion_r43703049 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -166,26 +166,63 @@ abstract class Star extends LeafExpress

[GitHub] spark pull request: [SPARK-10622] [core] [yarn] Differentiate dead...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-153201753 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-9836] [ML] Provide R-like summary stati...

2015-11-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/9413#discussion_r43702995 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala --- @@ -715,4 +724,63 @@ class LinearRegressionSuite extends SparkFu

[GitHub] spark pull request: [SPARK-10622] [core] [yarn] Differentiate dead...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-153201676 **[Test build #44841 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44841/consoleFull)** for PR 8887 at commit [`f02558c`](https://git

[GitHub] spark pull request: [SPARK-11467] [SQL] add Python API for stddev/...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9424#issuecomment-153201632 **[Test build #44853 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44853/consoleFull)** for PR 9424 at commit [`9526e7c`](https://gith

[GitHub] spark pull request: [SPARK-11467] [SQL] add Python API for stddev/...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9424#issuecomment-153201589 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11467] [SQL] add Python API for stddev/...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9424#issuecomment-153201578 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11329] [SQL] Support star expansion for...

2015-11-02 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9343#discussion_r43702945 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -166,26 +166,63 @@ abstract class Star extends LeafExpress

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153201546 **[Test build #1970 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1970/consoleFull)** for PR 9383 at commit [`df44fc6`](https://gi

[GitHub] spark pull request: [SPARK-11329] [SQL] Support star expansion for...

2015-11-02 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9343#discussion_r43702827 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -166,26 +166,63 @@ abstract class Star extends LeafExpress

[GitHub] spark pull request: [SPARK-11343] [ML] Allow float and double pred...

2015-11-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/9296#issuecomment-153201122 Merged into master. @dahlem Could you send a new PR to address @felixcheung 's comment? --- If your project is set up for it, you can reply to this email and have your r

[GitHub] spark pull request: [SPARK-10592] [ML] [PySpark] Deprecate weights...

2015-11-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9311 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-11467] [SQL] add Python API for stddev/...

2015-11-02 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/9424 [SPARK-11467] [SQL] add Python API for stddev/variance Add Python API for stddev/stddev_pop/stddev_samp/variance/var_pop/var_samp/skewness/kurtosis You can merge this pull request into a Git reposit

[GitHub] spark pull request: [SPARK-11343] [ML] Allow float and double pred...

2015-11-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9296 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-10592] [ML] [PySpark] Deprecate weights...

2015-11-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/9311#issuecomment-153201019 @dbtsai The comment on `LogisticRegressionAggregator` is not addressed yet. Could you send a PR to fix it? --- If your project is set up for it, you can reply to this e

[GitHub] spark pull request: [SPARK-10592] [ML] [PySpark] Deprecate weights...

2015-11-02 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/9311#issuecomment-153200678 Thanks. Merged into master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10286][ML][PySpark][Docs] Add @since an...

2015-11-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9275 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-10286][ML][PySpark][Docs] Add @since an...

2015-11-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/9275#issuecomment-153200053 Merged into master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153199533 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153199537 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153199120 **[Test build #44832 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44832/consoleFull)** for PR 9344 at commit [`d8ffd35`](https://git

[GitHub] spark pull request: [SPARK-11424] Guard against double-close() of ...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9423#issuecomment-153199436 [Test build #44852 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44852/consoleFull) for PR 9423 at commit [`cf008ca`](https://gith

[GitHub] spark pull request: [SPARK-11293] Fix shuffle memory leaks in Spil...

2015-11-02 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9260#issuecomment-153198660 Ping @yhuai, any objection to merging this to 1.5? A similar fix has already been incorporated into master as part of my memory manager consolidation patch. We could a

[GitHub] spark pull request: [SPARK-11424] Guard against double-close() of ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9423#issuecomment-153198334 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11424] Guard against double-close() of ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9423#issuecomment-153198351 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10592] [ML] [PySpark] Deprecate weights...

2015-11-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/9311#issuecomment-153198286 No, that is a compatible change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11329] [SQL] Support star expansion for...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9343#issuecomment-153197858 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11329] [SQL] Support star expansion for...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9343#issuecomment-153197856 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11329] [SQL] Support star expansion for...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9343#issuecomment-153197650 **[Test build #44840 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44840/consoleFull)** for PR 9343 at commit [`c55a7d2`](https://git

[GitHub] spark pull request: [SPARK-11424] Guard against double-close() of ...

2015-11-02 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/9423 [SPARK-11424] Guard against double-close() of RecordReaders (branch-1.3 backport) This is a branch-1.3 backport of #9382, a fix for SPARK-11424. You can merge this pull request into a Git reposit

[GitHub] spark pull request: [SPARK-11424] Guard against double-close() of ...

2015-11-02 Thread JoshRosen
Github user JoshRosen closed the pull request at: https://github.com/apache/spark/pull/9388 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[GitHub] spark pull request: [SPARK-11424] Guard against double-close() of ...

2015-11-02 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9388#issuecomment-153196656 Merged to branch-1.4 (1.4.2). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153196272 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153196273 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153196174 **[Test build #44831 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44831/consoleFull)** for PR 9344 at commit [`d8ffd35`](https://git

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153195860 **[Test build #44851 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44851/consoleFull)** for PR 9344 at commit [`6a5204f`](https://git

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153195867 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153195866 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153195513 **[Test build #44851 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44851/consoleFull)** for PR 9344 at commit [`6a5204f`](https://gith

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153195222 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153195202 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11020] [core] Wait for HDFS to leave sa...

2015-11-02 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9043#issuecomment-153194849 I have created https://issues.apache.org/jira/browse/SPARK-11466. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-11020] [core] Wait for HDFS to leave sa...

2015-11-02 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9043#issuecomment-153194708 hmm. If I run `FsHistoryProviderSuite` individually, it is fine. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153194638 @davies, @andrewor14, @rxin: this PR is now ready for review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-11020] [core] Wait for HDFS to leave sa...

2015-11-02 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9043#issuecomment-153194554 @vanzin Looks like this breaks hadoop1 tests. Please see https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/Spark-Master-SBT/AMPLAB_JENKINS_BUI

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153194220 **[Test build #44850 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44850/consoleFull)** for PR 9344 at commit [`852b5c9`](https://gith

[GitHub] spark pull request: [SPARK-11235] [network] Add ability to stream ...

2015-11-02 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/9206#issuecomment-153193676 ping @rxin @zsxwing This is blocking other stuff I'm working on so I'd really like to push this soon. --- If your project is set up for it, you can reply to th

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153192593 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11195][CORE] Use correct classloader fo...

2015-11-02 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9367#issuecomment-153192578 @choochootrain I think you can write your app in SparkSubmitSuite (https://github.com/apache/spark/blob/master/core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153192628 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10304][SQL] Partition discovery should ...

2015-11-02 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/8840#discussion_r43699404 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -184,7 +204,8 @@ private[sql] object PartitioningU

[GitHub] spark pull request: [SPARK-11275][SQL][WIP] Rollup and Cube Genera...

2015-11-02 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/9419#issuecomment-153192515 @rick-ibm Will add more comments to explain it. Especially, I will emphasize this design will expect the optimizer collapses these two projections into a single one.

[GitHub] spark pull request: [SPARK-11195][CORE] Use correct classloader fo...

2015-11-02 Thread choochootrain
Github user choochootrain commented on the pull request: https://github.com/apache/spark/pull/9367#issuecomment-153191848 i'm writing the test right now, is there an easy way to get the relative path to the assembled spark jar so I can compile my job against it? --- If your project i

[GitHub] spark pull request: [SPARK-10304][SQL] Partition discovery should ...

2015-11-02 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/8840#discussion_r43699130 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -77,8 +77,16 @@ private[sql] object PartitioningUt

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9393#issuecomment-153191362 **[Test build #44849 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44849/consoleFull)** for PR 9393 at commit [`d2ee5ee`](https://gith

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153190736 **[Test build #44847 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44847/consoleFull)** for PR 9422 at commit [`bec07da`](https://git

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153190742 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153190739 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9393#issuecomment-153190509 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9393#issuecomment-153190522 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153190310 **[Test build #44847 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44847/consoleFull)** for PR 9422 at commit [`bec07da`](https://gith

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153189652 **[Test build #44830 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44830/consoleFull)** for PR 9383 at commit [`3864095`](https://git

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153189858 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-9319][SPARKR] Add support for setting c...

2015-11-02 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/9218#discussion_r43698343 --- Diff: R/pkg/R/DataFrame.R --- @@ -276,6 +276,75 @@ setMethod("names<-", } }) +#' @rdname columns +#' @nam

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153189866 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153188893 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153188897 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-10658][SPARK-11421][PYSPARK][CORE] Prov...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9313#issuecomment-153188708 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-9319][SPARKR] Add support for setting c...

2015-11-02 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/9218#discussion_r43698022 --- Diff: R/pkg/R/DataFrame.R --- @@ -276,6 +276,75 @@ setMethod("names<-", } }) +#' @rdname columns +#' @nam

[GitHub] spark pull request: [SPARK-10658][SPARK-11421][PYSPARK][CORE] Prov...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9313#issuecomment-153188707 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9344#discussion_r43697998 --- Diff: core/src/test/scala/org/apache/spark/memory/UnifiedMemoryManagerSuite.scala --- @@ -163,18 +170,19 @@ class UnifiedMemoryManagerSuite extends Mem

[GitHub] spark pull request: [SPARK-10658][SPARK-11421][PYSPARK][CORE] Prov...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9313#issuecomment-153188705 **[Test build #44848 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44848/consoleFull)** for PR 9313 at commit [`72b9f36`](https://git

[GitHub] spark pull request: [SPARK-8402][MLLIB] DP Means Clustering

2015-11-02 Thread yu-iskw
Github user yu-iskw commented on a diff in the pull request: https://github.com/apache/spark/pull/6880#discussion_r43697828 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/DpMeans.scala --- @@ -0,0 +1,279 @@ +/* + * Licensed to the Apache Software Foundatio

[GitHub] spark pull request: [SPARK-10658][SPARK-11421][PYSPARK][CORE] Prov...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9313#issuecomment-153188257 **[Test build #44848 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44848/consoleFull)** for PR 9313 at commit [`72b9f36`](https://gith

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-02 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/8984#discussion_r43697786 --- Diff: R/pkg/R/types.R --- @@ -0,0 +1,41 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreeme

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-02 Thread felixcheung
Github user felixcheung commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-153188047 @olarayej from what I gather from our conversation, "map" = NA should be what we want initially. @sun-rui ? --- If your project is set up for it, you can reply to t

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153187920 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153187882 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10658][SPARK-11421][PYSPARK][CORE] Prov...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9313#issuecomment-153187887 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10658][SPARK-11421][PYSPARK][CORE] Prov...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9313#issuecomment-153187919 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/9422 [SPARK-10429] [SQL] make mutableProjection atomic Right now, SQL's mutable projection updates every value of the mutable project after it evaluates the corresponding expression. This makes the behav

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153187619 **[Test build #44846 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44846/consoleFull)** for PR 9344 at commit [`f976faa`](https://gith

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153186987 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153187016 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9344#discussion_r43696993 --- Diff: core/src/main/scala/org/apache/spark/memory/package.scala --- @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] spark pull request: SPARK-11442 Reduce numSlices for local metrics...

2015-11-02 Thread tedyu
Github user tedyu commented on the pull request: https://github.com/apache/spark/pull/9384#issuecomment-153186003 By adding a sleep after 'local metrics' finishes, I captured the following using ps command: ``` USER PID %CPU %MEMVSZ RSS TTY STAT hbase

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153186079 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153186094 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10592] [ML] [PySpark] Deprecate weights...

2015-11-02 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/9311#issuecomment-153183047 This looks good to me, but I wonder if changing from `val weights` into `def weights: Vector = coefficients` will cause any binary compatibility issue. --- If your proje

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153182155 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153182196 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153182320 **[Test build #44844 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44844/consoleFull)** for PR 9344 at commit [`b0d569d`](https://gith

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/9393#issuecomment-153182111 @yhuai This only applies to UDFs. What if a user uses built-in expressions, e.g.: ~~~scala val a = b + c val d = a * a ~~~ It may be useful t

<    1   2   3   4   5   6   7   8   9   10   >