[GitHub] spark pull request: SPARK-11344: Made ApplicationDescription and D...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9299#issuecomment-152981661 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11424] Guard against double-close() of ...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9388#issuecomment-153177605 [Test build #44826 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44826/consoleFull) for PR 9388 at commit

[GitHub] spark pull request: [SPARK-11424] Guard against double-close() of ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9388#issuecomment-153177730 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11424] Guard against double-close() of ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9388#issuecomment-153177734 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11359][STREAMING][KINESIS] Checkpoint t...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9421#issuecomment-153181727 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11359][STREAMING][KINESIS] Checkpoint t...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9421#issuecomment-153181593 **[Test build #44839 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44839/consoleFull)** for PR 9421 at commit

[GitHub] spark pull request: [SPARK-11389][WIP][CORE] Add support for off-h...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153181437 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11359][STREAMING][KINESIS] Checkpoint t...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9421#issuecomment-153181731 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153186079 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153186094 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-11442 Reduce numSlices for local metrics...

2015-11-02 Thread tedyu
Github user tedyu commented on the pull request: https://github.com/apache/spark/pull/9384#issuecomment-153186003 By adding a sleep after 'local metrics' finishes, I captured the following using ps command: ``` USER PID %CPU %MEMVSZ RSS TTY STAT hbase

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153186987 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153187016 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9344#discussion_r43697998 --- Diff: core/src/test/scala/org/apache/spark/memory/UnifiedMemoryManagerSuite.scala --- @@ -163,18 +170,19 @@ class UnifiedMemoryManagerSuite extends

[GitHub] spark pull request: [SPARK-10658][SPARK-11421][PYSPARK][CORE] Prov...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9313#issuecomment-153188705 **[Test build #44848 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44848/consoleFull)** for PR 9313 at commit

[GitHub] spark pull request: [SPARK-10658][SPARK-11421][PYSPARK][CORE] Prov...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9313#issuecomment-153188707 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9393#issuecomment-153190522 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9393#issuecomment-153191362 **[Test build #44849 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44849/consoleFull)** for PR 9393 at commit

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153192593 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11235] [network] Add ability to stream ...

2015-11-02 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/9206#issuecomment-153193676 ping @rxin @zsxwing This is blocking other stuff I'm working on so I'd really like to push this soon. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153194638 @davies, @andrewor14, @rxin: this PR is now ready for review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-11020] [core] Wait for HDFS to leave sa...

2015-11-02 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9043#issuecomment-153194708 hmm. If I run `FsHistoryProviderSuite` individually, it is fine. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-11020] [core] Wait for HDFS to leave sa...

2015-11-02 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9043#issuecomment-153194554 @vanzin Looks like this breaks hadoop1 tests. Please see

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153195202 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153195222 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11329] [SQL] Support star expansion for...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9343#issuecomment-153197858 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11329] [SQL] Support star expansion for...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9343#issuecomment-153197856 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11293] Fix shuffle memory leaks in Spil...

2015-11-02 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9260#issuecomment-153198660 Ping @yhuai, any objection to merging this to 1.5? A similar fix has already been incorporated into master as part of my memory manager consolidation patch. We could

[GitHub] spark pull request: [SPARK-11424] Guard against double-close() of ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9423#issuecomment-153198334 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11467] [SQL] add Python API for stddev/...

2015-11-02 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/9424 [SPARK-11467] [SQL] add Python API for stddev/variance Add Python API for stddev/stddev_pop/stddev_samp/variance/var_pop/var_samp/skewness/kurtosis You can merge this pull request into a Git

[GitHub] spark pull request: [SPARK-11343] [ML] Allow float and double pred...

2015-11-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/9296#issuecomment-153201122 Merged into master. @dahlem Could you send a new PR to address @felixcheung 's comment? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-10592] [ML] [PySpark] Deprecate weights...

2015-11-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9311 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-11329] [SQL] Support star expansion for...

2015-11-02 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9343#discussion_r43702827 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -166,26 +166,63 @@ abstract class Star extends

[GitHub] spark pull request: [SPARK-11329] [SQL] Support star expansion for...

2015-11-02 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9343#discussion_r43703147 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -166,26 +166,63 @@ abstract class Star extends

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9344#discussion_r43703108 --- Diff: core/src/main/java/org/apache/spark/memory/MemoryConsumer.java --- @@ -43,10 +41,14 @@ protected MemoryConsumer(TaskMemoryManager taskMemoryManager)

[GitHub] spark pull request: [SPARK-11389][WIP][CORE] Add support for off-h...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153180075 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10592] [ML] [PySpark] Deprecate weights...

2015-11-02 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/9311#discussion_r43694864 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -392,11 +392,14 @@ class LogisticRegression(override val

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153190739 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153190736 **[Test build #44847 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44847/consoleFull)** for PR 9422 at commit

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153190742 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11195][CORE] Use correct classloader fo...

2015-11-02 Thread choochootrain
Github user choochootrain commented on the pull request: https://github.com/apache/spark/pull/9367#issuecomment-153191848 i'm writing the test right now, is there an easy way to get the relative path to the assembled spark jar so I can compile my job against it? --- If your project

[GitHub] spark pull request: [SPARK-10304][SQL] Partition discovery should ...

2015-11-02 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/8840#discussion_r43699130 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -77,8 +77,16 @@ private[sql] object

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153195513 **[Test build #44851 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44851/consoleFull)** for PR 9344 at commit

[GitHub] spark pull request: [SPARK-11424] Guard against double-close() of ...

2015-11-02 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9388#issuecomment-153196656 Merged to branch-1.4 (1.4.2). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11424] Guard against double-close() of ...

2015-11-02 Thread JoshRosen
Github user JoshRosen closed the pull request at: https://github.com/apache/spark/pull/9388 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-11424] Guard against double-close() of ...

2015-11-02 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/9423 [SPARK-11424] Guard against double-close() of RecordReaders (branch-1.3 backport) This is a branch-1.3 backport of #9382, a fix for SPARK-11424. You can merge this pull request into a Git

[GitHub] spark pull request: [SPARK-11329] [SQL] Support star expansion for...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9343#issuecomment-153197650 **[Test build #44840 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44840/consoleFull)** for PR 9343 at commit

[GitHub] spark pull request: [SPARK-10592] [ML] [PySpark] Deprecate weights...

2015-11-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/9311#issuecomment-153198286 No, that is a compatible change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11424] Guard against double-close() of ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9423#issuecomment-153198351 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10286][ML][PySpark][Docs] Add @since an...

2015-11-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9275 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153199533 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10286][ML][PySpark][Docs] Add @since an...

2015-11-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/9275#issuecomment-153200053 Merged into master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10622] [core] [yarn] Differentiate dead...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-153201756 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153201802 ping @yhuai @JoshRosen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10622] [core] [yarn] Differentiate dead...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-153201753 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11329] [SQL] Support star expansion for...

2015-11-02 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9343#discussion_r43703049 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -166,26 +166,63 @@ abstract class Star extends

[GitHub] spark pull request: SPARK-11442 Reduce numSlices for local metrics...

2015-11-02 Thread tedyu
Github user tedyu commented on the pull request: https://github.com/apache/spark/pull/9384#issuecomment-153178210 SparkListenerSuite running out of memory is quite common on Jenkins:

[GitHub] spark pull request: [SPARK-11389][WIP][CORE] Add support for off-h...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153180020 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11389][WIP][CORE] Add support for off-h...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153180888 **[Test build #44843 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44843/consoleFull)** for PR 9344 at commit

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153182196 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153182320 **[Test build #44844 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44844/consoleFull)** for PR 9344 at commit

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153182155 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153187920 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10658][SPARK-11421][PYSPARK][CORE] Prov...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9313#issuecomment-153187887 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10658][SPARK-11421][PYSPARK][CORE] Prov...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9313#issuecomment-153187919 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153187882 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-9319][SPARKR] Add support for setting c...

2015-11-02 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/9218#discussion_r43698022 --- Diff: R/pkg/R/DataFrame.R --- @@ -276,6 +276,75 @@ setMethod("names<-", } }) +#' @rdname columns +#'

[GitHub] spark pull request: [SPARK-10658][SPARK-11421][PYSPARK][CORE] Prov...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9313#issuecomment-153188708 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153188897 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153188893 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153190310 **[Test build #44847 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44847/consoleFull)** for PR 9422 at commit

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153189858 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-9319][SPARKR] Add support for setting c...

2015-11-02 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/9218#discussion_r43698343 --- Diff: R/pkg/R/DataFrame.R --- @@ -276,6 +276,75 @@ setMethod("names<-", } }) +#' @rdname columns +#'

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153189652 **[Test build #44830 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44830/consoleFull)** for PR 9383 at commit

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153189866 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10304][SQL] Partition discovery should ...

2015-11-02 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/8840#discussion_r43699404 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -184,7 +204,8 @@ private[sql] object

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153192628 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11195][CORE] Use correct classloader fo...

2015-11-02 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9367#issuecomment-153192578 @choochootrain I think you can write your app in SparkSubmitSuite

[GitHub] spark pull request: [SPARK-11275][SQL][WIP] Rollup and Cube Genera...

2015-11-02 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/9419#issuecomment-153192515 @rick-ibm Will add more comments to explain it. Especially, I will emphasize this design will expect the optimizer collapses these two projections into a single one.

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153195860 **[Test build #44851 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44851/consoleFull)** for PR 9344 at commit

[GitHub] spark pull request: [SPARK-10592] [ML] [PySpark] Deprecate weights...

2015-11-02 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/9311#issuecomment-153200678 Thanks. Merged into master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11343] [ML] Allow float and double pred...

2015-11-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9296 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-10592] [ML] [PySpark] Deprecate weights...

2015-11-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/9311#issuecomment-153201019 @dbtsai The comment on `LogisticRegressionAggregator` is not addressed yet. Could you send a PR to fix it? --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-02 Thread olarayej
Github user olarayej commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-153179661 @sun-rui @felixcheung In types.R, should I then replace: COMPLEX_TYPES <- c("map"=NA, "array"=NA, "struct"=NA) by: COMPLEX_TYPES <-

[GitHub] spark pull request: [SPARK-9319][SPARKR] Add support for setting c...

2015-11-02 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/9218#discussion_r43694097 --- Diff: R/pkg/R/DataFrame.R --- @@ -276,6 +276,75 @@ setMethod("names<-", } }) +#' @rdname columns +#'

[GitHub] spark pull request: [SPARK-9319][SPARKR] Add support for setting c...

2015-11-02 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/9218#discussion_r43694062 --- Diff: R/pkg/R/DataFrame.R --- @@ -276,6 +276,75 @@ setMethod("names<-", } }) +#' @rdname columns +#'

[GitHub] spark pull request: [SPARK-11389][WIP][CORE] Add support for off-h...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153181426 **[Test build #44843 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44843/consoleFull)** for PR 9344 at commit

[GitHub] spark pull request: [SPARK-11389][WIP][CORE] Add support for off-h...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153181438 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10592] [ML] [PySpark] Deprecate weights...

2015-11-02 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/9311#discussion_r43694980 --- Diff: mllib/src/main/scala/org/apache/spark/ml/r/SparkRWrappers.scala --- @@ -51,13 +51,22 @@ private[r] object SparkRWrappers { pipeline.fit(df)

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/9393#issuecomment-153182111 @yhuai This only applies to UDFs. What if a user uses built-in expressions, e.g.: ~~~scala val a = b + c val d = a * a ~~~ It may be useful

[GitHub] spark pull request: [SPARK-10592] [ML] [PySpark] Deprecate weights...

2015-11-02 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/9311#issuecomment-153183047 This looks good to me, but I wonder if changing from `val weights` into `def weights: Vector = coefficients` will cause any binary compatibility issue. --- If your

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9344#discussion_r43696993 --- Diff: core/src/main/scala/org/apache/spark/memory/package.scala --- @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/9422 [SPARK-10429] [SQL] make mutableProjection atomic Right now, SQL's mutable projection updates every value of the mutable project after it evaluates the corresponding expression. This makes the

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153187619 **[Test build #44846 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44846/consoleFull)** for PR 9344 at commit

[GitHub] spark pull request: [SPARK-8402][MLLIB] DP Means Clustering

2015-11-02 Thread yu-iskw
Github user yu-iskw commented on a diff in the pull request: https://github.com/apache/spark/pull/6880#discussion_r43697828 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/DpMeans.scala --- @@ -0,0 +1,279 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-10658][SPARK-11421][PYSPARK][CORE] Prov...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9313#issuecomment-153188257 **[Test build #44848 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44848/consoleFull)** for PR 9313 at commit

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-02 Thread felixcheung
Github user felixcheung commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-153188047 @olarayej from what I gather from our conversation, "map" = NA should be what we want initially. @sun-rui ? --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-02 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/8984#discussion_r43697786 --- Diff: R/pkg/R/types.R --- @@ -0,0 +1,41 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9393#issuecomment-153190509 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153194220 **[Test build #44850 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44850/consoleFull)** for PR 9344 at commit

<    2   3   4   5   6   7   8   9   10   11   >