[GitHub] spark pull request #13796: [SPARK-7159][ML] Add multiclass logistic regressi...

2016-08-18 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/13796#discussion_r75349832 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/MultinomialLogisticRegression.scala --- @@ -0,0 +1,611 @@ +/* + * Licensed to the A

[GitHub] spark issue #14680: [SPARK-17101][SQL] Provide consistent format identifiers...

2016-08-18 Thread jaceklaskowski
Github user jaceklaskowski commented on the issue: https://github.com/apache/spark/pull/14680 @rxin @HyukjinKwon Mind reviewing it again and letting me know what you think? I know it's minor but would greatly appreciate having it merged at your earliest convenience. Thanks. --- If y

[GitHub] spark issue #14702: [SPARK-15694] Implement ScriptTransformation in sql/core

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14702 **[Test build #63995 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63995/consoleFull)** for PR 14702 at commit [`9dde09d`](https://github.com/apache/spark/commit/9

[GitHub] spark pull request #14639: [SPARK-17054][SPARKR] SparkR can not run in yarn-...

2016-08-18 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/14639#discussion_r75347400 --- Diff: R/pkg/R/sparkR.R --- @@ -344,6 +344,7 @@ sparkRHive.init <- function(jsc = NULL) { #' @note sparkR.session since 2.0.0 sparkR.session

[GitHub] spark pull request #14702: [SPARK-15694] Implement ScriptTransformation in s...

2016-08-18 Thread tejasapatil
GitHub user tejasapatil opened a pull request: https://github.com/apache/spark/pull/14702 [SPARK-15694] Implement ScriptTransformation in sql/core ## What changes were proposed in this pull request? Added `ScriptTransformationExec` which would run script operator in SQL mod

[GitHub] spark issue #14701: update-test

2016-08-18 Thread taiyangdixia
Github user taiyangdixia commented on the issue: https://github.com/apache/spark/pull/14701 ok --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fe

[GitHub] spark pull request #14701: update-test

2016-08-18 Thread taiyangdixia
Github user taiyangdixia closed the pull request at: https://github.com/apache/spark/pull/14701 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #14701: update-test

2016-08-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14701 Close this PR please? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #14701: update-test

2016-08-18 Thread taiyangdixia
Github user taiyangdixia commented on the issue: https://github.com/apache/spark/pull/14701 wq --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fe

[GitHub] spark issue #14701: update-test

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14701 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #14701: update-test

2016-08-18 Thread taiyangdixia
GitHub user taiyangdixia opened a pull request: https://github.com/apache/spark/pull/14701 update-test ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how t

[GitHub] spark issue #14558: [SPARK-16508][SparkR] Fix warnings on undocumented/dupli...

2016-08-18 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14558 Does having ... in setGeneric forces CRAN checks to require @param there? I think @mengxr point is that so long as the function definition also has ... we could have one @param at the fu

[GitHub] spark issue #14583: [SPARK-16994][SQL] PushDownPredicate should not ignore l...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14583 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14583: [SPARK-16994][SQL] PushDownPredicate should not ignore l...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14583 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63986/ Test PASSed. ---

[GitHub] spark issue #14583: [SPARK-16994][SQL] PushDownPredicate should not ignore l...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14583 **[Test build #63986 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63986/consoleFull)** for PR 14583 at commit [`eea0dfe`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14571: [SPARK-16983][SQL] Add `prettyName` for row_number, dens...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14571 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14571: [SPARK-16983][SQL] Add `prettyName` for row_number, dens...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14571 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63987/ Test PASSed. ---

[GitHub] spark pull request #14639: [SPARK-17054][SPARKR] SparkR can not run in yarn-...

2016-08-18 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14639#discussion_r75344423 --- Diff: R/pkg/R/sparkR.R --- @@ -344,6 +344,7 @@ sparkRHive.init <- function(jsc = NULL) { #' @note sparkR.session since 2.0.0 sparkR.session <-

[GitHub] spark issue #14571: [SPARK-16983][SQL] Add `prettyName` for row_number, dens...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14571 **[Test build #63987 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63987/consoleFull)** for PR 14571 at commit [`3184424`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14527: [SPARK-16938][SQL] `drop/dropDuplicate` should handle th...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14527 **[Test build #63988 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63988/consoleFull)** for PR 14527 at commit [`dd9c369`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14689: [SPARK-17098][SQL] Fix `NullPropagation` optimizer to ha...

2016-08-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14689 Hi, @hvanhovell . Could you review this PR when you have some time? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #14639: [SPARK-17054][SPARKR] SparkR can not run in yarn-cluster...

2016-08-18 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14639 @zjffdu Thanks for clarifying -- I now remember that in the YARN cluster mode there is no `SPARK_HOME` set. However in this case the JVM comes up first and the R process then connects to it. So in

[GitHub] spark issue #14689: [SPARK-17098][SQL] Fix `NullPropagation` optimizer to ha...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14689 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14689: [SPARK-17098][SQL] Fix `NullPropagation` optimizer to ha...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14689 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63985/ Test PASSed. ---

[GitHub] spark issue #14689: [SPARK-17098][SQL] Fix `NullPropagation` optimizer to ha...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14689 **[Test build #63985 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63985/consoleFull)** for PR 14689 at commit [`453775c`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14527: [SPARK-16938][SQL] `drop/dropDuplicate` should handle th...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14527 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63988/ Test PASSed. ---

[GitHub] spark issue #14527: [SPARK-16938][SQL] `drop/dropDuplicate` should handle th...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14527 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14623: [SPARK-17044][SQL] Make test files for window functions ...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14623 **[Test build #63994 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63994/consoleFull)** for PR 14623 at commit [`20604f6`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #14623: [SPARK-17044][SQL] Make test files for window functions ...

2016-08-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14623 Hi, @rxin . Could you review this second try for `window_function.sql`? I simplified the test for our eyeballs. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #14558: [SPARK-16508][SparkR] Fix warnings on undocumented/dupli...

2016-08-18 Thread junyangq
Github user junyangq commented on the issue: https://github.com/apache/spark/pull/14558 Yes, because the generic function and the method are grouped in the same doc and we can only give one description for both of the `...`, we want the actual meaning of them to be same. --- If your

[GitHub] spark pull request #14155: [SPARK-16498][SQL] move hive hack for data source...

2016-08-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14155#discussion_r75339230 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -144,16 +163,172 @@ private[spark] class HiveExternalCatalog(

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...

2016-08-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13680 for 1, currently we only store offsets for var-length element, we should store offset and length as the element content will be word-aligned. --- If your project is set up for it, you can reply t

[GitHub] spark issue #12004: [SPARK-7481][build] [WIP] Add Hadoop 2.6+ spark-cloud mo...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12004 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #12004: [SPARK-7481][build] [WIP] Add Hadoop 2.6+ spark-cloud mo...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12004 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63984/ Test PASSed. ---

[GitHub] spark issue #12004: [SPARK-7481][build] [WIP] Add Hadoop 2.6+ spark-cloud mo...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12004 **[Test build #63984 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63984/consoleFull)** for PR 12004 at commit [`0d9f122`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14079: [SPARK-8425][CORE] New Blacklist Mechanism

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14079 **[Test build #63993 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63993/consoleFull)** for PR 14079 at commit [`cec36c9`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #12790: [SPARK-15018][PYSPARK][ML] Fixed bug causing error if Py...

2016-08-18 Thread MechCoder
Github user MechCoder commented on the issue: https://github.com/apache/spark/pull/12790 If that's the case then the piece of documentation that promises the Pipeline to behave as an identity transformer when no stages are used, has to be changed (removed). --- If your project is se

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/14693 @rxin - I am not sure if there is an easy way to unit test this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark issue #14079: [SPARK-8425][CORE] New Blacklist Mechanism

2016-08-18 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/14079 @kayousterhout @tgravescs Sorry for the long-delay on my end, I have finally updated this. I think I have addressed all the naming / commenting issues -- I'm going to do a bit more testing on it and

[GitHub] spark issue #14693: [SPARK-17113][Shuffle] Job failure due to Executor OOM i...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14693 **[Test build #63991 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63991/consoleFull)** for PR 14693 at commit [`03f7316`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #14079: [SPARK-8425][CORE] New Blacklist Mechanism

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14079 **[Test build #63992 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63992/consoleFull)** for PR 14079 at commit [`b582d8e`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #14079: [SPARK-8425][CORE] New Blacklist Mechanism

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14079 **[Test build #63990 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63990/consoleFull)** for PR 14079 at commit [`a5fbce7`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14079: [SPARK-8425][CORE] New Blacklist Mechanism

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14079 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63990/ Test FAILed. ---

[GitHub] spark issue #14079: [SPARK-8425][CORE] New Blacklist Mechanism

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14079 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #14693: [SPARK-17113][Shuffle] Job failure due to Executo...

2016-08-18 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/14693#discussion_r75333091 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java --- @@ -522,7 +522,7 @@ public long spill() throws IO

[GitHub] spark issue #14079: [SPARK-8425][CORE] New Blacklist Mechanism

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14079 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #14079: [SPARK-8425][CORE] New Blacklist Mechanism

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14079 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63989/ Test FAILed. ---

[GitHub] spark issue #14079: [SPARK-8425][CORE] New Blacklist Mechanism

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14079 **[Test build #63990 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63990/consoleFull)** for PR 14079 at commit [`a5fbce7`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #14079: [SPARK-8425][CORE] New Blacklist Mechanism

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14079 **[Test build #63989 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63989/consoleFull)** for PR 14079 at commit [`f0428b4`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14079: [SPARK-8425][CORE] New Blacklist Mechanism

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14079 **[Test build #63989 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63989/consoleFull)** for PR 14079 at commit [`f0428b4`](https://github.com/apache/spark/commit/f

[GitHub] spark pull request #13796: [SPARK-7159][ML] Add multiclass logistic regressi...

2016-08-18 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/13796#discussion_r75329276 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/MultinomialLogisticRegression.scala --- @@ -0,0 +1,611 @@ +/* + * Licensed to the A

[GitHub] spark issue #14571: [SPARK-16983][SQL] Add `prettyName` for row_number, dens...

2016-08-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14571 Hi, @rxin . Could you review this PR again? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark issue #14527: [SPARK-16938][SQL] `drop/dropDuplicate` should handle th...

2016-08-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14527 Hi, @cloud-fan . Could you review this PR again? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark issue #14527: [SPARK-16938][SQL] `drop/dropDuplicate` should handle th...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14527 **[Test build #63988 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63988/consoleFull)** for PR 14527 at commit [`dd9c369`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #14583: [SPARK-16994][SQL] PushDownPredicate should not ignore l...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14583 **[Test build #63986 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63986/consoleFull)** for PR 14583 at commit [`eea0dfe`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #14571: [SPARK-16983][SQL] Add `prettyName` for row_number, dens...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14571 **[Test build #63987 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63987/consoleFull)** for PR 14571 at commit [`3184424`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #14689: [SPARK-17098][SQL] Fix `NullPropagation` optimizer to ha...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14689 **[Test build #63985 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63985/consoleFull)** for PR 14689 at commit [`453775c`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution by dedupli...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14452 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63983/ Test FAILed. ---

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution by dedupli...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14452 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution by dedupli...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14452 **[Test build #63983 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63983/consoleFull)** for PR 14452 at commit [`5ef961d`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14657: [SPARK-17068][SQL] Make view-usage visible during...

2016-08-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14657#discussion_r75314095 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -693,7 +694,11 @@ case class LocalLi

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-18 Thread mpjlu
Github user mpjlu commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r75314112 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/ChiSqSelector.scala --- @@ -189,11 +228,35 @@ class ChiSqSelector @Since("1.3.0") ( */

[GitHub] spark pull request #14673: [SPARK-15083] [Web UI] History Server can OOM due...

2016-08-18 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/14673#discussion_r75313702 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala --- @@ -93,6 +93,8 @@ class JobProgressListener(conf: SparkConf) extends

[GitHub] spark issue #12004: [SPARK-7481][build] [WIP] Add Hadoop 2.6+ spark-cloud mo...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12004 **[Test build #63984 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63984/consoleFull)** for PR 12004 at commit [`0d9f122`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...

2016-08-18 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13680 @cloud-fan I see. I think that we have already done 1. I will address 2. Am I correct? 1. in the `[offset or primitive values]`, we store the primitive values directly, or store length and of

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r75307704 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/ChiSqSelector.scala --- @@ -189,11 +228,35 @@ class ChiSqSelector @Since("1.3.0") ( */

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution by dedupli...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14452 **[Test build #63983 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63983/consoleFull)** for PR 14452 at commit [`5ef961d`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #14555: [SPARK-16965][MLLIB][PYSPARK] Fix bound checking for Spa...

2016-08-18 Thread adamzjw
Github user adamzjw commented on the issue: https://github.com/apache/spark/pull/14555 Hey, all. One of my colleagues run into a bug related to the discussion above. He called the Vectors.sparse(size: Int, indices: Array[Int], values: Array[Double]) without noticing that the

[GitHub] spark issue #14700: [SPARK-17127]Make unaligned access in unsafe available f...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14700 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63982/ Test FAILed. ---

[GitHub] spark issue #14700: [SPARK-17127]Make unaligned access in unsafe available f...

2016-08-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14700 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14700: [SPARK-17127]Make unaligned access in unsafe available f...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14700 **[Test build #63982 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63982/consoleFull)** for PR 14700 at commit [`24bcf05`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14118: [SPARK-16462][SPARK-16460][SPARK-15144][SQL] Make...

2016-08-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/14118#discussion_r75302086 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala --- @@ -370,7 +370,8 @@ class DataFrameReader private[sql](sparkSession: Sp

[GitHub] spark pull request #14118: [SPARK-16462][SPARK-16460][SPARK-15144][SQL] Make...

2016-08-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/14118#discussion_r75302012 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala --- @@ -370,7 +370,8 @@ class DataFrameReader private[sql](sparkSession: Sp

[GitHub] spark issue #14699: [SPARK-17125][SPARKR] Allow to specify spark config usin...

2016-08-18 Thread zjffdu
Github user zjffdu commented on the issue: https://github.com/apache/spark/pull/14699 Make sense, @felixcheung Thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14118: [SPARK-16462][SPARK-16460][SPARK-15144][SQL] Make CSV ca...

2016-08-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14118 This change looks good to me - I don't see any other reasons that `null` should not be read for `Boolean`, `TimestampType`, `DateType` and `StringType` inconsistently with other types. --- If

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...

2016-08-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13680 see https://github.com/apache/spark/pull/13680#discussion_r74798477 We'd like to change the format to: [numElements] [null bits] [offset or primitive values] [values] in the `[

[GitHub] spark pull request #14229: [SPARK-16447][ML][SparkR] LDA wrapper in SparkR

2016-08-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14229 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #14229: [SPARK-16447][ML][SparkR] LDA wrapper in SparkR

2016-08-18 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14229 LGTM merging. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, o

[GitHub] spark issue #14639: [SPARK-17054][SPARKR] SparkR can not run in yarn-cluster...

2016-08-18 Thread zjffdu
Github user zjffdu commented on the issue: https://github.com/apache/spark/pull/14639 There's no SPARK_HOME in yarn-cluster mode since the R process is in a remote host of the yarn cluster rather than in the client host. --- If your project is set up for it, you can reply to this em

[GitHub] spark issue #13867: [SPARK-16161][SQL] Ambiguous error message for unsupport...

2016-08-18 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13867 @ioana-delaney I lost track of this one. I am sorry about that. Could you update it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as wel

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-18 Thread mpjlu
Github user mpjlu commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r75294150 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/ChiSqSelector.scala --- @@ -189,11 +228,35 @@ class ChiSqSelector @Since("1.3.0") ( */

[GitHub] spark issue #14699: [SPARK-17125][SPARKR] Allow to specify spark config usin...

2016-08-18 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14699 Your example was failing for a different reason: ``` Error in invokeJava(isStatic = TRUE, className, methodName, ...) : java.lang.IllegalArgumentException: spark.executor.instances

[GitHub] spark pull request #14695: [SPARK-17117][SQL] 1 / NULL should not fail analy...

2016-08-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14695 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #14695: [SPARK-17117][SQL] 1 / NULL should not fail analysis

2016-08-18 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14695 LGTM. Merging to master/2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

[GitHub] spark issue #14686: [SPARK-16253][SQL] make spark sql compatible with hive s...

2016-08-18 Thread zenglinxi0615
Github user zenglinxi0615 commented on the issue: https://github.com/apache/spark/pull/14686 Have you tried it on spark 1.6.2? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

[GitHub] spark pull request #14656: [SPARK-17069] Expose spark.range() as table-value...

2016-08-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14656 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #14656: [SPARK-17069] Expose spark.range() as table-valued funct...

2016-08-18 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14656 LGTM. Merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #14643: [SPARK-17057][ML] ProbabilisticClassifierModels' ...

2016-08-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14643#discussion_r75290480 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/ProbabilisticClassifier.scala --- @@ -201,11 +201,18 @@ abstract class ProbabilisticClassif

[GitHub] spark pull request #14384: [Spark-16443][SparkR] Alternating Least Squares (...

2016-08-18 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/14384#discussion_r75290436 --- Diff: R/pkg/R/mllib.R --- @@ -632,3 +642,146 @@ setMethod("predict", signature(object = "AFTSurvivalRegressionModel"), function(object

[GitHub] spark pull request #14667: [SPARK-17087][MESOS] Documentation for Making Spa...

2016-08-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14667 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #14118: [SPARK-16462][SPARK-16460][SPARK-15144][SQL] Make CSV ca...

2016-08-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14118 CC @HyukjinKwon -- WDYT? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14667: [SPARK-17087][MESOS] Documentation for Making Spark on M...

2016-08-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14667 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i

[GitHub] spark issue #14447: [SPARK-16445][MLlib][SparkR] Multilayer Perceptron Class...

2016-08-18 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14447 @keypointt could you please rebase this to master? Only 2 more comments. cc @mengxr --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #14555: [SPARK-16965][MLLIB][PYSPARK] Fix bound checking for Spa...

2016-08-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14555 Last call for @mengxr or @jkbradley -- just want to make sure one's OK. Certainly good for correctness. --- If your project is set up for it, you can reply to this email and have your reply appear o

[GitHub] spark issue #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector based ...

2016-08-18 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/14597 Hi @srowen , I will update the Python API to match this changes. Now, the current Python API is not conflict with the changes. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #14700: [SPARK-17127]Make unaligned access in unsafe available f...

2016-08-18 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14700 @yimuxi did you test this on AArch64? It is impossible for us to test this on the current testing infra. --- If your project is set up for it, you can reply to this email and have your reply app

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r75289596 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/ChiSqSelector.scala --- @@ -189,11 +228,35 @@ class ChiSqSelector @Since("1.3.0") ( */

[GitHub] spark pull request #14447: [SPARK-16445][MLlib][SparkR] Multilayer Perceptro...

2016-08-18 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/14447#discussion_r75289453 --- Diff: mllib/src/main/scala/org/apache/spark/ml/r/MultilayerPerceptronClassifierWrapper.scala --- @@ -0,0 +1,135 @@ +/* + * Licensed to the A

[GitHub] spark issue #14700: [SPARK-17127]Make unaligned access in unsafe available f...

2016-08-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14700 **[Test build #63982 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63982/consoleFull)** for PR 14700 at commit [`24bcf05`](https://github.com/apache/spark/commit/2

[GitHub] spark pull request #14447: [SPARK-16445][MLlib][SparkR] Multilayer Perceptro...

2016-08-18 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/14447#discussion_r75289396 --- Diff: R/pkg/R/mllib.R --- @@ -533,6 +630,28 @@ setMethod("write.ml", signature(object = "KMeansModel", path = "character"), invisibl

<    1   2   3   4   5   6   7   >