[GitHub] spark pull request #14723: [SQL][WIP][Test] Supports object-based aggregatio...

2016-08-22 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/14723#discussion_r75622002 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/AggregateWithObjectAggregateBufferSuite.scala --- @@ -0,0 +1,156 @@ +/* + * Licensed to the

[GitHub] spark pull request #14723: [SQL][WIP][Test] Supports object-based aggregatio...

2016-08-22 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/14723#discussion_r75622250 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/interfaces.scala --- @@ -389,3 +389,89 @@ abstract class

[GitHub] spark pull request #14723: [SQL][WIP][Test] Supports object-based aggregatio...

2016-08-22 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/14723#discussion_r75622311 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/SortBasedAggregationIterator.scala --- @@ -90,6 +98,21 @@ class

[GitHub] spark issue #14723: [SQL][WIP][Test] Supports object-based aggregation funct...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14723 **[Test build #64179 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64179/consoleFull)** for PR 14723 at commit

[GitHub] spark issue #14723: [SQL][WIP][Test] Supports object-based aggregation funct...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14723 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14723: [SQL][WIP][Test] Supports object-based aggregation funct...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14723 **[Test build #64179 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64179/consoleFull)** for PR 14723 at commit

[GitHub] spark issue #14723: [SQL][WIP][Test] Supports object-based aggregation funct...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14723 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64179/ Test FAILed. ---

[GitHub] spark issue #14723: [SQL][WIP][Test] Supports object-based aggregation funct...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14723 **[Test build #64180 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64180/consoleFull)** for PR 14723 at commit

[GitHub] spark issue #14531: [SPARK-16943] [SPARK-16942] [SQL] Fix multiple bugs in C...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14531 **[Test build #64176 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64176/consoleFull)** for PR 14531 at commit

[GitHub] spark issue #14521: [SPARK-16935] [SQL] Verification of Function-related Ext...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14521 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64177/ Test PASSed. ---

[GitHub] spark issue #14521: [SPARK-16935] [SQL] Verification of Function-related Ext...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14521 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14745: [SPARK-16896][SQL] Handle duplicated field names in head...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14745 **[Test build #64178 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64178/consoleFull)** for PR 14745 at commit

[GitHub] spark issue #14745: [SPARK-16896][SQL] Handle duplicated field names in head...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14745 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14745: [SPARK-16896][SQL] Handle duplicated field names in head...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14745 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64178/ Test PASSed. ---

[GitHub] spark pull request #14746: [SPARK-17180] [SQL] Fix View Resolution Order in ...

2016-08-22 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/14746 [SPARK-17180] [SQL] Fix View Resolution Order in ALTER VIEW AS SELECT ### What changes were proposed in this pull request? In the current master branch, when users do not specify the

[GitHub] spark issue #14746: [SPARK-17180] [SQL] Fix View Resolution Order in ALTER V...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14746 **[Test build #64182 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64182/consoleFull)** for PR 14746 at commit

[GitHub] spark issue #14729: [SPARK-17167] [SQL] Issue Exceptions when Analyze Table ...

2016-08-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14729 @hvanhovell Not related to this PR. But I would like to ask that do we need to support temporary tables in `ANALYZE TABLE`? Because temporary tables (and views) are actually represented by logical

[GitHub] spark issue #14642: [SPARK-17056][Core] Fix a wrong assert regarding unroll ...

2016-08-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14642 ping @cloud-fan @JoshRosen since you have modified this file, do you think this change make sense? Thank you. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #14388: [SPARK-16362][SQL] Support ArrayType and StructType in v...

2016-08-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14388 @maver1ck Any results about the test? Thank you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14572: [SPARK-16552] [FOLLOW-UP] [SQL] Store the Inferred Schem...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14572 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14723: [SQL][WIP][Test] Supports object-based aggregation funct...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14723 **[Test build #64180 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64180/consoleFull)** for PR 14723 at commit

[GitHub] spark issue #14572: [SPARK-16552] [FOLLOW-UP] [SQL] Store the Inferred Schem...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14572 **[Test build #64175 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64175/consoleFull)** for PR 14572 at commit

[GitHub] spark issue #14723: [SQL][WIP][Test] Supports object-based aggregation funct...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14723 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14723: [SQL][WIP][Test] Supports object-based aggregation funct...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14723 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64180/ Test FAILed. ---

[GitHub] spark issue #14572: [SPARK-16552] [FOLLOW-UP] [SQL] Store the Inferred Schem...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14572 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64175/ Test PASSed. ---

[GitHub] spark issue #14531: [SPARK-16943] [SPARK-16942] [SQL] Fix multiple bugs in C...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14531 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14531: [SPARK-16943] [SPARK-16942] [SQL] Fix multiple bugs in C...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14531 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64176/ Test PASSed. ---

[GitHub] spark issue #14521: [SPARK-16935] [SQL] Verification of Function-related Ext...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14521 **[Test build #64177 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64177/consoleFull)** for PR 14521 at commit

[GitHub] spark issue #14625: [SPARK-17045] [SQL] Build/move Join-related test cases i...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14625 **[Test build #64181 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64181/consoleFull)** for PR 14625 at commit

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-08-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/13775 ping @yhuai Any chance you can review this? Thank you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #14537: [SPARK-16948][SQL] Querying empty partitioned orc tables...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14537 **[Test build #64183 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64183/consoleFull)** for PR 14537 at commit

[GitHub] spark issue #14659: [SPARK-16757] Set up Spark caller context to HDFS

2016-08-22 Thread Sherry302
Github user Sherry302 commented on the issue: https://github.com/apache/spark/pull/14659 Hi, @cnauroth Thank you very much for the review and suggestion. I have removed the spaces in the value of the caller context, and prepended "Spark" instead (refer to the commit

[GitHub] spark pull request #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-22 Thread winningsix
Github user winningsix commented on a diff in the pull request: https://github.com/apache/spark/pull/8880#discussion_r75631172 --- Diff: core/src/main/scala/org/apache/spark/crypto/CryptoConf.scala --- @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #14625: [SPARK-17045] [SQL] Build/move Join-related test cases i...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14625 **[Test build #64181 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64181/consoleFull)** for PR 14625 at commit

[GitHub] spark issue #14738: [MINOR][ML]Add expert param support to SharedParamsCodeG...

2016-08-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14738 OK, let's link this to the original JIRA then. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14625: [SPARK-17045] [SQL] Build/move Join-related test cases i...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14625 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64181/ Test FAILed. ---

[GitHub] spark issue #14625: [SPARK-17045] [SQL] Build/move Join-related test cases i...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14625 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14700: [SPARK-17127]Make unaligned access in unsafe available f...

2016-08-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14700 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #14747: [SPARK-17086] Fix an issue in QuantileDiscretizer

2016-08-22 Thread VinceShieh
GitHub user VinceShieh opened a pull request: https://github.com/apache/spark/pull/14747 [SPARK-17086] Fix an issue in QuantileDiscretizer ## What changes were proposed in this pull request? In cases when QuantileDiscretizerSuite is called upon a numeric array with

[GitHub] spark issue #14683: [SPARK-16968]Document additional options in jdbc Writer

2016-08-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14683 Merged to master as well --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14738: [SPARK-17090][MINOR][ML]Add expert param support to Shar...

2016-08-22 Thread hqzizania
Github user hqzizania commented on the issue: https://github.com/apache/spark/pull/14738 It is linked to SPARK-17090 now. @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #13704: [SPARK-15985][SQL] Eliminate redundant cast from an arra...

2016-08-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13704 @liancheng Could you please review this? cc: @cloud-fan , we are waiting for @liancheng 's review for a long time. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #11956: [SPARK-14098][SQL] Generate Java code that gets a float/...

2016-08-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/11956 @davies Would it be possible to share your opinions regarding these design questions among us? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #14723: [SQL][WIP][Test] Supports object-based aggregatio...

2016-08-22 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14723#discussion_r75632667 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/interfaces.scala --- @@ -389,3 +389,126 @@ abstract class

[GitHub] spark pull request #14700: [SPARK-17127]Make unaligned access in unsafe avai...

2016-08-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14700 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #14683: [SPARK-16968]Document additional options in jdbc ...

2016-08-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14683 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14747: [SPARK-17086][MLLIB] Fix an issue in QuantileDiscretizer

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14747 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14747: [SPARK-17086][MLLIB] Fix an issue in QuantileDiscretizer

2016-08-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14747 Hm, does `.distinct` guarantee that the ordering is preserved? It looks like it generally happens to. I guess we could add `.sorted` to make sure; it wouldn't be expensive, especially if it's

[GitHub] spark issue #14747: [SPARK-17086][MLLIB] Fix an issue in QuantileDiscretizer

2016-08-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14747 Jenkins add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix an issue in QuantileDiscretizer

2016-08-22 Thread VinceShieh
Github user VinceShieh commented on the issue: https://github.com/apache/spark/pull/14747 yes, the output from approxQuantile is a sorted array. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #14155: [SPARK-16498][SQL] move hive hack for data source...

2016-08-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14155#discussion_r75633327 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -200,22 +375,77 @@ private[spark] class

[GitHub] spark pull request #14738: [SPARK-17090][MINOR][ML]Add expert param support ...

2016-08-22 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/14738#discussion_r75633356 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/shared/SharedParamsCodeGen.scala --- @@ -167,11 +173,11 @@ private[shared] object

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix an issue in QuantileDiscretizer

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14747 **[Test build #64184 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64184/consoleFull)** for PR 14747 at commit

[GitHub] spark pull request #14738: [SPARK-17090][MINOR][ML]Add expert param support ...

2016-08-22 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/14738#discussion_r75633532 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/shared/sharedParams.scala --- @@ -397,13 +397,13 @@ private[ml] trait HasAggregationDepth

[GitHub] spark pull request #12889: [SPARK-15113][PySpark][ML] Add missing num featur...

2016-08-22 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/12889#discussion_r75633544 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -788,6 +788,8 @@ class

[GitHub] spark issue #12889: [SPARK-15113][PySpark][ML] Add missing num features num ...

2016-08-22 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/12889 jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix an issue in QuantileDiscretizer

2016-08-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14747 No, my question is whether it remains sorted after `.distinct`. We'd need that, but I'm not sure it's guaranteed. It's easy to add `.sorted` to guarantee it. --- If your project is set up for it,

[GitHub] spark issue #14659: [SPARK-16757] Set up Spark caller context to HDFS

2016-08-22 Thread Sherry302
Github user Sherry302 commented on the issue: https://github.com/apache/spark/pull/14659 Hi, @steveloughran Thanks a lot for the comments. In the audit log, if users set some configuration in spark-defaults.conf like `spark.eventLog.dir hdfs://localhost:9000/spark-history`,

[GitHub] spark pull request #14744: [SPARKR][SPARKSUBMIT] Allow to set sparkr shell c...

2016-08-22 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/14744#discussion_r75633858 --- Diff: docs/configuration.md --- @@ -1752,6 +1752,15 @@ showDF(properties, numRows = 200, truncate = FALSE) Executable for executing R scripts

[GitHub] spark issue #12889: [SPARK-15113][PySpark][ML] Add missing num features num ...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12889 **[Test build #64185 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64185/consoleFull)** for PR 12889 at commit

[GitHub] spark pull request #14692: [SPARK-17115] [SQL] decrease the threshold when s...

2016-08-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14692 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r75634212 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/ChiSqSelector.scala --- @@ -54,6 +54,29 @@ private[feature] trait ChiSqSelectorParams extends

[GitHub] spark issue #14659: [SPARK-16757] Set up Spark caller context to HDFS

2016-08-22 Thread Sherry302
Github user Sherry302 commented on the issue: https://github.com/apache/spark/pull/14659 Thanks a lot for adding me as “contributor” in Hadoop :) @steveloughran @cnauroth --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #14692: [SPARK-17115] [SQL] decrease the threshold when split ex...

2016-08-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14692 thanks, merging to master/2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #14738: [SPARK-17090][MINOR][ML]Add expert param support ...

2016-08-22 Thread hqzizania
Github user hqzizania commented on a diff in the pull request: https://github.com/apache/spark/pull/14738#discussion_r75634235 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/shared/SharedParamsCodeGen.scala --- @@ -167,11 +173,11 @@ private[shared] object

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r75634371 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/ChiSqSelector.scala --- @@ -91,8 +130,16 @@ final class ChiSqSelector @Since("1.6.0")

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r75634305 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/ChiSqSelector.scala --- @@ -54,6 +55,29 @@ private[feature] trait ChiSqSelectorParams extends

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r75634441 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/ChiSqSelector.scala --- @@ -69,21 +73,22 @@ class ChiSqSelectorModel @Since("1.3.0") (

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r75634631 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/ChiSqSelector.scala --- @@ -32,17 +32,21 @@ import org.apache.spark.rdd.RDD import

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-22 Thread mpjlu
Github user mpjlu commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r75634659 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/ChiSqSelector.scala --- @@ -54,6 +54,29 @@ private[feature] trait ChiSqSelectorParams extends

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r75634791 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/ChiSqSelector.scala --- @@ -91,8 +130,16 @@ final class ChiSqSelector @Since("1.6.0")

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix an issue in QuantileDiscretizer

2016-08-22 Thread VinceShieh
Github user VinceShieh commented on the issue: https://github.com/apache/spark/pull/14747 it seems Array.distinct will not break the sequence of the elements. But, you are right, we need guarantee the array is sorted. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r75634953 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/ChiSqSelector.scala --- @@ -189,11 +227,20 @@ class ChiSqSelector @Since("1.3.0") ( */

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r75635041 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/ChiSqSelector.scala --- @@ -189,11 +227,20 @@ class ChiSqSelector @Since("1.3.0") ( */

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r75634988 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/ChiSqSelector.scala --- @@ -189,11 +227,20 @@ class ChiSqSelector @Since("1.3.0") ( */

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r75635056 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/ChiSqSelector.scala --- @@ -189,11 +227,20 @@ class ChiSqSelector @Since("1.3.0") ( */

[GitHub] spark issue #14738: [SPARK-17090][MINOR][ML]Add expert param support to Shar...

2016-08-22 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/14738 It's better to update the PR title as ```[SPARK-17090][FOLLOW-UP][ML] ... ```. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix an issue in QuantileDiscretizer

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14747 **[Test build #64186 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64186/consoleFull)** for PR 14747 at commit

[GitHub] spark pull request #14747: [SPARK-17086][ML] Fix an issue in QuantileDiscret...

2016-08-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14747#discussion_r75635220 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala --- @@ -52,6 +52,25 @@ class QuantileDiscretizerSuite

[GitHub] spark issue #14747: [SPARK-17086][ML] Fix an issue in QuantileDiscretizer

2016-08-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14747 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #14715: [SPARK-17085] [Streaming] [Documentation and actual code...

2016-08-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14715 Merged to master/2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #14715: [SPARK-17085] [Streaming] [Documentation and actu...

2016-08-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14715 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14688: [SPARK-17095] [Documentation] [Latex and Scala doc do no...

2016-08-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14688 @jagadeesanas2 can you follow up on this to address other instances, including the one you opened the other PR for? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #14747: [SPARK-17086][ML] Fix an issue in QuantileDiscret...

2016-08-22 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/14747#discussion_r75636343 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala --- @@ -114,7 +114,7 @@ final class QuantileDiscretizer

[GitHub] spark issue #14663: [SPARK-17001] [ML] Enable standardScaler to standardize ...

2016-08-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14663 going once, going twice. This would simply let an operation proceed where it errored before, at the cost of giving a user a little more rope to hang him/herself. I think it unblocks a legitimate and

[GitHub] spark pull request #14747: [SPARK-17086][ML] Fix an issue in QuantileDiscret...

2016-08-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14747#discussion_r75636483 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala --- @@ -114,7 +114,7 @@ final class QuantileDiscretizer @Since("1.6.0")

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-22 Thread mpjlu
Github user mpjlu commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r75636510 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/ChiSqSelector.scala --- @@ -189,11 +227,20 @@ class ChiSqSelector @Since("1.3.0") ( */

[GitHub] spark pull request #14735: [SPARK-17173][SPARKR] R MLlib refactor, cleanup, ...

2016-08-22 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/14735#discussion_r75636716 --- Diff: R/pkg/R/mllib.R --- @@ -1027,7 +1009,7 @@ setMethod("spark.gaussianMixture", signature(data = "SparkDataFrame", formula = #' @export

[GitHub] spark issue #14239: [SPARK-16593] [CORE] Provide a pre-fetch mechanism to ac...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14239 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-22 Thread hqzizania
Github user hqzizania commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r75689943 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/ChiSqSelector.scala --- @@ -171,14 +180,47 @@ object ChiSqSelectorModel extends

[GitHub] spark issue #14649: [SPARK-17059][SQL] Allow FileFormat to specify partition...

2016-08-22 Thread andreweduffy
Github user andreweduffy commented on the issue: https://github.com/apache/spark/pull/14649 cc @davies @cloud-fan as well --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #10896: [SPARK-12978][SQL] Skip unnecessary final group-by when ...

2016-08-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/10896 @hvanhovell fixed. plz check again? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #10896: [SPARK-12978][SQL] Skip unnecessary final group-by when ...

2016-08-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/10896 **[Test build #64200 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64200/consoleFull)** for PR 10896 at commit

[GitHub] spark issue #10896: [SPARK-12978][SQL] Skip unnecessary final group-by when ...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/10896 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64200/ Test PASSed. ---

[GitHub] spark issue #10896: [SPARK-12978][SQL] Skip unnecessary final group-by when ...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/10896 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #10896: [SPARK-12978][SQL] Skip unnecessary final group-b...

2016-08-22 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/10896#discussion_r75693169 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/Aggregate.scala --- @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #14738: [SPARK-17090][FOLLOW-UP][ML]Add expert param supp...

2016-08-22 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/14738#discussion_r75689013 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/shared/SharedParamsCodeGen.scala --- @@ -153,6 +154,11 @@ private[shared] object

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-22 Thread hqzizania
Github user hqzizania commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r75689230 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/ChiSqSelector.scala --- @@ -171,14 +180,47 @@ object ChiSqSelectorModel extends

[GitHub] spark issue #14738: [SPARK-17090][FOLLOW-UP][ML]Add expert param support to ...

2016-08-22 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/14738 LGTM except the last minor issue. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14239: [SPARK-16593] [CORE] Provide a pre-fetch mechanism to ac...

2016-08-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14239 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64207/ Test FAILed. ---

  1   2   3   4   5   6   7   8   >