[GitHub] spark issue #20045: [Spark-22360][SQL][TEST] Add unit tests for Window Speci...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20045 **[Test build #86267 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86267/testReport)** for PR 20045 at commit

[GitHub] spark issue #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* for UDFs ...

2018-01-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20288 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86264/ Test PASSed. ---

[GitHub] spark issue #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* for UDFs ...

2018-01-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20288 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* for UDFs ...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20288 **[Test build #86264 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86264/testReport)** for PR 20288 at commit

[GitHub] spark pull request #20211: [SPARK-23011][PYTHON][SQL] Prepend missing groupi...

2018-01-17 Thread icexelloss
Github user icexelloss closed the pull request at: https://github.com/apache/spark/pull/20211 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20211: [SPARK-23011][PYTHON][SQL] Prepend missing grouping colu...

2018-01-17 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/20211 Yeah my bad. I am going to close this PR but I will keep that in mind for future reference. Thanks! --- - To unsubscribe,

[GitHub] spark issue #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* for UDFs ...

2018-01-17 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/20288 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* fo...

2018-01-17 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20288#discussion_r162089110 --- Diff: python/pyspark/sql/catalog.py --- @@ -224,92 +224,20 @@ def dropGlobalTempView(self, viewName): """

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-17 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/20203 yeah I think its probably fine to update the executors page that way -- lets at least see how it looks. Would the list include all stages ever blacklisted? Only those stages still running? The

[GitHub] spark pull request #20203: [SPARK-22577] [core] executor page blacklist stat...

2018-01-17 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20203#discussion_r162087946 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetBlacklist.scala --- @@ -128,13 +130,17 @@ private[scheduler] class TaskSetBlacklist(val

[GitHub] spark issue #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* for UDFs ...

2018-01-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20288 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86266/ Test FAILed. ---

[GitHub] spark issue #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* for UDFs ...

2018-01-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20288 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* for UDFs ...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20288 **[Test build #86266 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86266/testReport)** for PR 20288 at commit

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-17 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/20203 I think it would be nice to add something on the executors page as well. Perthaps we can put something like "for stage[, ,...]" if its only blacklisted for those tasksets. But I'm also

[GitHub] spark issue #20100: [SPARK-22913][SQL] Improved Hive Partition Pruning

2018-01-17 Thread ameent
Github user ameent commented on the issue: https://github.com/apache/spark/pull/20100 CCing @cloud-fan @tdas @HyukjinKwon @xubo245 I need help finding someone who can provide feedback on this pull request. This change reduces run-time of one of our use cases from 6 minutes to

[GitHub] spark issue #20294: [SPARK-23132][PYTHON][ML] Run doctests in ml.image when ...

2018-01-17 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/20294 nice fix! Great to see the doctests run. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20294: [SPARK-23132][PYTHON][ML] Run doctests in ml.imag...

2018-01-17 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/20294#discussion_r162082063 --- Diff: python/pyspark/ml/image.py --- @@ -194,9 +194,9 @@ def readImages(self, path, recursive=False, numPartitions=-1, :return: a

[GitHub] spark issue #20276: [SPARK-14948][SQL] disambiguate attributes in join condi...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20276 **[Test build #86278 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86278/testReport)** for PR 20276 at commit

[GitHub] spark issue #20276: [SPARK-14948][SQL] disambiguate attributes in join condi...

2018-01-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20276 > Is this only going to fix the case of joining DataFrames derived from the same DataFrame but not for self-joining? Yes. I think self-join is not fixable, `df.join(df, df("id") ===

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20023 **[Test build #86277 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86277/testReport)** for PR 20023 at commit

[GitHub] spark issue #20269: [SPARK-23029] [DOCS] Specifying default units of configu...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20269 **[Test build #4057 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4057/testReport)** for PR 20269 at commit

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20023 **[Test build #86276 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86276/testReport)** for PR 20023 at commit

[GitHub] spark issue #20216: [SPARK-23024][WEB-UI]Spark ui about the contents of the ...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20216 **[Test build #4056 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4056/testReport)** for PR 20216 at commit

[GitHub] spark pull request #20121: [SPARK-22882][ML][TESTS] ML test for structured s...

2018-01-17 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20121#discussion_r162052224 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala --- @@ -29,15 +29,14 @@ import

[GitHub] spark pull request #20121: [SPARK-22882][ML][TESTS] ML test for structured s...

2018-01-17 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20121#discussion_r162053757 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/MultilayerPerceptronClassifierSuite.scala --- @@ -75,11 +71,9 @@ class

[GitHub] spark pull request #20121: [SPARK-22882][ML][TESTS] ML test for structured s...

2018-01-17 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20121#discussion_r162037613 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/GBTClassifierSuite.scala --- @@ -169,59 +171,28 @@ class GBTClassifierSuite extends

[GitHub] spark pull request #20121: [SPARK-22882][ML][TESTS] ML test for structured s...

2018-01-17 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20121#discussion_r162058792 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/OneVsRestSuite.scala --- @@ -17,14 +17,13 @@ package

[GitHub] spark pull request #20121: [SPARK-22882][ML][TESTS] ML test for structured s...

2018-01-17 Thread smurakozi
Github user smurakozi commented on a diff in the pull request: https://github.com/apache/spark/pull/20121#discussion_r162051926 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/MultilayerPerceptronClassifierSuite.scala --- @@ -17,22 +17,18 @@ package

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2018-01-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r162064252 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/DecimalType.scala --- @@ -136,10 +137,52 @@ object DecimalType extends AbstractDataType

[GitHub] spark issue #20276: [SPARK-14948][SQL] disambiguate attributes in join condi...

2018-01-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20276 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20276: [SPARK-14948][SQL] disambiguate attributes in join condi...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20276 **[Test build #86275 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86275/testReport)** for PR 20276 at commit

[GitHub] spark issue #20276: [SPARK-14948][SQL] disambiguate attributes in join condi...

2018-01-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20276 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86275/ Test FAILed. ---

[GitHub] spark issue #20276: [SPARK-14948][SQL] disambiguate attributes in join condi...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20276 **[Test build #86275 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86275/testReport)** for PR 20276 at commit

[GitHub] spark issue #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* for UDFs ...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20288 **[Test build #86274 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86274/testReport)** for PR 20288 at commit

[GitHub] spark issue #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* for UDFs ...

2018-01-17 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20288 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* for UDFs ...

2018-01-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20288 Just fixed minor doc nits and double checked the built API documentation. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* for UDFs ...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20288 **[Test build #86273 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86273/testReport)** for PR 20288 at commit

[GitHub] spark pull request #20265: [SPARK-21783][SQL] Turn on ORC filter push-down b...

2018-01-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20265 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20265: [SPARK-21783][SQL] Turn on ORC filter push-down by defau...

2018-01-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20265 thanks, merging to master/2.3! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20292: [SPARK-23129][CORE] Make deserializeStream of Dis...

2018-01-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20292#discussion_r162054765 --- Diff: core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala --- @@ -547,9 +550,9 @@ class ExternalAppendOnlyMap[K, V, C](

[GitHub] spark pull request #20292: [SPARK-23129][CORE] Make deserializeStream of Dis...

2018-01-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20292#discussion_r162054687 --- Diff: core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala --- @@ -463,21 +463,21 @@ class ExternalAppendOnlyMap[K, V, C](

[GitHub] spark pull request #20292: [SPARK-23129][CORE] Make deserializeStream of Dis...

2018-01-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20292#discussion_r162054413 --- Diff: core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala --- @@ -463,21 +463,21 @@ class ExternalAppendOnlyMap[K, V, C](

[GitHub] spark issue #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* for UDFs ...

2018-01-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20288 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20023 **[Test build #86272 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86272/testReport)** for PR 20023 at commit

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2018-01-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r162053815 --- Diff: sql/core/src/test/resources/sql-tests/inputs/decimals.sql --- @@ -0,0 +1,53 @@ +-- +-- Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #20287: [SPARK-23121][WEB-UI] When the Spark Streaming app is ru...

2018-01-17 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20287 This is far from a fatal bug. The page doesn't render, but it should give some kind of error anyway. What change are you referring to? In any event, this change is unsuitable. ---

[GitHub] spark issue #18143: [SPARK-20919][SS] Simplificaiton of CachedKafkaConsumer ...

2018-01-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18143 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18143: [SPARK-20919][SS] Simplificaiton of CachedKafkaConsumer ...

2018-01-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18143 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86270/ Test PASSed. ---

[GitHub] spark issue #18143: [SPARK-20919][SS] Simplificaiton of CachedKafkaConsumer ...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18143 **[Test build #86270 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86270/testReport)** for PR 18143 at commit

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2018-01-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r162052996 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/DecimalType.scala --- @@ -136,10 +137,52 @@ object DecimalType extends AbstractDataType {

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2018-01-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r162052779 --- Diff: sql/core/src/test/resources/sql-tests/inputs/decimals.sql --- @@ -0,0 +1,53 @@ +-- +-- Licensed to the Apache Software Foundation

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2018-01-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r162051884 --- Diff: sql/core/src/test/resources/sql-tests/inputs/decimals.sql --- @@ -0,0 +1,53 @@ +-- +-- Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20023 LGTM. One thing we can improve is the golden file test framework. I found we sometimes repeat the test cases with a config on and off. We should write the test cases once and list the

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2018-01-17 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r162051255 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/DecimalPrecision.scala --- @@ -242,18 +278,36 @@ object DecimalPrecision

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2018-01-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r162051075 --- Diff: sql/core/src/test/resources/sql-tests/inputs/decimals.sql --- @@ -0,0 +1,53 @@ +-- +-- Licensed to the Apache Software Foundation

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2018-01-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r162050678 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/DecimalType.scala --- @@ -136,10 +137,52 @@ object DecimalType extends AbstractDataType

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2018-01-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r162050062 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/DecimalPrecision.scala --- @@ -242,18 +278,36 @@ object DecimalPrecision

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2018-01-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r162049618 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/DecimalPrecision.scala --- @@ -242,18 +278,36 @@ object DecimalPrecision

[GitHub] spark issue #20019: [SPARK-22361][SQL][TEST] Add unit test for Window Frames

2018-01-17 Thread gaborgsomogyi
Github user gaborgsomogyi commented on the issue: https://github.com/apache/spark/pull/20019 @gatorsmile @jiangxb1987 @smurakozi Thanks for the help! --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2018-01-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r162049277 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/DecimalPrecision.scala --- @@ -93,41 +97,76 @@ object DecimalPrecision

[GitHub] spark pull request #20023: [SPARK-22036][SQL] Decimal multiplication with hi...

2018-01-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20023#discussion_r162048685 --- Diff: docs/sql-programming-guide.md --- @@ -1795,6 +1795,11 @@ options. - Since Spark 2.3, when all inputs are binary, SQL `elt()` returns

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2018-01-17 Thread gaborgsomogyi
Github user gaborgsomogyi commented on the issue: https://github.com/apache/spark/pull/19893 @vanzin @squito @srowen @jiangxb1987 @henryr Big thanks to everybody for the constructive comments, learned a lot from them. I'll take a look at further possibilities like the

[GitHub] spark issue #20257: [SPARK-23048][ML] Add OneHotEncoderEstimator document an...

2018-01-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20257 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86269/ Test PASSed. ---

[GitHub] spark issue #20257: [SPARK-23048][ML] Add OneHotEncoderEstimator document an...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20257 **[Test build #86269 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86269/testReport)** for PR 20257 at commit

[GitHub] spark issue #20257: [SPARK-23048][ML] Add OneHotEncoderEstimator document an...

2018-01-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20257 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20294: [SPARK-23132][PYTHON][ML] Run doctests in ml.image when ...

2018-01-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20294 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86268/ Test PASSed. ---

[GitHub] spark issue #20294: [SPARK-23132][PYTHON][ML] Run doctests in ml.image when ...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20294 **[Test build #86268 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86268/testReport)** for PR 20294 at commit

[GitHub] spark issue #20294: [SPARK-23132][PYTHON][ML] Run doctests in ml.image when ...

2018-01-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20294 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregation func...

2018-01-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19872 Only few nits. LGTM but let me leave it to @ueshin. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregati...

2018-01-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19872#discussion_r162047655 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -360,9 +369,23 @@ abstract class SparkStrategies extends

[GitHub] spark pull request #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregati...

2018-01-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19872#discussion_r162047640 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala --- @@ -39,18 +38,20 @@ object

[GitHub] spark pull request #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregati...

2018-01-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19872#discussion_r162047620 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala --- @@ -27,7 +27,6 @@ import

[GitHub] spark pull request #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregati...

2018-01-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19872#discussion_r162047382 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -333,16 +339,19 @@ abstract class SparkStrategies extends

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20023 **[Test build #86271 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86271/testReport)** for PR 20023 at commit

[GitHub] spark pull request #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregati...

2018-01-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19872#discussion_r162047047 --- Diff: python/pyspark/sql/tests.py --- @@ -4279,6 +4272,425 @@ def test_unsupported_types(self):

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-17 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20023 the test failure is unrelated. A proof is that I just updated the migration section in the last commit and the previous one was passing all the tests. @cloud-fan @gatorsmile any more

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-17 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20023 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregati...

2018-01-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19872#discussion_r162046471 --- Diff: python/pyspark/sql/functions.py --- @@ -2214,6 +2216,37 @@ def pandas_udf(f=None, returnType=None, functionType=None): ..

[GitHub] spark issue #18143: [SPARK-20919][SS] Simplificaiton of CachedKafkaConsumer ...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18143 **[Test build #86270 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86270/testReport)** for PR 18143 at commit

[GitHub] spark issue #20211: [SPARK-23011][PYTHON][SQL] Prepend missing grouping colu...

2018-01-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20211 Yea, at least to me, PR description is helpful a lot when I actually review something or track the history :). --- - To

[GitHub] spark pull request #18143: [SPARK-20919][SS] Simplificaiton of CachedKafkaCo...

2018-01-17 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/18143#discussion_r162045340 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/CachedKafkaConsumer.scala --- @@ -45,9 +46,6 @@ private[kafka010] case

[GitHub] spark pull request #19993: [SPARK-22799][ML] Bucketizer should throw excepti...

2018-01-17 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19993#discussion_r162044954 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -249,6 +250,27 @@ object ParamValidators { def

[GitHub] spark issue #20257: [SPARK-23048][ML] Add OneHotEncoderEstimator document an...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20257 **[Test build #86269 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86269/testReport)** for PR 20257 at commit

[GitHub] spark issue #20294: [SPARK-23132][PYTHON][ML] Run doctests in ml.image when ...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20294 **[Test build #86268 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86268/testReport)** for PR 20294 at commit

[GitHub] spark pull request #20257: [SPARK-23048][ML] Add OneHotEncoderEstimator docu...

2018-01-17 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20257#discussion_r162043736 --- Diff: docs/ml-features.md --- @@ -783,11 +783,11 @@ Because this existing `OneHotEncoder` is a stateless transformer, it is not usab ##

[GitHub] spark pull request #19993: [SPARK-22799][ML] Bucketizer should throw excepti...

2018-01-17 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/19993#discussion_r162043704 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -249,6 +250,27 @@ object ParamValidators { def

[GitHub] spark pull request #20294: [SPARK-23132][PYTHON][ML] Run doctests in ml.imag...

2018-01-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20294#discussion_r162043164 --- Diff: python/pyspark/ml/image.py --- @@ -194,9 +194,9 @@ def readImages(self, path, recursive=False, numPartitions=-1, :return: a

[GitHub] spark issue #20294: [SPARK-23132][PYTHON][ML] Run doctests in ml.image when ...

2018-01-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20294 cc @MrBago and @imatiach-msft, could you take a look when you are available please? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20294: [SPARK-23132][PYTHON][ML] Run doctests in ml.imag...

2018-01-17 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/20294 [SPARK-23132][PYTHON][ML] Run doctests in ml.image when testing ## What changes were proposed in this pull request? This PR proposes to actually run the doctests in `ml/image.py`.

[GitHub] spark pull request #19993: [SPARK-22799][ML] Bucketizer should throw excepti...

2018-01-17 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/19993#discussion_r162042318 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -249,6 +250,27 @@ object ParamValidators { def

[GitHub] spark issue #20184: [SPARK-22987][Core] UnsafeExternalSorter cases OOM when ...

2018-01-17 Thread liutang123
Github user liutang123 commented on the issue: https://github.com/apache/spark/pull/20184 hi, @jerryshao , I try lazily allocate all the InputStream and byte arr in UnsafeSorterSpillReader. And would you please look at this when you have time? ---

[GitHub] spark pull request #20203: [SPARK-22577] [core] executor page blacklist stat...

2018-01-17 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20203#discussion_r162041751 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetBlacklist.scala --- @@ -128,13 +130,17 @@ private[scheduler] class TaskSetBlacklist(val

[GitHub] spark issue #20288: [SPARK-23122][PYTHON][SQL] Deprecate register* for UDFs ...

2018-01-17 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20288 LGTM pending Jenkins. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20023 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20023 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86260/ Test FAILed. ---

[GitHub] spark issue #20257: [SPARK-23048][ML] Add OneHotEncoderEstimator document an...

2018-01-17 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/20257 A couple minor comments, otherwise looks fine. I see we are changing the example names, so effectively removing the old examples. I'm ok with this, unless others have an objection? ---

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20023 **[Test build #86260 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86260/testReport)** for PR 20023 at commit

[GitHub] spark pull request #20257: [SPARK-23048][ML] Add OneHotEncoderEstimator docu...

2018-01-17 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/20257#discussion_r162040939 --- Diff: docs/ml-features.md --- @@ -783,11 +783,11 @@ Because this existing `OneHotEncoder` is a stateless transformer, it is not usab ##

[GitHub] spark issue #20265: [SPARK-21783][SQL] Turn on ORC filter push-down by defau...

2018-01-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20265 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86257/ Test PASSed. ---

[GitHub] spark issue #20265: [SPARK-21783][SQL] Turn on ORC filter push-down by defau...

2018-01-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20265 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

<    1   2   3   4   5   6   7   >