[GitHub] spark pull request #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-18 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r157683878 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowPythonRunner.scala --- @@ -126,18 +121,14 @@ class ArrowPythonRunner(

[GitHub] spark pull request #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-18 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r157683384 --- Diff: python/pyspark/sql/types.py --- @@ -1679,6 +1678,15 @@ def from_arrow_schema(arrow_schema): for field in arrow_schema])

[GitHub] spark pull request #20014: [SPARK-22827][CORE] Avoid throwing OutOfMemoryErr...

2017-12-18 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/20014#discussion_r157682902 --- Diff: core/src/main/java/org/apache/spark/memory/SparkOutOfMemoryError.java --- @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19884 **[Test build #85099 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85099/testReport)** for PR 19884 at commit

[GitHub] spark issue #20016: SPARK-22830 Scala Coding style has been improved in Spar...

2017-12-18 Thread chetkhatri
Github user chetkhatri commented on the issue: https://github.com/apache/spark/pull/20016 + @holdenk @sameeragarwal --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20016: SPARK-22830 Scala Coding style has been improved in Spar...

2017-12-18 Thread chetkhatri
Github user chetkhatri commented on the issue: https://github.com/apache/spark/pull/20016 @HyukjinKwon I agree with you ! But since Spark is scala project - A lot's of developers refer examples available here and if those are Java Developers they might don't understand that this is

[GitHub] spark pull request #20015: [SPARK-22829] Add new built-in function date_trun...

2017-12-18 Thread gczsjdy
Github user gczsjdy commented on a diff in the pull request: https://github.com/apache/spark/pull/20015#discussion_r157676669 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -1295,87 +1295,184 @@ case class

[GitHub] spark pull request #20015: [SPARK-22829] Add new built-in function date_trun...

2017-12-18 Thread gczsjdy
Github user gczsjdy commented on a diff in the pull request: https://github.com/apache/spark/pull/20015#discussion_r157678588 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -1295,87 +1295,184 @@ case class

[GitHub] spark pull request #20015: [SPARK-22829] Add new built-in function date_trun...

2017-12-18 Thread gczsjdy
Github user gczsjdy commented on a diff in the pull request: https://github.com/apache/spark/pull/20015#discussion_r157680290 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -944,9 +954,16 @@ object DateTimeUtils { date

[GitHub] spark pull request #20015: [SPARK-22829] Add new built-in function date_trun...

2017-12-18 Thread gczsjdy
Github user gczsjdy commented on a diff in the pull request: https://github.com/apache/spark/pull/20015#discussion_r157674840 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -1295,87 +1295,184 @@ case class

[GitHub] spark issue #20016: SPARK-22830 Scala Coding style has been improved in Spar...

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20016 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20016: SPARK-22830 Scala Coding style has been improved ...

2017-12-18 Thread chetkhatri
GitHub user chetkhatri opened a pull request: https://github.com/apache/spark/pull/20016 SPARK-22830 Scala Coding style has been improved in Spark Examples ## What changes were proposed in this pull request? * Under Spark Scala Examples: Some of the syntax were written like

[GitHub] spark pull request #20015: [SPARK-22829] Add new built-in function date_trun...

2017-12-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20015#discussion_r157673626 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -1295,87 +1295,184 @@ case class

[GitHub] spark pull request #20015: [SPARK-22829] Add new built-in function date_trun...

2017-12-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20015#discussion_r157677311 --- Diff: python/pyspark/sql/functions.py --- @@ -,6 +,24 @@ def trunc(date, format): return

[GitHub] spark pull request #19884: [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-18 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r157677769 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowConverters.scala --- @@ -86,21 +86,16 @@ private[sql] object

[GitHub] spark pull request #20015: [SPARK-22829] Add new built-in function date_trun...

2017-12-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20015#discussion_r157677136 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2797,6 +2797,21 @@ object functions { TruncDate(date.expr,

[GitHub] spark pull request #20015: [SPARK-22829] Add new built-in function date_trun...

2017-12-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20015#discussion_r157677400 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2797,6 +2797,21 @@ object functions { TruncDate(date.expr,

[GitHub] spark pull request #20015: [SPARK-22829] Add new built-in function date_trun...

2017-12-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20015#discussion_r157673559 --- Diff: python/pyspark/sql/functions.py --- @@ -,6 +,24 @@ def trunc(date, format): return

[GitHub] spark pull request #20015: [SPARK-22829] Add new built-in function date_trun...

2017-12-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20015#discussion_r157675835 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -1295,87 +1295,184 @@ case class

[GitHub] spark issue #20004: [Spark-22818][SQL] csv escape of quote escape

2017-12-18 Thread ep1804
Github user ep1804 commented on the issue: https://github.com/apache/spark/pull/20004 `CSVOptions` is changed NOT to set `charToEscapeQuoteEscaping` to `\u` by default, to allow the uniVocity parser to use `escape` as `charToEscapeQuoteEscaping` character. ---

[GitHub] spark issue #19975: [SPARK-22781][SS] Support creating streaming dataset wit...

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19975 **[Test build #85098 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85098/testReport)** for PR 19975 at commit

[GitHub] spark issue #20004: [Spark-22818][SQL] csv escape of quote escape

2017-12-18 Thread ep1804
Github user ep1804 commented on the issue: https://github.com/apache/spark/pull/20004 When `charToEscapeQuoteEscaping` is not set and `quote` and `escape` are different, uniVocity parser uses `escape` character as `charToEscapeQuoteEscaping` by default. This is why the test passes.

[GitHub] spark issue #19975: [SPARK-22781][SS] Support creating streaming dataset wit...

2017-12-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19975 Retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19984: [SPARK-22789] Map-only continuous processing execution

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19984 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85089/ Test FAILed. ---

[GitHub] spark issue #19984: [SPARK-22789] Map-only continuous processing execution

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19984 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19984: [SPARK-22789] Map-only continuous processing execution

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19984 **[Test build #85089 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85089/testReport)** for PR 19984 at commit

[GitHub] spark issue #20004: [Spark-22818][SQL] csv escape of quote escape

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20004 **[Test build #85097 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85097/testReport)** for PR 20004 at commit

[GitHub] spark issue #20004: [Spark-22818][SQL] csv escape of quote escape

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20004 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20004: [Spark-22818][SQL] csv escape of quote escape

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20004 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85096/ Test FAILed. ---

[GitHub] spark issue #20004: [Spark-22818][SQL] csv escape of quote escape

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20004 **[Test build #85096 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85096/testReport)** for PR 20004 at commit

[GitHub] spark issue #20004: [Spark-22818][SQL] csv escape of quote escape

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20004 **[Test build #85096 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85096/testReport)** for PR 20004 at commit

[GitHub] spark issue #19149: [SPARK-21652][SQL][FOLLOW-UP] Fix rule conflict caused b...

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19149 **[Test build #85095 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85095/testReport)** for PR 19149 at commit

[GitHub] spark issue #19149: [SPARK-21652][SQL][FOLLOW-UP] Fix rule conflict caused b...

2017-12-18 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19149 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20014: [SPARK-22827][CORE] Avoid throwing OutOfMemoryError in c...

2017-12-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/20014 Overall change lgtm. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20014: [SPARK-22827][CORE] Avoid throwing OutOfMemoryErr...

2017-12-18 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/20014#discussion_r157673852 --- Diff: core/src/main/java/org/apache/spark/memory/SparkOutOfMemoryError.java --- @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19950: [SPARK-22450][Core][MLLib][FollowUp] safely register cla...

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19950 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19950: [SPARK-22450][Core][MLLib][FollowUp] safely register cla...

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19950 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85084/ Test PASSed. ---

[GitHub] spark issue #19950: [SPARK-22450][Core][MLLib][FollowUp] safely register cla...

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19950 **[Test build #85084 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85084/testReport)** for PR 19950 at commit

[GitHub] spark issue #19950: [SPARK-22450][Core][MLLib][FollowUp] safely register cla...

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19950 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85085/ Test PASSed. ---

[GitHub] spark issue #19950: [SPARK-22450][Core][MLLib][FollowUp] safely register cla...

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19950 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19950: [SPARK-22450][Core][MLLib][FollowUp] safely register cla...

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19950 **[Test build #85085 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85085/testReport)** for PR 19950 at commit

[GitHub] spark issue #20012: [SPARK-22824] Restore old offset for binary compatibilit...

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20012 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85088/ Test PASSed. ---

[GitHub] spark issue #20012: [SPARK-22824] Restore old offset for binary compatibilit...

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20012 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20012: [SPARK-22824] Restore old offset for binary compatibilit...

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20012 **[Test build #85088 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85088/testReport)** for PR 20012 at commit

[GitHub] spark pull request #19746: [SPARK-22346][ML] VectorSizeHint Transformer for ...

2017-12-18 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/19746#discussion_r157668450 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/VectorSizeHint.scala --- @@ -0,0 +1,195 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19985: [SPARK-22791] [SQL] [SS] Redact Output of Explain

2017-12-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19985#discussion_r157667492 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/DataSourceScanExecRedactionSuite.scala --- @@ -52,4 +53,40 @@ class

[GitHub] spark issue #20015: [SPARK-22829] Add new built-in function date_trunc()

2017-12-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20015 OK. I am fine if you all guys strongly feel about this. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20015: [SPARK-22829] Add new built-in function date_trunc()

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20015 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85083/ Test PASSed. ---

[GitHub] spark issue #20015: [SPARK-22829] Add new built-in function date_trunc()

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20015 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20015: [SPARK-22829] Add new built-in function date_trunc()

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20015 **[Test build #85083 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85083/testReport)** for PR 20015 at commit

[GitHub] spark issue #20015: [SPARK-22829] Add new built-in function date_trunc()

2017-12-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20015 > after having a discussion with @cloud-fan, @gatorsmile, @rednaxelafx and Reynold Where did the discussion happen? Was this offline discussion? I also want to actively join in the

[GitHub] spark pull request #19717: [SPARK-22646] [Submission] Spark on Kubernetes - ...

2017-12-18 Thread yangw1234
Github user yangw1234 commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r157666054 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark-base/Dockerfile --- @@ -0,0 +1,47 @@ +# +# Licensed to the Apache

[GitHub] spark issue #19954: [SPARK-22757][Kubernetes] Enable use of remote dependenc...

2017-12-18 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/19954 @vanzin @mccheah @foxish Variables names have been shortened in https://github.com/apache/spark/pull/19954/commits/d50c61e46c43049b9ede615ffabd89a55c4adb0c, and traits with only a single

[GitHub] spark issue #19954: [SPARK-22757][Kubernetes] Enable use of remote dependenc...

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19954 **[Test build #85094 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85094/testReport)** for PR 19954 at commit

[GitHub] spark issue #19981: [SPARK-22786][SQL] only use AppStatusPlugin in history s...

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19981 **[Test build #85093 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85093/testReport)** for PR 19981 at commit

[GitHub] spark issue #19984: [SPARK-22789] Map-only continuous processing execution

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19984 **[Test build #85092 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85092/testReport)** for PR 19984 at commit

[GitHub] spark pull request #19981: [SPARK-22786][SQL] only use AppStatusPlugin in hi...

2017-12-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19981#discussion_r157663383 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListenerSuite.scala --- @@ -147,236 +159,246 @@ class SQLListenerSuite

[GitHub] spark issue #19981: [SPARK-22786][SQL] only use AppStatusPlugin in history s...

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19981 **[Test build #85091 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85091/testReport)** for PR 19981 at commit

[GitHub] spark pull request #19981: [SPARK-22786][SQL] only use AppStatusPlugin in hi...

2017-12-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19981#discussion_r157663315 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListenerSuite.scala --- @@ -36,11 +36,12 @@ import

[GitHub] spark issue #19985: [SPARK-22791] [SQL] [SS] Redact Output of Explain

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19985 **[Test build #85090 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85090/testReport)** for PR 19985 at commit

[GitHub] spark pull request #19981: [SPARK-22786][SQL] only use AppStatusPlugin in hi...

2017-12-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19981#discussion_r157662992 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -322,15 +321,18 @@ private[history] class

[GitHub] spark issue #20015: [SPARK-22829] Add new built-in function date_trunc()

2017-12-18 Thread youngbink
Github user youngbink commented on the issue: https://github.com/apache/spark/pull/20015 @HyukjinKwon Just took a look at this PR #14788. My point of mentioning those databases was just to give examples of the function that Spark doesn't support but other databases commonly

[GitHub] spark pull request #19149: [SPARK-21652][SQL][FOLLOW-UP] Fix rule conflict c...

2017-12-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19149#discussion_r157661943 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -47,7 +47,62 @@ abstract class

[GitHub] spark issue #20014: [SPARK-22827][CORE] Avoid throwing OutOfMemoryError in c...

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20014 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20014: [SPARK-22827][CORE] Avoid throwing OutOfMemoryError in c...

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20014 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85081/ Test PASSed. ---

[GitHub] spark issue #20014: [SPARK-22827][CORE] Avoid throwing OutOfMemoryError in c...

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20014 **[Test build #85081 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85081/testReport)** for PR 20014 at commit

[GitHub] spark issue #19904: [SPARK-22707][ML] Optimize CrossValidator memory occupat...

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19904 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19904: [SPARK-22707][ML] Optimize CrossValidator memory occupat...

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19904 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85086/ Test PASSed. ---

[GitHub] spark issue #19904: [SPARK-22707][ML] Optimize CrossValidator memory occupat...

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19904 **[Test build #85086 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85086/testReport)** for PR 19904 at commit

[GitHub] spark issue #20000: [SPARK-22815] [SQL] Keep PromotePrecision in Optimized P...

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/2 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85082/ Test PASSed. ---

[GitHub] spark issue #20000: [SPARK-22815] [SQL] Keep PromotePrecision in Optimized P...

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/2 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20000: [SPARK-22815] [SQL] Keep PromotePrecision in Optimized P...

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/2 **[Test build #85082 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85082/testReport)** for PR 2 at commit

[GitHub] spark issue #19984: [SPARK-22789] Map-only continuous processing execution

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19984 **[Test build #85089 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85089/testReport)** for PR 19984 at commit

[GitHub] spark issue #19892: [SPARK-22797][PySpark] Bucketizer support multi-column

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19892 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19892: [SPARK-22797][PySpark] Bucketizer support multi-column

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19892 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85087/ Test PASSed. ---

[GitHub] spark issue #19892: [SPARK-22797][PySpark] Bucketizer support multi-column

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19892 **[Test build #85087 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85087/testReport)** for PR 19892 at commit

[GitHub] spark issue #19988: [Spark-22795] [ML] Raise error when line search in First...

2017-12-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19988 Hm that's a good point. It means the optimization isn't sure it's making progress anymore because the input is too weird but I guess that doesn't say much about the actual quality of the solution

[GitHub] spark issue #20012: [SPARK-22824] Restore old offset for binary compatibilit...

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20012 **[Test build #85088 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85088/testReport)** for PR 20012 at commit

[GitHub] spark issue #19988: [Spark-22795] [ML] Raise error when line search in First...

2017-12-18 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19988 @srowen Wait... @jkbradley seems to have more thoughts about this: Question: When line search failed, does it mean the model is always meaning-less ? Maybe we need more discussion.

[GitHub] spark issue #19892: [SPARK-22797][PySpark] Bucketizer support multi-column

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19892 **[Test build #85087 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85087/testReport)** for PR 19892 at commit

[GitHub] spark issue #19984: [SPARK-22789] Map-only continuous processing execution

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19984 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19984: [SPARK-22789] Map-only continuous processing execution

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19984 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85079/ Test FAILed. ---

[GitHub] spark issue #19984: [SPARK-22789] Map-only continuous processing execution

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19984 **[Test build #85079 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85079/testReport)** for PR 19984 at commit

[GitHub] spark issue #19904: [SPARK-22707][ML] Optimize CrossValidator memory occupat...

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19904 **[Test build #85086 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85086/testReport)** for PR 19904 at commit

[GitHub] spark pull request #19864: [SPARK-22673][SQL] InMemoryRelation should utiliz...

2017-12-18 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/19864#discussion_r157653784 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/InMemoryColumnarQuerySuite.scala --- @@ -479,4 +485,43 @@ class

[GitHub] spark pull request #19950: [SPARK-22450][Core][MLLib][FollowUp] safely regis...

2017-12-18 Thread zhengruifeng
Github user zhengruifeng commented on a diff in the pull request: https://github.com/apache/spark/pull/19950#discussion_r157653695 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/TreePointSuite.scala --- @@ -0,0 +1,39 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19350: [SPARK-22126][ML][WIP] Fix model-specific optimization s...

2017-12-18 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19350 Design changed. I will create new PR for this later. New design is here https://docs.google.com/document/d/1xw5M4sp1e0eQie75yIt-r6-GTuD5vpFf_I6v-AFBM3M/edit?usp=sharing ---

[GitHub] spark pull request #19350: [SPARK-22126][ML][WIP] Fix model-specific optimiz...

2017-12-18 Thread WeichenXu123
Github user WeichenXu123 closed the pull request at: https://github.com/apache/spark/pull/19350 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19857: [SPARK-22667][ML][WIP] Fix model-specific optimization s...

2017-12-18 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19857 The design of this issue changed. @MrBago will take this over. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19857: [SPARK-22667][ML][WIP] Fix model-specific optimiz...

2017-12-18 Thread WeichenXu123
Github user WeichenXu123 closed the pull request at: https://github.com/apache/spark/pull/19857 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19950: [SPARK-22450][Core][MLLib][FollowUp] safely register cla...

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19950 **[Test build #85085 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85085/testReport)** for PR 19950 at commit

[GitHub] spark issue #20015: [SPARK-22829] Add new built-in function date_trunc()

2017-12-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20015 @cloud-fan and @youngbink how about reviving https://github.com/apache/spark/pull/14788 with a configuration to control this? AWS Redshift seems having `TRUNC` which just converts a

[GitHub] spark issue #19950: [SPARK-22450][Core][MLLib][FollowUp] safely register cla...

2017-12-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19950 **[Test build #85084 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85084/testReport)** for PR 19950 at commit

[GitHub] spark issue #19933: [SPARK-22744][CORE] Add a configuration to show the appl...

2017-12-18 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/19933 Thanks your time to discuss so much with me @srowen. Like MapReduce does, the submit host should be set automatically in submitting logical rather than expecting Spark application owner to set.

[GitHub] spark pull request #19864: [SPARK-22673][SQL] InMemoryRelation should utiliz...

2017-12-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19864#discussion_r157652482 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/InMemoryColumnarQuerySuite.scala --- @@ -479,4 +485,43 @@ class

[GitHub] spark pull request #19985: [SPARK-22791] [SQL] [SS] Redact Output of Explain

2017-12-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19985#discussion_r157652242 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/DataSourceScanExecRedactionSuite.scala --- @@ -52,4 +53,40 @@ class

[GitHub] spark pull request #19985: [SPARK-22791] [SQL] [SS] Redact Output of Explain

2017-12-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19985#discussion_r157652152 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala --- @@ -231,6 +231,13 @@ class QueryExecution(val sparkSession:

[GitHub] spark issue #19985: [SPARK-22791] [SQL] [SS] Redact Output of Explain

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19985 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85077/ Test PASSed. ---

[GitHub] spark issue #19985: [SPARK-22791] [SQL] [SS] Redact Output of Explain

2017-12-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19985 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20008: [SPARK-22822][TEST] Basic tests for FunctionArgum...

2017-12-18 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20008#discussion_r157651436 --- Diff: sql/core/src/test/resources/sql-tests/inputs/typeCoercion/native/decimalPrecision.sql --- @@ -0,0 +1,6883 @@ +-- +-- Licensed to the

  1   2   3   4   5   >