[GitHub] spark pull request #19861: [SPARK-22387][SQL] Propagate session configs to d...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19861#discussion_r157140363 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Utils.scala --- @@ -0,0 +1,58 @@ +/* + * Licensed to

[GitHub] spark issue #19861: [SPARK-22387][SQL] Propagate session configs to data sou...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19861 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84941/ Test PASSed. ---

[GitHub] spark issue #19861: [SPARK-22387][SQL] Propagate session configs to data sou...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19861 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19861: [SPARK-22387][SQL] Propagate session configs to data sou...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19861 **[Test build #84941 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84941/testReport)** for PR 19861 at commit

[GitHub] spark pull request #19981: [SPARK-22786][SQL] only use AppStatusPlugin in hi...

2017-12-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19981#discussion_r157137789 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala --- @@ -82,6 +82,19 @@ private[sql] class SharedState(val

[GitHub] spark issue #19987: [Test][WIP] add passing test.

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19987 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84944/ Test PASSed. ---

[GitHub] spark issue #19987: [Test][WIP] add passing test.

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19987 **[Test build #84944 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84944/testReport)** for PR 19987 at commit

[GitHub] spark issue #19987: [Test][WIP] add passing test.

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19987 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19986: [Test][WIP] add failing test.

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19986 **[Test build #84945 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84945/testReport)** for PR 19986 at commit

[GitHub] spark issue #19986: [Test][WIP] add failing test.

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19986 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84945/ Test FAILed. ---

[GitHub] spark issue #19986: [Test][WIP] add failing test.

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19986 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19981: [SPARK-22786][SQL] only use AppStatusPlugin in hi...

2017-12-14 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/19981#discussion_r157136615 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala --- @@ -82,6 +82,19 @@ private[sql] class SharedState(val

[GitHub] spark pull request #19981: [SPARK-22786][SQL] only use AppStatusPlugin in hi...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19981#discussion_r157136099 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListenerSuite.scala --- @@ -489,16 +501,17 @@ private case class

[GitHub] spark pull request #19981: [SPARK-22786][SQL] only use AppStatusPlugin in hi...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19981#discussion_r157135896 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListenerSuite.scala --- @@ -142,286 +163,277 @@ class SQLListenerSuite

[GitHub] spark pull request #19941: [SPARK-22753][SQL] Get rid of dataSource.writeAnd...

2017-12-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19941 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19981: [SPARK-22786][SQL] only use AppStatusPlugin in hi...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19981#discussion_r157135580 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListenerSuite.scala --- @@ -142,286 +163,277 @@ class SQLListenerSuite

[GitHub] spark issue #19941: [SPARK-22753][SQL] Get rid of dataSource.writeAndRead

2017-12-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19941 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19981: [SPARK-22786][SQL] only use AppStatusPlugin in hi...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19981#discussion_r157135495 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListenerSuite.scala --- @@ -142,286 +163,277 @@ class SQLListenerSuite

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-14 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r157135337 --- Diff: sql/core/src/main/java/org/apache/spark/sql/streaming/Trigger.java --- @@ -95,4 +96,57 @@ public static Trigger ProcessingTime(String interval) {

[GitHub] spark pull request #19985: [SPARK-22791] [SQL] [SS] Redact Output of Explain

2017-12-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19985#discussion_r157135279 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala --- @@ -418,6 +418,35 @@ class StreamSuite extends StreamTest {

[GitHub] spark pull request #19981: [SPARK-22786][SQL] only use AppStatusPlugin in hi...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19981#discussion_r157135252 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListenerSuite.scala --- @@ -36,14 +36,23 @@ import

[GitHub] spark issue #19986: [Test][WIP] add failing test.

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19986 **[Test build #84945 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84945/testReport)** for PR 19986 at commit

[GitHub] spark issue #19985: [SPARK-22791] [SQL] [SS] Redact Output of Explain

2017-12-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19985 cc @marmbrus @hvanhovell @zsxwing --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19987: [Test][WIP] add passing test.

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19987 **[Test build #84944 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84944/testReport)** for PR 19987 at commit

[GitHub] spark issue #19985: [SPARK-22791] [SQL] [SS] Redact Output of Explain

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19985 **[Test build #84946 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84946/testReport)** for PR 19985 at commit

[GitHub] spark pull request #19981: [SPARK-22786][SQL] only use AppStatusPlugin in hi...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19981#discussion_r157135091 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala --- @@ -82,6 +82,19 @@ private[sql] class SharedState(val sparkContext:

[GitHub] spark pull request #19987: [Test][WIP] add passing test.

2017-12-14 Thread MrBago
GitHub user MrBago opened a pull request: https://github.com/apache/spark/pull/19987 [Test][WIP] add passing test. missing import in python/pyspark/ml/tests.py, verifying with CI. You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request #19985: [SPARK-22791] [SQL] [SS] Redact Output of Explain

2017-12-14 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/19985 [SPARK-22791] [SQL] [SS] Redact Output of Explain ## What changes were proposed in this pull request? When calling explain on a query, the output can contain sensitive information. We

[GitHub] spark pull request #19986: [Test][WIP] add failing test.

2017-12-14 Thread MrBago
GitHub user MrBago opened a pull request: https://github.com/apache/spark/pull/19986 [Test][WIP] add failing test. missing import in python/pyspark/ml/tests.py, verifying with CI. You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request #19982: [SPARK-22787] [TEST] [SQL] Add a TPC-H query suit...

2017-12-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19982 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19981: [SPARK-22786][SQL] only use AppStatusPlugin in hi...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19981#discussion_r157134291 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusStore.scala --- @@ -25,21 +25,17 @@ import

[GitHub] spark issue #19981: [SPARK-22786][SQL] only use AppStatusPlugin in history s...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19981 **[Test build #84943 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84943/testReport)** for PR 19981 at commit

[GitHub] spark issue #19982: [SPARK-22787] [TEST] [SQL] Add a TPC-H query suite

2017-12-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19982 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19946: [SPARK-22648] [Scheduler] Spark on Kubernetes - D...

2017-12-14 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19946#discussion_r157133964 --- Diff: docs/running-on-kubernetes.md --- @@ -0,0 +1,502 @@ +--- +layout: global +title: Running Spark on Kubernetes +--- +* This will

[GitHub] spark issue #19982: [SPARK-22787] [TEST] [SQL] Add a TPC-H query suite

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19982 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84940/ Test PASSed. ---

[GitHub] spark issue #19982: [SPARK-22787] [TEST] [SQL] Add a TPC-H query suite

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19982 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19982: [SPARK-22787] [TEST] [SQL] Add a TPC-H query suite

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19982 **[Test build #84940 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84940/testReport)** for PR 19982 at commit

[GitHub] spark issue #19941: [SPARK-22753][SQL] Get rid of dataSource.writeAndRead

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19941 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84939/ Test PASSed. ---

[GitHub] spark issue #19941: [SPARK-22753][SQL] Get rid of dataSource.writeAndRead

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19941 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19941: [SPARK-22753][SQL] Get rid of dataSource.writeAndRead

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19941 **[Test build #84939 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84939/testReport)** for PR 19941 at commit

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r157130281 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala --- @@ -36,13 +36,14 @@ import

[GitHub] spark issue #19864: [SPARK-22673][SQL] InMemoryRelation should utilize exist...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19864 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84938/ Test PASSed. ---

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r157129866 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala --- @@ -118,309 +142,286 @@ class SQLListenerSuite extends

[GitHub] spark issue #19864: [SPARK-22673][SQL] InMemoryRelation should utilize exist...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19864 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19864: [SPARK-22673][SQL] InMemoryRelation should utilize exist...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19864 **[Test build #84938 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84938/testReport)** for PR 19864 at commit

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r157129912 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala --- @@ -118,309 +142,286 @@ class SQLListenerSuite extends

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r157129757 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala --- @@ -118,309 +142,286 @@ class SQLListenerSuite extends

[GitHub] spark issue #19949: [SPARK-22762][TEST] Basic tests for IfCoercion and CaseW...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19949 **[Test build #84942 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84942/testReport)** for PR 19949 at commit

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r157127908 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/GeneratedProjectionSuite.scala --- @@ -219,4 +219,31 @@ class

[GitHub] spark pull request #19977: [SPARK-22771][SQL] Concatenate binary inputs into...

2017-12-14 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19977#discussion_r157125671 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -50,15 +51,23 @@ import

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19884 Just for a refreshing reminder about Jenkins, I happened to check what we have in Jenkins roughly a month ago (just simply by printing out the versions within PySpark tests) in a specific

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r157124515 --- Diff: sql/core/src/main/java/org/apache/spark/sql/streaming/Trigger.java --- @@ -95,4 +96,57 @@ public static Trigger ProcessingTime(String

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r157124505 --- Diff: sql/core/src/main/java/org/apache/spark/sql/streaming/Trigger.java --- @@ -95,4 +96,57 @@ public static Trigger ProcessingTime(String

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r157124394 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1035,6 +1035,22 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2017-12-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r157124263 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1035,6 +1035,22 @@ object SQLConf { .booleanConf

[GitHub] spark issue #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0

2017-12-14 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19884 @wesm Yes, I'd like to use it asap to verify this patch and to confirm the behavior of my PR #18754 for `DecimalType` support. Thanks. ---

[GitHub] spark issue #19861: [SPARK-22387][SQL] Propagate session configs to data sou...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19861 **[Test build #84941 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84941/testReport)** for PR 19861 at commit

[GitHub] spark pull request #19977: [SPARK-22771][SQL] Concatenate binary inputs into...

2017-12-14 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19977#discussion_r157122909 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -50,15 +51,23 @@ import

[GitHub] spark pull request #19865: [SPARK-22668][SQL] Ensure no global variables in ...

2017-12-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19865#discussion_r157122860 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -217,6 +217,18 @@ class CodegenContext {

[GitHub] spark pull request #19949: [SPARK-22762][TEST] Basic tests for IfCoercion an...

2017-12-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19949#discussion_r157122725 --- Diff: sql/core/src/test/resources/sql-tests/inputs/typeCoercion/native/caseWhenCoercion.sql --- @@ -0,0 +1,200 @@ +-- +-- Licensed to the

[GitHub] spark issue #19949: [SPARK-22762][TEST] Basic tests for IfCoercion and CaseW...

2017-12-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19949 LGTM except the comment about `short` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19977: [SPARK-22771][SQL] Concatenate binary inputs into...

2017-12-14 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19977#discussion_r157122586 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -50,15 +51,23 @@ import

[GitHub] spark pull request #19977: [SPARK-22771][SQL] Concatenate binary inputs into...

2017-12-14 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19977#discussion_r157122613 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -50,15 +51,23 @@ import

[GitHub] spark pull request #19977: [SPARK-22771][SQL] Concatenate binary inputs into...

2017-12-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19977#discussion_r157122520 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -50,15 +51,23 @@ import

[GitHub] spark pull request #19977: [SPARK-22771][SQL] Concatenate binary inputs into...

2017-12-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19977#discussion_r157122430 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -50,15 +51,23 @@ import

[GitHub] spark issue #19982: [SPARK-22787] [TEST] [SQL] Add a TPC-H query suite

2017-12-14 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19982 yea, sure. I have much bandwidth now :) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19982: [SPARK-22787] [TEST] [SQL] Add a TPC-H query suite

2017-12-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19982 cc @maropu Feel free to submit a PR for adding SSB --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #19946: [SPARK-22648] [Scheduler] Spark on Kubernetes - D...

2017-12-14 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/19946#discussion_r157120443 --- Diff: docs/running-on-kubernetes.md --- @@ -0,0 +1,502 @@ +--- +layout: global +title: Running Spark on Kubernetes +--- +* This

[GitHub] spark issue #19982: [SPARK-22787] [TEST] [SQL] Add a TPC-H query suite

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19982 **[Test build #84940 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84940/testReport)** for PR 19982 at commit

[GitHub] spark issue #19773: [SPARK-22546][SQL] Supporting for changing column dataTy...

2017-12-14 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/19773 gental ping @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19982: [SPARK-22787] [TEST] [SQL] Add a TPC-H query suit...

2017-12-14 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19982#discussion_r157119974 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/BenchmarkQueryTest.scala --- @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19982: [SPARK-22787] [TEST] [SQL] Add a TPC-H query suit...

2017-12-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19982#discussion_r157119941 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/BenchmarkQueryTest.scala --- @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19982: [SPARK-22787] [TEST] [SQL] Add a TPC-H query suit...

2017-12-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19982#discussion_r157119768 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/BenchmarkQueryTest.scala --- @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19941: [SPARK-22753][SQL] Get rid of dataSource.writeAndRead

2017-12-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19941 LGTM pending Jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19941: [SPARK-22753][SQL] Get rid of dataSource.writeAndRead

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19941 **[Test build #84939 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84939/testReport)** for PR 19941 at commit

[GitHub] spark pull request #19941: [SPARK-22753][SQL] Get rid of dataSource.writeAnd...

2017-12-14 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/19941#discussion_r157118985 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala --- @@ -67,8 +67,9 @@ case class

[GitHub] spark pull request #19350: [SPARK-22126][ML][WIP] Fix model-specific optimiz...

2017-12-14 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/19350#discussion_r157118663 --- Diff: mllib/src/main/scala/org/apache/spark/ml/Estimator.scala --- @@ -82,5 +86,49 @@ abstract class Estimator[M <: Model[M]] extends

[GitHub] spark pull request #19864: [SPARK-22673][SQL] InMemoryRelation should utiliz...

2017-12-14 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/19864#discussion_r157118091 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -80,6 +80,14 @@ class CacheManager extends Logging {

[GitHub] spark pull request #19961: [SPARK-22496][SQL] thrift server adds operation l...

2017-12-14 Thread ChenjunZou
Github user ChenjunZou commented on a diff in the pull request: https://github.com/apache/spark/pull/19961#discussion_r157117909 --- Diff: sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/operation/ExecuteStatementOperation.java --- @@ -67,4 +68,16 @@ protected

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19041 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84935/ Test PASSed. ---

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19041 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19041 **[Test build #84935 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84935/testReport)** for PR 19041 at commit

[GitHub] spark issue #19864: [SPARK-22673][SQL] InMemoryRelation should utilize exist...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19864 **[Test build #84938 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84938/testReport)** for PR 19864 at commit

[GitHub] spark issue #19946: [SPARK-22648] [Scheduler] Spark on Kubernetes - Document...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19946 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19946: [SPARK-22648] [Scheduler] Spark on Kubernetes - Document...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19946 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84934/ Test PASSed. ---

[GitHub] spark issue #19946: [SPARK-22648] [Scheduler] Spark on Kubernetes - Document...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19946 **[Test build #84934 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84934/testReport)** for PR 19946 at commit

[GitHub] spark pull request #19977: [SPARK-22771][SQL] Concatenate binary inputs into...

2017-12-14 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19977#discussion_r157116687 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -50,15 +51,23 @@ import

[GitHub] spark issue #19982: [SPARK-22787] [TEST] [SQL] Add a TPC-H query suite

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19982 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19982: [SPARK-22787] [TEST] [SQL] Add a TPC-H query suite

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19982 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84936/ Test PASSed. ---

[GitHub] spark issue #19982: [SPARK-22787] [TEST] [SQL] Add a TPC-H query suite

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19982 **[Test build #84936 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84936/testReport)** for PR 19982 at commit

[GitHub] spark issue #19751: [SPARK-20653][core] Add cleaning of old elements from th...

2017-12-14 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/19751 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #16486: [SPARK-13610][ML] Create a Transformer to disasse...

2017-12-14 Thread leonfl
Github user leonfl closed the pull request at: https://github.com/apache/spark/pull/16486 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19975: [SPARK-22781][SS] Support creating streaming dataset wit...

2017-12-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19975 Also, @brkyvz . Could you review this PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19904: [SPARK-22707][ML] Optimize CrossValidator memory occupat...

2017-12-14 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19904 I discussed with @MrBago offline, I make a summary for what I thought now: I give 3 approaches which we can compare, after discussion I realized none of them is ideal, we have to make a

[GitHub] spark pull request #19977: [SPARK-22771][SQL] Concatenate binary inputs into...

2017-12-14 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19977#discussion_r157113499 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -50,15 +51,23 @@ import

[GitHub] spark pull request #19982: [SPARK-22787] [TEST] [SQL] Add a TPC-H query suit...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19982#discussion_r157113164 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/TPCHQuerySuite.scala --- @@ -0,0 +1,122 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19848: [SPARK-22162] Executors and the driver should use consis...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19848 I found Spark SQL always use 0 as job id... How hadoop committers work with job id? only for recovery? --- - To unsubscribe,

[GitHub] spark issue #19977: [SPARK-22771][SQL] Concatenate binary inputs into a bina...

2017-12-14 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19977 checked; ``` hive> create table t1(a string, b string); hive> create view v1 as select a || b from t1; hive> describe v1; OK _c0 string

[GitHub] spark pull request #19848: [SPARK-22162] Executors and the driver should use...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19848#discussion_r157111981 --- Diff: core/src/main/scala/org/apache/spark/internal/io/SparkHadoopWriter.scala --- @@ -60,17 +60,17 @@ object SparkHadoopWriter extends Logging {

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-12-14 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19811 For some types rarely seen, I think we don't need to make them in array for now because we might have just few elements there. ---

  1   2   3   4   5   6   7   8   >