[GitHub] spark issue #20155: [SPARK-22961][REGRESSION] Constant columns should genera...

2018-01-05 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/20155 A late LGTM! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20013: [SPARK-20657][core] Speed up rendering of the sta...

2018-01-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20013#discussion_r159943385 --- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala --- @@ -119,118 +121,115 @@ private class LiveTask( import

[GitHub] spark issue #19848: [SPARK-22162] Executors and the driver should use consis...

2018-01-05 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/19848 Done. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20096: [SPARK-22908] Add kafka source and sink for continuous p...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20096 **[Test build #85728 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85728/testReport)** for PR 20096 at commit

[GitHub] spark pull request #20167: Allow providing Mesos principal & secret via file...

2018-01-05 Thread rvesse
GitHub user rvesse opened a pull request: https://github.com/apache/spark/pull/20167 Allow providing Mesos principal & secret via files (SPARK-16501) ## What changes were proposed in this pull request? This commit modifies the Mesos submission client to allow the principal

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2018-01-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r159955132 --- Diff: core/src/test/scala/org/apache/spark/ThreadAudit.scala --- @@ -0,0 +1,126 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2018-01-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r159954589 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -27,19 +27,55 @@ import org.apache.spark.util.AccumulatorContext /**

[GitHub] spark issue #20166: [SPARK-22973][SQL] Fix incorrect results of Casting Map ...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20166 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20166: [SPARK-22973][SQL] Fix incorrect results of Casting Map ...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20166 **[Test build #85727 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85727/testReport)** for PR 20166 at commit

[GitHub] spark issue #20154: [SPARK-22960][k8s] Make build-push-docker-images.sh more...

2018-01-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20154 ARGs can have default values, so we could do that if we decide to use the Docker Hub infra. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20147: [SPARK-22940][SQL] HiveExternalCatalogVersionsSui...

2018-01-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20147 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20166: [SPARK-22973][SQL] Fix incorrect results of Casting Map ...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20166 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20166: [SPARK-22973][SQL] Fix incorrect results of Casting Map ...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20166 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85726/ Test FAILed. ---

[GitHub] spark issue #20166: [SPARK-22973][SQL] Fix incorrect results of Casting Map ...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20166 **[Test build #85726 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85726/testReport)** for PR 20166 at commit

[GitHub] spark issue #20168: SPARK-22730 Add ImageSchema support for non-integer imag...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20168 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20168: SPARK-22730 Add ImageSchema support for non-integ...

2018-01-05 Thread tomasatdatabricks
GitHub user tomasatdatabricks opened a pull request: https://github.com/apache/spark/pull/20168 SPARK-22730 Add ImageSchema support for non-integer image formats ## What changes were proposed in this pull request? Added functionality to handle all OpenCV modes to ImageSchema:

[GitHub] spark issue #20166: [SPARK-22973][SQL] Fix incorrect results of Casting Map ...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20166 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85727/ Test PASSed. ---

[GitHub] spark issue #20167: Allow providing Mesos principal & secret via files (SPAR...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20167 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20147: [SPARK-22940][SQL] HiveExternalCatalogVersionsSui...

2018-01-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20147#discussion_r159939270 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala --- @@ -85,6 +93,34 @@ class

[GitHub] spark issue #20013: [SPARK-20657][core] Speed up rendering of the stages pag...

2018-01-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20013 > I wonder which changes double the disk usage? It's the new indices, more explicitly the values, not the keys. I tried changing the disk layout to write all the indices in a new namespace

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2018-01-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r159955747 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/SharedSQLContext.scala --- @@ -17,4 +17,22 @@ package org.apache.spark.sql.test

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2018-01-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r159954825 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -27,19 +27,55 @@ import org.apache.spark.util.AccumulatorContext /**

[GitHub] spark issue #20154: [SPARK-22960][k8s] Make build-push-docker-images.sh more...

2018-01-05 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/20154 @vanzin it seems using `ARG` in `FROM` is only supported since Docker version 17.06. It does not work for earlier versions of Docker. See

[GitHub] spark issue #20170: [SPARK-22960][K8S] Revert use of ARG base_image in image...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20170 **[Test build #85734 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85734/testReport)** for PR 20170 at commit

[GitHub] spark issue #20142: [SPARK-22930][PYTHON][SQL] Improve the description of Ve...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20142 **[Test build #85732 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85732/testReport)** for PR 20142 at commit

[GitHub] spark issue #20142: [SPARK-22930][PYTHON][SQL] Improve the description of Ve...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20142 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85732/ Test FAILed. ---

[GitHub] spark issue #20142: [SPARK-22930][PYTHON][SQL] Improve the description of Ve...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20142 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20142: [SPARK-22930][PYTHON][SQL] Improve the descriptio...

2018-01-05 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20142#discussion_r159987954 --- Diff: python/pyspark/sql/tests.py --- @@ -3950,6 +3974,33 @@ def test_vectorized_udf_timestamps_respect_session_timezone(self):

[GitHub] spark issue #11994: [SPARK-14151] Expose metrics Source and Sink interface

2018-01-05 Thread CodingCat
Github user CodingCat commented on the issue: https://github.com/apache/spark/pull/11994 @jerryshao I mean we also need to provide BaseReporter trait ```scala trait Sink { protected val reporter: BaseReporter = createReporter() def createReporter():

[GitHub] spark issue #20169: [SPARK-17088][hive] Fix 'sharesHadoopClasses' option whe...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20169 **[Test build #85733 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85733/testReport)** for PR 20169 at commit

[GitHub] spark pull request #20132: [SPARK-13030][ML] Follow-up cleanups for OneHotEn...

2018-01-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20132 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20142: [SPARK-22930][PYTHON][SQL] Improve the description of Ve...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20142 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20142: [SPARK-22930][PYTHON][SQL] Improve the description of Ve...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20142 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85729/ Test PASSed. ---

[GitHub] spark issue #20142: [SPARK-22930][PYTHON][SQL] Improve the description of Ve...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20142 **[Test build #85729 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85729/testReport)** for PR 20142 at commit

[GitHub] spark pull request #20142: [SPARK-22930][PYTHON][SQL] Improve the descriptio...

2018-01-05 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20142#discussion_r159994714 --- Diff: python/pyspark/sql/tests.py --- @@ -3950,6 +3974,33 @@ def test_vectorized_udf_timestamps_respect_session_timezone(self):

[GitHub] spark issue #20096: [SPARK-22908] Add kafka source and sink for continuous p...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20096 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20096: [SPARK-22908] Add kafka source and sink for continuous p...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20096 **[Test build #85728 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85728/testReport)** for PR 20096 at commit

[GitHub] spark pull request #20096: [SPARK-22908] Add kafka source and sink for conti...

2018-01-05 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20096#discussion_r159983463 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/CachedKafkaConsumer.scala --- @@ -261,6 +261,10 @@ private[kafka010] case

[GitHub] spark issue #20142: [SPARK-22930][PYTHON][SQL] Improve the description of Ve...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20142 **[Test build #85732 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85732/testReport)** for PR 20142 at commit

[GitHub] spark pull request #20169: [SPARK-17088][hive] Fix 'sharesHadoopClasses' opt...

2018-01-05 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/20169 [SPARK-17088][hive] Fix 'sharesHadoopClasses' option when creating client. Because the call to the constructor of HiveClientImpl crosses class loader boundaries, different versions of the same

[GitHub] spark issue #20142: [SPARK-22930][PYTHON][SQL] Improve the description of Ve...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20142 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85731/ Test FAILed. ---

[GitHub] spark issue #20170: [SPARK-22960][K8S] Revert use of ARG base_image in image...

2018-01-05 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20170 LGTM, thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20142: [SPARK-22930][PYTHON][SQL] Improve the description of Ve...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20142 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20142: [SPARK-22930][PYTHON][SQL] Improve the description of Ve...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20142 **[Test build #85731 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85731/testReport)** for PR 20142 at commit

[GitHub] spark pull request #20013: [SPARK-20657][core] Speed up rendering of the sta...

2018-01-05 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20013#discussion_r159992789 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -110,107 +114,240 @@ private[spark] class AppStatusStore( if

[GitHub] spark pull request #20013: [SPARK-20657][core] Speed up rendering of the sta...

2018-01-05 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20013#discussion_r159990658 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -110,107 +114,240 @@ private[spark] class AppStatusStore( if

[GitHub] spark pull request #20013: [SPARK-20657][core] Speed up rendering of the sta...

2018-01-05 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20013#discussion_r159988995 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -110,107 +114,240 @@ private[spark] class AppStatusStore( if

[GitHub] spark issue #20142: [SPARK-22930][PYTHON][SQL] Improve the description of Ve...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20142 **[Test build #85729 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85729/testReport)** for PR 20142 at commit

[GitHub] spark pull request #20097: [SPARK-22912] v2 data source support in MicroBatc...

2018-01-05 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20097#discussion_r159985442 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryManager.scala --- @@ -240,31 +240,35 @@ class StreamingQueryManager

[GitHub] spark issue #20132: [SPARK-13030][ML] Follow-up cleanups for OneHotEncoderEs...

2018-01-05 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/20132 Thanks! Merging with master and branch-2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20142: [SPARK-22930][PYTHON][SQL] Improve the descriptio...

2018-01-05 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/20142#discussion_r159983977 --- Diff: python/pyspark/sql/tests.py --- @@ -3950,6 +3974,33 @@ def test_vectorized_udf_timestamps_respect_session_timezone(self):

[GitHub] spark issue #20142: [SPARK-22930][PYTHON][SQL] Improve the description of Ve...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20142 **[Test build #85735 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85735/testReport)** for PR 20142 at commit

[GitHub] spark pull request #20170: [SPARK-22960][K8S] Revert use of ARG base_image i...

2018-01-05 Thread liyinan926
GitHub user liyinan926 opened a pull request: https://github.com/apache/spark/pull/20170 [SPARK-22960][K8S] Revert use of ARG base_image in images ## What changes were proposed in this pull request? This PR reverts the `ARG base_image` before `FROM` in the images of driver,

[GitHub] spark issue #20170: [SPARK-22960][K8S] Revert use of ARG base_image in image...

2018-01-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20170 LGTM. I'll merge when tests finish even though this doesn't affect them. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20169: [SPARK-17088][hive] Fix 'sharesHadoopClasses' option whe...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20169 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20169: [SPARK-17088][hive] Fix 'sharesHadoopClasses' option whe...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20169 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85733/ Test PASSed. ---

[GitHub] spark issue #20029: [SPARK-22793][SQL]Memory leak in Spark Thrift Server

2018-01-05 Thread liufengdb
Github user liufengdb commented on the issue: https://github.com/apache/spark/pull/20029 lgtm! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20029: [SPARK-22793][SQL]Memory leak in Spark Thrift Server

2018-01-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20029 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20098: [SPARK-22914][DEPLOY] Register history.ui.port

2018-01-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20098 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20135: [SPARK-22937][SQL] SQL elt output binary for bina...

2018-01-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20135 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-05 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160016468 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcReadBenchmark.scala --- @@ -0,0 +1,435 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #20013: [SPARK-20657][core] Speed up rendering of the sta...

2018-01-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20013#discussion_r160004385 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -110,107 +114,240 @@ private[spark] class AppStatusStore( if

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-01-05 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20146 Seems to me we can't set string indexer order for R glm. A workaround is to encode the Species manually first. Then let R glm and spark.glm to fit the encoded Species column, instead of the

[GitHub] spark pull request #20096: [SPARK-22908] Add kafka source and sink for conti...

2018-01-05 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20096#discussion_r159985929 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetReader.scala --- @@ -145,6 +149,19 @@ private[kafka010] class

[GitHub] spark pull request #20096: [SPARK-22908] Add kafka source and sink for conti...

2018-01-05 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20096#discussion_r160008897 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaContinuousSuite.scala --- @@ -0,0 +1,248 @@ +/* + * Licensed to

[GitHub] spark pull request #20096: [SPARK-22908] Add kafka source and sink for conti...

2018-01-05 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20096#discussion_r160007884 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -418,11 +418,16 @@ abstract class StreamExecution(

[GitHub] spark pull request #20096: [SPARK-22908] Add kafka source and sink for conti...

2018-01-05 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20096#discussion_r160006676 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaContinuousWriter.scala --- @@ -0,0 +1,113 @@ +/* + * Licensed to

[GitHub] spark pull request #20096: [SPARK-22908] Add kafka source and sink for conti...

2018-01-05 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20096#discussion_r160004815 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaContinuousReader.scala --- @@ -0,0 +1,224 @@ +/* + * Licensed to

[GitHub] spark pull request #20096: [SPARK-22908] Add kafka source and sink for conti...

2018-01-05 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20096#discussion_r160005666 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriteTask.scala --- @@ -74,8 +56,49 @@ private[kafka010] class

[GitHub] spark issue #20169: [SPARK-17088][hive] Fix 'sharesHadoopClasses' option whe...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20169 **[Test build #85733 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85733/testReport)** for PR 20169 at commit

[GitHub] spark issue #20097: [SPARK-22912] v2 data source support in MicroBatchExecut...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20097 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20097: [SPARK-22912] v2 data source support in MicroBatchExecut...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20097 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85730/ Test PASSed. ---

[GitHub] spark issue #20098: [SPARK-22914][DEPLOY] Register history.ui.port

2018-01-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20098 Merging to master / 2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20135: [SPARK-22937][SQL] SQL elt output binary for binary inpu...

2018-01-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20135 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #17968: [SPARK-9792] Make DenseMatrix equality semantical

2018-01-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17968 cc @WeichenXu123 @yanboliang --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #17968: [SPARK-9792] Make DenseMatrix equality semantical

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17968 **[Test build #85738 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85738/testReport)** for PR 17968 at commit

[GitHub] spark issue #17968: [SPARK-9792] Make DenseMatrix equality semantical

2018-01-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17968 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #17968: [SPARK-9792] Make DenseMatrix equality semantical

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17968 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85738/ Test PASSed. ---

[GitHub] spark issue #20170: [SPARK-22960][K8S] Revert use of ARG base_image in image...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20170 **[Test build #85734 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85734/testReport)** for PR 20170 at commit

[GitHub] spark issue #17968: [SPARK-9792] Make DenseMatrix equality semantical

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17968 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-05 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160016334 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.scala --- @@ -0,0 +1,493 @@ +/* + *

[GitHub] spark issue #20163: [SPARK-22966][PySpark] Spark SQL should handle Python UD...

2018-01-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20163 @ueshin @icexelloss @cloud-fan @rednaxelafx, which one would you prefer? To me, I like 1 at most. If the perf diff is trivial, 2. is also fine. If 3. works fine, I think I am also fine

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160017101 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.scala --- @@ -0,0 +1,493 @@ +/* + *

[GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader

2018-01-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19943#discussion_r160017124 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.scala --- @@ -0,0 +1,493 @@ +/* + *

[GitHub] spark issue #20142: [SPARK-22930][PYTHON][SQL] Improve the description of Ve...

2018-01-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20142 LGTM except for the one minor comment --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20013: [SPARK-20657][core] Speed up rendering of the sta...

2018-01-05 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/20013#discussion_r160017934 --- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala --- @@ -119,118 +121,115 @@ private class LiveTask( import

[GitHub] spark issue #20013: [SPARK-20657][core] Speed up rendering of the stages pag...

2018-01-05 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/20013 lgtm --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20142: [SPARK-22930][PYTHON][SQL] Improve the description of Ve...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20142 **[Test build #85735 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85735/testReport)** for PR 20142 at commit

[GitHub] spark issue #20142: [SPARK-22930][PYTHON][SQL] Improve the description of Ve...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20142 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20142: [SPARK-22930][PYTHON][SQL] Improve the description of Ve...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20142 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85735/ Test PASSed. ---

[GitHub] spark issue #20135: [SPARK-22937][SQL] SQL elt output binary for binary inpu...

2018-01-05 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20135 @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20013: [SPARK-20657][core] Speed up rendering of the stages pag...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20013 **[Test build #85737 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85737/testReport)** for PR 20013 at commit

[GitHub] spark pull request #20170: [SPARK-22960][K8S] Revert use of ARG base_image i...

2018-01-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20170 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20076: [SPARK-21786][SQL] When acquiring 'compressionCodecClass...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20076 **[Test build #85741 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85741/testReport)** for PR 20076 at commit

[GitHub] spark issue #20076: [SPARK-21786][SQL] When acquiring 'compressionCodecClass...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20076 **[Test build #85741 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85741/testReport)** for PR 20076 at commit

[GitHub] spark issue #20076: [SPARK-21786][SQL] When acquiring 'compressionCodecClass...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20076 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85741/ Test FAILed. ---

[GitHub] spark issue #20076: [SPARK-21786][SQL] When acquiring 'compressionCodecClass...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20076 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20168: SPARK-22730 Add ImageSchema support for non-integ...

2018-01-05 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r160016683 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -37,20 +37,51 @@ import org.apache.spark.sql.types._

[GitHub] spark issue #20168: SPARK-22730 Add ImageSchema support for non-integer imag...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20168 **[Test build #85742 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85742/testReport)** for PR 20168 at commit

[GitHub] spark pull request #20142: [SPARK-22930][PYTHON][SQL] Improve the descriptio...

2018-01-05 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20142#discussion_r160017026 --- Diff: python/pyspark/sql/tests.py --- @@ -3567,6 +3580,18 @@ def tearDownClass(cls): time.tzset()

  1   2   3   >