[GitHub] spark issue #19218: [SPARK-21786][SQL] The 'spark.sql.parquet.compression.co...

2017-09-15 Thread fjh100456
Github user fjh100456 commented on the issue: https://github.com/apache/spark/pull/19218 @dongjoon-hyun Thank you very much, I'll fix it now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-15 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r139278875 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,307 @@ case class

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-15 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r139278847 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -503,69 +504,307 @@ case class

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-09-15 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r139278830 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1179,6 +1179,26 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark issue #19230: [SPARK-22003][SQL] support array column in vectorized re...

2017-09-15 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19230 @liufengdb The PR description looks like an end-to-end failure. I'm curious are you facing the failure in an end-to-end case? ---

[GitHub] spark pull request #19230: [SPARK-22003][SQL] support array column in vector...

2017-09-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19230#discussion_r139278205 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnVectorSuite.scala --- @@ -0,0 +1,202 @@ +/* + * Licensed to the

[GitHub] spark issue #19239: [SPARK-22017] Take minimum of all watermark execs in Str...

2017-09-15 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/19239 thank you @joseph-torres merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19239: [SPARK-22017] Take minimum of all watermark execs...

2017-09-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19239 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19239: [SPARK-22017] Take minimum of all watermark execs in Str...

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19239 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81837/ Test PASSed. ---

[GitHub] spark issue #19239: [SPARK-22017] Take minimum of all watermark execs in Str...

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19239 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19239: [SPARK-22017] Take minimum of all watermark execs in Str...

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19239 **[Test build #81837 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81837/testReport)** for PR 19239 at commit

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-15 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/16578 > * There is a minor change needed though, "parquetFormat: ParquetFileFormat" should be replaced by "fileFormat: FileFormat" as there is no dependency on the actual ParquetFileFormat class defined

[GitHub] spark issue #19219: [SPARK-21993][SQL][WIP] Close sessionState when finish

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19219 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19219: [SPARK-21993][SQL][WIP] Close sessionState when finish

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19219 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81838/ Test FAILed. ---

[GitHub] spark issue #19219: [SPARK-21993][SQL][WIP] Close sessionState when finish

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19219 **[Test build #81838 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81838/testReport)** for PR 19219 at commit

[GitHub] spark issue #19219: [SPARK-21993][SQL][WIP] Close sessionState when finish

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19219 **[Test build #81838 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81838/testReport)** for PR 19219 at commit

[GitHub] spark issue #17862: [SPARK-20602] [ML]Adding LBFGS optimizer and Squared_hin...

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17862 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19230: [SPARK-22003][SQL] support array column in vectorized re...

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19230 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81835/ Test PASSed. ---

[GitHub] spark issue #17862: [SPARK-20602] [ML]Adding LBFGS optimizer and Squared_hin...

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17862 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81836/ Test PASSed. ---

[GitHub] spark issue #19230: [SPARK-22003][SQL] support array column in vectorized re...

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19230 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17862: [SPARK-20602] [ML]Adding LBFGS optimizer and Squared_hin...

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17862 **[Test build #81836 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81836/testReport)** for PR 17862 at commit

[GitHub] spark issue #19230: [SPARK-22003][SQL] support array column in vectorized re...

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19230 **[Test build #81835 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81835/testReport)** for PR 19230 at commit

[GitHub] spark issue #19239: [SPARK-22017] Take minimum of all watermark execs in Str...

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19239 **[Test build #81837 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81837/testReport)** for PR 19239 at commit

[GitHub] spark issue #17862: [SPARK-20602] [ML]Adding LBFGS optimizer and Squared_hin...

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17862 **[Test build #81836 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81836/testReport)** for PR 17862 at commit

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Python Vectorized UDFs

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18659 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Python Vectorized UDFs

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18659 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81834/ Test FAILed. ---

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Python Vectorized UDFs

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18659 **[Test build #81834 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81834/testReport)** for PR 18659 at commit

[GitHub] spark pull request #19230: [SPARK-22003][SQL] support array column in vector...

2017-09-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19230#discussion_r139272552 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnVector.java --- @@ -16,6 +16,7 @@ */ package

[GitHub] spark issue #19180: [SPARK-21967][CORE] org.apache.spark.unsafe.types.UTF8St...

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19180 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81831/ Test PASSed. ---

[GitHub] spark issue #19180: [SPARK-21967][CORE] org.apache.spark.unsafe.types.UTF8St...

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19180 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19180: [SPARK-21967][CORE] org.apache.spark.unsafe.types.UTF8St...

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19180 **[Test build #81831 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81831/testReport)** for PR 19180 at commit

[GitHub] spark issue #19239: [SPARK-22017] Take minimum of all watermark execs in Str...

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19239 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19239: [SPARK-22017] Take minimum of all watermark execs in Str...

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19239 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81832/ Test PASSed. ---

[GitHub] spark issue #19239: [SPARK-22017] Take minimum of all watermark execs in Str...

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19239 **[Test build #81832 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81832/testReport)** for PR 19239 at commit

[GitHub] spark issue #19230: [SPARK-22003][SQL] support array column in vectorized re...

2017-09-15 Thread liufengdb
Github user liufengdb commented on the issue: https://github.com/apache/spark/pull/19230 @viirya @cloud-fan unit test updated. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19230: [SPARK-22003][SQL] support array column in vectorized re...

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19230 **[Test build #81835 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81835/testReport)** for PR 19230 at commit

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19250 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19250 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81833/ Test FAILed. ---

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19250 **[Test build #81833 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81833/testReport)** for PR 19250 at commit

[GitHub] spark pull request #18659: [SPARK-21190][PYSPARK][WIP] Python Vectorized UDF...

2017-09-15 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18659#discussion_r139261343 --- Diff: python/pyspark/serializers.py --- @@ -199,6 +211,33 @@ def __repr__(self): return "ArrowSerializer" +class

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Python Vectorized UDFs

2017-09-15 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/18659 @ueshin , I merged your tests and added support for `**kwargs` to use "size" for 0-parameter UDFs. Do you think this might be a little better to be called "length" or "output_length"?

[GitHub] spark pull request #19239: [SPARK-22017] Take minimum of all watermark execs...

2017-09-15 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/19239#discussion_r139259901 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/EventTimeWatermarkSuite.scala --- @@ -300,6 +300,67 @@ class EventTimeWatermarkSuite extends

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Python Vectorized UDFs

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18659 **[Test build #81834 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81834/testReport)** for PR 18659 at commit

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19250 **[Test build #81833 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81833/testReport)** for PR 19250 at commit

[GitHub] spark issue #19239: [SPARK-22017] Take minimum of all watermark execs in Str...

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19239 **[Test build #81832 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81832/testReport)** for PR 19239 at commit

[GitHub] spark issue #19074: [SPARK-21714][CORE][BACKPORT-2.2] Avoiding re-uploading ...

2017-09-15 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19074 @loneknightpy can you file a new bug instead of comment on a closed PR? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19074: [SPARK-21714][CORE][BACKPORT-2.2] Avoiding re-upl...

2017-09-15 Thread loneknightpy
Github user loneknightpy commented on a diff in the pull request: https://github.com/apache/spark/pull/19074#discussion_r139248521 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -311,12 +317,16 @@ object SparkSubmit extends CommandLineUtils {

[GitHub] spark pull request #19074: [SPARK-21714][CORE][BACKPORT-2.2] Avoiding re-upl...

2017-09-15 Thread loneknightpy
Github user loneknightpy commented on a diff in the pull request: https://github.com/apache/spark/pull/19074#discussion_r139247692 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -366,7 +376,7 @@ object SparkSubmit extends CommandLineUtils {

[GitHub] spark pull request #19074: [SPARK-21714][CORE][BACKPORT-2.2] Avoiding re-upl...

2017-09-15 Thread loneknightpy
Github user loneknightpy commented on a diff in the pull request: https://github.com/apache/spark/pull/19074#discussion_r139248109 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -376,8 +386,8 @@ object SparkSubmit extends CommandLineUtils {

[GitHub] spark issue #19180: [SPARK-21967][CORE] org.apache.spark.unsafe.types.UTF8St...

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19180 **[Test build #81831 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81831/testReport)** for PR 19180 at commit

[GitHub] spark issue #19180: [SPARK-21967][CORE] org.apache.spark.unsafe.types.UTF8St...

2017-09-15 Thread original-brownbear
Github user original-brownbear commented on the issue: https://github.com/apache/spark/pull/19180 @srowen alright then, switched to the JDK comparison left the rest as is :) --- - To unsubscribe, e-mail:

[GitHub] spark issue #19211: [SPARK-18838][core] Add separate listener queues to Live...

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19211 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19211: [SPARK-18838][core] Add separate listener queues to Live...

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19211 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81826/ Test PASSed. ---

[GitHub] spark issue #19211: [SPARK-18838][core] Add separate listener queues to Live...

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19211 **[Test build #81826 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81826/testReport)** for PR 19211 at commit

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19250 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81830/ Test FAILed. ---

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19250 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19250 **[Test build #81830 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81830/testReport)** for PR 19250 at commit

[GitHub] spark issue #18978: [SPARK-21737][YARN]Create communication channel between ...

2017-09-15 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/18978 we can close for now, got busy with some other stuff hope to get back to this soon --- - To unsubscribe, e-mail:

[GitHub] spark issue #18978: [SPARK-21737][YARN]Create communication channel between ...

2017-09-15 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18978 Any updates here? Or should we close this PR until this is properly investigated? --- - To unsubscribe, e-mail:

[GitHub] spark issue #18897: [SPARK-21655][YARN] Support Kill CLI for Yarn mode

2017-09-15 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18897 Can we close this PR until #18978 is figured out? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19250 **[Test build #81830 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81830/testReport)** for PR 19250 at commit

[GitHub] spark pull request #19250: [SPARK-12297] Table timezone correction for Times...

2017-09-15 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19250#discussion_r139234142 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala --- @@ -92,7 +92,7 @@ case class

[GitHub] spark pull request #19250: [SPARK-12297] Table timezone correction for Times...

2017-09-15 Thread squito
GitHub user squito opened a pull request: https://github.com/apache/spark/pull/19250 [SPARK-12297] Table timezone correction for Timestamps ## What changes were proposed in this pull request? When reading and writing data, spark will adjust timestamp data based on the

[GitHub] spark issue #19246: [SPARK-22025] Speeding up fromInternal for StructField

2017-09-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19246 Hi, @maver1ck . Could you add your benchmark result on PR description for reviewers? That will be your commit log finally. > greatly speed up function calling ---

[GitHub] spark issue #19249: [SPARK-22032] Speed up StructType.fromInternal

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19249 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81829/ Test PASSed. ---

[GitHub] spark issue #19249: [SPARK-22032] Speed up StructType.fromInternal

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19249 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19249: [SPARK-22032] Speed up StructType.fromInternal

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19249 **[Test build #81829 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81829/testReport)** for PR 19249 at commit

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2017-09-15 Thread brad-kaiser
Github user brad-kaiser commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r139223468 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerMessages.scala --- @@ -116,4 +121,11 @@ private[spark] object BlockManagerMessages {

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2017-09-15 Thread brad-kaiser
Github user brad-kaiser commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r139223492 --- Diff: docs/configuration.md --- @@ -1705,6 +1705,26 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2017-09-15 Thread brad-kaiser
Github user brad-kaiser commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r139223409 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerMessages.scala --- @@ -32,8 +32,13 @@ private[spark] object BlockManagerMessages {

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2017-09-15 Thread brad-kaiser
Github user brad-kaiser commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r139223321 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -1231,6 +1231,7 @@ private[spark] class BlockManager( def

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2017-09-15 Thread brad-kaiser
Github user brad-kaiser commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r139223199 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -612,25 +623,26 @@ class

[GitHub] spark issue #19249: [SPARK-22032] Speed up StructType.fromInternal

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19249 **[Test build #81829 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81829/testReport)** for PR 19249 at commit

[GitHub] spark issue #19249: [SPARK-22032] Speed up StructType.fromInternal

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19249 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81828/ Test FAILed. ---

[GitHub] spark issue #19249: [SPARK-22032] Speed up StructType.fromInternal

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19249 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19249: [SPARK-22032] Speed up StructType.fromInternal

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19249 **[Test build #81828 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81828/testReport)** for PR 19249 at commit

[GitHub] spark issue #19249: [SPARK-22032] Speed up StructType.fromInternal

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19249 **[Test build #81828 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81828/testReport)** for PR 19249 at commit

[GitHub] spark issue #19249: [SPARK-22032] Speed up StructType.fromInternal

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19249 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81827/ Test FAILed. ---

[GitHub] spark issue #19249: [SPARK-22032] Speed up StructType.fromInternal

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19249 **[Test build #81827 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81827/testReport)** for PR 19249 at commit

[GitHub] spark issue #19249: [SPARK-22032] Speed up StructType.fromInternal

2017-09-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19249 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19249: [SPARK-22032] Speed up StructType.fromInternal

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19249 **[Test build #81827 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81827/testReport)** for PR 19249 at commit

[GitHub] spark pull request #19249: [SPARK-22032] Speed up StructType.fromInternal

2017-09-15 Thread maver1ck
GitHub user maver1ck opened a pull request: https://github.com/apache/spark/pull/19249 [SPARK-22032] Speed up StructType.fromInternal ## What changes were proposed in this pull request? StructType.fromInternal is calling f.fromInternal(v) for every field. We can use

[GitHub] spark issue #19211: [SPARK-18838][core] Add separate listener queues to Live...

2017-09-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19211 **[Test build #81826 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81826/testReport)** for PR 19211 at commit

[GitHub] spark pull request #19238: [SPARK-22016][SQL] Add HiveDialect for JDBC conne...

2017-09-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19238#discussion_r139203753 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala --- @@ -1103,6 +1103,17 @@ class JDBCSuite extends SparkFunSuite

[GitHub] spark pull request #19238: [SPARK-22016][SQL] Add HiveDialect for JDBC conne...

2017-09-15 Thread danielfx90
Github user danielfx90 commented on a diff in the pull request: https://github.com/apache/spark/pull/19238#discussion_r139202470 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala --- @@ -1103,6 +1103,17 @@ class JDBCSuite extends SparkFunSuite

[GitHub] spark pull request #19238: [SPARK-22016][SQL] Add HiveDialect for JDBC conne...

2017-09-15 Thread danielfx90
Github user danielfx90 commented on a diff in the pull request: https://github.com/apache/spark/pull/19238#discussion_r139201902 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala --- @@ -1103,6 +1103,17 @@ class JDBCSuite extends SparkFunSuite

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-09-15 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/15770 I will address the review comments soon. Thanks! @WeichenXu123 --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19211: [SPARK-18838][core] Add separate listener queues ...

2017-09-15 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19211#discussion_r139199768 --- Diff: core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala --- @@ -65,53 +60,61 @@ private[spark] class LiveListenerBus(conf: SparkConf)

[GitHub] spark pull request #19248: [SPARK-22027] Add missing explanation of default ...

2017-09-15 Thread exKAZUu
Github user exKAZUu commented on a diff in the pull request: https://github.com/apache/spark/pull/19248#discussion_r139198364 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/shared/sharedParams.scala --- @@ -44,7 +44,7 @@ private[ml] trait HasRegParam extends Params {

[GitHub] spark pull request #19196: [SPARK-21977] SinglePartition optimizations break...

2017-09-15 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/19196#discussion_r139197660 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingAggregationSuite.scala --- @@ -381,4 +388,187 @@ class StreamingAggregationSuite

[GitHub] spark issue #19227: [SPARK-20060][CORE] Support accessing secure Hadoop clus...

2017-09-15 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19227 > If people have a standalone cluster they have secured off for only a set of users and they all run under a single user, having access to secure HDFS is still very useful. No disputing

[GitHub] spark issue #19243: [SPARK-21780][R] Simpler Dataset.sample API in R

2017-09-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19243 Sure. This one is abit tricky. Let me try to find out a better way. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19243: [SPARK-21780][R] Simpler Dataset.sample API in R

2017-09-15 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19243 let me think about this a bit... --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #15544: [SPARK-17997] [SQL] Add an aggregation function f...

2017-09-15 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15544#discussion_r139188790 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/ApproxCountDistinctForIntervals.scala --- @@ -0,0 +1,248 @@

[GitHub] spark pull request #15544: [SPARK-17997] [SQL] Add an aggregation function f...

2017-09-15 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15544#discussion_r139190161 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/ApproxCountDistinctForIntervals.scala --- @@ -0,0 +1,248 @@

[GitHub] spark pull request #15544: [SPARK-17997] [SQL] Add an aggregation function f...

2017-09-15 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15544#discussion_r139190528 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/ApproxCountDistinctForIntervals.scala --- @@ -0,0 +1,248 @@

[GitHub] spark pull request #15544: [SPARK-17997] [SQL] Add an aggregation function f...

2017-09-15 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15544#discussion_r139188326 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -270,6 +270,7 @@ object FunctionRegistry {

[GitHub] spark pull request #19238: [SPARK-22016][SQL] Add HiveDialect for JDBC conne...

2017-09-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19238#discussion_r139188633 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala --- @@ -1103,6 +1103,17 @@ class JDBCSuite extends SparkFunSuite

[GitHub] spark issue #19237: [SPARK-21987][SQL] fix a compatibility issue of sql even...

2017-09-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19237 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19211: [SPARK-18838][core] Add separate listener queues ...

2017-09-15 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19211#discussion_r139187703 --- Diff: core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala --- @@ -65,53 +60,61 @@ private[spark] class LiveListenerBus(conf:

  1   2   3   >