[GitHub] [spark] SparkQA commented on issue #24416: [SPARK-27521][SQL] move data source v2 to catalyst module

2019-05-27 Thread GitBox
SparkQA commented on issue #24416: [SPARK-27521][SQL] move data source v2 to catalyst module URL: https://github.com/apache/spark/pull/24416#issuecomment-496388376 **[Test build #105858 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105858/testReport)** fo

[GitHub] [spark] AmplabJenkins commented on issue #24416: [SPARK-27521][SQL] move data source v2 to catalyst module

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24416: [SPARK-27521][SQL] move data source v2 to catalyst module URL: https://github.com/apache/spark/pull/24416#issuecomment-496387863 Merged build finished. Test PASSed. This is an automate

[GitHub] [spark] AmplabJenkins removed a comment on issue #24416: [SPARK-27521][SQL] move data source v2 to catalyst module

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24416: [SPARK-27521][SQL] move data source v2 to catalyst module URL: https://github.com/apache/spark/pull/24416#issuecomment-496387865 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.

[GitHub] [spark] AmplabJenkins removed a comment on issue #24416: [SPARK-27521][SQL] move data source v2 to catalyst module

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24416: [SPARK-27521][SQL] move data source v2 to catalyst module URL: https://github.com/apache/spark/pull/24416#issuecomment-496387863 Merged build finished. Test PASSed. This is an

[GitHub] [spark] AmplabJenkins commented on issue #24416: [SPARK-27521][SQL] move data source v2 to catalyst module

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24416: [SPARK-27521][SQL] move data source v2 to catalyst module URL: https://github.com/apache/spark/pull/24416#issuecomment-496387865 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley

[GitHub] [spark] swapnilushinde commented on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions.

2019-05-27 Thread GitBox
swapnilushinde commented on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions. URL: https://github.com/apache/spark/pull/24724#issuecomment-496382800 Hello @HyukjinKwon @MaxGekk - Above simple example was just for an ill

[GitHub] [spark] wenxuanguan commented on issue #24711: [SPARK-27859][SS] Use efficient sorting instead of `.sorted.reverse` sequence

2019-05-27 Thread GitBox
wenxuanguan commented on issue #24711: [SPARK-27859][SS] Use efficient sorting instead of `.sorted.reverse` sequence URL: https://github.com/apache/spark/pull/24711#issuecomment-496380982 > yea LGTM too Thank you for review ---

[GitHub] [spark] HeartSaVioR commented on a change in pull request #23576: [SPARK-26655] [SS] Support multiple aggregates in append mode

2019-05-27 Thread GitBox
HeartSaVioR commented on a change in pull request #23576: [SPARK-26655] [SS] Support multiple aggregates in append mode URL: https://github.com/apache/spark/pull/23576#discussion_r287939969 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/WatermarkTrac

[GitHub] [spark] Ngone51 commented on issue #24569: [SPARK-23191][CORE] Warn rather than terminate when duplicate worker register happens

2019-05-27 Thread GitBox
Ngone51 commented on issue #24569: [SPARK-23191][CORE] Warn rather than terminate when duplicate worker register happens URL: https://github.com/apache/spark/pull/24569#issuecomment-496376452 Thank you @cloud-fan This is an

[GitHub] [spark] HyukjinKwon commented on issue #24711: [SPARK-27859][SS] Use efficient sorting instead of `.sorted.reverse` sequence

2019-05-27 Thread GitBox
HyukjinKwon commented on issue #24711: [SPARK-27859][SS] Use efficient sorting instead of `.sorted.reverse` sequence URL: https://github.com/apache/spark/pull/24711#issuecomment-496376373 yea LGTM too This is an automated mes

[GitHub] [spark] HyukjinKwon closed pull request #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions.

2019-05-27 Thread GitBox
HyukjinKwon closed pull request #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions. URL: https://github.com/apache/spark/pull/24724 This is an automated message fr

[GitHub] [spark] HyukjinKwon commented on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions.

2019-05-27 Thread GitBox
HyukjinKwon commented on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions. URL: https://github.com/apache/spark/pull/24724#issuecomment-496375843 There's virtually no diff: ```scala case class Person(name: String

[GitHub] [spark] HyukjinKwon commented on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions.

2019-05-27 Thread GitBox
HyukjinKwon commented on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions. URL: https://github.com/apache/spark/pull/24724#issuecomment-496375267 API itself is two lines. It's one liner or two liner - workaround is easy. I

[GitHub] [spark] dongjoon-hyun commented on issue #24711: [SPARK-27859][SS] Use efficient sorting instead of `.sorted.reverse` sequence

2019-05-27 Thread GitBox
dongjoon-hyun commented on issue #24711: [SPARK-27859][SS] Use efficient sorting instead of `.sorted.reverse` sequence URL: https://github.com/apache/spark/pull/24711#issuecomment-496374107 You're welcome, @wenxuanguan . This

[GitHub] [spark] dongjoon-hyun edited a comment on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions.

2019-05-27 Thread GitBox
dongjoon-hyun edited a comment on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions. URL: https://github.com/apache/spark/pull/24724#issuecomment-496373474 First of all, the followings are the most frequent use cases. (And

[GitHub] [spark] dongjoon-hyun commented on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions.

2019-05-27 Thread GitBox
dongjoon-hyun commented on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions. URL: https://github.com/apache/spark/pull/24724#issuecomment-496373474 First of all, the followings are the most frequent use cases. 1. HEADER

[GitHub] [spark] dongjoon-hyun edited a comment on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions.

2019-05-27 Thread GitBox
dongjoon-hyun edited a comment on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions. URL: https://github.com/apache/spark/pull/24724#issuecomment-496373474 First of all, the followings are the most frequent use cases. 1

[GitHub] [spark] HyukjinKwon closed pull request #24716: [SPARK-27848][R][BUILD] AppVeyor change to latest R version (3.6.0)

2019-05-27 Thread GitBox
HyukjinKwon closed pull request #24716: [SPARK-27848][R][BUILD] AppVeyor change to latest R version (3.6.0) URL: https://github.com/apache/spark/pull/24716 This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] HyukjinKwon commented on issue #24716: [SPARK-25944][R][BUILD] AppVeyor change to latest R version (3.6.0)

2019-05-27 Thread GitBox
HyukjinKwon commented on issue #24716: [SPARK-25944][R][BUILD] AppVeyor change to latest R version (3.6.0) URL: https://github.com/apache/spark/pull/24716#issuecomment-496372569 Merged to master. This is an automated message

[GitHub] [spark] wenxuanguan commented on issue #24711: [SPARK-27859][SS] Use efficient sorting instead of `.sorted.reverse` sequence

2019-05-27 Thread GitBox
wenxuanguan commented on issue #24711: [SPARK-27859][SS] Use efficient sorting instead of `.sorted.reverse` sequence URL: https://github.com/apache/spark/pull/24711#issuecomment-496371722 @srowen @dongjoon-hyun Thank you for review --

[GitHub] [spark] AmplabJenkins removed a comment on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC URL: https://github.com/apache/spark/pull/24043#issuecomment-496369660 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] AmplabJenkins removed a comment on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC URL: https://github.com/apache/spark/pull/24043#issuecomment-496369663 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.ed

[GitHub] [spark] AmplabJenkins commented on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC URL: https://github.com/apache/spark/pull/24043#issuecomment-496369660 Merged build finished. Test PASSed. This is an automated message

[GitHub] [spark] AmplabJenkins commented on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC URL: https://github.com/apache/spark/pull/24043#issuecomment-496369663 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkin

[GitHub] [spark] SparkQA removed a comment on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC

2019-05-27 Thread GitBox
SparkQA removed a comment on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC URL: https://github.com/apache/spark/pull/24043#issuecomment-496340435 **[Test build #105853 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105853/testReport)** for

[GitHub] [spark] SparkQA commented on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC

2019-05-27 Thread GitBox
SparkQA commented on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC URL: https://github.com/apache/spark/pull/24043#issuecomment-496369350 **[Test build #105853 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105853/testReport)** for PR 2404

[GitHub] [spark] AmplabJenkins removed a comment on issue #24382: [SPARK-27330][SS] support task abort in foreach writer

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24382: [SPARK-27330][SS] support task abort in foreach writer URL: https://github.com/apache/spark/pull/24382#issuecomment-483678508 Can one of the admins verify this patch? This is a

[GitHub] [spark] HeartSaVioR commented on issue #24382: [SPARK-27330][SS] support task abort in foreach writer

2019-05-27 Thread GitBox
HeartSaVioR commented on issue #24382: [SPARK-27330][SS] support task abort in foreach writer URL: https://github.com/apache/spark/pull/24382#issuecomment-496369272 test this please This is an automated message from the Apach

[GitHub] [spark] swapnilushinde edited a comment on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions.

2019-05-27 Thread GitBox
swapnilushinde edited a comment on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions. URL: https://github.com/apache/spark/pull/24724#issuecomment-496367606 Hi, @dongjoon-hyun Thanks for reply. Yes, I use this API sometime

[GitHub] [spark] HyukjinKwon commented on issue #24716: [SPARK-25944][R][BUILD] AppVeyor change to latest R version (3.6.0)

2019-05-27 Thread GitBox
HyukjinKwon commented on issue #24716: [SPARK-25944][R][BUILD] AppVeyor change to latest R version (3.6.0) URL: https://github.com/apache/spark/pull/24716#issuecomment-496368610 Oops, thanks This is an automated message from

[GitHub] [spark] swapnilushinde commented on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions.

2019-05-27 Thread GitBox
swapnilushinde commented on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions. URL: https://github.com/apache/spark/pull/24724#issuecomment-496367606 Hi, @dongjoon-hyun Thanks for reply. Yes, I use this API sometimes as wel

[GitHub] [spark] dongjoon-hyun commented on issue #24716: [SPARK-25944][R][BUILD] AppVeyor change to latest R version (3.6.0)

2019-05-27 Thread GitBox
dongjoon-hyun commented on issue #24716: [SPARK-25944][R][BUILD] AppVeyor change to latest R version (3.6.0) URL: https://github.com/apache/spark/pull/24716#issuecomment-496366021 BTW, @HyukjinKwon . Could you fix the PR description? > R 3.5.1 is released 2019-04-26. It seems to

[GitHub] [spark] dongjoon-hyun commented on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions.

2019-05-27 Thread GitBox
dongjoon-hyun commented on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions. URL: https://github.com/apache/spark/pull/24724#issuecomment-496365550 Hi, @swapnilushinde . Thank you for making a PR, but do you the following?

[GitHub] [spark] dongjoon-hyun closed pull request #24711: [SPARK-27859][SS] Use efficient sorting instead of `.sorted.reverse` sequence

2019-05-27 Thread GitBox
dongjoon-hyun closed pull request #24711: [SPARK-27859][SS] Use efficient sorting instead of `.sorted.reverse` sequence URL: https://github.com/apache/spark/pull/24711 This is an automated message from the Apache Git Service

[GitHub] [spark] dongjoon-hyun commented on issue #24711: [SPARK-27859][SS] Use efficient sorting instead of `.sorted.reverse` sequence

2019-05-27 Thread GitBox
dongjoon-hyun commented on issue #24711: [SPARK-27859][SS] Use efficient sorting instead of `.sorted.reverse` sequence URL: https://github.com/apache/spark/pull/24711#issuecomment-496364119 Merged to master. This is an automa

[GitHub] [spark] dongjoon-hyun commented on issue #24711: [Minor][SS] Use efficient sorting instead of `.sorted.reverse` sequence

2019-05-27 Thread GitBox
dongjoon-hyun commented on issue #24711: [Minor][SS] Use efficient sorting instead of `.sorted.reverse` sequence URL: https://github.com/apache/spark/pull/24711#issuecomment-496363920 I'll create for you. This is an automated

[GitHub] [spark] dongjoon-hyun commented on issue #24711: [Minor][SS]avoid inefficient sort when getLatest in HDFSMetadataLog

2019-05-27 Thread GitBox
dongjoon-hyun commented on issue #24711: [Minor][SS]avoid inefficient sort when getLatest in HDFSMetadataLog URL: https://github.com/apache/spark/pull/24711#issuecomment-496363597 Also, please update PR title and description. You didn't include the changes in `streaming/ui/BatchPage.scala`

[GitHub] [spark] dongjoon-hyun edited a comment on issue #24711: [Minor][SS]avoid inefficient sort when getLatest in HDFSMetadataLog

2019-05-27 Thread GitBox
dongjoon-hyun edited a comment on issue #24711: [Minor][SS]avoid inefficient sort when getLatest in HDFSMetadataLog URL: https://github.com/apache/spark/pull/24711#issuecomment-496362843 Thank you for pinging me, @wenxuanguan . Please make a JIRA issue and use the ID in the PR title. This

[GitHub] [spark] dongjoon-hyun commented on issue #24711: [Minor][SS]avoid inefficient sort when getLatest in HDFSMetadataLog

2019-05-27 Thread GitBox
dongjoon-hyun commented on issue #24711: [Minor][SS]avoid inefficient sort when getLatest in HDFSMetadataLog URL: https://github.com/apache/spark/pull/24711#issuecomment-496362843 Thank you for pinging me, @wenxuanguan . Please make a JIRA issue and use the ID. This is trivial but worth fo

[GitHub] [spark] AmplabJenkins removed a comment on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions.

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions. URL: https://github.com/apache/spark/pull/24724#issuecomment-496360880 Can one of the admins verify this patch? ---

[GitHub] [spark] AmplabJenkins commented on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions.

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions. URL: https://github.com/apache/spark/pull/24724#issuecomment-496361177 Can one of the admins verify this patch?

[GitHub] [spark] AmplabJenkins removed a comment on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions.

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions. URL: https://github.com/apache/spark/pull/24724#issuecomment-496360804 Can one of the admins verify this patch? ---

[GitHub] [spark] AmplabJenkins removed a comment on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead.

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead. URL: https://github.com/apache/spark/pull/24671#issuecomment-496360724 Test PASSed. Refer to this link for build results (access ri

[GitHub] [spark] AmplabJenkins removed a comment on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead.

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead. URL: https://github.com/apache/spark/pull/24671#issuecomment-496360721 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins commented on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions.

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions. URL: https://github.com/apache/spark/pull/24724#issuecomment-496360880 Can one of the admins verify this patch?

[GitHub] [spark] wenxuanguan commented on issue #24711: [Minor][SS]avoid inefficient sort when getLatest in HDFSMetadataLog

2019-05-27 Thread GitBox
wenxuanguan commented on issue #24711: [Minor][SS]avoid inefficient sort when getLatest in HDFSMetadataLog URL: https://github.com/apache/spark/pull/24711#issuecomment-496360902 @dongjoon-hyun @HyukjinKwon Can you please have a look? ---

[GitHub] [spark] AmplabJenkins commented on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead.

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead. URL: https://github.com/apache/spark/pull/24671#issuecomment-496360721 Merged build finished. Test PASSed. ---

[GitHub] [spark] AmplabJenkins commented on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions.

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions. URL: https://github.com/apache/spark/pull/24724#issuecomment-496360804 Can one of the admins verify this patch?

[GitHub] [spark] AmplabJenkins commented on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead.

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead. URL: https://github.com/apache/spark/pull/24671#issuecomment-496360724 Test PASSed. Refer to this link for build results (access rights to C

[GitHub] [spark] swapnilushinde opened a new pull request #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions.

2019-05-27 Thread GitBox
swapnilushinde opened a new pull request #24724: User friendly dataset, dataframe generation for csv datasources without explicit StructType definitions. URL: https://github.com/apache/spark/pull/24724 ## What changes were proposed in this pull request? Many users frequently load struct

[GitHub] [spark] SparkQA commented on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead.

2019-05-27 Thread GitBox
SparkQA commented on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead. URL: https://github.com/apache/spark/pull/24671#issuecomment-496360449 **[Test build #105854 has finished](https://amplab.cs.berkeley.edu/jenkins/jo

[GitHub] [spark] SparkQA removed a comment on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead.

2019-05-27 Thread GitBox
SparkQA removed a comment on issue #24671: [SPARK-27811][Core][Docs]Improve docs about spark.driver.memoryOverhead and spark.executor.memoryOverhead. URL: https://github.com/apache/spark/pull/24671#issuecomment-496341756 **[Test build #105854 has started](https://amplab.cs.berkeley.edu/jen

[GitHub] [spark] AmplabJenkins removed a comment on issue #24472: [SPARK-27578][SQL] Support INTERVAL ... HOUR TO SECOND syntax

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24472: [SPARK-27578][SQL] Support INTERVAL ... HOUR TO SECOND syntax URL: https://github.com/apache/spark/pull/24472#issuecomment-496360440 Merged build finished. Test PASSed. This is

[GitHub] [spark] AmplabJenkins removed a comment on issue #24472: [SPARK-27578][SQL] Support INTERVAL ... HOUR TO SECOND syntax

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24472: [SPARK-27578][SQL] Support INTERVAL ... HOUR TO SECOND syntax URL: https://github.com/apache/spark/pull/24472#issuecomment-496360443 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.

[GitHub] [spark] AmplabJenkins commented on issue #24472: [SPARK-27578][SQL] Support INTERVAL ... HOUR TO SECOND syntax

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24472: [SPARK-27578][SQL] Support INTERVAL ... HOUR TO SECOND syntax URL: https://github.com/apache/spark/pull/24472#issuecomment-496360440 Merged build finished. Test PASSed. This is an auto

[GitHub] [spark] AmplabJenkins commented on issue #24472: [SPARK-27578][SQL] Support INTERVAL ... HOUR TO SECOND syntax

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24472: [SPARK-27578][SQL] Support INTERVAL ... HOUR TO SECOND syntax URL: https://github.com/apache/spark/pull/24472#issuecomment-496360443 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berke

[GitHub] [spark] SparkQA removed a comment on issue #24472: [SPARK-27578][SQL] Support INTERVAL ... HOUR TO SECOND syntax

2019-05-27 Thread GitBox
SparkQA removed a comment on issue #24472: [SPARK-27578][SQL] Support INTERVAL ... HOUR TO SECOND syntax URL: https://github.com/apache/spark/pull/24472#issuecomment-496340414 **[Test build #105852 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105852/testR

[GitHub] [spark] SparkQA commented on issue #24472: [SPARK-27578][SQL] Support INTERVAL ... HOUR TO SECOND syntax

2019-05-27 Thread GitBox
SparkQA commented on issue #24472: [SPARK-27578][SQL] Support INTERVAL ... HOUR TO SECOND syntax URL: https://github.com/apache/spark/pull/24472#issuecomment-496360207 **[Test build #105852 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105852/testReport)*

[GitHub] [spark] dongjoon-hyun edited a comment on issue #24472: [SPARK-27578][SQL] Support INTERVAL ... HOUR TO SECOND syntax

2019-05-27 Thread GitBox
dongjoon-hyun edited a comment on issue #24472: [SPARK-27578][SQL] Support INTERVAL ... HOUR TO SECOND syntax URL: https://github.com/apache/spark/pull/24472#issuecomment-496358371 Hi, @gatorsmile and @cloud-fan . Could you give us some directional advice, please? - First, this PR

[GitHub] [spark] SparkQA commented on issue #24689: [SPARK-26946][SQL][FOLLOWUP] Handle lookupCatalog function not defined

2019-05-27 Thread GitBox
SparkQA commented on issue #24689: [SPARK-26946][SQL][FOLLOWUP] Handle lookupCatalog function not defined URL: https://github.com/apache/spark/pull/24689#issuecomment-496359179 **[Test build #105857 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105857/test

[GitHub] [spark] AmplabJenkins removed a comment on issue #24700: [SPARK-27834][SQL][R][PYTHON] Make separate PySpark/SparkR vectorization configurations

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24700: [SPARK-27834][SQL][R][PYTHON] Make separate PySpark/SparkR vectorization configurations URL: https://github.com/apache/spark/pull/24700#issuecomment-496358953 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins removed a comment on issue #24700: [SPARK-27834][SQL][R][PYTHON] Make separate PySpark/SparkR vectorization configurations

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24700: [SPARK-27834][SQL][R][PYTHON] Make separate PySpark/SparkR vectorization configurations URL: https://github.com/apache/spark/pull/24700#issuecomment-496358955 Test PASSed. Refer to this link for build results (access rights to CI server ne

[GitHub] [spark] AmplabJenkins removed a comment on issue #24689: [SPARK-26946][SQL][FOLLOWUP] Handle lookupCatalog function not defined

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24689: [SPARK-26946][SQL][FOLLOWUP] Handle lookupCatalog function not defined URL: https://github.com/apache/spark/pull/24689#issuecomment-496358913 Merged build finished. Test PASSed. ---

[GitHub] [spark] AmplabJenkins removed a comment on issue #24689: [SPARK-26946][SQL][FOLLOWUP] Handle lookupCatalog function not defined

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24689: [SPARK-26946][SQL][FOLLOWUP] Handle lookupCatalog function not defined URL: https://github.com/apache/spark/pull/24689#issuecomment-496358914 Test PASSed. Refer to this link for build results (access rights to CI server needed): https

[GitHub] [spark] AmplabJenkins commented on issue #24700: [SPARK-27834][SQL][R][PYTHON] Make separate PySpark/SparkR vectorization configurations

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24700: [SPARK-27834][SQL][R][PYTHON] Make separate PySpark/SparkR vectorization configurations URL: https://github.com/apache/spark/pull/24700#issuecomment-496358955 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #24700: [SPARK-27834][SQL][R][PYTHON] Make separate PySpark/SparkR vectorization configurations

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24700: [SPARK-27834][SQL][R][PYTHON] Make separate PySpark/SparkR vectorization configurations URL: https://github.com/apache/spark/pull/24700#issuecomment-496358953 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins commented on issue #24689: [SPARK-26946][SQL][FOLLOWUP] Handle lookupCatalog function not defined

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24689: [SPARK-26946][SQL][FOLLOWUP] Handle lookupCatalog function not defined URL: https://github.com/apache/spark/pull/24689#issuecomment-496358913 Merged build finished. Test PASSed. This i

[GitHub] [spark] AmplabJenkins commented on issue #24689: [SPARK-26946][SQL][FOLLOWUP] Handle lookupCatalog function not defined

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24689: [SPARK-26946][SQL][FOLLOWUP] Handle lookupCatalog function not defined URL: https://github.com/apache/spark/pull/24689#issuecomment-496358914 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://ampla

[GitHub] [spark] SparkQA removed a comment on issue #24700: [SPARK-27834][SQL][R][PYTHON] Make separate PySpark/SparkR vectorization configurations

2019-05-27 Thread GitBox
SparkQA removed a comment on issue #24700: [SPARK-27834][SQL][R][PYTHON] Make separate PySpark/SparkR vectorization configurations URL: https://github.com/apache/spark/pull/24700#issuecomment-496328630 **[Test build #105851 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullR

[GitHub] [spark] SparkQA commented on issue #24700: [SPARK-27834][SQL][R][PYTHON] Make separate PySpark/SparkR vectorization configurations

2019-05-27 Thread GitBox
SparkQA commented on issue #24700: [SPARK-27834][SQL][R][PYTHON] Make separate PySpark/SparkR vectorization configurations URL: https://github.com/apache/spark/pull/24700#issuecomment-496358643 **[Test build #105851 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestB

[GitHub] [spark] dongjoon-hyun commented on issue #24472: [SPARK-27578][SQL] Support INTERVAL ... HOUR TO SECOND syntax

2019-05-27 Thread GitBox
dongjoon-hyun commented on issue #24472: [SPARK-27578][SQL] Support INTERVAL ... HOUR TO SECOND syntax URL: https://github.com/apache/spark/pull/24472#issuecomment-496358371 Hi, @gatorsmile and @cloud-fan . Could you give us some directional advice, please? - First, this PR wants

[GitHub] [spark] AmplabJenkins removed a comment on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics URL: https://github.com/apache/spark/pull/24717#issuecomment-496356858 Merged build finished. Test PASSed. This

[GitHub] [spark] AmplabJenkins removed a comment on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics URL: https://github.com/apache/spark/pull/24717#issuecomment-496356864 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://ampl

[GitHub] [spark] cloud-fan closed pull request #24569: [SPARK-23191][CORE] Warn rather than terminate when duplicate worker register happens

2019-05-27 Thread GitBox
cloud-fan closed pull request #24569: [SPARK-23191][CORE] Warn rather than terminate when duplicate worker register happens URL: https://github.com/apache/spark/pull/24569 This is an automated message from the Apache Git Ser

[GitHub] [spark] jzhuge commented on a change in pull request #24689: [SPARK-26946][SQL][FOLLOWUP] Handle lookupCatalog function not defined

2019-05-27 Thread GitBox
jzhuge commented on a change in pull request #24689: [SPARK-26946][SQL][FOLLOWUP] Handle lookupCatalog function not defined URL: https://github.com/apache/spark/pull/24689#discussion_r287922975 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalog/v2/LookupCatalog.

[GitHub] [spark] AmplabJenkins commented on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics URL: https://github.com/apache/spark/pull/24717#issuecomment-496356858 Merged build finished. Test PASSed. This is an a

[GitHub] [spark] AmplabJenkins commented on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics URL: https://github.com/apache/spark/pull/24717#issuecomment-496356864 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.be

[GitHub] [spark] SparkQA removed a comment on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics

2019-05-27 Thread GitBox
SparkQA removed a comment on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics URL: https://github.com/apache/spark/pull/24717#issuecomment-496346720 **[Test build #105856 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105856/te

[GitHub] [spark] SparkQA commented on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics

2019-05-27 Thread GitBox
SparkQA commented on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics URL: https://github.com/apache/spark/pull/24717#issuecomment-496356711 **[Test build #105856 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105856/testRepor

[GitHub] [spark] cloud-fan commented on issue #24569: [SPARK-23191][CORE] Warn rather than terminate when duplicate worker register happens

2019-05-27 Thread GitBox
cloud-fan commented on issue #24569: [SPARK-23191][CORE] Warn rather than terminate when duplicate worker register happens URL: https://github.com/apache/spark/pull/24569#issuecomment-496356630 thanks, merging to master! This

[GitHub] [spark] cloud-fan commented on issue #24696: [SPARK-27832][SQL] Don't decompress and create column batch when the task is completed

2019-05-27 Thread GitBox
cloud-fan commented on issue #24696: [SPARK-27832][SQL] Don't decompress and create column batch when the task is completed URL: https://github.com/apache/spark/pull/24696#issuecomment-496355838 > At the moment, the returned batch is also immediately closed I'm a little lost here. Ca

[GitHub] [spark] dongjoon-hyun edited a comment on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types

2019-05-27 Thread GitBox
dongjoon-hyun edited a comment on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types URL: https://github.com/apache/spark/pull/24722#issuecomment-496351652 @gcmerz . What is your id in Apache JIRA? If you don't have, please create one.

[GitHub] [spark] dongjoon-hyun commented on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types

2019-05-27 Thread GitBox
dongjoon-hyun commented on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types URL: https://github.com/apache/spark/pull/24722#issuecomment-496351652 @gcmerz . What is your id in Apache JIRA? If you don't have, please create one. Then,

[GitHub] [spark] zhengruifeng commented on issue #14325: [SPARK-16692] [ML] Add multi label classification evaluator, DataFrame

2019-05-27 Thread GitBox
zhengruifeng commented on issue #14325: [SPARK-16692] [ML] Add multi label classification evaluator, DataFrame URL: https://github.com/apache/spark/pull/14325#issuecomment-496350955 What's the progress now? @liwzhi @WeichenXu123 @srowen If @liwzhi are not working on this, can I take it

[GitHub] [spark] dongjoon-hyun closed pull request #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types

2019-05-27 Thread GitBox
dongjoon-hyun closed pull request #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types URL: https://github.com/apache/spark/pull/24722 This is an automated message from the Apac

[GitHub] [spark] AmplabJenkins removed a comment on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types URL: https://github.com/apache/spark/pull/24722#issuecomment-496349036 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins removed a comment on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types URL: https://github.com/apache/spark/pull/24722#issuecomment-496349037 Test PASSed. Refer to this link for build results (access rights to CI serve

[GitHub] [spark] AmplabJenkins commented on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types URL: https://github.com/apache/spark/pull/24722#issuecomment-496349037 Test PASSed. Refer to this link for build results (access rights to CI server needed

[GitHub] [spark] AmplabJenkins commented on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types URL: https://github.com/apache/spark/pull/24722#issuecomment-496349036 Merged build finished. Test PASSed. --

[GitHub] [spark] dongjoon-hyun commented on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types

2019-05-27 Thread GitBox
dongjoon-hyun commented on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types URL: https://github.com/apache/spark/pull/24722#issuecomment-496348939 Merged to `master` and `branch-2.4`. -

[GitHub] [spark] SparkQA removed a comment on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types

2019-05-27 Thread GitBox
SparkQA removed a comment on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types URL: https://github.com/apache/spark/pull/24722#issuecomment-496344329 **[Test build #105855 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkP

[GitHub] [spark] SparkQA commented on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types

2019-05-27 Thread GitBox
SparkQA commented on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types URL: https://github.com/apache/spark/pull/24722#issuecomment-496348853 **[Test build #105855 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequ

[GitHub] [spark] dongjoon-hyun commented on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types

2019-05-27 Thread GitBox
dongjoon-hyun commented on issue #24722: [SPARK-27858][SQL] Fix for avro deserialization on union types with multiple non-null types URL: https://github.com/apache/spark/pull/24722#issuecomment-496348246 You're welcome. Thank you for swift update. --

[GitHub] [spark] dongjoon-hyun commented on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC

2019-05-27 Thread GitBox
dongjoon-hyun commented on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC URL: https://github.com/apache/spark/pull/24043#issuecomment-496347909 Lastly, it would be great if you can add some performance comparisons between Parquet/ORC merge schema in the PR description. This

[GitHub] [spark] AmplabJenkins removed a comment on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics URL: https://github.com/apache/spark/pull/24717#issuecomment-496347648 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amp

[GitHub] [spark] AmplabJenkins removed a comment on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics

2019-05-27 Thread GitBox
AmplabJenkins removed a comment on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics URL: https://github.com/apache/spark/pull/24717#issuecomment-496347641 Merged build finished. Test PASSed. This

[GitHub] [spark] AmplabJenkins commented on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics URL: https://github.com/apache/spark/pull/24717#issuecomment-496347648 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.b

[GitHub] [spark] AmplabJenkins commented on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics

2019-05-27 Thread GitBox
AmplabJenkins commented on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics URL: https://github.com/apache/spark/pull/24717#issuecomment-496347641 Merged build finished. Test PASSed. This is an a

[GitHub] [spark] SparkQA commented on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics

2019-05-27 Thread GitBox
SparkQA commented on issue #24717: [SPARK-27847][ML] One-Pass MultilabelMetrics & MulticlassMetrics URL: https://github.com/apache/spark/pull/24717#issuecomment-496346720 **[Test build #105856 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105856/testReport

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #24043: [SPARK-11412][SQL] Support merge schema for ORC

2019-05-27 Thread GitBox
dongjoon-hyun commented on a change in pull request #24043: [SPARK-11412][SQL] Support merge schema for ORC URL: https://github.com/apache/spark/pull/24043#discussion_r287913435 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFileOperator.scala ## @

  1   2   3   4   5   6   7   >