[GitHub] [spark] ulysses-you commented on issue #27212: [SPARK-30515][SQL] Refactor SimplifyBinaryComparison to reduce the time complexity

2020-01-14 Thread GitBox
ulysses-you commented on issue #27212: [SPARK-30515][SQL] Refactor SimplifyBinaryComparison to reduce the time complexity URL: https://github.com/apache/spark/pull/27212#issuecomment-574539982 Good catch ! Thanks. This is an

[GitHub] [spark] cloud-fan commented on a change in pull request #26961: [SPARK-29708][SQL] Correct aggregated values when grouping sets are duplicated

2020-01-14 Thread GitBox
cloud-fan commented on a change in pull request #26961: [SPARK-29708][SQL] Correct aggregated values when grouping sets are duplicated URL: https://github.com/apache/spark/pull/26961#discussion_r366733463 ## File path:

[GitHub] [spark] cloud-fan commented on a change in pull request #26961: [SPARK-29708][SQL] Correct aggregated values when grouping sets are duplicated

2020-01-14 Thread GitBox
cloud-fan commented on a change in pull request #26961: [SPARK-29708][SQL] Correct aggregated values when grouping sets are duplicated URL: https://github.com/apache/spark/pull/26961#discussion_r366733010 ## File path:

[GitHub] [spark] AmplabJenkins commented on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server URL: https://github.com/apache/spark/pull/27208#issuecomment-574538029 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server URL: https://github.com/apache/spark/pull/27208#issuecomment-574538036 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server URL: https://github.com/apache/spark/pull/27208#issuecomment-574538029 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server URL: https://github.com/apache/spark/pull/27208#issuecomment-574538036 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server

2020-01-14 Thread GitBox
SparkQA commented on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server URL: https://github.com/apache/spark/pull/27208#issuecomment-574537375 **[Test build #116748 has

[GitHub] [spark] SparkQA removed a comment on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server

2020-01-14 Thread GitBox
SparkQA removed a comment on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server URL: https://github.com/apache/spark/pull/27208#issuecomment-574495800 **[Test build #116748 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server URL: https://github.com/apache/spark/pull/27208#issuecomment-574536603 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server URL: https://github.com/apache/spark/pull/27208#issuecomment-574536595 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server URL: https://github.com/apache/spark/pull/27208#issuecomment-574536595 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server URL: https://github.com/apache/spark/pull/27208#issuecomment-574536603 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server

2020-01-14 Thread GitBox
SparkQA removed a comment on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server URL: https://github.com/apache/spark/pull/27208#issuecomment-574494199 **[Test build #116746 has

[GitHub] [spark] AmplabJenkins removed a comment on issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle Project nodes

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle Project nodes URL: https://github.com/apache/spark/pull/20345#issuecomment-574535699 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID URL: https://github.com/apache/spark/pull/26918#issuecomment-574535678 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID URL: https://github.com/apache/spark/pull/26918#issuecomment-574535669 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle Project nodes

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle Project nodes URL: https://github.com/apache/spark/pull/20345#issuecomment-574535705 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle Project nodes

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle Project nodes URL: https://github.com/apache/spark/pull/20345#issuecomment-574535705 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] cloud-fan commented on issue #24118: [SPARK-26736][SQL] if filter condition `And` has non-determined sub function it does not do partition prunning

2020-01-14 Thread GitBox
cloud-fan commented on issue #24118: [SPARK-26736][SQL] if filter condition `And` has non-determined sub function it does not do partition prunning URL: https://github.com/apache/spark/pull/24118#issuecomment-574535706 Seems we need to update `HiveTableScans` to use `ScanOperation` as

[GitHub] [spark] AmplabJenkins commented on issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle Project nodes

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle Project nodes URL: https://github.com/apache/spark/pull/20345#issuecomment-574535699 Merged build finished. Test PASSed. This

[GitHub] [spark] AmplabJenkins removed a comment on issue #25827: [SPARK-29128][SQL] Split predicate code in OR expressions

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #25827: [SPARK-29128][SQL] Split predicate code in OR expressions URL: https://github.com/apache/spark/pull/25827#issuecomment-574535667 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID URL: https://github.com/apache/spark/pull/26918#issuecomment-574535678 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-574535634 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server

2020-01-14 Thread GitBox
SparkQA commented on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server URL: https://github.com/apache/spark/pull/27208#issuecomment-574535775 **[Test build #116746 has

[GitHub] [spark] HyukjinKwon closed pull request #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md

2020-01-14 Thread GitBox
HyukjinKwon closed pull request #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md URL: https://github.com/apache/spark/pull/27194 This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-574535630 Merged build finished. Test PASSed.

[GitHub] [spark] HyukjinKwon closed pull request #27174: [SPARK-30490][SQL] Eliminate compiler warnings in Avro datasource

2020-01-14 Thread GitBox
HyukjinKwon closed pull request #27174: [SPARK-30490][SQL] Eliminate compiler warnings in Avro datasource URL: https://github.com/apache/spark/pull/27174 This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] AmplabJenkins removed a comment on issue #25827: [SPARK-29128][SQL] Split predicate code in OR expressions

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #25827: [SPARK-29128][SQL] Split predicate code in OR expressions URL: https://github.com/apache/spark/pull/25827#issuecomment-574535656 Merged build finished. Test PASSed. This is an

[GitHub] [spark] AmplabJenkins commented on issue #25827: [SPARK-29128][SQL] Split predicate code in OR expressions

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #25827: [SPARK-29128][SQL] Split predicate code in OR expressions URL: https://github.com/apache/spark/pull/25827#issuecomment-574535656 Merged build finished. Test PASSed. This is an

[GitHub] [spark] AmplabJenkins commented on issue #25827: [SPARK-29128][SQL] Split predicate code in OR expressions

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #25827: [SPARK-29128][SQL] Split predicate code in OR expressions URL: https://github.com/apache/spark/pull/25827#issuecomment-574535667 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID URL: https://github.com/apache/spark/pull/26918#issuecomment-574535669 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-574535634 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-574535630 Merged build finished. Test PASSed.

[GitHub] [spark] SparkQA commented on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID

2020-01-14 Thread GitBox
SparkQA commented on issue #26918: [SPARK-30279][SQL] Support 32 or more grouping attributes for GROUPING_ID URL: https://github.com/apache/spark/pull/26918#issuecomment-574535173 **[Test build #116760 has

[GitHub] [spark] SparkQA commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

2020-01-14 Thread GitBox
SparkQA commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-574535205 **[Test build #116759 has

[GitHub] [spark] SparkQA commented on issue #25827: [SPARK-29128][SQL] Split predicate code in OR expressions

2020-01-14 Thread GitBox
SparkQA commented on issue #25827: [SPARK-29128][SQL] Split predicate code in OR expressions URL: https://github.com/apache/spark/pull/25827#issuecomment-574535177 **[Test build #116761 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116761/testReport)**

[GitHub] [spark] SparkQA commented on issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle Project nodes

2020-01-14 Thread GitBox
SparkQA commented on issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle Project nodes URL: https://github.com/apache/spark/pull/20345#issuecomment-574535203 **[Test build #116762 has

[GitHub] [spark] guykhazma commented on issue #27157: [SPARK-30475][SQL] File source V2: Push data filters for file listing

2020-01-14 Thread GitBox
guykhazma commented on issue #27157: [SPARK-30475][SQL] File source V2: Push data filters for file listing URL: https://github.com/apache/spark/pull/27157#issuecomment-574533965 @gengliangwang see also this [PR](https://github.com/apache/spark/pull/17322) which originally added the

[GitHub] [spark] maropu commented on issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle Project nodes

2020-01-14 Thread GitBox
maropu commented on issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle Project nodes URL: https://github.com/apache/spark/pull/20345#issuecomment-574533947 retest this please This is an automated message

[GitHub] [spark] maropu commented on issue #25827: [SPARK-29128][SQL] Split predicate code in OR expressions

2020-01-14 Thread GitBox
maropu commented on issue #25827: [SPARK-29128][SQL] Split predicate code in OR expressions URL: https://github.com/apache/spark/pull/25827#issuecomment-574534079 retest this please This is an automated message from the

[GitHub] [spark] ayudovin edited a comment on issue #27030: [SPARK-30244][SQL] Emit pre/post events for "Partition" methods in ExternalCatalogWithListener

2020-01-14 Thread GitBox
ayudovin edited a comment on issue #27030: [SPARK-30244][SQL] Emit pre/post events for "Partition" methods in ExternalCatalogWithListener URL: https://github.com/apache/spark/pull/27030#issuecomment-574527485 > Hi, @ayudovin . Please check the PR content once more carefully. Thanks!

[GitHub] [spark] maropu commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

2020-01-14 Thread GitBox
maropu commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-574533844 retest this please This is an automated

[GitHub] [spark] ayudovin commented on a change in pull request #27030: [SPARK-30244][SQL] Emit pre/post events for "Partition" methods in ExternalCatalogWithListener

2020-01-14 Thread GitBox
ayudovin commented on a change in pull request #27030: [SPARK-30244][SQL] Emit pre/post events for "Partition" methods in ExternalCatalogWithListener URL: https://github.com/apache/spark/pull/27030#discussion_r366727729 ## File path:

[GitHub] [spark] AmplabJenkins removed a comment on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md URL: https://github.com/apache/spark/pull/27194#issuecomment-574532330 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins commented on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md URL: https://github.com/apache/spark/pull/27194#issuecomment-574532317 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md URL: https://github.com/apache/spark/pull/27194#issuecomment-574532317 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md URL: https://github.com/apache/spark/pull/27194#issuecomment-574532330 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md

2020-01-14 Thread GitBox
SparkQA commented on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md URL: https://github.com/apache/spark/pull/27194#issuecomment-574532206 **[Test build #116757 has

[GitHub] [spark] SparkQA removed a comment on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md

2020-01-14 Thread GitBox
SparkQA removed a comment on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md URL: https://github.com/apache/spark/pull/27194#issuecomment-574528638 **[Test build #116757 has

[GitHub] [spark] maropu closed pull request #19410: [SPARK-22184][CORE][GRAPHX] GraphX fails in case of insufficient memory and checkpoints enabled

2020-01-14 Thread GitBox
maropu closed pull request #19410: [SPARK-22184][CORE][GRAPHX] GraphX fails in case of insufficient memory and checkpoints enabled URL: https://github.com/apache/spark/pull/19410 This is an automated message from the Apache

[GitHub] [spark] maropu commented on issue #19410: [SPARK-22184][CORE][GRAPHX] GraphX fails in case of insufficient memory and checkpoints enabled

2020-01-14 Thread GitBox
maropu commented on issue #19410: [SPARK-22184][CORE][GRAPHX] GraphX fails in case of insufficient memory and checkpoints enabled URL: https://github.com/apache/spark/pull/19410#issuecomment-574531717 I'll close this pr and the corresponding jira because this pr is stale. If necessary,

[GitHub] [spark] MaxGekk commented on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md

2020-01-14 Thread GitBox
MaxGekk commented on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md URL: https://github.com/apache/spark/pull/27194#issuecomment-574531673 I have attached the screenshot.

[GitHub] [spark] cloud-fan commented on a change in pull request #26881: [SPARK-30252][SQL] Disallow negative scale of Decimal

2020-01-14 Thread GitBox
cloud-fan commented on a change in pull request #26881: [SPARK-30252][SQL] Disallow negative scale of Decimal URL: https://github.com/apache/spark/pull/26881#discussion_r366725522 ## File path: sql/core/src/test/resources/sql-tests/results/ansi/literals.sql.out ## @@

[GitHub] [spark] cloud-fan commented on a change in pull request #26881: [SPARK-30252][SQL] Disallow negative scale of Decimal

2020-01-14 Thread GitBox
cloud-fan commented on a change in pull request #26881: [SPARK-30252][SQL] Disallow negative scale of Decimal URL: https://github.com/apache/spark/pull/26881#discussion_r366725133 ## File path:

[GitHub] [spark] maropu edited a comment on issue #24118: [SPARK-26736][SQL] if filter condition `And` has non-determined sub function it does not do partition prunning

2020-01-14 Thread GitBox
maropu edited a comment on issue #24118: [SPARK-26736][SQL] if filter condition `And` has non-determined sub function it does not do partition prunning URL: https://github.com/apache/spark/pull/24118#issuecomment-574530598 It seems this pr intended to target hive tables? I tried the

[GitHub] [spark] cloud-fan commented on a change in pull request #26881: [SPARK-30252][SQL] Disallow negative scale of Decimal

2020-01-14 Thread GitBox
cloud-fan commented on a change in pull request #26881: [SPARK-30252][SQL] Disallow negative scale of Decimal URL: https://github.com/apache/spark/pull/26881#discussion_r366725192 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/types/DecimalSuite.scala ##

[GitHub] [spark] maropu commented on issue #24118: [SPARK-26736][SQL] if filter condition `And` has non-determined sub function it does not do partition prunning

2020-01-14 Thread GitBox
maropu commented on issue #24118: [SPARK-26736][SQL] if filter condition `And` has non-determined sub function it does not do partition prunning URL: https://github.com/apache/spark/pull/24118#issuecomment-574530598 It seems this pr intentd to target hive tables? I tried the example query

[GitHub] [spark] AmplabJenkins removed a comment on issue #27030: [SPARK-30244][SQL] Emit pre/post events for "Partition" methods in ExternalCatalogWithListener

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27030: [SPARK-30244][SQL] Emit pre/post events for "Partition" methods in ExternalCatalogWithListener URL: https://github.com/apache/spark/pull/27030#issuecomment-574529042 Test PASSed. Refer to this link for build results (access rights to CI

[GitHub] [spark] cloud-fan commented on a change in pull request #26881: [SPARK-30252][SQL] Disallow negative scale of Decimal

2020-01-14 Thread GitBox
cloud-fan commented on a change in pull request #26881: [SPARK-30252][SQL] Disallow negative scale of Decimal URL: https://github.com/apache/spark/pull/26881#discussion_r366724047 ## File path: python/pyspark/sql/tests/test_types.py ## @@ -204,12 +204,16 @@ def

[GitHub] [spark] cloud-fan commented on a change in pull request #26881: [SPARK-30252][SQL] Disallow negative scale of Decimal

2020-01-14 Thread GitBox
cloud-fan commented on a change in pull request #26881: [SPARK-30252][SQL] Disallow negative scale of Decimal URL: https://github.com/apache/spark/pull/26881#discussion_r366724203 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@

[GitHub] [spark] AmplabJenkins removed a comment on issue #27030: [SPARK-30244][SQL] Emit pre/post events for "Partition" methods in ExternalCatalogWithListener

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27030: [SPARK-30244][SQL] Emit pre/post events for "Partition" methods in ExternalCatalogWithListener URL: https://github.com/apache/spark/pull/27030#issuecomment-574529036 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #27030: [SPARK-30244][SQL] Emit pre/post events for "Partition" methods in ExternalCatalogWithListener

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27030: [SPARK-30244][SQL] Emit pre/post events for "Partition" methods in ExternalCatalogWithListener URL: https://github.com/apache/spark/pull/27030#issuecomment-574529042 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins commented on issue #27030: [SPARK-30244][SQL] Emit pre/post events for "Partition" methods in ExternalCatalogWithListener

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27030: [SPARK-30244][SQL] Emit pre/post events for "Partition" methods in ExternalCatalogWithListener URL: https://github.com/apache/spark/pull/27030#issuecomment-574529036 Merged build finished. Test PASSed.

[GitHub] [spark] SparkQA commented on issue #27198: [SPARK-27986][SQL][FOLLOWUP] Respect filter in sql/toString of AggregateExpression

2020-01-14 Thread GitBox
SparkQA commented on issue #27198: [SPARK-27986][SQL][FOLLOWUP] Respect filter in sql/toString of AggregateExpression URL: https://github.com/apache/spark/pull/27198#issuecomment-574528628 **[Test build #116756 has

[GitHub] [spark] SparkQA commented on issue #27030: [SPARK-30244][SQL] Emit pre/post events for "Partition" methods in ExternalCatalogWithListener

2020-01-14 Thread GitBox
SparkQA commented on issue #27030: [SPARK-30244][SQL] Emit pre/post events for "Partition" methods in ExternalCatalogWithListener URL: https://github.com/apache/spark/pull/27030#issuecomment-574528630 **[Test build #116758 has

[GitHub] [spark] SparkQA commented on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md

2020-01-14 Thread GitBox
SparkQA commented on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md URL: https://github.com/apache/spark/pull/27194#issuecomment-574528638 **[Test build #116757 has

[GitHub] [spark] ayudovin commented on issue #27030: [SPARK-30244][SQL] Emit pre/post events for "Partition" methods in ExternalCatalogWithListener

2020-01-14 Thread GitBox
ayudovin commented on issue #27030: [SPARK-30244][SQL] Emit pre/post events for "Partition" methods in ExternalCatalogWithListener URL: https://github.com/apache/spark/pull/27030#issuecomment-574527485 > Hi, @ayudovin . Please check the PR content once more carefully. Thanks! I have

[GitHub] [spark] AmplabJenkins removed a comment on issue #27213: [SPARK-30516][SQL] statistic estimation of FileScan should take partitionFilters and partition number into account

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27213: [SPARK-30516][SQL] statistic estimation of FileScan should take partitionFilters and partition number into account URL: https://github.com/apache/spark/pull/27213#issuecomment-574526788 Test PASSed. Refer to this link for build results

[GitHub] [spark] AmplabJenkins removed a comment on issue #26973: [SPARK-30323][SQL] Support filters pushdown in CSV datasource

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #26973: [SPARK-30323][SQL] Support filters pushdown in CSV datasource URL: https://github.com/apache/spark/pull/26973#issuecomment-574526831 Merged build finished. Test PASSed. This

[GitHub] [spark] AmplabJenkins removed a comment on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md URL: https://github.com/apache/spark/pull/27194#issuecomment-574526814 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md URL: https://github.com/apache/spark/pull/27194#issuecomment-574526820 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins removed a comment on issue #27198: [SPARK-27986][SQL][FOLLOWUP] Respect filter in sql/toString of AggregateExpression

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27198: [SPARK-27986][SQL][FOLLOWUP] Respect filter in sql/toString of AggregateExpression URL: https://github.com/apache/spark/pull/27198#issuecomment-574526763 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #27213: [SPARK-30516][SQL] statistic estimation of FileScan should take partitionFilters and partition number into account

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27213: [SPARK-30516][SQL] statistic estimation of FileScan should take partitionFilters and partition number into account URL: https://github.com/apache/spark/pull/27213#issuecomment-574526778 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #26973: [SPARK-30323][SQL] Support filters pushdown in CSV datasource

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #26973: [SPARK-30323][SQL] Support filters pushdown in CSV datasource URL: https://github.com/apache/spark/pull/26973#issuecomment-574526842 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #27198: [SPARK-27986][SQL][FOLLOWUP] Respect filter in sql/toString of AggregateExpression

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27198: [SPARK-27986][SQL][FOLLOWUP] Respect filter in sql/toString of AggregateExpression URL: https://github.com/apache/spark/pull/27198#issuecomment-574526771 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins commented on issue #27213: [SPARK-30516][SQL] statistic estimation of FileScan should take partitionFilters and partition number into account

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27213: [SPARK-30516][SQL] statistic estimation of FileScan should take partitionFilters and partition number into account URL: https://github.com/apache/spark/pull/27213#issuecomment-574526788 Test PASSed. Refer to this link for build results (access

[GitHub] [spark] AmplabJenkins commented on issue #26973: [SPARK-30323][SQL] Support filters pushdown in CSV datasource

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #26973: [SPARK-30323][SQL] Support filters pushdown in CSV datasource URL: https://github.com/apache/spark/pull/26973#issuecomment-574526831 Merged build finished. Test PASSed. This is an

[GitHub] [spark] AmplabJenkins commented on issue #26973: [SPARK-30323][SQL] Support filters pushdown in CSV datasource

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #26973: [SPARK-30323][SQL] Support filters pushdown in CSV datasource URL: https://github.com/apache/spark/pull/26973#issuecomment-574526842 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #27213: [SPARK-30516][SQL] statistic estimation of FileScan should take partitionFilters and partition number into account

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27213: [SPARK-30516][SQL] statistic estimation of FileScan should take partitionFilters and partition number into account URL: https://github.com/apache/spark/pull/27213#issuecomment-574526778 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md URL: https://github.com/apache/spark/pull/27194#issuecomment-574526814 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #27198: [SPARK-27986][SQL][FOLLOWUP] Respect filter in sql/toString of AggregateExpression

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27198: [SPARK-27986][SQL][FOLLOWUP] Respect filter in sql/toString of AggregateExpression URL: https://github.com/apache/spark/pull/27198#issuecomment-574526763 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #27198: [SPARK-27986][SQL][FOLLOWUP] Respect filter in sql/toString of AggregateExpression

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27198: [SPARK-27986][SQL][FOLLOWUP] Respect filter in sql/toString of AggregateExpression URL: https://github.com/apache/spark/pull/27198#issuecomment-574526771 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27194: [SPARK-30505][DOCS] Deprecate Avro option `ignoreExtension` in sql-data-sources-avro.md URL: https://github.com/apache/spark/pull/27194#issuecomment-574526820 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server URL: https://github.com/apache/spark/pull/27208#issuecomment-574526271 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server URL: https://github.com/apache/spark/pull/27208#issuecomment-574526263 Merged build finished. Test PASSed.

[GitHub] [spark] maropu edited a comment on issue #12302: [SPARK-14390][GraphX] Make initialization step in Pregel optional.

2020-01-14 Thread GitBox
maropu edited a comment on issue #12302: [SPARK-14390][GraphX] Make initialization step in Pregel optional. URL: https://github.com/apache/spark/pull/12302#issuecomment-574525828 I'll close this cuz the corresponding jira has been already closed.

[GitHub] [spark] AmplabJenkins commented on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server URL: https://github.com/apache/spark/pull/27208#issuecomment-574526263 Merged build finished. Test PASSed.

[GitHub] [spark] SparkQA commented on issue #26973: [SPARK-30323][SQL] Support filters pushdown in CSV datasource

2020-01-14 Thread GitBox
SparkQA commented on issue #26973: [SPARK-30323][SQL] Support filters pushdown in CSV datasource URL: https://github.com/apache/spark/pull/26973#issuecomment-574526334 **[Test build #116755 has

[GitHub] [spark] AmplabJenkins commented on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server URL: https://github.com/apache/spark/pull/27208#issuecomment-574526271 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #27213: [SPARK-30516][SQL] statistic estimation of FileScan should take partitionFilters and partition number into account

2020-01-14 Thread GitBox
SparkQA commented on issue #27213: [SPARK-30516][SQL] statistic estimation of FileScan should take partitionFilters and partition number into account URL: https://github.com/apache/spark/pull/27213#issuecomment-574526311 **[Test build #116754 has

[GitHub] [spark] maropu closed pull request #12302: [SPARK-14390][GraphX] Make initialization step in Pregel optional.

2020-01-14 Thread GitBox
maropu closed pull request #12302: [SPARK-14390][GraphX] Make initialization step in Pregel optional. URL: https://github.com/apache/spark/pull/12302 This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] SparkQA removed a comment on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server

2020-01-14 Thread GitBox
SparkQA removed a comment on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server URL: https://github.com/apache/spark/pull/27208#issuecomment-574492639 **[Test build #116745 has

[GitHub] [spark] maropu commented on issue #12302: [SPARK-14390][GraphX] Make initialization step in Pregel optional.

2020-01-14 Thread GitBox
maropu commented on issue #12302: [SPARK-14390][GraphX] Make initialization step in Pregel optional. URL: https://github.com/apache/spark/pull/12302#issuecomment-574525828 I'll close this pr and the corresponding jira cuz the pr is stale.

[GitHub] [spark] SparkQA commented on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server

2020-01-14 Thread GitBox
SparkQA commented on issue #27208: [WIP][SPARK-30481][CORE] Integrate event log compactor into Spark History Server URL: https://github.com/apache/spark/pull/27208#issuecomment-574525747 **[Test build #116745 has

[GitHub] [spark] fuwhu opened a new pull request #27213: [SPARK-30516][SQL] statistic estimation of FileScan should take partitionFilters and partition number into account

2020-01-14 Thread GitBox
fuwhu opened a new pull request #27213: [SPARK-30516][SQL] statistic estimation of FileScan should take partitionFilters and partition number into account URL: https://github.com/apache/spark/pull/27213 ### What changes were proposed in this pull request? Refine

[GitHub] [spark] AmplabJenkins removed a comment on issue #27212: [SPARK-30515][SQL] Refactor SimplifyBinaryComparison to reduce the time complexity

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27212: [SPARK-30515][SQL] Refactor SimplifyBinaryComparison to reduce the time complexity URL: https://github.com/apache/spark/pull/27212#issuecomment-574522594 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #27212: [SPARK-30515][SQL] Refactor SimplifyBinaryComparison to reduce the time complexity

2020-01-14 Thread GitBox
AmplabJenkins removed a comment on issue #27212: [SPARK-30515][SQL] Refactor SimplifyBinaryComparison to reduce the time complexity URL: https://github.com/apache/spark/pull/27212#issuecomment-574522601 Test PASSed. Refer to this link for build results (access rights to CI server

[GitHub] [spark] AmplabJenkins commented on issue #27212: [SPARK-30515][SQL] Refactor SimplifyBinaryComparison to reduce the time complexity

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27212: [SPARK-30515][SQL] Refactor SimplifyBinaryComparison to reduce the time complexity URL: https://github.com/apache/spark/pull/27212#issuecomment-574522594 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #27212: [SPARK-30515][SQL] Refactor SimplifyBinaryComparison to reduce the time complexity

2020-01-14 Thread GitBox
AmplabJenkins commented on issue #27212: [SPARK-30515][SQL] Refactor SimplifyBinaryComparison to reduce the time complexity URL: https://github.com/apache/spark/pull/27212#issuecomment-574522601 Test PASSed. Refer to this link for build results (access rights to CI server needed):

  1   2   3   4   5   6   7   8   9   10   >