[jira] [Created] (SPARK-35608) Support AQE optimizer side transformUpWithPruning

2021-06-02 Thread XiDuo You (Jira)
XiDuo You created SPARK-35608: - Summary: Support AQE optimizer side transformUpWithPruning Key: SPARK-35608 URL: https://issues.apache.org/jira/browse/SPARK-35608 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-35585) Support propagate empty relation through project/filter

2021-06-02 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35585. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32724

[jira] [Assigned] (SPARK-35585) Support propagate empty relation through project/filter

2021-06-02 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35585: --- Assignee: XiDuo You > Support propagate empty relation through project/filter >

[jira] [Commented] (SPARK-35396) Support to manual close/release entries in MemoryStore and InMemoryRelation instead of replying on GC

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355562#comment-17355562 ] Apache Spark commented on SPARK-35396: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-34808) Removes outer join if it only has distinct on streamed side

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355599#comment-17355599 ] Apache Spark commented on SPARK-34808: -- User 'wangyum' has created a pull request for this issue:

[jira] [Created] (SPARK-35611) Introduce the strategy on mismatched offset for start offset timestamp on Kafka data source

2021-06-02 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-35611: Summary: Introduce the strategy on mismatched offset for start offset timestamp on Kafka data source Key: SPARK-35611 URL: https://issues.apache.org/jira/browse/SPARK-35611

[jira] [Created] (SPARK-35609) Add style rules to prohibit to use a Guava's API which is incompatible with newer versions

2021-06-02 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35609: -- Summary: Add style rules to prohibit to use a Guava's API which is incompatible with newer versions Key: SPARK-35609 URL: https://issues.apache.org/jira/browse/SPARK-35609

[jira] [Assigned] (SPARK-35604) Fix condition check for FULL OUTER sort merge join

2021-06-02 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-35604: -- Assignee: Cheng Su > Fix condition check for FULL OUTER sort merge join >

[jira] [Resolved] (SPARK-35604) Fix condition check for FULL OUTER sort merge join

2021-06-02 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-35604. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32736

[jira] [Assigned] (SPARK-32975) [K8S] - executor fails to be restarted after it goes to ERROR/Failure state

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32975: Assignee: (was: Apache Spark) > [K8S] - executor fails to be restarted after it goes

[jira] [Assigned] (SPARK-32975) [K8S] - executor fails to be restarted after it goes to ERROR/Failure state

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32975: Assignee: Apache Spark > [K8S] - executor fails to be restarted after it goes to

[jira] [Commented] (SPARK-32975) [K8S] - executor fails to be restarted after it goes to ERROR/Failure state

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355513#comment-17355513 ] Apache Spark commented on SPARK-32975: -- User 'cchriswu' has created a pull request for this issue:

[jira] [Commented] (SPARK-35568) UnsupportedOperationException: WholeStageCodegen (3) does not implement doExecuteBroadcast

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1735#comment-1735 ] Apache Spark commented on SPARK-35568: -- User 'JkSelf' has created a pull request for this issue:

[jira] [Commented] (SPARK-35610) Memory leak in Spark interpreter

2021-06-02 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355614#comment-17355614 ] Attila Zsolt Piros commented on SPARK-35610: I am working on this, soon a PR will be opened.

[jira] [Updated] (SPARK-35610) Memory leak in Spark interpreter

2021-06-02 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-35610: --- Description: I have identified this leak by running the Livy tests (I know it is

[jira] [Updated] (SPARK-35610) Memory leak in Spark interpreter

2021-06-02 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-35610: --- Description: I have identified this leak by running the Livy tests (I know it is

[jira] [Updated] (SPARK-35604) Fix condition check for FULL OUTER sort merge join

2021-06-02 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-35604: - Issue Type: Improvement (was: Documentation) > Fix condition check for FULL OUTER sort merge join >

[jira] [Commented] (SPARK-35608) Support AQE optimizer side transformUpWithPruning

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1739#comment-1739 ] Apache Spark commented on SPARK-35608: -- User 'ulysses-you' has created a pull request for this

[jira] [Commented] (SPARK-35523) Fix the default value in Data Source Options page

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355607#comment-17355607 ] Apache Spark commented on SPARK-35523: -- User 'itholic' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35523) Fix the default value in Data Source Options page

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35523: Assignee: (was: Apache Spark) > Fix the default value in Data Source Options page >

[jira] [Assigned] (SPARK-35523) Fix the default value in Data Source Options page

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35523: Assignee: Apache Spark > Fix the default value in Data Source Options page >

[jira] [Created] (SPARK-35610) Memory leak in Spark interpreter

2021-06-02 Thread Attila Zsolt Piros (Jira)
Attila Zsolt Piros created SPARK-35610: -- Summary: Memory leak in Spark interpreter Key: SPARK-35610 URL: https://issues.apache.org/jira/browse/SPARK-35610 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-35074) spark.jars.xxx configs should be moved to config/package.scala

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35074: Assignee: Apache Spark > spark.jars.xxx configs should be moved to config/package.scala

[jira] [Issue Comment Deleted] (SPARK-35074) spark.jars.xxx configs should be moved to config/package.scala

2021-06-02 Thread dc-heros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dc-heros updated SPARK-35074: - Comment: was deleted (was: how about spark.yarn.*** and spark.kerberos.***, should they also be

[jira] [Commented] (SPARK-35074) spark.jars.xxx configs should be moved to config/package.scala

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355619#comment-17355619 ] Apache Spark commented on SPARK-35074: -- User 'dgd-contributor' has created a pull request for this

[jira] [Assigned] (SPARK-35074) spark.jars.xxx configs should be moved to config/package.scala

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35074: Assignee: (was: Apache Spark) > spark.jars.xxx configs should be moved to

[jira] [Assigned] (SPARK-35609) Add style rules to prohibit to use a Guava's API which is incompatible with newer versions

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35609: Assignee: Apache Spark (was: Kousuke Saruta) > Add style rules to prohibit to use a

[jira] [Assigned] (SPARK-35609) Add style rules to prohibit to use a Guava's API which is incompatible with newer versions

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35609: Assignee: Kousuke Saruta (was: Apache Spark) > Add style rules to prohibit to use a

[jira] [Commented] (SPARK-35609) Add style rules to prohibit to use a Guava's API which is incompatible with newer versions

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355529#comment-17355529 ] Apache Spark commented on SPARK-35609: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-35074) spark.jars.xxx configs should be moved to config/package.scala

2021-06-02 Thread dc-heros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355527#comment-17355527 ] dc-heros commented on SPARK-35074: -- how about spark.yarn.*** and spark.kerberos.***, should they also

[jira] [Commented] (SPARK-35396) Support to manual close/release entries in MemoryStore and InMemoryRelation instead of replying on GC

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355561#comment-17355561 ] Apache Spark commented on SPARK-35396: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Updated] (SPARK-35590) pyspark v3.1.1removed pyspark.streaming.kafka?

2021-06-02 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-35590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 楚中天 updated SPARK-35590: Description: I could not import the module that name kafka from pyspark.streaming,and I did not found it in the

[jira] [Updated] (SPARK-35610) Memory leak in Spark interpreter

2021-06-02 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-35610: --- Description: I have identified this leak by running the Livy tests (I know it is

[jira] [Assigned] (SPARK-35608) Support AQE optimizer side transformUpWithPruning

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35608: Assignee: (was: Apache Spark) > Support AQE optimizer side transformUpWithPruning >

[jira] [Assigned] (SPARK-35608) Support AQE optimizer side transformUpWithPruning

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35608: Assignee: Apache Spark > Support AQE optimizer side transformUpWithPruning >

[jira] [Commented] (SPARK-35608) Support AQE optimizer side transformUpWithPruning

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355560#comment-17355560 ] Apache Spark commented on SPARK-35608: -- User 'ulysses-you' has created a pull request for this

[jira] [Updated] (SPARK-35610) Memory leak in Spark interpreter

2021-06-02 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-35610: --- Description: I have identified this leak by running the Livy tests (I know it is

[jira] [Updated] (SPARK-35609) Add style rules to prohibit to use a Guava's API which is incompatible with newer versions

2021-06-02 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35609: --- Description: SPARK-30272 replaced Objects.toStringHelper which is an APIs Guava 14 provides

[jira] [Assigned] (SPARK-35610) Memory leak in Spark interpreter

2021-06-02 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros reassigned SPARK-35610: -- Assignee: Attila Zsolt Piros > Memory leak in Spark interpreter >

[jira] [Updated] (SPARK-35610) Memory leak in Spark interpreter

2021-06-02 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-35610: --- Description: I have identified this leak by running the Livy tests (I know it is

[jira] [Updated] (SPARK-35612) Support LZ4 compression in ORC data source

2021-06-02 Thread Han (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Han updated SPARK-35612: Description: Apache ORC supports LZ4 compression, but we cannot set LZ4 compression in the ORC data source

[jira] [Resolved] (SPARK-34623) Deduplicate window expressions

2021-06-02 Thread Tanel Kiis (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanel Kiis resolved SPARK-34623. Resolution: Won't Do > Deduplicate window expressions > -- > >

[jira] [Resolved] (SPARK-32801) Make InferFiltersFromConstraints take in account EqualNullSafe

2021-06-02 Thread Tanel Kiis (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanel Kiis resolved SPARK-32801. Resolution: Won't Do > Make InferFiltersFromConstraints take in account EqualNullSafe >

[jira] [Assigned] (SPARK-35612) Support LZ4 compression in ORC data source

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35612: Assignee: (was: Apache Spark) > Support LZ4 compression in ORC data source >

[jira] [Commented] (SPARK-35612) Support LZ4 compression in ORC data source

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355743#comment-17355743 ] Apache Spark commented on SPARK-35612: -- User 'fornaix' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35612) Support LZ4 compression in ORC data source

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35612: Assignee: Apache Spark > Support LZ4 compression in ORC data source >

[jira] [Commented] (SPARK-32975) [K8S] - executor fails to be restarted after it goes to ERROR/Failure state

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355836#comment-17355836 ] Apache Spark commented on SPARK-32975: -- User 'cchriswu' has created a pull request for this issue:

[jira] [Updated] (SPARK-35610) Memory leak in Spark interpreter

2021-06-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35610: -- Component/s: (was: Tests) > Memory leak in Spark interpreter >

[jira] [Resolved] (SPARK-35610) Memory leak in Spark interpreter

2021-06-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35610. --- Fix Version/s: 3.0.3 3.1.3 3.2.0 Resolution:

[jira] [Assigned] (SPARK-35611) Introduce the strategy on mismatched offset for start offset timestamp on Kafka data source

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35611: Assignee: Apache Spark > Introduce the strategy on mismatched offset for start offset

[jira] [Assigned] (SPARK-35611) Introduce the strategy on mismatched offset for start offset timestamp on Kafka data source

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35611: Assignee: (was: Apache Spark) > Introduce the strategy on mismatched offset for

[jira] [Commented] (SPARK-35611) Introduce the strategy on mismatched offset for start offset timestamp on Kafka data source

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355647#comment-17355647 ] Apache Spark commented on SPARK-35611: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Updated] (SPARK-35610) Memory leak in Spark interpreter

2021-06-02 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-35610: --- Description: I have identified this leak by running the Livy tests (I know it is

[jira] [Updated] (SPARK-35610) Memory leak in Spark interpreter

2021-06-02 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-35610: --- Description: I have identified this leak by running the Livy tests (I know it is

[jira] [Commented] (SPARK-33696) Upgrade built-in Hive to 2.3.8

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355711#comment-17355711 ] Apache Spark commented on SPARK-33696: -- User 'wangyum' has created a pull request for this issue:

[jira] [Created] (SPARK-35612) Support LZ4 compression in ORC data source

2021-06-02 Thread Han (Jira)
Han created SPARK-35612: --- Summary: Support LZ4 compression in ORC data source Key: SPARK-35612 URL: https://issues.apache.org/jira/browse/SPARK-35612 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-35610) Memory leak in Spark interpreter

2021-06-02 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-35610: --- Description: I have identified this leak by running the Livy tests (I know it is

[jira] [Resolved] (SPARK-21957) Add current_user function

2021-06-02 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21957. - Fix Version/s: 3.2.0 Assignee: Kent Yao Resolution: Fixed > Add current_user

[jira] [Reopened] (SPARK-21957) Add current_user function

2021-06-02 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-21957: - > Add current_user function > - > > Key: SPARK-21957 >

[jira] [Commented] (SPARK-34943) Upgrade flake8 to 3.8.0 or above in Jenkins

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355699#comment-17355699 ] Apache Spark commented on SPARK-34943: -- User 'pingsutw' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34943) Upgrade flake8 to 3.8.0 or above in Jenkins

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34943: Assignee: Apache Spark (was: Shane Knapp) > Upgrade flake8 to 3.8.0 or above in Jenkins

[jira] [Assigned] (SPARK-34943) Upgrade flake8 to 3.8.0 or above in Jenkins

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34943: Assignee: Shane Knapp (was: Apache Spark) > Upgrade flake8 to 3.8.0 or above in Jenkins

[jira] [Updated] (SPARK-35610) Memory leak in Spark interpreter

2021-06-02 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-35610: --- Description: I have identified this leak by running the Livy tests (I know it is

[jira] [Assigned] (SPARK-35059) Group exception messages in hive/execution

2021-06-02 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35059: --- Assignee: jiaan.geng > Group exception messages in hive/execution >

[jira] [Resolved] (SPARK-35059) Group exception messages in hive/execution

2021-06-02 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35059. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32694

[jira] [Updated] (SPARK-35610) Memory leak in Spark interpreter

2021-06-02 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-35610: --- Description: I have identified this leak by running the Livy tests (I know it is

[jira] [Assigned] (SPARK-35610) Memory leak in Spark interpreter

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35610: Assignee: Apache Spark (was: Attila Zsolt Piros) > Memory leak in Spark interpreter >

[jira] [Assigned] (SPARK-35610) Memory leak in Spark interpreter

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35610: Assignee: Attila Zsolt Piros (was: Apache Spark) > Memory leak in Spark interpreter >

[jira] [Commented] (SPARK-35610) Memory leak in Spark interpreter

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355685#comment-17355685 ] Apache Spark commented on SPARK-35610: -- User 'attilapiros' has created a pull request for this

[jira] [Updated] (SPARK-35613) Cache commonly occurring strings from SQLMetrics, JsonProtocol and AccumulatorV2

2021-06-02 Thread Venkata krishnan Sowrirajan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata krishnan Sowrirajan updated SPARK-35613: Summary: Cache commonly occurring strings from SQLMetrics,

[jira] [Assigned] (SPARK-35613) Cache commonly occurring strings from SQLMetrics, JsonProtocol and AccumulatorV2

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35613: Assignee: Apache Spark > Cache commonly occurring strings from SQLMetrics, JsonProtocol

[jira] [Assigned] (SPARK-35613) Cache commonly occurring strings from SQLMetrics, JsonProtocol and AccumulatorV2

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35613: Assignee: (was: Apache Spark) > Cache commonly occurring strings from SQLMetrics,

[jira] [Commented] (SPARK-35613) Cache commonly occurring strings from SQLMetrics, JsonProtocol and AccumulatorV2

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355923#comment-17355923 ] Apache Spark commented on SPARK-35613: -- User 'venkata91' has created a pull request for this issue:

[jira] [Created] (SPARK-35613) Cache commonly occurring strings from SQLMetrics and in JsonProtocol

2021-06-02 Thread Venkata krishnan Sowrirajan (Jira)
Venkata krishnan Sowrirajan created SPARK-35613: --- Summary: Cache commonly occurring strings from SQLMetrics and in JsonProtocol Key: SPARK-35613 URL:

[jira] [Commented] (SPARK-34859) Vectorized parquet reader needs synchronization among pages for column index

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355890#comment-17355890 ] Apache Spark commented on SPARK-34859: -- User 'sunchao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35617) Update GitHub Action docker image to 20210602

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35617: Assignee: Apache Spark > Update GitHub Action docker image to 20210

[jira] [Assigned] (SPARK-35617) Update GitHub Action docker image to 20210602

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35617: Assignee: (was: Apache Spark) > Update GitHub Action docker image to 20210

[jira] [Commented] (SPARK-35617) Update GitHub Action docker image to 20210602

2021-06-02 Thread Apache Spark (Jira)
for this issue: https://github.com/apache/spark/pull/32755 > Update GitHub Action docker image to 20210602 > - > > Key: SPARK-35617 > URL: https://issues.apache.org/jira/browse/SPARK-35617 > Project: Spar

[jira] [Comment Edited] (SPARK-35602) Job crashes with java.io.UTFDataFormatException: encoded string too long

2021-06-02 Thread dejan miljkovic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356138#comment-17356138 ] dejan miljkovic edited comment on SPARK-35602 at 6/3/21, 3:44 AM: -- Hi

[jira] [Updated] (SPARK-35546) Properly handle race conditions in RemoteBlockPushResolver for access to the internal ConcurrentHashMaps with multiple app attempts enabled

2021-06-02 Thread Ye Zhou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou updated SPARK-35546: Summary: Properly handle race conditions in RemoteBlockPushResolver for access to the internal

[jira] [Updated] (SPARK-35546) Properly handle race conditions in RemoteBlockPushResolver for access to the internal ConcurrentHashMaps to handle multiple app attempts

2021-06-02 Thread Ye Zhou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou updated SPARK-35546: Summary: Properly handle race conditions in RemoteBlockPushResolver for access to the internal

[jira] [Updated] (SPARK-35546) Properly handle race conditions in RemoteBlockPushResolver to support push based shuffle with multiple app attempts enabled

2021-06-02 Thread Ye Zhou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou updated SPARK-35546: Summary: Properly handle race conditions in RemoteBlockPushResolver to support push based shuffle with

[jira] [Assigned] (SPARK-35619) Refactor LinearRegression - make huber support virtual centering

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35619: Assignee: (was: Apache Spark) > Refactor LinearRegression - make huber support

[jira] [Assigned] (SPARK-35619) Refactor LinearRegression - make huber support virtual centering

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35619: Assignee: Apache Spark > Refactor LinearRegression - make huber support virtual

[jira] [Commented] (SPARK-35619) Refactor LinearRegression - make huber support virtual centering

2021-06-02 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356079#comment-17356079 ] Apache Spark commented on SPARK-35619: -- User 'zhengruifeng' has created a pull request for this

[jira] [Resolved] (SPARK-35620) Remove documentation build in Python linter

2021-06-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35620. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32760

[jira] [Resolved] (SPARK-35528) Add more options at Data Source Options pages

2021-06-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35528. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32757

[jira] [Assigned] (SPARK-35528) Add more options at Data Source Options pages

2021-06-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35528: Assignee: Haejoon Lee > Add more options at Data Source Options pages >

[jira] [Resolved] (SPARK-31689) ShuffleBlockFetchIterator keeps localBlocks in its memory even though it never uses it

2021-06-02 Thread Chandni Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh resolved SPARK-31689. --- Resolution: Incomplete This will be addressed with SPARK-32922 where this change is

[jira] [Created] (SPARK-35617) Update GitHub Action docker image to 20210602

2021-06-02 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-35617: - Summary: Update GitHub Action docker image to 20210602 Key: SPARK-35617 URL: https://issues.apache.org/jira/browse/SPARK-35617 Project: Spark Issue Type

[jira] [Assigned] (SPARK-35617) Update GitHub Action docker image to 20210602

2021-06-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35617: - Assignee: Dongjoon Hyun > Update GitHub Action docker image to 20210

[jira] [Resolved] (SPARK-35617) Update GitHub Action docker image to 20210602

2021-06-02 Thread Dongjoon Hyun (Jira)
://github.com/apache/spark/pull/32755] > Update GitHub Action docker image to 20210602 > - > > Key: SPARK-35617 > URL: https://issues.apache.org/jira/browse/SPARK-35617 > Project: Spark >

[jira] [Commented] (SPARK-35602) Job crashes with java.io.UTFDataFormatException: encoded string too long

2021-06-02 Thread dejan miljkovic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356159#comment-17356159 ] dejan miljkovic commented on SPARK-35602: - The problem is in

[jira] [Commented] (SPARK-35555) Match to the behaviour of pandas' quantile on boolean

2021-06-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-3?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356173#comment-17356173 ] Hyukjin Kwon commented on SPARK-3: -- https://github.com/pandas-dev/pandas/issues/41792

[jira] [Resolved] (SPARK-35570) Shuffle file leak with external shuffle service enable

2021-06-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35570. -- Resolution: Invalid > Shuffle file leak with external shuffle service enable >

[jira] [Commented] (SPARK-35570) Shuffle file leak with external shuffle service enable

2021-06-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356182#comment-17356182 ] Hyukjin Kwon commented on SPARK-35570: -- I think it's more like a question on the design. You would

[jira] [Created] (SPARK-35616) Make astype data-type-based

2021-06-02 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35616: Summary: Make astype data-type-based Key: SPARK-35616 URL: https://issues.apache.org/jira/browse/SPARK-35616 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-35618) Resolve star expressions in subquery

2021-06-02 Thread Allison Wang (Jira)
Allison Wang created SPARK-35618: Summary: Resolve star expressions in subquery Key: SPARK-35618 URL: https://issues.apache.org/jira/browse/SPARK-35618 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-35621) Add rule id pruning to the TypeCoercion rule

2021-06-02 Thread Yingyi Bu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356090#comment-17356090 ] Yingyi Bu commented on SPARK-35621: --- Work items: * Factor the manual recursion in

[jira] [Issue Comment Deleted] (SPARK-35058) Group exception messages in hive/client

2021-06-02 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-35058: --- Comment: was deleted (was: I'm working on.) > Group exception messages in hive/client >

  1   2   >