[jira] [Assigned] (SPARK-32634) Introduce sort-based fallback mechanism for shuffled hash join

[ https://issues.apache.org/jira/browse/SPARK-32634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32634: Assignee: Apache Spark > Introduce sort-based fallback mechanism for shuffled hash join

[jira] [Commented] (SPARK-32634) Introduce sort-based fallback mechanism for shuffled hash join

[ https://issues.apache.org/jira/browse/SPARK-32634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324173#comment-17324173 ] Apache Spark commented on SPARK-32634: -- User 'c21' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32634) Introduce sort-based fallback mechanism for shuffled hash join

[ https://issues.apache.org/jira/browse/SPARK-32634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32634: Assignee: (was: Apache Spark) > Introduce sort-based fallback mechanism for shuffled

[jira] [Comment Edited] (SPARK-34738) Upgrade Minikube and kubernetes and move to docker 'virtualization' layer

[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324167#comment-17324167 ] Attila Zsolt Piros edited comment on SPARK-34738 at 4/17/21, 4:27 AM:

[jira] [Commented] (SPARK-34738) Upgrade Minikube and kubernetes and move to docker 'virtualization' layer

[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324167#comment-17324167 ] Attila Zsolt Piros commented on SPARK-34738: My new commit:

[jira] [Comment Edited] (SPARK-34738) Upgrade Minikube and kubernetes and move to docker 'virtualization' layer

[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324165#comment-17324165 ] Attila Zsolt Piros edited comment on SPARK-34738 at 4/17/21, 3:56 AM:

[jira] [Commented] (SPARK-34738) Upgrade Minikube and kubernetes and move to docker 'virtualization' layer

[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324165#comment-17324165 ] Attila Zsolt Piros commented on SPARK-34738: @shane I can modify my PR to skip the PV tests

[jira] [Commented] (SPARK-34738) Upgrade Minikube and kubernetes and move to docker 'virtualization' layer

[ https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324131#comment-17324131 ] Shane Knapp commented on SPARK-34738: - alright, my canary build w/skipping the PV integration test

[jira] [Commented] (SPARK-35108) Pickle produces incorrect key labels for GenericRowWithSchema (data corruption)

[ https://issues.apache.org/jira/browse/SPARK-35108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324088#comment-17324088 ] Robert Joseph Evans commented on SPARK-35108: - If you have SPARK_HOME set when you run

[jira] [Updated] (SPARK-35108) Pickle produces incorrect key labels for GenericRowWithSchema (data corruption)

[ https://issues.apache.org/jira/browse/SPARK-35108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated SPARK-35108: Attachment: test.sh test.py > Pickle produces incorrect key

[jira] [Created] (SPARK-35108) Pickle produces incorrect key labels for GenericRowWithSchema (data corruption)

Robert Joseph Evans created SPARK-35108: --- Summary: Pickle produces incorrect key labels for GenericRowWithSchema (data corruption) Key: SPARK-35108 URL: https://issues.apache.org/jira/browse/SPARK-35108

[jira] [Assigned] (SPARK-35107) Parse unit-to-unit interval literals to ANSI intervals

[ https://issues.apache.org/jira/browse/SPARK-35107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35107: Assignee: Max Gekk (was: Apache Spark) > Parse unit-to-unit interval literals to ANSI

[jira] [Commented] (SPARK-35107) Parse unit-to-unit interval literals to ANSI intervals

[ https://issues.apache.org/jira/browse/SPARK-35107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324034#comment-17324034 ] Apache Spark commented on SPARK-35107: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35107) Parse unit-to-unit interval literals to ANSI intervals

[ https://issues.apache.org/jira/browse/SPARK-35107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35107: Assignee: Apache Spark (was: Max Gekk) > Parse unit-to-unit interval literals to ANSI

[jira] [Created] (SPARK-35107) Parse unit-to-unit interval literals to ANSI intervals

Max Gekk created SPARK-35107: Summary: Parse unit-to-unit interval literals to ANSI intervals Key: SPARK-35107 URL: https://issues.apache.org/jira/browse/SPARK-35107 Project: Spark Issue Type:

[jira] [Commented] (SPARK-35097) Add column name to SparkUpgradeException about ancient datetime

[ https://issues.apache.org/jira/browse/SPARK-35097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323998#comment-17323998 ] angerszhu commented on SPARK-35097: --- [~smilegator] Sure, working on this. > Add column name to

[jira] [Commented] (SPARK-35103) Improve type coercion rule performance

[ https://issues.apache.org/jira/browse/SPARK-35103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323986#comment-17323986 ] Apache Spark commented on SPARK-35103: -- User 'sigmod' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35103) Improve type coercion rule performance

[ https://issues.apache.org/jira/browse/SPARK-35103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35103: Assignee: (was: Apache Spark) > Improve type coercion rule performance >

[jira] [Assigned] (SPARK-35103) Improve type coercion rule performance

[ https://issues.apache.org/jira/browse/SPARK-35103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35103: Assignee: Apache Spark > Improve type coercion rule performance >

[jira] [Commented] (SPARK-35103) Improve type coercion rule performance

[ https://issues.apache.org/jira/browse/SPARK-35103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323984#comment-17323984 ] Apache Spark commented on SPARK-35103: -- User 'sigmod' has created a pull request for this issue:

[jira] [Commented] (SPARK-35097) Add column name to SparkUpgradeException about ancient datetime

[ https://issues.apache.org/jira/browse/SPARK-35097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323943#comment-17323943 ] Xiao Li commented on SPARK-35097: - Can you help fix this, [~angerszhuuu] ? > Add column name to

[jira] [Commented] (SPARK-35106) HadoopMapReduceCommitProtocol performs bad rename when dynamic partition overwrite is used

[ https://issues.apache.org/jira/browse/SPARK-35106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323924#comment-17323924 ] Apache Spark commented on SPARK-35106: -- User 'xkrogen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35106) HadoopMapReduceCommitProtocol performs bad rename when dynamic partition overwrite is used

[ https://issues.apache.org/jira/browse/SPARK-35106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35106: Assignee: (was: Apache Spark) > HadoopMapReduceCommitProtocol performs bad rename

[jira] [Assigned] (SPARK-35106) HadoopMapReduceCommitProtocol performs bad rename when dynamic partition overwrite is used

[ https://issues.apache.org/jira/browse/SPARK-35106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35106: Assignee: Apache Spark > HadoopMapReduceCommitProtocol performs bad rename when dynamic

[jira] [Created] (SPARK-35106) HadoopMapReduceCommitProtocol performs bad rename when dynamic partition overwrite is used

Erik Krogen created SPARK-35106: --- Summary: HadoopMapReduceCommitProtocol performs bad rename when dynamic partition overwrite is used Key: SPARK-35106 URL: https://issues.apache.org/jira/browse/SPARK-35106

[jira] [Commented] (SPARK-19842) Informational Referential Integrity Constraints Support in Spark

[ https://issues.apache.org/jira/browse/SPARK-19842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323773#comment-17323773 ] Ste Millington commented on SPARK-19842: Is there any progress on this, has anything landed in

[jira] [Updated] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianhui Wang updated SPARK-30602: - Description: In a large deployment of a Spark compute infrastructure, Spark shuffle is

[jira] [Assigned] (SPARK-35105) Support multiple paths for ADD FILE/JAR/ARCHIVE commands

[ https://issues.apache.org/jira/browse/SPARK-35105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35105: Assignee: Kousuke Saruta (was: Apache Spark) > Support multiple paths for ADD

[jira] [Assigned] (SPARK-35105) Support multiple paths for ADD FILE/JAR/ARCHIVE commands

[ https://issues.apache.org/jira/browse/SPARK-35105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35105: Assignee: Apache Spark (was: Kousuke Saruta) > Support multiple paths for ADD

[jira] [Commented] (SPARK-35105) Support multiple paths for ADD FILE/JAR/ARCHIVE commands

[ https://issues.apache.org/jira/browse/SPARK-35105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323618#comment-17323618 ] Apache Spark commented on SPARK-35105: -- User 'sarutak' has created a pull request for this issue:

[jira] [Created] (SPARK-35105) Support multiple paths for ADD FILE/JAR/ARCHIVE commands

Kousuke Saruta created SPARK-35105: -- Summary: Support multiple paths for ADD FILE/JAR/ARCHIVE commands Key: SPARK-35105 URL: https://issues.apache.org/jira/browse/SPARK-35105 Project: Spark

[jira] [Resolved] (SPARK-34995) Port/integrate Koalas remaining codes into PySpark

[ https://issues.apache.org/jira/browse/SPARK-34995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34995. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32197

[jira] [Assigned] (SPARK-34995) Port/integrate Koalas remaining codes into PySpark

[ https://issues.apache.org/jira/browse/SPARK-34995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-34995: Assignee: Haejoon Lee > Port/integrate Koalas remaining codes into PySpark >

[jira] [Assigned] (SPARK-34494) Move JSON data source options from Python and Scala into a single page.

[ https://issues.apache.org/jira/browse/SPARK-34494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34494: Assignee: Apache Spark > Move JSON data source options from Python and Scala into a

[jira] [Commented] (SPARK-34494) Move JSON data source options from Python and Scala into a single page.

[ https://issues.apache.org/jira/browse/SPARK-34494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17322718#comment-17322718 ] Apache Spark commented on SPARK-34494: -- User 'itholic' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34494) Move JSON data source options from Python and Scala into a single page.

[ https://issues.apache.org/jira/browse/SPARK-34494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34494: Assignee: (was: Apache Spark) > Move JSON data source options from Python and Scala

[jira] [Resolved] (SPARK-35104) Fix ugly indentation of multiple JSON records in a single split file generated by JacksonGenerator when pretty option is true

[ https://issues.apache.org/jira/browse/SPARK-35104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35104. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32203

[jira] [Updated] (SPARK-34494) Move JSON data source options from Python and Scala into a single page.

[ https://issues.apache.org/jira/browse/SPARK-34494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34494: Summary: Move JSON data source options from Python and Scala into a single page. (was: move JSON

[jira] [Updated] (SPARK-34494) move JSON data source options from Python and Scala into a single page.

[ https://issues.apache.org/jira/browse/SPARK-34494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34494: Summary: move JSON data source options from Python and Scala into a single page. (was: Move

[jira] [Resolved] (SPARK-35083) Support remote scheduler pool file

[ https://issues.apache.org/jira/browse/SPARK-35083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35083. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32184

[jira] [Assigned] (SPARK-35083) Support remote scheduler pool file

[ https://issues.apache.org/jira/browse/SPARK-35083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35083: - Assignee: ulysses you > Support remote scheduler pool file >

[jira] [Updated] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianhui Wang updated SPARK-30602: - Description: 白月山火禾In a large deployment of a Spark compute infrastructure, Spark shuffle is

[jira] [Assigned] (SPARK-35104) Fix ugly indentation of multiple JSON records in a single split file generated by JacksonGenerator when pretty option is true

[ https://issues.apache.org/jira/browse/SPARK-35104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35104: Assignee: Kousuke Saruta (was: Apache Spark) > Fix ugly indentation of multiple JSON

[jira] [Commented] (SPARK-35104) Fix ugly indentation of multiple JSON records in a single split file generated by JacksonGenerator when pretty option is true

[ https://issues.apache.org/jira/browse/SPARK-35104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17322647#comment-17322647 ] Apache Spark commented on SPARK-35104: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35104) Fix ugly indentation of multiple JSON records in a single split file generated by JacksonGenerator when pretty option is true

[ https://issues.apache.org/jira/browse/SPARK-35104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35104: Assignee: Apache Spark (was: Kousuke Saruta) > Fix ugly indentation of multiple JSON

[jira] [Updated] (SPARK-35104) Fix ugly indentation of multiple JSON records in a single split file generated by JacksonGenerator when pretty option is true

[ https://issues.apache.org/jira/browse/SPARK-35104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35104: --- Description: When writing multiple JSON records into a single split file with pretty option

[jira] [Created] (SPARK-35104) Fix ugly indentation of multiple JSON records in a single split file generated by JacksonGenerator when pretty option is true

Kousuke Saruta created SPARK-35104: -- Summary: Fix ugly indentation of multiple JSON records in a single split file generated by JacksonGenerator when pretty option is true Key: SPARK-35104 URL: