[jira] [Updated] (SPARK-34818) Reorder the items in User Guide

[ https://issues.apache.org/jira/browse/SPARK-34818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34818: - Component/s: docs > Reorder the items in User Guide > --- > >

[jira] [Assigned] (SPARK-34818) Reorder the items in User Guide

[ https://issues.apache.org/jira/browse/SPARK-34818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34818: Assignee: Hyukjin Kwon (was: Apache Spark) > Reorder the items in User Guide >

[jira] [Assigned] (SPARK-34818) Reorder the items in User Guide

[ https://issues.apache.org/jira/browse/SPARK-34818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34818: Assignee: Apache Spark (was: Hyukjin Kwon) > Reorder the items in User Guide >

[jira] [Commented] (SPARK-34818) Reorder the items in User Guide

[ https://issues.apache.org/jira/browse/SPARK-34818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305896#comment-17305896 ] Apache Spark commented on SPARK-34818: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Resolved] (SPARK-34225) Jars or file paths which contain spaces are generating FileNotFoundException exception

[ https://issues.apache.org/jira/browse/SPARK-34225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-34225. Fix Version/s: 3.1.2 3.2.0 Assignee: Kousuke Saruta

[jira] [Created] (SPARK-34818) Reorder the items in User Guide

Hyukjin Kwon created SPARK-34818: Summary: Reorder the items in User Guide Key: SPARK-34818 URL: https://issues.apache.org/jira/browse/SPARK-34818 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-34817) read parquet uint8/16/32 logical types

[ https://issues.apache.org/jira/browse/SPARK-34817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34817: Assignee: (was: Apache Spark) > read parquet uint8/16/32 logical types >

[jira] [Commented] (SPARK-34817) read parquet uint8/16/32 logical types

[ https://issues.apache.org/jira/browse/SPARK-34817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305872#comment-17305872 ] Apache Spark commented on SPARK-34817: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34817) read parquet uint8/16/32 logical types

[ https://issues.apache.org/jira/browse/SPARK-34817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34817: Assignee: Apache Spark > read parquet uint8/16/32 logical types >

[jira] [Issue Comment Deleted] (SPARK-33604) Group exception messages in sql/execution

[ https://issues.apache.org/jira/browse/SPARK-33604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-33604: --- Comment: was deleted (was: I'm working on.) > Group exception messages in sql/execution >

[jira] [Created] (SPARK-34817) read parquet uint8/16/32 logical types

Kent Yao created SPARK-34817: Summary: read parquet uint8/16/32 logical types Key: SPARK-34817 URL: https://issues.apache.org/jira/browse/SPARK-34817 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-34816) Support for Parquet unsigned LogicalTypes

[ https://issues.apache.org/jira/browse/SPARK-34816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34816: - Description: Parquet supports some unsigned datatypes. Here is the definition related in

[jira] [Commented] (SPARK-33604) Group exception messages in sql/execution

[ https://issues.apache.org/jira/browse/SPARK-33604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305862#comment-17305862 ] Apache Spark commented on SPARK-33604: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33604) Group exception messages in sql/execution

[ https://issues.apache.org/jira/browse/SPARK-33604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33604: Assignee: (was: Apache Spark) > Group exception messages in sql/execution >

[jira] [Assigned] (SPARK-33604) Group exception messages in sql/execution

[ https://issues.apache.org/jira/browse/SPARK-33604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33604: Assignee: Apache Spark > Group exception messages in sql/execution >

[jira] [Updated] (SPARK-34786) read parquet uint64 as decimal

[ https://issues.apache.org/jira/browse/SPARK-34786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-34786: - Parent: SPARK-34816 Issue Type: Sub-task (was: Improvement) > read parquet uint64 as decimal >

[jira] [Created] (SPARK-34816) Support for Parquet unsigned LogicalTypes

Kent Yao created SPARK-34816: Summary: Support for Parquet unsigned LogicalTypes Key: SPARK-34816 URL: https://issues.apache.org/jira/browse/SPARK-34816 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-34087) a memory leak occurs when we clone the spark session

[ https://issues.apache.org/jira/browse/SPARK-34087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305858#comment-17305858 ] Apache Spark commented on SPARK-34087: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Commented] (SPARK-34087) a memory leak occurs when we clone the spark session

[ https://issues.apache.org/jira/browse/SPARK-34087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305859#comment-17305859 ] Apache Spark commented on SPARK-34087: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Commented] (SPARK-34801) java.lang.NoSuchMethodException: org.apache.hadoop.hive.ql.metadata.Hive.loadPartition

[ https://issues.apache.org/jira/browse/SPARK-34801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305854#comment-17305854 ] zhaojk commented on SPARK-34801: This problem occurred when we installed Apache Spark3.0.2-hadoop3.2 in

[jira] [Updated] (SPARK-34802) Move simplify expression rules before push down rules

[ https://issues.apache.org/jira/browse/SPARK-34802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-34802: Summary: Move simplify expression rules before push down rules (was: Optimize Optimizer

[jira] [Resolved] (SPARK-34470) VectorSlicer use ordering if possible

[ https://issues.apache.org/jira/browse/SPARK-34470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-34470. -- Resolution: Resolved > VectorSlicer use ordering if possible >

[jira] [Assigned] (SPARK-34470) VectorSlicer use ordering if possible

[ https://issues.apache.org/jira/browse/SPARK-34470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng reassigned SPARK-34470: Assignee: zhengruifeng > VectorSlicer use ordering if possible >

[jira] [Commented] (SPARK-34757) Spark submit should ignore cache for SNAPSHOT dependencies

[ https://issues.apache.org/jira/browse/SPARK-34757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305837#comment-17305837 ] Apache Spark commented on SPARK-34757: -- User 'bozhang2820' has created a pull request for this

[jira] [Commented] (SPARK-34757) Spark submit should ignore cache for SNAPSHOT dependencies

[ https://issues.apache.org/jira/browse/SPARK-34757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305836#comment-17305836 ] Apache Spark commented on SPARK-34757: -- User 'bozhang2820' has created a pull request for this

[jira] [Assigned] (SPARK-34815) Update CSVBenchmark

[ https://issues.apache.org/jira/browse/SPARK-34815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34815: Assignee: (was: Apache Spark) > Update CSVBenchmark > > >

[jira] [Updated] (SPARK-34815) Update CSVBenchmark

[ https://issues.apache.org/jira/browse/SPARK-34815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-34815: - Priority: Minor (was: Major) > Update CSVBenchmark > > >

[jira] [Commented] (SPARK-34815) Update CSVBenchmark

[ https://issues.apache.org/jira/browse/SPARK-34815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305833#comment-17305833 ] Apache Spark commented on SPARK-34815: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-34815) Update CSVBenchmark

[ https://issues.apache.org/jira/browse/SPARK-34815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34815: Assignee: Apache Spark > Update CSVBenchmark > > >

[jira] [Created] (SPARK-34815) Update CSVBenchmark

Hyukjin Kwon created SPARK-34815: Summary: Update CSVBenchmark Key: SPARK-34815 URL: https://issues.apache.org/jira/browse/SPARK-34815 Project: Spark Issue Type: Improvement

[jira] [Issue Comment Deleted] (SPARK-34775) Push down limit through window when partitionSpec is not empty

[ https://issues.apache.org/jira/browse/SPARK-34775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-34775: Comment: was deleted (was: User 'viirya' has created a pull request for this issue:

[jira] [Created] (SPARK-34814) LikeSimplification should handle NULL

Yuming Wang created SPARK-34814: --- Summary: LikeSimplification should handle NULL Key: SPARK-34814 URL: https://issues.apache.org/jira/browse/SPARK-34814 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-34772) RebaseDateTime loadRebaseRecords should use Spark classloader instead of context

[ https://issues.apache.org/jira/browse/SPARK-34772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305810#comment-17305810 ] Apache Spark commented on SPARK-34772: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-34772) RebaseDateTime loadRebaseRecords should use Spark classloader instead of context

[ https://issues.apache.org/jira/browse/SPARK-34772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305809#comment-17305809 ] Apache Spark commented on SPARK-34772: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Resolved] (SPARK-34809) Enable spark.hadoopRDD.ignoreEmptySplits by default

[ https://issues.apache.org/jira/browse/SPARK-34809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34809. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31909

[jira] [Assigned] (SPARK-34809) Enable spark.hadoopRDD.ignoreEmptySplits by default

[ https://issues.apache.org/jira/browse/SPARK-34809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34809: - Assignee: Dongjoon Hyun > Enable spark.hadoopRDD.ignoreEmptySplits by default >

[jira] [Assigned] (SPARK-34813) Remove Scala 2.13 build GitHub Action job from branch-3.1

[ https://issues.apache.org/jira/browse/SPARK-34813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34813: - Assignee: Dongjoon Hyun > Remove Scala 2.13 build GitHub Action job from branch-3.1 >

[jira] [Resolved] (SPARK-34813) Remove Scala 2.13 build GitHub Action job from branch-3.1

[ https://issues.apache.org/jira/browse/SPARK-34813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34813. --- Fix Version/s: 3.1.2 Resolution: Fixed Issue resolved by pull request 31915

[jira] [Assigned] (SPARK-34813) Remove Scala 2.13 build GitHub Action job from branch-3.1

[ https://issues.apache.org/jira/browse/SPARK-34813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34813: Assignee: (was: Apache Spark) > Remove Scala 2.13 build GitHub Action job from

[jira] [Assigned] (SPARK-34813) Remove Scala 2.13 build GitHub Action job from branch-3.1

[ https://issues.apache.org/jira/browse/SPARK-34813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34813: Assignee: Apache Spark > Remove Scala 2.13 build GitHub Action job from branch-3.1 >

[jira] [Commented] (SPARK-34813) Remove Scala 2.13 build GitHub Action job from branch-3.1

[ https://issues.apache.org/jira/browse/SPARK-34813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305770#comment-17305770 ] Apache Spark commented on SPARK-34813: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-34813) Remove Scala 2.13 build GitHub Action job from branch-3.1

[ https://issues.apache.org/jira/browse/SPARK-34813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34813: -- Summary: Remove Scala 2.13 build GitHub Action job from branch-3.1 (was: Remove Scala 2.13

[jira] [Updated] (SPARK-34813) Remove Scala 2.13 build from branch-3.1

[ https://issues.apache.org/jira/browse/SPARK-34813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34813: -- Component/s: Project Infra > Remove Scala 2.13 build from branch-3.1 >

[jira] [Created] (SPARK-34813) Remove Scala 2.13 build from branch-3.1

Dongjoon Hyun created SPARK-34813: - Summary: Remove Scala 2.13 build from branch-3.1 Key: SPARK-34813 URL: https://issues.apache.org/jira/browse/SPARK-34813 Project: Spark Issue Type: Test

[jira] [Resolved] (SPARK-34811) Redact fs.s3a.access.key like secret and token

[ https://issues.apache.org/jira/browse/SPARK-34811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34811. --- Fix Version/s: 3.0.3 3.1.2 2.4.8

[jira] [Assigned] (SPARK-34811) Redact fs.s3a.access.key like secret and token

[ https://issues.apache.org/jira/browse/SPARK-34811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34811: - Assignee: Dongjoon Hyun > Redact fs.s3a.access.key like secret and token >

[jira] [Created] (SPARK-34812) RowNumberLike and RankLike should not be nullable

Tanel Kiis created SPARK-34812: -- Summary: RowNumberLike and RankLike should not be nullable Key: SPARK-34812 URL: https://issues.apache.org/jira/browse/SPARK-34812 Project: Spark Issue Type:

[jira] [Commented] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305738#comment-17305738 ] Min Shen commented on SPARK-30602: -- Just an update for where we are: The team at LinkedIn has been

[jira] [Commented] (SPARK-33122) Remove redundant aggregates in the Optimzier

[ https://issues.apache.org/jira/browse/SPARK-33122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305675#comment-17305675 ] Apache Spark commented on SPARK-33122: -- User 'tanelk' has created a pull request for this issue:

[jira] [Updated] (SPARK-34796) Codegen compilation error for query with LIMIT operator and without AQE

[ https://issues.apache.org/jira/browse/SPARK-34796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-34796: - Fix Version/s: 3.1.2 > Codegen compilation error for query with LIMIT operator and

[jira] [Updated] (SPARK-34581) BoundAttribute issue after optimization by BooleanSimplification and PushFoldableIntoBranches

[ https://issues.apache.org/jira/browse/SPARK-34581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-34581: --- Affects Version/s: 3.0.2 3.1.1 > BoundAttribute issue after optimization by

[jira] [Commented] (SPARK-34581) BoundAttribute issue after optimization by BooleanSimplification and PushFoldableIntoBranches

[ https://issues.apache.org/jira/browse/SPARK-34581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305654#comment-17305654 ] Apache Spark commented on SPARK-34581: -- User 'peter-toth' has created a pull request for this

[jira] [Assigned] (SPARK-34581) BoundAttribute issue after optimization by BooleanSimplification and PushFoldableIntoBranches

[ https://issues.apache.org/jira/browse/SPARK-34581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34581: Assignee: Apache Spark > BoundAttribute issue after optimization by

[jira] [Assigned] (SPARK-34581) BoundAttribute issue after optimization by BooleanSimplification and PushFoldableIntoBranches

[ https://issues.apache.org/jira/browse/SPARK-34581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34581: Assignee: (was: Apache Spark) > BoundAttribute issue after optimization by

[jira] [Commented] (SPARK-34581) BoundAttribute issue after optimization by BooleanSimplification and PushFoldableIntoBranches

[ https://issues.apache.org/jira/browse/SPARK-34581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305653#comment-17305653 ] Apache Spark commented on SPARK-34581: -- User 'peter-toth' has created a pull request for this

[jira] [Assigned] (SPARK-34784) Upgrade Jackson to 2.12.2

[ https://issues.apache.org/jira/browse/SPARK-34784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-34784: --- Assignee: Xingchao, Zhang > Upgrade Jackson to 2.12.2 > - > >

[jira] [Resolved] (SPARK-34784) Upgrade Jackson to 2.12.2

[ https://issues.apache.org/jira/browse/SPARK-34784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-34784. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31878

[jira] [Commented] (SPARK-26625) spark.redaction.regex should include oauthToken

[ https://issues.apache.org/jira/browse/SPARK-26625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305630#comment-17305630 ] Dongjoon Hyun commented on SPARK-26625: --- I assigned this issue to [~vinooganesh] and backported

[jira] [Assigned] (SPARK-26625) spark.redaction.regex should include oauthToken

[ https://issues.apache.org/jira/browse/SPARK-26625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-26625: - Assignee: Vinoo Ganesh > spark.redaction.regex should include oauthToken >

[jira] [Updated] (SPARK-26625) spark.redaction.regex should include oauthToken

[ https://issues.apache.org/jira/browse/SPARK-26625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26625: -- Fix Version/s: 2.4.8 > spark.redaction.regex should include oauthToken >

[jira] [Commented] (SPARK-34811) Redact fs.s3a.access.key like secret and token

[ https://issues.apache.org/jira/browse/SPARK-34811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305629#comment-17305629 ] Apache Spark commented on SPARK-34811: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-34811) Redact fs.s3a.access.key like secret and token

[ https://issues.apache.org/jira/browse/SPARK-34811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305628#comment-17305628 ] Apache Spark commented on SPARK-34811: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-34811) Redact fs.s3a.access.key like secret and token

[ https://issues.apache.org/jira/browse/SPARK-34811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34811: Assignee: (was: Apache Spark) > Redact fs.s3a.access.key like secret and token >

[jira] [Assigned] (SPARK-34811) Redact fs.s3a.access.key like secret and token

[ https://issues.apache.org/jira/browse/SPARK-34811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34811: Assignee: Apache Spark > Redact fs.s3a.access.key like secret and token >

[jira] [Created] (SPARK-34811) Redact fs.s3a.access.key like secret and token

Dongjoon Hyun created SPARK-34811: - Summary: Redact fs.s3a.access.key like secret and token Key: SPARK-34811 URL: https://issues.apache.org/jira/browse/SPARK-34811 Project: Spark Issue Type: