[jira] [Created] (SPARK-37978) Remove the useless ChunkFetchFailureException class

weixiuli created SPARK-37978: Summary: Remove the useless ChunkFetchFailureException class Key: SPARK-37978 URL: https://issues.apache.org/jira/browse/SPARK-37978 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-27442) ParquetFileFormat fails to read column named with invalid characters

[ https://issues.apache.org/jira/browse/SPARK-27442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-27442: --- Assignee: angerszhu > ParquetFileFormat fails to read column named with invalid characters

[jira] [Resolved] (SPARK-27442) ParquetFileFormat fails to read column named with invalid characters

[ https://issues.apache.org/jira/browse/SPARK-27442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27442. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35229

[jira] [Assigned] (SPARK-37929) Support cascade mode for `dropNamespace` API

[ https://issues.apache.org/jira/browse/SPARK-37929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37929: --- Assignee: dch nguyen > Support cascade mode for `dropNamespace` API >

[jira] [Resolved] (SPARK-37929) Support cascade mode for `dropNamespace` API

[ https://issues.apache.org/jira/browse/SPARK-37929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37929. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35246

[jira] [Reopened] (SPARK-28516) Data Type Formatting Functions: `to_char`

[ https://issues.apache.org/jira/browse/SPARK-28516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng reopened SPARK-28516: > Data Type Formatting Functions: `to_char` > - > >

[jira] [Assigned] (SPARK-37967) ConstantFolding/ Literal.create support ObjectType

[ https://issues.apache.org/jira/browse/SPARK-37967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37967: --- Assignee: angerszhu > ConstantFolding/ Literal.create support ObjectType >

[jira] [Resolved] (SPARK-37967) ConstantFolding/ Literal.create support ObjectType

[ https://issues.apache.org/jira/browse/SPARK-37967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37967. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35255

[jira] [Assigned] (SPARK-37977) Upgrade ORC to 1.6.13

[ https://issues.apache.org/jira/browse/SPARK-37977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37977: Assignee: Dongjoon Hyun > Upgrade ORC to 1.6.13 > - > >

[jira] [Resolved] (SPARK-37977) Upgrade ORC to 1.6.13

[ https://issues.apache.org/jira/browse/SPARK-37977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37977. -- Fix Version/s: 3.2.2 Resolution: Fixed Issue resolved by pull request 35266

[jira] [Commented] (SPARK-37971) Apply and evaluate expressions row-wise in a Spark DataFrame

[ https://issues.apache.org/jira/browse/SPARK-37971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479820#comment-17479820 ] Hyukjin Kwon commented on SPARK-37971: -- [~carlosft] you can manually transpose and perform row-wise

[jira] [Updated] (SPARK-37971) Apply and evaluate expressions row-wise in a Spark DataFrame

[ https://issues.apache.org/jira/browse/SPARK-37971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37971: - Priority: Major (was: Critical) > Apply and evaluate expressions row-wise in a Spark DataFrame

[jira] [Commented] (SPARK-37976) All tasks finish but spark job does not conclude. Forever waits for [DONE]

[ https://issues.apache.org/jira/browse/SPARK-37976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479819#comment-17479819 ] Hyukjin Kwon commented on SPARK-37976: -- [~nitinsiwach] are you able to provide the reproducer? >

[jira] [Updated] (SPARK-37976) All tasks finish but spark job does not conclude. Forever waits for [DONE]

[ https://issues.apache.org/jira/browse/SPARK-37976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37976: - Component/s: YARN > All tasks finish but spark job does not conclude. Forever waits for [DONE]

[jira] [Updated] (SPARK-37976) All tasks finish but spark job does not conclude. Forever waits for [DONE]

[ https://issues.apache.org/jira/browse/SPARK-37976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37976: - Priority: Major (was: Blocker) > All tasks finish but spark job does not conclude. Forever

[jira] [Assigned] (SPARK-37977) Upgrade ORC to 1.6.13

[ https://issues.apache.org/jira/browse/SPARK-37977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37977: Assignee: (was: Apache Spark) > Upgrade ORC to 1.6.13 > - > >

[jira] [Commented] (SPARK-37977) Upgrade ORC to 1.6.13

[ https://issues.apache.org/jira/browse/SPARK-37977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479818#comment-17479818 ] Apache Spark commented on SPARK-37977: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-37977) Upgrade ORC to 1.6.13

[ https://issues.apache.org/jira/browse/SPARK-37977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37977: Assignee: Apache Spark > Upgrade ORC to 1.6.13 > - > >

[jira] [Assigned] (SPARK-37888) Unify v1 and v2 DESCRIBE TABLE tests

[ https://issues.apache.org/jira/browse/SPARK-37888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37888: Assignee: (was: Apache Spark) > Unify v1 and v2 DESCRIBE TABLE tests >

[jira] [Assigned] (SPARK-37888) Unify v1 and v2 DESCRIBE TABLE tests

[ https://issues.apache.org/jira/browse/SPARK-37888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37888: Assignee: Apache Spark > Unify v1 and v2 DESCRIBE TABLE tests >

[jira] [Commented] (SPARK-37888) Unify v1 and v2 DESCRIBE TABLE tests

[ https://issues.apache.org/jira/browse/SPARK-37888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479817#comment-17479817 ] Apache Spark commented on SPARK-37888: -- User 'imback82' has created a pull request for this issue:

[jira] [Created] (SPARK-37977) Upgrade ORC to 1.6.13

Dongjoon Hyun created SPARK-37977: - Summary: Upgrade ORC to 1.6.13 Key: SPARK-37977 URL: https://issues.apache.org/jira/browse/SPARK-37977 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-37963) Need to update Partition URI after renaming table in InMemoryCatalog

[ https://issues.apache.org/jira/browse/SPARK-37963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479813#comment-17479813 ] Huaxin Gao commented on SPARK-37963: Changed the fix version to 3.2.2 for now. Will change back if

[jira] [Updated] (SPARK-37963) Need to update Partition URI after renaming table in InMemoryCatalog

[ https://issues.apache.org/jira/browse/SPARK-37963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-37963: --- Fix Version/s: 3.2.2 (was: 3.2.1) > Need to update Partition URI after

[jira] [Comment Edited] (SPARK-30661) KMeans blockify input vectors

[ https://issues.apache.org/jira/browse/SPARK-30661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479793#comment-17479793 ] zhengruifeng edited comment on SPARK-30661 at 1/21/22, 2:54 AM:

[jira] [Updated] (SPARK-30661) KMeans blockify input vectors

[ https://issues.apache.org/jira/browse/SPARK-30661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-30661: - Attachment: blockify_kmeans.png > KMeans blockify input vectors > -

[jira] [Commented] (SPARK-30661) KMeans blockify input vectors

[ https://issues.apache.org/jira/browse/SPARK-30661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479793#comment-17479793 ] zhengruifeng commented on SPARK-30661: -- according to

[jira] [Commented] (SPARK-37037) Improve byte array sort by unify compareTo function of UTF8String and ByteArray

[ https://issues.apache.org/jira/browse/SPARK-37037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479778#comment-17479778 ] Apache Spark commented on SPARK-37037: -- User 'ulysses-you' has created a pull request for this

[jira] [Commented] (SPARK-37037) Improve byte array sort by unify compareTo function of UTF8String and ByteArray

[ https://issues.apache.org/jira/browse/SPARK-37037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479777#comment-17479777 ] Apache Spark commented on SPARK-37037: -- User 'ulysses-you' has created a pull request for this

[jira] [Commented] (SPARK-37897) Filter with subexpression elimination may cause query failed

[ https://issues.apache.org/jira/browse/SPARK-37897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479764#comment-17479764 ] hujiahua commented on SPARK-37897: -- [~viirya] Thank you for your reply. But I'm curious what is the

[jira] [Commented] (SPARK-23897) Guava version

[ https://issues.apache.org/jira/browse/SPARK-23897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479744#comment-17479744 ] phoebe chen commented on SPARK-23897: - The currently in-use Guava version 14.0.1 has following

[jira] [Commented] (SPARK-37806) Support minimum number of tasks per executor before being rolling

[ https://issues.apache.org/jira/browse/SPARK-37806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479724#comment-17479724 ] Apache Spark commented on SPARK-37806: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-37976) All tasks finish but spark job does not conclude. Forever waits for [DONE]

Nitin Siwach created SPARK-37976: Summary: All tasks finish but spark job does not conclude. Forever waits for [DONE] Key: SPARK-37976 URL: https://issues.apache.org/jira/browse/SPARK-37976 Project:

[jira] [Resolved] (SPARK-37083) Inline type hints for python/pyspark/accumulators.py

[ https://issues.apache.org/jira/browse/SPARK-37083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37083. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-37083) Inline type hints for python/pyspark/accumulators.py

[ https://issues.apache.org/jira/browse/SPARK-37083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37083: -- Assignee: dch nguyen > Inline type hints for python/pyspark/accumulators.py

[jira] [Commented] (SPARK-37974) Implement vectorized DELTA_BYTE_ARRAY and DELTA_LENGTH_BYTE_ARRAY encodings for Parquet V2 support

[ https://issues.apache.org/jira/browse/SPARK-37974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479642#comment-17479642 ] Apache Spark commented on SPARK-37974: -- User 'parthchandra' has created a pull request for this

[jira] [Assigned] (SPARK-37974) Implement vectorized DELTA_BYTE_ARRAY and DELTA_LENGTH_BYTE_ARRAY encodings for Parquet V2 support

[ https://issues.apache.org/jira/browse/SPARK-37974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37974: Assignee: Apache Spark > Implement vectorized DELTA_BYTE_ARRAY and

[jira] [Commented] (SPARK-37974) Implement vectorized DELTA_BYTE_ARRAY and DELTA_LENGTH_BYTE_ARRAY encodings for Parquet V2 support

[ https://issues.apache.org/jira/browse/SPARK-37974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479641#comment-17479641 ] Apache Spark commented on SPARK-37974: -- User 'parthchandra' has created a pull request for this

[jira] [Assigned] (SPARK-37974) Implement vectorized DELTA_BYTE_ARRAY and DELTA_LENGTH_BYTE_ARRAY encodings for Parquet V2 support

[ https://issues.apache.org/jira/browse/SPARK-37974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37974: Assignee: (was: Apache Spark) > Implement vectorized DELTA_BYTE_ARRAY and

[jira] [Created] (SPARK-37975) Implement vectorized BYTE_STREAM_SPLIT encoding for Parquet V2 support

Parth Chandra created SPARK-37975: - Summary: Implement vectorized BYTE_STREAM_SPLIT encoding for Parquet V2 support Key: SPARK-37975 URL: https://issues.apache.org/jira/browse/SPARK-37975 Project:

[jira] [Updated] (SPARK-37974) Implement vectorized DELTA_BYTE_ARRAY and DELTA_LENGTH_BYTE_ARRAY encodings for Parquet V2 support

[ https://issues.apache.org/jira/browse/SPARK-37974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated SPARK-37974: -- External issue ID: (was: SPARK-36879) > Implement vectorized DELTA_BYTE_ARRAY and

[jira] [Created] (SPARK-37974) Implement vectorized DELTA_BYTE_ARRAY and DELTA_LENGTH_BYTE_ARRAY encodings for Parquet V2 support

Parth Chandra created SPARK-37974: - Summary: Implement vectorized DELTA_BYTE_ARRAY and DELTA_LENGTH_BYTE_ARRAY encodings for Parquet V2 support Key: SPARK-37974 URL:

[jira] [Updated] (SPARK-37973) Directly call super.getDefaultReadLimit when scala issue 12523 is fixed

[ https://issues.apache.org/jira/browse/SPARK-37973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Jerry Peng updated SPARK-37973: -- Description: In regards to [https://github.com/apache/spark/pull/35238] and more

[jira] [Created] (SPARK-37973) Directly call super.getDefaultReadLimit when scala issue 12523 is fixed

Boyang Jerry Peng created SPARK-37973: - Summary: Directly call super.getDefaultReadLimit when scala issue 12523 is fixed Key: SPARK-37973 URL: https://issues.apache.org/jira/browse/SPARK-37973

[jira] [Resolved] (SPARK-37968) Upgrade commons-collections3 to commons-collections4

[ https://issues.apache.org/jira/browse/SPARK-37968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37968. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35257

[jira] [Assigned] (SPARK-37968) Upgrade commons-collections3 to commons-collections4

[ https://issues.apache.org/jira/browse/SPARK-37968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37968: - Assignee: Yang Jie > Upgrade commons-collections3 to commons-collections4 >

[jira] [Commented] (SPARK-36879) Support Parquet v2 data page encodings for the vectorized path

[ https://issues.apache.org/jira/browse/SPARK-36879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479602#comment-17479602 ] Apache Spark commented on SPARK-36879: -- User 'parthchandra' has created a pull request for this

[jira] [Assigned] (SPARK-37972) Typing incompatibilities with numpy==1.22.x

[ https://issues.apache.org/jira/browse/SPARK-37972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37972: Assignee: Apache Spark > Typing incompatibilities with numpy==1.22.x >

[jira] [Assigned] (SPARK-37972) Typing incompatibilities with numpy==1.22.x

[ https://issues.apache.org/jira/browse/SPARK-37972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37972: Assignee: (was: Apache Spark) > Typing incompatibilities with numpy==1.22.x >

[jira] [Commented] (SPARK-37972) Typing incompatibilities with numpy==1.22.x

[ https://issues.apache.org/jira/browse/SPARK-37972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479448#comment-17479448 ] Apache Spark commented on SPARK-37972: -- User 'zero323' has created a pull request for this issue:

[jira] [Updated] (SPARK-37972) Typing incompatibilities with numpy==1.22.x

[ https://issues.apache.org/jira/browse/SPARK-37972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37972: --- Description: When type checked against {{numpy==1.22}} mypy detects following

[jira] [Updated] (SPARK-37972) Typing incompatibilities with numpy==1.22.x

[ https://issues.apache.org/jira/browse/SPARK-37972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37972: --- Summary: Typing incompatibilities with numpy==1.22.x (was: Typing

[jira] [Created] (SPARK-37972) Typing incompatibilities with numpy~=1.22

Maciej Szymkiewicz created SPARK-37972: -- Summary: Typing incompatibilities with numpy~=1.22 Key: SPARK-37972 URL: https://issues.apache.org/jira/browse/SPARK-37972 Project: Spark Issue

[jira] [Updated] (SPARK-37971) Apply and evaluate expressions row-wise in a Spark DataFrame

[ https://issues.apache.org/jira/browse/SPARK-37971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carlos Gameiro updated SPARK-37971: --- Description: This functionality would serve very specific use cases. Consider a DataFrame

[jira] [Updated] (SPARK-37971) Apply and evaluate expressions row-wise in a Spark DataFrame

[ https://issues.apache.org/jira/browse/SPARK-37971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carlos Gameiro updated SPARK-37971: --- Description: This functionality would serve very specific use cases. Consider a DataFrame

[jira] [Updated] (SPARK-37971) Apply and evaluate expressions row-wise in a Spark DataFrame

[ https://issues.apache.org/jira/browse/SPARK-37971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carlos Gameiro updated SPARK-37971: --- Summary: Apply and evaluate expressions row-wise in a Spark DataFrame (was: Apply and

[jira] [Updated] (SPARK-37971) Apply and evaluate expressiosn row-wise in a Spark DataFrame

[ https://issues.apache.org/jira/browse/SPARK-37971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carlos Gameiro updated SPARK-37971: --- Description: This functionality would serve very specific use cases. Consider a DataFrame

[jira] [Updated] (SPARK-37971) Apply and evaluate expressiosn row-wise in a Spark DataFrame

[ https://issues.apache.org/jira/browse/SPARK-37971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carlos Gameiro updated SPARK-37971: --- Summary: Apply and evaluate expressiosn row-wise in a Spark DataFrame (was: Apply and

[jira] [Updated] (SPARK-37971) Apply and evaluate expressiosn row-wise in a DataFrame

[ https://issues.apache.org/jira/browse/SPARK-37971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carlos Gameiro updated SPARK-37971: --- Description: This functionality would serve very specific use cases. Consider a DataFrame

[jira] [Updated] (SPARK-37971) Apply and evaluate expressiosn row-wise in a DataFrame

[ https://issues.apache.org/jira/browse/SPARK-37971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carlos Gameiro updated SPARK-37971: --- Description: This functionality would serve very specific use cases. Consider a DataFrame

[jira] [Updated] (SPARK-37971) Apply and evaluate expressiosn row-wise in a DataFrame

[ https://issues.apache.org/jira/browse/SPARK-37971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carlos Gameiro updated SPARK-37971: --- Description: This functionality would serve very specific use cases. Consider a DataFrame

[jira] [Created] (SPARK-37971) Apply and evaluate expressiosn row-wise in a DataFrame

Carlos Gameiro created SPARK-37971: -- Summary: Apply and evaluate expressiosn row-wise in a DataFrame Key: SPARK-37971 URL: https://issues.apache.org/jira/browse/SPARK-37971 Project: Spark

[jira] [Commented] (SPARK-37533) New SQL function: try_element_at

[ https://issues.apache.org/jira/browse/SPARK-37533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479380#comment-17479380 ] Apache Spark commented on SPARK-37533: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-37533) New SQL function: try_element_at

[ https://issues.apache.org/jira/browse/SPARK-37533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479381#comment-17479381 ] Apache Spark commented on SPARK-37533: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-35028) ANSI mode: disallow group by aliases

[ https://issues.apache.org/jira/browse/SPARK-35028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479378#comment-17479378 ] Apache Spark commented on SPARK-35028: -- User 'gengliangwang' has created a pull request for this

[jira] [Resolved] (SPARK-37963) Need to update Partition URI after renaming table in InMemoryCatalog

[ https://issues.apache.org/jira/browse/SPARK-37963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-37963. Fix Version/s: 3.3.0 3.2.1 Resolution: Fixed Issue resolved by

[jira] [Commented] (SPARK-37970) Introduce a new interface on streaming data source to notify the latest seen offset

[ https://issues.apache.org/jira/browse/SPARK-37970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479335#comment-17479335 ] Apache Spark commented on SPARK-37970: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Assigned] (SPARK-37970) Introduce a new interface on streaming data source to notify the latest seen offset

[ https://issues.apache.org/jira/browse/SPARK-37970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37970: Assignee: Apache Spark > Introduce a new interface on streaming data source to notify

[jira] [Assigned] (SPARK-37970) Introduce a new interface on streaming data source to notify the latest seen offset

[ https://issues.apache.org/jira/browse/SPARK-37970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37970: Assignee: (was: Apache Spark) > Introduce a new interface on streaming data source

[jira] [Created] (SPARK-37970) Introduce a new interface on streaming data source to notify the latest seen offset

Jungtaek Lim created SPARK-37970: Summary: Introduce a new interface on streaming data source to notify the latest seen offset Key: SPARK-37970 URL: https://issues.apache.org/jira/browse/SPARK-37970

[jira] [Commented] (SPARK-37970) Introduce a new interface on streaming data source to notify the latest seen offset

[ https://issues.apache.org/jira/browse/SPARK-37970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479268#comment-17479268 ] Jungtaek Lim commented on SPARK-37970: -- Will submit a PR soon. > Introduce a new interface on

[jira] [Commented] (SPARK-37969) Hive Serde insert should check schema before execution

[ https://issues.apache.org/jira/browse/SPARK-37969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479242#comment-17479242 ] Apache Spark commented on SPARK-37969: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-37969) Hive Serde insert should check schema before execution

[ https://issues.apache.org/jira/browse/SPARK-37969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479241#comment-17479241 ] Apache Spark commented on SPARK-37969: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-37969) Hive Serde insert should check schema before execution

[ https://issues.apache.org/jira/browse/SPARK-37969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37969: Assignee: (was: Apache Spark) > Hive Serde insert should check schema before

[jira] [Assigned] (SPARK-37969) Hive Serde insert should check schema before execution

[ https://issues.apache.org/jira/browse/SPARK-37969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37969: Assignee: Apache Spark > Hive Serde insert should check schema before execution >

[jira] [Created] (SPARK-37969) Hive Serde insert should check schema before execution

angerszhu created SPARK-37969: - Summary: Hive Serde insert should check schema before execution Key: SPARK-37969 URL: https://issues.apache.org/jira/browse/SPARK-37969 Project: Spark Issue Type:

[jira] [Updated] (SPARK-36057) Support volcano/alternative schedulers

[ https://issues.apache.org/jira/browse/SPARK-36057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-36057: Shepherd: Holden Karau Affects Version/s: 3.3.0 (was:

[jira] [Resolved] (SPARK-37708) pyspark adding third-party Dependencies on k8s

[ https://issues.apache.org/jira/browse/SPARK-37708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jingxiong zhong resolved SPARK-37708. - Resolution: Not A Problem > pyspark adding third-party Dependencies on k8s >

[jira] [Resolved] (SPARK-28137) Data Type Formatting Functions: `to_number`

[ https://issues.apache.org/jira/browse/SPARK-28137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28137. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35060

[jira] [Assigned] (SPARK-28137) Data Type Formatting Functions: `to_number`

[ https://issues.apache.org/jira/browse/SPARK-28137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-28137: --- Assignee: jiaan.geng > Data Type Formatting Functions: `to_number` >

[jira] [Assigned] (SPARK-37922) Combine to one cast if we can safely up-cast two casts

[ https://issues.apache.org/jira/browse/SPARK-37922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37922: --- Assignee: Yuming Wang > Combine to one cast if we can safely up-cast two casts >

[jira] [Resolved] (SPARK-37922) Combine to one cast if we can safely up-cast two casts

[ https://issues.apache.org/jira/browse/SPARK-37922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37922. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35220

[jira] [Commented] (SPARK-37968) Upgrade commons-collections3 to commons-collections4

[ https://issues.apache.org/jira/browse/SPARK-37968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479198#comment-17479198 ] Apache Spark commented on SPARK-37968: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-37968) Upgrade commons-collections3 to commons-collections4

[ https://issues.apache.org/jira/browse/SPARK-37968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37968: Assignee: Apache Spark > Upgrade commons-collections3 to commons-collections4 >

[jira] [Assigned] (SPARK-37968) Upgrade commons-collections3 to commons-collections4

[ https://issues.apache.org/jira/browse/SPARK-37968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37968: Assignee: (was: Apache Spark) > Upgrade commons-collections3 to commons-collections4

[jira] [Commented] (SPARK-37968) Upgrade commons-collections3 to commons-collections4

[ https://issues.apache.org/jira/browse/SPARK-37968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479197#comment-17479197 ] Apache Spark commented on SPARK-37968: -- User 'LuciferYang' has created a pull request for this

[jira] [Created] (SPARK-37968) Upgrade commons-collections3 to commons-collections4

Yang Jie created SPARK-37968: Summary: Upgrade commons-collections3 to commons-collections4 Key: SPARK-37968 URL: https://issues.apache.org/jira/browse/SPARK-37968 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-37931) Quote the column name if needed

[ https://issues.apache.org/jira/browse/SPARK-37931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37931: --- Assignee: PengLei > Quote the column name if needed > --- > >

[jira] [Resolved] (SPARK-37931) Quote the column name if needed

[ https://issues.apache.org/jira/browse/SPARK-37931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37931. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35227

[jira] [Commented] (SPARK-37933) Limit push down for parquet datasource v2

[ https://issues.apache.org/jira/browse/SPARK-37933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479173#comment-17479173 ] Apache Spark commented on SPARK-37933: -- User 'stczwd' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37933) Limit push down for parquet datasource v2

[ https://issues.apache.org/jira/browse/SPARK-37933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37933: Assignee: Jackey Lee (was: Apache Spark) > Limit push down for parquet datasource v2 >

[jira] [Assigned] (SPARK-37933) Limit push down for parquet datasource v2

[ https://issues.apache.org/jira/browse/SPARK-37933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37933: Assignee: Apache Spark (was: Jackey Lee) > Limit push down for parquet datasource v2 >

[jira] [Updated] (SPARK-37960) Support aggregate push down with CASE ... WHEN ... ELSE ... END

[ https://issues.apache.org/jira/browse/SPARK-37960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-37960: --- Summary: Support aggregate push down with CASE ... WHEN ... ELSE ... END (was: Support aggregate

[jira] [Commented] (SPARK-37831) Add task partition id in metrics

[ https://issues.apache.org/jira/browse/SPARK-37831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479162#comment-17479162 ] Apache Spark commented on SPARK-37831: -- User 'stczwd' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37967) ConstantFolding/ Literal.create support ObjectType

[ https://issues.apache.org/jira/browse/SPARK-37967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37967: Assignee: (was: Apache Spark) > ConstantFolding/ Literal.create support ObjectType >

[jira] [Commented] (SPARK-37967) ConstantFolding/ Literal.create support ObjectType

[ https://issues.apache.org/jira/browse/SPARK-37967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479158#comment-17479158 ] Apache Spark commented on SPARK-37967: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-37967) ConstantFolding/ Literal.create support ObjectType

[ https://issues.apache.org/jira/browse/SPARK-37967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37967: Assignee: Apache Spark > ConstantFolding/ Literal.create support ObjectType >