[jira] [Resolved] (SPARK-33573) Server side metrics related to push-based shuffle

2023-01-11 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-33573. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-33573) Server side metrics related to push-based shuffle

2023-01-11 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-33573: --- Assignee: Minchu Yang > Server side metrics related to push-based shuffle

[jira] [Resolved] (SPARK-41998) Reeuse test_readwriter test cases

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41998. -- Assignee: Hyukjin Kwon Resolution: Fixed Fixed in

[jira] [Assigned] (SPARK-41968) Refactor ProtobufSerDe to ProtobufSerDe[T]

2023-01-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-41968: -- Assignee: Yang Jie > Refactor ProtobufSerDe to ProtobufSerDe[T] >

[jira] [Resolved] (SPARK-41968) Refactor ProtobufSerDe to ProtobufSerDe[T]

2023-01-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-41968. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39487

[jira] [Commented] (SPARK-42019) Reuse pyspark.sql.tests.test_types test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675823#comment-17675823 ] Apache Spark commented on SPARK-42019: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-42019) Reuse pyspark.sql.tests.test_types test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42019: Assignee: Apache Spark > Reuse pyspark.sql.tests.test_types test cases >

[jira] [Assigned] (SPARK-42019) Reuse pyspark.sql.tests.test_types test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42019: Assignee: (was: Apache Spark) > Reuse pyspark.sql.tests.test_types test cases >

[jira] [Commented] (SPARK-42019) Reuse pyspark.sql.tests.test_types test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675822#comment-17675822 ] Apache Spark commented on SPARK-42019: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Created] (SPARK-42024) createDataFrame should corse types of string float to float

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42024: Summary: createDataFrame should corse types of string float to float Key: SPARK-42024 URL: https://issues.apache.org/jira/browse/SPARK-42024 Project: Spark

[jira] [Created] (SPARK-42023) createDataFrame should corse types of string false to bool false

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42023: Summary: createDataFrame should corse types of string false to bool false Key: SPARK-42023 URL: https://issues.apache.org/jira/browse/SPARK-42023 Project: Spark

[jira] [Created] (SPARK-42022) createDataFrame should autogenerate missing column names

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42022: Summary: createDataFrame should autogenerate missing column names Key: SPARK-42022 URL: https://issues.apache.org/jira/browse/SPARK-42022 Project: Spark

[jira] [Created] (SPARK-42021) createDataFrame with array.array

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42021: Summary: createDataFrame with array.array Key: SPARK-42021 URL: https://issues.apache.org/jira/browse/SPARK-42021 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-42020) createDataFrame with UDT

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42020: Summary: createDataFrame with UDT Key: SPARK-42020 URL: https://issues.apache.org/jira/browse/SPARK-42020 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-42019) Reuse pyspark.sql.tests.test_group test cases

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42019: Summary: Reuse pyspark.sql.tests.test_group test cases Key: SPARK-42019 URL: https://issues.apache.org/jira/browse/SPARK-42019 Project: Spark Issue Type:

[jira] [Commented] (SPARK-42010) Reuse pyspark.sql.tests.test_column test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675818#comment-17675818 ] Apache Spark commented on SPARK-42010: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-42019) Reuse pyspark.sql.tests.test_types test cases

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42019: - Summary: Reuse pyspark.sql.tests.test_types test cases (was: Reuse

[jira] [Updated] (SPARK-42006) Test parity: pyspark.sql.tests.test_group, test_serde, test_datasources and test_column

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42006: - Description: See https://issues.apache.org/jira/browse/SPARK-41652 and

[jira] [Assigned] (SPARK-42010) Reuse pyspark.sql.tests.test_column test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42010: Assignee: (was: Apache Spark) > Reuse pyspark.sql.tests.test_column test cases >

[jira] [Commented] (SPARK-42010) Reuse pyspark.sql.tests.test_column test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675817#comment-17675817 ] Apache Spark commented on SPARK-42010: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-42018) Test parity: pyspark.sql.tests.test_types

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42018: - Description: See https://issues.apache.org/jira/browse/SPARK-41652 and

[jira] [Assigned] (SPARK-42010) Reuse pyspark.sql.tests.test_column test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42010: Assignee: Apache Spark > Reuse pyspark.sql.tests.test_column test cases >

[jira] [Updated] (SPARK-42018) Test parity: pyspark.sql.tests.test_types

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42018: - Epic Link: SPARK-39375 > Test parity: pyspark.sql.tests.test_types >

[jira] [Created] (SPARK-42018) Test parity: pyspark.sql.tests.test_types

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42018: Summary: Test parity: pyspark.sql.tests.test_types Key: SPARK-42018 URL: https://issues.apache.org/jira/browse/SPARK-42018 Project: Spark Issue Type:

[jira] [Created] (SPARK-42017) Different error type AnalysisException vs SparkConnectAnalysisException

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42017: Summary: Different error type AnalysisException vs SparkConnectAnalysisException Key: SPARK-42017 URL: https://issues.apache.org/jira/browse/SPARK-42017 Project:

[jira] [Updated] (SPARK-42016) Type inconsistency of struct and map when accessing the nested column

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42016: - Parent: (was: SPARK-41281) Issue Type: Bug (was: Sub-task) > Type inconsistency of

[jira] [Updated] (SPARK-42016) Type inconsistency of struct and map when accessing the nested column

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42016: - Epic Link: (was: SPARK-39375) > Type inconsistency of struct and map when accessing the

[jira] [Updated] (SPARK-42016) Type inconsistency of struct and map when accessing the nested column

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42016: - Epic Link: SPARK-39375 > Type inconsistency of struct and map when accessing the nested column

[jira] [Updated] (SPARK-42016) Type inconsistency of struct and map when accessing the nested column

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42016: - Parent: SPARK-41282 Issue Type: Sub-task (was: Bug) > Type inconsistency of struct and

[jira] [Created] (SPARK-42016) Type inconsistency of struct and map when accessing the nested column

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42016: Summary: Type inconsistency of struct and map when accessing the nested column Key: SPARK-42016 URL: https://issues.apache.org/jira/browse/SPARK-42016 Project: Spark

[jira] [Assigned] (SPARK-42009) Reuse pyspark.sql.tests.test_serde test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42009: Assignee: Apache Spark > Reuse pyspark.sql.tests.test_serde test cases >

[jira] [Assigned] (SPARK-42009) Reuse pyspark.sql.tests.test_serde test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42009: Assignee: (was: Apache Spark) > Reuse pyspark.sql.tests.test_serde test cases >

[jira] [Commented] (SPARK-42009) Reuse pyspark.sql.tests.test_serde test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675814#comment-17675814 ] Apache Spark commented on SPARK-42009: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-42006) Test parity: pyspark.sql.tests.test_group, test_serde, test_datasources and test_column

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42006: - Summary: Test parity: pyspark.sql.tests.test_group, test_serde, test_datasources and

[jira] [Updated] (SPARK-42010) Reuse pyspark.sql.tests.test_column test cases

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42010: - Summary: Reuse pyspark.sql.tests.test_column test cases (was: Reuse

[jira] [Deleted] (SPARK-42015) Support struct as a key in map

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon deleted SPARK-42015: - > Support struct as a key in map > -- > > Key:

[jira] [Created] (SPARK-42015) Support struct as a key in map

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42015: Summary: Support struct as a key in map Key: SPARK-42015 URL: https://issues.apache.org/jira/browse/SPARK-42015 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-42014) Support aware datetimes

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42014: Summary: Support aware datetimes Key: SPARK-42014 URL: https://issues.apache.org/jira/browse/SPARK-42014 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-42008) Reuse pyspark.sql.tests.test_datasources test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675812#comment-17675812 ] Apache Spark commented on SPARK-42008: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-42008) Reuse pyspark.sql.tests.test_datasources test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675811#comment-17675811 ] Apache Spark commented on SPARK-42008: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-42008) Reuse pyspark.sql.tests.test_datasources test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42008: Assignee: (was: Apache Spark) > Reuse pyspark.sql.tests.test_datasources test cases

[jira] [Assigned] (SPARK-42008) Reuse pyspark.sql.tests.test_datasources test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42008: Assignee: Apache Spark > Reuse pyspark.sql.tests.test_datasources test cases >

[jira] [Created] (SPARK-42012) Implement DataFrameReader.orc

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42012: Summary: Implement DataFrameReader.orc Key: SPARK-42012 URL: https://issues.apache.org/jira/browse/SPARK-42012 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-42013) Implement DataFrameReader.text to take multiple paths

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42013: Summary: Implement DataFrameReader.text to take multiple paths Key: SPARK-42013 URL: https://issues.apache.org/jira/browse/SPARK-42013 Project: Spark Issue

[jira] [Created] (SPARK-42011) Implement DataFrameReader.csv

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42011: Summary: Implement DataFrameReader.csv Key: SPARK-42011 URL: https://issues.apache.org/jira/browse/SPARK-42011 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-42007) Reuse pyspark.sql.tests.test_group test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42007: Assignee: (was: Apache Spark) > Reuse pyspark.sql.tests.test_group test cases >

[jira] [Assigned] (SPARK-42007) Reuse pyspark.sql.tests.test_group test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42007: Assignee: Apache Spark > Reuse pyspark.sql.tests.test_group test cases >

[jira] [Commented] (SPARK-42007) Reuse pyspark.sql.tests.test_group test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675809#comment-17675809 ] Apache Spark commented on SPARK-42007: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-42008) Reuse pyspark.sql.tests.test_datasources test cases

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42008: - Summary: Reuse pyspark.sql.tests.test_datasources test cases (was: Reuse

[jira] [Updated] (SPARK-42008) Reuse pyspark.sql.tests.test_types test cases

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42008: - Summary: Reuse pyspark.sql.tests.test_types test cases (was: Reeuse

[jira] [Updated] (SPARK-42009) Reuse pyspark.sql.tests.test_serde test cases

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42009: - Summary: Reuse pyspark.sql.tests.test_serde test cases (was: Reeuse

[jira] [Updated] (SPARK-42007) Reeuse pyspark.sql.tests.test_group test cases

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42007: - Summary: Reeuse pyspark.sql.tests.test_group test cases (was: Reeuse

[jira] [Updated] (SPARK-42007) Reuse pyspark.sql.tests.test_group test cases

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42007: - Summary: Reuse pyspark.sql.tests.test_group test cases (was: Reeuse

[jira] [Created] (SPARK-42010) Reuse pyspark.sql.tests.test_types test cases

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42010: Summary: Reuse pyspark.sql.tests.test_types test cases Key: SPARK-42010 URL: https://issues.apache.org/jira/browse/SPARK-42010 Project: Spark Issue Type:

[jira] [Created] (SPARK-42008) Reeuse pyspark.sql.tests.test_types test cases

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42008: Summary: Reeuse pyspark.sql.tests.test_types test cases Key: SPARK-42008 URL: https://issues.apache.org/jira/browse/SPARK-42008 Project: Spark Issue Type:

[jira] [Created] (SPARK-42009) Reeuse pyspark.sql.tests.test_serde test cases

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42009: Summary: Reeuse pyspark.sql.tests.test_serde test cases Key: SPARK-42009 URL: https://issues.apache.org/jira/browse/SPARK-42009 Project: Spark Issue Type:

[jira] [Created] (SPARK-42007) Reeuse pyspark.sql.tests.test_readwriter test cases

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42007: Summary: Reeuse pyspark.sql.tests.test_readwriter test cases Key: SPARK-42007 URL: https://issues.apache.org/jira/browse/SPARK-42007 Project: Spark Issue

[jira] [Created] (SPARK-42006) Test parity: pyspark.sql.tests.test_group, test_serde, test_datasources and test_types

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42006: Summary: Test parity: pyspark.sql.tests.test_group, test_serde, test_datasources and test_types Key: SPARK-42006 URL: https://issues.apache.org/jira/browse/SPARK-42006

[jira] [Updated] (SPARK-42006) Test parity: pyspark.sql.tests.test_group, test_serde, test_datasources and test_types

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42006: - Epic Link: SPARK-39375 > Test parity: pyspark.sql.tests.test_group, test_serde,

[jira] [Updated] (SPARK-42005) SparkR cannot collect dataframe with NA in a date column along with another timestamp column

2023-01-11 Thread Vivek Atal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vivek Atal updated SPARK-42005: --- Description: This issue seems to be related with

[jira] [Updated] (SPARK-42005) SparkR cannot collect dataframe with NA in a date column along with another timestamp column

2023-01-11 Thread Vivek Atal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vivek Atal updated SPARK-42005: --- Description: This issue seems to be related with

[jira] [Created] (SPARK-42005) SparkR cannot collect dataframe with NA in a date column along with another timestamp column

2023-01-11 Thread Vivek Atal (Jira)
Vivek Atal created SPARK-42005: -- Summary: SparkR cannot collect dataframe with NA in a date column along with another timestamp column Key: SPARK-42005 URL: https://issues.apache.org/jira/browse/SPARK-42005

[jira] [Resolved] (SPARK-41996) KafkaMicroBatchV2SourceSuite failed for topic partitions unavailable test due to kafka operations taking longer

2023-01-11 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-41996. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39520

[jira] [Assigned] (SPARK-41996) KafkaMicroBatchV2SourceSuite failed for topic partitions unavailable test due to kafka operations taking longer

2023-01-11 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-41996: Assignee: Anish Shrigondekar > KafkaMicroBatchV2SourceSuite failed for topic partitions

[jira] [Created] (SPARK-42004) Migrate "XX000" sqlState onto `INTERNAL_ERROR`

2023-01-11 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-42004: --- Summary: Migrate "XX000" sqlState onto `INTERNAL_ERROR` Key: SPARK-42004 URL: https://issues.apache.org/jira/browse/SPARK-42004 Project: Spark Issue Type:

[jira] [Commented] (SPARK-41990) Filtering by composite field name like `field name` doesn't work with pushDownPredicate = true

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675795#comment-17675795 ] Apache Spark commented on SPARK-41990: -- User 'panbingkun' has created a pull request for this

[jira] [Assigned] (SPARK-41990) Filtering by composite field name like `field name` doesn't work with pushDownPredicate = true

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41990: Assignee: Apache Spark > Filtering by composite field name like `field name` doesn't

[jira] [Assigned] (SPARK-41990) Filtering by composite field name like `field name` doesn't work with pushDownPredicate = true

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41990: Assignee: (was: Apache Spark) > Filtering by composite field name like `field name`

[jira] [Commented] (SPARK-41990) Filtering by composite field name like `field name` doesn't work with pushDownPredicate = true

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675793#comment-17675793 ] Apache Spark commented on SPARK-41990: -- User 'panbingkun' has created a pull request for this

[jira] [Assigned] (SPARK-42003) Reduce duplicate code in ResolveGroupByAll

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42003: Assignee: Gengliang Wang (was: Apache Spark) > Reduce duplicate code in

[jira] [Assigned] (SPARK-42003) Reduce duplicate code in ResolveGroupByAll

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42003: Assignee: Apache Spark (was: Gengliang Wang) > Reduce duplicate code in

[jira] [Commented] (SPARK-42003) Reduce duplicate code in ResolveGroupByAll

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675792#comment-17675792 ] Apache Spark commented on SPARK-42003: -- User 'gengliangwang' has created a pull request for this

[jira] [Created] (SPARK-42003) Reduce duplicate code in ResolveGroupByAll

2023-01-11 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-42003: -- Summary: Reduce duplicate code in ResolveGroupByAll Key: SPARK-42003 URL: https://issues.apache.org/jira/browse/SPARK-42003 Project: Spark Issue Type:

[jira] [Commented] (SPARK-41998) Reeuse test_readwriter test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675788#comment-17675788 ] Apache Spark commented on SPARK-41998: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-41998) Reeuse test_readwriter test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41998: Assignee: Apache Spark > Reeuse test_readwriter test cases >

[jira] [Commented] (SPARK-41998) Reeuse test_readwriter test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675787#comment-17675787 ] Apache Spark commented on SPARK-41998: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-41998) Reeuse test_readwriter test cases

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41998: Assignee: (was: Apache Spark) > Reeuse test_readwriter test cases >

[jira] [Created] (SPARK-42002) Implement DataFrameWriterV2 (ReadwriterV2Tests)

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42002: Summary: Implement DataFrameWriterV2 (ReadwriterV2Tests) Key: SPARK-42002 URL: https://issues.apache.org/jira/browse/SPARK-42002 Project: Spark Issue Type:

[jira] [Created] (SPARK-42001) Unexpected schema set to DefaultSource plan (ReadwriterTests.test_save_and_load)

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42001: Summary: Unexpected schema set to DefaultSource plan (ReadwriterTests.test_save_and_load) Key: SPARK-42001 URL: https://issues.apache.org/jira/browse/SPARK-42001

[jira] [Updated] (SPARK-42000) saveAsTable fail to find the default source (ReadwriterTests.test_insert_into)

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42000: - Summary: saveAsTable fail to find the default source (ReadwriterTests.test_insert_into) (was:

[jira] [Updated] (SPARK-41999) NPE for bucketed write (ReadwriterTests.test_bucketed_write)

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41999: - Summary: NPE for bucketed write (ReadwriterTests.test_bucketed_write) (was: NPE for bucketed

[jira] [Created] (SPARK-42000) saveAsTable fail to find the default source

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42000: Summary: saveAsTable fail to find the default source Key: SPARK-42000 URL: https://issues.apache.org/jira/browse/SPARK-42000 Project: Spark Issue Type:

[jira] [Created] (SPARK-41999) NPE for bucketed write

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41999: Summary: NPE for bucketed write Key: SPARK-41999 URL: https://issues.apache.org/jira/browse/SPARK-41999 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-41997) Test parity: pyspark.sql.tests.test_readwriter

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41997: - Description: See https://issues.apache.org/jira/browse/SPARK-41652 and

[jira] [Created] (SPARK-41998) Reeuse test_readwriter test cases

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41998: Summary: Reeuse test_readwriter test cases Key: SPARK-41998 URL: https://issues.apache.org/jira/browse/SPARK-41998 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-41997) Test parity: pyspark.sql.tests.test_readwriter

2023-01-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-41997: Summary: Test parity: pyspark.sql.tests.test_readwriter Key: SPARK-41997 URL: https://issues.apache.org/jira/browse/SPARK-41997 Project: Spark Issue Type:

[jira] [Commented] (SPARK-41887) Support DataFrame hint parameter to be list

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675783#comment-17675783 ] Apache Spark commented on SPARK-41887: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-41996) KafkaMicroBatchV2SourceSuite failed for topic partitions unavailable test due to kafka operations taking longer

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675778#comment-17675778 ] Apache Spark commented on SPARK-41996: -- User 'anishshri-db' has created a pull request for this

[jira] [Assigned] (SPARK-41996) KafkaMicroBatchV2SourceSuite failed for topic partitions unavailable test due to kafka operations taking longer

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41996: Assignee: (was: Apache Spark) > KafkaMicroBatchV2SourceSuite failed for topic

[jira] [Assigned] (SPARK-41996) KafkaMicroBatchV2SourceSuite failed for topic partitions unavailable test due to kafka operations taking longer

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41996: Assignee: Apache Spark > KafkaMicroBatchV2SourceSuite failed for topic partitions

[jira] [Commented] (SPARK-41996) KafkaMicroBatchV2SourceSuite failed for topic partitions unavailable test due to kafka operations taking longer

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675777#comment-17675777 ] Apache Spark commented on SPARK-41996: -- User 'anishshri-db' has created a pull request for this

[jira] [Resolved] (SPARK-41980) Enable test_functions_broadcast

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41980. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39500

[jira] [Assigned] (SPARK-41980) Enable test_functions_broadcast

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41980: Assignee: Hyukjin Kwon > Enable test_functions_broadcast >

[jira] [Commented] (SPARK-41996) KafkaMicroBatchV2SourceSuite failed for topic partitions unavailable test due to kafka operations taking longer

2023-01-11 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675764#comment-17675764 ] Anish Shrigondekar commented on SPARK-41996: Have the fix and will send the PR out soon  

[jira] [Created] (SPARK-41996) KafkaMicroBatchV2SourceSuite failed for topic partitions unavailable test due to kafka operations taking longer

2023-01-11 Thread Anish Shrigondekar (Jira)
Anish Shrigondekar created SPARK-41996: -- Summary: KafkaMicroBatchV2SourceSuite failed for topic partitions unavailable test due to kafka operations taking longer Key: SPARK-41996 URL:

[jira] [Resolved] (SPARK-41591) Implement functionality for training a PyTorch file locally

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41591. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39188

[jira] [Assigned] (SPARK-41591) Implement functionality for training a PyTorch file locally

2023-01-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41591: Assignee: Rithwik Ediga Lakhamsani > Implement functionality for training a PyTorch file

[jira] [Commented] (SPARK-41995) schema_of_json only accepts foldable expressions

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675745#comment-17675745 ] Apache Spark commented on SPARK-41995: -- User 'eric-maynard' has created a pull request for this

[jira] [Commented] (SPARK-41995) schema_of_json only accepts foldable expressions

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675744#comment-17675744 ] Apache Spark commented on SPARK-41995: -- User 'eric-maynard' has created a pull request for this

[jira] [Assigned] (SPARK-41995) schema_of_json only accepts foldable expressions

2023-01-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41995: Assignee: (was: Apache Spark) > schema_of_json only accepts foldable expressions >

  1   2   3   >