[jira] [Created] (SPARK-42049) Improve AliasAwareOutputExpression

2023-01-12 Thread XiDuo You (Jira)
XiDuo You created SPARK-42049: - Summary: Improve AliasAwareOutputExpression Key: SPARK-42049 URL: https://issues.apache.org/jira/browse/SPARK-42049 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-42001) Unexpected schema set to DefaultSource plan (ReadwriterTests.test_save_and_load)

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42001: Assignee: (was: Apache Spark) > Unexpected schema set to DefaultSource plan >

[jira] [Assigned] (SPARK-42001) Unexpected schema set to DefaultSource plan (ReadwriterTests.test_save_and_load)

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42001: Assignee: Apache Spark > Unexpected schema set to DefaultSource plan >

[jira] [Commented] (SPARK-42001) Unexpected schema set to DefaultSource plan (ReadwriterTests.test_save_and_load)

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676501#comment-17676501 ] Apache Spark commented on SPARK-42001: -- User 'zhengruifeng' has created a pull request for this

[jira] [Updated] (SPARK-40819) Parquet INT64 (TIMESTAMP(NANOS,true)) now throwing Illegal Parquet type instead of automatically converting to LongType

2023-01-12 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-40819: -- Labels: regression (was: correctness) > Parquet INT64 (TIMESTAMP(NANOS,true)) now throwing

[jira] [Resolved] (SPARK-41986) Introduce shuffle on SinglePartition

2023-01-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41986. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39512

[jira] [Assigned] (SPARK-41986) Introduce shuffle on SinglePartition

2023-01-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41986: - Assignee: Yuming Wang > Introduce shuffle on SinglePartition >

[jira] [Updated] (SPARK-40819) Parquet INT64 (TIMESTAMP(NANOS,true)) now throwing Illegal Parquet type instead of automatically converting to LongType

2023-01-12 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-40819: -- Labels: correctness (was: ) > Parquet INT64 (TIMESTAMP(NANOS,true)) now throwing Illegal

[jira] [Updated] (SPARK-41914) Sorting issue with partitioned-writing and planned write optimization disabled

2023-01-12 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-41914: -- Labels: correctness (was: ) > Sorting issue with partitioned-writing and planned write

[jira] [Updated] (SPARK-26345) Parquet support Column indexes

2023-01-12 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-26345: -- Labels: (was: correctness) > Parquet support Column indexes >

[jira] [Updated] (SPARK-26345) Parquet support Column indexes

2023-01-12 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-26345: -- Labels: correctness (was: ) > Parquet support Column indexes >

[jira] [Updated] (SPARK-41959) Improve v1 writes with empty2null

2023-01-12 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-41959: -- Labels: correctness (was: ) > Improve v1 writes with empty2null >

[jira] [Assigned] (SPARK-42016) Type inconsistency of struct and map when accessing the nested column

2023-01-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-42016: - Assignee: Ruifeng Zheng > Type inconsistency of struct and map when accessing the

[jira] [Resolved] (SPARK-42016) Type inconsistency of struct and map when accessing the nested column

2023-01-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-42016. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39547

[jira] [Resolved] (SPARK-42042) DataFrameReader should support StructType schema

2023-01-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-42042. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39545

[jira] [Assigned] (SPARK-42042) DataFrameReader should support StructType schema

2023-01-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-42042: - Assignee: Ruifeng Zheng > DataFrameReader should support StructType schema >

[jira] [Commented] (SPARK-41900) Support data type int8

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676494#comment-17676494 ] Apache Spark commented on SPARK-41900: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-41900) Support data type int8

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41900: Assignee: (was: Apache Spark) > Support data type int8 > -- > >

[jira] [Assigned] (SPARK-41900) Support data type int8

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41900: Assignee: Apache Spark > Support data type int8 > -- > >

[jira] [Commented] (SPARK-41900) Support data type int8

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676491#comment-17676491 ] Apache Spark commented on SPARK-41900: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-41900) Support data type int8

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676493#comment-17676493 ] Apache Spark commented on SPARK-41900: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-42047) Literal should support numpy datatypes

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676488#comment-17676488 ] Apache Spark commented on SPARK-42047: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-42047) Literal should support numpy datatypes

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676489#comment-17676489 ] Apache Spark commented on SPARK-42047: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-42047) Literal should support numpy datatypes

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42047: Assignee: (was: Apache Spark) > Literal should support numpy datatypes >

[jira] [Assigned] (SPARK-42047) Literal should support numpy datatypes

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42047: Assignee: Apache Spark > Literal should support numpy datatypes >

[jira] [Assigned] (SPARK-42037) Rename AMPLAB_ to SPARK_ prefix in build environment variables

2023-01-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-42037: - Assignee: Dongjoon Hyun > Rename AMPLAB_ to SPARK_ prefix in build environment

[jira] [Resolved] (SPARK-42037) Rename AMPLAB_ to SPARK_ prefix in build environment variables

2023-01-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-42037. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39539

[jira] [Created] (SPARK-42048) Different column name of lit(np.int8)

2023-01-12 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-42048: - Summary: Different column name of lit(np.int8) Key: SPARK-42048 URL: https://issues.apache.org/jira/browse/SPARK-42048 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-42047) Literal should support numpy datatypes

2023-01-12 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-42047: - Summary: Literal should support numpy datatypes Key: SPARK-42047 URL: https://issues.apache.org/jira/browse/SPARK-42047 Project: Spark Issue Type:

[jira] [Updated] (SPARK-42046) Add `connect-client-jvm` to connect module

2023-01-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42046: -- Issue Type: Bug (was: Improvement) > Add `connect-client-jvm` to connect module >

[jira] [Assigned] (SPARK-42046) Add `connect-client-jvm` to connect module

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42046: Assignee: (was: Apache Spark) > Add `connect-client-jvm` to connect module >

[jira] [Assigned] (SPARK-42046) Add `connect-client-jvm` to connect module

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42046: Assignee: Apache Spark > Add `connect-client-jvm` to connect module >

[jira] [Commented] (SPARK-42046) Add `connect-client-jvm` to connect module

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676473#comment-17676473 ] Apache Spark commented on SPARK-42046: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-42046) Add `connect-client-jvm` to connect module

2023-01-12 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-42046: - Summary: Add `connect-client-jvm` to connect module Key: SPARK-42046 URL: https://issues.apache.org/jira/browse/SPARK-42046 Project: Spark Issue Type:

[jira] [Updated] (SPARK-42037) Rename AMPLAB_ to SPARK_ prefix in build environment variables

2023-01-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42037: -- Summary: Rename AMPLAB_ to SPARK_ prefix in build environment variables (was: Remove

[jira] [Assigned] (SPARK-42014) Support aware datetimes

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42014: Assignee: (was: Apache Spark) > Support aware datetimes > --- >

[jira] [Assigned] (SPARK-42014) Support aware datetimes

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42014: Assignee: Apache Spark > Support aware datetimes > --- > >

[jira] [Commented] (SPARK-42014) Support aware datetimes

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676456#comment-17676456 ] Apache Spark commented on SPARK-42014: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-42016) Type inconsistency of struct and map when accessing the nested column

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676451#comment-17676451 ] Apache Spark commented on SPARK-42016: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-42016) Type inconsistency of struct and map when accessing the nested column

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676450#comment-17676450 ] Apache Spark commented on SPARK-42016: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-42016) Type inconsistency of struct and map when accessing the nested column

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42016: Assignee: Apache Spark > Type inconsistency of struct and map when accessing the nested

[jira] [Assigned] (SPARK-42016) Type inconsistency of struct and map when accessing the nested column

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42016: Assignee: (was: Apache Spark) > Type inconsistency of struct and map when accessing

[jira] [Resolved] (SPARK-41494) Make json functions support options

2023-01-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41494. --- Resolution: Resolved > Make json functions support options >

[jira] [Assigned] (SPARK-41847) DataFrame mapfield,structlist invalid type

2023-01-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41847: - Assignee: Ruifeng Zheng > DataFrame mapfield,structlist invalid type >

[jira] [Assigned] (SPARK-41837) DataFrame.createDataFrame datatype conversion error

2023-01-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41837: - Assignee: Ruifeng Zheng > DataFrame.createDataFrame datatype conversion error >

[jira] [Assigned] (SPARK-41838) DataFrame.show() fix map printing

2023-01-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41838: - Assignee: Ruifeng Zheng > DataFrame.show() fix map printing >

[jira] [Assigned] (SPARK-41746) SparkSession.createDataFrame does not support nested datatypes

2023-01-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41746: - Assignee: Ruifeng Zheng > SparkSession.createDataFrame does not support nested

[jira] [Resolved] (SPARK-41746) SparkSession.createDataFrame does not support nested datatypes

2023-01-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41746. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39535

[jira] [Resolved] (SPARK-41835) Implement `transform_keys` function

2023-01-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41835. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39535

[jira] [Assigned] (SPARK-41835) Implement `transform_keys` function

2023-01-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41835: - Assignee: Ruifeng Zheng > Implement `transform_keys` function >

[jira] [Resolved] (SPARK-41847) DataFrame mapfield,structlist invalid type

2023-01-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41847. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39535

[jira] [Resolved] (SPARK-41837) DataFrame.createDataFrame datatype conversion error

2023-01-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41837. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39535

[jira] [Resolved] (SPARK-41836) Implement `transform_values` function

2023-01-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41836. --- Resolution: Fixed Issue resolved by pull request 39535

[jira] [Resolved] (SPARK-41838) DataFrame.show() fix map printing

2023-01-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41838. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39535

[jira] [Assigned] (SPARK-42045) Round/Bround should return an error on integral overflow

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42045: Assignee: Apache Spark (was: Gengliang Wang) > Round/Bround should return an error on

[jira] [Assigned] (SPARK-42045) Round/Bround should return an error on integral overflow

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42045: Assignee: Gengliang Wang (was: Apache Spark) > Round/Bround should return an error on

[jira] [Commented] (SPARK-42045) Round/Bround should return an error on integral overflow

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676437#comment-17676437 ] Apache Spark commented on SPARK-42045: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-42045) Round/Bround should return an error on integral overflow

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676436#comment-17676436 ] Apache Spark commented on SPARK-42045: -- User 'gengliangwang' has created a pull request for this

[jira] [Created] (SPARK-42045) Round/Bround should return an error on integral overflow

2023-01-12 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-42045: -- Summary: Round/Bround should return an error on integral overflow Key: SPARK-42045 URL: https://issues.apache.org/jira/browse/SPARK-42045 Project: Spark

[jira] [Commented] (SPARK-40143) ANSI mode: allow explicitly casting fraction strings as Integral types

2023-01-12 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676435#comment-17676435 ] Gengliang Wang commented on SPARK-40143: It's hard to define which cast(string as int) syntax

[jira] [Resolved] (SPARK-40143) ANSI mode: allow explicitly casting fraction strings as Integral types

2023-01-12 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-40143. Resolution: Won't Do > ANSI mode: allow explicitly casting fraction strings as Integral

[jira] [Assigned] (SPARK-42042) DataFrameReader should support StructType schema

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42042: Assignee: (was: Apache Spark) > DataFrameReader should support StructType schema >

[jira] [Assigned] (SPARK-42042) DataFrameReader should support StructType schema

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42042: Assignee: Apache Spark > DataFrameReader should support StructType schema >

[jira] [Commented] (SPARK-42042) DataFrameReader should support StructType schema

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676430#comment-17676430 ] Apache Spark commented on SPARK-42042: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-42028) Support Pandas DF to Spark DF with Nanosecond Timestamps

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676421#comment-17676421 ] Apache Spark commented on SPARK-42028: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-42028) Support Pandas DF to Spark DF with Nanosecond Timestamps

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676420#comment-17676420 ] Apache Spark commented on SPARK-42028: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-42027) CreateDataframe from Pandas with Struct and Timestamp

2023-01-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42027: - Component/s: PySpark (was: Connect) > CreateDataframe from Pandas with

[jira] [Commented] (SPARK-42027) CreateDataframe from Pandas with Struct and Timestamp

2023-01-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676418#comment-17676418 ] Hyukjin Kwon commented on SPARK-42027: -- Converted to a general issue in PySpark. > CreateDataframe

[jira] [Updated] (SPARK-42027) CreateDataframe from Pandas with Struct and Timestamp

2023-01-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42027: - Epic Link: (was: SPARK-39375) > CreateDataframe from Pandas with Struct and Timestamp >

[jira] [Updated] (SPARK-42027) CreateDataframe from Pandas with Struct and Timestamp

2023-01-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42027: - Parent: (was: SPARK-41279) Issue Type: Bug (was: Sub-task) > CreateDataframe from

[jira] [Updated] (SPARK-42027) CreateDataframe from Pandas with Struct and Timestamp

2023-01-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42027: - Epic Link: SPARK-39375 > CreateDataframe from Pandas with Struct and Timestamp >

[jira] [Commented] (SPARK-42044) Fix wrong error message for `MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY`

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676410#comment-17676410 ] Apache Spark commented on SPARK-42044: -- User 'itholic' has created a pull request for this issue:

[jira] [Commented] (SPARK-42044) Fix wrong error message for `MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY`

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676409#comment-17676409 ] Apache Spark commented on SPARK-42044: -- User 'itholic' has created a pull request for this issue:

[jira] [Commented] (SPARK-41591) Implement functionality for training a PyTorch file locally

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676408#comment-17676408 ] Apache Spark commented on SPARK-41591: -- User 'harupy' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42044) Fix wrong error message for `MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY`

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42044: Assignee: Apache Spark > Fix wrong error message for

[jira] [Assigned] (SPARK-42044) Fix wrong error message for `MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY`

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42044: Assignee: (was: Apache Spark) > Fix wrong error message for

[jira] [Commented] (SPARK-41591) Implement functionality for training a PyTorch file locally

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676407#comment-17676407 ] Apache Spark commented on SPARK-41591: -- User 'harupy' has created a pull request for this issue:

[jira] [Created] (SPARK-42044) Fix wrong error message for `MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY`

2023-01-12 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-42044: --- Summary: Fix wrong error message for `MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY` Key: SPARK-42044 URL: https://issues.apache.org/jira/browse/SPARK-42044 Project: Spark

[jira] [Commented] (SPARK-42004) Migrate "XX000" sqlState onto `INTERNAL_ERROR`

2023-01-12 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676404#comment-17676404 ] Haejoon Lee commented on SPARK-42004: - I'm working on this > Migrate "XX000" sqlState onto

[jira] [Commented] (SPARK-42043) Basic Scala Client Result Implementation

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676401#comment-17676401 ] Apache Spark commented on SPARK-42043: -- User 'zhenlineo' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42043) Basic Scala Client Result Implementation

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42043: Assignee: (was: Apache Spark) > Basic Scala Client Result Implementation >

[jira] [Assigned] (SPARK-42043) Basic Scala Client Result Implementation

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42043: Assignee: Apache Spark > Basic Scala Client Result Implementation >

[jira] [Commented] (SPARK-42043) Basic Scala Client Result Implementation

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676400#comment-17676400 ] Apache Spark commented on SPARK-42043: -- User 'zhenlineo' has created a pull request for this issue:

[jira] [Updated] (SPARK-42043) Basic Scala Client Result Implementation

2023-01-12 Thread Zhen Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Li updated SPARK-42043: Description: Adding the basic scala client Result implementation. Add some tests to verify the result can

[jira] [Created] (SPARK-42043) Basic Scala Client

2023-01-12 Thread Zhen Li (Jira)
Zhen Li created SPARK-42043: --- Summary: Basic Scala Client Key: SPARK-42043 URL: https://issues.apache.org/jira/browse/SPARK-42043 Project: Spark Issue Type: Improvement Components:

[jira] [Created] (SPARK-42042) DataFrameReader should support StructType schema

2023-01-12 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-42042: - Summary: DataFrameReader should support StructType schema Key: SPARK-42042 URL: https://issues.apache.org/jira/browse/SPARK-42042 Project: Spark Issue

[jira] [Updated] (SPARK-42041) DataFrameReader should support list of pathes

2023-01-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-42041: -- Description: for example, {code:java} def load( self, path:

[jira] [Created] (SPARK-42041) DataFrameReader should support list of pathes

2023-01-12 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-42041: - Summary: DataFrameReader should support list of pathes Key: SPARK-42041 URL: https://issues.apache.org/jira/browse/SPARK-42041 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-41926) Add Github action test job with RocksDB as UI backend

2023-01-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41926. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39435

[jira] [Resolved] (SPARK-41991) Interpreted mode subexpression elimination can throw exception during insert

2023-01-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41991. -- Fix Version/s: 3.4.0 Assignee: Bruce Robbins Resolution: Fixed Fixed in

[jira] [Assigned] (SPARK-42039) SPJ: Remove Option in KeyGroupedPartitioning#partitionValues

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42039: Assignee: Apache Spark > SPJ: Remove Option in KeyGroupedPartitioning#partitionValues >

[jira] [Commented] (SPARK-42039) SPJ: Remove Option in KeyGroupedPartitioning#partitionValues

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676385#comment-17676385 ] Apache Spark commented on SPARK-42039: -- User 'sunchao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42039) SPJ: Remove Option in KeyGroupedPartitioning#partitionValues

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42039: Assignee: (was: Apache Spark) > SPJ: Remove Option in

[jira] [Created] (SPARK-42040) SPJ: Introduce a new API for V2 input partition to report partition size

2023-01-12 Thread Chao Sun (Jira)
Chao Sun created SPARK-42040: Summary: SPJ: Introduce a new API for V2 input partition to report partition size Key: SPARK-42040 URL: https://issues.apache.org/jira/browse/SPARK-42040 Project: Spark

[jira] [Created] (SPARK-42039) SPJ: Remove Option in KeyGroupedPartitioning#partitionValues

2023-01-12 Thread Chao Sun (Jira)
Chao Sun created SPARK-42039: Summary: SPJ: Remove Option in KeyGroupedPartitioning#partitionValues Key: SPARK-42039 URL: https://issues.apache.org/jira/browse/SPARK-42039 Project: Spark Issue

[jira] [Created] (SPARK-42038) SPJ: Support partially clustered distribution

2023-01-12 Thread Chao Sun (Jira)
Chao Sun created SPARK-42038: Summary: SPJ: Support partially clustered distribution Key: SPARK-42038 URL: https://issues.apache.org/jira/browse/SPARK-42038 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-42037) Remove `AMPLAB_` prefix in build environment variables

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676378#comment-17676378 ] Apache Spark commented on SPARK-42037: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-42037) Remove `AMPLAB_` prefix in build environment variables

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42037: Assignee: Apache Spark > Remove `AMPLAB_` prefix in build environment variables >

[jira] [Commented] (SPARK-42037) Remove `AMPLAB_` prefix in build environment variables

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676377#comment-17676377 ] Apache Spark commented on SPARK-42037: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-42037) Remove `AMPLAB_` prefix in build environment variables

2023-01-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42037: Assignee: (was: Apache Spark) > Remove `AMPLAB_` prefix in build environment

  1   2   3   >