[jira] [Resolved] (SPARK-45370) Fix python test when ansi mode enabled

2023-10-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45370. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43168

[jira] [Assigned] (SPARK-45370) Fix python test when ansi mode enabled

2023-10-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45370: Assignee: BingKun Pan > Fix python test when ansi mode enabled >

[jira] [Assigned] (SPARK-45566) Introduce Pandas-like testing utils for Pandas API on Spark

2023-10-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45566: Assignee: Haejoon Lee > Introduce Pandas-like testing utils for Pandas API on Spark >

[jira] [Resolved] (SPARK-45566) Introduce Pandas-like testing utils for Pandas API on Spark

2023-10-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45566. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43398

[jira] [Assigned] (SPARK-45572) Enable doctest of Frame.transpose

2023-10-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45572: Assignee: Ruifeng Zheng > Enable doctest of Frame.transpose >

[jira] [Resolved] (SPARK-45572) Enable doctest of Frame.transpose

2023-10-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45572. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43399

[jira] [Resolved] (SPARK-45562) XML: Make 'rowTag' a required option

2023-10-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45562. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43389

[jira] [Assigned] (SPARK-45562) XML: Make 'rowTag' a required option

2023-10-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45562: Assignee: Sandip Agarwala > XML: Make 'rowTag' a required option >

[jira] [Assigned] (SPARK-45567) Remove redundant if in org.apache.spark.sql.connect.execution.ExecuteGrpcResponseSender#run

2023-10-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45567: Assignee: Min Zhao > Remove redundant if in >

[jira] [Resolved] (SPARK-45567) Remove redundant if in org.apache.spark.sql.connect.execution.ExecuteGrpcResponseSender#run

2023-10-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45567. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43395

[jira] [Assigned] (SPARK-45517) Expand more exception constructors to support error framework parameters

2023-10-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45517: Assignee: Yihong He > Expand more exception constructors to support error framework

[jira] [Resolved] (SPARK-45517) Expand more exception constructors to support error framework parameters

2023-10-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45517. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43368

[jira] [Assigned] (SPARK-45538) pyspark connect overwrite_partitions bug

2023-10-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45538: Assignee: xie shuiahu > pyspark connect overwrite_partitions bug >

[jira] [Resolved] (SPARK-45538) pyspark connect overwrite_partitions bug

2023-10-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45538. -- Fix Version/s: 4.0.0 3.5.1 Resolution: Fixed Fixed in

[jira] [Updated] (SPARK-45486) make add_artifact idempotent

2023-10-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-45486: - Language: (was: Scala) > make add_artifact idempotent > > >

[jira] [Updated] (SPARK-45485) make add_artifact idempotent

2023-10-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-45485: - Description: Similar to the [Python

[jira] [Updated] (SPARK-45486) make add_artifact idempotent

2023-10-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-45486: - Priority: Minor (was: Major) > make add_artifact idempotent > > >

[jira] [Updated] (SPARK-45486) make add_artifact idempotent

2023-10-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-45486: - Summary: make add_artifact idempotent (was: [CONNECT][SCALA] Update user agent to recognize

[jira] [Resolved] (SPARK-45476) Raise exception directly instead of calling `resolveColumnsByPosition`

2023-10-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45476. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42762

[jira] [Assigned] (SPARK-45476) Raise exception directly instead of calling `resolveColumnsByPosition`

2023-10-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45476: Assignee: Haejoon Lee > Raise exception directly instead of calling

[jira] [Resolved] (SPARK-45535) Do not compile tests for snapshots and release build

2023-10-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45535. -- Resolution: Invalid > Do not compile tests for snapshots and release build >

[jira] [Created] (SPARK-45535) Do not compile tests for snapshots and release build

2023-10-13 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-45535: Summary: Do not compile tests for snapshots and release build Key: SPARK-45535 URL: https://issues.apache.org/jira/browse/SPARK-45535 Project: Spark Issue

[jira] [Assigned] (SPARK-45526) Refine docstring of `options` for dataframe reader and writer

2023-10-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45526: Assignee: Hyukjin Kwon (was: Allison Wang) > Refine docstring of `options` for

[jira] [Assigned] (SPARK-45526) Refine docstring of `options` for dataframe reader and writer

2023-10-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45526: Assignee: Allison Wang > Refine docstring of `options` for dataframe reader and writer >

[jira] [Resolved] (SPARK-45526) Refine docstring of `options` for dataframe reader and writer

2023-10-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45526. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43358

[jira] [Resolved] (SPARK-45528) Improve the example of DataFrameReader/Writer.options to take a dictionary

2023-10-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45528. -- Resolution: Duplicate > Improve the example of DataFrameReader/Writer.options to take a

[jira] [Created] (SPARK-45528) Improve the example of DataFrameReader/Writer.options to take a dictionary

2023-10-12 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-45528: Summary: Improve the example of DataFrameReader/Writer.options to take a dictionary Key: SPARK-45528 URL: https://issues.apache.org/jira/browse/SPARK-45528 Project:

[jira] [Resolved] (SPARK-45516) Include QueryContext in SparkThrowable proto message

2023-10-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45516. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43352

[jira] [Assigned] (SPARK-45516) Include QueryContext in SparkThrowable proto message

2023-10-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45516: Assignee: Yihong He > Include QueryContext in SparkThrowable proto message >

[jira] [Assigned] (SPARK-43664) Fix TABLE_OR_VIEW_NOT_FOUND from SQLParityTests

2023-10-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43664: Assignee: Haejoon Lee > Fix TABLE_OR_VIEW_NOT_FOUND from SQLParityTests >

[jira] [Resolved] (SPARK-43664) Fix TABLE_OR_VIEW_NOT_FOUND from SQLParityTests

2023-10-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43664. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43237

[jira] [Resolved] (SPARK-45488) XML: Add support for value in 'rowTag' element

2023-10-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45488. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43319

[jira] [Resolved] (SPARK-45221) Refine docstring of `DataFrameReader.parquet`

2023-10-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45221. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43301

[jira] [Assigned] (SPARK-45221) Refine docstring of `DataFrameReader.parquet`

2023-10-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45221: Assignee: Allison Wang > Refine docstring of `DataFrameReader.parquet` >

[jira] [Resolved] (SPARK-45416) Sanity check that Spark Connect returns arrow batches in order

2023-10-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45416. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43219

[jira] [Assigned] (SPARK-45416) Sanity check that Spark Connect returns arrow batches in order

2023-10-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45416: Assignee: Juliusz Sompolski > Sanity check that Spark Connect returns arrow batches in

[jira] [Resolved] (SPARK-45479) Recover Python build in branch-3.3, 3.4 and 3.5

2023-10-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45479. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43306

[jira] [Assigned] (SPARK-45479) Recover Python build in branch-3.3, 3.4 and 3.5

2023-10-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45479: Assignee: Hyukjin Kwon > Recover Python build in branch-3.3, 3.4 and 3.5 >

[jira] [Created] (SPARK-45479) Recover Python build in branch-3.3, 3.4 and 3.5

2023-10-10 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-45479: Summary: Recover Python build in branch-3.3, 3.4 and 3.5 Key: SPARK-45479 URL: https://issues.apache.org/jira/browse/SPARK-45479 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-45475) Should use DataFrame.foreachPartition instead of RDD.foreachPartition in JdbcUtils

2023-10-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45475. -- Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-45475) Should use DataFrame.foreachPartition instead of RDD.foreachPartition in JdbcUtils

2023-10-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45475: Assignee: Hyukjin Kwon > Should use DataFrame.foreachPartition instead of

[jira] [Resolved] (SPARK-45450) Fix imports according to PEP8: pyspark.pandas and pyspark (core)

2023-10-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45450. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43257

[jira] [Updated] (SPARK-45475) Should use DataFrame.foreachPartition instead of RDD.foreachPartition in JdbcUtils

2023-10-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-45475: - Summary: Should use DataFrame.foreachPartition instead of RDD.foreachPartition in JdbcUtils

[jira] [Created] (SPARK-45475) Should use DataFrame.foreachBatch instead of RDD.foreachBatch in JdbcUtils

2023-10-09 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-45475: Summary: Should use DataFrame.foreachBatch instead of RDD.foreachBatch in JdbcUtils Key: SPARK-45475 URL: https://issues.apache.org/jira/browse/SPARK-45475 Project:

[jira] [Resolved] (SPARK-43299) JVM Client throw StreamingQueryException when error handling is implemented

2023-10-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43299. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42859

[jira] [Assigned] (SPARK-43299) JVM Client throw StreamingQueryException when error handling is implemented

2023-10-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43299: Assignee: Yihong He > JVM Client throw StreamingQueryException when error handling is

[jira] [Created] (SPARK-45450) Fix imports according to PEP8: pyspark.pandas and pyspark (core)

2023-10-06 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-45450: Summary: Fix imports according to PEP8: pyspark.pandas and pyspark (core) Key: SPARK-45450 URL: https://issues.apache.org/jira/browse/SPARK-45450 Project: Spark

[jira] [Created] (SPARK-45448) Fix imports according to PEP8: pyspark.testing, pyspark.mllib, pyspark.resource and pyspark.streaming

2023-10-06 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-45448: Summary: Fix imports according to PEP8: pyspark.testing, pyspark.mllib, pyspark.resource and pyspark.streaming Key: SPARK-45448 URL:

[jira] [Assigned] (SPARK-45394) Retry handling for add_artifact

2023-10-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45394: Assignee: Alice Sayutina > Retry handling for add_artifact >

[jira] [Resolved] (SPARK-45394) Retry handling for add_artifact

2023-10-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45394. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43216

[jira] [Updated] (SPARK-45396) Add doc entry for `pyspark.ml.connect` module

2023-10-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-45396: - Target Version/s: 3.5.1 (was: 3.5.0, 3.5.1) > Add doc entry for `pyspark.ml.connect` module >

[jira] [Updated] (SPARK-45396) Add doc entry for `pyspark.ml.connect` module

2023-10-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-45396: - Fix Version/s: 3.5.1 > Add doc entry for `pyspark.ml.connect` module >

[jira] [Resolved] (SPARK-45420) Add DataType.fromDDL into PySpark

2023-10-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45420. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43226

[jira] [Assigned] (SPARK-45420) Add DataType.fromDDL into PySpark

2023-10-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45420: Assignee: Hyukjin Kwon > Add DataType.fromDDL into PySpark >

[jira] [Created] (SPARK-45420) Add DataType.fromDDL into PySpark

2023-10-04 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-45420: Summary: Add DataType.fromDDL into PySpark Key: SPARK-45420 URL: https://issues.apache.org/jira/browse/SPARK-45420 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-45386) Correctness issue when persisting using StorageLevel.NONE

2023-10-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45386. -- Fix Version/s: 3.5.1 Resolution: Fixed Issue resolved by pull request 43213

[jira] [Assigned] (SPARK-45386) Correctness issue when persisting using StorageLevel.NONE

2023-10-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45386: Assignee: Emil Ejbyfeldt > Correctness issue when persisting using StorageLevel.NONE >

[jira] [Resolved] (SPARK-45412) Validate the plan and session in DataFrame.__init__

2023-10-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45412. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43215

[jira] [Assigned] (SPARK-45412) Validate the plan and session in DataFrame.__init__

2023-10-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45412: Assignee: Ruifeng Zheng > Validate the plan and session in DataFrame.__init__ >

[jira] [Resolved] (SPARK-45345) Refactor release-build.sh

2023-10-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45345. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43212

[jira] [Assigned] (SPARK-45345) Refactor release-build.sh

2023-10-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45345: Assignee: Hyukjin Kwon > Refactor release-build.sh > - > >

[jira] [Resolved] (SPARK-45396) Add doc entry for `pyspark.ml.connect` module

2023-10-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45396. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43210

[jira] [Assigned] (SPARK-45396) Add doc entry for `pyspark.ml.connect` module

2023-10-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45396: Assignee: Weichen Xu > Add doc entry for `pyspark.ml.connect` module >

[jira] [Deleted] (SPARK-45411) Fix snapshot build in publish_snapshot.yml

2023-10-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon deleted SPARK-45411: - > Fix snapshot build in publish_snapshot.yml > -- > >

[jira] [Created] (SPARK-45411) Fix snapshot build in publish_snapshot.yml

2023-10-04 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-45411: Summary: Fix snapshot build in publish_snapshot.yml Key: SPARK-45411 URL: https://issues.apache.org/jira/browse/SPARK-45411 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-45406) Delete schema from DataFrame constructor

2023-10-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45406. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43206

[jira] [Assigned] (SPARK-45406) Delete schema from DataFrame constructor

2023-10-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45406: Assignee: Ruifeng Zheng > Delete schema from DataFrame constructor >

[jira] [Resolved] (SPARK-43620) Support `Column` for SparkConnectColumn.__getitem__

2023-10-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43620. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43120

[jira] [Assigned] (SPARK-43620) Support `Column` for SparkConnectColumn.__getitem__

2023-10-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43620: Assignee: Haejoon Lee > Support `Column` for SparkConnectColumn.__getitem__ >

[jira] [Assigned] (SPARK-45399) XML: Add XML Options using newOption

2023-10-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45399: Assignee: Sandip Agarwala > XML: Add XML Options using newOption >

[jira] [Resolved] (SPARK-45399) XML: Add XML Options using newOption

2023-10-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45399. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43201

[jira] [Resolved] (SPARK-45347) Include SparkThrowable in FetchErrorDetailsResponse

2023-10-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45347. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43136

[jira] [Assigned] (SPARK-45347) Include SparkThrowable in FetchErrorDetailsResponse

2023-10-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45347: Assignee: Yihong He > Include SparkThrowable in FetchErrorDetailsResponse >

[jira] [Deleted] (SPARK-45395) Include protobuf module into setup.py

2023-10-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon deleted SPARK-45395: - > Include protobuf module into setup.py > - > >

[jira] [Created] (SPARK-45395) Include protobuf module into setup.py

2023-10-02 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-45395: Summary: Include protobuf module into setup.py Key: SPARK-45395 URL: https://issues.apache.org/jira/browse/SPARK-45395 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-44762) Add more documentation and examples for using job tags for interrupt

2023-09-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44762: Assignee: Juliusz Sompolski > Add more documentation and examples for using job tags for

[jira] [Resolved] (SPARK-44762) Add more documentation and examples for using job tags for interrupt

2023-09-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44762. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43182

[jira] [Resolved] (SPARK-45359) DataFrame.{columns, colRegex, explain} should raise exceptions when plan is invalid

2023-09-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45359. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43150

[jira] [Assigned] (SPARK-45359) DataFrame.{columns, colRegex, explain} should raise exceptions when plan is invalid

2023-09-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45359: Assignee: Ruifeng Zheng > DataFrame.{columns, colRegex, explain} should raise exceptions

[jira] [Resolved] (SPARK-45358) Remove shim classes for Hive prior 2.0.0

2023-09-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45358. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43151

[jira] [Assigned] (SPARK-45358) Remove shim classes for Hive prior 2.0.0

2023-09-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45358: Assignee: Cheng Pan > Remove shim classes for Hive prior 2.0.0 >

[jira] [Resolved] (SPARK-45302) Remove PID communication between Python workers when no demon is used

2023-09-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45302. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43087

[jira] [Assigned] (SPARK-45302) Remove PID communication between Python workers when no demon is used

2023-09-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45302: Assignee: Hyukjin Kwon > Remove PID communication between Python workers when no demon

[jira] [Resolved] (SPARK-45339) Pyspark should log errors it retries

2023-09-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45339. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43127

[jira] [Assigned] (SPARK-45339) Pyspark should log errors it retries

2023-09-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45339: Assignee: Alice Sayutina > Pyspark should log errors it retries >

[jira] [Assigned] (SPARK-45328) Remove Hive support prior to 2.0.0

2023-09-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45328: Assignee: Hyukjin Kwon > Remove Hive support prior to 2.0.0 >

[jira] [Resolved] (SPARK-45328) Remove Hive support prior to 2.0.0

2023-09-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45328. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43116

[jira] [Assigned] (SPARK-45309) Remove all SystemUtils.isJavaVersionAtLeast with JDK 9

2023-09-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45309: Assignee: Hyukjin Kwon > Remove all SystemUtils.isJavaVersionAtLeast with JDK 9 >

[jira] [Resolved] (SPARK-45309) Remove all SystemUtils.isJavaVersionAtLeast with JDK 9

2023-09-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45309. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43098

[jira] [Resolved] (SPARK-45323) Upgrade snappy to 1.1.10.4

2023-09-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45323. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43109

[jira] [Assigned] (SPARK-45323) Upgrade snappy to 1.1.10.4

2023-09-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45323: Assignee: Bjørn Jørgensen > Upgrade snappy to 1.1.10.4 > -- > >

[jira] [Resolved] (SPARK-45333) Fix one unit mistake related to spark.eventLog.buffer.kb

2023-09-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45333. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42294

[jira] [Assigned] (SPARK-45333) Fix one unit mistake related to spark.eventLog.buffer.kb

2023-09-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45333: Assignee: Mengran Lan > Fix one unit mistake related to spark.eventLog.buffer.kb >

[jira] [Assigned] (SPARK-45315) Drop JDK 8/11 and make JDK 17 by default

2023-09-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45315: Assignee: Hyukjin Kwon > Drop JDK 8/11 and make JDK 17 by default >

[jira] [Created] (SPARK-45328) Remove Hive support prior to 2.0.0

2023-09-25 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-45328: Summary: Remove Hive support prior to 2.0.0 Key: SPARK-45328 URL: https://issues.apache.org/jira/browse/SPARK-45328 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-45317) Handle null filename in stack traces of exceptions

2023-09-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45317: Assignee: Yihong He > Handle null filename in stack traces of exceptions >

[jira] [Resolved] (SPARK-45317) Handle null filename in stack traces of exceptions

2023-09-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45317. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43103

[jira] [Assigned] (SPARK-45298) Remove the workaround for JDK-8228469 in SPARK-31959 test

2023-09-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45298: Assignee: Hyukjin Kwon > Remove the workaround for JDK-8228469 in SPARK-31959 test >

[jira] [Resolved] (SPARK-45298) Remove the workaround for JDK-8228469 in SPARK-31959 test

2023-09-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45298. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43083

[jira] [Resolved] (SPARK-45295) Remove Utils.isMemberClass workaround for JDK 8

2023-09-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45295. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43080

<    6   7   8   9   10   11   12   13   14   15   >