[jira] [Assigned] (SPARK-49641) Include `table_funcs` and `variant_funcs` in the built-in function list doc

2024-09-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49641: Assignee: BingKun Pan > Include `table_funcs` and `variant_funcs` in the built-in functio

[jira] [Resolved] (SPARK-49641) Include `table_funcs` and `variant_funcs` in the built-in function list doc

2024-09-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49641. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48106 [https://gi

[jira] [Updated] (SPARK-49422) Create a shared KeyValueGroupedDataset interface

2024-09-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-49422: - Fix Version/s: (was: 4.0.0) > Create a shared KeyValueGroupedDataset interface > ---

[jira] [Reopened] (SPARK-49422) Create a shared KeyValueGroupedDataset interface

2024-09-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-49422: -- reverted at https://github.com/apache/spark/commit/f3c8d26eb0c3fd7f77950eb08c70bb2a9ab6493c > Cr

[jira] [Updated] (SPARK-49495) Document and Feature Preview on master branch via Live GitHub Pages Updates

2024-09-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-49495: - Fix Version/s: (was: 4.0.0) > Document and Feature Preview on master branch via Live GitHub

[jira] [Reopened] (SPARK-49495) Document and Feature Preview on master branch via Live GitHub Pages Updates

2024-09-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-49495: -- Assignee: (was: Kent Yao) Reverted in https://github.com/apache/spark/commit/7de71a2ec7

[jira] [Resolved] (SPARK-49684) Get rid of unnecessary global locks from Spark Connect service

2024-09-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49684. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48131 [https://gi

[jira] [Assigned] (SPARK-49684) Get rid of unnecessary global locks from Spark Connect service

2024-09-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49684: Assignee: Changgyoo Park > Get rid of unnecessary global locks from Spark Connect service

[jira] [Resolved] (SPARK-49688) Data race between SparkListener and SparkConnectServiceSuite over ExecuteHolder

2024-09-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49688. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48142 [https://gi

[jira] [Assigned] (SPARK-49688) Data race between SparkListener and SparkConnectServiceSuite over ExecuteHolder

2024-09-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49688: Assignee: Changgyoo Park > Data race between SparkListener and SparkConnectServiceSuite o

[jira] [Assigned] (SPARK-49673) Increase maxBatchSize for Connect's sqlCommandResult

2024-09-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49673: Assignee: Robert Dillitz > Increase maxBatchSize for Connect's sqlCommandResult > ---

[jira] [Resolved] (SPARK-49673) Increase maxBatchSize for Connect's sqlCommandResult

2024-09-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49673. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48122 [https://gi

[jira] [Assigned] (SPARK-49548) Get rid of coarse-locking in SparkConnectSessionManager

2024-09-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49548: Assignee: Changgyoo Park > Get rid of coarse-locking in SparkConnectSessionManager >

[jira] [Resolved] (SPARK-49544) Severe lock contention in SparkConnectExecutionManager

2024-09-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49544. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48034 [https://gi

[jira] [Resolved] (SPARK-49548) Get rid of coarse-locking in SparkConnectSessionManager

2024-09-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49548. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48036 [https://gi

[jira] [Assigned] (SPARK-49544) Severe lock contention in SparkConnectExecutionManager

2024-09-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49544: Assignee: Changgyoo Park > Severe lock contention in SparkConnectExecutionManager > -

[jira] [Assigned] (SPARK-49553) Remove experimental API notes for pandas related functions

2024-09-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49553: Assignee: Allison Wang > Remove experimental API notes for pandas related functions > ---

[jira] [Resolved] (SPARK-49553) Remove experimental API notes for pandas related functions

2024-09-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49553. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48042 [https://gi

[jira] [Resolved] (SPARK-49582) Improve "dispatch_window_method" utility and docstring

2024-09-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49582. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48056 [https://gi

[jira] [Assigned] (SPARK-49582) Improve "dispatch_window_method" utility and docstring

2024-09-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49582: Assignee: Xinrong Meng > Improve "dispatch_window_method" utility and docstring > ---

[jira] [Resolved] (SPARK-49576) Upload Python logs in CI

2024-09-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49576. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48048 [https://gi

[jira] [Resolved] (SPARK-49536) Add error handling for python streaming data source record prefetching

2024-09-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49536. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48023 [https://gi

[jira] [Assigned] (SPARK-49536) Add error handling for python streaming data source record prefetching

2024-09-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49536: Assignee: Chaoqin Li > Add error handling for python streaming data source record prefetc

[jira] [Created] (SPARK-49545) Increase timeout for build from 3 to 4 hours

2024-09-09 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-49545: Summary: Increase timeout for build from 3 to 4 hours Key: SPARK-49545 URL: https://issues.apache.org/jira/browse/SPARK-49545 Project: Spark Issue Type: Impr

[jira] [Resolved] (SPARK-49532) Improve documentation of "plotting.sample_ratio" option

2024-09-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49532. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48013 [https://gi

[jira] [Assigned] (SPARK-49532) Improve documentation of "plotting.sample_ratio" option

2024-09-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49532: Assignee: Xinrong Meng > Improve documentation of "plotting.sample_ratio" option > --

[jira] [Assigned] (SPARK-49284) Create a shared Catalog interface

2024-09-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49284: Assignee: Herman van Hövell > Create a shared Catalog interface > ---

[jira] [Resolved] (SPARK-49284) Create a shared Catalog interface

2024-09-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49284. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47991 [https://gi

[jira] [Resolved] (SPARK-49511) Attempt to apply formatting rules to sql/api

2024-09-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49511. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47989 [https://gi

[jira] [Assigned] (SPARK-49525) Log improvement for server side streaming query listener bus listener

2024-09-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49525: Assignee: Wei Liu > Log improvement for server side streaming query listener bus listener

[jira] [Assigned] (SPARK-49526) Windows-style paths are unsupported in ArtifactManager

2024-09-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49526: Assignee: Venkata Sai Akhil Gudesa > Windows-style paths are unsupported in ArtifactManag

[jira] [Resolved] (SPARK-49526) Windows-style paths are unsupported in ArtifactManager

2024-09-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49526. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48003 [https://gi

[jira] [Assigned] (SPARK-49458) ReattachExecute does not supply server-side session id

2024-09-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49458: Assignee: Changgyoo Park > ReattachExecute does not supply server-side session id > -

[jira] [Assigned] (SPARK-48960) Makes spark-submit works with Spark connect

2024-09-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48960: Assignee: Hyukjin Kwon > Makes spark-submit works with Spark connect > --

[jira] [Resolved] (SPARK-49477) Improve pandas udf return type error message

2024-09-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49477. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47942 [https://gi

[jira] [Assigned] (SPARK-49456) Spark website doesn't properly scroll to hash links

2024-08-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49456: Assignee: Neil Ramaswamy > Spark website doesn't properly scroll to hash links > ---

[jira] [Resolved] (SPARK-49456) Spark website doesn't properly scroll to hash links

2024-08-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49456. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47925 [https://gi

[jira] [Resolved] (SPARK-49449) Remove string and binary from metadata in spec

2024-08-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49449. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47917 [https://gi

[jira] [Assigned] (SPARK-49402) Fix Binder integration in PySpark documentation

2024-08-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49402: Assignee: Hyukjin Kwon > Fix Binder integration in PySpark documentation > --

[jira] [Resolved] (SPARK-49402) Fix Binder integration in PySpark documentation

2024-08-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49402. -- Fix Version/s: 3.4.4 4.0.0 3.5.3 Resolution: Fixed

[jira] [Created] (SPARK-49402) Fix Binder integration in PySpark documentation

2024-08-26 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-49402: Summary: Fix Binder integration in PySpark documentation Key: SPARK-49402 URL: https://issues.apache.org/jira/browse/SPARK-49402 Project: Spark Issue Type: B

[jira] [Resolved] (SPARK-49387) Fix type hint for `accuracy` in `percentile_approx` and `approx_percentile`

2024-08-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49387. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47869 [https://gi

[jira] [Assigned] (SPARK-49387) Fix type hint for `accuracy` in `percentile_approx` and `approx_percentile`

2024-08-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49387: Assignee: Ruifeng Zheng > Fix type hint for `accuracy` in `percentile_approx` and `approx

[jira] [Resolved] (SPARK-49370) maven.scaladoc.skip should not affect test code compilation

2024-08-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49370. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47858 [https://gi

[jira] [Assigned] (SPARK-49370) maven.scaladoc.skip should not affect test code compilation

2024-08-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49370: Assignee: Cheng Pan > maven.scaladoc.skip should not affect test code compilation > -

[jira] [Resolved] (SPARK-49347) Deprecate SaprkR

2024-08-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49347. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47842 [https://gi

[jira] [Assigned] (SPARK-49347) Deprecate SaprkR

2024-08-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49347: Assignee: Hyukjin Kwon > Deprecate SaprkR > > > Key: SPA

[jira] [Assigned] (SPARK-49368) Avoid accessing protobuf lite classes directly

2024-08-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49368: Assignee: Cheng Pan > Avoid accessing protobuf lite classes directly > --

[jira] [Resolved] (SPARK-49368) Avoid accessing protobuf lite classes directly

2024-08-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49368. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47857 [https://gi

[jira] [Resolved] (SPARK-49359) allow StagedTableCatalog implementations to fall back to non-atomic write

2024-08-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49359. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47848 [https://gi

[jira] [Assigned] (SPARK-49359) allow StagedTableCatalog implementations to fall back to non-atomic write

2024-08-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49359: Assignee: Wenchen Fan > allow StagedTableCatalog implementations to fall back to non-atom

[jira] [Assigned] (SPARK-49344) Support `json_normalize` for Pandas API on Spark

2024-08-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49344: Assignee: Haejoon Lee > Support `json_normalize` for Pandas API on Spark > --

[jira] [Resolved] (SPARK-49344) Support `json_normalize` for Pandas API on Spark

2024-08-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49344. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47840 [https://gi

[jira] [Resolved] (SPARK-49301) [PySpark] Chunk arrow data passed to Python worker

2024-08-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49301. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47804 [https://gi

[jira] [Updated] (SPARK-49347) Deprecate SaprkR

2024-08-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-49347: - Issue Type: Task (was: Improvement) > Deprecate SaprkR > > > K

[jira] [Updated] (SPARK-49347) Deprecate SaprkR

2024-08-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-49347: - Priority: Critical (was: Major) > Deprecate SaprkR > > > Key:

[jira] [Created] (SPARK-49347) Deprecate SaprkR

2024-08-21 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-49347: Summary: Deprecate SaprkR Key: SPARK-49347 URL: https://issues.apache.org/jira/browse/SPARK-49347 Project: Spark Issue Type: Improvement Components

[jira] [Resolved] (SPARK-49121) Support from_protobuf and to_protobuf for SQL functions

2024-08-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49121. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47716 [https://gi

[jira] [Assigned] (SPARK-49121) Support from_protobuf and to_protobuf for SQL functions

2024-08-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49121: Assignee: Haejoon Lee > Support from_protobuf and to_protobuf for SQL functions > ---

[jira] [Resolved] (SPARK-49345) Make sure using the current running Spark Session

2024-08-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49345. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47838 [https://gi

[jira] [Resolved] (SPARK-49323) Move MockObserver from Spark Connect Server's test folder to the Server's main folder

2024-08-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49323. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47816 [https://gi

[jira] [Assigned] (SPARK-49323) Move MockObserver from Spark Connect Server's test folder to the Server's main folder

2024-08-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49323: Assignee: Thang Long Vu > Move MockObserver from Spark Connect Server's test folder to t

[jira] [Resolved] (SPARK-49026) Create conversions Column API to Connect protos

2024-08-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49026. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47812 [https://gi

[jira] [Resolved] (SPARK-49274) Support java serialization in AgnosticEncoders

2024-08-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49274. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47806 [https://gi

[jira] [Resolved] (SPARK-49269) Improve performance and memory footprint of all VALUES clauses

2024-08-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49269. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47791 [https://gi

[jira] [Assigned] (SPARK-49269) Improve performance and memory footprint of all VALUES clauses

2024-08-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49269: Assignee: Costas Zarifis > Improve performance and memory footprint of all VALUES clauses

[jira] [Resolved] (SPARK-49263) Spark Connect python client: Consistently handle boolean Dataframe reader options

2024-08-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49263. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47790 [https://gi

[jira] [Assigned] (SPARK-49263) Spark Connect python client: Consistently handle boolean Dataframe reader options

2024-08-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49263: Assignee: Juliusz Sompolski > Spark Connect python client: Consistently handle boolean Da

[jira] [Assigned] (SPARK-49257) Make `bundler install` can retry

2024-08-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49257: Assignee: BingKun Pan > Make `bundler install` can retry > --

[jira] [Resolved] (SPARK-49257) Make `bundler install` can retry

2024-08-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49257. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47784 [https://gi

[jira] [Assigned] (SPARK-49260) Should not prepend the classes path of sql/core module in Spark Connect Shell

2024-08-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49260: Assignee: Yang Jie > Should not prepend the classes path of sql/core module in Spark Conn

[jira] [Resolved] (SPARK-49260) Should not prepend the classes path of sql/core module in Spark Connect Shell

2024-08-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49260. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47786 [https://gi

[jira] [Resolved] (SPARK-49022) Integrate Basic ColumnNode API in Column

2024-08-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49022. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47688 [https://gi

[jira] [Resolved] (SPARK-48966) Improve error message with invalid unresolved column reference in UDTF call

2024-08-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48966. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47447 [https://gi

[jira] [Assigned] (SPARK-48966) Improve error message with invalid unresolved column reference in UDTF call

2024-08-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48966: Assignee: Daniel > Improve error message with invalid unresolved column reference in UDTF

[jira] [Resolved] (SPARK-49221) sbt compilation warning: `Regular tasks always evaluate task dependencies (.value) regardless of if expressions`

2024-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49221. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47739 [https://gi

[jira] [Assigned] (SPARK-49221) sbt compilation warning: `Regular tasks always evaluate task dependencies (.value) regardless of if expressions`

2024-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49221: Assignee: Yang Jie > sbt compilation warning: `Regular tasks always evaluate task depende

[jira] [Assigned] (SPARK-49215) Document the NaN handling in df.na.drop

2024-08-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49215: Assignee: Ruifeng Zheng > Document the NaN handling in df.na.drop > -

[jira] [Resolved] (SPARK-49215) Document the NaN handling in df.na.drop

2024-08-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49215. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47731 [https://gi

[jira] [Resolved] (SPARK-49201) Reimplement hist plot with Spark SQL

2024-08-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49201. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47708 [https://gi

[jira] [Resolved] (SPARK-49198) Prune more jars required for Spark Connect shell

2024-08-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49198. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47705 [https://gi

[jira] [Resolved] (SPARK-49173) Change Spark Connect shell prompt from `@` to `scala>`

2024-08-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49173. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47679 [https://gi

[jira] [Assigned] (SPARK-49173) Change Spark Connect shell prompt from `@` to `scala>`

2024-08-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49173: Assignee: Hyukjin Kwon > Change Spark Connect shell prompt from `@` to `scala>` > ---

[jira] [Updated] (SPARK-49194) Makes Spark scripts work with Spark connect

2024-08-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-49194: - Description: This is an umbrella ticket that includes tasks making Spark scripts such as bin/pys

[jira] [Created] (SPARK-49198) Prune more jars required for Spark Connect shell

2024-08-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-49198: Summary: Prune more jars required for Spark Connect shell Key: SPARK-49198 URL: https://issues.apache.org/jira/browse/SPARK-49198 Project: Spark Issue Type:

[jira] [Created] (SPARK-49195) Embed script level parsing logic into SparkSubmitCommandBuilder

2024-08-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-49195: Summary: Embed script level parsing logic into SparkSubmitCommandBuilder Key: SPARK-49195 URL: https://issues.apache.org/jira/browse/SPARK-49195 Project: Spark

[jira] [Updated] (SPARK-49173) Change Spark Connect shell prompt from `@` to `scala>`

2024-08-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-49173: - Parent: SPARK-49194 Issue Type: Sub-task (was: Improvement) > Change Spark Connect shel

[jira] [Updated] (SPARK-48960) Makes spark-submit works with Spark connect

2024-08-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-48960: - Parent: SPARK-49194 Issue Type: Sub-task (was: Improvement) > Makes spark-submit works

[jira] [Updated] (SPARK-48936) Makes spark-shell works with Spark connect

2024-08-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-48936: - Parent: SPARK-49194 Issue Type: Sub-task (was: Improvement) > Makes spark-shell works w

[jira] [Updated] (SPARK-49171) Update Spark Shell documentation with Spark Connect

2024-08-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-49171: - Summary: Update Spark Shell documentation with Spark Connect (was: Update Spark Shell with Spar

[jira] [Updated] (SPARK-49171) Update Spark Shell documentation with Spark Connect

2024-08-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-49171: - Parent: SPARK-49194 Issue Type: Sub-task (was: Improvement) > Update Spark Shell docume

[jira] [Resolved] (SPARK-49193) Improve the performance of RowSetUtils.toColumnBasedSet

2024-08-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49193. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47699 [https://gi

[jira] [Resolved] (SPARK-48936) Makes spark-shell works with Spark connect

2024-08-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-48936. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47402 [https://gi

[jira] [Assigned] (SPARK-48936) Makes spark-shell works with Spark connect

2024-08-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-48936: Assignee: Hyukjin Kwon > Makes spark-shell works with Spark connect > ---

[jira] [Assigned] (SPARK-49177) Make GA's `build_error_docs` run only once

2024-08-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49177: Assignee: BingKun Pan > Make GA's `build_error_docs` run only once >

[jira] [Resolved] (SPARK-49177) Make GA's `build_error_docs` run only once

2024-08-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49177. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47682 [https://gi

[jira] [Resolved] (SPARK-49176) Fix `spark.ui.custom.executor.log.url` docs by adding K8s

2024-08-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49176. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47681 [https://gi

[jira] [Created] (SPARK-49173) Change Spark Connect shell prompt from `@` to `scala>`

2024-08-08 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-49173: Summary: Change Spark Connect shell prompt from `@` to `scala>` Key: SPARK-49173 URL: https://issues.apache.org/jira/browse/SPARK-49173 Project: Spark Issue

[jira] [Created] (SPARK-49171) Update Spark Shell with Spark Connect

2024-08-08 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-49171: Summary: Update Spark Shell with Spark Connect Key: SPARK-49171 URL: https://issues.apache.org/jira/browse/SPARK-49171 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-49142) Lower Spark Connect client log messages to debug level

2024-08-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49142: Assignee: Niranjan Jayakar > Lower Spark Connect client log messages to debug level > ---

  1   2   3   4   5   6   7   8   9   10   >