[jira] [Assigned] (SPARK-38809) Implement option to skip null values in symmetric hash impl of stream-stream joins

[ https://issues.apache.org/jira/browse/SPARK-38809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-38809: Assignee: Anish Shrigondekar > Implement option to skip null values in symmetric hash

[jira] [Resolved] (SPARK-38809) Implement option to skip null values in symmetric hash impl of stream-stream joins

[ https://issues.apache.org/jira/browse/SPARK-38809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-38809. -- Fix Version/s: 3.3.0 3.2.2 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-38827) Improve the test coverage for pyspark/find_spark_home.py

[ https://issues.apache.org/jira/browse/SPARK-38827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38827: Assignee: (was: Apache Spark) > Improve the test coverage for

[jira] [Commented] (SPARK-38827) Improve the test coverage for pyspark/find_spark_home.py

[ https://issues.apache.org/jira/browse/SPARK-38827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519286#comment-17519286 ] Apache Spark commented on SPARK-38827: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-38827) Improve the test coverage for pyspark/find_spark_home.py

[ https://issues.apache.org/jira/browse/SPARK-38827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38827: Assignee: Apache Spark > Improve the test coverage for pyspark/find_spark_home.py >

[jira] [Commented] (SPARK-38826) dropFieldIfAllNull option does not work for empty JSON struct

[ https://issues.apache.org/jira/browse/SPARK-38826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519285#comment-17519285 ] Apache Spark commented on SPARK-38826: -- User 'morvenhuang' has created a pull request for this

[jira] [Assigned] (SPARK-38826) dropFieldIfAllNull option does not work for empty JSON struct

[ https://issues.apache.org/jira/browse/SPARK-38826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38826: Assignee: (was: Apache Spark) > dropFieldIfAllNull option does not work for empty

[jira] [Assigned] (SPARK-38826) dropFieldIfAllNull option does not work for empty JSON struct

[ https://issues.apache.org/jira/browse/SPARK-38826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38826: Assignee: Apache Spark > dropFieldIfAllNull option does not work for empty JSON struct >

[jira] [Updated] (SPARK-38827) Improve the test coverage for pyspark/find_spark_home.py

[ https://issues.apache.org/jira/browse/SPARK-38827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38827: - Priority: Minor (was: Major) > Improve the test coverage for pyspark/find_spark_home.py >

[jira] [Created] (SPARK-38827) Improve the test coverage for pyspark/find_spark_home.py

Hyukjin Kwon created SPARK-38827: Summary: Improve the test coverage for pyspark/find_spark_home.py Key: SPARK-38827 URL: https://issues.apache.org/jira/browse/SPARK-38827 Project: Spark

[jira] [Assigned] (SPARK-38531) "Prune unrequired child index" branch of ColumnPruning has wrong condition

[ https://issues.apache.org/jira/browse/SPARK-38531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-38531: --- Assignee: Min Yang > "Prune unrequired child index" branch of ColumnPruning has wrong

[jira] [Resolved] (SPARK-38531) "Prune unrequired child index" branch of ColumnPruning has wrong condition

[ https://issues.apache.org/jira/browse/SPARK-38531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38531. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35864

[jira] [Updated] (SPARK-38826) dropFieldIfAllNull option does not work for empty JSON struct

[ https://issues.apache.org/jira/browse/SPARK-38826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] morvenhuang updated SPARK-38826: Description: As stated in the doc,  {quote}dropFieldIfAllNull Whether to ignore column of all

[jira] [Created] (SPARK-38826) dropFieldIfAllNull option does not work for empty JSON struct

morvenhuang created SPARK-38826: --- Summary: dropFieldIfAllNull option does not work for empty JSON struct Key: SPARK-38826 URL: https://issues.apache.org/jira/browse/SPARK-38826 Project: Spark

[jira] [Commented] (SPARK-38819) Run Pandas on Spark with Pandas 1.4.x

[ https://issues.apache.org/jira/browse/SPARK-38819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519278#comment-17519278 ] Hyukjin Kwon commented on SPARK-38819: -- Nice!! > Run Pandas on Spark with Pandas 1.4.x >

[jira] [Resolved] (SPARK-38806) Unable to initialize the empty pyspark.pandas dataframe

[ https://issues.apache.org/jira/browse/SPARK-38806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38806. -- Resolution: Invalid > Unable to initialize the empty pyspark.pandas dataframe >

[jira] [Resolved] (SPARK-38798) Make `spark.file.transferTo` as an `ConfigEntry`

[ https://issues.apache.org/jira/browse/SPARK-38798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38798. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36079

[jira] [Assigned] (SPARK-38798) Make `spark.file.transferTo` as an `ConfigEntry`

[ https://issues.apache.org/jira/browse/SPARK-38798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38798: Assignee: Yang Jie > Make `spark.file.transferTo` as an `ConfigEntry` >

[jira] [Resolved] (SPARK-38755) Add file to address missing pandas general functions

[ https://issues.apache.org/jira/browse/SPARK-38755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38755. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36034

[jira] [Assigned] (SPARK-38755) Add file to address missing pandas general functions

[ https://issues.apache.org/jira/browse/SPARK-38755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38755: Assignee: Haejoon Lee > Add file to address missing pandas general functions >

[jira] [Resolved] (SPARK-38800) Explicitly document the supported pandas version.

[ https://issues.apache.org/jira/browse/SPARK-38800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38800. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36095

[jira] [Assigned] (SPARK-38800) Explicitly document the supported pandas version.

[ https://issues.apache.org/jira/browse/SPARK-38800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38800: Assignee: Haejoon Lee > Explicitly document the supported pandas version. >

[jira] [Commented] (SPARK-38707) Allow user to insert into only certain columns of a table

[ https://issues.apache.org/jira/browse/SPARK-38707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519259#comment-17519259 ] morvenhuang commented on SPARK-38707: - Seems already a PR there,

[jira] [Resolved] (SPARK-38765) Implement `inplace` parameter of `Series.clip`

[ https://issues.apache.org/jira/browse/SPARK-38765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38765. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36041

[jira] [Assigned] (SPARK-38765) Implement `inplace` parameter of `Series.clip`

[ https://issues.apache.org/jira/browse/SPARK-38765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38765: Assignee: Xinrong Meng > Implement `inplace` parameter of `Series.clip` >

[jira] [Resolved] (SPARK-38726) Support `how` parameter of `MultiIndex.dropna`

[ https://issues.apache.org/jira/browse/SPARK-38726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38726. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36028

[jira] [Assigned] (SPARK-38726) Support `how` parameter of `MultiIndex.dropna`

[ https://issues.apache.org/jira/browse/SPARK-38726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38726: Assignee: Xinrong Meng > Support `how` parameter of `MultiIndex.dropna` >

[jira] [Updated] (SPARK-38819) Run Pandas on Spark with Pandas 1.4.x

[ https://issues.apache.org/jira/browse/SPARK-38819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-38819: Description: This is a umbrella to track issues when pandas upgrade to 1.4.x   I disable the

[jira] [Updated] (SPARK-38803) Set minio cpu to 250m (0.25) in K8s IT

[ https://issues.apache.org/jira/browse/SPARK-38803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-38803: Summary: Set minio cpu to 250m (0.25) in K8s IT (was: Add `minio` tag for K8s IT) > Set minio

[jira] [Commented] (SPARK-37575) null values should be saved as nothing rather than quoted empty Strings "" with default settings

[ https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519235#comment-17519235 ] Apache Spark commented on SPARK-37575: -- User 'anchovYu' has created a pull request for this issue:

[jira] [Commented] (SPARK-38825) Add a test to cover parquet notIn filter

[ https://issues.apache.org/jira/browse/SPARK-38825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519223#comment-17519223 ] Apache Spark commented on SPARK-38825: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38825) Add a test to cover parquet notIn filter

[ https://issues.apache.org/jira/browse/SPARK-38825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38825: Assignee: Apache Spark > Add a test to cover parquet notIn filter >

[jira] [Assigned] (SPARK-38825) Add a test to cover parquet notIn filter

[ https://issues.apache.org/jira/browse/SPARK-38825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38825: Assignee: (was: Apache Spark) > Add a test to cover parquet notIn filter >

[jira] [Commented] (SPARK-38825) Add a test to cover parquet notIn filter

[ https://issues.apache.org/jira/browse/SPARK-38825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519222#comment-17519222 ] Apache Spark commented on SPARK-38825: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Created] (SPARK-38825) Add a test to cover parquet notIn filter

Huaxin Gao created SPARK-38825: -- Summary: Add a test to cover parquet notIn filter Key: SPARK-38825 URL: https://issues.apache.org/jira/browse/SPARK-38825 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-38817) Upgrade kubernetes-client to 5.12.2

[ https://issues.apache.org/jira/browse/SPARK-38817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38817: -- Description: ``` Volcano extension doesn't support queue API ``` > Upgrade

[jira] [Resolved] (SPARK-38802) Support spark.kubernetes.test.(driver|executor)RequestCores

[ https://issues.apache.org/jira/browse/SPARK-38802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38802. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36087

[jira] [Assigned] (SPARK-38802) Support spark.kubernetes.test.(driver|executor)RequestCores

[ https://issues.apache.org/jira/browse/SPARK-38802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38802: - Assignee: Yikun Jiang > Support spark.kubernetes.test.(driver|executor)RequestCores >

[jira] [Updated] (SPARK-38817) Upgrade kubernetes-client to 5.12.2

[ https://issues.apache.org/jira/browse/SPARK-38817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38817: -- Parent: SPARK-36057 Issue Type: Sub-task (was: Bug) > Upgrade kubernetes-client to

[jira] [Resolved] (SPARK-38817) Upgrade kubernetes-client to 5.12.2

[ https://issues.apache.org/jira/browse/SPARK-38817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38817. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 36098

[jira] [Assigned] (SPARK-38817) Upgrade kubernetes-client to 5.12.2

[ https://issues.apache.org/jira/browse/SPARK-38817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38817: - Assignee: Yikun Jiang > Upgrade kubernetes-client to 5.12.2 >

[jira] [Updated] (SPARK-38817) Upgrade kubernetes-client to 5.12.2

[ https://issues.apache.org/jira/browse/SPARK-38817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38817: -- Issue Type: Bug (was: Improvement) > Upgrade kubernetes-client to 5.12.2 >

[jira] [Updated] (SPARK-38817) Upgrade kubernetes-client to 5.12.2

[ https://issues.apache.org/jira/browse/SPARK-38817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38817: -- Affects Version/s: 3.3.0 (was: 3.4.0) > Upgrade kubernetes-client

[jira] [Commented] (SPARK-38648) SPIP: Simplified API for DL Inferencing

[ https://issues.apache.org/jira/browse/SPARK-38648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519186#comment-17519186 ] Lee Yang commented on SPARK-38648: -- FYI, here is some proof-of-concept code (with example notebooks):

[jira] [Commented] (SPARK-37047) Add overloads for lpad and rpad for BINARY strings

[ https://issues.apache.org/jira/browse/SPARK-37047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519095#comment-17519095 ] Apache Spark commented on SPARK-37047: -- User 'anchovYu' has created a pull request for this issue:

[jira] [Commented] (SPARK-37047) Add overloads for lpad and rpad for BINARY strings

[ https://issues.apache.org/jira/browse/SPARK-37047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519094#comment-17519094 ] Apache Spark commented on SPARK-37047: -- User 'anchovYu' has created a pull request for this issue:

[jira] [Updated] (SPARK-38823) Incorrect result of dataset reduceGroups in java

[ https://issues.apache.org/jira/browse/SPARK-38823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] IKozar updated SPARK-38823: --- Description: {code:java} @Data @NoArgsConstructor @AllArgsConstructor public static class Item

[jira] [Updated] (SPARK-38823) Incorrect result of dataset reduceGroups in java

[ https://issues.apache.org/jira/browse/SPARK-38823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] IKozar updated SPARK-38823: --- Description: {code:java} @Data @NoArgsConstructor @AllArgsConstructor public static class Item

[jira] [Created] (SPARK-38824) Bug in async commit of Kafka offset in DirectKafkaInputDStream

SOUVIK PAUL created SPARK-38824: --- Summary: Bug in async commit of Kafka offset in DirectKafkaInputDStream Key: SPARK-38824 URL: https://issues.apache.org/jira/browse/SPARK-38824 Project: Spark

[jira] [Created] (SPARK-38823) Incorrect result of dataset reduceGroups in java

IKozar created SPARK-38823: -- Summary: Incorrect result of dataset reduceGroups in java Key: SPARK-38823 URL: https://issues.apache.org/jira/browse/SPARK-38823 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-18208) Executor OOM due to a memory leak in BytesToBytesMap

[ https://issues.apache.org/jira/browse/SPARK-18208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519057#comment-17519057 ] Sachit commented on SPARK-18208: [~jiexiong]  [~dongjoon] : Getting same issue in Spark 2.4.4 . Could

[jira] [Commented] (SPARK-32268) Bloom Filter Join

[ https://issues.apache.org/jira/browse/SPARK-32268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519047#comment-17519047 ] Apache Spark commented on SPARK-32268: -- User 'andylam-db' has created a pull request for this

[jira] [Resolved] (SPARK-38818) Fix the docs of try_multiply/try_subtract/ANSI cast

[ https://issues.apache.org/jira/browse/SPARK-38818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-38818. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 36099

[jira] [Commented] (SPARK-37013) `select format_string('%0$s', 'Hello')` has different behavior when using java 8 and Java 17

[ https://issues.apache.org/jira/browse/SPARK-37013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519006#comment-17519006 ] Apache Spark commented on SPARK-37013: -- User 'anchovYu' has created a pull request for this issue:

[jira] [Commented] (SPARK-37013) `select format_string('%0$s', 'Hello')` has different behavior when using java 8 and Java 17

[ https://issues.apache.org/jira/browse/SPARK-37013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519007#comment-17519007 ] Apache Spark commented on SPARK-37013: -- User 'anchovYu' has created a pull request for this issue:

[jira] [Updated] (SPARK-38792) Regression in time executor takes to do work sometime after v3.0.1 ?

[ https://issues.apache.org/jira/browse/SPARK-38792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Guinther updated SPARK-38792: --- Attachment: executor-timing-debug-number-4.jpg > Regression in time executor takes to do

[jira] [Updated] (SPARK-38792) Regression in time executor takes to do work sometime after v3.0.1 ?

[ https://issues.apache.org/jira/browse/SPARK-38792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Guinther updated SPARK-38792: --- Attachment: executor-timing-debug-number-2.jpg > Regression in time executor takes to do

[jira] [Updated] (SPARK-38792) Regression in time executor takes to do work sometime after v3.0.1 ?

[ https://issues.apache.org/jira/browse/SPARK-38792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Guinther updated SPARK-38792: --- Attachment: executor-timing-debug-number-5.jpg > Regression in time executor takes to do

[jira] [Commented] (SPARK-38792) Regression in time executor takes to do work sometime after v3.0.1 ?

[ https://issues.apache.org/jira/browse/SPARK-38792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518998#comment-17518998 ] Danny Guinther commented on SPARK-38792: I added yet another kind of dummy job that aims to

[jira] [Updated] (SPARK-37643) when charVarcharAsString is true, char datatype partition table query incorrect

[ https://issues.apache.org/jira/browse/SPARK-37643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YuanGuanhu updated SPARK-37643: --- Description: This ticket aim at fixing the bug that does not apply right-padding for char types

[jira] [Commented] (SPARK-36194) Add a logical plan visitor to propagate the distinct attributes

[ https://issues.apache.org/jira/browse/SPARK-36194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518966#comment-17518966 ] Apache Spark commented on SPARK-36194: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-36194) Add a logical plan visitor to propagate the distinct attributes

[ https://issues.apache.org/jira/browse/SPARK-36194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518965#comment-17518965 ] Apache Spark commented on SPARK-36194: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Created] (SPARK-38822) Raise indexError when insert loc is out of bounds

Yikun Jiang created SPARK-38822: --- Summary: Raise indexError when insert loc is out of bounds Key: SPARK-38822 URL: https://issues.apache.org/jira/browse/SPARK-38822 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-38760) Implement DataFrame.observe in PySpark

[ https://issues.apache.org/jira/browse/SPARK-38760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38760: Assignee: Hyukjin Kwon > Implement DataFrame.observe in PySpark >

[jira] [Resolved] (SPARK-38760) Implement DataFrame.observe in PySpark

[ https://issues.apache.org/jira/browse/SPARK-38760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38760. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36082

[jira] [Assigned] (SPARK-38804) Add StreamingQueryManager.removeListener in PySpark

[ https://issues.apache.org/jira/browse/SPARK-38804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38804: Assignee: Hyukjin Kwon > Add StreamingQueryManager.removeListener in PySpark >

[jira] [Resolved] (SPARK-38804) Add StreamingQueryManager.removeListener in PySpark

[ https://issues.apache.org/jira/browse/SPARK-38804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38804. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36086

[jira] [Updated] (SPARK-38819) Run Pandas on Spark with Pandas 1.4.x

[ https://issues.apache.org/jira/browse/SPARK-38819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-38819: Description: This is a umbrella to track issues when pandas upgrade to 1.4.x > Run Pandas on

[jira] [Updated] (SPARK-38821) test_nsmallest test failed tue to pandas 1.4.1/1.4.2 bug

[ https://issues.apache.org/jira/browse/SPARK-38821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-38821: Summary: test_nsmallest test failed tue to pandas 1.4.1/1.4.2 bug (was: part of test_nsmallest

[jira] [Created] (SPARK-38820) test_boolean_ops failed due to dtype mismatch

Yikun Jiang created SPARK-38820: --- Summary: test_boolean_ops failed due to dtype mismatch Key: SPARK-38820 URL: https://issues.apache.org/jira/browse/SPARK-38820 Project: Spark Issue Type:

[jira] [Created] (SPARK-38821) part of test_nsmallest test failed tue to pandas 1.4.1/1.4.2 bug

Yikun Jiang created SPARK-38821: --- Summary: part of test_nsmallest test failed tue to pandas 1.4.1/1.4.2 bug Key: SPARK-38821 URL: https://issues.apache.org/jira/browse/SPARK-38821 Project: Spark

[jira] [Created] (SPARK-38819) Run Pandas on Spark with Pandas 1.4.x

Yikun Jiang created SPARK-38819: --- Summary: Run Pandas on Spark with Pandas 1.4.x Key: SPARK-38819 URL: https://issues.apache.org/jira/browse/SPARK-38819 Project: Spark Issue Type: Umbrella

[jira] [Commented] (SPARK-38818) Fix the docs of try_multiply/try_subtract/ANSI cast

[ https://issues.apache.org/jira/browse/SPARK-38818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518872#comment-17518872 ] Apache Spark commented on SPARK-38818: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-38818) Fix the docs of try_multiply/try_subtract/ANSI cast

[ https://issues.apache.org/jira/browse/SPARK-38818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38818: Assignee: Apache Spark (was: Gengliang Wang) > Fix the docs of

[jira] [Assigned] (SPARK-38818) Fix the docs of try_multiply/try_subtract/ANSI cast

[ https://issues.apache.org/jira/browse/SPARK-38818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38818: Assignee: Gengliang Wang (was: Apache Spark) > Fix the docs of

[jira] [Commented] (SPARK-38818) Fix the docs of try_multiply/try_subtract/ANSI cast

[ https://issues.apache.org/jira/browse/SPARK-38818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518871#comment-17518871 ] Apache Spark commented on SPARK-38818: -- User 'gengliangwang' has created a pull request for this

[jira] [Created] (SPARK-38818) Fix the docs of try_multiply/try_subtract/ANSI cast

Gengliang Wang created SPARK-38818: -- Summary: Fix the docs of try_multiply/try_subtract/ANSI cast Key: SPARK-38818 URL: https://issues.apache.org/jira/browse/SPARK-38818 Project: Spark

[jira] [Commented] (SPARK-38817) Upgrade kubernetes-client to 5.12.2

[ https://issues.apache.org/jira/browse/SPARK-38817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518826#comment-17518826 ] Apache Spark commented on SPARK-38817: -- User 'Yikun' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38817) Upgrade kubernetes-client to 5.12.2

[ https://issues.apache.org/jira/browse/SPARK-38817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38817: Assignee: Apache Spark > Upgrade kubernetes-client to 5.12.2 >

[jira] [Assigned] (SPARK-38817) Upgrade kubernetes-client to 5.12.2

[ https://issues.apache.org/jira/browse/SPARK-38817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38817: Assignee: (was: Apache Spark) > Upgrade kubernetes-client to 5.12.2 >

[jira] [Created] (SPARK-38817) Upgrade kubernetes-client to 5.12.2

Yikun Jiang created SPARK-38817: --- Summary: Upgrade kubernetes-client to 5.12.2 Key: SPARK-38817 URL: https://issues.apache.org/jira/browse/SPARK-38817 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-38548) New SQL function: try_sum

[ https://issues.apache.org/jira/browse/SPARK-38548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518815#comment-17518815 ] Apache Spark commented on SPARK-38548: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-38548) New SQL function: try_sum

[ https://issues.apache.org/jira/browse/SPARK-38548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518814#comment-17518814 ] Apache Spark commented on SPARK-38548: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-38110) Use error classes in the compilation errors of windows

[ https://issues.apache.org/jira/browse/SPARK-38110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518813#comment-17518813 ] Goutam Ghosh commented on SPARK-38110: -- [~maxgekk] can I work on this ? > Use error classes in

[jira] [Created] (SPARK-38816) Wrong code results in random matrix generator in spark-als algorithm

Nickolay created SPARK-38816: Summary: Wrong code results in random matrix generator in spark-als algorithm Key: SPARK-38816 URL: https://issues.apache.org/jira/browse/SPARK-38816 Project: Spark

[jira] [Updated] (SPARK-38815) Jdbc driver class not found when enable hive and call jdbc action function multiple times

[ https://issues.apache.org/jira/browse/SPARK-38815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zery updated SPARK-38815: - Summary: Jdbc driver class not found when enable hive and call jdbc action function multiple times (was:

[jira] [Updated] (SPARK-38815) Jdbc driver class not found when enable hive and call multiple jdbc action function

[ https://issues.apache.org/jira/browse/SPARK-38815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zery updated SPARK-38815: - Summary: Jdbc driver class not found when enable hive and call multiple jdbc action function (was: Don't

[jira] [Updated] (SPARK-38815) Don't found jdbc driver class when enable hive and call multiple jdbc action function

[ https://issues.apache.org/jira/browse/SPARK-38815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zery updated SPARK-38815: - Summary: Don't found jdbc driver class when enable hive and call multiple jdbc action function (was: Not found

[jira] [Updated] (SPARK-38815) Not found jdbc driver class when enable hive and call multiple jdbc action function

[ https://issues.apache.org/jira/browse/SPARK-38815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zery updated SPARK-38815: - Summary: Not found jdbc driver class when enable hive and call multiple jdbc action function (was: Not found

[jira] [Created] (SPARK-38815) Not found jdbc driver class when enable hive and call multiple jdbc action

zery created SPARK-38815: Summary: Not found jdbc driver class when enable hive and call multiple jdbc action Key: SPARK-38815 URL: https://issues.apache.org/jira/browse/SPARK-38815 Project: Spark

[jira] [Commented] (SPARK-38803) Add `minio` tag for K8s IT

[ https://issues.apache.org/jira/browse/SPARK-38803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518690#comment-17518690 ] Apache Spark commented on SPARK-38803: -- User 'Yikun' has created a pull request for this issue: