[jira] [Created] (SPARK-45814) ArrowConverters.createEmptyArrowBatch may cause memory leak

xie shuiahu created SPARK-45814: --- Summary: ArrowConverters.createEmptyArrowBatch may cause memory leak Key: SPARK-45814 URL: https://issues.apache.org/jira/browse/SPARK-45814 Project: Spark

[jira] [Updated] (SPARK-45811) XML: Refine docstring of from_xml

[ https://issues.apache.org/jira/browse/SPARK-45811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45811: --- Labels: pull-request-available (was: ) > XML: Refine docstring of from_xml >

[jira] [Created] (SPARK-45813) Return the observed metrics from commands

Takuya Ueshin created SPARK-45813: - Summary: Return the observed metrics from commands Key: SPARK-45813 URL: https://issues.apache.org/jira/browse/SPARK-45813 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45813) Return the observed metrics from commands

[ https://issues.apache.org/jira/browse/SPARK-45813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45813: --- Labels: pull-request-available (was: ) > Return the observed metrics from commands >

[jira] [Updated] (SPARK-45815) Provide an interface for Streaming sources to add _metadata columns

[ https://issues.apache.org/jira/browse/SPARK-45815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaohua Zhao updated SPARK-45815: Component/s: Structured Streaming > Provide an interface for Streaming sources to add _metadata

[jira] [Updated] (SPARK-45815) Provide an interface for Streaming sources to add _metadata columns

[ https://issues.apache.org/jira/browse/SPARK-45815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45815: --- Labels: pull-request-available (was: ) > Provide an interface for Streaming sources to add

[jira] [Assigned] (SPARK-45804) Add spark.ui.threadDump.flamegraphEnabled config to switch flame graph on/off

[ https://issues.apache.org/jira/browse/SPARK-45804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45804: - Assignee: Kent Yao > Add spark.ui.threadDump.flamegraphEnabled config to switch flame

[jira] [Resolved] (SPARK-45812) Upgrade Pandas to 2.1.2

[ https://issues.apache.org/jira/browse/SPARK-45812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45812. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43689

[jira] [Updated] (SPARK-45013) Flaky Test with NPE: track allocated resources by taskId

[ https://issues.apache.org/jira/browse/SPARK-45013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45013: --- Labels: pull-request-available (was: ) > Flaky Test with NPE: track allocated resources by

[jira] [Resolved] (SPARK-34444) Pushdown scalar-subquery filter to FileSourceScan

[ https://issues.apache.org/jira/browse/SPARK-3?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You resolved SPARK-3. --- Fix Version/s: 4.0.0 Resolution: Fixed > Pushdown scalar-subquery filter to FileSourceScan >

[jira] [Updated] (SPARK-45814) ArrowConverters.createEmptyArrowBatch may cause memory leak

[ https://issues.apache.org/jira/browse/SPARK-45814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45814: --- Labels: pull-request-available (was: ) > ArrowConverters.createEmptyArrowBatch may cause

[jira] [Created] (SPARK-45815) Provide an interface for Streaming sources to add _metadata columns

Yaohua Zhao created SPARK-45815: --- Summary: Provide an interface for Streaming sources to add _metadata columns Key: SPARK-45815 URL: https://issues.apache.org/jira/browse/SPARK-45815 Project: Spark

[jira] [Updated] (SPARK-45815) Provide an interface for Streaming sources to add _metadata columns

[ https://issues.apache.org/jira/browse/SPARK-45815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaohua Zhao updated SPARK-45815: Description: Currently, only the native V1 file-based streaming source can read the `_metadata`

[jira] [Resolved] (SPARK-45804) Add spark.ui.threadDump.flamegraphEnabled config to switch flame graph on/off

[ https://issues.apache.org/jira/browse/SPARK-45804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45804. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43674

[jira] [Resolved] (SPARK-45013) Flaky Test with NPE: track allocated resources by taskId

[ https://issues.apache.org/jira/browse/SPARK-45013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45013. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43693

[jira] [Resolved] (SPARK-45775) Drop table skiped when CatalogV2Util loadTable meet unexpected Exception

[ https://issues.apache.org/jira/browse/SPARK-45775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] konwu resolved SPARK-45775. --- Resolution: Invalid > Drop table skiped when CatalogV2Util loadTable meet unexpected Exception >

[jira] [Updated] (SPARK-45816) Return null when overflowing during casting from timestamp to integers

[ https://issues.apache.org/jira/browse/SPARK-45816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45816: --- Labels: pull-request-available (was: ) > Return null when overflowing during casting from

[jira] [Assigned] (SPARK-45013) Flaky Test with NPE: track allocated resources by taskId

[ https://issues.apache.org/jira/browse/SPARK-45013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45013: - Assignee: Kent Yao > Flaky Test with NPE: track allocated resources by taskId >

[jira] [Created] (SPARK-45816) Return null when overflowing during casting from timestamp to integers

L. C. Hsieh created SPARK-45816: --- Summary: Return null when overflowing during casting from timestamp to integers Key: SPARK-45816 URL: https://issues.apache.org/jira/browse/SPARK-45816 Project: Spark

[jira] [Assigned] (SPARK-45556) Inconsistent status code between web page and REST API when exception is thrown

[ https://issues.apache.org/jira/browse/SPARK-45556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45556: -- Assignee: Apache Spark > Inconsistent status code between web page and REST API when

[jira] [Updated] (SPARK-45803) Remove the no longer used `RpcAbortException`.

[ https://issues.apache.org/jira/browse/SPARK-45803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45803: --- Labels: pull-request-available (was: ) > Remove the no longer used `RpcAbortException`. >

[jira] [Created] (SPARK-45803) Remove the no longer used `RpcAbortException`.

Yang Jie created SPARK-45803: Summary: Remove the no longer used `RpcAbortException`. Key: SPARK-45803 URL: https://issues.apache.org/jira/browse/SPARK-45803 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-45556) Inconsistent status code between web page and REST API when exception is thrown

[ https://issues.apache.org/jira/browse/SPARK-45556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45556: -- Assignee: (was: Apache Spark) > Inconsistent status code between web page and

[jira] [Assigned] (SPARK-45527) Task fraction resource request is not expected

[ https://issues.apache.org/jira/browse/SPARK-45527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45527: -- Assignee: (was: Apache Spark) > Task fraction resource request is not expected >

[jira] [Created] (SPARK-45802) Remove no longer needed Java majorVersion checks in `Platform`

Yang Jie created SPARK-45802: Summary: Remove no longer needed Java majorVersion checks in `Platform` Key: SPARK-45802 URL: https://issues.apache.org/jira/browse/SPARK-45802 Project: Spark

[jira] [Updated] (SPARK-45802) Remove no longer needed Java majorVersion checks in `Platform`

[ https://issues.apache.org/jira/browse/SPARK-45802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45802: --- Labels: pull-request-available (was: ) > Remove no longer needed Java majorVersion checks

[jira] [Assigned] (SPARK-45527) Task fraction resource request is not expected

[ https://issues.apache.org/jira/browse/SPARK-45527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45527: -- Assignee: Apache Spark > Task fraction resource request is not expected >

[jira] [Assigned] (SPARK-45798) Assert server-side session ID in Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-45798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45798: -- Assignee: Apache Spark > Assert server-side session ID in Spark Connect >

[jira] [Assigned] (SPARK-45798) Assert server-side session ID in Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-45798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45798: -- Assignee: (was: Apache Spark) > Assert server-side session ID in Spark Connect >

[jira] [Commented] (SPARK-45758) Introduce a mapper for hadoop compression codecs

[ https://issues.apache.org/jira/browse/SPARK-45758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17783169#comment-17783169 ] Jiaan Geng commented on SPARK-45758: Resolved by https://github.com/apache/spark/pull/43620 >

[jira] [Updated] (SPARK-45804) Add spark.ui.threadDump.flamegraphEnabled config to switch flame graph on/off

[ https://issues.apache.org/jira/browse/SPARK-45804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45804: --- Labels: pull-request-available (was: ) > Add spark.ui.threadDump.flamegraphEnabled config

[jira] [Resolved] (SPARK-45793) Improve the built-in compression codecs

[ https://issues.apache.org/jira/browse/SPARK-45793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-45793. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43659

[jira] [Created] (SPARK-45804) Add spark.ui.threadDump.flamegraphEnabled config to switch flame graph on/off

Kent Yao created SPARK-45804: Summary: Add spark.ui.threadDump.flamegraphEnabled config to switch flame graph on/off Key: SPARK-45804 URL: https://issues.apache.org/jira/browse/SPARK-45804 Project: Spark

[jira] [Created] (SPARK-45805) Eliminate magic numbers in withOrigin

Max Gekk created SPARK-45805: Summary: Eliminate magic numbers in withOrigin Key: SPARK-45805 URL: https://issues.apache.org/jira/browse/SPARK-45805 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-45805) Eliminate magic numbers in withOrigin

[ https://issues.apache.org/jira/browse/SPARK-45805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45805: --- Labels: pull-request-available (was: ) > Eliminate magic numbers in withOrigin >

[jira] [Resolved] (SPARK-45758) Introduce a mapper for hadoop compression codecs

[ https://issues.apache.org/jira/browse/SPARK-45758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-45758. Resolution: Resolved > Introduce a mapper for hadoop compression codecs >

[jira] [Updated] (SPARK-41635) GROUP BY ALL

[ https://issues.apache.org/jira/browse/SPARK-41635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-41635: --- Labels: pull-request-available (was: ) > GROUP BY ALL > > >

[jira] [Created] (SPARK-45806) GROUP BY ALL don't work in ansi mode

XiaozongCui created SPARK-45806: --- Summary: GROUP BY ALL don't work in ansi mode Key: SPARK-45806 URL: https://issues.apache.org/jira/browse/SPARK-45806 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-45807) DataSourceV2: Improve ViewCatalog API

Eduard Tudenhoefner created SPARK-45807: --- Summary: DataSourceV2: Improve ViewCatalog API Key: SPARK-45807 URL: https://issues.apache.org/jira/browse/SPARK-45807 Project: Spark Issue

[jira] [Created] (SPARK-45808) Improve error details for Spark Connect Client in Python

Martin Grund created SPARK-45808: Summary: Improve error details for Spark Connect Client in Python Key: SPARK-45808 URL: https://issues.apache.org/jira/browse/SPARK-45808 Project: Spark

[jira] [Updated] (SPARK-45808) Improve error details for Spark Connect Client in Python

[ https://issues.apache.org/jira/browse/SPARK-45808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45808: --- Labels: pull-request-available (was: ) > Improve error details for Spark Connect Client in

[jira] [Created] (SPARK-45809) Refine docstring of `lit`

Hyukjin Kwon created SPARK-45809: Summary: Refine docstring of `lit` Key: SPARK-45809 URL: https://issues.apache.org/jira/browse/SPARK-45809 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-45809) Refine docstring of `lit`

[ https://issues.apache.org/jira/browse/SPARK-45809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45809: --- Labels: pull-request-available (was: ) > Refine docstring of `lit` >

[jira] [Created] (SPARK-45810) Create API to stop consuming rows from the input table

Daniel created SPARK-45810: -- Summary: Create API to stop consuming rows from the input table Key: SPARK-45810 URL: https://issues.apache.org/jira/browse/SPARK-45810 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45186) XML: Refine docstring of schema_of_xml

[ https://issues.apache.org/jira/browse/SPARK-45186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-45186: - Summary: XML: Refine docstring of schema_of_xml (was: XML: Refine docstring of from_xml,

[jira] [Created] (SPARK-45811) XML: Refine docstring of from_xml

Hyukjin Kwon created SPARK-45811: Summary: XML: Refine docstring of from_xml Key: SPARK-45811 URL: https://issues.apache.org/jira/browse/SPARK-45811 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-45186) XML: Refine docstring of schema_of_xml

[ https://issues.apache.org/jira/browse/SPARK-45186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45186: --- Labels: pull-request-available (was: ) > XML: Refine docstring of schema_of_xml >

[jira] [Updated] (SPARK-45810) Create API to stop consuming rows from the input table

[ https://issues.apache.org/jira/browse/SPARK-45810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45810: --- Labels: pull-request-available (was: ) > Create API to stop consuming rows from the input

[jira] [Updated] (SPARK-45258) Refine docstring of `sum`

[ https://issues.apache.org/jira/browse/SPARK-45258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45258: --- Labels: pull-request-available (was: ) > Refine docstring of `sum` >

[jira] [Resolved] (SPARK-45805) Eliminate magic numbers in withOrigin

[ https://issues.apache.org/jira/browse/SPARK-45805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth resolved SPARK-45805. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43671

[jira] [Updated] (SPARK-45259) Refine docstring of `count`

[ https://issues.apache.org/jira/browse/SPARK-45259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45259: --- Labels: pull-request-available (was: ) > Refine docstring of `count` >

[jira] [Updated] (SPARK-45807) DataSourceV2: Improve ViewCatalog API

[ https://issues.apache.org/jira/browse/SPARK-45807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45807: --- Labels: pull-request-available (was: ) > DataSourceV2: Improve ViewCatalog API >

[jira] [Resolved] (SPARK-45801) Introduce two helper methods for `QueryTest` that accept the `Array` type `expectedAnswer`

[ https://issues.apache.org/jira/browse/SPARK-45801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-45801. -- Resolution: Won't Fix > Introduce two helper methods for `QueryTest` that accept the `Array` type >

[jira] [Updated] (SPARK-43402) FileSourceScanExec supports push down data filter with scalar subquery

[ https://issues.apache.org/jira/browse/SPARK-43402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43402: --- Labels: pull-request-available (was: ) > FileSourceScanExec supports push down data filter

[jira] [Assigned] (SPARK-45773) Refine docstring of `SparkSession.builder.config`

[ https://issues.apache.org/jira/browse/SPARK-45773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45773: Assignee: Allison Wang > Refine docstring of `SparkSession.builder.config` >

[jira] [Resolved] (SPARK-45773) Refine docstring of `SparkSession.builder.config`

[ https://issues.apache.org/jira/browse/SPARK-45773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45773. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43639

[jira] [Updated] (SPARK-45786) Inaccurate Decimal multiplication and division results

[ https://issues.apache.org/jira/browse/SPARK-45786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45786: --- Labels: pull-request-available (was: ) > Inaccurate Decimal multiplication and division

[jira] [Updated] (SPARK-45260) Refine docstring of count_distinct

[ https://issues.apache.org/jira/browse/SPARK-45260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45260: --- Labels: pull-request-available (was: ) > Refine docstring of count_distinct >

[jira] [Updated] (SPARK-45222) Refine docstring of `DataFrameReader.json`

[ https://issues.apache.org/jira/browse/SPARK-45222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45222: --- Labels: pull-request-available (was: ) > Refine docstring of `DataFrameReader.json` >

[jira] [Resolved] (SPARK-45803) Remove the no longer used `RpcAbortException`.

[ https://issues.apache.org/jira/browse/SPARK-45803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45803. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43673

[jira] [Assigned] (SPARK-45803) Remove the no longer used `RpcAbortException`.

[ https://issues.apache.org/jira/browse/SPARK-45803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45803: - Assignee: Yang Jie > Remove the no longer used `RpcAbortException`. >

[jira] [Updated] (SPARK-44728) Improve PySpark documentations

[ https://issues.apache.org/jira/browse/SPARK-44728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-44728: - Fix Version/s: (was: 4.0.0) > Improve PySpark documentations >

[jira] [Updated] (SPARK-45223) Refine docstring of `Column.when`

[ https://issues.apache.org/jira/browse/SPARK-45223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45223: --- Labels: pull-request-available (was: ) > Refine docstring of `Column.when` >

[jira] [Created] (SPARK-45812) Upgrade Pandas to 2.1.2

Haejoon Lee created SPARK-45812: --- Summary: Upgrade Pandas to 2.1.2 Key: SPARK-45812 URL: https://issues.apache.org/jira/browse/SPARK-45812 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-45812) Upgrade Pandas to 2.1.2

[ https://issues.apache.org/jira/browse/SPARK-45812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45812: --- Labels: pull-request-available (was: ) > Upgrade Pandas to 2.1.2 > ---