[jira] [Updated] (SPARK-38584) Unify the data validation

[ https://issues.apache.org/jira/browse/SPARK-38584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-38584: - Description: 1, input vector validation is missing in most algorithms, when the input dataset

[jira] [Updated] (SPARK-38521) Throw Exception if overwriting hive partition table with dynamic and staticPartitionOverwriteMode

[ https://issues.apache.org/jira/browse/SPARK-38521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackey Lee updated SPARK-38521: --- Affects Version/s: 3.4.0 (was: 3.3.0) > Throw Exception if overwriting

[jira] [Updated] (SPARK-38521) Throw Exception if overwriting hive partition table with dynamic and staticPartitionOverwriteMode

[ https://issues.apache.org/jira/browse/SPARK-38521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackey Lee updated SPARK-38521: --- Fix Version/s: 3.4.0 (was: 3.3.0) > Throw Exception if overwriting hive

[jira] [Updated] (SPARK-38427) DataFilter pushed down with PartitionFilter for Orc

[ https://issues.apache.org/jira/browse/SPARK-38427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackey Lee updated SPARK-38427: --- Affects Version/s: 3.4.0 (was: 3.3.0) > DataFilter pushed down with

[jira] [Updated] (SPARK-38440) DataFilter pushed down with PartitionFilter fro Parquet V1 Datasource

[ https://issues.apache.org/jira/browse/SPARK-38440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackey Lee updated SPARK-38440: --- Affects Version/s: 3.4.0 (was: 3.3.0) > DataFilter pushed down with

[jira] [Updated] (SPARK-38433) Add Shell Code Style Check Action

[ https://issues.apache.org/jira/browse/SPARK-38433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackey Lee updated SPARK-38433: --- Affects Version/s: 3.4.0 (was: 3.3.0) > Add Shell Code Style Check

[jira] [Updated] (SPARK-37933) Limit push down for parquet datasource v2

[ https://issues.apache.org/jira/browse/SPARK-37933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackey Lee updated SPARK-37933: --- Affects Version/s: 3.4.0 (was: 3.3.0) > Limit push down for parquet

[jira] [Updated] (SPARK-38041) DataFilter pushed down with PartitionFilter

[ https://issues.apache.org/jira/browse/SPARK-38041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackey Lee updated SPARK-38041: --- Affects Version/s: 3.4.0 (was: 3.3.0) > DataFilter pushed down with

[jira] [Updated] (SPARK-37919) SQL UI shows accurate Metrics with stage retries

[ https://issues.apache.org/jira/browse/SPARK-37919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackey Lee updated SPARK-37919: --- Affects Version/s: 3.4.0 (was: 3.3.0) > SQL UI shows accurate Metrics

[jira] [Updated] (SPARK-37831) Add task partition id in metrics

[ https://issues.apache.org/jira/browse/SPARK-37831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackey Lee updated SPARK-37831: --- Affects Version/s: (was: 3.2.1) > Add task partition id in metrics >

[jira] [Resolved] (SPARK-38563) Upgrade to Py4J 0.10.9.5

[ https://issues.apache.org/jira/browse/SPARK-38563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38563. -- Fix Version/s: 3.3.0 3.2.2 Resolution: Fixed Issue resolved by pull

[jira] [Commented] (SPARK-38593) Incorporate numRowsDroppedByWatermark metric from SessionWindowStateStoreRestoreExec into StateOperatorProgress

[ https://issues.apache.org/jira/browse/SPARK-38593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508568#comment-17508568 ] Apache Spark commented on SPARK-38593: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Commented] (SPARK-38204) All state operators are at a risk of inconsistency between state partitioning and operator partitioning

[ https://issues.apache.org/jira/browse/SPARK-38204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508542#comment-17508542 ] Apache Spark commented on SPARK-38204: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Commented] (SPARK-38204) All state operators are at a risk of inconsistency between state partitioning and operator partitioning

[ https://issues.apache.org/jira/browse/SPARK-38204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508541#comment-17508541 ] Apache Spark commented on SPARK-38204: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Commented] (SPARK-38592) Column name contains back tick `

[ https://issues.apache.org/jira/browse/SPARK-38592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508536#comment-17508536 ] Hyukjin Kwon commented on SPARK-38592: -- [~JuzDDM] mind showing the fully self-contained reproducer?

[jira] [Updated] (SPARK-38592) Column name contains back tick `

[ https://issues.apache.org/jira/browse/SPARK-38592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38592: - Labels: (was: bulk-closed) > Column name contains back tick ` >

[jira] [Assigned] (SPARK-38563) Upgrade to Py4J 0.10.9.5

[ https://issues.apache.org/jira/browse/SPARK-38563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38563: Assignee: Hyukjin Kwon (was: Apache Spark) > Upgrade to Py4J 0.10.9.5 >

[jira] [Assigned] (SPARK-38563) Upgrade to Py4J 0.10.9.5

[ https://issues.apache.org/jira/browse/SPARK-38563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38563: Assignee: Apache Spark (was: Hyukjin Kwon) > Upgrade to Py4J 0.10.9.5 >

[jira] [Updated] (SPARK-38584) Unify the data validation

[ https://issues.apache.org/jira/browse/SPARK-38584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-38584: - Description: 1, input vector validation is missing in most algorithms, when the input dataset

[jira] [Reopened] (SPARK-38563) Upgrade to Py4J 0.10.9.5

[ https://issues.apache.org/jira/browse/SPARK-38563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-38563: -- > Upgrade to Py4J 0.10.9.5 > > > Key: SPARK-38563 >

[jira] [Commented] (SPARK-38563) Upgrade to Py4J 0.10.9.5

[ https://issues.apache.org/jira/browse/SPARK-38563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508524#comment-17508524 ] Apache Spark commented on SPARK-38563: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-38563) Upgrade to Py4J 0.10.9.5

[ https://issues.apache.org/jira/browse/SPARK-38563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38563: - Fix Version/s: (was: 3.2.2) > Upgrade to Py4J 0.10.9.5 > > >

[jira] [Updated] (SPARK-38563) Upgrade to Py4J 0.10.9.5

[ https://issues.apache.org/jira/browse/SPARK-38563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38563: - Summary: Upgrade to Py4J 0.10.9.5 (was: Upgrade to Py4J 0.10.9.4) > Upgrade to Py4J 0.10.9.5 >

[jira] [Commented] (SPARK-33236) Enable Push-based shuffle service to store state in NM level DB for work preserving restart

[ https://issues.apache.org/jira/browse/SPARK-33236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508501#comment-17508501 ] Apache Spark commented on SPARK-33236: -- User 'zhouyejoe' has created a pull request for this issue:

[jira] [Commented] (SPARK-33236) Enable Push-based shuffle service to store state in NM level DB for work preserving restart

[ https://issues.apache.org/jira/browse/SPARK-33236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508499#comment-17508499 ] Apache Spark commented on SPARK-33236: -- User 'zhouyejoe' has created a pull request for this issue:

[jira] [Commented] (SPARK-33236) Enable Push-based shuffle service to store state in NM level DB for work preserving restart

[ https://issues.apache.org/jira/browse/SPARK-33236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508498#comment-17508498 ] Ye Zhou commented on SPARK-33236: - WIP PR posted [https://github.com/apache/spark/pull/35906.]  >

[jira] [Commented] (SPARK-38563) Upgrade to Py4J 0.10.9.4

[ https://issues.apache.org/jira/browse/SPARK-38563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508492#comment-17508492 ] Dongjoon Hyun commented on SPARK-38563: --- This is reverted from `master` and `branch-3.3`. >

[jira] [Updated] (SPARK-38563) Upgrade to Py4J 0.10.9.4

[ https://issues.apache.org/jira/browse/SPARK-38563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38563: -- Fix Version/s: (was: 3.3.0) (was: 3.2.2) > Upgrade to Py4J

[jira] [Resolved] (SPARK-38563) Upgrade to Py4J 0.10.9.4

[ https://issues.apache.org/jira/browse/SPARK-38563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38563. --- Fix Version/s: 3.2.2 Assignee: Hyukjin Kwon Resolution: Fixed > Upgrade to

[jira] [Reopened] (SPARK-38563) Upgrade to Py4J 0.10.9.4

[ https://issues.apache.org/jira/browse/SPARK-38563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-38563: --- Assignee: (was: Hyukjin Kwon) > Upgrade to Py4J 0.10.9.4 > >

[jira] [Updated] (SPARK-38592) Column name contains back tick `

[ https://issues.apache.org/jira/browse/SPARK-38592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dennis Du updated SPARK-38592: -- Description: Try to modify the data frame to ensure column names have no special characters.

[jira] [Commented] (SPARK-38593) Incorporate numRowsDroppedByWatermark metric from SessionWindowStateStoreRestoreExec into StateOperatorProgress

[ https://issues.apache.org/jira/browse/SPARK-38593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508471#comment-17508471 ] Apache Spark commented on SPARK-38593: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38593) Incorporate numRowsDroppedByWatermark metric from SessionWindowStateStoreRestoreExec into StateOperatorProgress

[ https://issues.apache.org/jira/browse/SPARK-38593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38593: Assignee: Apache Spark > Incorporate numRowsDroppedByWatermark metric from >

[jira] [Assigned] (SPARK-38593) Incorporate numRowsDroppedByWatermark metric from SessionWindowStateStoreRestoreExec into StateOperatorProgress

[ https://issues.apache.org/jira/browse/SPARK-38593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38593: Assignee: Apache Spark > Incorporate numRowsDroppedByWatermark metric from >

[jira] [Assigned] (SPARK-38593) Incorporate numRowsDroppedByWatermark metric from SessionWindowStateStoreRestoreExec into StateOperatorProgress

[ https://issues.apache.org/jira/browse/SPARK-38593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38593: Assignee: (was: Apache Spark) > Incorporate numRowsDroppedByWatermark metric from >

[jira] [Commented] (SPARK-38593) Incorporate numRowsDroppedByWatermark metric from SessionWindowStateStoreRestoreExec into StateOperatorProgress

[ https://issues.apache.org/jira/browse/SPARK-38593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508470#comment-17508470 ] Apache Spark commented on SPARK-38593: -- User 'viirya' has created a pull request for this issue:

[jira] [Created] (SPARK-38593) Incorporate numRowsDroppedByWatermark metric from SessionWindowStateStoreRestoreExec into StateOperatorProgress

L. C. Hsieh created SPARK-38593: --- Summary: Incorporate numRowsDroppedByWatermark metric from SessionWindowStateStoreRestoreExec into StateOperatorProgress Key: SPARK-38593 URL:

[jira] [Resolved] (SPARK-37425) Inline type hints for python/pyspark/mllib/recommendation.py

[ https://issues.apache.org/jira/browse/SPARK-37425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37425. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-37425) Inline type hints for python/pyspark/mllib/recommendation.py

[ https://issues.apache.org/jira/browse/SPARK-37425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37425: -- Assignee: dch nguyen > Inline type hints for

[jira] [Commented] (SPARK-2489) Unsupported parquet datatype optional fixed_len_byte_array

[ https://issues.apache.org/jira/browse/SPARK-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508438#comment-17508438 ] Nicolas Luiz Ribeiro Veiga commented on SPARK-2489: --- Can we reopen this issue? I

[jira] [Updated] (SPARK-37244) Build and test on Python 3.10

[ https://issues.apache.org/jira/browse/SPARK-37244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37244: -- Labels: releasenotes (was: ) > Build and test on Python 3.10 > -

[jira] [Commented] (SPARK-38563) Upgrade to Py4J 0.10.9.4

[ https://issues.apache.org/jira/browse/SPARK-38563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508433#comment-17508433 ] Apache Spark commented on SPARK-38563: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-38563) Upgrade to Py4J 0.10.9.4

[ https://issues.apache.org/jira/browse/SPARK-38563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508429#comment-17508429 ] Apache Spark commented on SPARK-38563: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-38563) Upgrade to Py4J 0.10.9.4

[ https://issues.apache.org/jira/browse/SPARK-38563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508430#comment-17508430 ] Apache Spark commented on SPARK-38563: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-2489) Unsupported parquet datatype optional fixed_len_byte_array

[ https://issues.apache.org/jira/browse/SPARK-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508407#comment-17508407 ] Apache Spark commented on SPARK-2489: - User 'nicolaslrveiga' has created a pull request for this

[jira] [Commented] (SPARK-2489) Unsupported parquet datatype optional fixed_len_byte_array

[ https://issues.apache.org/jira/browse/SPARK-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508406#comment-17508406 ] Apache Spark commented on SPARK-2489: - User 'nicolaslrveiga' has created a pull request for this

[jira] [Commented] (SPARK-38194) Make memory overhead factor configurable

[ https://issues.apache.org/jira/browse/SPARK-38194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508405#comment-17508405 ] Dongjoon Hyun commented on SPARK-38194: --- This is reverted from branch-3.3 via

[jira] [Updated] (SPARK-38194) Make memory overhead factor configurable

[ https://issues.apache.org/jira/browse/SPARK-38194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38194: -- Summary: Make memory overhead factor configurable (was: Make Yarn memory overhead factor

[jira] [Updated] (SPARK-38194) Make Yarn memory overhead factor configurable

[ https://issues.apache.org/jira/browse/SPARK-38194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38194: -- Affects Version/s: 3.4.0 (was: 3.2.1) > Make Yarn memory overhead

[jira] [Updated] (SPARK-38194) Make memory overhead factor configurable

[ https://issues.apache.org/jira/browse/SPARK-38194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38194: -- Component/s: Kubernetes Mesos > Make memory overhead factor configurable >

[jira] [Updated] (SPARK-38194) Make Yarn memory overhead factor configurable

[ https://issues.apache.org/jira/browse/SPARK-38194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38194: -- Fix Version/s: 3.4.0 (was: 3.3.0) > Make Yarn memory overhead factor

[jira] [Commented] (SPARK-38194) Make Yarn memory overhead factor configurable

[ https://issues.apache.org/jira/browse/SPARK-38194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508370#comment-17508370 ] Apache Spark commented on SPARK-38194: -- User 'Kimahriman' has created a pull request for this

[jira] [Commented] (SPARK-38194) Make Yarn memory overhead factor configurable

[ https://issues.apache.org/jira/browse/SPARK-38194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508369#comment-17508369 ] Apache Spark commented on SPARK-38194: -- User 'Kimahriman' has created a pull request for this

[jira] [Commented] (SPARK-38194) Make Yarn memory overhead factor configurable

[ https://issues.apache.org/jira/browse/SPARK-38194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508358#comment-17508358 ] Apache Spark commented on SPARK-38194: -- User 'tgravescs' has created a pull request for this issue:

[jira] [Updated] (SPARK-38592) Column name contains back tick `

[ https://issues.apache.org/jira/browse/SPARK-38592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dennis Du updated SPARK-38592: -- Description: Try to modify the data frame to ensure column names have no special characters.

[jira] [Created] (SPARK-38592) Column name contains back tick `

Dennis Du created SPARK-38592: - Summary: Column name contains back tick ` Key: SPARK-38592 URL: https://issues.apache.org/jira/browse/SPARK-38592 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-35066) Spark 3.1.1 is slower than 3.0.2 by 4-5 times

[ https://issues.apache.org/jira/browse/SPARK-35066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated SPARK-35066: Description: Hi, The following snippet code runs 4-5 times slower when it's used in Apache Spark

[jira] [Updated] (SPARK-35066) Spark 3.1.1 is slower than 3.0.2 by 4-5 times

[ https://issues.apache.org/jira/browse/SPARK-35066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated SPARK-35066: Description: Hi, The following snippet code runs 4-5 times slower when it's used in Apache Spark

[jira] [Updated] (SPARK-35066) Spark 3.1.1 is slower than 3.0.2 by 4-5 times

[ https://issues.apache.org/jira/browse/SPARK-35066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maziyar PANAHI updated SPARK-35066: --- Description: Hi, The following snippet code runs 4-5 times slower when it's used in Apache

[jira] [Updated] (SPARK-35066) Spark 3.1.1 is slower than 3.0.2 by 4-5 times

[ https://issues.apache.org/jira/browse/SPARK-35066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maziyar PANAHI updated SPARK-35066: --- Description: Hi, The following snippet code runs 4-5 times slower when it's used in Apache

[jira] [Updated] (SPARK-35066) Spark 3.1.1 is slower than 3.0.2 by 4-5 times

[ https://issues.apache.org/jira/browse/SPARK-35066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maziyar PANAHI updated SPARK-35066: --- Description: Hi, The following snippet code runs 4-5 times slower when it's used in Apache

[jira] [Updated] (SPARK-35066) Spark 3.1.1 is slower than 3.0.2 by 4-5 times

[ https://issues.apache.org/jira/browse/SPARK-35066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maziyar PANAHI updated SPARK-35066: --- Attachment: image-2022-03-17-17-19-34-906.png > Spark 3.1.1 is slower than 3.0.2 by 4-5

[jira] [Updated] (SPARK-35066) Spark 3.1.1 is slower than 3.0.2 by 4-5 times

[ https://issues.apache.org/jira/browse/SPARK-35066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maziyar PANAHI updated SPARK-35066: --- Attachment: image-2022-03-17-17-19-11-655.png > Spark 3.1.1 is slower than 3.0.2 by 4-5

[jira] [Updated] (SPARK-35066) Spark 3.1.1 is slower than 3.0.2 by 4-5 times

[ https://issues.apache.org/jira/browse/SPARK-35066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maziyar PANAHI updated SPARK-35066: --- Attachment: Screenshot 2021-04-08 at 15.13.19-1.png > Spark 3.1.1 is slower than 3.0.2 by

[jira] [Updated] (SPARK-35066) Spark 3.1.1 is slower than 3.0.2 by 4-5 times

[ https://issues.apache.org/jira/browse/SPARK-35066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maziyar PANAHI updated SPARK-35066: --- Attachment: image-2022-03-17-17-18-36-793.png > Spark 3.1.1 is slower than 3.0.2 by 4-5

[jira] [Commented] (SPARK-38577) Interval types are not truncated to the expected endField when creating a DataFrame via Duration

[ https://issues.apache.org/jira/browse/SPARK-38577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508269#comment-17508269 ] Robert Joseph Evans commented on SPARK-38577: - This is especially problematic because it is

[jira] [Assigned] (SPARK-38591) Add flatMapSortedGroups to KeyValueGroupedDataset

[ https://issues.apache.org/jira/browse/SPARK-38591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38591: Assignee: (was: Apache Spark) > Add flatMapSortedGroups to KeyValueGroupedDataset >

[jira] [Assigned] (SPARK-38591) Add flatMapSortedGroups to KeyValueGroupedDataset

[ https://issues.apache.org/jira/browse/SPARK-38591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38591: Assignee: Apache Spark > Add flatMapSortedGroups to KeyValueGroupedDataset >

[jira] [Commented] (SPARK-38591) Add flatMapSortedGroups to KeyValueGroupedDataset

[ https://issues.apache.org/jira/browse/SPARK-38591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508265#comment-17508265 ] Apache Spark commented on SPARK-38591: -- User 'EnricoMi' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38591) Add flatMapSortedGroups to KeyValueGroupedDataset

[ https://issues.apache.org/jira/browse/SPARK-38591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38591: Assignee: Apache Spark > Add flatMapSortedGroups to KeyValueGroupedDataset >

[jira] [Created] (SPARK-38591) Add flatMapSortedGroups to KeyValueGroupedDataset

Enrico Minack created SPARK-38591: - Summary: Add flatMapSortedGroups to KeyValueGroupedDataset Key: SPARK-38591 URL: https://issues.apache.org/jira/browse/SPARK-38591 Project: Spark Issue

[jira] [Commented] (SPARK-38544) Upgrade log4j2 to 2.17.2

[ https://issues.apache.org/jira/browse/SPARK-38544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508195#comment-17508195 ] Apache Spark commented on SPARK-38544: -- User 'jackylee-ch' has created a pull request for this

[jira] [Assigned] (SPARK-38590) New SQL function: try_to_binary

[ https://issues.apache.org/jira/browse/SPARK-38590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38590: Assignee: Apache Spark (was: Gengliang Wang) > New SQL function: try_to_binary >

[jira] [Commented] (SPARK-38590) New SQL function: try_to_binary

[ https://issues.apache.org/jira/browse/SPARK-38590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508158#comment-17508158 ] Apache Spark commented on SPARK-38590: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-38590) New SQL function: try_to_binary

[ https://issues.apache.org/jira/browse/SPARK-38590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38590: Assignee: Gengliang Wang (was: Apache Spark) > New SQL function: try_to_binary >

[jira] [Created] (SPARK-38590) New SQL function: try_to_binary

Gengliang Wang created SPARK-38590: -- Summary: New SQL function: try_to_binary Key: SPARK-38590 URL: https://issues.apache.org/jira/browse/SPARK-38590 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-38589) New SQL function: try_avg

[ https://issues.apache.org/jira/browse/SPARK-38589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38589: Assignee: Apache Spark (was: Gengliang Wang) > New SQL function: try_avg >

[jira] [Assigned] (SPARK-38589) New SQL function: try_avg

[ https://issues.apache.org/jira/browse/SPARK-38589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38589: Assignee: Gengliang Wang (was: Apache Spark) > New SQL function: try_avg >

[jira] [Commented] (SPARK-38589) New SQL function: try_avg

[ https://issues.apache.org/jira/browse/SPARK-38589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508156#comment-17508156 ] Apache Spark commented on SPARK-38589: -- User 'gengliangwang' has created a pull request for this

[jira] [Created] (SPARK-38589) New SQL function: try_avg

Gengliang Wang created SPARK-38589: -- Summary: New SQL function: try_avg Key: SPARK-38589 URL: https://issues.apache.org/jira/browse/SPARK-38589 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-38587) Validating new location for rename command should use formatted names

[ https://issues.apache.org/jira/browse/SPARK-38587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508135#comment-17508135 ] Apache Spark commented on SPARK-38587: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38587) Validating new location for rename command should use formatted names

[ https://issues.apache.org/jira/browse/SPARK-38587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38587: Assignee: (was: Apache Spark) > Validating new location for rename command should

[jira] [Commented] (SPARK-38587) Validating new location for rename command should use formatted names

[ https://issues.apache.org/jira/browse/SPARK-38587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508134#comment-17508134 ] Apache Spark commented on SPARK-38587: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38587) Validating new location for rename command should use formatted names

[ https://issues.apache.org/jira/browse/SPARK-38587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38587: Assignee: Apache Spark > Validating new location for rename command should use formatted

[jira] [Commented] (SPARK-38575) Duduplicate branch specification in GitHub Actions workflow

[ https://issues.apache.org/jira/browse/SPARK-38575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508128#comment-17508128 ] Apache Spark commented on SPARK-38575: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-38588) Validate input dataset of LinearSVC

[ https://issues.apache.org/jira/browse/SPARK-38588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38588: Assignee: (was: Apache Spark) > Validate input dataset of LinearSVC >

[jira] [Commented] (SPARK-38588) Validate input dataset of LinearSVC

[ https://issues.apache.org/jira/browse/SPARK-38588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508118#comment-17508118 ] Apache Spark commented on SPARK-38588: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-38588) Validate input dataset of LinearSVC

[ https://issues.apache.org/jira/browse/SPARK-38588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38588: Assignee: Apache Spark > Validate input dataset of LinearSVC >

[jira] [Commented] (SPARK-38588) Validate input dataset of LinearSVC

[ https://issues.apache.org/jira/browse/SPARK-38588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508119#comment-17508119 ] Apache Spark commented on SPARK-38588: -- User 'zhengruifeng' has created a pull request for this

[jira] [Created] (SPARK-38588) Validate input dataset of LinearSVC

zhengruifeng created SPARK-38588: Summary: Validate input dataset of LinearSVC Key: SPARK-38588 URL: https://issues.apache.org/jira/browse/SPARK-38588 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-38587) Validating new location for rename command should use formatted names

Kent Yao created SPARK-38587: Summary: Validating new location for rename command should use formatted names Key: SPARK-38587 URL: https://issues.apache.org/jira/browse/SPARK-38587 Project: Spark

[jira] [Commented] (SPARK-38575) Duduplicate branch specification in GitHub Actions workflow

[ https://issues.apache.org/jira/browse/SPARK-38575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508096#comment-17508096 ] Apache Spark commented on SPARK-38575: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Resolved] (SPARK-38586) Trigger notifying workflow in branch-3.3 and other future branches

[ https://issues.apache.org/jira/browse/SPARK-38586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38586. -- Fix Version/s: 3.3.0 Assignee: Hyukjin Kwon Resolution: Fixed Fixed in

[jira] [Assigned] (SPARK-38586) Trigger notifying workflow in branch-3.3 and other future branches

[ https://issues.apache.org/jira/browse/SPARK-38586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38586: Assignee: (was: Apache Spark) > Trigger notifying workflow in branch-3.3 and other

[jira] [Assigned] (SPARK-38586) Trigger notifying workflow in branch-3.3 and other future branches

[ https://issues.apache.org/jira/browse/SPARK-38586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38586: Assignee: Apache Spark > Trigger notifying workflow in branch-3.3 and other future

[jira] [Commented] (SPARK-38586) Trigger notifying workflow in branch-3.3 and other future branches

[ https://issues.apache.org/jira/browse/SPARK-38586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508087#comment-17508087 ] Apache Spark commented on SPARK-38586: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Created] (SPARK-38586) Trigger notifying workflow in branch-3.3 and other future branches

Hyukjin Kwon created SPARK-38586: Summary: Trigger notifying workflow in branch-3.3 and other future branches Key: SPARK-38586 URL: https://issues.apache.org/jira/browse/SPARK-38586 Project: Spark

[jira] [Commented] (SPARK-38575) Duduplicate branch specification in GitHub Actions workflow

[ https://issues.apache.org/jira/browse/SPARK-38575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508085#comment-17508085 ] Apache Spark commented on SPARK-38575: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-38566) Revert the parser changes for DEFAULT column support

[ https://issues.apache.org/jira/browse/SPARK-38566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-38566: Assignee: Max Gekk > Revert the parser changes for DEFAULT column support >

[jira] [Resolved] (SPARK-38566) Revert the parser changes for DEFAULT column support

[ https://issues.apache.org/jira/browse/SPARK-38566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-38566. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35885

  1   2   >