[jira] [Updated] (SPARK-43397) Log executor decommission duration

[ https://issues.apache.org/jira/browse/SPARK-43397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongwei Zhu updated SPARK-43397: - Description: Log executor decommission duration. > Log executor decommission duration >

[jira] [Created] (SPARK-43397) Log executor decommission duration

Zhongwei Zhu created SPARK-43397: Summary: Log executor decommission duration Key: SPARK-43397 URL: https://issues.apache.org/jira/browse/SPARK-43397 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-43381) Make 'transformStatCov' lazy

[ https://issues.apache.org/jira/browse/SPARK-43381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43381. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41055 [https://gi

[jira] [Assigned] (SPARK-43381) Make 'transformStatCov' lazy

[ https://issues.apache.org/jira/browse/SPARK-43381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43381: Assignee: Ruifeng Zheng > Make 'transformStatCov' lazy > > >

[jira] [Updated] (SPARK-43383) Add statistics rowCount for LocalRelation

[ https://issues.apache.org/jira/browse/SPARK-43383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43383: Fix Version/s: (was: 3.5.0) > Add statistics rowCount for LocalRelation >

[jira] [Resolved] (SPARK-43392) Sequence expression can overflow

[ https://issues.apache.org/jira/browse/SPARK-43392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jia Fan resolved SPARK-43392. - Resolution: Duplicate Duplicate with SPARK-43393 > Sequence expression can overflow > -

[jira] [Updated] (SPARK-42750) Support INSERT INTO by name

[ https://issues.apache.org/jira/browse/SPARK-42750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jia Fan updated SPARK-42750: Issue Type: New Feature (was: Bug) > Support INSERT INTO by name > --- > >

[jira] [Commented] (SPARK-42750) Support INSERT INTO by name

[ https://issues.apache.org/jira/browse/SPARK-42750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720272#comment-17720272 ] Jia Fan commented on SPARK-42750: - Please review https://github.com/apache/spark/pull/40

[jira] [Created] (SPARK-43396) Add config to control max ratio of decommissioning executors

Zhongwei Zhu created SPARK-43396: Summary: Add config to control max ratio of decommissioning executors Key: SPARK-43396 URL: https://issues.apache.org/jira/browse/SPARK-43396 Project: Spark

[jira] [Resolved] (SPARK-43329) driver and executors shared same Kubernetes PVC in Spark 3.4+

[ https://issues.apache.org/jira/browse/SPARK-43329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43329. --- Fix Version/s: 3.4.1 Assignee: Qian Sun Resolution: Fixed Thank you for repo

[jira] [Commented] (SPARK-39006) Show a directional error message for PVC Dynamic Allocation Failure

[ https://issues.apache.org/jira/browse/SPARK-39006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720266#comment-17720266 ] Dongjoon Hyun commented on SPARK-39006: --- This is reverted via [https://github.com

[jira] [Updated] (SPARK-43342) Revert SPARK-39006 Show a directional error message for executor PVC dynamic allocation failure

[ https://issues.apache.org/jira/browse/SPARK-43342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43342: -- Summary: Revert SPARK-39006 Show a directional error message for executor PVC dynamic allocati

[jira] [Assigned] (SPARK-43342) Spark in Kubernetes mode throws IllegalArgumentException when using static PVC

[ https://issues.apache.org/jira/browse/SPARK-43342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43342: - Assignee: Qian Sun > Spark in Kubernetes mode throws IllegalArgumentException when usin

[jira] [Resolved] (SPARK-43342) Spark in Kubernetes mode throws IllegalArgumentException when using static PVC

[ https://issues.apache.org/jira/browse/SPARK-43342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43342. --- Fix Version/s: 3.4.1 Resolution: Fixed Issue resolved by pull request 41069 [https://

[jira] [Commented] (SPARK-43361) Allow deserializing protobuf enum fields as integers

[ https://issues.apache.org/jira/browse/SPARK-43361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720264#comment-17720264 ] Hudson commented on SPARK-43361: User 'justaparth' has created a pull request for this i

[jira] [Updated] (SPARK-43361) Allow deserializing protobuf enum fields as integers

[ https://issues.apache.org/jira/browse/SPARK-43361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Upadhyay updated SPARK-43361: --- Description: When deserializing protobuf enum fields, the spark-protobuf library will deser

[jira] [Updated] (SPARK-43361) Allow deserializing protobuf enum fields as integers

[ https://issues.apache.org/jira/browse/SPARK-43361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Upadhyay updated SPARK-43361: --- Description: When deserializing protobuf enum fields, the spark-protobuf library will deser

[jira] [Resolved] (SPARK-43324) DataSource V2: Handle UPDATE commands for delta-based sources

[ https://issues.apache.org/jira/browse/SPARK-43324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43324. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41028 [https://

[jira] [Assigned] (SPARK-43324) DataSource V2: Handle UPDATE commands for delta-based sources

[ https://issues.apache.org/jira/browse/SPARK-43324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43324: - Assignee: Anton Okolnychyi > DataSource V2: Handle UPDATE commands for delta-based sour

[jira] [Commented] (SPARK-43351) Support Golang in Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-43351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720241#comment-17720241 ] BoYang commented on SPARK-43351: Yes, thanks Ruifeng for the comment and suggestion! Wou

[jira] [Comment Edited] (SPARK-43382) Read and write csv and json files. Archive files such as zip or gz are supported

[ https://issues.apache.org/jira/browse/SPARK-43382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720110#comment-17720110 ] melin edited comment on SPARK-43382 at 5/6/23 3:57 PM: --- There is a

[jira] [Comment Edited] (SPARK-43382) Read and write csv and json files. Archive files such as zip or gz are supported

[ https://issues.apache.org/jira/browse/SPARK-43382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720110#comment-17720110 ] melin edited comment on SPARK-43382 at 5/6/23 3:57 PM: --- There is a

[jira] [Resolved] (SPARK-43395) Exclude macOS tar extended metadata in make-distribution.sh

[ https://issues.apache.org/jira/browse/SPARK-43395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-43395. -- Fix Version/s: 3.3.3 3.4.1 3.5.0 3.2.4

[jira] [Assigned] (SPARK-43395) Exclude macOS tar extended metadata in make-distribution.sh

[ https://issues.apache.org/jira/browse/SPARK-43395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-43395: Assignee: Cheng Pan > Exclude macOS tar extended metadata in make-distribution.sh > -

[jira] [Assigned] (SPARK-38462) Use error classes in org.apache.spark.executor

[ https://issues.apache.org/jira/browse/SPARK-38462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-38462: Assignee: Bo Zhang > Use error classes in org.apache.spark.executor > ---

[jira] [Resolved] (SPARK-38462) Use error classes in org.apache.spark.executor

[ https://issues.apache.org/jira/browse/SPARK-38462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-38462. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41048 [https://github.com

[jira] [Commented] (SPARK-43310) Dataset.observe is ignored when writing to Kafka with batch query

[ https://issues.apache.org/jira/browse/SPARK-43310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720195#comment-17720195 ] Jia Fan commented on SPARK-43310: - The reason are kafka sink use `SaveIntoDataSourceComm

[jira] [Updated] (SPARK-43310) Dataset.observe is ignored when writing to Kafka with batch query

[ https://issues.apache.org/jira/browse/SPARK-43310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jia Fan updated SPARK-43310: Attachment: image-2023-05-06-18-08-41-335.png > Dataset.observe is ignored when writing to Kafka with batc

[jira] [Commented] (SPARK-43351) Support Golang in Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-43351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720184#comment-17720184 ] Ruifeng Zheng commented on SPARK-43351: --- Thanks for starting this work. But before

[jira] [Updated] (SPARK-43351) Support Golang in Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-43351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43351: -- Affects Version/s: 3.5.0 (was: 3.4.0) > Support Golang in Spark Con

[jira] [Resolved] (SPARK-43371) minimize duplication across layers for chmod

[ https://issues.apache.org/jira/browse/SPARK-43371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-43371. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 37 [https://github.

[jira] [Assigned] (SPARK-43371) minimize duplication across layers for chmod

[ https://issues.apache.org/jira/browse/SPARK-43371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-43371: --- Assignee: Yikun Jiang > minimize duplication across layers for chmod >

[jira] [Assigned] (SPARK-43387) Provide human readable name for error _LEGACY_ERROR_TEMP_1168

[ https://issues.apache.org/jira/browse/SPARK-43387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-43387: Assignee: Eric Ogren > Provide human readable name for error _LEGACY_ERROR_TEMP_1168 > --

[jira] [Commented] (SPARK-43387) Provide human readable name for error _LEGACY_ERROR_TEMP_1168

[ https://issues.apache.org/jira/browse/SPARK-43387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720175#comment-17720175 ] ASF GitHub Bot commented on SPARK-43387: User 'eogren' has created a pull reques

[jira] [Resolved] (SPARK-43387) Provide human readable name for error _LEGACY_ERROR_TEMP_1168

[ https://issues.apache.org/jira/browse/SPARK-43387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-43387. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41068 [https://github.com

[jira] [Updated] (SPARK-43387) Provide human readable name for error _LEGACY_ERROR_TEMP_1168

[ https://issues.apache.org/jira/browse/SPARK-43387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-43387: - Parent: SPARK-37935 Issue Type: Sub-task (was: Improvement) > Provide human readable name for e

[jira] [Commented] (SPARK-43043) Improve the performance of MapOutputTracker.updateMapOutput

[ https://issues.apache.org/jira/browse/SPARK-43043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720174#comment-17720174 ] ASF GitHub Bot commented on SPARK-43043: User 'jiangxb1987' has created a pull r

[jira] [Created] (SPARK-43395) Excluding macOS tar extended metadata in make-distribution.sh

Cheng Pan created SPARK-43395: - Summary: Excluding macOS tar extended metadata in make-distribution.sh Key: SPARK-43395 URL: https://issues.apache.org/jira/browse/SPARK-43395 Project: Spark Issu

[jira] [Updated] (SPARK-43395) Exclude macOS tar extended metadata in make-distribution.sh

[ https://issues.apache.org/jira/browse/SPARK-43395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-43395: -- Summary: Exclude macOS tar extended metadata in make-distribution.sh (was: Excluding macOS tar extend

[jira] [Updated] (SPARK-43394) Upgrade maven to 3.8.8

[ https://issues.apache.org/jira/browse/SPARK-43394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-43394: -- Priority: Trivial (was: Major) > Upgrade maven to 3.8.8 > -- > >

[jira] [Created] (SPARK-43394) Upgrade maven to 3.8.8

Cheng Pan created SPARK-43394: - Summary: Upgrade maven to 3.8.8 Key: SPARK-43394 URL: https://issues.apache.org/jira/browse/SPARK-43394 Project: Spark Issue Type: Improvement Components