[jira] [Assigned] (SPARK-39084) df.rdd.isEmpty() results in unexpected executor failure and JVM crash

[ https://issues.apache.org/jira/browse/SPARK-39084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39084: Assignee: (was: Apache Spark) > df.rdd.isEmpty() results in unexpected executor

[jira] [Assigned] (SPARK-39084) df.rdd.isEmpty() results in unexpected executor failure and JVM crash

[ https://issues.apache.org/jira/browse/SPARK-39084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39084: Assignee: Apache Spark > df.rdd.isEmpty() results in unexpected executor failure and JVM

[jira] [Commented] (SPARK-39084) df.rdd.isEmpty() results in unexpected executor failure and JVM crash

[ https://issues.apache.org/jira/browse/SPARK-39084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17530604#comment-17530604 ] Apache Spark commented on SPARK-39084: -- User 'sadikovi' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39083) Fix FsHistoryProvider race condition between update and clean app data

[ https://issues.apache.org/jira/browse/SPARK-39083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39083: Assignee: (was: Apache Spark) > Fix FsHistoryProvider race condition between update

[jira] [Assigned] (SPARK-39083) Fix FsHistoryProvider race condition between update and clean app data

[ https://issues.apache.org/jira/browse/SPARK-39083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39083: Assignee: Apache Spark > Fix FsHistoryProvider race condition between update and clean

[jira] [Commented] (SPARK-39083) Fix FsHistoryProvider race condition between update and clean app data

[ https://issues.apache.org/jira/browse/SPARK-39083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17530603#comment-17530603 ] Apache Spark commented on SPARK-39083: -- User 'tanvn' has created a pull request for this issue:

[jira] [Commented] (SPARK-25355) Support --proxy-user for Spark on K8s

[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17530602#comment-17530602 ] Shrikant commented on SPARK-25355: -- [~gaborgsomogyi] ConnectionRefusedExcpetion is not there, only

[jira] [Commented] (SPARK-39064) FileNotFoundException while reading from Kafka

[ https://issues.apache.org/jira/browse/SPARK-39064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17530601#comment-17530601 ] Sushmita chauhan commented on SPARK-39064: -- I tried a lot to resolve the issue but not resolved

[jira] [Updated] (SPARK-39084) df.rdd.isEmpty() results in unexpected executor failure and JVM crash

[ https://issues.apache.org/jira/browse/SPARK-39084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Sadikov updated SPARK-39084: - Description: It was discovered that a particular data distribution in a DataFrame with groupBy

[jira] [Commented] (SPARK-39084) df.rdd.isEmpty() results in unexpected executor failure and JVM crash

[ https://issues.apache.org/jira/browse/SPARK-39084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17530600#comment-17530600 ] Ivan Sadikov commented on SPARK-39084: -- I am going to open a PR to fix this shortly. >

[jira] [Created] (SPARK-39084) df.rdd.isEmpty() results in unexpected executor failure and JVM crash

Ivan Sadikov created SPARK-39084: Summary: df.rdd.isEmpty() results in unexpected executor failure and JVM crash Key: SPARK-39084 URL: https://issues.apache.org/jira/browse/SPARK-39084 Project: Spark

[jira] [Updated] (SPARK-39083) Fix FsHistoryProvider race condition between update and clean app data

[ https://issues.apache.org/jira/browse/SPARK-39083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vu Tan updated SPARK-39083: --- Description: After SPARK-29043, FsHistoryProvider will list the log info without waiting all

[jira] [Updated] (SPARK-39083) Fix FsHistoryProvider race condition between update and clean app data

[ https://issues.apache.org/jira/browse/SPARK-39083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vu Tan updated SPARK-39083: --- Description: After SPARK-29043, FsHistoryProvider will list the log info without waiting all

[jira] [Created] (SPARK-39083) Fix FsHistoryProvider race condition between update and clean app data

Vu Tan created SPARK-39083: -- Summary: Fix FsHistoryProvider race condition between update and clean app data Key: SPARK-39083 URL: https://issues.apache.org/jira/browse/SPARK-39083 Project: Spark

[jira] [Resolved] (SPARK-39082) Fix FsHistoryProvider race condition between update and clean app data

[ https://issues.apache.org/jira/browse/SPARK-39082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vu Tan resolved SPARK-39082. Resolution: Duplicate > Fix FsHistoryProvider race condition between update and clean app data >

[jira] [Updated] (SPARK-39082) Fix FsHistoryProvider race condition between update and clean app data

[ https://issues.apache.org/jira/browse/SPARK-39082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vu Tan updated SPARK-39082: --- Fix Version/s: (was: 3.3.0) (was: 3.2.1) > Fix FsHistoryProvider race condition

[jira] [Created] (SPARK-39082) Fix FsHistoryProvider race condition between update and clean app data

Vu Tan created SPARK-39082: -- Summary: Fix FsHistoryProvider race condition between update and clean app data Key: SPARK-39082 URL: https://issues.apache.org/jira/browse/SPARK-39082 Project: Spark

[jira] [Commented] (SPARK-37474) Migrate SparkR documentation to pkgdown

[ https://issues.apache.org/jira/browse/SPARK-37474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17530590#comment-17530590 ] Apache Spark commented on SPARK-37474: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-37474) Migrate SparkR documentation to pkgdown

[ https://issues.apache.org/jira/browse/SPARK-37474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17530591#comment-17530591 ] Apache Spark commented on SPARK-37474: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-39070) Pivoting a dataframe after two explodes raises an org.codehaus.commons.compiler.CompileException

[ https://issues.apache.org/jira/browse/SPARK-39070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17530589#comment-17530589 ] L. C. Hsieh commented on SPARK-39070: - Hmm, SPARK-34638 is a feature requirement but this is a bug

[jira] [Commented] (SPARK-39064) FileNotFoundException while reading from Kafka

[ https://issues.apache.org/jira/browse/SPARK-39064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17530585#comment-17530585 ] Hyukjin Kwon commented on SPARK-39064: -- Spark 2.4.x is EOL. Mind trying Spark 3.x and see if the

[jira] [Updated] (SPARK-39064) FileNotFoundException while reading from Kafka

[ https://issues.apache.org/jira/browse/SPARK-39064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39064: - Fix Version/s: (was: 2.4.8) > FileNotFoundException while reading from Kafka >

[jira] [Updated] (SPARK-39064) FileNotFoundException while reading from Kafka

[ https://issues.apache.org/jira/browse/SPARK-39064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39064: - Target Version/s: (was: 2.4.8) > FileNotFoundException while reading from Kafka >

[jira] [Updated] (SPARK-39064) FileNotFoundException while reading from Kafka

[ https://issues.apache.org/jira/browse/SPARK-39064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39064: - Priority: Major (was: Critical) > FileNotFoundException while reading from Kafka >

[jira] [Commented] (SPARK-39066) Run `./dev/test-dependencies.sh --replace-manifest ` use Maven 3.8.5 produce wrong result

[ https://issues.apache.org/jira/browse/SPARK-39066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17530584#comment-17530584 ] Hyukjin Kwon commented on SPARK-39066: -- You should better use `./build/mvn` to pin the version. we

[jira] [Commented] (SPARK-39070) Pivoting a dataframe after two explodes raises an org.codehaus.commons.compiler.CompileException

[ https://issues.apache.org/jira/browse/SPARK-39070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17530583#comment-17530583 ] Hyukjin Kwon commented on SPARK-39070: -- It's actually a duplicate of SPARK-34638. cc [~viirya] FYI

[jira] [Resolved] (SPARK-39070) Pivoting a dataframe after two explodes raises an org.codehaus.commons.compiler.CompileException

[ https://issues.apache.org/jira/browse/SPARK-39070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39070. -- Resolution: Duplicate > Pivoting a dataframe after two explodes raises an >

[jira] [Commented] (SPARK-39070) Pivoting a dataframe after two explodes raises an org.codehaus.commons.compiler.CompileException

[ https://issues.apache.org/jira/browse/SPARK-39070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17530582#comment-17530582 ] Hyukjin Kwon commented on SPARK-39070: -- I can't reproduce this in the current master branch, but I

[jira] (SPARK-39070) Pivoting a dataframe after two explodes raises an org.codehaus.commons.compiler.CompileException

[ https://issues.apache.org/jira/browse/SPARK-39070 ] Hyukjin Kwon deleted comment on SPARK-39070: -- was (Author: hyukjin.kwon): The error cannot be reproduced in Apache Spark. Maybe it's Databricks specific issue. You might have to better

[jira] [Commented] (SPARK-39070) Pivoting a dataframe after two explodes raises an org.codehaus.commons.compiler.CompileException

[ https://issues.apache.org/jira/browse/SPARK-39070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17530581#comment-17530581 ] Hyukjin Kwon commented on SPARK-39070: -- The error cannot be reproduced in Apache Spark. Maybe it's

[jira] [Resolved] (SPARK-39067) Upgrade scala-maven-plugin to 4.6.1

[ https://issues.apache.org/jira/browse/SPARK-39067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-39067. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36406

[jira] [Assigned] (SPARK-39067) Upgrade scala-maven-plugin to 4.6.1

[ https://issues.apache.org/jira/browse/SPARK-39067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-39067: Assignee: Yang Jie > Upgrade scala-maven-plugin to 4.6.1 >

[jira] [Updated] (SPARK-39067) Upgrade scala-maven-plugin to 4.6.1

[ https://issues.apache.org/jira/browse/SPARK-39067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-39067: - Priority: Trivial (was: Minor) > Upgrade scala-maven-plugin to 4.6.1 >

[jira] [Updated] (SPARK-38918) Nested column pruning should filter out attributes that do not belong to the current relation

[ https://issues.apache.org/jira/browse/SPARK-38918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-38918: Fix Version/s: 3.1.3 3.0.4 > Nested column pruning should filter out

[jira] [Commented] (SPARK-38659) PySpark ResourceWarning: unclosed socket

[ https://issues.apache.org/jira/browse/SPARK-38659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17530554#comment-17530554 ] Gergely Kalmár commented on SPARK-38659: Yes, our test suite is failing because of these

[jira] [Assigned] (SPARK-38729) Test the error class: FAILED_SET_ORIGINAL_PERMISSION_BACK

[ https://issues.apache.org/jira/browse/SPARK-38729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-38729: Assignee: panbingkun > Test the error class: FAILED_SET_ORIGINAL_PERMISSION_BACK >

[jira] [Resolved] (SPARK-38729) Test the error class: FAILED_SET_ORIGINAL_PERMISSION_BACK

[ https://issues.apache.org/jira/browse/SPARK-38729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-38729. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36409

[jira] [Resolved] (SPARK-38737) Test the error classes: INVALID_FIELD_NAME

[ https://issues.apache.org/jira/browse/SPARK-38737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-38737. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36404

[jira] [Updated] (SPARK-38737) Test the error classes: INVALID_FIELD_NAME

[ https://issues.apache.org/jira/browse/SPARK-38737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-38737: - Reporter: panbingkun (was: Max Gekk) > Test the error classes: INVALID_FIELD_NAME >

[jira] [Resolved] (SPARK-38700) Use error classes in the execution errors of save mode

[ https://issues.apache.org/jira/browse/SPARK-38700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-38700. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36350

[jira] [Assigned] (SPARK-38700) Use error classes in the execution errors of save mode

[ https://issues.apache.org/jira/browse/SPARK-38700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-38700: Assignee: panbingkun > Use error classes in the execution errors of save mode >