[jira] [Created] (SPARK-49042) CodeGenerator: Error calculating stats of compiled class. java.lang.UnsupportedOperationException: empty.max

2024-07-29 Thread Arnaud Nauwynck (Jira)
Arnaud Nauwynck created SPARK-49042: --- Summary: CodeGenerator: Error calculating stats of compiled class. java.lang.UnsupportedOperationException: empty.max Key: SPARK-49042 URL: https://issues.apache.org/jira/br

[jira] [Updated] (SPARK-48575) spark.history.fs.update.interval calling too many directory pollings when spark log dir contains many sparkEvent apps

2024-06-14 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-48575: Description: In the case of a sparkLog dir containing "lot" of spark eventLogs sub-dirs (

[jira] [Updated] (SPARK-48575) spark.history.fs.update.interval calling too many directory pollings when spark log dir contains many sparkEvent apps

2024-06-09 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-48575: Attachment: EventLogFileReaders.patch > spark.history.fs.update.interval calling too many

[jira] [Updated] (SPARK-48575) spark.history.fs.update.interval calling too many directory pollings when spark log dir contains many sparkEvent apps

2024-06-09 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-48575: Attachment: (was: EventLogFileReaders.patch) > spark.history.fs.update.interval callin

[jira] [Updated] (SPARK-48575) spark.history.fs.update.interval calling too many directory pollings when spark log dir contains many sparkEvent apps

2024-06-08 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-48575: Attachment: EventLogFileReaders.patch > spark.history.fs.update.interval calling too many

[jira] [Updated] (SPARK-48575) spark.history.fs.update.interval calling too many directory pollings when spark log dir contains many sparkEvent apps

2024-06-08 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-48575: Description: In the case of a sparkLog dir containing "lot" of spark eventLogs sub-dirs (

[jira] [Updated] (SPARK-48575) spark.history.fs.update.interval calling too many directory pollings when spark log dir contains many sparkEvent apps

2024-06-08 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-48575: Description: In the case of a sparkLog dir containing "lot" of spark eventLogs sub-dirs (

[jira] [Created] (SPARK-48575) spark.history.fs.update.interval calling too many directory pollings when spark log dir contains many sparkEvent apps

2024-06-08 Thread Arnaud Nauwynck (Jira)
Arnaud Nauwynck created SPARK-48575: --- Summary: spark.history.fs.update.interval calling too many directory pollings when spark log dir contains many sparkEvent apps Key: SPARK-48575 URL: https://issues.apache.o

[jira] [Updated] (SPARK-46659) Add customizable TaskScheduling param, to avoid randomly choosing executor for tasks, and downscale on low micro-batches activity

2024-01-16 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-46659: Description: When using dynamicAllocation (but not spark.decommission.enabled=true) with a

[jira] [Updated] (SPARK-46661) Add customizable property spark.dynamicAllocation.lastExecutorIdleTimeout for last remaining executor, defaulting to spark.dynamicAllocation.executorIdleTimeout

2024-01-10 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-46661: Description: when using dynamicAllocation, the parameter "spark.dynamicAllocation.executo

[jira] [Created] (SPARK-46661) Add customizable property spark.dynamicAllocation.lastExecutorIdleTimeout for last remaining executor, defaulting to spark.dynamicAllocation.executorIdleTimeout

2024-01-10 Thread Arnaud Nauwynck (Jira)
Arnaud Nauwynck created SPARK-46661: --- Summary: Add customizable property spark.dynamicAllocation.lastExecutorIdleTimeout for last remaining executor, defaulting to spark.dynamicAllocation.executorIdleTimeout Key: SPARK-46661

[jira] [Updated] (SPARK-46659) Add customizable TaskScheduling param, to avoid randomly choosing executor for tasks, and downscale on low micro-batches activity

2024-01-10 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-46659: Description: When using dynamicAllocation (but not spark.decommission.enabled=true) with a

[jira] [Updated] (SPARK-46659) Add customizable TaskScheduling param, to avoid randomly choosing executor for tasks, and downscale on low micro-batches activity

2024-01-10 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-46659: Description: When using dynamicAllocation (but not spark.decommission.enabled=true) with a

[jira] [Updated] (SPARK-46659) Add customizable TaskScheduling param, to avoid randomly choosing executor for tasks, and downscale on low micro-batches activity

2024-01-10 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-46659: Description: When using dynamicAllocation (but not spark.decommission.enabled=true) with a

[jira] [Created] (SPARK-46659) Add customizable TaskScheduling param, to avoid randomly choosing executor for tasks, and downscale on low micro-batches activity

2024-01-10 Thread Arnaud Nauwynck (Jira)
Arnaud Nauwynck created SPARK-46659: --- Summary: Add customizable TaskScheduling param, to avoid randomly choosing executor for tasks, and downscale on low micro-batches activity Key: SPARK-46659 URL: https://issu

[jira] [Created] (SPARK-46429) avoid duplicate Classes and Resources in classpath of SPARK_HOME/jars/*.jar

2023-12-16 Thread Arnaud Nauwynck (Jira)
Arnaud Nauwynck created SPARK-46429: --- Summary: avoid duplicate Classes and Resources in classpath of SPARK_HOME/jars/*.jar Key: SPARK-46429 URL: https://issues.apache.org/jira/browse/SPARK-46429 Pro

[jira] [Updated] (SPARK-45897) java.lang.NoClassDefFoundError: javax/servlet/Servlet incompatibilities upgrading springboot-dependencies 2.7 to 3.* (package javax.* -> jakarta.*)

2023-11-12 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-45897: Description: updating springboot-dependencies 2.7 to 3.1 breaks spark with springboot com

[jira] [Updated] (SPARK-45897) java.lang.NoClassDefFoundError: javax/servlet/Servlet incompatibilities upgrading springboot-dependencies 2.7 to 3.* (package javax.* -> jakarta.*)

2023-11-12 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-45897: Attachment: test-springboot-spark.zip > java.lang.NoClassDefFoundError: javax/servlet/Serv

[jira] [Updated] (SPARK-45897) java.lang.NoClassDefFoundError: javax/servlet/Servlet incompatibilities upgrading springboot-dependencies 2.7 to 3.* (package javax.* -> jakarta.*)

2023-11-12 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-45897: Description: updating springboot-dependencies 2.7 to 3.1 breaks spark with springboot com

[jira] [Created] (SPARK-45897) java.lang.NoClassDefFoundError: javax/servlet/Servlet incompatibilities upgrading springboot-dependencies 2.7 to 3.* (package javax.* -> jakarta.*)

2023-11-12 Thread Arnaud Nauwynck (Jira)
Arnaud Nauwynck created SPARK-45897: --- Summary: java.lang.NoClassDefFoundError: javax/servlet/Servlet incompatibilities upgrading springboot-dependencies 2.7 to 3.* (package javax.* -> jakarta.*) Key: SPARK-45897

[jira] [Updated] (SPARK-40642) wrong doc on memory tuning regarding String object memory size, changed since version>=9

2022-10-03 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-40642: Description: The documentation is wrong regarding memory consumption of java.lang.String h

[jira] [Updated] (SPARK-40642) wrong doc on memory tuning regarding String object memory size, changed since version>=9

2022-10-03 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-40642: Description: The documentation is wrong regarding memory consumption of java.lang.String h

[jira] [Created] (SPARK-40642) wrong doc on memory tuning regarding String object memory size, changed since version>=9

2022-10-03 Thread Arnaud Nauwynck (Jira)
Arnaud Nauwynck created SPARK-40642: --- Summary: wrong doc on memory tuning regarding String object memory size, changed since version>=9 Key: SPARK-40642 URL: https://issues.apache.org/jira/browse/SPARK-40642

[jira] [Commented] (SPARK-33164) SPIP: add SQL support to "SELECT * (EXCEPT someColumn) FROM .." equivalent to DataSet.dropColumn(someColumn)

2020-10-18 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17216174#comment-17216174 ] Arnaud Nauwynck commented on SPARK-33164: - Notice that there is also the featur

[jira] [Comment Edited] (SPARK-33164) SPIP: add SQL support to "SELECT * (EXCEPT someColumn) FROM .." equivalent to DataSet.dropColumn(someColumn)

2020-10-15 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17215016#comment-17215016 ] Arnaud Nauwynck edited comment on SPARK-33164 at 10/15/20, 9:35 PM: --

[jira] [Commented] (SPARK-33164) SPIP: add SQL support to "SELECT * (EXCEPT someColumn) FROM .." equivalent to DataSet.dropColumn(someColumn)

2020-10-15 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17215016#comment-17215016 ] Arnaud Nauwynck commented on SPARK-33164: - some links of interrest in source cod

[jira] [Comment Edited] (SPARK-33164) SPIP: add SQL support to "SELECT * (EXCEPT someColumn) FROM .." equivalent to DataSet.dropColumn(someColumn)

2020-10-15 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17215016#comment-17215016 ] Arnaud Nauwynck edited comment on SPARK-33164 at 10/15/20, 9:24 PM: --

[jira] [Updated] (SPARK-33164) SPIP: add SQL support to "SELECT * (EXCEPT someColumn) FROM .." equivalent to DataSet.dropColumn(someColumn)

2020-10-15 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-33164: Description: *Q1.* What are you trying to do? Articulate your objectives using absolutely

[jira] [Updated] (SPARK-33164) SPIP: add SQL support to "SELECT * (EXCEPT someColumn) FROM .." equivalent to DataSet.dropColumn(someColumn)

2020-10-15 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-33164: Description: *Q1.* What are you trying to do? Articulate your objectives using absolutely

[jira] [Updated] (SPARK-33164) SPIP: add SQL support to "SELECT * (EXCEPT someColumn) FROM .." equivalent to DataSet.dropColumn(someColumn)

2020-10-15 Thread Arnaud Nauwynck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnaud Nauwynck updated SPARK-33164: Description: *Q1.* What are you trying to do? Articulate your objectives using absolutely

[jira] [Created] (SPARK-33164) SPIP: add SQL support to "SELECT * (EXCEPT someColumn) FROM .." equivalent to DataSet.dropColumn(someColumn)

2020-10-15 Thread Arnaud Nauwynck (Jira)
Arnaud Nauwynck created SPARK-33164: --- Summary: SPIP: add SQL support to "SELECT * (EXCEPT someColumn) FROM .." equivalent to DataSet.dropColumn(someColumn) Key: SPARK-33164 URL: https://issues.apache.org/jira/br