[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172055#comment-14172055 ] Saisai Shao commented on SPARK-3948: Hi Josh, thanks for your help. I don't think it's

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172062#comment-14172062 ] Josh Rosen commented on SPARK-3948: --- Hi [~jerryshao], To make sure that I understand,

[jira] [Created] (SPARK-3954) promote the speed of convert files to RDDS

宿荣全 created SPARK-3954: -- Summary: promote the speed of convert files to RDDS Key: SPARK-3954 URL: https://issues.apache.org/jira/browse/SPARK-3954 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172080#comment-14172080 ] Josh Rosen commented on SPARK-3948: --- I'm surprised that the lack of {{append}} didn't

[jira] [Created] (SPARK-3955) Different versions between jackson-mapper-asl and jackson-core-asl

Jongyoul Lee created SPARK-3955: --- Summary: Different versions between jackson-mapper-asl and jackson-core-asl Key: SPARK-3955 URL: https://issues.apache.org/jira/browse/SPARK-3955 Project: Spark

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172116#comment-14172116 ] Mridul Muralidharan commented on SPARK-3948: [~joshrosen] Assuming there are

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172117#comment-14172117 ] Saisai Shao commented on SPARK-3948: Hi Josh, I think for old code without append,

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172134#comment-14172134 ] Mridul Muralidharan commented on SPARK-3948: [~jerryshao] Just to clarify,

[jira] [Commented] (SPARK-3955) Different versions between jackson-mapper-asl and jackson-core-asl

[ https://issues.apache.org/jira/browse/SPARK-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172148#comment-14172148 ] Sean Owen commented on SPARK-3955: -- Looks like Jackson is managed to version 1.8.8 for

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172152#comment-14172152 ] Saisai Shao commented on SPARK-3948: Hi [~mridul], what I observed is that, after

[jira] [Closed] (SPARK-3898) History Web UI display incorrectly.

[ https://issues.apache.org/jira/browse/SPARK-3898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zzc closed SPARK-3898. -- Resolution: Not a Problem Target Version/s: 1.2.0 History Web UI display incorrectly.

[jira] [Commented] (SPARK-3954) promote the speed of convert files to RDDS

[ https://issues.apache.org/jira/browse/SPARK-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172167#comment-14172167 ] Apache Spark commented on SPARK-3954: - User 'surq' has created a pull request for this

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172225#comment-14172225 ] Mridul Muralidharan commented on SPARK-3948: That is weird, I tried a stripped

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172226#comment-14172226 ] Mridul Muralidharan commented on SPARK-3948: Note, t is just some file with a

[jira] [Commented] (SPARK-3939) NPE caused by SessionState.out not set in thriftserver2

[ https://issues.apache.org/jira/browse/SPARK-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172230#comment-14172230 ] Apache Spark commented on SPARK-3939: - User 'adrian-wang' has created a pull request

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172371#comment-14172371 ] Saisai Shao commented on SPARK-3948: Hi [~mridulm80], thanks a lot for your

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172387#comment-14172387 ] Praveen Seluka commented on SPARK-3174: --- [~andrewor] - Can you also comment on the

[jira] [Commented] (SPARK-2075) Anonymous classes are missing from Spark distribution

[ https://issues.apache.org/jira/browse/SPARK-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172394#comment-14172394 ] Iulian Dragos commented on SPARK-2075: -- The Scala compiler produces stable names for

[jira] [Commented] (SPARK-3629) Improvements to YARN doc

[ https://issues.apache.org/jira/browse/SPARK-3629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172416#comment-14172416 ] ssj commented on SPARK-3629: I have pushed an request on github about it.

[jira] [Commented] (SPARK-3629) Improvements to YARN doc

[ https://issues.apache.org/jira/browse/SPARK-3629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172444#comment-14172444 ] Apache Spark commented on SPARK-3629: - User 'ssjssh' has created a pull request for

[jira] [Commented] (SPARK-1209) SparkHadoopUtil should not use package org.apache.hadoop

[ https://issues.apache.org/jira/browse/SPARK-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172479#comment-14172479 ] Apache Spark commented on SPARK-1209: - User 'srowen' has created a pull request for

[jira] [Commented] (SPARK-2429) Hierarchical Implementation of KMeans

[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172486#comment-14172486 ] RJ Nowling commented on SPARK-2429: --- Great to know! I'm glad that isn't a bottleneck.

[jira] [Commented] (SPARK-3873) Scala style: check import ordering

[ https://issues.apache.org/jira/browse/SPARK-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172552#comment-14172552 ] Marcelo Vanzin commented on SPARK-3873: --- It doesn't seem like I can unassing bugs...

[jira] [Commented] (SPARK-1209) SparkHadoopUtil should not use package org.apache.hadoop

[ https://issues.apache.org/jira/browse/SPARK-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172571#comment-14172571 ] Sean Owen commented on SPARK-1209: -- ... and why wouldn't you, that's the title of the

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172582#comment-14172582 ] Josh Rosen commented on SPARK-3948: --- That kernel bug seems like it could explain why the

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172585#comment-14172585 ] Mridul Muralidharan commented on SPARK-3948: [~jerryshao] great work ! I

[jira] [Commented] (SPARK-3704) the types not match adding value form spark row to hive row in SparkSQLOperationManager

[ https://issues.apache.org/jira/browse/SPARK-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172645#comment-14172645 ] JIng Wang commented on SPARK-3704: -- Will this fix be merged to branch-v1.1 for the next

[jira] [Updated] (SPARK-3956) Python API for Distributed Matrix

[ https://issues.apache.org/jira/browse/SPARK-3956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-3956: -- Priority: Minor (was: Blocker) Python API for Distributed Matrix -

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172861#comment-14172861 ] Marcelo Vanzin commented on SPARK-3174: --- [~praveenseluka] I took a quick look at

[jira] [Commented] (SPARK-3936) Incorrect result in GraphX BytecodeUtils with closures + class/object methods

[ https://issues.apache.org/jira/browse/SPARK-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172867#comment-14172867 ] Apache Spark commented on SPARK-3936: - User 'jegonzal' has created a pull request for

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172895#comment-14172895 ] Josh Rosen commented on SPARK-3948: --- [~mridulm80], that explicit position-checking and

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172917#comment-14172917 ] Fi commented on SPARK-2883: --- This sounds great! Is this a scala only patch, or would it also

[jira] [Commented] (SPARK-3630) Identify cause of Kryo+Snappy PARSING_ERROR

[ https://issues.apache.org/jira/browse/SPARK-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172926#comment-14172926 ] Josh Rosen commented on SPARK-3630: --- I think that there may be multiple causes of these

[jira] [Updated] (SPARK-3630) Identify cause of Kryo+Snappy PARSING_ERROR

[ https://issues.apache.org/jira/browse/SPARK-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3630: -- Target Version/s: 1.1.1, 1.2.0 Affects Version/s: 1.2.0 Assignee: Josh Rosen

[jira] [Created] (SPARK-3959) SqlParser fails to parse literal -9223372036854775808 (Long.MinValue).

Kousuke Saruta created SPARK-3959: - Summary: SqlParser fails to parse literal -9223372036854775808 (Long.MinValue). Key: SPARK-3959 URL: https://issues.apache.org/jira/browse/SPARK-3959 Project:

[jira] [Created] (SPARK-3960) We cannot apply unary minus only to literal.

Kousuke Saruta created SPARK-3960: - Summary: We cannot apply unary minus only to literal. Key: SPARK-3960 URL: https://issues.apache.org/jira/browse/SPARK-3960 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-3960) We cannot apply unary minus only to literal.

[ https://issues.apache.org/jira/browse/SPARK-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3960: -- Description: Because of the wrong syntax definition, we cannot apply unary minus only to

[jira] [Updated] (SPARK-3960) We can apply unary minus only to literal.

[ https://issues.apache.org/jira/browse/SPARK-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3960: -- Summary: We can apply unary minus only to literal. (was: We cannot apply unary minus only to

[jira] [Commented] (SPARK-3958) Possible stream-corruption issues in TorrentBroadcast

[ https://issues.apache.org/jira/browse/SPARK-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172949#comment-14172949 ] Josh Rosen commented on SPARK-3958: --- Digging into this stacktrace in more detail:

[jira] [Commented] (SPARK-3960) We can apply unary minus only to literal.

[ https://issues.apache.org/jira/browse/SPARK-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172986#comment-14172986 ] Apache Spark commented on SPARK-3960: - User 'sarutak' has created a pull request for

[jira] [Commented] (SPARK-3937) Unsafe memory access inside of Snappy library

[ https://issues.apache.org/jira/browse/SPARK-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173025#comment-14173025 ] Josh Rosen commented on SPARK-3937: --- Another occurrence of this problem, running a

[jira] [Updated] (SPARK-3958) Possible stream-corruption issues in TorrentBroadcast

[ https://issues.apache.org/jira/browse/SPARK-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3958: -- Target Version/s: 1.2.0 (was: 1.1.1, 1.2.0) Possible stream-corruption issues in TorrentBroadcast

[jira] [Updated] (SPARK-3958) Possible stream-corruption issues in TorrentBroadcast

[ https://issues.apache.org/jira/browse/SPARK-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3958: -- Affects Version/s: (was: 1.1.0) Removing 1.1.0 as an affected version for now, since the stacktrace

[jira] [Commented] (SPARK-3958) Possible stream-corruption issues in TorrentBroadcast

[ https://issues.apache.org/jira/browse/SPARK-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173154#comment-14173154 ] Josh Rosen commented on SPARK-3958: --- I think that I can safely rule out problems in

[jira] [Commented] (SPARK-3955) Different versions between jackson-mapper-asl and jackson-core-asl

[ https://issues.apache.org/jira/browse/SPARK-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173166#comment-14173166 ] Apache Spark commented on SPARK-3955: - User 'jongyoul' has created a pull request for

[jira] [Created] (SPARK-3961) Python API for mllib.feature

Davies Liu created SPARK-3961: - Summary: Python API for mllib.feature Key: SPARK-3961 URL: https://issues.apache.org/jira/browse/SPARK-3961 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-3961) Python API for mllib.feature

[ https://issues.apache.org/jira/browse/SPARK-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173172#comment-14173172 ] Apache Spark commented on SPARK-3961: - User 'davies' has created a pull request for

[jira] [Commented] (SPARK-3958) Possible stream-corruption issues in TorrentBroadcast

[ https://issues.apache.org/jira/browse/SPARK-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173199#comment-14173199 ] Saisai Shao commented on SPARK-3958: Hi Josh, have you tried other compression like

[jira] [Commented] (SPARK-3958) Possible stream-corruption issues in TorrentBroadcast

[ https://issues.apache.org/jira/browse/SPARK-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173207#comment-14173207 ] Josh Rosen commented on SPARK-3958: --- Hi [~jerryshao], I don't have a reliable

[jira] [Updated] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-3720: -- Attachment: orc.diff This is the diff for orc file support. Because I also work on this item in

[jira] [Updated] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-2883: -- Attachment: orc.diff Just for completion, I also attach the diff here besides spark-3720. Spark

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173269#comment-14173269 ] Zhan Zhang commented on SPARK-2883: --- It is scala patch, and I am not very familiar with

[jira] [Commented] (SPARK-1209) SparkHadoop{MapRed,MapReduce}Util should not use package org.apache.hadoop

[ https://issues.apache.org/jira/browse/SPARK-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173284#comment-14173284 ] Patrick Wendell commented on SPARK-1209: Got it! Yeah so anything that is

[jira] [Updated] (SPARK-3954) Optimization to FileInputDStream

[ https://issues.apache.org/jira/browse/SPARK-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3954: --- Summary: Optimization to FileInputDStream (was: promote the speed of convert files to RDDS)

[jira] [Updated] (SPARK-3954) promote the speed of convert files to RDDS

[ https://issues.apache.org/jira/browse/SPARK-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3954: --- Component/s: (was: Input/Output) Streaming promote the speed of

[jira] [Commented] (SPARK-3951) Make the external-* jars fat jars

[ https://issues.apache.org/jira/browse/SPARK-3951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173289#comment-14173289 ] Patrick Wendell commented on SPARK-3951: I think the best solution here is

[jira] [Created] (SPARK-3962) Mark spark dependency as provided in external libraries

Patrick Wendell created SPARK-3962: -- Summary: Mark spark dependency as provided in external libraries Key: SPARK-3962 URL: https://issues.apache.org/jira/browse/SPARK-3962 Project: Spark

[jira] [Updated] (SPARK-3962) Mark spark dependency as provided in external libraries

[ https://issues.apache.org/jira/browse/SPARK-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3962: --- Description: Right now there is not an easy way for users to link against the external

[jira] [Resolved] (SPARK-1561) sbt/sbt assembly generates too many local files

[ https://issues.apache.org/jira/browse/SPARK-1561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1561. Resolution: Not a Problem Does not seem to be an issue any more - resolving per comment

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173300#comment-14173300 ] Patrick Wendell commented on SPARK-3431: [~srowen] - just wondering, is it trivial

[jira] [Created] (SPARK-3963) Support getting task-scoped properties from TaskContext

Patrick Wendell created SPARK-3963: -- Summary: Support getting task-scoped properties from TaskContext Key: SPARK-3963 URL: https://issues.apache.org/jira/browse/SPARK-3963 Project: Spark

[jira] [Updated] (SPARK-3963) Support getting task-scoped properties from TaskContext

[ https://issues.apache.org/jira/browse/SPARK-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3963: --- Description: This is a proposal for a minor feature. Given stabilization of the TaskContext

[jira] [Created] (SPARK-3964) Python API for Hypothesis testing

Davies Liu created SPARK-3964: - Summary: Python API for Hypothesis testing Key: SPARK-3964 URL: https://issues.apache.org/jira/browse/SPARK-3964 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-3965) Spark assembly for hadoop2 contains avro-mapred for hadoop1

David Jacot created SPARK-3965: -- Summary: Spark assembly for hadoop2 contains avro-mapred for hadoop1 Key: SPARK-3965 URL: https://issues.apache.org/jira/browse/SPARK-3965 Project: Spark Issue

[jira] [Created] (SPARK-3966) Fix nullabilities of Cast related to DateType.

Takuya Ueshin created SPARK-3966: Summary: Fix nullabilities of Cast related to DateType. Key: SPARK-3966 URL: https://issues.apache.org/jira/browse/SPARK-3966 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3966) Fix nullabilities of Cast related to DateType.

[ https://issues.apache.org/jira/browse/SPARK-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173400#comment-14173400 ] Apache Spark commented on SPARK-3966: - User 'ueshin' has created a pull request for

[jira] [Commented] (SPARK-3963) Support getting task-scoped properties from TaskContext

[ https://issues.apache.org/jira/browse/SPARK-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173404#comment-14173404 ] Patrick Wendell commented on SPARK-3963: [~rxin] and [~adav] I'd be interested in

[jira] [Updated] (SPARK-3963) Support getting task-scoped properties from TaskContext

[ https://issues.apache.org/jira/browse/SPARK-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3963: --- Description: This is a proposal for a minor feature. Given stabilization of the TaskContext

[jira] [Commented] (SPARK-3885) Provide mechanism to remove accumulators once they are no longer used

[ https://issues.apache.org/jira/browse/SPARK-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173407#comment-14173407 ] Nathan Kronenfeld commented on SPARK-3885: -- I tried reusing accumulators and