[
https://issues.apache.org/jira/browse/SPARK-60571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-60571:
-----------------------------------
Assignee: Shujing Yang
> Set the task result size accumulator in place in TaskResultGetter
> -----------------------------------------------------------------
>
> Key: SPARK-60571
> URL: https://issues.apache.org/jira/browse/SPARK-60571
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 5.0.0
> Reporter: Shujing Yang
> Assignee: Shujing Yang
> Priority: Major
> Labels: pull-request-available
>
> When TaskResultGetter receives a task result, it sets the task result size on
> the RESULT_SIZE accumulator with map, which rebuilds the whole accumulator
> update sequence once per task result on the driver. The sequence can hold
> hundreds of entries when SQL metrics are involved, and each name check
> allocates a Some. The matching accumulator was already mutated in place, so
> the rebuilt sequence held the same objects. Walk the updates with foreach,
> set the matching LongAccumulator in place, and use Option.contains for the
> name check. No behavior change.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
