Chandni Singh created SPARK-31689: ------------------------------------- Summary: ShuffleBlockFetchIterator keeps localBlocks in its memory even though it never uses it Key: SPARK-31689 URL: https://issues.apache.org/jira/browse/SPARK-31689 Project: Spark Issue Type: Bug Components: Shuffle Affects Versions: 2.4.5 Reporter: Chandni Singh
The {{localBlocks}} is created and used in the {{initialize}} method of ShuffleBlockFetchIterator but is never used after that. It can be local to the initialize method instead of being a field in the {{ShuffleBlockFetchIterator}} instance. It hold son to memory until iterator instance is alive which is unnecessary. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org