[
https://issues.apache.org/jira/browse/SPARK-39988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-39988:
-------------------------------------
Assignee: Yang Jie
> LevelDBIterator not close after used in `RemoteBlockPushResolver`,
> `YarnShuffleService` and `ExternalShuffleBlockResolver`
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-39988
> URL: https://issues.apache.org/jira/browse/SPARK-39988
> Project: Spark
> Issue Type: Bug
> Components: Spark Core, YARN
> Affects Versions: 3.4.0
> Reporter: Yang Jie
> Assignee: Yang Jie
> Priority: Major
>
> For example:
> {code:java}
> @VisibleForTesting
> static ConcurrentMap<AppExecId, ExecutorShuffleInfo>
> reloadRegisteredExecutors(DB db)
> throws IOException {
> ConcurrentMap<AppExecId, ExecutorShuffleInfo> registeredExecutors =
> Maps.newConcurrentMap();
> if (db != null) {
> DBIterator itr = db.iterator();
> itr.seek(APP_KEY_PREFIX.getBytes(StandardCharsets.UTF_8));
> while (itr.hasNext()) {
> Map.Entry<byte[], byte[]> e = itr.next();
> String key = new String(e.getKey(), StandardCharsets.UTF_8);
> if (!key.startsWith(APP_KEY_PREFIX)) {
> break;
> }
> AppExecId id = parseDbAppExecKey(key);
> logger.info("Reloading registered executors: " + id.toString());
> ExecutorShuffleInfo shuffleInfo = mapper.readValue(e.getValue(),
> ExecutorShuffleInfo.class);
> registeredExecutors.put(id, shuffleInfo);
> }
> }
> return registeredExecutors;
> } {code}
> DBIterator in above code should call close after used, otherwise, there is no
> other place to close it
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
