[
https://issues.apache.org/jira/browse/IGNITE-980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Veselovsky updated IGNITE-980:
-----------------------------------
Description:
Search for usages of
org.apache.ignite.internal.processors.hadoop.HadoopUtils#fileSystemForMrUser
and see if returned Fs should be closed.
Now the policy is straightforward: if we take the Fs with caching, we don't
close it. If we take it w/o caching, we close it.
We also may add closing of all the cached instances upon Ignite node top.
was:
FileSystem is taken as a local variable and used in the following methods.
Need to check if we should do that as try (FileSystem fs = ...) { ... }, of use
the Fs without closing.
1)
org.apache.ignite.internal.processors.hadoop.v2.HadoopV2JobResourceManager#cleanupStagingDirectory
2)
org.apache.ignite.internal.processors.hadoop.v2.HadoopV2JobResourceManager#processFiles
3)
org.apache.ignite.internal.processors.hadoop.v2.HadoopV2JobResourceManager#prepareJobEnvironment
4) org.apache.ignite.hadoop.fs.IgniteHadoopFileSystemCounterWriter#write
> Investigate if we should close() the FileSystems in MapRed task
> implementations.
> --------------------------------------------------------------------------------
>
> Key: IGNITE-980
> URL: https://issues.apache.org/jira/browse/IGNITE-980
> Project: Ignite
> Issue Type: Task
> Components: hadoop
> Affects Versions: sprint-5
> Reporter: Ivan Veselovsky
> Assignee: Ivan Veselovsky
> Priority: Minor
> Fix For: sprint-6
>
>
> Search for usages of
> org.apache.ignite.internal.processors.hadoop.HadoopUtils#fileSystemForMrUser
> and see if returned Fs should be closed.
> Now the policy is straightforward: if we take the Fs with caching, we don't
> close it. If we take it w/o caching, we close it.
> We also may add closing of all the cached instances upon Ignite node top.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
