[
https://issues.apache.org/jira/browse/FLINK-22734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler reassigned FLINK-22734:
----------------------------------------
Assignee: Chesnay Schepler
> ExtractionUtils#getClassReader causes "open too many files" error
> -----------------------------------------------------------------
>
> Key: FLINK-22734
> URL: https://issues.apache.org/jira/browse/FLINK-22734
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
> Affects Versions: 1.12.2
> Reporter: zhiping lin
> Assignee: Chesnay Schepler
> Priority: Major
>
> when we get classReader in ExtractionUtils
>
> {code:java}
> private static ClassReader getClassReader(Class<?> cls) {
> final String className = cls.getName().replaceFirst("^.*\\.", "") +
> ".class";
> try {
> return new ClassReader(cls.getResourceAsStream(className));
>
> } catch (IOException e) {
> throw new IllegalStateException("Could not instantiate
> ClassReader.", e);
> }
> }
> {code}
> we open a inputStream by "cls.getResourceAsStream(className)" and set it as a
> construct param for ClassReader without any other steps to close it. Also
> ClassReader's construct method with only a outside inputStream, classReader
> won't close the inputStream while it finished read the file. This will leads
> to fd leak with "open too many files" error when we parse a lot independent
> sqls with different envs in one jvm(the case is that we want to extract the
> table/data lineage of our flink jobs).
> {code:java}
> lsof -p 8011 | wc -l
> 10534
> {code}
> After checked all files and I found they are opened when loaded the udf jars
> for each loop to build the env. Then I test this method with try-with-source
> and it works well. So I wonder if it is necessary to change a little bit just
> like this
>
> {code:java}
> private static ClassReader getClassReader(Class<?> cls) {
> final String className = cls.getName().replaceFirst("^.*\\.", "") +
> ".class";
> try (InputStream inputStream = cls.getResourceAsStream(className)) {
> return new ClassReader(inputStream);
> } catch (IOException e) {
> throw new IllegalStateException("Could not instantiate ClassReader.",
> e);
> }
> }
> {code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
