[
https://issues.apache.org/jira/browse/PARQUET-783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Allman updated PARQUET-783:
-----------------------------------
Affects Version/s: (was: 1.9.1)
> H2SeekableInputStream does not close its underlying FSDataInputStream,
> leading to connection leaks
> --------------------------------------------------------------------------------------------------
>
> Key: PARQUET-783
> URL: https://issues.apache.org/jira/browse/PARQUET-783
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
> Affects Versions: 1.9.0
> Reporter: Michael Allman
> Assignee: Michael Allman
> Priority: Critical
> Fix For: 1.10.0
>
>
> {{ParquetFileReader}} opens a {{SeekableInputStream}} to read a footer. In
> the process, it opens a new {{FSDataInputStream}} and wraps it. However,
> {{H2SeekableInputStream}} does not override the {{close}} method. Therefore,
> when {{ParquetFileReader}} closes it, the underlying {{FSDataInputStream}} is
> not closed. As a result, these stale connections can exhaust a clusters' data
> nodes' connection resources and lead to mysterious HDFS read failures in HDFS
> clients, e.g.
> {noformat}
> org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block:
> BP-905337612-172.16.70.103-1444328960665:blk_1720536852_646811517
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
