[
https://issues.apache.org/jira/browse/PARQUET-783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15752914#comment-15752914
]
Ryan Blue commented on PARQUET-783:
-----------------------------------
Yeah, we can make that happen. In the future, could you bring this up on the
mailing list or open an issue to release 1.9.1? Many of us just saw that this
was resolved and not your latest comment to get it in a patch release. Thanks,
[~michael]!
> H2SeekableInputStream does not close its underlying FSDataInputStream,
> leading to connection leaks
> --------------------------------------------------------------------------------------------------
>
> Key: PARQUET-783
> URL: https://issues.apache.org/jira/browse/PARQUET-783
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
> Affects Versions: 1.9.0
> Reporter: Michael Allman
> Assignee: Michael Allman
> Priority: Critical
> Fix For: 1.10.0
>
>
> {{ParquetFileReader}} opens a {{SeekableInputStream}} to read a footer. In
> the process, it opens a new {{FSDataInputStream}} and wraps it. However,
> {{H2SeekableInputStream}} does not override the {{close}} method. Therefore,
> when {{ParquetFileReader}} closes it, the underlying {{FSDataInputStream}} is
> not closed. As a result, these stale connections can exhaust a clusters' data
> nodes' connection resources and lead to mysterious HDFS read failures in HDFS
> clients, e.g.
> {noformat}
> org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block:
> BP-905337612-172.16.70.103-1444328960665:blk_1720536852_646811517
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
