[jira] [Commented] (PARQUET-783) H2SeekableInputStream does not close its underlying FSDataInputStream, leading to connection leaks

2017-06-09 Thread Julien Le Dem (JIRA) 

[ 
https://issues.apache.org/jira/browse/PARQUET-783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16044715#comment-16044715
 ] 

Julien Le Dem commented on PARQUET-783:
---

Hi [~fuka], I created a jira ticket to make a 1.9.1 release: PARQUET-1027
We should link to it any JIRA we think should be added and get it started soon.


> H2SeekableInputStream does not close its underlying FSDataInputStream, 
> leading to connection leaks
> --
>
> Key: PARQUET-783
> URL: https://issues.apache.org/jira/browse/PARQUET-783
> Project: Parquet
>  Issue Type: Bug
>  Components: parquet-mr
>Affects Versions: 1.9.0
>Reporter: Michael Allman
>Assignee: Michael Allman
>Priority: Critical
> Fix For: 1.10.0
>
>
> {{ParquetFileReader}} opens a {{SeekableInputStream}} to read a footer. In 
> the process, it opens a new {{FSDataInputStream}} and wraps it. However, 
> {{H2SeekableInputStream}} does not override the {{close}} method. Therefore, 
> when {{ParquetFileReader}} closes it, the underlying {{FSDataInputStream}} is 
> not closed. As a result, these stale connections can exhaust a clusters' data 
> nodes' connection resources and lead to mysterious HDFS read failures in HDFS 
> clients, e.g.
> {noformat}
> org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: 
> BP-905337612-172.16.70.103-1444328960665:blk_1720536852_646811517
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (PARQUET-783) H2SeekableInputStream does not close its underlying FSDataInputStream, leading to connection leaks

2017-05-31 Thread Rohit Aggarwal (JIRA) 

[ 
https://issues.apache.org/jira/browse/PARQUET-783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16031811#comment-16031811
 ] 

Rohit Aggarwal commented on PARQUET-783:


We've been hit by this same issue as well. Any chance of 1.9.1 happening soon? 
Thanks! 

> H2SeekableInputStream does not close its underlying FSDataInputStream, 
> leading to connection leaks
> --
>
> Key: PARQUET-783
> URL: https://issues.apache.org/jira/browse/PARQUET-783
> Project: Parquet
>  Issue Type: Bug
>  Components: parquet-mr
>Affects Versions: 1.9.0
>Reporter: Michael Allman
>Assignee: Michael Allman
>Priority: Critical
> Fix For: 1.10.0
>
>
> {{ParquetFileReader}} opens a {{SeekableInputStream}} to read a footer. In 
> the process, it opens a new {{FSDataInputStream}} and wraps it. However, 
> {{H2SeekableInputStream}} does not override the {{close}} method. Therefore, 
> when {{ParquetFileReader}} closes it, the underlying {{FSDataInputStream}} is 
> not closed. As a result, these stale connections can exhaust a clusters' data 
> nodes' connection resources and lead to mysterious HDFS read failures in HDFS 
> clients, e.g.
> {noformat}
> org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: 
> BP-905337612-172.16.70.103-1444328960665:blk_1720536852_646811517
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (PARQUET-783) H2SeekableInputStream does not close its underlying FSDataInputStream, leading to connection leaks

2016-12-15 Thread Ryan Blue (JIRA) 

[ 
https://issues.apache.org/jira/browse/PARQUET-783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15752914#comment-15752914
 ] 

Ryan Blue commented on PARQUET-783:
---

Yeah, we can make that happen. In the future, could you bring this up on the 
mailing list or open an issue to release 1.9.1? Many of us just saw that this 
was resolved and not your latest comment to get it in a patch release. Thanks, 
[~michael]!

> H2SeekableInputStream does not close its underlying FSDataInputStream, 
> leading to connection leaks
> --
>
> Key: PARQUET-783
> URL: https://issues.apache.org/jira/browse/PARQUET-783
> Project: Parquet
>  Issue Type: Bug
>  Components: parquet-mr
>Affects Versions: 1.9.0
>Reporter: Michael Allman
>Assignee: Michael Allman
>Priority: Critical
> Fix For: 1.10.0
>
>
> {{ParquetFileReader}} opens a {{SeekableInputStream}} to read a footer. In 
> the process, it opens a new {{FSDataInputStream}} and wraps it. However, 
> {{H2SeekableInputStream}} does not override the {{close}} method. Therefore, 
> when {{ParquetFileReader}} closes it, the underlying {{FSDataInputStream}} is 
> not closed. As a result, these stale connections can exhaust a clusters' data 
> nodes' connection resources and lead to mysterious HDFS read failures in HDFS 
> clients, e.g.
> {noformat}
> org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: 
> BP-905337612-172.16.70.103-1444328960665:blk_1720536852_646811517
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PARQUET-783) H2SeekableInputStream does not close its underlying FSDataInputStream, leading to connection leaks

2016-12-05 Thread Michael Allman (JIRA) 

[ 
https://issues.apache.org/jira/browse/PARQUET-783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724105#comment-15724105
 ] 

Michael Allman commented on PARQUET-783:


Any chance we can get this in 1.9.1? This bug makes parquet 1.9 unusable for us.

> H2SeekableInputStream does not close its underlying FSDataInputStream, 
> leading to connection leaks
> --
>
> Key: PARQUET-783
> URL: https://issues.apache.org/jira/browse/PARQUET-783
> Project: Parquet
>  Issue Type: Bug
>  Components: parquet-mr
>Affects Versions: 1.9.0
>Reporter: Michael Allman
>Assignee: Michael Allman
>Priority: Critical
> Fix For: 1.10.0
>
>
> {{ParquetFileReader}} opens a {{SeekableInputStream}} to read a footer. In 
> the process, it opens a new {{FSDataInputStream}} and wraps it. However, 
> {{H2SeekableInputStream}} does not override the {{close}} method. Therefore, 
> when {{ParquetFileReader}} closes it, the underlying {{FSDataInputStream}} is 
> not closed. As a result, these stale connections can exhaust a clusters' data 
> nodes' connection resources and lead to mysterious HDFS read failures in HDFS 
> clients, e.g.
> {noformat}
> org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: 
> BP-905337612-172.16.70.103-1444328960665:blk_1720536852_646811517
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PARQUET-783) H2SeekableInputStream does not close its underlying FSDataInputStream, leading to connection leaks

2016-12-05 Thread Michael Allman (JIRA) 

[ 
https://issues.apache.org/jira/browse/PARQUET-783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15722771#comment-15722771
 ] 

Michael Allman commented on PARQUET-783:


Hi [~gszadovszky]. Thanks for the advice. I don't seem to be able to assign 
this ticket to myself. Maybe you can do that for me?

> H2SeekableInputStream does not close its underlying FSDataInputStream, 
> leading to connection leaks
> --
>
> Key: PARQUET-783
> URL: https://issues.apache.org/jira/browse/PARQUET-783
> Project: Parquet
>  Issue Type: Bug
>  Components: parquet-mr
>Affects Versions: 1.9.0, 1.9.1
>Reporter: Michael Allman
>Priority: Critical
>
> {{ParquetFileReader}} opens a {{SeekableInputStream}} to read a footer. In 
> the process, it opens a new {{FSDataInputStream}} and wraps it. However, 
> {{H2SeekableInputStream}} does not override the {{close}} method. Therefore, 
> when {{ParquetFileReader}} closes it, the underlying {{FSDataInputStream}} is 
> not closed. As a result, these stale connections can exhaust a clusters' data 
> nodes' connection resources and lead to mysterious HDFS read failures in HDFS 
> clients, e.g.
> {noformat}
> org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: 
> BP-905337612-172.16.70.103-1444328960665:blk_1720536852_646811517
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PARQUET-783) H2SeekableInputStream does not close its underlying FSDataInputStream, leading to connection leaks

2016-12-05 Thread Gabor Szadovszky (JIRA) 

[ 
https://issues.apache.org/jira/browse/PARQUET-783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15721918#comment-15721918
 ] 

Gabor Szadovszky commented on PARQUET-783:
--

Hi [~michael],
As you have linked the PR you should press the button "Submit Patch" so that 
the status of this jira will be highlighted for the committers.
I would also suggest assigning the JIRA for yourself.
Thanks a lot.

> H2SeekableInputStream does not close its underlying FSDataInputStream, 
> leading to connection leaks
> --
>
> Key: PARQUET-783
> URL: https://issues.apache.org/jira/browse/PARQUET-783
> Project: Parquet
>  Issue Type: Bug
>  Components: parquet-mr
>Affects Versions: 1.9.0, 1.9.1
>Reporter: Michael Allman
>Priority: Critical
>
> {{ParquetFileReader}} opens a {{SeekableInputStream}} to read a footer. In 
> the process, it opens a new {{FSDataInputStream}} and wraps it. However, 
> {{H2SeekableInputStream}} does not override the {{close}} method. Therefore, 
> when {{ParquetFileReader}} closes it, the underlying {{FSDataInputStream}} is 
> not closed. As a result, these stale connections can exhaust a clusters' data 
> nodes' connection resources and lead to mysterious HDFS read failures in HDFS 
> clients, e.g.
> {noformat}
> org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: 
> BP-905337612-172.16.70.103-1444328960665:blk_1720536852_646811517
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)