[jira] [Updated] (HIVE-21005) LLAP: Reading more stripes per-split leaks ZlibCodecs

2018-12-11 Thread Gopal V (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HIVE-21005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gopal V updated HIVE-21005:
---
   Resolution: Fixed
Fix Version/s: 4.0.0
   Status: Resolved  (was: Patch Available)

Pushed to master, thanks [~ndembla]

> LLAP: Reading more stripes per-split leaks ZlibCodecs
> -
>
> Key: HIVE-21005
> URL: https://issues.apache.org/jira/browse/HIVE-21005
> Project: Hive
>  Issue Type: Bug
>  Components: llap
>Reporter: Gopal V
>Assignee: Nita Dembla
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21005.patch
>
>
> OrcEncodedDataReader - calls ensureDataReader in a loop, overwriting itself
> {code}
> for (int stripeIxMod = 0; stripeIxMod < stripeRgs.length; ++stripeIxMod) {
> 
> // 6.2. Ensure we have stripe metadata. We might have read it before 
> for RG filtering.
> if (stripeMetadatas != null) {
>   stripeMetadata = stripeMetadatas.get(stripeIxMod);
> } else {
> ...
>   ensureDataReader();
> ...
> }
> {code}
> {code}
>   private void ensureDataReader() throws IOException {
> ...
> stripeReader = orcReader.encodedReader(
> fileKey, dw, dw, useObjectPools ? POOL_FACTORY : null, trace, 
> useCodecPool, cacheTag);
> {code}
> creates new encodedReader without closing previous stripe's encoded reader.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-21005) LLAP: Reading more stripes per-split leaks ZlibCodecs

2018-12-10 Thread Gopal V (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HIVE-21005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gopal V updated HIVE-21005:
---
Status: Patch Available  (was: Open)

> LLAP: Reading more stripes per-split leaks ZlibCodecs
> -
>
> Key: HIVE-21005
> URL: https://issues.apache.org/jira/browse/HIVE-21005
> Project: Hive
>  Issue Type: Bug
>  Components: llap
>Reporter: Gopal V
>Assignee: Nita Dembla
>Priority: Major
> Attachments: HIVE-21005.patch
>
>
> OrcEncodedDataReader - calls ensureDataReader in a loop, overwriting itself
> {code}
> for (int stripeIxMod = 0; stripeIxMod < stripeRgs.length; ++stripeIxMod) {
> 
> // 6.2. Ensure we have stripe metadata. We might have read it before 
> for RG filtering.
> if (stripeMetadatas != null) {
>   stripeMetadata = stripeMetadatas.get(stripeIxMod);
> } else {
> ...
>   ensureDataReader();
> ...
> }
> {code}
> {code}
>   private void ensureDataReader() throws IOException {
> ...
> stripeReader = orcReader.encodedReader(
> fileKey, dw, dw, useObjectPools ? POOL_FACTORY : null, trace, 
> useCodecPool, cacheTag);
> {code}
> creates new encodedReader without closing previous stripe's encoded reader.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-21005) LLAP: Reading more stripes per-split leaks ZlibCodecs

2018-12-06 Thread Nita Dembla (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HIVE-21005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nita Dembla updated HIVE-21005:
---
Attachment: HIVE-21005.patch

> LLAP: Reading more stripes per-split leaks ZlibCodecs
> -
>
> Key: HIVE-21005
> URL: https://issues.apache.org/jira/browse/HIVE-21005
> Project: Hive
>  Issue Type: Bug
>  Components: llap
>Reporter: Gopal V
>Assignee: Nita Dembla
>Priority: Major
> Attachments: HIVE-21005.patch
>
>
> OrcEncodedDataReader - calls ensureDataReader in a loop, overwriting itself
> {code}
> for (int stripeIxMod = 0; stripeIxMod < stripeRgs.length; ++stripeIxMod) {
> 
> // 6.2. Ensure we have stripe metadata. We might have read it before 
> for RG filtering.
> if (stripeMetadatas != null) {
>   stripeMetadata = stripeMetadatas.get(stripeIxMod);
> } else {
> ...
>   ensureDataReader();
> ...
> }
> {code}
> {code}
>   private void ensureDataReader() throws IOException {
> ...
> stripeReader = orcReader.encodedReader(
> fileKey, dw, dw, useObjectPools ? POOL_FACTORY : null, trace, 
> useCodecPool, cacheTag);
> {code}
> creates new encodedReader without closing previous stripe's encoded reader.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-21005) LLAP: Reading more stripes per-split leaks ZlibCodecs

2018-12-04 Thread Gopal V (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HIVE-21005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gopal V updated HIVE-21005:
---
Component/s: llap

> LLAP: Reading more stripes per-split leaks ZlibCodecs
> -
>
> Key: HIVE-21005
> URL: https://issues.apache.org/jira/browse/HIVE-21005
> Project: Hive
>  Issue Type: Bug
>  Components: llap
>Reporter: Gopal V
>Priority: Major
>
> OrcEncodedDataReader - calls ensureDataReader in a loop, overwriting itself
> {code}
> for (int stripeIxMod = 0; stripeIxMod < stripeRgs.length; ++stripeIxMod) {
> 
> // 6.2. Ensure we have stripe metadata. We might have read it before 
> for RG filtering.
> if (stripeMetadatas != null) {
>   stripeMetadata = stripeMetadatas.get(stripeIxMod);
> } else {
> ...
>   ensureDataReader();
> ...
> }
> {code}
> {code}
>   private void ensureDataReader() throws IOException {
> ...
> stripeReader = orcReader.encodedReader(
> fileKey, dw, dw, useObjectPools ? POOL_FACTORY : null, trace, 
> useCodecPool, cacheTag);
> {code}
> creates new encodedReader without closing previous stripe's encoded reader.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)