[jira] [Updated] (HIVE-22583) LLAP cache always misses with non-vectorized serde readers such as OpenCSV
[ https://issues.apache.org/jira/browse/HIVE-22583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ádám Szita updated HIVE-22583: -- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Committed to master. Thanks for the review Slim! > LLAP cache always misses with non-vectorized serde readers such as OpenCSV > -- > > Key: HIVE-22583 > URL: https://issues.apache.org/jira/browse/HIVE-22583 > Project: Hive > Issue Type: Bug > Components: llap >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22583.0.patch, HIVE-22583.1.patch, > HIVE-22583.2.patch, HIVE-22583.3.patch > > > Although after the first read LLAP cache stores data of tables that are not > using the LazySimple serde, the stored data is then never used in the future > subsequent queries, causing a full cache miss and re-read each time. > Problem is rooted in SerdeEncodedDataReader#cacheFileData is not taking care > of creating an entry for the root/struct column of the table. The only cases > this is taken care of are when a vectorized reader is used _(e.g. > LazySimpleSerde's LazySimpleDeserializeRead)_, where > SerdeEncodedDataReader#processAsyncCacheData takes care of this. > This can be reproduced by either using a custom serde, like OpenCSV or using > LazySimpleSerde, but turning off _hive.llap.io.encode.vector.serde.enabled_. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22583) LLAP cache always misses with non-vectorized serde readers such as OpenCSV
[ https://issues.apache.org/jira/browse/HIVE-22583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ádám Szita updated HIVE-22583: -- Attachment: HIVE-22583.3.patch > LLAP cache always misses with non-vectorized serde readers such as OpenCSV > -- > > Key: HIVE-22583 > URL: https://issues.apache.org/jira/browse/HIVE-22583 > Project: Hive > Issue Type: Bug > Components: llap >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Attachments: HIVE-22583.0.patch, HIVE-22583.1.patch, > HIVE-22583.2.patch, HIVE-22583.3.patch > > > Although after the first read LLAP cache stores data of tables that are not > using the LazySimple serde, the stored data is then never used in the future > subsequent queries, causing a full cache miss and re-read each time. > Problem is rooted in SerdeEncodedDataReader#cacheFileData is not taking care > of creating an entry for the root/struct column of the table. The only cases > this is taken care of are when a vectorized reader is used _(e.g. > LazySimpleSerde's LazySimpleDeserializeRead)_, where > SerdeEncodedDataReader#processAsyncCacheData takes care of this. > This can be reproduced by either using a custom serde, like OpenCSV or using > LazySimpleSerde, but turning off _hive.llap.io.encode.vector.serde.enabled_. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22583) LLAP cache always misses with non-vectorized serde readers such as OpenCSV
[ https://issues.apache.org/jira/browse/HIVE-22583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ádám Szita updated HIVE-22583: -- Attachment: HIVE-22583.2.patch > LLAP cache always misses with non-vectorized serde readers such as OpenCSV > -- > > Key: HIVE-22583 > URL: https://issues.apache.org/jira/browse/HIVE-22583 > Project: Hive > Issue Type: Bug > Components: llap >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Attachments: HIVE-22583.0.patch, HIVE-22583.1.patch, > HIVE-22583.2.patch > > > Although after the first read LLAP cache stores data of tables that are not > using the LazySimple serde, the stored data is then never used in the future > subsequent queries, causing a full cache miss and re-read each time. > Problem is rooted in SerdeEncodedDataReader#cacheFileData is not taking care > of creating an entry for the root/struct column of the table. The only cases > this is taken care of are when a vectorized reader is used _(e.g. > LazySimpleSerde's LazySimpleDeserializeRead)_, where > SerdeEncodedDataReader#processAsyncCacheData takes care of this. > This can be reproduced by either using a custom serde, like OpenCSV or using > LazySimpleSerde, but turning off _hive.llap.io.encode.vector.serde.enabled_. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22583) LLAP cache always misses with non-vectorized serde readers such as OpenCSV
[ https://issues.apache.org/jira/browse/HIVE-22583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ádám Szita updated HIVE-22583: -- Attachment: HIVE-22583.1.patch > LLAP cache always misses with non-vectorized serde readers such as OpenCSV > -- > > Key: HIVE-22583 > URL: https://issues.apache.org/jira/browse/HIVE-22583 > Project: Hive > Issue Type: Bug > Components: llap >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Attachments: HIVE-22583.0.patch, HIVE-22583.1.patch > > > Although after the first read LLAP cache stores data of tables that are not > using the LazySimple serde, the stored data is then never used in the future > subsequent queries, causing a full cache miss and re-read each time. > Problem is rooted in SerdeEncodedDataReader#cacheFileData is not taking care > of creating an entry for the root/struct column of the table. The only cases > this is taken care of are when a vectorized reader is used _(e.g. > LazySimpleSerde's LazySimpleDeserializeRead)_, where > SerdeEncodedDataReader#processAsyncCacheData takes care of this. > This can be reproduced by either using a custom serde, like OpenCSV or using > LazySimpleSerde, but turning off _hive.llap.io.encode.vector.serde.enabled_. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22583) LLAP cache always misses with non-vectorized serde readers such as OpenCSV
[ https://issues.apache.org/jira/browse/HIVE-22583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ádám Szita updated HIVE-22583: -- Attachment: HIVE-22583.0.patch > LLAP cache always misses with non-vectorized serde readers such as OpenCSV > -- > > Key: HIVE-22583 > URL: https://issues.apache.org/jira/browse/HIVE-22583 > Project: Hive > Issue Type: Bug > Components: llap >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Attachments: HIVE-22583.0.patch > > > Although after the first read LLAP cache stores data of tables that are not > using the LazySimple serde, the stored data is then never used in the future > subsequent queries, causing a full cache miss and re-read each time. > Problem is rooted in SerdeEncodedDataReader#cacheFileData is not taking care > of creating an entry for the root/struct column of the table. The only cases > this is taken care of are when a vectorized reader is used _(e.g. > LazySimpleSerde's LazySimpleDeserializeRead)_, where > SerdeEncodedDataReader#processAsyncCacheData takes care of this. > This can be reproduced by either using a custom serde, like OpenCSV or using > LazySimpleSerde, but turning off _hive.llap.io.encode.vector.serde.enabled_. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22583) LLAP cache always misses with non-vectorized serde readers such as OpenCSV
[ https://issues.apache.org/jira/browse/HIVE-22583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ádám Szita updated HIVE-22583: -- Status: Patch Available (was: Open) > LLAP cache always misses with non-vectorized serde readers such as OpenCSV > -- > > Key: HIVE-22583 > URL: https://issues.apache.org/jira/browse/HIVE-22583 > Project: Hive > Issue Type: Bug > Components: llap >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Attachments: HIVE-22583.0.patch > > > Although after the first read LLAP cache stores data of tables that are not > using the LazySimple serde, the stored data is then never used in the future > subsequent queries, causing a full cache miss and re-read each time. > Problem is rooted in SerdeEncodedDataReader#cacheFileData is not taking care > of creating an entry for the root/struct column of the table. The only cases > this is taken care of are when a vectorized reader is used _(e.g. > LazySimpleSerde's LazySimpleDeserializeRead)_, where > SerdeEncodedDataReader#processAsyncCacheData takes care of this. > This can be reproduced by either using a custom serde, like OpenCSV or using > LazySimpleSerde, but turning off _hive.llap.io.encode.vector.serde.enabled_. -- This message was sent by Atlassian Jira (v8.3.4#803005)
