[jira] [Updated] (HIVE-9605) Remove parquet nested objects from wrapper writable objects
[ https://issues.apache.org/jira/browse/HIVE-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-9605: --- Resolution: Fixed Fix Version/s: parquet-branch Status: Resolved (was: Patch Available) Committed to branch! > Remove parquet nested objects from wrapper writable objects > --- > > Key: HIVE-9605 > URL: https://issues.apache.org/jira/browse/HIVE-9605 > Project: Hive > Issue Type: Sub-task >Affects Versions: 0.14.0 >Reporter: Sergio Peña >Assignee: Sergio Peña > Fix For: parquet-branch > > Attachments: HIVE-9605.3.patch, HIVE-9605.4.patch > > > Parquet nested types are using an extra wrapper object (ArrayWritable) as a > wrapper of map and list elements. This extra object is not needed and causing > unnecessary memory allocations. > An example of code is on HiveCollectionConverter.java: > {noformat} > public void end() { > parent.set(index, wrapList(new ArrayWritable( > Writable.class, list.toArray(new Writable[list.size()]; > } > {noformat} > This object is later unwrapped on AbstractParquetMapInspector, i.e.: > {noformat} > final Writable[] mapContainer = ((ArrayWritable) data).get(); > final Writable[] mapArray = ((ArrayWritable) mapContainer[0]).get(); > for (final Writable obj : mapArray) { > ... > } > {noformat} > We should get rid of this wrapper object to save time and memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9605) Remove parquet nested objects from wrapper writable objects
[ https://issues.apache.org/jira/browse/HIVE-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-9605: -- Status: Patch Available (was: Open) > Remove parquet nested objects from wrapper writable objects > --- > > Key: HIVE-9605 > URL: https://issues.apache.org/jira/browse/HIVE-9605 > Project: Hive > Issue Type: Sub-task >Affects Versions: 0.14.0 >Reporter: Sergio Peña >Assignee: Sergio Peña > Attachments: HIVE-9605.3.patch, HIVE-9605.4.patch > > > Parquet nested types are using an extra wrapper object (ArrayWritable) as a > wrapper of map and list elements. This extra object is not needed and causing > unnecessary memory allocations. > An example of code is on HiveCollectionConverter.java: > {noformat} > public void end() { > parent.set(index, wrapList(new ArrayWritable( > Writable.class, list.toArray(new Writable[list.size()]; > } > {noformat} > This object is later unwrapped on AbstractParquetMapInspector, i.e.: > {noformat} > final Writable[] mapContainer = ((ArrayWritable) data).get(); > final Writable[] mapArray = ((ArrayWritable) mapContainer[0]).get(); > for (final Writable obj : mapArray) { > ... > } > {noformat} > We should get rid of this wrapper object to save time and memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9605) Remove parquet nested objects from wrapper writable objects
[ https://issues.apache.org/jira/browse/HIVE-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-9605: -- Attachment: HIVE-9605.4.patch > Remove parquet nested objects from wrapper writable objects > --- > > Key: HIVE-9605 > URL: https://issues.apache.org/jira/browse/HIVE-9605 > Project: Hive > Issue Type: Sub-task >Affects Versions: 0.14.0 >Reporter: Sergio Peña >Assignee: Sergio Peña > Attachments: HIVE-9605.3.patch, HIVE-9605.4.patch > > > Parquet nested types are using an extra wrapper object (ArrayWritable) as a > wrapper of map and list elements. This extra object is not needed and causing > unnecessary memory allocations. > An example of code is on HiveCollectionConverter.java: > {noformat} > public void end() { > parent.set(index, wrapList(new ArrayWritable( > Writable.class, list.toArray(new Writable[list.size()]; > } > {noformat} > This object is later unwrapped on AbstractParquetMapInspector, i.e.: > {noformat} > final Writable[] mapContainer = ((ArrayWritable) data).get(); > final Writable[] mapArray = ((ArrayWritable) mapContainer[0]).get(); > for (final Writable obj : mapArray) { > ... > } > {noformat} > We should get rid of this wrapper object to save time and memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9605) Remove parquet nested objects from wrapper writable objects
[ https://issues.apache.org/jira/browse/HIVE-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-9605: -- Attachment: HIVE-9605.3.patch Fixed unit tests. This patch depends from HIVE-9333. Once it is merged, I will submit this patch. > Remove parquet nested objects from wrapper writable objects > --- > > Key: HIVE-9605 > URL: https://issues.apache.org/jira/browse/HIVE-9605 > Project: Hive > Issue Type: Sub-task >Affects Versions: 0.14.0 >Reporter: Sergio Peña >Assignee: Sergio Peña > Attachments: HIVE-9605.3.patch > > > Parquet nested types are using an extra wrapper object (ArrayWritable) as a > wrapper of map and list elements. This extra object is not needed and causing > unnecessary memory allocations. > An example of code is on HiveCollectionConverter.java: > {noformat} > public void end() { > parent.set(index, wrapList(new ArrayWritable( > Writable.class, list.toArray(new Writable[list.size()]; > } > {noformat} > This object is later unwrapped on AbstractParquetMapInspector, i.e.: > {noformat} > final Writable[] mapContainer = ((ArrayWritable) data).get(); > final Writable[] mapArray = ((ArrayWritable) mapContainer[0]).get(); > for (final Writable obj : mapArray) { > ... > } > {noformat} > We should get rid of this wrapper object to save time and memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9605) Remove parquet nested objects from wrapper writable objects
[ https://issues.apache.org/jira/browse/HIVE-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-9605: -- Attachment: (was: HIVE-9605.2.patch) > Remove parquet nested objects from wrapper writable objects > --- > > Key: HIVE-9605 > URL: https://issues.apache.org/jira/browse/HIVE-9605 > Project: Hive > Issue Type: Sub-task >Affects Versions: 0.14.0 >Reporter: Sergio Peña >Assignee: Sergio Peña > Attachments: HIVE-9605.3.patch > > > Parquet nested types are using an extra wrapper object (ArrayWritable) as a > wrapper of map and list elements. This extra object is not needed and causing > unnecessary memory allocations. > An example of code is on HiveCollectionConverter.java: > {noformat} > public void end() { > parent.set(index, wrapList(new ArrayWritable( > Writable.class, list.toArray(new Writable[list.size()]; > } > {noformat} > This object is later unwrapped on AbstractParquetMapInspector, i.e.: > {noformat} > final Writable[] mapContainer = ((ArrayWritable) data).get(); > final Writable[] mapArray = ((ArrayWritable) mapContainer[0]).get(); > for (final Writable obj : mapArray) { > ... > } > {noformat} > We should get rid of this wrapper object to save time and memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9605) Remove parquet nested objects from wrapper writable objects
[ https://issues.apache.org/jira/browse/HIVE-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-9605: -- Status: Open (was: Patch Available) > Remove parquet nested objects from wrapper writable objects > --- > > Key: HIVE-9605 > URL: https://issues.apache.org/jira/browse/HIVE-9605 > Project: Hive > Issue Type: Sub-task >Affects Versions: 0.14.0 >Reporter: Sergio Peña >Assignee: Sergio Peña > Attachments: HIVE-9605.2.patch > > > Parquet nested types are using an extra wrapper object (ArrayWritable) as a > wrapper of map and list elements. This extra object is not needed and causing > unnecessary memory allocations. > An example of code is on HiveCollectionConverter.java: > {noformat} > public void end() { > parent.set(index, wrapList(new ArrayWritable( > Writable.class, list.toArray(new Writable[list.size()]; > } > {noformat} > This object is later unwrapped on AbstractParquetMapInspector, i.e.: > {noformat} > final Writable[] mapContainer = ((ArrayWritable) data).get(); > final Writable[] mapArray = ((ArrayWritable) mapContainer[0]).get(); > for (final Writable obj : mapArray) { > ... > } > {noformat} > We should get rid of this wrapper object to save time and memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9605) Remove parquet nested objects from wrapper writable objects
[ https://issues.apache.org/jira/browse/HIVE-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-9605: -- Status: Patch Available (was: In Progress) > Remove parquet nested objects from wrapper writable objects > --- > > Key: HIVE-9605 > URL: https://issues.apache.org/jira/browse/HIVE-9605 > Project: Hive > Issue Type: Sub-task >Affects Versions: 0.14.0 >Reporter: Sergio Peña >Assignee: Sergio Peña > Attachments: HIVE-9605.2.patch > > > Parquet nested types are using an extra wrapper object (ArrayWritable) as a > wrapper of map and list elements. This extra object is not needed and causing > unnecessary memory allocations. > An example of code is on HiveCollectionConverter.java: > {noformat} > public void end() { > parent.set(index, wrapList(new ArrayWritable( > Writable.class, list.toArray(new Writable[list.size()]; > } > {noformat} > This object is later unwrapped on AbstractParquetMapInspector, i.e.: > {noformat} > final Writable[] mapContainer = ((ArrayWritable) data).get(); > final Writable[] mapArray = ((ArrayWritable) mapContainer[0]).get(); > for (final Writable obj : mapArray) { > ... > } > {noformat} > We should get rid of this wrapper object to save time and memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9605) Remove parquet nested objects from wrapper writable objects
[ https://issues.apache.org/jira/browse/HIVE-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-9605: -- Attachment: HIVE-9605.2.patch > Remove parquet nested objects from wrapper writable objects > --- > > Key: HIVE-9605 > URL: https://issues.apache.org/jira/browse/HIVE-9605 > Project: Hive > Issue Type: Sub-task >Affects Versions: 0.14.0 >Reporter: Sergio Peña >Assignee: Sergio Peña > Attachments: HIVE-9605.2.patch > > > Parquet nested types are using an extra wrapper object (ArrayWritable) as a > wrapper of map and list elements. This extra object is not needed and causing > unnecessary memory allocations. > An example of code is on HiveCollectionConverter.java: > {noformat} > public void end() { > parent.set(index, wrapList(new ArrayWritable( > Writable.class, list.toArray(new Writable[list.size()]; > } > {noformat} > This object is later unwrapped on AbstractParquetMapInspector, i.e.: > {noformat} > final Writable[] mapContainer = ((ArrayWritable) data).get(); > final Writable[] mapArray = ((ArrayWritable) mapContainer[0]).get(); > for (final Writable obj : mapArray) { > ... > } > {noformat} > We should get rid of this wrapper object to save time and memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)