----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35950/#review89721 -----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/DataWritableWriter.java (lines 71 - 72) <https://reviews.apache.org/r/35950/#comment142419> Add the comments before the declarations of messageWriter. ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/DataWritableWriter.java (line 73) <https://reviews.apache.org/r/35950/#comment142418> No need to initialized by null val. ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/DataWritableWriter.java (line 107) <https://reviews.apache.org/r/35950/#comment142422> I don't follow Why rename to schema here. ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/DataWritableWriter.java (line 182) <https://reviews.apache.org/r/35950/#comment142424> groupSchema -> groupType ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/DataWritableWriter.java (line 352) <https://reviews.apache.org/r/35950/#comment142425> ByteDataWrter? ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/DataWritableWriter.java (line 538) <https://reviews.apache.org/r/35950/#comment142420> DateDataWriter - cheng xu On June 28, 2015, 8:29 a.m., Sergio Pena wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/35950/ > ----------------------------------------------------------- > > (Updated June 28, 2015, 8:29 a.m.) > > > Review request for hive, Ryan Blue, cheng xu, and Dong Chen. > > > Bugs: HIVE-11131 > https://issues.apache.org/jira/browse/HIVE-11131 > > > Repository: hive-git > > > Description > ------- > > Implemented data type writers that will be created before the first Hive row > is written to Parquet. These writers contain information about object > inspectors and schema of a specific data type, and calls the specific > addXXXX() method used by Parquet for each data type. > > > Diffs > ----- > > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/DataWritableWriter.java > c195c3ec3ddae19bf255fc2c9633f8bf4390f428 > > Diff: https://reviews.apache.org/r/35950/diff/ > > > Testing > ------- > > Tests from TestDataWritableWriter run OK. > > I run other tests with micro-becnhmarks, and I got some better results from > this new implemntation: > > Using repeated rows across the file, this is the throughput increase using 1 > million records: > > bigint boolean double float int string > 7.598 7.491 7.488 7.588 7.53 0.270 (before) > 10.137 11.511 10.155 10.297 10.242 0.286 (after) > > Using random rows across the file, the is the throughput increase using 1 > million records: > > bigint boolean double float int string > 5.268 7.723 4.107 4.173 4.729 0.20 (before) > 6.236 10.466 5.944 4.749 5.234 0.22 (after) > > > Thanks, > > Sergio Pena > >
