Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/11057 )
Change subject: IMPALA-5050: Add support to read TIMESTAMP_MILLIS and TIMESTAMP_MICROS from Parquet ...................................................................... Patch Set 16: (8 comments) http://gerrit.cloudera.org:8080/#/c/11057/16/be/src/exec/hdfs-parquet-scanner.cc File be/src/exec/hdfs-parquet-scanner.cc: http://gerrit.cloudera.org:8080/#/c/11057/16/be/src/exec/hdfs-parquet-scanner.cc@518 PS16, Line 518: (col_type.IsTimestampType()) { : stats_read = ColumnStatsReader::ReadTimestampStatFromThrift(col_chunk, col_order, : *node->element, ColumnStatsReader::StatsField::MIN, : CreateTimestampHelper(*node->element), static_cast<TimestampValue*>(slot)); : } else { : stats_read = ColumnStatsReader::ReadFromThrift( : col_chunk, col_type, col_order, ColumnStatsReader::StatsField::MIN, slot); : } > I still prefer to hide that logic in ReadFromThrift(), but feel free to lea I am still thinking about a way to make this nicer without major refactoring. http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/exec/hdfs-parquet-scanner.cc File be/src/exec/hdfs-parquet-scanner.cc: http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/exec/hdfs-parquet-scanner.cc@516 PS18, Line 516: == "le") > nit: stats_field Done http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/exec/parquet-column-readers.cc File be/src/exec/parquet-column-readers.cc: http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/exec/parquet-column-readers.cc@768 PS18, Line 768: if (UNLIKELY(!TimestampValue::IsValidDate(val->date()))) { > A short comment hinting at why IsValidTime() is not needed might be useful Thanks for spotting this - the asymmetry came in with a rebase, and was not intentional. The logic worked without problems, but did not express the different assumptions about memcopied and converted timestamps. http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/exec/parquet-common.h File be/src/exec/parquet-common.h: http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/exec/parquet-common.h@433 PS18, Line 433: ParquetTimestampHelper > The name is ok but a little vague. I wonder if ParquetTimestampDecoder woul Done http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/exec/parquet-common.cc File be/src/exec/parquet-common.cc: http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/exec/parquet-common.cc@114 PS18, Line 114: if (needs_conversion) { > nit: can fit conditional on one line Done http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/exec/parquet-common.cc@123 PS18, Line 123: v->UtcToLocal(*timezone_, &repeated_period_start); > Ah, this is clever. Thanks :) http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/util/dict-encoding.h File be/src/util/dict-encoding.h: http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/util/dict-encoding.h@383 PS18, Line 383: inline > nit: inline is already implied for functions in the class body Done http://gerrit.cloudera.org:8080/#/c/11057/18/be/src/util/dict-encoding.h@383 PS18, Line 383: ResetInner > I feel like Decode() or DecodeHelper() would be clearer, since it describes Done -- To view, visit http://gerrit.cloudera.org:8080/11057 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4c7c01fffa31b3d2ca3480adf6ff851137dadac3 Gerrit-Change-Number: 11057 Gerrit-PatchSet: 16 Gerrit-Owner: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Attila Jeges <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Mon, 05 Nov 2018 19:47:15 +0000 Gerrit-HasComments: Yes
