Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/14982 )
Change subject: IMPALA-8801: Date type support for ORC scanner ...................................................................... Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/14982/2/be/src/exec/orc-metadata-utils.cc File be/src/exec/orc-metadata-utils.cc: http://gerrit.cloudera.org:8080/#/c/14982/2/be/src/exec/orc-metadata-utils.cc@181 PS2, Line 181: case orc::TypeKind::DATE: : if (type.type == TYPE_DATE) return Status::OK(); : break; : d > I gave this a quick look an it seems that supporting the use case mentioned I'm ok to do this in another JIRA. We can modify OrcTimestampReader to support reading orc::TimestampVectorBatch into Date type slots. In its constructor it knows which kind of slots (timestamp or date) it's writting to. So in ReadValue() it can have different behaviors based on different modes (timestamp values => timestamp slots / timestamp values => date slots). We can do the same on OrcDateColumnReader to let it support reading ORC Date values into Timestamp type slots. Note that the life cycle of a OrcColumnReader is within the life cycle of the HdfsOrcScanner which only reads a split of an ORC file, and an ORC file can't have two types for one column (e.g. column1 is timestamp in stripe1 and is date in stripe2). So we don't need to deal with different batch types in UpdateInputBatch(). BTW, It'd be better to add test coverage for this type compactibility check in test_scanners.py (See TestOrc.test_type_conversions). -- To view, visit http://gerrit.cloudera.org:8080/14982 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I672a2cdd2452a46b676e0e36942fd310f55c4956 Gerrit-Change-Number: 14982 Gerrit-PatchSet: 4 Gerrit-Owner: Gabor Kaszab <[email protected]> Gerrit-Reviewer: Attila Jeges <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Gabor Kaszab <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Norbert Luksa <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Mon, 13 Jan 2020 08:14:40 +0000 Gerrit-HasComments: Yes
