Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17039 )
Change subject: IMPALA-10485: part(1): make ORC column reader creation independent of schema resolution ...................................................................... IMPALA-10485: part(1): make ORC column reader creation independent of schema resolution Currently ORC schema resolution is position-based only. Then the positional information is reused during ORC column reader creation. This prevents adding other column resolution strategies, e.g. column resolution by name or Iceberg field id. It also prevents schema evolution, as table metadata and file metadata should be in sync. This patch makes column reader creation independent of schema resolution. It does this by creating a mapping between slot/tuple descriptors and ORC type ids during schema resolution and use this mapping during column reader creation. Now further patches just need to add support for other column resolution strategies, the column readers will be created accordingly. Testing: * no additional tests as it is neither a bug fix nor a new feature Change-Id: I0f7d521f9397c5188fadc7996cee0bd1650d363e Reviewed-on: http://gerrit.cloudera.org:8080/17039 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/exec/hdfs-orc-scanner.cc M be/src/exec/hdfs-orc-scanner.h M be/src/exec/orc-column-readers.cc M be/src/exec/orc-column-readers.h M be/src/exec/orc-metadata-utils.cc M be/src/exec/orc-metadata-utils.h M tests/query_test/test_scanners.py 7 files changed, 80 insertions(+), 138 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17039 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I0f7d521f9397c5188fadc7996cee0bd1650d363e Gerrit-Change-Number: 17039 Gerrit-PatchSet: 6 Gerrit-Owner: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Gabor Kaszab <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
