Csaba Ringhofer has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21492
Change subject: WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu ...................................................................... WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu Before this commit only read support was implemented (convert_kudu_utc_timestamps=true). This change adds write support: if write_kudu_utc_timestamps=true, then timestamps are converted from local time to utc during INSERT/UPSERT to Kudu. In case of ambigious conversions (DST changes) the earlier possible UTC timestamp is written. To be able to read back Kudu tables written by Impala correctly convert_kudu_utc_timestamps and write_kudu_utc_timestamps need to have the same value. The conversion is implemented by adding to_utc_timestamp() to inserted timestamp expressions during planning. This allows doing the same conversion during the pre-insert sorting and partitioning. Read support is implemented differently - in that case the plan is not changed and the scanner does the conversion. Other changes: - Before this change, verification of tests with TIMESTAMP results were skipped when the file format is Kudu. This shouldn't be necessary so the skipping was removed. Haven't run the full test suite yet, for this reason the change is WIP. Change-Id: Ibb4995a64e042e7bb261fcc6e6bf7ffce61e9bd1 --- M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaService.thrift M common/thrift/Query.thrift M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/util/ExprUtil.java M testdata/workloads/functional-query/queries/QueryTest/kudu_timestamp_conversion.test M tests/common/test_result_verifier.py M tests/query_test/test_kudu.py 9 files changed, 86 insertions(+), 23 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/21492/1 -- To view, visit http://gerrit.cloudera.org:8080/21492 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ibb4995a64e042e7bb261fcc6e6bf7ffce61e9bd1 Gerrit-Change-Number: 21492 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer <[email protected]>
