[Impala-ASF-CR] IMPALA-10171: Create query options for local time related flags
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16469 ) Change subject: IMPALA-10171: Create query options for local time related flags .. Patch Set 8: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Gerrit-Change-Number: 16469 Gerrit-PatchSet: 8 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 18 Sep 2020 22:12:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10171: Create query options for local time related flags
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16469 ) Change subject: IMPALA-10171: Create query options for local time related flags .. IMPALA-10171: Create query options for local time related flags convert_legacy_hive_parquet_utc_timestamps and use_local_tz_for_unix_timestamp_conversions were controllable only by flags until now. After this change the old flags are only used on the Coordinator to set the defaults for the query options. If default_query_options also sets these query options, it will take precedence over the old flags. Possible follow up work: - the old flags could be deprecated, as default_query_options can be used to set this on a server level - testing these functionalities no longer needs custom cluster tests - rewriting the existing tests could speed up test execution Testing: - expr-test was rewritten to use the query option instead of the flag - extended the custom cluster tests for the old flags to also use the query options - ran related tests Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Reviewed-on: http://gerrit.cloudera.org:8080/16469 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/exec/parquet/hdfs-parquet-scanner.cc M be/src/exec/parquet/parquet-column-readers.cc M be/src/exprs/expr-test.cc M be/src/exprs/timestamp-functions.h M be/src/runtime/runtime-state.cc M be/src/runtime/runtime-state.h M be/src/service/impala-server.cc M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M testdata/workloads/functional-query/queries/QueryTest/utc-timestamp-functions.test M tests/custom_cluster/test_hive_parquet_timestamp_conversion.py M tests/custom_cluster/test_local_tz_conversion.py 14 files changed, 183 insertions(+), 37 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/16469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Gerrit-Change-Number: 16469 Gerrit-PatchSet: 9 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-10171: Create query options for local time related flags
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16469 ) Change subject: IMPALA-10171: Create query options for local time related flags .. Patch Set 8: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6448/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/16469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Gerrit-Change-Number: 16469 Gerrit-PatchSet: 8 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 18 Sep 2020 16:48:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10171: Create query options for local time related flags
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16469 ) Change subject: IMPALA-10171: Create query options for local time related flags .. Patch Set 8: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Gerrit-Change-Number: 16469 Gerrit-PatchSet: 8 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 18 Sep 2020 16:48:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10171: Create query options for local time related flags
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16469 ) Change subject: IMPALA-10171: Create query options for local time related flags .. Patch Set 7: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Gerrit-Change-Number: 16469 Gerrit-PatchSet: 7 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 18 Sep 2020 16:15:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10171: Create query options for local time related flags
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16469 ) Change subject: IMPALA-10171: Create query options for local time related flags .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7193/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Gerrit-Change-Number: 16469 Gerrit-PatchSet: 6 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 18 Sep 2020 08:18:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10171: Create query options for local time related flags
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16469 ) Change subject: IMPALA-10171: Create query options for local time related flags .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7192/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Gerrit-Change-Number: 16469 Gerrit-PatchSet: 5 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 18 Sep 2020 08:17:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10171: Create query options for local time related flags
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16469 ) Change subject: IMPALA-10171: Create query options for local time related flags .. Patch Set 7: (3 comments) In path 5 and 6 I didn't rebase with a fresh enough version, please ignore them. http://gerrit.cloudera.org:8080/#/c/16469/4/be/src/exprs/timestamp-functions.h File be/src/exprs/timestamp-functions.h: http://gerrit.cloudera.org:8080/#/c/16469/4/be/src/exprs/timestamp-functions.h@112 PS4, Line 112: /// query option use_local_tz_for_unix_timestamp_conversions. If it is true, the > These references to the flag need updating. Done http://gerrit.cloudera.org:8080/#/c/16469/4/common/thrift/ImpalaInternalService.thrift File common/thrift/ImpalaInternalService.thrift: http://gerrit.cloudera.org:8080/#/c/16469/4/common/thrift/ImpalaInternalService.thrift@452 PS4, Line 452: > nit: extra space. Done http://gerrit.cloudera.org:8080/#/c/16469/4/testdata/workloads/functional-query/queries/QueryTest/utc-timestamp-functions.test File testdata/workloads/functional-query/queries/QueryTest/utc-timestamp-functions.test: http://gerrit.cloudera.org:8080/#/c/16469/4/testdata/workloads/functional-query/queries/QueryTest/utc-timestamp-functions.test@20 PS4, Line 20: oca > local-timestamp-functions.test, no? oops, done -- To view, visit http://gerrit.cloudera.org:8080/16469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Gerrit-Change-Number: 16469 Gerrit-PatchSet: 7 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 18 Sep 2020 08:13:22 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10171: Create query options for local time related flags
Hello Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16469 to look at the new patch set (#6). Change subject: IMPALA-10171: Create query options for local time related flags .. IMPALA-10171: Create query options for local time related flags convert_legacy_hive_parquet_utc_timestamps and use_local_tz_for_unix_timestamp_conversions were controllable only by flags until now. After this change the old flags are only used on the Coordinator to set the defaults for the query options. If default_query_options also sets these query options, it will take precedence over the old flags. Possible follow up work: - the old flags could be deprecated, as default_query_options can be used to set this on a server level - testing these functionalities no longer needs custom cluster tests - rewriting the existing tests could speed up test execution Testing: - expr-test was rewritten to use the query option instead of the flag - extended the custom cluster tests for the old flags to also use the query options - ran related tests Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 --- M be/src/exec/parquet/hdfs-parquet-scanner.cc M be/src/exec/parquet/parquet-column-readers.cc M be/src/exprs/expr-test.cc M be/src/exprs/timestamp-functions.h M be/src/runtime/runtime-state.cc M be/src/runtime/runtime-state.h M be/src/service/impala-server.cc M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M testdata/workloads/functional-query/queries/QueryTest/utc-timestamp-functions.test M tests/custom_cluster/test_hive_parquet_timestamp_conversion.py M tests/custom_cluster/test_local_tz_conversion.py 14 files changed, 183 insertions(+), 37 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/69/16469/6 -- To view, visit http://gerrit.cloudera.org:8080/16469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Gerrit-Change-Number: 16469 Gerrit-PatchSet: 6 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-10171: Create query options for local time related flags
Hello Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16469 to look at the new patch set (#5). Change subject: IMPALA-10171: Create query options for local time related flags .. IMPALA-10171: Create query options for local time related flags convert_legacy_hive_parquet_utc_timestamps and use_local_tz_for_unix_timestamp_conversions were controllable only by flags until now. After this change the old flags are only used on the Coordinator to set the defaults for the query options. If default_query_options also sets these query options, it will take precedence over the old flags. Possible follow up work: - the old flags could be deprecated, as default_query_options can be used to set this on a server level - testing these functionalities no longer needs custom cluster tests - rewriting the existing tests could speed up test execution Testing: - expr-test was rewritten to use the query option instead of the flag - extended the custom cluster tests for the old flags to also use the query options - ran related tests Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 --- M be/src/exec/parquet/hdfs-parquet-scanner.cc M be/src/exec/parquet/parquet-column-readers.cc M be/src/exprs/expr-test.cc M be/src/exprs/timestamp-functions.h M be/src/runtime/runtime-state.cc M be/src/runtime/runtime-state.h M be/src/service/impala-server.cc M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M testdata/workloads/functional-query/queries/QueryTest/utc-timestamp-functions.test M tests/custom_cluster/test_hive_parquet_timestamp_conversion.py M tests/custom_cluster/test_local_tz_conversion.py 14 files changed, 183 insertions(+), 37 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/69/16469/5 -- To view, visit http://gerrit.cloudera.org:8080/16469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Gerrit-Change-Number: 16469 Gerrit-PatchSet: 5 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-10171: Create query options for local time related flags
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16469 ) Change subject: IMPALA-10171: Create query options for local time related flags .. Patch Set 4: (3 comments) LGTM aside from some comments http://gerrit.cloudera.org:8080/#/c/16469/4/be/src/exprs/timestamp-functions.h File be/src/exprs/timestamp-functions.h: http://gerrit.cloudera.org:8080/#/c/16469/4/be/src/exprs/timestamp-functions.h@112 PS4, Line 112: /// FLAGS_use_local_tz_for_unix_timestamp_conversions. If the flag is true, the These references to the flag need updating. http://gerrit.cloudera.org:8080/#/c/16469/4/common/thrift/ImpalaInternalService.thrift File common/thrift/ImpalaInternalService.thrift: http://gerrit.cloudera.org:8080/#/c/16469/4/common/thrift/ImpalaInternalService.thrift@452 PS4, Line 452: nit: extra space. http://gerrit.cloudera.org:8080/#/c/16469/4/testdata/workloads/functional-query/queries/QueryTest/utc-timestamp-functions.test File testdata/workloads/functional-query/queries/QueryTest/utc-timestamp-functions.test: http://gerrit.cloudera.org:8080/#/c/16469/4/testdata/workloads/functional-query/queries/QueryTest/utc-timestamp-functions.test@20 PS4, Line 20: utc local-timestamp-functions.test, no? Maybe also copy this comment to each query so it's clear that it applies to all of them. -- To view, visit http://gerrit.cloudera.org:8080/16469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Gerrit-Change-Number: 16469 Gerrit-PatchSet: 4 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 17 Sep 2020 20:49:47 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10171: Create query options for local time related flags
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16469 ) Change subject: IMPALA-10171: Create query options for local time related flags .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Gerrit-Change-Number: 16469 Gerrit-PatchSet: 4 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 17 Sep 2020 17:45:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10171: Create query options for local time related flags
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16469 ) Change subject: IMPALA-10171: Create query options for local time related flags .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7191/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Gerrit-Change-Number: 16469 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 17 Sep 2020 12:44:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10171: Create query options for local time related flags
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16469 ) Change subject: IMPALA-10171: Create query options for local time related flags .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7190/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Gerrit-Change-Number: 16469 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 17 Sep 2020 12:34:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10171: Create query options for local time related flags
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16469 ) Change subject: IMPALA-10171: Create query options for local time related flags .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6438/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/16469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Gerrit-Change-Number: 16469 Gerrit-PatchSet: 4 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 17 Sep 2020 12:24:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10171: Create query options for local time related flags
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16469 ) Change subject: IMPALA-10171: Create query options for local time related flags .. Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/16469/2/be/src/service/query-options.h File be/src/service/query-options.h: http://gerrit.cloudera.org:8080/#/c/16469/2/be/src/service/query-options.h@216 PS2, Line 216: QUERY_OPT_FN(use_local_tz_for_unix_timestamp_conversions,\ > line too long (105 > 90) Done http://gerrit.cloudera.org:8080/#/c/16469/2/be/src/service/query-options.h@218 PS2, Line 218: QUERY_OPT_FN(convert_legacy_hive_parquet_utc_timestamps,\ > line too long (103 > 90) Done -- To view, visit http://gerrit.cloudera.org:8080/16469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Gerrit-Change-Number: 16469 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 17 Sep 2020 12:24:35 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10171: Create query options for local time related flags
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16469 to look at the new patch set (#3). Change subject: IMPALA-10171: Create query options for local time related flags .. IMPALA-10171: Create query options for local time related flags convert_legacy_hive_parquet_utc_timestamps and use_local_tz_for_unix_timestamp_conversions were controllable only by flags until now. After this change the old flags are only used on the Coordinator to set the defaults for the query options. If default_query_options also sets these query options, it will take precedence over the old flags. Possible follow up work: - the old flags could be deprecated, as default_query_options can be used to set this on a server level - testing these functionalities no longer needs custom cluster tests - rewriting the existing tests could speed up test execution Testing: - expr-test was rewritten to use the query option instead of the flag - extended the custom cluster tests for the old flags to also use the query options - ran related tests Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 --- M be/src/exec/parquet/hdfs-parquet-scanner.cc M be/src/exec/parquet/parquet-column-readers.cc M be/src/exprs/expr-test.cc M be/src/runtime/runtime-state.cc M be/src/runtime/runtime-state.h M be/src/service/impala-server.cc M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M testdata/workloads/functional-query/queries/QueryTest/utc-timestamp-functions.test M tests/custom_cluster/test_hive_parquet_timestamp_conversion.py M tests/custom_cluster/test_local_tz_conversion.py 13 files changed, 165 insertions(+), 34 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/69/16469/3 -- To view, visit http://gerrit.cloudera.org:8080/16469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Gerrit-Change-Number: 16469 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10171: Create query options for local time related flags
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16469 ) Change subject: IMPALA-10171: Create query options for local time related flags .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/16469/2/be/src/service/query-options.h File be/src/service/query-options.h: http://gerrit.cloudera.org:8080/#/c/16469/2/be/src/service/query-options.h@216 PS2, Line 216: QUERY_OPT_FN(use_local_tz_for_unix_timestamp_conversions, USE_LOCAL_TZ_FOR_UNIX_TIMESTAMP_CONVERSIONS,\ line too long (105 > 90) http://gerrit.cloudera.org:8080/#/c/16469/2/be/src/service/query-options.h@218 PS2, Line 218: QUERY_OPT_FN(convert_legacy_hive_parquet_utc_timestamps, CONVERT_LEGACY_HIVE_PARQUET_UTC_TIMESTAMPS,\ line too long (103 > 90) -- To view, visit http://gerrit.cloudera.org:8080/16469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Gerrit-Change-Number: 16469 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 17 Sep 2020 12:13:10 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10171: Create query options for local time related flags
Csaba Ringhofer has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16469 Change subject: IMPALA-10171: Create query options for local time related flags .. IMPALA-10171: Create query options for local time related flags convert_legacy_hive_parquet_utc_timestamps and use_local_tz_for_unix_timestamp_conversions were controllable only by flags until now. After this change the old flags are only used on the Coordinator to set the defaults for the query options. If default_query_options also sets these query options, it will take precedence over the old flags. Possible follow up work: - the old flags could be deprecated, as default_query_options can be used to set this on a server level - testing these functionalities no longer needs custom cluster tests - rewriting the existing tests could speed up test execution Testing: - expr-test was rewritten to use the query option instead of the flag - extended the custom cluster tests for the old flags to also use the query options - ran related tests Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 --- M be/src/exec/parquet/hdfs-parquet-scanner.cc M be/src/exec/parquet/parquet-column-readers.cc M be/src/exprs/expr-test.cc M be/src/runtime/runtime-state.cc M be/src/runtime/runtime-state.h M be/src/service/impala-server.cc M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M testdata/workloads/functional-query/queries/QueryTest/utc-timestamp-functions.test M tests/custom_cluster/test_hive_parquet_timestamp_conversion.py M tests/custom_cluster/test_local_tz_conversion.py 13 files changed, 165 insertions(+), 34 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/69/16469/2 -- To view, visit http://gerrit.cloudera.org:8080/16469 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I5c4252d1c8f8e224c1d0e8234f09374bcc0c6f68 Gerrit-Change-Number: 16469 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer
