[Impala-ASF-CR] IMPALA-8572: improve QueryEventHook execution
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13748 ) Change subject: IMPALA-8572: improve QueryEventHook execution .. Patch Set 10: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3813/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13748 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb88422f7cfe86947d11ce57d2b4c63e57d1b643 Gerrit-Change-Number: 13748 Gerrit-PatchSet: 10 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Wed, 03 Jul 2019 05:41:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8572: improve QueryEventHook execution
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13748 ) Change subject: IMPALA-8572: improve QueryEventHook execution .. Patch Set 9: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3812/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13748 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb88422f7cfe86947d11ce57d2b4c63e57d1b643 Gerrit-Change-Number: 13748 Gerrit-PatchSet: 9 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Wed, 03 Jul 2019 05:41:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8709: Add Damerau-Levenshtein edit distance built-in function
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13794 ) Change subject: IMPALA-8709: Add Damerau-Levenshtein edit distance built-in function .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3811/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13794 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib759817ec15e7075bf49d51e494e45c8af4db94d Gerrit-Change-Number: 13794 Gerrit-PatchSet: 1 Gerrit-Owner: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 03 Jul 2019 05:25:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8572: improve QueryEventHook execution
Hello Fredy Wijaya, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13748 to look at the new patch set (#10). Change subject: IMPALA-8572: improve QueryEventHook execution .. IMPALA-8572: improve QueryEventHook execution WIP Testing: TBD Change-Id: Ibb88422f7cfe86947d11ce57d2b4c63e57d1b643 --- M be/src/service/impala-server.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift A fe/src/main/java/org/apache/impala/hooks/FixedCapacityQueryHookExecutor.java M fe/src/main/java/org/apache/impala/hooks/QueryEventHookManager.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/Frontend.java A fe/src/test/java/org/apache/impala/hooks/FixedCapacityQueryHookExecutorTest.java M fe/src/test/java/org/apache/impala/hooks/QueryEventHookManagerTest.java M query-event-hook-api/src/main/java/org/apache/impala/hooks/QueryEventHook.java 10 files changed, 767 insertions(+), 109 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/13748/10 -- To view, visit http://gerrit.cloudera.org:8080/13748 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ibb88422f7cfe86947d11ce57d2b4c63e57d1b643 Gerrit-Change-Number: 13748 Gerrit-PatchSet: 10 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] IMPALA-8572: improve QueryEventHook execution
Hello Fredy Wijaya, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13748 to look at the new patch set (#9). Change subject: IMPALA-8572: improve QueryEventHook execution .. IMPALA-8572: improve QueryEventHook execution WIP Testing: TBD Change-Id: Ibb88422f7cfe86947d11ce57d2b4c63e57d1b643 --- M be/src/service/impala-server.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift A fe/src/main/java/org/apache/impala/hooks/FixedCapacityQueryHookExecutor.java M fe/src/main/java/org/apache/impala/hooks/QueryEventHookManager.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/Frontend.java A fe/src/test/java/org/apache/impala/hooks/FixedCapacityQueryHookExecutorTest.java M fe/src/test/java/org/apache/impala/hooks/QueryEventHookManagerTest.java M query-event-hook-api/src/main/java/org/apache/impala/hooks/QueryEventHook.java 10 files changed, 658 insertions(+), 109 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/13748/9 -- To view, visit http://gerrit.cloudera.org:8080/13748 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ibb88422f7cfe86947d11ce57d2b4c63e57d1b643 Gerrit-Change-Number: 13748 Gerrit-PatchSet: 9 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] IMPALA-8709: Add Damerau-Levenshtein edit distance built-in function
Greg Rahn has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13794 Change subject: IMPALA-8709: Add Damerau-Levenshtein edit distance built-in function .. IMPALA-8709: Add Damerau-Levenshtein edit distance built-in function This patch adds new built-in functions to calculate Damerau-Levenshtein edit distance. Implmented as dle_dst() and damerau_levenshtein(). If either value is NULL or both values are NULL returns NULL which differs from Netezza's dle_dst() which returns the length of the not NULL value or 0 if both values are NULL. The NULL behavior matches the existing levenshtein() function. Testing: - Added unit tests to expr-test.cc - Manualal testing on over 1400 string pairs from http://marvin.cs.uidaho.edu/misspell.html and results match Netezza Change-Id: Ib759817ec15e7075bf49d51e494e45c8af4db94d --- M be/src/exprs/expr-test.cc M be/src/exprs/string-functions-ir.cc M be/src/exprs/string-functions.h M common/function-registry/impala_functions.py 4 files changed, 110 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/13794/1 -- To view, visit http://gerrit.cloudera.org:8080/13794 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ib759817ec15e7075bf49d51e494e45c8af4db94d Gerrit-Change-Number: 13794 Gerrit-PatchSet: 1 Gerrit-Owner: Greg Rahn
[Impala-ASF-CR] IMPALA-8663 : FileMetadataLoader should skip hidden and tmp directories
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13665 ) Change subject: IMPALA-8663 : FileMetadataLoader should skip hidden and tmp directories .. Patch Set 12: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3810/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13665 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2c4a22908304fe9e377d77d6c18d401c3f3294aa Gerrit-Change-Number: 13665 Gerrit-PatchSet: 12 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 03 Jul 2019 03:30:52 + Gerrit-HasComments: No
[Impala-ASF-CR] build: use thin static archives
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13775 ) Change subject: build: use thin static archives .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3b54b5be8658c951914758c406cca55d4cc1756e Gerrit-Change-Number: 13775 Gerrit-PatchSet: 3 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Wed, 03 Jul 2019 03:08:16 + Gerrit-HasComments: No
[Impala-ASF-CR] build: use thin static archives
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13775 ) Change subject: build: use thin static archives .. build: use thin static archives This enables thin static archives for our internal libraries during the build. This makes linking much faster since the static archives just point to object files instead of copying them. This reduces the size of intermediate '.a' files for my debug build from about 1.4GB to 58MB. Incremental rebuilds are slightly sped up, though maybe in the realm of noise (likely the performance improvement depends on how much RAM is available for buffer caching the IO): Without patch incremental build of catalogd after modifying CatalogObjects.thrift: real0m53.433s user7m6.400s sys 1m21.610s With patch: real0m44.772s user7m6.632s sys 1m16.870s Change-Id: I3b54b5be8658c951914758c406cca55d4cc1756e Reviewed-on: http://gerrit.cloudera.org:8080/13775 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M CMakeLists.txt 1 file changed, 9 insertions(+), 0 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/13775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I3b54b5be8658c951914758c406cca55d4cc1756e Gerrit-Change-Number: 13775 Gerrit-PatchSet: 4 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8673: Add query option to force plan hints for insert queries
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13753 ) Change subject: IMPALA-8673: Add query option to force plan hints for insert queries .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13753 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1c3f213402b8e4d1940f96738ad21edf800fa43a Gerrit-Change-Number: 13753 Gerrit-PatchSet: 4 Gerrit-Owner: Abhishek Rawat Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 03 Jul 2019 02:54:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8673: Add query option to force plan hints for insert queries
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13753 ) Change subject: IMPALA-8673: Add query option to force plan hints for insert queries .. IMPALA-8673: Add query option to force plan hints for insert queries IMPALA-5293 enabled the pre-insert clustering by default. This could cause performance regression and this change provides a query option for setting default hints for INSERT statement. New query option 'DEFAULT_HINTS_INSERT_STATEMENT' was added. It also supports adding multiple supported hints when separated by ':' set DEFAULT_HINTS_INSERT_STATEMENT=[clustered|noclustered]; set DEFAULT_HINTS_INSERT_STATEMENT=[shuffle|noshuffle]; set DEFAULT_HINTS_INSERT_STATEMENT= [clustered|noclustered]:[shuffle|noshuffle]; If a given insert statement already has plan hints in the query text, the default hints, if any, are all ignored. This is because, if a query has plan hints specified by the user, we don't want to override it. When a default hint is set, and there is an INSERT statement without any plan hints in the query text, the default hints have the same affect as they would have had, if they were applied as plan hints in the query text. So these default hints have the same application and restrictions as the existing plan hints for INSERT statement. The default hints apply to HDFS and Kudu table formats and are ignored for HBase table format. Testing: - Added unit tests in AnalyzeDDLTest for CTAS. - Added unit tests in AnalyzeStmtsTest for insert statements. - Added unit tests in PlannerTest validating the plan for various scenarios involving different combinations of default hints. Change-Id: I1c3f213402b8e4d1940f96738ad21edf800fa43a Reviewed-on: http://gerrit.cloudera.org:8080/13753 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java A testdata/workloads/functional-planner/queries/PlannerTest/insert-default-clustered-noshuffle.test A testdata/workloads/functional-planner/queries/PlannerTest/insert-default-clustered-shuffle.test A testdata/workloads/functional-planner/queries/PlannerTest/insert-default-clustered.test A testdata/workloads/functional-planner/queries/PlannerTest/insert-default-noclustered-noshuffle.test A testdata/workloads/functional-planner/queries/PlannerTest/insert-default-noclustered-shuffle.test A testdata/workloads/functional-planner/queries/PlannerTest/insert-default-noclustered.test A testdata/workloads/functional-planner/queries/PlannerTest/insert-default-noshuffle.test A testdata/workloads/functional-planner/queries/PlannerTest/insert-default-shuffle.test 17 files changed, 1,359 insertions(+), 4 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/13753 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I1c3f213402b8e4d1940f96738ad21edf800fa43a Gerrit-Change-Number: 13753 Gerrit-PatchSet: 5 Gerrit-Owner: Abhishek Rawat Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8663 : FileMetadataLoader should skip hidden and tmp directories
Vihang Karajgaonkar has uploaded a new patch set (#12). ( http://gerrit.cloudera.org:8080/13665 ) Change subject: IMPALA-8663 : FileMetadataLoader should skip hidden and tmp directories .. IMPALA-8663 : FileMetadataLoader should skip hidden and tmp directories The FileMetadataLoader is used to load the file information in when the table is loaded. By default, it lists all the files in the table/partition directory. Currently, it only skips the filenames which are invalid (hidden files and ones starting with "_" etc). However, it does not skip the directories which are temporary or hidden. In case of Hive when data is inserted into a table, it creates a temporary staging directory which is a hidden directory under the table location. When the insert in hive is completed, such staging directories are removed. But if there is a refresh called during that time, FileMetadataLoader will add the files in the staging directory as well. Not only this could cause temporary invalid results but it causes table to go in a bad state when these temporary directories are removed. The only work-around in such a case to issue a refresh on the table again. This patch adds logic in the filemetadataloader to ignore such temporary staging directories. Unfortunately, hadoop does not provide a API which can recursively list files in a directory and skip certain directories. This patch adds a new FilterIterator which wraps around existing listFiles, listStatus and RecursingIterator to skip the hidden directories from the listing result. Also, the existing code to recover partitions implements its own recursion logic which includes path validation. This already skips such hidden directories since they do not conform to the partition spec. The patch does a minor modification to this method by directly calling the listStatusIterator instead of going through FileSystemUtil#listStatus whiche uses the filtering remote iterator now. Testing: 1. Added a new tests as well as modified existing ones which were related to cover interesting cases. 2. Ran concurrent inserts from Hive while issuing refresh in a loop on Impala side. Earlier this would cause the table to go into a bad state. Now, it works fine for the staging directories. It still runs into a FileNotFoundException from the impalad when there are insert overwrite statements in Hive Change-Id: I2c4a22908304fe9e377d77d6c18d401c3f3294aa --- M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java M fe/src/test/java/org/apache/impala/catalog/FileMetadataLoaderTest.java A fe/src/test/java/org/apache/impala/common/FileSystemUtilTest.java M fe/src/test/java/org/apache/impala/util/AcidUtilsTest.java M tests/metadata/test_recursive_listing.py 6 files changed, 236 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/13665/12 -- To view, visit http://gerrit.cloudera.org:8080/13665 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2c4a22908304fe9e377d77d6c18d401c3f3294aa Gerrit-Change-Number: 13665 Gerrit-PatchSet: 12 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8341: [DOCS] Added a note about the requirement for existing dirs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13793 ) Change subject: IMPALA-8341: [DOCS] Added a note about the requirement for existing dirs .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/374/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/13793 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5feff3ab7c09ee681098ec3e630977cb92f1 Gerrit-Change-Number: 13793 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 03 Jul 2019 02:00:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8734: Reload table schema on TBLPROPERTIES change
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13785 ) Change subject: IMPALA-8734: Reload table schema on TBLPROPERTIES change .. IMPALA-8734: Reload table schema on TBLPROPERTIES change Prior to this patch, an INVALIDATE METADATA was required when altering the TBLPROPERTIES for the changes to take effect. With this patch the table schema is automatically reloaded on TBLPROPERTIES change. Testing: - Added a new test in test_ddl.py - Ran test_ddl.py Change-Id: I2a43a962c2a456f3ddc078b2924f551fccb5c2ad Reviewed-on: http://gerrit.cloudera.org:8080/13785 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M tests/metadata/test_ddl.py 2 files changed, 20 insertions(+), 0 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/13785 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I2a43a962c2a456f3ddc078b2924f551fccb5c2ad Gerrit-Change-Number: 13785 Gerrit-PatchSet: 6 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8734: Reload table schema on TBLPROPERTIES change
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13785 ) Change subject: IMPALA-8734: Reload table schema on TBLPROPERTIES change .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13785 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2a43a962c2a456f3ddc078b2924f551fccb5c2ad Gerrit-Change-Number: 13785 Gerrit-PatchSet: 5 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 03 Jul 2019 01:57:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8341: [DOCS] Added a note about the requirement for existing dirs
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/13793 ) Change subject: IMPALA-8341: [DOCS] Added a note about the requirement for existing dirs .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13793 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5feff3ab7c09ee681098ec3e630977cb92f1 Gerrit-Change-Number: 13793 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 03 Jul 2019 01:37:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8341: [DOCS] Added a note about the experiemental feature
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/13788 ) Change subject: IMPALA-8341: [DOCS] Added a note about the experiemental feature .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/13788/1/docs/topics/impala_data_cache.xml File docs/topics/impala_data_cache.xml: http://gerrit.cloudera.org:8080/#/c/13788/1/docs/topics/impala_data_cache.xml@64 PS1, Line 64: The specified directories must exist in the local filesystem of each Impala Daemon. > or Impala will fail to start. Done; Added in a new patch as this was already merged. -- To view, visit http://gerrit.cloudera.org:8080/13788 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If9f5795a28738f7df435a3bea06ffa1a216fd04e Gerrit-Change-Number: 13788 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Wed, 03 Jul 2019 01:37:35 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8341: [DOCS] Added a note about the requirement for existing dirs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13793 ) Change subject: IMPALA-8341: [DOCS] Added a note about the requirement for existing dirs .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/374/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/13793 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5feff3ab7c09ee681098ec3e630977cb92f1 Gerrit-Change-Number: 13793 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 03 Jul 2019 01:37:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8341: [DOCS] Added a note about the requirement for existing dirs
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13793 Change subject: IMPALA-8341: [DOCS] Added a note about the requirement for existing dirs .. IMPALA-8341: [DOCS] Added a note about the requirement for existing dirs Change-Id: I5feff3ab7c09ee681098ec3e630977cb92f1 --- M docs/topics/impala_data_cache.xml 1 file changed, 3 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/93/13793/1 -- To view, visit http://gerrit.cloudera.org:8080/13793 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I5feff3ab7c09ee681098ec3e630977cb92f1 Gerrit-Change-Number: 13793 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-8341: [DOCS] Added a note about the experiemental feature
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13788 ) Change subject: IMPALA-8341: [DOCS] Added a note about the experiemental feature .. IMPALA-8341: [DOCS] Added a note about the experiemental feature Change-Id: If9f5795a28738f7df435a3bea06ffa1a216fd04e Reviewed-on: http://gerrit.cloudera.org:8080/13788 Tested-by: Impala Public Jenkins Reviewed-by: Michael Ho --- M docs/topics/impala_data_cache.xml 1 file changed, 4 insertions(+), 0 deletions(-) Approvals: Impala Public Jenkins: Verified Michael Ho: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/13788 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: If9f5795a28738f7df435a3bea06ffa1a216fd04e Gerrit-Change-Number: 13788 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho
[Impala-ASF-CR] IMPALA-8341: [DOCS] Added a note about the experiemental feature
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/13788 ) Change subject: IMPALA-8341: [DOCS] Added a note about the experiemental feature .. Patch Set 1: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/13788/1/docs/topics/impala_data_cache.xml File docs/topics/impala_data_cache.xml: http://gerrit.cloudera.org:8080/#/c/13788/1/docs/topics/impala_data_cache.xml@64 PS1, Line 64: The specified directories must exist in the local filesystem of each Impala Daemon. or Impala will fail to start. -- To view, visit http://gerrit.cloudera.org:8080/13788 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If9f5795a28738f7df435a3bea06ffa1a216fd04e Gerrit-Change-Number: 13788 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Wed, 03 Jul 2019 01:28:49 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8732: Use a serialized descriptor table in TQueryCtx
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13772 ) Change subject: IMPALA-8732: Use a serialized descriptor table in TQueryCtx .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3809/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13772 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I458aa62dd4d1e4e4a7b1869a604623a69f3b2d9a Gerrit-Change-Number: 13772 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Wed, 03 Jul 2019 01:01:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8732: Use a serialized descriptor table in TQueryCtx
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/13772 ) Change subject: IMPALA-8732: Use a serialized descriptor table in TQueryCtx .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/13772/1/be/src/runtime/coordinator-backend-state.cc File be/src/runtime/coordinator-backend-state.cc: http://gerrit.cloudera.org:8080/#/c/13772/1/be/src/runtime/coordinator-backend-state.cc@200 PS1, Line 200: sidecar.__set_query_ctx(query_ctx()); Makes me wonder if we should factor out the descriptor table from query_ctx() and send the entire serialized descriptor table byte array as a sidecar instead to avoid another malloc + copy in line 213 below. If the above makes sense, it could be a follow-up change to make backporting easier. http://gerrit.cloudera.org:8080/#/c/13772/1/be/src/runtime/descriptors.h File be/src/runtime/descriptors.h: http://gerrit.cloudera.org:8080/#/c/13772/1/be/src/runtime/descriptors.h@480 PS1, Line 480: thrift_tbl stale http://gerrit.cloudera.org:8080/#/c/13772/1/be/src/runtime/descriptors.h@485 PS1, Line 485: thrift_tbl stale -- To view, visit http://gerrit.cloudera.org:8080/13772 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I458aa62dd4d1e4e4a7b1869a604623a69f3b2d9a Gerrit-Change-Number: 13772 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Wed, 03 Jul 2019 00:48:36 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8732: Use a serialized descriptor table in TQueryCtx
Hello Michael Ho, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13772 to look at the new patch set (#2). Change subject: IMPALA-8732: Use a serialized descriptor table in TQueryCtx .. IMPALA-8732: Use a serialized descriptor table in TQueryCtx In IMPALA-8732, there is contention in tcmalloc when sending the ExecQueryFInstances messages for a query referencing a large number of partitions. This is because each thread in the ExecEnv::exec_rpc_thread_pool_ is making a copy of the TQueryCtx, which contains the TDescriptorTable and a large map of THdfsPartition objects. Every thread in the exec_rpc_thread_pool_ is doing this simultaneously. The threads are copying this structure, but this structure is the same across all messages for this query. Copying a large map of THdfsPartition objects is wasteful, especially considering that the coordinator does not need to access any of this information before sending it out to executors. This changes TQueryCtx to contain a TDescriptorTableSerialized, which is a binary blob containing the serialized form of TDescriptorTable. This is serialized in the frontend and passed directly through to executors. The old unserialized TDescriptorTable form is maintained to enable frontend planner tests (which use incomplete structures lacking some required fields and cannot be serialized). Testing: - Core and exhaustive tests pass Change-Id: I458aa62dd4d1e4e4a7b1869a604623a69f3b2d9a --- M be/src/runtime/coordinator.cc M be/src/runtime/descriptors.cc M be/src/runtime/descriptors.h M be/src/runtime/query-state.cc M common/thrift/Descriptors.thrift M common/thrift/ImpalaInternalService.thrift M fe/src/main/java/org/apache/impala/analysis/DescriptorTable.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java 9 files changed, 90 insertions(+), 16 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/72/13772/2 -- To view, visit http://gerrit.cloudera.org:8080/13772 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I458aa62dd4d1e4e4a7b1869a604623a69f3b2d9a Gerrit-Change-Number: 13772 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho
[Impala-ASF-CR] IMPALA-8732: Use a serialized descriptor table in TQueryCtx
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/13772 ) Change subject: IMPALA-8732: Use a serialized descriptor table in TQueryCtx .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/13772/1/common/thrift/ImpalaInternalService.thrift File common/thrift/ImpalaInternalService.thrift: http://gerrit.cloudera.org:8080/#/c/13772/1/common/thrift/ImpalaInternalService.thrift@469 PS1, Line 469: // with incomplete structures (e.g. THdfsTable without the required nullPartitionKeyValue > line too long (91 > 90) Done -- To view, visit http://gerrit.cloudera.org:8080/13772 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I458aa62dd4d1e4e4a7b1869a604623a69f3b2d9a Gerrit-Change-Number: 13772 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Wed, 03 Jul 2019 00:22:23 + Gerrit-HasComments: Yes
[Impala-ASF-CR] [DOCS] \N needs to be escaped in Sqoop for Impala to recognize as null
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13773 ) Change subject: [DOCS] \N needs to be escaped in Sqoop for Impala to recognize as null .. [DOCS] \N needs to be escaped in Sqoop for Impala to recognize as null Change-Id: I581f532c2f865e93d8922661c389791f41805b64 Reviewed-on: http://gerrit.cloudera.org:8080/13773 Tested-by: Impala Public Jenkins Reviewed-by: Joe McDonnell --- M docs/topics/impala_txtfile.xml 1 file changed, 28 insertions(+), 23 deletions(-) Approvals: Impala Public Jenkins: Verified Joe McDonnell: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/13773 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I581f532c2f865e93d8922661c389791f41805b64 Gerrit-Change-Number: 13773 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] [DOCS] \N needs to be escaped in Sqoop for Impala to recognize as null
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/13773 ) Change subject: [DOCS] \N needs to be escaped in Sqoop for Impala to recognize as null .. Patch Set 2: Code-Review+2 I looked up the sqoop documentation, and this change is correct. https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_importing_data_into_hive Thanks for making this change. -- To view, visit http://gerrit.cloudera.org:8080/13773 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I581f532c2f865e93d8922661c389791f41805b64 Gerrit-Change-Number: 13773 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 02 Jul 2019 23:55:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13777 ) Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL .. Patch Set 5: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/373/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/13777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f Gerrit-Change-Number: 13777 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 02 Jul 2019 22:12:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/13777 ) Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/13777/4/docs/topics/impala_metadata.xml File docs/topics/impala_metadata.xml: http://gerrit.cloudera.org:8080/#/c/13777/4/docs/topics/impala_metadata.xml@54 PS4, Line 54: imapald > typo Done -- To view, visit http://gerrit.cloudera.org:8080/13777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f Gerrit-Change-Number: 13777 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 02 Jul 2019 21:49:43 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13777 ) Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL .. Patch Set 5: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/373/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/13777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f Gerrit-Change-Number: 13777 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 02 Jul 2019 21:49:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL
Hello Bharath Vissapragada, Vihang Karajgaonkar, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13777 to look at the new patch set (#5). Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL .. IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL - Also made a few formatting changes. - Removed the Preview Release note for Invalidation of Metadata cache. Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f --- M docs/topics/impala_metadata.xml 1 file changed, 57 insertions(+), 31 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/77/13777/5 -- To view, visit http://gerrit.cloudera.org:8080/13777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f Gerrit-Change-Number: 13777 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] build: use thin static archives
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13775 ) Change subject: build: use thin static archives .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3808/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3b54b5be8658c951914758c406cca55d4cc1756e Gerrit-Change-Number: 13775 Gerrit-PatchSet: 2 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 02 Jul 2019 21:41:10 + Gerrit-HasComments: No
[Impala-ASF-CR] build: use thin static archives
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13775 ) Change subject: build: use thin static archives .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3b54b5be8658c951914758c406cca55d4cc1756e Gerrit-Change-Number: 13775 Gerrit-PatchSet: 3 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 02 Jul 2019 21:31:01 + Gerrit-HasComments: No
[Impala-ASF-CR] build: use thin static archives
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13775 ) Change subject: build: use thin static archives .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4577/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3b54b5be8658c951914758c406cca55d4cc1756e Gerrit-Change-Number: 13775 Gerrit-PatchSet: 3 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 02 Jul 2019 21:31:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/13777 ) Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL .. Patch Set 4: Code-Review+1 (1 comment) I'll let Vihang take another look. http://gerrit.cloudera.org:8080/#/c/13777/4/docs/topics/impala_metadata.xml File docs/topics/impala_metadata.xml: http://gerrit.cloudera.org:8080/#/c/13777/4/docs/topics/impala_metadata.xml@54 PS4, Line 54: imapald typo -- To view, visit http://gerrit.cloudera.org:8080/13777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f Gerrit-Change-Number: 13777 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 02 Jul 2019 21:30:02 + Gerrit-HasComments: Yes
[Impala-ASF-CR] build: use thin static archives
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/13775 ) Change subject: build: use thin static archives .. Patch Set 2: Code-Review+2 This makes sense to me. -- To view, visit http://gerrit.cloudera.org:8080/13775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3b54b5be8658c951914758c406cca55d4cc1756e Gerrit-Change-Number: 13775 Gerrit-PatchSet: 2 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 02 Jul 2019 21:19:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8673: Add query option to force plan hints for insert queries
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13753 ) Change subject: IMPALA-8673: Add query option to force plan hints for insert queries .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4576/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13753 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1c3f213402b8e4d1940f96738ad21edf800fa43a Gerrit-Change-Number: 13753 Gerrit-PatchSet: 4 Gerrit-Owner: Abhishek Rawat Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 02 Jul 2019 21:15:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8673: Add query option to force plan hints for insert queries
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13753 ) Change subject: IMPALA-8673: Add query option to force plan hints for insert queries .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13753 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1c3f213402b8e4d1940f96738ad21edf800fa43a Gerrit-Change-Number: 13753 Gerrit-PatchSet: 4 Gerrit-Owner: Abhishek Rawat Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 02 Jul 2019 21:15:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8673: Add query option to force plan hints for insert queries
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/13753 ) Change subject: IMPALA-8673: Add query option to force plan hints for insert queries .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13753 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1c3f213402b8e4d1940f96738ad21edf800fa43a Gerrit-Change-Number: 13753 Gerrit-PatchSet: 3 Gerrit-Owner: Abhishek Rawat Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 02 Jul 2019 21:15:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13777 ) Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL .. Patch Set 4: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/372/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/13777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f Gerrit-Change-Number: 13777 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 02 Jul 2019 21:14:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13777 ) Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL .. Patch Set 3: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/371/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/13777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f Gerrit-Change-Number: 13777 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 02 Jul 2019 21:12:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13777 ) Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL .. Patch Set 4: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/372/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/13777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f Gerrit-Change-Number: 13777 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 02 Jul 2019 21:08:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL
Hello Bharath Vissapragada, Vihang Karajgaonkar, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13777 to look at the new patch set (#4). Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL .. IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL - Also made a few formatting changes. - Removed the Preview Release note for Invalidation of Metadata cache. Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f --- M docs/topics/impala_metadata.xml 1 file changed, 57 insertions(+), 31 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/77/13777/4 -- To view, visit http://gerrit.cloudera.org:8080/13777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f Gerrit-Change-Number: 13777 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13777 ) Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL .. Patch Set 3: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/371/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/13777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f Gerrit-Change-Number: 13777 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 02 Jul 2019 21:00:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL
Hello Bharath Vissapragada, Vihang Karajgaonkar, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13777 to look at the new patch set (#3). Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL .. IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL - Also made a few formatting changes. - Removed the Preview Release note for Invalidation of Metadata cache. Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f --- M docs/topics/impala_metadata.xml 1 file changed, 49 insertions(+), 31 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/77/13777/3 -- To view, visit http://gerrit.cloudera.org:8080/13777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f Gerrit-Change-Number: 13777 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] build: use thin static archives
Hello Tim Armstrong, Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13775 to look at the new patch set (#2). Change subject: build: use thin static archives .. build: use thin static archives This enables thin static archives for our internal libraries during the build. This makes linking much faster since the static archives just point to object files instead of copying them. This reduces the size of intermediate '.a' files for my debug build from about 1.4GB to 58MB. Incremental rebuilds are slightly sped up, though maybe in the realm of noise (likely the performance improvement depends on how much RAM is available for buffer caching the IO): Without patch incremental build of catalogd after modifying CatalogObjects.thrift: real0m53.433s user7m6.400s sys 1m21.610s With patch: real0m44.772s user7m6.632s sys 1m16.870s Change-Id: I3b54b5be8658c951914758c406cca55d4cc1756e --- M CMakeLists.txt 1 file changed, 9 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/75/13775/2 -- To view, visit http://gerrit.cloudera.org:8080/13775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I3b54b5be8658c951914758c406cca55d4cc1756e Gerrit-Change-Number: 13775 Gerrit-PatchSet: 2 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13777 ) Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL .. Patch Set 2: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/370/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/13777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f Gerrit-Change-Number: 13777 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 02 Jul 2019 20:54:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8673: Add query option to force plan hints for insert queries
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13753 ) Change subject: IMPALA-8673: Add query option to force plan hints for insert queries .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3807/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13753 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1c3f213402b8e4d1940f96738ad21edf800fa43a Gerrit-Change-Number: 13753 Gerrit-PatchSet: 3 Gerrit-Owner: Abhishek Rawat Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 02 Jul 2019 20:34:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13777 ) Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/370/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/13777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f Gerrit-Change-Number: 13777 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 02 Jul 2019 20:32:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL
Hello Bharath Vissapragada, Vihang Karajgaonkar, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13777 to look at the new patch set (#2). Change subject: IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL .. IMPALA-8519: [DOCS] Doc the limitation in insert events from SparkSQL Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f --- M docs/topics/impala_metadata.xml 1 file changed, 10 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/77/13777/2 -- To view, visit http://gerrit.cloudera.org:8080/13777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I36cfc7e592ed2588a8c1f8375033d60492b27a4f Gerrit-Change-Number: 13777 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-8734: Reload table schema on TBLPROPERTIES change
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13785 ) Change subject: IMPALA-8734: Reload table schema on TBLPROPERTIES change .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3806/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13785 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2a43a962c2a456f3ddc078b2924f551fccb5c2ad Gerrit-Change-Number: 13785 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 02 Jul 2019 20:31:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8734: Reload table schema on TBLPROPERTIES change
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13785 ) Change subject: IMPALA-8734: Reload table schema on TBLPROPERTIES change .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13785 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2a43a962c2a456f3ddc078b2924f551fccb5c2ad Gerrit-Change-Number: 13785 Gerrit-PatchSet: 5 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 02 Jul 2019 20:13:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8734: Reload table schema on TBLPROPERTIES change
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13785 ) Change subject: IMPALA-8734: Reload table schema on TBLPROPERTIES change .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4575/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13785 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2a43a962c2a456f3ddc078b2924f551fccb5c2ad Gerrit-Change-Number: 13785 Gerrit-PatchSet: 5 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 02 Jul 2019 20:13:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8734: Reload table schema on TBLPROPERTIES change
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/13785 ) Change subject: IMPALA-8734: Reload table schema on TBLPROPERTIES change .. Patch Set 4: Code-Review+2 patch looks good to me. -- To view, visit http://gerrit.cloudera.org:8080/13785 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2a43a962c2a456f3ddc078b2924f551fccb5c2ad Gerrit-Change-Number: 13785 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 02 Jul 2019 20:11:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8673: Add query option to force plan hints for insert queries
Abhishek Rawat has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/13753 ) Change subject: IMPALA-8673: Add query option to force plan hints for insert queries .. IMPALA-8673: Add query option to force plan hints for insert queries IMPALA-5293 enabled the pre-insert clustering by default. This could cause performance regression and this change provides a query option for setting default hints for INSERT statement. New query option 'DEFAULT_HINTS_INSERT_STATEMENT' was added. It also supports adding multiple supported hints when separated by ':' set DEFAULT_HINTS_INSERT_STATEMENT=[clustered|noclustered]; set DEFAULT_HINTS_INSERT_STATEMENT=[shuffle|noshuffle]; set DEFAULT_HINTS_INSERT_STATEMENT= [clustered|noclustered]:[shuffle|noshuffle]; If a given insert statement already has plan hints in the query text, the default hints, if any, are all ignored. This is because, if a query has plan hints specified by the user, we don't want to override it. When a default hint is set, and there is an INSERT statement without any plan hints in the query text, the default hints have the same affect as they would have had, if they were applied as plan hints in the query text. So these default hints have the same application and restrictions as the existing plan hints for INSERT statement. The default hints apply to HDFS and Kudu table formats and are ignored for HBase table format. Testing: - Added unit tests in AnalyzeDDLTest for CTAS. - Added unit tests in AnalyzeStmtsTest for insert statements. - Added unit tests in PlannerTest validating the plan for various scenarios involving different combinations of default hints. Change-Id: I1c3f213402b8e4d1940f96738ad21edf800fa43a --- M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java A testdata/workloads/functional-planner/queries/PlannerTest/insert-default-clustered-noshuffle.test A testdata/workloads/functional-planner/queries/PlannerTest/insert-default-clustered-shuffle.test A testdata/workloads/functional-planner/queries/PlannerTest/insert-default-clustered.test A testdata/workloads/functional-planner/queries/PlannerTest/insert-default-noclustered-noshuffle.test A testdata/workloads/functional-planner/queries/PlannerTest/insert-default-noclustered-shuffle.test A testdata/workloads/functional-planner/queries/PlannerTest/insert-default-noclustered.test A testdata/workloads/functional-planner/queries/PlannerTest/insert-default-noshuffle.test A testdata/workloads/functional-planner/queries/PlannerTest/insert-default-shuffle.test 17 files changed, 1,359 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/53/13753/3 -- To view, visit http://gerrit.cloudera.org:8080/13753 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1c3f213402b8e4d1940f96738ad21edf800fa43a Gerrit-Change-Number: 13753 Gerrit-PatchSet: 3 Gerrit-Owner: Abhishek Rawat Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8673: Add query option to force plan hints for insert queries
Abhishek Rawat has posted comments on this change. (
http://gerrit.cloudera.org:8080/13753 )
Change subject: IMPALA-8673: Add query option to force plan hints for insert
queries
..
Patch Set 2:
(3 comments)
Thanks for reviewing the code. Please review the latest patch.
http://gerrit.cloudera.org:8080/#/c/13753/2/fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
File fe/src/main/java/org/apache/impala/analysis/InsertStmt.java:
http://gerrit.cloudera.org:8080/#/c/13753/2/fe/src/main/java/org/apache/impala/analysis/InsertStmt.java@854
PS2, Line 854: planHints_.add(new PlanHint(hint));
> nit: planHints_.add(new PlanHint(hint.trim()));
Done
http://gerrit.cloudera.org:8080/#/c/13753/2/fe/src/main/java/org/apache/impala/analysis/InsertStmt.java@842
PS2, Line 842: if (planHints_.isEmpty() &&
: ((table_ instanceof FeHBaseTable) ||
:
!(analyzer.getQueryOptions().isSetDefault_hints_insert_statement( return;
:
: // Set up the plan hints from query option
DEFAULT_HINTS_INSERT_STATEMENT.
: // Default hint is ignored, if the original statement had
hints.
: if (planHints_.isEmpty() &&
:
analyzer.getQueryOptions().isSetDefault_hints_insert_statement()) {
: String defaultHints =
:
analyzer.getQueryOptions().getDefault_hints_insert_statement();
: for (String hint: defaultHints.trim().split(":")) {
: hint = hint.trim();
: planHints_.add(new PlanHint(hint));
: }
: }
:
: if (table_ instanceof FeHBaseTable) {
: throw new AnalysisException(String.format("INSERT hints
are only supported for " +
: "inserting into Hdfs and Kudu tables: %s",
getTargetTableName()));
: }
> I think this can be formatted to be more readable. The checks on the hbase
Done
http://gerrit.cloudera.org:8080/#/c/13753/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java:
http://gerrit.cloudera.org:8080/#/c/13753/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java@2038
PS2, Line 2038: AnalyzesOk(String.format("insert into table
functional_kudu.alltypes " +
> Mind adding some checks that
Very good point. I have added the new checks.
--
To view, visit http://gerrit.cloudera.org:8080/13753
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1c3f213402b8e4d1940f96738ad21edf800fa43a
Gerrit-Change-Number: 13753
Gerrit-PatchSet: 2
Gerrit-Owner: Abhishek Rawat
Gerrit-Reviewer: Abhishek Rawat
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Tue, 02 Jul 2019 19:52:50 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8734: Reload table schema on TBLPROPERTIES change
Fredy Wijaya has posted comments on this change. (
http://gerrit.cloudera.org:8080/13785 )
Change subject: IMPALA-8734: Reload table schema on TBLPROPERTIES change
..
Patch Set 4:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/13785/3/tests/metadata/test_ddl.py
File tests/metadata/test_ddl.py:
http://gerrit.cloudera.org:8080/#/c/13785/3/tests/metadata/test_ddl.py@697
PS3, Line 697: self.execute_query_expect_success(self.client, "create table
{0}.{1} (c1 string)"
> Is there a reason for not creating a new test? The new code seems to be com
No reason, I thought it would be simpler, but yeah it seems creating a new test
is better. Done.
http://gerrit.cloudera.org:8080/#/c/13785/3/tests/metadata/test_ddl.py@702
PS3, Line 702: ct_s
> Using a less "null like" value could make the test's intention clearer - it
I was following the test case, but yeah using a different value is more
readable. Done.
http://gerrit.cloudera.org:8080/#/c/13785/3/tests/metadata/test_ddl.py@701
PS3, Line 701:file_data="\nfoo\n")
: self.execute_query_expect_success(self.client,
: "alter table {0}.{1} set
tblproperties"
:
"('serialization.null.format'='foo')"
: .format(unique_database,
tbl_name))
: result = self.execute_query_expect_success(self.client,
:"select * from
{0}.{1}"
:
.format(unique_database, tbl_name))
: assert len(result.data) == 2
: assert result.data[0] == ''
> First I thought that it would be nice to create a utility function for this
TIL. Done.
--
To view, visit http://gerrit.cloudera.org:8080/13785
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2a43a962c2a456f3ddc078b2924f551fccb5c2ad
Gerrit-Change-Number: 13785
Gerrit-PatchSet: 4
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Anurag Mantripragada
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Tue, 02 Jul 2019 19:52:12 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8734: Reload table schema on TBLPROPERTIES change
Fredy Wijaya has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/13785 ) Change subject: IMPALA-8734: Reload table schema on TBLPROPERTIES change .. IMPALA-8734: Reload table schema on TBLPROPERTIES change Prior to this patch, an INVALIDATE METADATA was required when altering the TBLPROPERTIES for the changes to take effect. With this patch the table schema is automatically reloaded on TBLPROPERTIES change. Testing: - Added a new test in test_ddl.py - Ran test_ddl.py Change-Id: I2a43a962c2a456f3ddc078b2924f551fccb5c2ad --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M tests/metadata/test_ddl.py 2 files changed, 20 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/85/13785/4 -- To view, visit http://gerrit.cloudera.org:8080/13785 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2a43a962c2a456f3ddc078b2924f551fccb5c2ad Gerrit-Change-Number: 13785 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8543: More diagnostics for TAcceptQueueServer
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13790 ) Change subject: IMPALA-8543: More diagnostics for TAcceptQueueServer .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3805/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13790 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I33b32352b457a2c8ec7bae6da46bb9c555dc9c36 Gerrit-Change-Number: 13790 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Tue, 02 Jul 2019 19:19:54 + Gerrit-HasComments: No
[Impala-ASF-CR] build: use thin static archives
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/13775 ) Change subject: build: use thin static archives .. Patch Set 1: > Patch Set 1: > > I tried this out and I couldn't tell a difference when doing buildall.sh > -skiptests. Can you include a measurement of what got faster? hmm, it should make the generated .a files much smaller and thus link faster. Let me run some tests, maybe I put this in the wrong spot in the CMakeLists.txt file (I copied it from Kudu where we saw a speedup but will admit I didn't verify it in Impala) -- To view, visit http://gerrit.cloudera.org:8080/13775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3b54b5be8658c951914758c406cca55d4cc1756e Gerrit-Change-Number: 13775 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 02 Jul 2019 19:17:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7734: Catalog and Statestore memz page shows useless memory
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13670 ) Change subject: IMPALA-7734: Catalog and Statestore memz page shows useless memory .. IMPALA-7734: Catalog and Statestore memz page shows useless memory The MemTracker is removed from StatestoreD main and CatalogD main as it is not used. The /memz page related metrics are refactored to a a separate file. When the MemTracker is not available during callback initialization it should not be part of the callback either. Therefore, when MemTracker is not part of the callback it is not displayed. Before/after pictures of the /memz page can be found in the Jira. Change-Id: I1a42434b98aec1d9e0be6ae52325049378fdde23 Reviewed-on: http://gerrit.cloudera.org:8080/13670 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/catalog/catalogd-main.cc M be/src/runtime/exec-env.cc M be/src/statestore/statestored-main.cc M be/src/util/CMakeLists.txt M be/src/util/default-path-handlers.cc M be/src/util/default-path-handlers.h A be/src/util/memusage-path-handlers.cc A be/src/util/memusage-path-handlers.h M www/memz.tmpl 9 files changed, 193 insertions(+), 90 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/13670 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I1a42434b98aec1d9e0be6ae52325049378fdde23 Gerrit-Change-Number: 13670 Gerrit-PatchSet: 12 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-7734: Catalog and Statestore memz page shows useless memory
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13670 ) Change subject: IMPALA-7734: Catalog and Statestore memz page shows useless memory .. Patch Set 11: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13670 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1a42434b98aec1d9e0be6ae52325049378fdde23 Gerrit-Change-Number: 13670 Gerrit-PatchSet: 11 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Tue, 02 Jul 2019 19:14:11 + Gerrit-HasComments: No
[Impala-ASF-CR] build: use thin static archives
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/13775 ) Change subject: build: use thin static archives .. Patch Set 1: I tried this out and I couldn't tell a difference when doing buildall.sh -skiptests. Can you include a measurement of what got faster? -- To view, visit http://gerrit.cloudera.org:8080/13775 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3b54b5be8658c951914758c406cca55d4cc1756e Gerrit-Change-Number: 13775 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 02 Jul 2019 18:48:01 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] \N needs to be escaped in Sqoop for Impala to recognize as null
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13773 ) Change subject: [DOCS] \N needs to be escaped in Sqoop for Impala to recognize as null .. Patch Set 2: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/369/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/13773 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I581f532c2f865e93d8922661c389791f41805b64 Gerrit-Change-Number: 13773 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 02 Jul 2019 18:44:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8543: More diagnostics for TAcceptQueueServer
Michael Ho has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13790 Change subject: IMPALA-8543: More diagnostics for TAcceptQueueServer .. IMPALA-8543: More diagnostics for TAcceptQueueServer This change adds more logging information in TAcceptQueueServer to help diagnose issues at various stages of client connections establishment. Two new metrics are also added to measure the connection setup time and the wait time for service threads to be available. Change-Id: I33b32352b457a2c8ec7bae6da46bb9c555dc9c36 --- M be/src/rpc/TAcceptQueueServer.cpp M be/src/rpc/TAcceptQueueServer.h M common/thrift/metrics.json 3 files changed, 180 insertions(+), 22 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/13790/1 -- To view, visit http://gerrit.cloudera.org:8080/13790 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I33b32352b457a2c8ec7bae6da46bb9c555dc9c36 Gerrit-Change-Number: 13790 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Ho
[Impala-ASF-CR] [DOCS] \N needs to be escaped in Sqoop for Impala to recognize as null
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13773 ) Change subject: [DOCS] \N needs to be escaped in Sqoop for Impala to recognize as null .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/369/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/13773 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I581f532c2f865e93d8922661c389791f41805b64 Gerrit-Change-Number: 13773 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 02 Jul 2019 18:21:46 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] \N needs to be escaped in Sqoop for Impala to recognize as null
Hello Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13773 to look at the new patch set (#2). Change subject: [DOCS] \N needs to be escaped in Sqoop for Impala to recognize as null .. [DOCS] \N needs to be escaped in Sqoop for Impala to recognize as null Change-Id: I581f532c2f865e93d8922661c389791f41805b64 --- M docs/topics/impala_txtfile.xml 1 file changed, 28 insertions(+), 23 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/73/13773/2 -- To view, visit http://gerrit.cloudera.org:8080/13773 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I581f532c2f865e93d8922661c389791f41805b64 Gerrit-Change-Number: 13773 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] [DOCS] \N needs to be escaped in Sqoop for Impala to recognize as null
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/13773 ) Change subject: [DOCS] \N needs to be escaped in Sqoop for Impala to recognize as null .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/13773/1/docs/topics/impala_txtfile.xml File docs/topics/impala_txtfile.xml: http://gerrit.cloudera.org:8080/#/c/13773/1/docs/topics/impala_txtfile.xml@308 PS1, Line 308: --null-string '\\N' --null-non-string : '\\N' > On the commandline, there is a difference between single and double quotes: In Impala shell, only \\N worked both with single and double quotes. http://gerrit.cloudera.org:8080/#/c/13773/1/docs/topics/impala_txtfile.xml@392 PS1, Line 392: specify : to represent > The "\N" in the old version got dropped, so this is curren Done http://gerrit.cloudera.org:8080/#/c/13773/1/docs/topics/impala_txtfile.xml@394 PS1, Line 394: needs to be escaped as '\\N' when : used in a string. > I'm a bit confused about what this is trying to get across. In the actual t Done -- To view, visit http://gerrit.cloudera.org:8080/13773 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I581f532c2f865e93d8922661c389791f41805b64 Gerrit-Change-Number: 13773 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 02 Jul 2019 18:21:44 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8341: [DOCS] Added a note about the experiemental feature
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13788 ) Change subject: IMPALA-8341: [DOCS] Added a note about the experiemental feature .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/368/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/13788 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If9f5795a28738f7df435a3bea06ffa1a216fd04e Gerrit-Change-Number: 13788 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Tue, 02 Jul 2019 18:10:42 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] \N needs to be escaped in Sqoop for Impala to recognize as null
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/13773 ) Change subject: [DOCS] \N needs to be escaped in Sqoop for Impala to recognize as null .. Patch Set 1: (3 comments) What is the origin of this change? Did someone complain about this? http://gerrit.cloudera.org:8080/#/c/13773/1/docs/topics/impala_txtfile.xml File docs/topics/impala_txtfile.xml: http://gerrit.cloudera.org:8080/#/c/13773/1/docs/topics/impala_txtfile.xml@308 PS1, Line 308: --null-string '\\N' --null-non-string : '\\N' On the commandline, there is a difference between single and double quotes: $ echo "\\N" \N $ echo '\\N' \\N In this case, I believe it should be either double quotes with the escaping or single quotes without the escaping. i.e. "\\N" or '\N' http://gerrit.cloudera.org:8080/#/c/13773/1/docs/topics/impala_txtfile.xml@392 PS1, Line 392: specify : to represent The "\N" in the old version got dropped, so this is currently: "specify to represent" I think it should be: "specify \N to represent" http://gerrit.cloudera.org:8080/#/c/13773/1/docs/topics/impala_txtfile.xml@394 PS1, Line 394: needs to be escaped as '\\N' when : used in a string. I'm a bit confused about what this is trying to get across. In the actual text file, it should be \N. Users can write text files several different ways. Some require escaping, some don't. I think I would rather leave this paragraph the way it was. -- To view, visit http://gerrit.cloudera.org:8080/13773 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I581f532c2f865e93d8922661c389791f41805b64 Gerrit-Change-Number: 13773 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 02 Jul 2019 18:10:14 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8734: Reload table schema on TBLPROPERTIES change
Anurag Mantripragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/13785 ) Change subject: IMPALA-8734: Reload table schema on TBLPROPERTIES change .. Patch Set 3: Code-Review+1 Looks good to me -- To view, visit http://gerrit.cloudera.org:8080/13785 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2a43a962c2a456f3ddc078b2924f551fccb5c2ad Gerrit-Change-Number: 13785 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 02 Jul 2019 17:50:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8734: Reload table schema on TBLPROPERTIES change
Csaba Ringhofer has posted comments on this change. (
http://gerrit.cloudera.org:8080/13785 )
Change subject: IMPALA-8734: Reload table schema on TBLPROPERTIES change
..
Patch Set 3:
(3 comments)
I wrote some comments for the test, but probably the simplest way would be to
move the whole logic to alter_table.test. Some strings could be inserted to the
table, check select *, then serialization.null.format could be set to one of
the values, then check select * again.
http://gerrit.cloudera.org:8080/#/c/13785/3/tests/metadata/test_ddl.py
File tests/metadata/test_ddl.py:
http://gerrit.cloudera.org:8080/#/c/13785/3/tests/metadata/test_ddl.py@697
PS3, Line 697: # IMPALA-8734: Force a table schema reload when setting
table properties.
Is there a reason for not creating a new test? The new code seems to be
completely independent from the earlier things in
test_create_alter_tbl_properties
http://gerrit.cloudera.org:8080/#/c/13785/3/tests/metadata/test_ddl.py@702
PS3, Line 702: null
Using a less "null like" value could make the test's intention clearer - it was
not obvious for me that "null" is not interpreted as NULL by default.
http://gerrit.cloudera.org:8080/#/c/13785/3/tests/metadata/test_ddl.py@701
PS3, Line 701: temp_file, temp_path = tempfile.mkstemp()
: os.write(temp_file, "\nnull\n")
: os.close(temp_file)
: subprocess.check_call(["hdfs", "dfs", "-put", "-f",
temp_path,
:
"/test-warehouse/{0}.db/{1}/f".format(unique_database,
:
tbl_name)])
: self.execute_query_expect_success(self.client,
: "alter table {0}.{1} set
tblproperties"
:
"('serialization.null.format'='null')"
: .format(unique_database,
tbl_name))
First I thought that it would be nice to create a utility function for this,
then I realized that there is already one:
https://github.com/apache/impala/blob/aee0abd76b762e57ce9f0a2e40a9a8b4f97dc986/tests/util/hdfs_util.py#L172
--
To view, visit http://gerrit.cloudera.org:8080/13785
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2a43a962c2a456f3ddc078b2924f551fccb5c2ad
Gerrit-Change-Number: 13785
Gerrit-PatchSet: 3
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Tue, 02 Jul 2019 17:50:33 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8341: [DOCS] Added a note about the experiemental feature
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13788 ) Change subject: IMPALA-8341: [DOCS] Added a note about the experiemental feature .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/368/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/13788 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If9f5795a28738f7df435a3bea06ffa1a216fd04e Gerrit-Change-Number: 13788 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 02 Jul 2019 17:50:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8341: [DOCS] Added a note about the experiemental feature
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13788 Change subject: IMPALA-8341: [DOCS] Added a note about the experiemental feature .. IMPALA-8341: [DOCS] Added a note about the experiemental feature Change-Id: If9f5795a28738f7df435a3bea06ffa1a216fd04e --- M docs/topics/impala_data_cache.xml 1 file changed, 4 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/13788/1 -- To view, visit http://gerrit.cloudera.org:8080/13788 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: If9f5795a28738f7df435a3bea06ffa1a216fd04e Gerrit-Change-Number: 13788 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-7322: Add storage wait time to profile
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13786 ) Change subject: IMPALA-7322: Add storage wait time to profile .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3804/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13786 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7447f8c8e7e50eb71d18643859d2e3de865368d2 Gerrit-Change-Number: 13786 Gerrit-PatchSet: 1 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Tue, 02 Jul 2019 17:43:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8734: Reload table schema on TBLPROPERTIES change
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13785 ) Change subject: IMPALA-8734: Reload table schema on TBLPROPERTIES change .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3803/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13785 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2a43a962c2a456f3ddc078b2924f551fccb5c2ad Gerrit-Change-Number: 13785 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 02 Jul 2019 17:32:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7322: Add storage wait time to profile
Yongzhi Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/13786 ) Change subject: IMPALA-7322: Add storage wait time to profile .. Patch Set 1: The original review is in https://gerrit.cloudera.org/c/13738/ -- To view, visit http://gerrit.cloudera.org:8080/13786 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7447f8c8e7e50eb71d18643859d2e3de865368d2 Gerrit-Change-Number: 13786 Gerrit-PatchSet: 1 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Tue, 02 Jul 2019 17:04:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7322: Add storage wait time to profile
Yongzhi Chen has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13786 Change subject: IMPALA-7322: Add storage wait time to profile .. IMPALA-7322: Add storage wait time to profile Add metrics to record storage wait time for operations with metadata load in catalog for hdfs, kudu and hbase tables. Pass storage wait time from catalog to fe through thrift and log total storage load time in query profile. Storage-load-time is the amount of time spent loading metadata from the underlying storage layer (e.g. S3, HDFS, Kudu, HBase), which does not include the amount of time spending loading data from HMS. Testing: Ran queries that can trigger all of, none of or some of the related tables loading. Check query profile for each query. Check catalog metrics for each table. Add unit tests to test_observability.py Ran all core tests. Sample output: Profile:(storage-load-time is the added property): After ran a hbase query (Metadata load finished is divided into several lines because of limitation of commit message): Query Compilation: 4s401ms - Metadata load started: 661.084us (661.084us) - Metadata load finished. loaded-tables=1/1 load-requests=1 catalog-updates=3 storage-load-time=233ms: 3s819ms (3s819ms) - Analysis finished: 3s820ms (763.979us) - Value transfer graph computed: 3s820ms (63.193us) Catalog metrics(this sample is from a hdfs table): storage-metadata-load-duration: Count: 1 Mean rate: 0.0085 1 min. rate: 0.032 5 min. rate: 0.1386 15 min. rate: 0.177 Min (msec): 111 Max (msec): 111 Mean (msec): 111.1802 Median (msec): 111.1802 75th-% (msec): 111.1802 95th-% (msec): 111.1802 99th-% (msec): 111.1802 Change-Id: I7447f8c8e7e50eb71d18643859d2e3de865368d2 --- M common/thrift/CatalogObjects.thrift M fe/src/main/java/org/apache/impala/analysis/StmtMetadataLoader.java M fe/src/main/java/org/apache/impala/catalog/HBaseTable.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/catalog/Table.java M tests/common/skip.py M tests/query_test/test_observability.py 8 files changed, 144 insertions(+), 30 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/13786/1 -- To view, visit http://gerrit.cloudera.org:8080/13786 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I7447f8c8e7e50eb71d18643859d2e3de865368d2 Gerrit-Change-Number: 13786 Gerrit-PatchSet: 1 Gerrit-Owner: Yongzhi Chen
[Impala-ASF-CR] IMPALA-8734: Reload table schema on TBLPROPERTIES change
Fredy Wijaya has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13785 Change subject: IMPALA-8734: Reload table schema on TBLPROPERTIES change .. IMPALA-8734: Reload table schema on TBLPROPERTIES change Prior to this patch, an INVALIDATE METADATA was required when altering the TBLPROPERTIES for the changes to take effect. With this patch the table schema is automatically reloaded on TBLPROPERTIES change. Testing: - Added a new test in test_ddl.py - Ran test_ddl.py Change-Id: I2a43a962c2a456f3ddc078b2924f551fccb5c2ad --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M tests/metadata/test_ddl.py 2 files changed, 25 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/85/13785/3 -- To view, visit http://gerrit.cloudera.org:8080/13785 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I2a43a962c2a456f3ddc078b2924f551fccb5c2ad Gerrit-Change-Number: 13785 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/13765 ) Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. Patch Set 3: (6 comments) Still trying to understand the overall approach here. The commit message seems to indicate that the NULL check is not necessary for complex types, but I don't really understand why; plus the JIRA description in IMPALA-8718 seems to indicate the NULL check is necessary. http://gerrit.cloudera.org:8080/#/c/13765/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13765/3//COMMIT_MSG@11 PS3, Line 11: make right side exprs nullable, it makes a wrapper nit: what is "it", the Impala planner? http://gerrit.cloudera.org:8080/#/c/13765/3//COMMIT_MSG@16 PS3, Line 16: cannot both be null and non-null. But during its This line is a bit confusing. Can you clarify what you mean here? http://gerrit.cloudera.org:8080/#/c/13765/3//COMMIT_MSG@17 PS3, Line 17: checking whether the wrapper needed, because of code what do you mean by "because of code limitation"? http://gerrit.cloudera.org:8080/#/c/13765/3/testdata/workloads/functional-query/queries/QueryTest/complex_joins.test File testdata/workloads/functional-query/queries/QueryTest/complex_joins.test: http://gerrit.cloudera.org:8080/#/c/13765/3/testdata/workloads/functional-query/queries/QueryTest/complex_joins.test@5 PS3, Line 5: create table array_tbl (col1 string, col2 array ) stored as parquet; databases such as "functional_parquet" already have tables such as "complextypestbl" that you can use. See "test_nested_types.py" for examples. http://gerrit.cloudera.org:8080/#/c/13765/3/testdata/workloads/functional-query/queries/QueryTest/complex_joins.test@35 PS3, Line 35: select t.col1, rank() over(order by t.col1) as rnk is the analytical function necessary to re-produce the error? otherwise I think it is just adding unnecessary complexity to the test case. http://gerrit.cloudera.org:8080/#/c/13765/3/tests/query_test/test_complex.py File tests/query_test/test_complex.py: http://gerrit.cloudera.org:8080/#/c/13765/3/tests/query_test/test_complex.py@23 PS3, Line 23: class TestComplex(ImpalaTestSuite): You can just add this test to test_nested_types.py instead of creating a new file. I think the pattern in the Impala codebase is to refer to array, map, and struct types as "nested" types, rather than "complex" types. -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 3 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Tue, 02 Jul 2019 15:55:31 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8710: Increase allowed bit width to 64 for bit packing
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/13737 ) Change subject: IMPALA-8710: Increase allowed bit width to 64 for bit packing .. Patch Set 5: Code-Review+1 (4 comments) http://gerrit.cloudera.org:8080/#/c/13737/5/be/src/util/bit-packing.inline.h File be/src/util/bit-packing.inline.h: http://gerrit.cloudera.org:8080/#/c/13737/5/be/src/util/bit-packing.inline.h@172 PS5, Line 172: constexpr int NO_WORDS = LAST_WORD_IDX - FIRST_WORD_IDX; nit: You don't use abbreviations at other places, so at first it reads like "no words". Maybe WORDS_TO_READ? http://gerrit.cloudera.org:8080/#/c/13737/5/be/src/util/bit-packing.inline.h@179 PS5, Line 179: constexpr bool CAN_READ_64 = FIRST_BIT_IDX - FIRST_BIT_OFFSET + 64 <= BIT_WIDTH * 32; nit: This is about not to read past the end of the buffer, right? Can you add comment for it? http://gerrit.cloudera.org:8080/#/c/13737/5/be/src/util/bit-stream-utils-test.cc File be/src/util/bit-stream-utils-test.cc: http://gerrit.cloudera.org:8080/#/c/13737/5/be/src/util/bit-stream-utils-test.cc@158 PS5, Line 158: int64_t nit: uint64_t? http://gerrit.cloudera.org:8080/#/c/13737/5/be/src/util/bit-stream-utils-test.cc@158 PS5, Line 158: 0LL very nit: sometimes in the code there is only L or UL, and sometimes LL, I think UL is the choice for most places. It doesn't really matter, but makes the reader to think for a sec why this specific suffix is used here. -- To view, visit http://gerrit.cloudera.org:8080/13737 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3ff3b387cbe8e41dd78c45411ef54de4afa8 Gerrit-Change-Number: 13737 Gerrit-PatchSet: 5 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 02 Jul 2019 15:48:48 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
Attila Jeges has posted comments on this change. (
http://gerrit.cloudera.org:8080/13722 )
Change subject: IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
..
Patch Set 2:
(16 comments)
http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/runtime/datetime-iso-sql-format-tokenizer.cc
File be/src/runtime/datetime-iso-sql-format-tokenizer.cc:
http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/runtime/datetime-iso-sql-format-tokenizer.cc@114
PS2, Line 114: ProcessNextToken
Mention in a comment that this function is doing greedy-matching and finds the
longest matching token.
http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/runtime/datetime-iso-sql-format-tokenizer.cc@122
PS2, Line 122: auto
const auto
http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/runtime/datetime-iso-sql-format-tokenizer.cc@140
PS2, Line 140: auto
const auto
http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/runtime/timestamp-parse-util.h
File be/src/runtime/timestamp-parse-util.h:
http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/runtime/timestamp-parse-util.h@67
PS2, Line 67: If it failed then
nit: Otherwise
http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/runtime/timestamp-parse-util.cc
File be/src/runtime/timestamp-parse-util.cc:
http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/runtime/timestamp-parse-util.cc@43
PS2, Line 43:
nit: Double space.
http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/runtime/timestamp-parse-util.cc@43
PS2, Line 43:
nit:double space.
http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/runtime/timestamp-parse-util.cc@130
PS2, Line 130: boost::posix_time::time_duration
No need for a fully qualified name, 'time_duration' is available in this
namespace.
http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/runtime/timestamp-parse-util.cc@150
PS2, Line 150: int day_offset = AdjustWithTimezone(t, dt_result.tz_offset);
Before this change, timezone-adjustment was done only when dt_ctx.has_time_toks
was set. Any reason you changed this behavior? Isn't this a breaking change?
http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/runtime/timestamp-parse-util.cc@231
PS2, Line 231: tok.len != 4
tok.len < 4
http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/runtime/timestamp-parse-util.cc@232
PS2, Line 232: int adjust_factor = std::pow(10, tok.len);
: num_val %= adjust_factor;
Isn't this a breaking change. "% 100" vs "% adjust_factor" ?
http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/service/impala-server.cc
File be/src/service/impala-server.cc:
http://gerrit.cloudera.org:8080/#/c/13722/2/be/src/service/impala-server.cc@1119
PS2, Line 1119:
query_ctx->__set_now_string(query_ctx->client_request.query_options.now_string);
Is this query option used for testing only? Please add a comment to explain.
http://gerrit.cloudera.org:8080/#/c/13722/2/common/thrift/Exprs.thrift
File common/thrift/Exprs.thrift:
http://gerrit.cloudera.org:8080/#/c/13722/2/common/thrift/Exprs.thrift@144
PS2, Line 144: TCastExpr
Consider renaming to TCastFormatExpr or something similar to emphasize that
this is only used for cast expressions with a format clause.
http://gerrit.cloudera.org:8080/#/c/13722/2/fe/src/main/cup/sql-parser.cup
File fe/src/main/cup/sql-parser.cup:
http://gerrit.cloudera.org:8080/#/c/13722/2/fe/src/main/cup/sql-parser.cup@2937
PS2, Line 2937: cast_format_expr
Consider renaming this to 'cast_format_val'
http://gerrit.cloudera.org:8080/#/c/13722/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java:
http://gerrit.cloudera.org:8080/#/c/13722/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java@3088
PS2, Line 3088: public void TestCastFormatClauseFromTimestamp() throws
AnalysisException {
: String to_timestamp_cast = "cast('05-01-2017' as
timestamp)";
: AnalysisError("select cast("+to_timestamp_cast+" as
DATETIME FORMAT 'MM-dd-')",
: "Unsupported data type: DATETIME");
: AnalysisError("select cast("+to_timestamp_cast+" as DATE
FORMAT 'MM-dd-')",
: "FORMAT clause is not applicable from TIMESTAMP to
DATE");
: AnalysisError("select cast("+to_timestamp_cast+" AS INT
FORMAT 'MM-dd-')",
: "FORMAT clause is not applicable from TIMESTAMP to
INT");
: AnalysisError("select cast("+to_timestamp_cast+" AS BOOLEAN
FORMAT 'MM-dd-')",
: "FORMAT clause is not applicable from TIMESTAMP to
BOOLEAN");
: AnalysisError("select cast("+to_timestamp_cast+" AS DOUBLE
FORMAT 'MM-dd-')",
: "FORMAT clause is not applicable from TIMESTAMP to
DOUBLE");
: AnalysisError("select cast("+to_timestamp_cast+" AS STRING
FORMAT '')",
:
[Impala-ASF-CR] IMPALA-7734: Catalog and Statestore memz page shows useless memory
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13670 ) Change subject: IMPALA-7734: Catalog and Statestore memz page shows useless memory .. Patch Set 11: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13670 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1a42434b98aec1d9e0be6ae52325049378fdde23 Gerrit-Change-Number: 13670 Gerrit-PatchSet: 11 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Tue, 02 Jul 2019 13:23:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7734: Catalog and Statestore memz page shows useless memory
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13670 ) Change subject: IMPALA-7734: Catalog and Statestore memz page shows useless memory .. Patch Set 11: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4574/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13670 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1a42434b98aec1d9e0be6ae52325049378fdde23 Gerrit-Change-Number: 13670 Gerrit-PatchSet: 11 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Tue, 02 Jul 2019 13:23:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7734: Catalog and Statestore memz page shows useless memory
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/13670 ) Change subject: IMPALA-7734: Catalog and Statestore memz page shows useless memory .. Patch Set 10: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13670 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1a42434b98aec1d9e0be6ae52325049378fdde23 Gerrit-Change-Number: 13670 Gerrit-PatchSet: 10 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Tue, 02 Jul 2019 13:22:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7734: Catalog and Statestore memz page shows useless memory
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13670 ) Change subject: IMPALA-7734: Catalog and Statestore memz page shows useless memory .. Patch Set 10: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3802/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13670 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1a42434b98aec1d9e0be6ae52325049378fdde23 Gerrit-Change-Number: 13670 Gerrit-PatchSet: 10 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Tue, 02 Jul 2019 13:20:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7734: Catalog and Statestore memz page shows useless memory
Tamas Mate has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/13670 ) Change subject: IMPALA-7734: Catalog and Statestore memz page shows useless memory .. IMPALA-7734: Catalog and Statestore memz page shows useless memory The MemTracker is removed from StatestoreD main and CatalogD main as it is not used. The /memz page related metrics are refactored to a a separate file. When the MemTracker is not available during callback initialization it should not be part of the callback either. Therefore, when MemTracker is not part of the callback it is not displayed. Before/after pictures of the /memz page can be found in the Jira. Change-Id: I1a42434b98aec1d9e0be6ae52325049378fdde23 --- M be/src/catalog/catalogd-main.cc M be/src/runtime/exec-env.cc M be/src/statestore/statestored-main.cc M be/src/util/CMakeLists.txt M be/src/util/default-path-handlers.cc M be/src/util/default-path-handlers.h A be/src/util/memusage-path-handlers.cc A be/src/util/memusage-path-handlers.h M www/memz.tmpl 9 files changed, 193 insertions(+), 90 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/70/13670/10 -- To view, visit http://gerrit.cloudera.org:8080/13670 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1a42434b98aec1d9e0be6ae52325049378fdde23 Gerrit-Change-Number: 13670 Gerrit-PatchSet: 10 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-7734: Catalog and Statestore memz page shows useless memory
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13670 ) Change subject: IMPALA-7734: Catalog and Statestore memz page shows useless memory .. Patch Set 9: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/3801/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/13670 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1a42434b98aec1d9e0be6ae52325049378fdde23 Gerrit-Change-Number: 13670 Gerrit-PatchSet: 9 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Tue, 02 Jul 2019 12:20:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7734: Catalog and Statestore memz page shows useless memory
Tamas Mate has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/13670 ) Change subject: IMPALA-7734: Catalog and Statestore memz page shows useless memory .. IMPALA-7734: Catalog and Statestore memz page shows useless memory The MemTracker is removed from StatestoreD main and CatalogD main as it is not used. The /memz page related metrics are refactored to a a separate file. When the MemTracker is not available during callback initialization it should not be part of the callback either. Therefore, when MemTracker is not part of the callback it is not displayed. Before/after pictures of the /memz page can be found in the Jira. Change-Id: I1a42434b98aec1d9e0be6ae52325049378fdde23 --- M be/src/catalog/catalogd-main.cc M be/src/statestore/statestored-main.cc M be/src/util/CMakeLists.txt M be/src/util/default-path-handlers.cc M be/src/util/default-path-handlers.h A be/src/util/memusage-path-handlers.cc A be/src/util/memusage-path-handlers.h M www/memz.tmpl 8 files changed, 194 insertions(+), 85 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/70/13670/9 -- To view, visit http://gerrit.cloudera.org:8080/13670 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1a42434b98aec1d9e0be6ae52325049378fdde23 Gerrit-Change-Number: 13670 Gerrit-PatchSet: 9 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tamas Mate
