[Impala-ASF-CR] IMPALA-12683: Fix wrong event time for batched events
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20864 ) Change subject: IMPALA-12683: Fix wrong event time for batched events .. IMPALA-12683: Fix wrong event time for batched events When updating the last-synced-event-id after processing a batch of partition events, we use the last event id. We should do the same when updating last-synced-event-time. However, currently BatchPartitionEvent uses getEventTime() from the parent class. It actually returns the event time of the first event. We should override it to use the last event. Tests - Ran MetastoreEventsProcessorTest.testDisableEventSyncFlag 200 times. Change-Id: I82efe18dd28fe8af47f8c66cc8c5eb8e6f8dfd2b Reviewed-on: http://gerrit.cloudera.org:8080/20864 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java 1 file changed, 9 insertions(+), 0 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/20864 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I82efe18dd28fe8af47f8c66cc8c5eb8e6f8dfd2b Gerrit-Change-Number: 20864 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala
[Impala-ASF-CR] IMPALA-12683: Fix wrong event time for batched events
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20864 ) Change subject: IMPALA-12683: Fix wrong event time for batched events .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/20864 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I82efe18dd28fe8af47f8c66cc8c5eb8e6f8dfd2b Gerrit-Change-Number: 20864 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Comment-Date: Sat, 06 Jan 2024 04:42:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12643 (part 2): Fallback to safe libraries on error in resolve minidumps.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20863 ) Change subject: IMPALA-12643 (part 2): Fallback to safe libraries on error in resolve_minidumps.py .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/14894/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20863 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6bb4c9f65f9c27bb3b86c7ff2f3a6a48e258ef01 Gerrit-Change-Number: 20863 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Sat, 06 Jan 2024 04:40:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12643 (part 2): Fallback to safe libraries on error in resolve minidumps.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20863 ) Change subject: IMPALA-12643 (part 2): Fallback to safe libraries on error in resolve_minidumps.py .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/14893/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20863 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6bb4c9f65f9c27bb3b86c7ff2f3a6a48e258ef01 Gerrit-Change-Number: 20863 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Sat, 06 Jan 2024 04:37:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12643 (part 2): Fallback to safe libraries on error in resolve minidumps.py
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/20863 ) Change subject: IMPALA-12643 (part 2): Fallback to safe libraries on error in resolve_minidumps.py .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/20863/1/bin/resolve_minidumps.py File bin/resolve_minidumps.py: http://gerrit.cloudera.org:8080/#/c/20863/1/bin/resolve_minidumps.py@423 PS1, Line 423: return > Might be useful to dump the exception as well. Good idea, added the traceback for the exception -- To view, visit http://gerrit.cloudera.org:8080/20863 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6bb4c9f65f9c27bb3b86c7ff2f3a6a48e258ef01 Gerrit-Change-Number: 20863 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Sat, 06 Jan 2024 04:13:46 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12643 (part 2): Fallback to safe libraries on error in resolve minidumps.py
Hello Michael Smith, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20863 to look at the new patch set (#3). Change subject: IMPALA-12643 (part 2): Fallback to safe libraries on error in resolve_minidumps.py .. IMPALA-12643 (part 2): Fallback to safe libraries on error in resolve_minidumps.py Since resolve_minidumps.py's call to minidump_stackwalk can go haywire due to bad symbols in shared libraries, this adds a fallback mechanism where it tries again with a "safe" list of shared libraries. These are limited to the ones that make the most difference in resolving minidumps (libc, libstdc++, and libjvm). The list of safe libraries can be customized via the --safe_library_list. Testing: - Verified that this uses the fallback on Centos 7 and resolves the minidumps successfully. Change-Id: I6bb4c9f65f9c27bb3b86c7ff2f3a6a48e258ef01 --- M bin/resolve_minidumps.py 1 file changed, 73 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/20863/3 -- To view, visit http://gerrit.cloudera.org:8080/20863 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6bb4c9f65f9c27bb3b86c7ff2f3a6a48e258ef01 Gerrit-Change-Number: 20863 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith
[Impala-ASF-CR] IMPALA-12643 (part 2): Fallback to safe libraries on error in resolve minidumps.py
Hello Michael Smith, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20863 to look at the new patch set (#2). Change subject: IMPALA-12643 (part 2): Fallback to safe libraries on error in resolve_minidumps.py .. IMPALA-12643 (part 2): Fallback to safe libraries on error in resolve_minidumps.py Since resolve_minidumps.py's call to minidump_stackwalk can go haywire due to bad symbols in shared libraries, this adds a fallback mechanism where it tries again with a "safe" list of shared libraries. These are limited to the ones that make the most difference in resolving minidumps (libc, libstdc++, and libjvm). The list of safe libraries can be customized via the --safe_library_list. Testing: - Verified that this uses the fallback on Centos 7 and resolves the minidumps successfully. Change-Id: I6bb4c9f65f9c27bb3b86c7ff2f3a6a48e258ef01 --- M bin/resolve_minidumps.py 1 file changed, 73 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/20863/2 -- To view, visit http://gerrit.cloudera.org:8080/20863 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6bb4c9f65f9c27bb3b86c7ff2f3a6a48e258ef01 Gerrit-Change-Number: 20863 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith
[Impala-ASF-CR] IMPALA-10048: Go parallel for dump breakpad symbols.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20802 ) Change subject: IMPALA-10048: Go parallel for dump_breakpad_symbols.py .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10129/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/20802 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I93427bb07f1d9718bd6df90acfd247210b54294d Gerrit-Change-Number: 20802 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Sat, 06 Jan 2024 04:14:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12643 (part 2): Fallback to safe libraries on error in resolve minidumps.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20863 ) Change subject: IMPALA-12643 (part 2): Fallback to safe libraries on error in resolve_minidumps.py .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/20863/2/bin/resolve_minidumps.py File bin/resolve_minidumps.py: http://gerrit.cloudera.org:8080/#/c/20863/2/bin/resolve_minidumps.py@424 PS2, Line 424: e flake8: F841 local variable 'e' is assigned to but never used -- To view, visit http://gerrit.cloudera.org:8080/20863 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6bb4c9f65f9c27bb3b86c7ff2f3a6a48e258ef01 Gerrit-Change-Number: 20863 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Sat, 06 Jan 2024 04:11:17 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20367 ) Change subject: IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs .. Patch Set 25: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/14892/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20367 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia250d0a943838086c187e5cb7c60035e5a564bbf Gerrit-Change-Number: 20367 Gerrit-PatchSet: 25 Gerrit-Owner: Sai Hemanth Gantasala Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Comment-Date: Sat, 06 Jan 2024 02:11:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs
Hello Quanlong Huang, k.venureddy2...@gmail.com, Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20367 to look at the new patch set (#25). Change subject: IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs .. IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs The idea is that when any DDL/DML operation is performed by Impala, it also syncs the db/table to its latest event ID as per HMS. This way updates to a db/table's are applied in the same order as they appear in the Notification log table in HMS which ensures consistency. Currently catalogD applies any updates received from Impala clients in-place. Instead it should perform an HMS operation first and then replay all the HMS events since the last synced event id. Implementation: when the enable_sync_to_latest_event_on_ddls flag is set to true, we do the DDL/DML operation first, i.e., perform HMS operation and then sync the db/table in the catalogD's cache to the latest event in HMS for the corresponding db/table. Currently we fetch all events greater than the db/table's lastSyncEventId and filter them in the events processor to sync only the current db/table events. Once HIVE-27499 is implemented, we can directly fetch the events only for the respective db/table and process them. Currently, there is no efficient way to identify if there are pending events for a db/table. Set 'enable_sync_to_latest_event_on_ddls'to true to enable this feature. Performance impact: DDL/DML might need more time to execute due to fetching and applying other events for corresponding metadata object. Note: We don't modify the cache using MetastoreEventsProcessor for alter table rename operation as this is a complex operation regarding cache modification (IMPALA-12553 has more details about this). We also don't modify the cache this way for the truncate table operation (IMPALA-12636 has more details about this). We don't modify cache using above process for 'refresh table'/'invalidate metadata table' commands. Few more properties are added to the 'file_metadata_reload_properties' config like numFiles, numRows, totalSize e.t.c (see IMPALA-11534 for more details). Testing: 1) Added few tests in the MetaStoreEventProcessorForTest to verify this feature that simulates the metadata sync between HMS and Impala. 2) Added few tests in the CatalogHmsSyncToLatestEventIdTest class to the metadata sync between HMS end point, Catalog Metastore Server and Impala. The HMS end point serves as common interface to metadata changes outside the current Impala service such as Hive, Spark or other Impala service. Also verified the table lastSyncEventId is updated after the events are sync and confirmed that metastore event processor ignored these synced events. 3) Added some end-to-end tests in test_sync_to_latest_hms_events.py Change-Id: Ia250d0a943838086c187e5cb7c60035e5a564bbf --- M be/src/catalog/catalog-server.cc M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/catalog/TableLoader.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M fe/src/test/java/org/apache/impala/catalog/metastore/CatalogHmsSyncToLatestEventIdTest.java A tests/custom_cluster/test_sync_to_latest_hms_events.py A tests/metadata/__init__.py M tests/metadata/test_ddl.py M tests/metadata/test_ddl_base.py M tests/metadata/test_event_processing.py M tests/metadata/test_recover_partitions.py 18 files changed, 1,111 insertions(+), 368 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/67/20367/25 -- To view, visit http://gerrit.cloudera.org:8080/20367 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia250d0a943838086c187e5cb7c60035e5a564bbf Gerrit-Change-Number: 20367 Gerrit-PatchSet: 25 Gerrit-Owner: Sai Hemanth Gantasala Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala
[Impala-ASF-CR] IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs
Sai Hemanth Gantasala has posted comments on this change. ( http://gerrit.cloudera.org:8080/20367 ) Change subject: IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs .. Patch Set 25: (1 comment) http://gerrit.cloudera.org:8080/#/c/20367/24/tests/metadata/test_ddl.py File tests/metadata/test_ddl.py: http://gerrit.cloudera.org:8080/#/c/20367/24/tests/metadata/test_ddl.py@156 PS24, Line 156: @ > Can we address this by using a different name, e.g. "_test_truncate_cleans_ Ack -- To view, visit http://gerrit.cloudera.org:8080/20367 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia250d0a943838086c187e5cb7c60035e5a564bbf Gerrit-Change-Number: 20367 Gerrit-PatchSet: 25 Gerrit-Owner: Sai Hemanth Gantasala Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Comment-Date: Sat, 06 Jan 2024 01:44:31 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12683: Fix wrong event time for batched events
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20864 ) Change subject: IMPALA-12683: Fix wrong event time for batched events .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10128/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/20864 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I82efe18dd28fe8af47f8c66cc8c5eb8e6f8dfd2b Gerrit-Change-Number: 20864 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Comment-Date: Sat, 06 Jan 2024 00:03:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12683: Fix wrong event time for batched events
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20864 ) Change subject: IMPALA-12683: Fix wrong event time for batched events .. Patch Set 2: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/10127/ -- To view, visit http://gerrit.cloudera.org:8080/20864 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I82efe18dd28fe8af47f8c66cc8c5eb8e6f8dfd2b Gerrit-Change-Number: 20864 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Comment-Date: Fri, 05 Jan 2024 23:30:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12643 (part 2): Fallback to safe libraries on error in resolve minidumps.py
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20863 ) Change subject: IMPALA-12643 (part 2): Fallback to safe libraries on error in resolve_minidumps.py .. Patch Set 1: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/20863/1/bin/resolve_minidumps.py File bin/resolve_minidumps.py: http://gerrit.cloudera.org:8080/#/c/20863/1/bin/resolve_minidumps.py@423 PS1, Line 423: except Exception: Might be useful to dump the exception as well. -- To view, visit http://gerrit.cloudera.org:8080/20863 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6bb4c9f65f9c27bb3b86c7ff2f3a6a48e258ef01 Gerrit-Change-Number: 20863 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Fri, 05 Jan 2024 22:59:12 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/20367 ) Change subject: IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs .. Patch Set 24: (1 comment) http://gerrit.cloudera.org:8080/#/c/20367/24/tests/metadata/test_ddl.py File tests/metadata/test_ddl.py: http://gerrit.cloudera.org:8080/#/c/20367/24/tests/metadata/test_ddl.py@156 PS24, Line 156: @ > flake8: F811 redefinition of unused 'test_truncate_cleans_hdfs_files' from Can we address this by using a different name, e.g. "_test_truncate_cleans_hdfs_files_helper", or "_test_truncate_cleans_hdfs_files_impl"? BTW, the helper function name shouldn't start with "test". Otherwise, pytest will consider it as a test. https://docs.pytest.org/en/7.4.x/explanation/goodpractices.html#conventions-for-python-test-discovery -- To view, visit http://gerrit.cloudera.org:8080/20367 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia250d0a943838086c187e5cb7c60035e5a564bbf Gerrit-Change-Number: 20367 Gerrit-PatchSet: 24 Gerrit-Owner: Sai Hemanth Gantasala Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Comment-Date: Fri, 05 Jan 2024 23:01:22 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10048: Go parallel for dump breakpad symbols.py
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20802 ) Change subject: IMPALA-10048: Go parallel for dump_breakpad_symbols.py .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/20802 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I93427bb07f1d9718bd6df90acfd247210b54294d Gerrit-Change-Number: 20802 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Fri, 05 Jan 2024 23:01:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11909: Use absolute path when calling resolve minidumps.py
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20801 ) Change subject: IMPALA-11909: Use absolute path when calling resolve_minidumps.py .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/20801 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I063843554b52d3e8ed79ee32d9fd4c90d059c482 Gerrit-Change-Number: 20801 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Fri, 05 Jan 2024 23:00:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12643 (part 1): Limit memory consumption for resolve minidumps.py
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20862 ) Change subject: IMPALA-12643 (part 1): Limit memory consumption for resolve_minidumps.py .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/20862 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4db8facb8a037327228c3714e047e0d1f0fe1d94 Gerrit-Change-Number: 20862 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Fri, 05 Jan 2024 22:55:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20367 ) Change subject: IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs .. Patch Set 24: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/14891/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/20367 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia250d0a943838086c187e5cb7c60035e5a564bbf Gerrit-Change-Number: 20367 Gerrit-PatchSet: 24 Gerrit-Owner: Sai Hemanth Gantasala Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Comment-Date: Fri, 05 Jan 2024 22:46:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12681: Release file descriptors for partially written temporary files
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20852 ) Change subject: IMPALA-12681: Release file descriptors for partially written temporary files .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/14890/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20852 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I58a2bac419ced806d6f5a32bcdf24d79e078ab14 Gerrit-Change-Number: 20852 Gerrit-PatchSet: 6 Gerrit-Owner: Yida Wu Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Fri, 05 Jan 2024 22:43:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs
Sai Hemanth Gantasala has posted comments on this change. ( http://gerrit.cloudera.org:8080/20367 ) Change subject: IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs .. Patch Set 24: (3 comments) http://gerrit.cloudera.org:8080/#/c/20367/21//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/20367/21//COMMIT_MSG@27 PS21, Line 27: Set 'enable_sync_to_latest_event_on_ddls'to true to enable this > I mean this paragraph is incomplete. This patch doesn't set this flag to tr Ack http://gerrit.cloudera.org:8080/#/c/20367/21/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java: http://gerrit.cloudera.org:8080/#/c/20367/21/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@2676 PS21, Line 2676: org.apache.hadoop.hive.metastore.api.Table msTbl = null; > > When the table is reloaded and it is not a fullReload, then we need to se Ack http://gerrit.cloudera.org:8080/#/c/20367/23/tests/custom_cluster/test_sync_to_latest_hms_events.py File tests/custom_cluster/test_sync_to_latest_hms_events.py: http://gerrit.cloudera.org:8080/#/c/20367/23/tests/custom_cluster/test_sync_to_latest_hms_events.py@60 PS23, Line 60: TestDdlStatements.test_truncate_cleans_hdfs_files(self.client, > Can we use TestDdlStatements.test_truncate_cleans_hdfs_files() directly? Ack -- To view, visit http://gerrit.cloudera.org:8080/20367 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia250d0a943838086c187e5cb7c60035e5a564bbf Gerrit-Change-Number: 20367 Gerrit-PatchSet: 24 Gerrit-Owner: Sai Hemanth Gantasala Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Comment-Date: Fri, 05 Jan 2024 22:19:05 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs
Hello Quanlong Huang, k.venureddy2...@gmail.com, Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20367 to look at the new patch set (#24). Change subject: IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs .. IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs The idea is that when any DDL/DML operation is performed by Impala, it also syncs the db/table to its latest event ID as per HMS. This way updates to a db/table's are applied in the same order as they appear in the Notification log table in HMS which ensures consistency. Currently catalogD applies any updates received from Impala clients in-place. Instead it should perform an HMS operation first and then replay all the HMS events since the last synced event id. Implementation: when the enable_sync_to_latest_event_on_ddls flag is set to true, we do the DDL/DML operation first, i.e., perform HMS operation and then sync the db/table in the catalogD's cache to the latest event in HMS for the corresponding db/table. Currently we fetch all events greater than the db/table's lastSyncEventId and filter them in the events processor to sync only the current db/table events. Once HIVE-27499 is implemented, we can directly fetch the events only for the respective db/table and process them. Currently, there is no efficient way to identify if there are pending events for a db/table. Set 'enable_sync_to_latest_event_on_ddls'to true to enable this feature. Performance impact: DDL/DML might need more time to execute due to fetching and applying other events for corresponding metadata object. Note: We don't modify the cache using MetastoreEventsProcessor for alter table rename operation as this is a complex operation regarding cache modification (IMPALA-12553 has more details about this). We also don't modify the cache this way for the truncate table operation (IMPALA-12636 has more details about this). We don't modify cache using above process for 'refresh table'/'invalidate metadata table' commands. Few more properties are added to the 'file_metadata_reload_properties' config like numFiles, numRows, totalSize e.t.c (see IMPALA-11534 for more details). Testing: 1) Added few tests in the MetaStoreEventProcessorForTest to verify this feature that simulates the metadata sync between HMS and Impala. 2) Added few tests in the CatalogHmsSyncToLatestEventIdTest class to the metadata sync between HMS end point, Catalog Metastore Server and Impala. The HMS end point serves as common interface to metadata changes outside the current Impala service such as Hive, Spark or other Impala service. Also verified the table lastSyncEventId is updated after the events are sync and confirmed that metastore event processor ignored these synced events. 3) Added some end-to-end tests in test_sync_to_latest_hms_events.py Change-Id: Ia250d0a943838086c187e5cb7c60035e5a564bbf --- M be/src/catalog/catalog-server.cc M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/catalog/TableLoader.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M fe/src/test/java/org/apache/impala/catalog/metastore/CatalogHmsSyncToLatestEventIdTest.java A tests/custom_cluster/test_sync_to_latest_hms_events.py A tests/metadata/__init__.py M tests/metadata/test_ddl.py M tests/metadata/test_ddl_base.py M tests/metadata/test_event_processing.py M tests/metadata/test_recover_partitions.py 18 files changed, 1,107 insertions(+), 368 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/67/20367/24 -- To view, visit http://gerrit.cloudera.org:8080/20367 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia250d0a943838086c187e5cb7c60035e5a564bbf Gerrit-Change-Number: 20367 Gerrit-PatchSet: 24 Gerrit-Owner: Sai Hemanth Gantasala Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala
[Impala-ASF-CR] IMPALA-12681: Release file descriptors for partially written temporary files
Yida Wu has posted comments on this change. ( http://gerrit.cloudera.org:8080/20852 ) Change subject: IMPALA-12681: Release file descriptors for partially written temporary files .. Patch Set 6: (2 comments) http://gerrit.cloudera.org:8080/#/c/20852/5/tests/custom_cluster/test_scratch_disk.py File tests/custom_cluster/test_scratch_disk.py: http://gerrit.cloudera.org:8080/#/c/20852/5/tests/custom_cluster/test_scratch_disk.py@300 PS5, Line 300: for impalad in self.cluster.impalads: > You can get the list of impalads with Thanks, this is better. Done. http://gerrit.cloudera.org:8080/#/c/20852/5/tests/custom_cluster/test_scratch_disk.py@335 PS5, Line 335: @pytest.mark.execute_serially > It would make sense to check this on each scratch test, not just on remote Done -- To view, visit http://gerrit.cloudera.org:8080/20852 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I58a2bac419ced806d6f5a32bcdf24d79e078ab14 Gerrit-Change-Number: 20852 Gerrit-PatchSet: 6 Gerrit-Owner: Yida Wu Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Fri, 05 Jan 2024 22:17:25 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12681: Release file descriptors for partially written temporary files
Yida Wu has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/20852 ) Change subject: IMPALA-12681: Release file descriptors for partially written temporary files .. IMPALA-12681: Release file descriptors for partially written temporary files This patch fixes a bug where partially written temporary files are removed without releasing the file descriptors. This patch fixes the bug by adding a call to Close() of the local file writer during the Delete() of the DiskFile class, which could be called when the local buffer file is being evicted or the query ends, ensuring proper release of the file handle. Testing: Passed core tests. Additionally, a check has been added in the test test_scratch_disk.py to verify that there are no deleted files in the /proc/x/fd/ directory. Change-Id: I58a2bac419ced806d6f5a32bcdf24d79e078ab14 --- M be/src/runtime/io/disk-file.cc M be/src/runtime/io/disk-io-mgr-test.cc M be/src/runtime/io/local-file-writer.cc M be/src/runtime/io/local-file-writer.h M be/src/runtime/tmp-file-mgr-test.cc M tests/custom_cluster/test_scratch_disk.py 6 files changed, 41 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/52/20852/6 -- To view, visit http://gerrit.cloudera.org:8080/20852 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I58a2bac419ced806d6f5a32bcdf24d79e078ab14 Gerrit-Change-Number: 20852 Gerrit-PatchSet: 6 Gerrit-Owner: Yida Wu Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yida Wu
[Impala-ASF-CR] IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20367 ) Change subject: IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs .. Patch Set 24: (12 comments) http://gerrit.cloudera.org:8080/#/c/20367/24/tests/metadata/test_ddl.py File tests/metadata/test_ddl.py: http://gerrit.cloudera.org:8080/#/c/20367/24/tests/metadata/test_ddl.py@153 PS24, Line 153: t flake8: F821 undefined name 'test_truncate_cleans_hdfs_files' http://gerrit.cloudera.org:8080/#/c/20367/24/tests/metadata/test_ddl.py@156 PS24, Line 156: @ flake8: F811 redefinition of unused 'test_truncate_cleans_hdfs_files' from line 150 http://gerrit.cloudera.org:8080/#/c/20367/24/tests/metadata/test_ddl.py@252 PS24, Line 252: @ flake8: F811 redefinition of unused 'test_metadata_after_alter_database' from line 249 http://gerrit.cloudera.org:8080/#/c/20367/24/tests/metadata/test_ddl.py@270 PS24, Line 270: @ flake8: F811 redefinition of unused 'test_alter_table_set_owner' from line 267 http://gerrit.cloudera.org:8080/#/c/20367/24/tests/metadata/test_ddl.py@286 PS24, Line 286: @ flake8: F811 redefinition of unused 'test_alter_view_set_owner' from line 283 http://gerrit.cloudera.org:8080/#/c/20367/24/tests/metadata/test_ddl.py@498 PS24, Line 498: @ flake8: F811 redefinition of unused 'test_drop_partition_with_purge' from line 493 http://gerrit.cloudera.org:8080/#/c/20367/24/tests/metadata/test_ddl.py@646 PS24, Line 646: r flake8: E501 line too long (92 > 90 characters) http://gerrit.cloudera.org:8080/#/c/20367/24/tests/metadata/test_ddl.py@651 PS24, Line 651: @ flake8: F811 redefinition of unused 'test_create_alter_bulk_partition' from line 644 http://gerrit.cloudera.org:8080/#/c/20367/24/tests/metadata/test_ddl.py@698 PS24, Line 698: @ flake8: F811 redefinition of unused 'test_alter_table_set_fileformat' from line 694 http://gerrit.cloudera.org:8080/#/c/20367/24/tests/metadata/test_ddl.py@724 PS24, Line 724: @ flake8: F811 redefinition of unused 'test_alter_table_create_many_partitions' from line 721 http://gerrit.cloudera.org:8080/#/c/20367/24/tests/metadata/test_ddl.py@748 PS24, Line 748: @ flake8: F811 redefinition of unused 'test_create_alter_tbl_properties' from line 745 http://gerrit.cloudera.org:8080/#/c/20367/24/tests/metadata/test_ddl.py@808 PS24, Line 808: @ flake8: F811 redefinition of unused 'test_alter_tbl_properties_reload' from line 805 -- To view, visit http://gerrit.cloudera.org:8080/20367 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia250d0a943838086c187e5cb7c60035e5a564bbf Gerrit-Change-Number: 20367 Gerrit-PatchSet: 24 Gerrit-Owner: Sai Hemanth Gantasala Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Comment-Date: Fri, 05 Jan 2024 22:20:12 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12683: Fix wrong event time for batched events
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20864 ) Change subject: IMPALA-12683: Fix wrong event time for batched events .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/20864 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I82efe18dd28fe8af47f8c66cc8c5eb8e6f8dfd2b Gerrit-Change-Number: 20864 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Comment-Date: Fri, 05 Jan 2024 18:53:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12683: Fix wrong event time for batched events
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20864 ) Change subject: IMPALA-12683: Fix wrong event time for batched events .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10127/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/20864 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I82efe18dd28fe8af47f8c66cc8c5eb8e6f8dfd2b Gerrit-Change-Number: 20864 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Comment-Date: Fri, 05 Jan 2024 18:53:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12683: Fix wrong event time for batched events
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20864 ) Change subject: IMPALA-12683: Fix wrong event time for batched events .. Patch Set 1: Code-Review+2 Promote to +2. -- To view, visit http://gerrit.cloudera.org:8080/20864 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I82efe18dd28fe8af47f8c66cc8c5eb8e6f8dfd2b Gerrit-Change-Number: 20864 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Comment-Date: Fri, 05 Jan 2024 18:53:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12683: Fix wrong event time for batched events
Sai Hemanth Gantasala has posted comments on this change. ( http://gerrit.cloudera.org:8080/20864 ) Change subject: IMPALA-12683: Fix wrong event time for batched events .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/20864 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I82efe18dd28fe8af47f8c66cc8c5eb8e6f8dfd2b Gerrit-Change-Number: 20864 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Comment-Date: Fri, 05 Jan 2024 18:46:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12683: Fix wrong event time for batched events
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20864 ) Change subject: IMPALA-12683: Fix wrong event time for batched events .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/20864 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I82efe18dd28fe8af47f8c66cc8c5eb8e6f8dfd2b Gerrit-Change-Number: 20864 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Comment-Date: Fri, 05 Jan 2024 18:28:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12660: TSAN error in ImpalaServer::QueryStateRecord::Init
Michael Smith has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20842 ) Change subject: IMPALA-12660: TSAN error in ImpalaServer::QueryStateRecord::Init .. IMPALA-12660: TSAN error in ImpalaServer::QueryStateRecord::Init The new SimultaneousMultipleQueriesOneSession test within internal-server-test.cc has revealed a data race condition where the Impala web UI can read TExecRequest in the QueryDriver while the frontend is updating this object. Since the fix would require adding locks to the critical query planning path and the only impact is the UI showing slightly outdated data, this race condition is being ignored. Change-Id: I2c553576f03b7503f77f4aa1d3ea8086fff0e43b Reviewed-on: http://gerrit.cloudera.org:8080/20842 Reviewed-by: Michael Smith Tested-by: Michael Smith --- M bin/tsan-suppressions.txt 1 file changed, 10 insertions(+), 0 deletions(-) Approvals: Michael Smith: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/20842 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I2c553576f03b7503f77f4aa1d3ea8086fff0e43b Gerrit-Change-Number: 20842 Gerrit-PatchSet: 4 Gerrit-Owner: Jason Fehr Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jason Fehr Gerrit-Reviewer: Michael Smith
[Impala-ASF-CR] IMPALA-12660: TSAN error in ImpalaServer::QueryStateRecord::Init
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20842 ) Change subject: IMPALA-12660: TSAN error in ImpalaServer::QueryStateRecord::Init .. Patch Set 3: Verified+1 Code-Review+2 Verified in manual TSAN run. -- To view, visit http://gerrit.cloudera.org:8080/20842 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2c553576f03b7503f77f4aa1d3ea8086fff0e43b Gerrit-Change-Number: 20842 Gerrit-PatchSet: 3 Gerrit-Owner: Jason Fehr Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jason Fehr Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Fri, 05 Jan 2024 18:13:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12681: Release file descriptors for partially written temporary files
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/20852 ) Change subject: IMPALA-12681: Release file descriptors for partially written temporary files .. Patch Set 5: (2 comments) http://gerrit.cloudera.org:8080/#/c/20852/5/tests/custom_cluster/test_scratch_disk.py File tests/custom_cluster/test_scratch_disk.py: http://gerrit.cloudera.org:8080/#/c/20852/5/tests/custom_cluster/test_scratch_disk.py@300 PS5, Line 300: daemons = ['impalad'] You can get the list of impalads with self.cluster.impalads and get their pids with get_pid() http://gerrit.cloudera.org:8080/#/c/20852/5/tests/custom_cluster/test_scratch_disk.py@335 PS5, Line 335: self.check_deleted_file_fd() It would make sense to check this on each scratch test, not just on remote ones, e.g. by calling it in teardown_method -- To view, visit http://gerrit.cloudera.org:8080/20852 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I58a2bac419ced806d6f5a32bcdf24d79e078ab14 Gerrit-Change-Number: 20852 Gerrit-PatchSet: 5 Gerrit-Owner: Yida Wu Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Fri, 05 Jan 2024 15:55:18 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12356: Fix first ALTER PARTITION event from Hive could be treated as self event
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20486 ) Change subject: IMPALA-12356: Fix first ALTER_PARTITION event from Hive could be treated as self event .. Patch Set 16: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/20486 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I23c2affa3fe32c0b3843bff5e4c0018dce9060d3 Gerrit-Change-Number: 20486 Gerrit-PatchSet: 16 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Comment-Date: Fri, 05 Jan 2024 15:57:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12681: Release file descriptors for partially written temporary files
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20852 ) Change subject: IMPALA-12681: Release file descriptors for partially written temporary files .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/14889/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20852 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I58a2bac419ced806d6f5a32bcdf24d79e078ab14 Gerrit-Change-Number: 20852 Gerrit-PatchSet: 5 Gerrit-Owner: Yida Wu Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Fri, 05 Jan 2024 14:39:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12681: Release file descriptors for partially written temporary files
Yida Wu has posted comments on this change. ( http://gerrit.cloudera.org:8080/20852 ) Change subject: IMPALA-12681: Release file descriptors for partially written temporary files .. Patch Set 5: The new patch fixes some test cases, and added a check in test_scratch_disk.py to automatically check the path. Also manually tested the case to cancel the query in the middle during spilling, everything looks fine, I don't see any deleted files in /proc/x/fd. -- To view, visit http://gerrit.cloudera.org:8080/20852 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I58a2bac419ced806d6f5a32bcdf24d79e078ab14 Gerrit-Change-Number: 20852 Gerrit-PatchSet: 5 Gerrit-Owner: Yida Wu Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Fri, 05 Jan 2024 14:20:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12681: Release file descriptors for partially written temporary files
Yida Wu has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/20852 ) Change subject: IMPALA-12681: Release file descriptors for partially written temporary files .. IMPALA-12681: Release file descriptors for partially written temporary files This patch fixes a bug where partially written temporary files are removed without releasing the file descriptors. This patch fixes the bug by adding a call to Close() of the local file writer during the Delete() of the DiskFile class, which could be called when the local buffer file is being evicted or the query ends, ensuring proper release of the file handle. Testing: Passed core tests. Additionally, a check has been added in the test test_scratch_disk.py to verify that there are no deleted files in the /proc/x/fd/ directory. Change-Id: I58a2bac419ced806d6f5a32bcdf24d79e078ab14 --- M be/src/runtime/io/disk-file.cc M be/src/runtime/io/disk-io-mgr-test.cc M be/src/runtime/io/local-file-writer.cc M be/src/runtime/io/local-file-writer.h M be/src/runtime/tmp-file-mgr-test.cc M tests/custom_cluster/test_scratch_disk.py 6 files changed, 50 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/52/20852/5 -- To view, visit http://gerrit.cloudera.org:8080/20852 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I58a2bac419ced806d6f5a32bcdf24d79e078ab14 Gerrit-Change-Number: 20852 Gerrit-PatchSet: 5 Gerrit-Owner: Yida Wu Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yida Wu
[Impala-ASF-CR] IMPALA-12670: getIfPresent should throw the cause of error
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20853 ) Change subject: IMPALA-12670: getIfPresent should throw the cause of error .. IMPALA-12670: getIfPresent should throw the cause of error CatalogdMetaProvider maintains a map (a Guava cache) as its local catalog cache. It has a piggyback mechanism to load metadata from catalogd that when concurrent threads want to load the same content (identified by the same key) from catalogd, only one of them actually sends the request and load the result into the cache. Other threads wait and get the result when the work is done. The piggyback mechanism is implemented by putting a Future object as the value when the key doesn't exist in the cache. The Future object handles the loading. Other threads that want the same value just invoke Future.get() to wait. See more in the comments in loadWithCaching(). If there are any errors thrown in the loading process, Future.get() will encapsulate the error into an ExecutionException and throw it instead. The cause could be an InconsistentMetadataFetchException which indicates FE should retry the planning. It's handled in Frontend#getTExecRequest(). In loadWithCaching(), we try to throw the cause of the exception thrown from Future.get(). So the InconsistentMetadataFetchException can be handled as expected. However, in getIfPresent(), the error handling is inconsistent that it try to throw the current exception. That causes retriable failures can't be retried. Note that this is an existing bug but got more easy to be hitted after IMPALA-11501 because getIfPresent() is now used in LocalDb#getTableIfCached() which is used in many places. This patch fixes getIfPresent() to have the same logic of using the Future object (including error handling) as loadWithCaching(). Also adds more loggings in both catalogd and impalad sides when the lookup status is abnormal. In order to test the loading error more easily, this patch adds a hidden flag, inject_failure_ratio_in_catalog_fetch, to randomly inject retriable errors. Tests - Ran test_local_catalog_ddls_with_invalidate_metadata 700 times. - Add e2e test that will easily fail without this fix. Change-Id: I74268ba2bb700988107780e13ffbdbb4c767d09d Reviewed-on: http://gerrit.cloudera.org:8080/20853 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/exec/catalog-op-executor.cc M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java M tests/custom_cluster/test_local_catalog.py 4 files changed, 64 insertions(+), 12 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/20853 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I74268ba2bb700988107780e13ffbdbb4c767d09d Gerrit-Change-Number: 20853 Gerrit-PatchSet: 6 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto
[Impala-ASF-CR] IMPALA-12670: getIfPresent should throw the cause of error
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20853 ) Change subject: IMPALA-12670: getIfPresent should throw the cause of error .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/20853 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I74268ba2bb700988107780e13ffbdbb4c767d09d Gerrit-Change-Number: 20853 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Fri, 05 Jan 2024 14:00:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/20367 ) Change subject: IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs .. Patch Set 23: (3 comments) http://gerrit.cloudera.org:8080/#/c/20367/21//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/20367/21//COMMIT_MSG@27 PS21, Line 27: Set 'enable_sync_to_latest_event_on_ddls' to true. > We just need to set the flag 'enable_sync_to_latest_event_on_ddls' right? I I mean this paragraph is incomplete. This patch doesn't set this flag to true by default. Do you want to say "Set 'enable_sync_to_latest_event_on_ddls' to true to enable this feature" ? We'd better mention the performance impact on this flag, i.e. DDL/DML might need more time to execute due to fetching and applying other events. http://gerrit.cloudera.org:8080/#/c/20367/21/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java: http://gerrit.cloudera.org:8080/#/c/20367/21/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@2676 PS21, Line 2676: org.apache.hadoop.hive.metastore.api.Table msTbl = null; > When the table is reloaded and it is not a fullReload, then we need to > setRefreshEventId to currentHMSEventId so that older events can be skipped. What if the current reload just reloads the file metadata, and there is an older event that needs to reload the table schema? The older event shouldn't be skipped in this case. I think the explanation should be: if 'eventId' is 'currentHmsEventId', all the older events on this table should have been processed so it's ok to update the lastRefreshEventId. http://gerrit.cloudera.org:8080/#/c/20367/23/tests/custom_cluster/test_sync_to_latest_hms_events.py File tests/custom_cluster/test_sync_to_latest_hms_events.py: http://gerrit.cloudera.org:8080/#/c/20367/23/tests/custom_cluster/test_sync_to_latest_hms_events.py@60 PS23, Line 60: class_instance.test_truncate_cleans_hdfs_files(self.client, self.filesystem_client, Can we use TestDdlStatements.test_truncate_cleans_hdfs_files() directly? -- To view, visit http://gerrit.cloudera.org:8080/20367 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia250d0a943838086c187e5cb7c60035e5a564bbf Gerrit-Change-Number: 20367 Gerrit-PatchSet: 23 Gerrit-Owner: Sai Hemanth Gantasala Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Comment-Date: Fri, 05 Jan 2024 12:34:58 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12683: Fix wrong event time for batched events
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20864 ) Change subject: IMPALA-12683: Fix wrong event time for batched events .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/14888/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20864 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I82efe18dd28fe8af47f8c66cc8c5eb8e6f8dfd2b Gerrit-Change-Number: 20864 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 05 Jan 2024 12:24:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12683: Fix wrong event time for batched events
Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20864 Change subject: IMPALA-12683: Fix wrong event time for batched events .. IMPALA-12683: Fix wrong event time for batched events When updating the last-synced-event-id after processing a batch of partition events, we use the last event id. We should do the same when updating last-synced-event-time. However, currently BatchPartitionEvent uses getEventTime() from the parent class. It actually returns the event time of the first event. We should override it to use the last event. Tests - Ran MetastoreEventsProcessorTest.testDisableEventSyncFlag 200 times. Change-Id: I82efe18dd28fe8af47f8c66cc8c5eb8e6f8dfd2b --- M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java 1 file changed, 9 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/64/20864/1 -- To view, visit http://gerrit.cloudera.org:8080/20864 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I82efe18dd28fe8af47f8c66cc8c5eb8e6f8dfd2b Gerrit-Change-Number: 20864 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang
[Impala-ASF-CR] IMPALA-12681: Release file descriptors for partially written temporary files
Yida Wu has posted comments on this change. ( http://gerrit.cloudera.org:8080/20852 ) Change subject: IMPALA-12681: Release file descriptors for partially written temporary files .. Patch Set 4: Thanks Csaba and Abhishek for the review. The patch still needs to change one or two tests before merge. Still testing, will upload a new patch soon. -- To view, visit http://gerrit.cloudera.org:8080/20852 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I58a2bac419ced806d6f5a32bcdf24d79e078ab14 Gerrit-Change-Number: 20852 Gerrit-PatchSet: 4 Gerrit-Owner: Yida Wu Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Fri, 05 Jan 2024 11:42:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12356: Fix first ALTER PARTITION event from Hive could be treated as self event
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20486 ) Change subject: IMPALA-12356: Fix first ALTER_PARTITION event from Hive could be treated as self event .. Patch Set 16: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10126/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/20486 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I23c2affa3fe32c0b3843bff5e4c0018dce9060d3 Gerrit-Change-Number: 20486 Gerrit-PatchSet: 16 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Comment-Date: Fri, 05 Jan 2024 11:18:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12681: Release file descriptors for partially written temporary files
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/20852 ) Change subject: IMPALA-12681: Release file descriptors for partially written temporary files .. Patch Set 4: Code-Review+2 it can be merged from my side - tests could be considered in a follow up Jira. Ideally Impala would never have hanging deleted files, so we could check after test runs that there are no FDs like this -- To view, visit http://gerrit.cloudera.org:8080/20852 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I58a2bac419ced806d6f5a32bcdf24d79e078ab14 Gerrit-Change-Number: 20852 Gerrit-PatchSet: 4 Gerrit-Owner: Yida Wu Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Fri, 05 Jan 2024 10:48:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12356: Fix first ALTER PARTITION event from Hive could be treated as self event
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20486 ) Change subject: IMPALA-12356: Fix first ALTER_PARTITION event from Hive could be treated as self event .. Patch Set 16: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/10124/ -- To view, visit http://gerrit.cloudera.org:8080/20486 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I23c2affa3fe32c0b3843bff5e4c0018dce9060d3 Gerrit-Change-Number: 20486 Gerrit-PatchSet: 16 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sai Hemanth Gantasala Gerrit-Comment-Date: Fri, 05 Jan 2024 09:43:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12670: getIfPresent should throw the cause of error
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20853 ) Change subject: IMPALA-12670: getIfPresent should throw the cause of error .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/20853 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I74268ba2bb700988107780e13ffbdbb4c767d09d Gerrit-Change-Number: 20853 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Fri, 05 Jan 2024 09:20:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12670: getIfPresent should throw the cause of error
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20853 ) Change subject: IMPALA-12670: getIfPresent should throw the cause of error .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10125/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/20853 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I74268ba2bb700988107780e13ffbdbb4c767d09d Gerrit-Change-Number: 20853 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Fri, 05 Jan 2024 09:20:30 + Gerrit-HasComments: No
