[
https://issues.apache.org/jira/browse/IMPALA-9357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17063702#comment-17063702
]
ASF subversion and git services commented on IMPALA-9357:
---------------------------------------------------------
Commit f41710d1bf2d33e89b318cb2c83781be0bc83f42 in impala's branch
refs/heads/branch-3.4.0 from Vihang Karajgaonkar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=f41710d ]
IMPALA-9357: Fix race condition in alter_database event
The race condition is exposed intermittently, on certain builds which
causes test_event_processing::test_self_events test to fail. This
happens because we are checking for self-event identifiers in the Db
object without taking a lock. When a DDL like 'comment on
database is 'test'' is executed, it is possible that the event
processor thread is triggered as soon as the ALTER_DATABASE event is
generated. This may cause event processor fail the self-event detection
since the self-event identifiers are not yet added to the Db object.
The fix adds a Db lock similar to Table lock. Alter db operations
in CatalogOpExecutor now take db locks instead of metastoreDdlLock_
which makes it consistent with table locking protocol.
Testing:
1. Ran existing tests for events processor.
2. This test was failing on centos6 frequently (failed in 1/3 times).
After the fix I ran the test in a loop for 24 hrs (197 iterations) and
the test didn't fail.
3. Ran core tests with CDP and CDH builds.
Change-Id: I472fd8a55740769ee5cdb84e48422a4ab39a8d1e
Reviewed-on: http://gerrit.cloudera.org:8080/15260
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
(cherry picked from commit 1d27b91a36f687f958b02738ba7899652b2cfec7)
> Fix race condition in alter_database event
> ------------------------------------------
>
> Key: IMPALA-9357
> URL: https://issues.apache.org/jira/browse/IMPALA-9357
> Project: IMPALA
> Issue Type: Bug
> Reporter: Fang-Yu Rao
> Assignee: Vihang Karajgaonkar
> Priority: Blocker
> Labels: broken-build, flaky-test
>
> The EE test {{test_self_events}} seems flaky. In what follows the stacktrace
> is provided.
> {code:java}
> Stacktrace
> custom_cluster/test_event_processing.py:167: in test_self_events
> self.__run_self_events_test(unique_database, True)
> custom_cluster/test_event_processing.py:232: in __run_self_events_test
> self.__exec_sql_and_check_selfevent_counter(stmt, use_impala)
> custom_cluster/test_event_processing.py:388: in
> __exec_sql_and_check_selfevent_counter
> assert self_events_after > self_events
> E assert 1 > 1
> {code}
> Since this test was recently added by
> https://issues.apache.org/jira/browse/IMPALA-9101 and reviewed by
> [~stigahuang], maybe you could provide some insight into it? The JIRA is
> assigned to [~vihangk1] for now, but please feel free to re-assign it to
> others as appropriate. Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
