Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20507 )
Change subject: IMPALA-12460: Add lag and histogram of event processing in the log ...................................................................... IMPALA-12460: Add lag and histogram of event processing in the log This patch logs the lag of the event processing which is (latestEventTime - lastSyncedEventTime) at the end of processing an event batch. If the batch is slow to process, we also log the top-10 expensive events and the top-10 targets that contribute to this. Admins can decide whether to disable event processing on some tables. For table and partition level events, the target name is the table name. For db level events, the target name is the db name. For events that don't have db/table names, e.g. COMMIT_TXN, the target name is a constant string - "CLUSTER_WIDE". Log parsing example for expensive events: -- In shell: $ grep 'expensive events' catalogd.INFO > expensive-events.log -- In Python: import re EVENT_PATTERN = re.compile(r"\(type=(\w+), id=(\d+), target=([\w\.]+), duration_ms=(\d+)\)") re.findall(EVENT_PATTERN, log_line) Log parsing example for top targets in event processing: -- In shell: $ grep 'targets in event processing' catalogd.INFO > slow-targets.log -- In Python: import re re.findall(r"\(target=([\w\.]+), duration_ms=(\d+)\)", log_line) This patch also adds a thread annotation on the event-processor thread so we know what event is under processing using the jstacks. The unit of the event time is second. This patch fixes some variable names that show them with ms. Tests: - Manually add some delay in event processing, verify the logs and jstacks. Change-Id: Ib9421b5e26bfa2324217ec9695fbd81636727d22 Reviewed-on: http://gerrit.cloudera.org:8080/20507 Reviewed-by: Daniel Becker <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java 2 files changed, 87 insertions(+), 21 deletions(-) Approvals: Daniel Becker: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/20507 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ib9421b5e26bfa2324217ec9695fbd81636727d22 Gerrit-Change-Number: 20507 Gerrit-PatchSet: 8 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Daniel Becker <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Reviewer: Wenzhe Zhou <[email protected]>
