[
https://issues.apache.org/jira/browse/IMPALA-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16590804#comment-16590804
]
ASF subversion and git services commented on IMPALA-7433:
---------------------------------------------------------
Commit df1865856a7a9988f946c38deeb39cbb3fff80da in impala's branch
refs/heads/master from [[email protected]]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=df18658 ]
IMPALA-7433: reduce logging on executors
Moved logs to -v=2 for reasons described in the JIRA. Added
more details to some existing log messages or new
less-frequent log messages so that useful information is not
removed.
Sample logging for an executor after the change:
I0813 12:10:50.249850 31250 impala-internal-service.cc:49]
ExecQueryFInstances(): query_id=fd4ae28bc993236e:27343be100000000
coord=tarmstrong-box:22000 #instances=2
I0813 12:10:50.250722 31256 query-state.cc:477] Executing instance.
instance_id=fd4ae28bc993236e:27343be100000006 fragment_idx=1
per_fragment_instance_idx=2 coord_state_idx=1 #in-flight=1
I0813 12:10:50.250804 31259 query-state.cc:477] Executing instance.
instance_id=fd4ae28bc993236e:27343be100000003 fragment_idx=2
per_fragment_instance_idx=2 coord_state_idx=1 #in-flight=2
I0813 12:10:50.374167 31259 query-state.cc:485] Instance completed.
instance_id=fd4ae28bc993236e:27343be100000003 #in-flight=1 status=OK
I0813 12:10:50.375370 31269 krpc-data-stream-mgr.cc:294] DeregisterRecvr():
fragment_instance_id=fd4ae28bc993236e:27343be100000006, node=3
I0813 12:10:50.417552 31256 query-state.cc:485] Instance completed.
instance_id=fd4ae28bc993236e:27343be100000006 #in-flight=0 status=OK
I0813 12:10:50.418007 31256 query-exec-mgr.cc:179] ReleaseQueryState():
deleted query_id=fd4ae28bc993236e:27343be100000000
Change-Id: I6c1db44acc6def2b05a4fd032c63716e08cdf5ff
Reviewed-on: http://gerrit.cloudera.org:8080/11202
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Reduce volume of -v=1 logs on executors
> ---------------------------------------
>
> Key: IMPALA-7433
> URL: https://issues.apache.org/jira/browse/IMPALA-7433
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 3.1.0
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Fix For: Impala 3.1.0
>
>
> We're currently logging a lot of things per query on executors that is
> redundant or unnecessary, e.g.:
> * ExecQueryFInstances() and StartQueryFInstances() are both logged but are
> redundant
> * Some random low-level details are logged, e.g. "Max row batch queue size
> for scan node '0' in fragment instance '940cdcd50664a51:368ab6690000001e':
> 180"
> I think we should cut back the logging at -v=1, which is the default log
> level most deployments run at:
> * Do log key finstance lifecycle events that are useful to understand what
> finstances are running and if/when failures occurred
> * Don't log things redundantly
> * Don't log lower-level events that occur in the normal flow of query
> execution (e.g. memory limits, queue lengths). These things have limited
> diagnostic value and should be in the profile, if anywhere
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
