[
https://issues.apache.org/jira/browse/KAFKA-4829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steven Schlansker updated KAFKA-4829:
-------------------------------------
Description:
Currently I see this every commit interval:
{code}
2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Committing
task StreamTask 1_31
2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Committing
task StreamTask 2_31
{code}
We have ~10 tasks in our topology, 4 topics, and 32 partitions per topic.
This means every commit interval we log a few hundred lines of the above
which is an order of magnitude chattier than anything else in the log
during normal operations.
To improve visibility of important messages, we should reduce the chattiness of
normal commits and highlight abnormal commits. An example proposal:
existing message is fine at TRACE level for diagnostics
{{TRACE o.a.k.s.p.i.StreamThread - Committing task StreamTask 1_31}}
normal fast case, wrap them all up into one summary line
{{INFO o.a.k.s.p.i.StreamThreads - 64 stream tasks committed in 25ms}}
some kind of threshold / messaging in case it doesn't complete quickly or logs
an exception
{{ERROR o.a.k.s.p.i.StreamThread - StreamTask 1_32 did not commit in 100ms}}
was:
Currently I see this every commit interval:
{code}
2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Committing
task StreamTask 1_31
2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Committing
task StreamTask 2_31
{code}
We have ~10 tasks in our topology, 4 topics, and 32 partitions per topic.
This means every commit interval we log a few hundred lines of the above
which is an order of magnitude chattier than anything else in the log
during normal operations.
To improve visibility of important messages, we should reduce the chattiness of
normal commits and highlight abnormal commits. An example proposal:
existing message is fine at TRACE level for diagnostics
{{TRACE o.a.k.s.p.i.StreamThread - Committing task StreamTask 1_31}}
normal fast case, wrap them all up into one summary line
{{INFO o.a.k.s.p.i.StreamThreads - 64 stream tasks committed in 25ms}}
some kind of threshold / messaging in case it doesn't complete quickly
or logs an exception
{{ERROR o.a.k.s.p.i.StreamThread - StreamTask 1_32 did not commit in 100ms}}
Thoughts?
> Improve logging of StreamTask commits
> -------------------------------------
>
> Key: KAFKA-4829
> URL: https://issues.apache.org/jira/browse/KAFKA-4829
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Affects Versions: 0.10.2.0
> Reporter: Steven Schlansker
> Priority: Minor
>
> Currently I see this every commit interval:
> {code}
> 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
> o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Committing
> task StreamTask 1_31
> 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
> o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Committing
> task StreamTask 2_31
> {code}
> We have ~10 tasks in our topology, 4 topics, and 32 partitions per topic.
> This means every commit interval we log a few hundred lines of the above
> which is an order of magnitude chattier than anything else in the log
> during normal operations.
> To improve visibility of important messages, we should reduce the chattiness
> of normal commits and highlight abnormal commits. An example proposal:
> existing message is fine at TRACE level for diagnostics
> {{TRACE o.a.k.s.p.i.StreamThread - Committing task StreamTask 1_31}}
> normal fast case, wrap them all up into one summary line
> {{INFO o.a.k.s.p.i.StreamThreads - 64 stream tasks committed in 25ms}}
> some kind of threshold / messaging in case it doesn't complete quickly or
> logs an exception
> {{ERROR o.a.k.s.p.i.StreamThread - StreamTask 1_32 did not commit in 100ms}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
