[
https://issues.apache.org/jira/browse/STORM-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16303483#comment-16303483
]
Jungtaek Lim commented on STORM-2862:
-------------------------------------
[~hmcc]
You can extract component name and task ID from thread name (only if it is
logged from executor threads) though I agree it is not that easy to build parse
format.
The format of executor thread is like
Thread-<threadId>-<componentId>-executor<executorId>, and the format of
executorId is \[startTaskId endTaskId].
You could construct regex for {code} Thread-[0-9]+-(.+)-executor\[([0-9]+)
([0-9]+)\] {code}
Btw, I'm not sure we can modify the log format, which may be backward
incompatible for the cases which need to parse log messages.
> More flexible logging in multilang (Python, Ruby, JS)
> -----------------------------------------------------
>
> Key: STORM-2862
> URL: https://issues.apache.org/jira/browse/STORM-2862
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-client, storm-multilang
> Affects Versions: 2.0.0, 1.1.1
> Reporter: Heather McCartney
> Assignee: Heather McCartney
> Priority: Trivial
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> We're running a Storm topology written in Python, using storm.py from
> storm-multilang. As well as human-readable logs, the topology is also
> configured to write JSON logs which are sent to ELK.
> At the moment, when storm-core receives a "log" command, it outputs the pid,
> component name, and the message it received, like so:
> {{ShellLog pid:<pid>, name:<component name> <message>}}
> The code that does this is (currently) in [ShellBolt line
> 254|https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/task/ShellBolt.java#L254]
> and [ShellSpout line
> 227|https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/spout/ShellSpout.java#L227].
> As well as the pid and component name, it would be great to have the task ID,
> tuple ID, and the ID of the originating tuple - but this would make parsing
> the string even more laborious than it is now, and would make the default log
> message too long.
> Would it be possible to put contextual information like this in the
> [ThreadContext|https://logging.apache.org/log4j/2.x/manual/thread-context.html]
> instead? Then our JSON layout could read from the context instead of parsing
> the string, and human-readable logs could use "%mdc" in the PatternLayout
> format string.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)