Heather McCartney created STORM-2862:
----------------------------------------
Summary: More flexible logging in multilang (Python, Ruby, JS)
Key: STORM-2862
URL: https://issues.apache.org/jira/browse/STORM-2862
Project: Apache Storm
Issue Type: Improvement
Components: storm-client, storm-multilang
Affects Versions: 2.0.0, 1.1.1
Reporter: Heather McCartney
Assignee: Heather McCartney
Priority: Trivial
We're running a Storm topology written in Python, using storm.py from
storm-multilang. As well as human-readable logs, the topology is also
configured to write JSON logs which are sent to ELK.
At the moment, when storm-core receives a "log" command, it outputs the pid,
component name, and the message it received, like so:
{{ShellLog pid:<pid>, name:<component name> <message>}}
The code that does this is (currently) in [ShellBolt line
254|https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/task/ShellBolt.java#L254]
and [ShellSpout line
227|https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/spout/ShellSpout.java#L227].
As well as the pid and component name, it would be great to have the task ID,
tuple ID, and the ID of the originating tuple - but this would make parsing the
string even more laborious than it is now, and would make the default log
message too long.
Would it be possible to put contextual information like this in the
[ThreadContext|https://logging.apache.org/log4j/2.x/manual/thread-context.html]
instead? Then our JSON layout could read from the context instead of parsing
the string, and human-readable logs could use "%mdc" in the PatternLayout
format string.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
