[ 
https://issues.apache.org/jira/browse/STORM-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16311149#comment-16311149
 ] 

Heather McCartney commented on STORM-2862:
------------------------------------------

Thanks [~kabhwan], I didn't know that.

What about the ID of the originating tuple, for tracking tasks through the 
topology? Is this available anywhere that I haven't thought of?

I understand about not modifying the log format. In that case, would it be 
possible to make this configurable? e.g.

* keep the part of the change that centralises the logging logic in 
ShellLogHandler
* make the {{handleLog}} method protected
* we subclass ShellLogHandler e.g. com.mycompany.CustomShellLogHandler and 
override {{handleLog}} to write the ThreadContext
* make the ShellLogHandler class a configuration option, so the default is 
{{org.apache.storm.utils.ShellLogHandler}} but we could set it to 
{{com.mycompany.CustomShellLogHandler}}

Or would you rather not add another configuration option for this?

> More flexible logging in multilang (Python, Ruby, JS)
> -----------------------------------------------------
>
>                 Key: STORM-2862
>                 URL: https://issues.apache.org/jira/browse/STORM-2862
>             Project: Apache Storm
>          Issue Type: Improvement
>          Components: storm-client, storm-multilang
>    Affects Versions: 2.0.0, 1.1.1
>            Reporter: Heather McCartney
>            Assignee: Heather McCartney
>            Priority: Trivial
>              Labels: pull-request-available
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> We're running a Storm topology written in Python, using storm.py from 
> storm-multilang. As well as human-readable logs, the topology is also 
> configured to write JSON logs which are sent to ELK.
> At the moment, when storm-core receives a "log" command, it outputs the pid, 
> component name, and the message it received, like so:
> {{ShellLog pid:<pid>, name:<component name> <message>}}
> The code that does this is (currently) in [ShellBolt line 
> 254|https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/task/ShellBolt.java#L254]
>  and [ShellSpout line 
> 227|https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/spout/ShellSpout.java#L227].
> As well as the pid and component name, it would be great to have the task ID, 
> tuple ID, and the ID of the originating tuple - but this would make parsing 
> the string even more laborious than it is now, and would make the default log 
> message too long. 
> Would it be possible to put contextual information like this in the 
> [ThreadContext|https://logging.apache.org/log4j/2.x/manual/thread-context.html]
>  instead? Then our JSON layout could read from the context instead of parsing 
> the string, and human-readable logs could use "%mdc" in the PatternLayout 
> format string.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Reply via email to