[
https://issues.apache.org/jira/browse/STORM-3163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mal Graty updated STORM-3163:
-----------------------------
Description:
tl;dr: ShellLogHandler is handed context in one thread, before being used
exclusively from another, this obstructs sane usage of slf4j's MDC feature
which is thread local.
---
`ShellBolt` instantiates the `ShellLogHandler` and calls its `setUpContext` as
part of `prepare`, immediately before it spawns its `BoltReaderRunnable` and
`BoltWriterRunnable` threads which are responsible for communication with the
`ShellProcess` that's already been spawned.
The `ShellLogHandler` is used exclusively from `BoltReaderRunnable`. The upshot
of this is that `setUpContext` is executed in the task thread (i.e.
`Thread-21-joiner-executor[2 2]`) while the `log` method is executed in the
anonymous thread (i.e. `Thread-30`) running the `BoltReaderRunnable`.
This creates a problem when trying to leverage `slf4j`'s `MDC` (or `NDC`) which
are used for augmenting log messages with additional information which is
localised and persisted at the thread level.
The current work around for this is to store all relevant context on the
`ShellLogHandler` during `setUpContext`, and then write it into the `MDC`
during the `log` call, and taking precautions around that state being thread
safe. The thread safety requirement is the first drawback, the second is that
each `log` call has the additional overhead of either unconditionally writing
to the `MDC` or checking that the `MDC` is already populated. Neither is very
appealing.
The suggested solution is to pass `stormConf`, `_process`, and `_context` into
the constructor of `BoltReaderRunnable` (and perhaps also `BoltWriterRunnable`)
and instantiate `ShellLogHandler` and call its `setUpContext` from that thread.
was:
`ShellBolt` instantiates the `ShellLogHandler` and calls its `setUpContext` as
part of `prepare`, immediately before it spawns its `BoltReaderRunnable` and
`BoltWriterRunnable` threads which are responsible for communication with the
`ShellProcess` that's already been spawned.
The `ShellLogHandler` is used exclusively from `BoltReaderRunnable`. The upshot
of this is that `setUpContext` is executed in the task thread (i.e.
`Thread-21-joiner-executor[2 2]`) while the `log` method is executed in the
anonymous thread (i.e. `Thread-30`) running the `BoltReaderRunnable`.
This creates a problem when trying to leverage `slf4j`'s `MDC` (or `NDC`) which
are used for augmenting log messages with additional information which is
localised and persisted at the thread level.
The current work around for this is to store all relevant context on the
`ShellLogHandler` during `setUpContext`, and then write it into the `MDC`
during the `log` call, and taking precautions around that state being thread
safe. The thread safety requirement is the first drawback, the second is that
each `log` call has the additional overhead of either unconditionally writing
to the `MDC` or checking that the `MDC` is already populated. Neither is very
appealing.
The suggested solution is to pass `stormConf`, `_process`, and `_context` into
the constructor of `BoltReaderRunnable` (and perhaps also `BoltWriterRunnable`)
and instantiate `ShellLogHandler` and call its `setUpContext` from that thread.
> ShellLogHandler loses thread context between setup and use
> ----------------------------------------------------------
>
> Key: STORM-3163
> URL: https://issues.apache.org/jira/browse/STORM-3163
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-core
> Affects Versions: 1.2.2
> Reporter: Mal Graty
> Priority: Trivial
> Labels: Logging
>
> tl;dr: ShellLogHandler is handed context in one thread, before being used
> exclusively from another, this obstructs sane usage of slf4j's MDC feature
> which is thread local.
> ---
> `ShellBolt` instantiates the `ShellLogHandler` and calls its `setUpContext`
> as part of `prepare`, immediately before it spawns its `BoltReaderRunnable`
> and `BoltWriterRunnable` threads which are responsible for communication with
> the `ShellProcess` that's already been spawned.
> The `ShellLogHandler` is used exclusively from `BoltReaderRunnable`. The
> upshot of this is that `setUpContext` is executed in the task thread (i.e.
> `Thread-21-joiner-executor[2 2]`) while the `log` method is executed in the
> anonymous thread (i.e. `Thread-30`) running the `BoltReaderRunnable`.
> This creates a problem when trying to leverage `slf4j`'s `MDC` (or `NDC`)
> which are used for augmenting log messages with additional information which
> is localised and persisted at the thread level.
> The current work around for this is to store all relevant context on the
> `ShellLogHandler` during `setUpContext`, and then write it into the `MDC`
> during the `log` call, and taking precautions around that state being thread
> safe. The thread safety requirement is the first drawback, the second is that
> each `log` call has the additional overhead of either unconditionally writing
> to the `MDC` or checking that the `MDC` is already populated. Neither is very
> appealing.
> The suggested solution is to pass `stormConf`, `_process`, and `_context`
> into the constructor of `BoltReaderRunnable` (and perhaps also
> `BoltWriterRunnable`) and instantiate `ShellLogHandler` and call its
> `setUpContext` from that thread.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
