[
https://issues.apache.org/jira/browse/STORM-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14640965#comment-14640965
]
Arun Mahadevan commented on STORM-954:
--------------------------------------
Putting down high level thoughts below.
Have an internal bolt (EventInspector) that is capable of receiving tuples from
any spout or bolt. EventInspector would log the tuples and any additional meta
data it received into a log file which can be used to inspect the events via
the storm UI.
Users can turn on event inspection for specific spouts or bolts (or globally
for all) via the UI and this flag is passed on to the workers via a zookeeper
watch. This will also ensure turning on/off the event inspection without
restarting the topology.
We will add edges (streams) from all spouts and bolts to EventInspector during
topology construction. Worker / Executors send out the tuples and additional
metadata to EventInspector for those spouts and bolts which have the flag
turned on.
To keep the load under control, we let users specify (or auto apply) some
sampling percentage and also write out data to log files in batches for better
performance.
> Toplogy Event Inspector
> -----------------------
>
> Key: STORM-954
> URL: https://issues.apache.org/jira/browse/STORM-954
> Project: Apache Storm
> Issue Type: Improvement
> Reporter: Sriharsha Chintalapani
> Assignee: Arun Mahadevan
>
> •Ability to view tuples flowing through the topology
> •Ability to turn on/off debug events without having to stop/restart topology
> •Default debug events is off
> •User should be able to select a specific Spout or Bolt and see incoming
> events and outgoing events
> •We could put a configurable numbers of events to view (e.g. last 100 events
> or last 1 minute)
> •Tuple stream to have following info
> •Message id, batch/transaction id, name/value pair, timestamp, acked (boolean)
> •All the above to be available from Storm UI
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
