I have a storm cluster that's running three different topologies - soon to become 6. I was wondering if there was a way to separate the log statements for each topology code and maybe even keep it distinct from the log statements of the workers themselves? This would enable developers to easily see what's going on in the topologies that they wrote vs what's going on in other topologies and the storm cluster itself.
Has anyone solved this? I am thinking two possible end solutions: a) Different log files on each machine: for eg worker-6701.log, worker-6702.log, supervisor.log, twitter_counter_topology.log, facebook_counter_topology.log or b) Some kind of a marker on each log statement within the worker files. This could be filtered later by another system like logstash. For eg: [twitter_counter_topology] processed tweets from account @stormprocessor * Not sure if this should be posted to the users or the dev group but figured I should start here to see if anyone has a solution. -- - varun :)
