Varun,
If your topologies using Logger to write debug or info you can
add logger to cluster.xml for example
<logger name="storm.starter.WordCountTopology" additivity="false">
<level value="INFO" />
<appender-ref ref="wordcount" />
</logger>
and use a custom appender
<appender name="wordcount"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${storm.log.dir}/wordcount.log</file>
<rollingPolicy
class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${storm.log.dir}/wordcount.log.%i</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>9</maxIndex>
</rollingPolicy>
<triggeringPolicy
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>100MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss} %c{1} [%p] %m%n</pattern>
</encoder>
</appender>
I've tested the above with trunk and by adding Logger to
wordcounttopology.
-Harsha
On Thu, Aug 28, 2014, at 07:51 AM, Bobby Evans wrote:
> If you are running storm in distributed mode the each worker has a
> different log file based off of the worker port. If you use the security
> branch each log file also has the topology id in it too.
>
> - Bobby
>
> On 7/23/14, 3:18 PM, "Varun Vijayaraghavan" <[email protected]> wrote:
>
> >We have a storm cluster that's running three different topologies - and
> >this number is expected increase for new features. Is there a way to
> >separate the log statements for each topology code and maybe even keep it
> >distinct from the log statements of the workers themselves? Either by
> >having separate log files per topology or adding some kind of tagging to
> >each log statement.. or any other better alternative.
> >
> >This would enable developers to easily see what's going on in the
> >topologies that they wrote vs what's going on in other topologies and the
> >storm cluster itself.
> >
> >Thanks!
> >
> >--
> >- varun :)
>
