Re: worker.yaml and worker.pid in workers.artifacts director
As far as I know, the path has to be {STORM_WORKERS_ARTIFACTS_DIR}/{worker
id}/{worker port}/worker.pid. Maybe someone has different answer to it.
> On Apr 18, 2018, at 4:24 PM, Mitchell Rathbun (BLOOMBERG/ 731 LEX)
> wrote:
>
> I was hoping to change where they are sent at a more granular level. For
> example, say I update the worker.xml file to update the worker.log file
> paths. Say I set it to
> {STORM_WORKERS_ARTIFACTS_DIR}//logs. Making this
> change, worker.log* will be sent to
> {STORM_WORKERS_ARTIFACTS_DIR}//logs, but
> worker.yaml and worker.pid will still be sent to
> {STORM_WORKERS_ARTIFACTS_DIR}/{worker id}/{worker port}. Is there anyway to
> specify the path of worker.yaml and worker.pid as the same as the other log
> files?
>
> Sent from Bloomberg Professional for Android
>
>
> - Original Message -
> From: Ethan Li >
> At: 18-Apr-2018 16:19:31
>
> Hi Mitchell,
>
>
> worker.yaml and worker.pid are used by storm itself.
>
> They will be put into {STORM_WORKERS_ARTIFACTS_DIR}/{worker id}/{worker port}
> (see
> https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/utils/ConfigUtils.java#L259-L261
>
>
>
>
> https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/utils/ConfigUtils.java#L104-L114
>
> )
>
>
> If you want to change their location, you can set this config:
> “STORM_WORKERS_ARTIFACTS_DIR” (
> https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/Config.java#L1751-L1757
>
> )
>
>
> Ethan
>
>
>> On Apr 18, 2018, at 1:12 PM, Mitchell Rathbun (BLOOMBERG/ 731 LEX)
>> > wrote:
>>
>> I have noticed that when I run a topology, all the worker related logs are
>> written by default to "${sys:workers.artifacts}/${sys:storm.id
>> }/${sys:worker.port}/${sys:logfile.name}", as specified
>> by the default worker.xml file. I have noticed that if I update the filename
>> used in the worker.xml file, all of the logs are sent to the specified
>> directory. However, the original specified path will still be populated with
>> just worker.pid and worker.yaml files. What are these files used for? Is it
>> possible to update the log directory specified by worker.xml and then have
>> the worker.pid and worker.yaml files written to that same directory?
>
Re: worker.yaml and worker.pid in workers.artifacts director
I was hoping to change where they are sent at a more granular level. For
example, say I update the worker.xml file to update the worker.log file paths.
Say I set it to {STORM_WORKERS_ARTIFACTS_DIR}//logs.
Making this change, worker.log* will be sent to
{STORM_WORKERS_ARTIFACTS_DIR}//logs, but worker.yaml
and worker.pid will still be sent to {STORM_WORKERS_ARTIFACTS_DIR}/{worker
id}/{worker port}. Is there anyway to specify the path of worker.yaml and
worker.pid as the same as the other log files?
Sent from Bloomberg Professional for Android
- Original Message -
From: Ethan Li
At: 18-Apr-2018 16:19:31
Hi Mitchell,
worker.yaml and worker.pid are used by storm itself.
They will be put into {STORM_WORKERS_ARTIFACTS_DIR}/{worker id}/{worker port}
(see
https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/utils/ConfigUtils.java#L259-L261
https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/utils/ConfigUtils.java#L104-L114)
If you want to change their location, you can set this config:
“STORM_WORKERS_ARTIFACTS_DIR” (
https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/Config.java#L1751-L1757)
Ethan
On Apr 18, 2018, at 1:12 PM, Mitchell Rathbun (BLOOMBERG/ 731 LEX)
wrote:
I have noticed that when I run a topology, all the worker related logs are
written by default to
"${sys:workers.artifacts}/${sys:storm.id}/${sys:worker.port}/${sys:logfile.name}",
as specified by the default worker.xml file. I have noticed that if I update
the filename used in the worker.xml file, all of the logs are sent to the
specified directory. However, the original specified path will still be
populated with just worker.pid and worker.yaml files. What are these files used
for? Is it possible to update the log directory specified by worker.xml and
then have the worker.pid and worker.yaml files written to that same directory?
Re: RE: Storm using all the CPU in disruptor-flush-task-pool
Simon, Were you been able to craft a standalone test case to reproduce the issue ? any progress on this issue ?-roshan
Re: worker.yaml and worker.pid in workers.artifacts directory
Hi Mitchell,
worker.yaml and worker.pid are used by storm itself.
They will be put into {STORM_WORKERS_ARTIFACTS_DIR}/{worker id}/{worker port}
(see
https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/utils/ConfigUtils.java#L259-L261
https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/utils/ConfigUtils.java#L104-L114
)
If you want to change their location, you can set this config:
“STORM_WORKERS_ARTIFACTS_DIR” (
https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/Config.java#L1751-L1757
)
Ethan
> On Apr 18, 2018, at 1:12 PM, Mitchell Rathbun (BLOOMBERG/ 731 LEX)
> wrote:
>
> I have noticed that when I run a topology, all the worker related logs are
> written by default to
> "${sys:workers.artifacts}/${sys:storm.id}/${sys:worker.port}/${sys:logfile.name}",
> as specified by the default worker.xml file. I have noticed that if I update
> the filename used in the worker.xml file, all of the logs are sent to the
> specified directory. However, the original specified path will still be
> populated with just worker.pid and worker.yaml files. What are these files
> used for? Is it possible to update the log directory specified by worker.xml
> and then have the worker.pid and worker.yaml files written to that same
> directory?
worker.yaml and worker.pid in workers.artifacts directory
I have noticed that when I run a topology, all the worker related logs are
written by default to
"${sys:workers.artifacts}/${sys:storm.id}/${sys:worker.port}/${sys:logfile.name}",
as specified by the default worker.xml file. I have noticed that if I update
the filename used in the worker.xml file, all of the logs are sent to the
specified directory. However, the original specified path will still be
populated with just worker.pid and worker.yaml files. What are these files used
for? Is it possible to update the log directory specified by worker.xml and
then have the worker.pid and worker.yaml files written to that same directory?
