When running Storm using LocalCluster, it starts an in process zookeeper server. Normally, when running a zookeeper server, it is up to the administrator to remove old logs so the directory specified by datalogdir doesn't fill up. The zookeeper config settings autopurge.snapRetainCount and autopurge.purgeInterval allow this to be done automatically. I am wondering if Storm automatically handles the clean up of these logs or provides a config file for the above settings to be changed from defaults. From looking through the code, it seems that it doesn't. I am guessing this is because local mode is for testing and isn't really meant to support long running topologies. Does running Storm through LocalCluster provide a way to prevent Zookeeper server logs from overflowing? And in general, is LocalCluster meant to be used for long running topologies in production, or just testing?
In Process Zookeeper Log Clean Up
Mitchell Rathbun (BLOOMBERG/ 731 LEX) Fri, 08 Dec 2017 14:38:11 -0800
- In Process Zookeeper Log Clean Up Mitchell Rathbun (BLOOMBERG/ 731 LEX)
- Re: In Process Zookeeper Log Cl... Manish Sharma
- Re: In Process Zookeeper Log Cl... Mitchell Rathbun (BLOOMBERG/ 731 LEX)
