Re: log files not being cleaned up despite purgeInterval
I would first check the permission on zkCleanup.sh and the bin folder. Sounds like zookeeper user has no access to the /zk/bin directory. That might also explain why it is not getting deleted by the zk instance. And I'm not sure in this one, but did you try giving the full path to the txn log files like bash -ex ./zkCleanup.sh /data/version-2 -n 3 as root? I think this script might be expecting the full path, including the version-2 directory. Regards, Norbert On Fri, Jul 19, 2019 at 2:00 PM Koen De Groote wrote: > Hello Norbert, > > I've set up a new environment which then reached at least 4 *.log files > All snapshots and log files are kept in /data/version-2/(default for the > image) > > I went into the zookeeper container and executed: > > bash -ex ./zkCleanup.sh /data -n 3 > > As root, this changes nothing. There are still 4 *.log files > > Changing to the zookeeper user, I get the following output: > > Path '/zookeeper-3.4.13/bin' does not exist. > Usage: > PurgeTxnLog dataLogDir [snapDir] -n count > dataLogDir -- path to the txn log directory > snapDir -- path to the snapshot directory > count -- the number of old snaps/logs you want to keep, value should be > greater than or equal to 3 > > And the 4 *.log files still exist. > Also printing the usage, indicating, to me at least, that something about > the input is wrong, even though it is identical to the one used as root, > which did not result in this output. > > No actual error messages seem to be printed or logged anywhere. > > Not sure what to do next. > > > > On Fri, Jul 19, 2019 at 11:01 AM Norbert Kalmar > wrote: > > > Hi Koen, > > > > It should do just as you said. You can also set > autopurge.snapRetainCount, > > bu default it is set to 3, so if you didn't set anything it is not a > reason > > to keep old logs. > > > > As a plan B you could use zkCleanup.sh [snapshotDir] -n 3 to delete all > > except the last 3 log files. You can add this to a cron job. > > > > As for why the old log files not getting deleted, could be something > > related to the docker image, maybe a permission problem? Do you see any > > errors in the server log? > > > > Regards, > > Norbert > > > > On Thu, Jul 18, 2019 at 9:25 PM Koen De Groote < > > koen.degro...@limecraft.com> > > wrote: > > > > > Greetings, > > > > > > Working with Zookeeper version 3.4.13 in the official docker image. > > > > > > I was under the impression that the setting "autopurge.purgeInterval=1" > > > meant that log files would be cleaned up every hour. > > > > > > Instead, I now find that months of these files are just sitting in > their > > > directory, untouched. > > > > > > So perhaps I'm wrong about that, but I'm not sure. > > > > > > What I wish to achieve is that these log files stop accumulating and > keep > > > only the most recent. Is there a way to achieve this? Or are they > merely > > > historical and can they be deleted freely? > > > > > > Kind regards, > > > Koen De Groote > > > > > >
Re: log files not being cleaned up despite purgeInterval
Hello Norbert, I've set up a new environment which then reached at least 4 *.log files All snapshots and log files are kept in /data/version-2/(default for the image) I went into the zookeeper container and executed: bash -ex ./zkCleanup.sh /data -n 3 As root, this changes nothing. There are still 4 *.log files Changing to the zookeeper user, I get the following output: Path '/zookeeper-3.4.13/bin' does not exist. Usage: PurgeTxnLog dataLogDir [snapDir] -n count dataLogDir -- path to the txn log directory snapDir -- path to the snapshot directory count -- the number of old snaps/logs you want to keep, value should be greater than or equal to 3 And the 4 *.log files still exist. Also printing the usage, indicating, to me at least, that something about the input is wrong, even though it is identical to the one used as root, which did not result in this output. No actual error messages seem to be printed or logged anywhere. Not sure what to do next. On Fri, Jul 19, 2019 at 11:01 AM Norbert Kalmar wrote: > Hi Koen, > > It should do just as you said. You can also set autopurge.snapRetainCount, > bu default it is set to 3, so if you didn't set anything it is not a reason > to keep old logs. > > As a plan B you could use zkCleanup.sh [snapshotDir] -n 3 to delete all > except the last 3 log files. You can add this to a cron job. > > As for why the old log files not getting deleted, could be something > related to the docker image, maybe a permission problem? Do you see any > errors in the server log? > > Regards, > Norbert > > On Thu, Jul 18, 2019 at 9:25 PM Koen De Groote < > koen.degro...@limecraft.com> > wrote: > > > Greetings, > > > > Working with Zookeeper version 3.4.13 in the official docker image. > > > > I was under the impression that the setting "autopurge.purgeInterval=1" > > meant that log files would be cleaned up every hour. > > > > Instead, I now find that months of these files are just sitting in their > > directory, untouched. > > > > So perhaps I'm wrong about that, but I'm not sure. > > > > What I wish to achieve is that these log files stop accumulating and keep > > only the most recent. Is there a way to achieve this? Or are they merely > > historical and can they be deleted freely? > > > > Kind regards, > > Koen De Groote > > >
Re: log files not being cleaned up despite purgeInterval
Hi Koen, It should do just as you said. You can also set autopurge.snapRetainCount, bu default it is set to 3, so if you didn't set anything it is not a reason to keep old logs. As a plan B you could use zkCleanup.sh [snapshotDir] -n 3 to delete all except the last 3 log files. You can add this to a cron job. As for why the old log files not getting deleted, could be something related to the docker image, maybe a permission problem? Do you see any errors in the server log? Regards, Norbert On Thu, Jul 18, 2019 at 9:25 PM Koen De Groote wrote: > Greetings, > > Working with Zookeeper version 3.4.13 in the official docker image. > > I was under the impression that the setting "autopurge.purgeInterval=1" > meant that log files would be cleaned up every hour. > > Instead, I now find that months of these files are just sitting in their > directory, untouched. > > So perhaps I'm wrong about that, but I'm not sure. > > What I wish to achieve is that these log files stop accumulating and keep > only the most recent. Is there a way to achieve this? Or are they merely > historical and can they be deleted freely? > > Kind regards, > Koen De Groote >
