Bug#696228: graphite-carbon: logrotate and internal carbon log rotation interact badly
Hello Jonnas, On Sun, Jan 13, 2013 at 04:44:49PM +0100, Jonas Genannt wrote: This looks like the twisted log class used in carbon is still doing internal logrotation when the file exceeds 1M in size. I have updated the patch against Carbon, it now disables rotateLength. That is by default 1M. with your latest patch it now works like expected: -rw-r--r-- 1 _graphite root 6410536 Jan 17 11:00 console.log -rw-r--r-- 1 _graphite _graphite 368 Jan 15 06:31 console.log.1 6:31 is nearly the time when logrotate kicks in and today no rotation. This bug can be closed now. Not directly related: Are you aware of any plans to package python-graphite-web in debian ? At the moment we are using python setup.py --command-packages=stdeb.command debianize to get the package. Many thanks, Hermann -- Netzwerkadministration/Zentrale Dienste, Interdiziplinaeres Zentrum fuer wissenschaftliches Rechnen der Universitaet Heidelberg IWR; INF 368; 69120 Heidelberg; Tel: (06221)54-8236 Fax: -5224 Email: hermann.la...@iwr.uni-heidelberg.de -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#696228: graphite-carbon: logrotate and internal carbon log rotation interact badly
Hello Hermann, with your latest patch it now works like expected: -rw-r--r-- 1 _graphite root 6410536 Jan 17 11:00 console.log -rw-r--r-- 1 _graphite _graphite 368 Jan 15 06:31 console.log.1 6:31 is nearly the time when logrotate kicks in and today no rotation. This bug can be closed now. cool - thanks for testing and reporting it. Not directly related: Are you aware of any plans to package python-graphite-web in debian ? At the moment we are using python setup.py --command-packages=stdeb.command debianize to get the package. See here: http://ftp-master.debian.org/new/graphite-web_0.9.10+debian-1.html Is already done and uploaded, sitting since 3 weeks in NEW. I'm also working on python-ceres, the next generation library. (Replacement for whisper in Graphite 0.10) It will be uploaded then graphite-web hits unstable. Greets, Jonas -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#696228: graphite-carbon: logrotate and internal carbon log rotation interact badly
severity 696228 grave tags 696228 + pending thanks Hello, On Sun, Jan 13, 2013 at 04:44:49PM +0100, Jonas Genannt wrote: This looks like the twisted log class used in carbon is still doing internal logrotation when the file exceeds 1M in size. I have updated the patch against Carbon, it now disables rotateLength. That is by default 1M. with your latest patch it now works like expected: -rw-r--r-- 1 _graphite root 6410536 Jan 17 11:00 console.log -rw-r--r-- 1 _graphite _graphite 368 Jan 15 06:31 console.log.1 6:31 is nearly the time when logrotate kicks in and today no rotation. This bug can be closed now. summary of that bug report: - on package update or reconfigure the postinst script hangs - internal logrotate of carbon confuses system logrotate eats up space on disc That bug should also fixed in Wheezy. Current Version in Wheezy is: 0.9.10-1 I will prepare new packages. Thanks, Jonas -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#696228: graphite-carbon: logrotate and internal carbon log rotation interact badly
Hello Hermann, On Thu, Jan 03, 2013 at 12:46:34PM +0100, Hermann Lauer wrote: looks still somehow messy, but deleted now all console.log* files as you said and will watch if a new mess builds up during the next days. Deleted the default logrotate rule for carbon a few days ago and are seeing now: -rw-r--r-- 1 _graphite _graphite 145 Jan 11 08:52 console.log.1 This looks like the twisted log class used in carbon is still doing internal logrotation when the file exceeds 1M in size. thanks for your testing again, great work! :) I think I have found the problem: http://anonscm.debian.org/gitweb/?p=pkg-graphite/packages/graphite-carbon.git;a=blob;f=debian/patches/disable_internal_logroate.patch;h=77e3314fed0bee01bbb910a78d3fdbe8adc526eb;hb=53cc86269a2683592d96b5d65dc9fcb04a677a86 I have updated the patch against Carbon, it now disables rotateLength. That is by default 1M. Could you please test again? Thanks, Jonas -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#696228: graphite-carbon: logrotate and internal carbon log rotation interact badly
Hello Jonas, On Thu, Jan 03, 2013 at 12:46:34PM +0100, Hermann Lauer wrote: looks still somehow messy, but deleted now all console.log* files as you said and will watch if a new mess builds up during the next days. Deleted the default logrotate rule for carbon a few days ago and are seeing now: -rw-r--r-- 1 _graphite _graphite 118924 Jan 11 09:53 console.log -rw-r--r-- 1 _graphite _graphite 145 Jan 11 08:52 console.log.1 -rw-r--r-- 1 _graphite _graphite 143 Jan 8 02:33 console.log.10 -rw-r--r-- 1 _graphite _graphite 112 Jan 7 17:45 console.log.11 -rw-r--r-- 1 _graphite _graphite 124 Jan 7 08:57 console.log.12 -rw-r--r-- 1 _graphite _graphite 133 Jan 7 00:12 console.log.13 -rw-r--r-- 1 _graphite _graphite 133 Jan 6 15:31 console.log.14 -rw-r- 1 _graphite root 0 Jan 6 06:50 console.log.15 -rw-r--r-- 1 root root 7591235 Jan 6 06:50 console.log.16 -rw-r--r-- 1 _graphite _graphite 164 Jan 11 00:26 console.log.2 -rw-r--r-- 1 _graphite _graphite 163 Jan 10 15:58 console.log.3 -rw-r--r-- 1 _graphite _graphite 134 Jan 10 07:08 console.log.4 -rw-r--r-- 1 _graphite _graphite 100 Jan 9 22:19 console.log.5 -rw-r--r-- 1 _graphite _graphite 159 Jan 9 13:28 console.log.6 -rw-r--r-- 1 _graphite _graphite 118 Jan 9 04:47 console.log.7 -rw-r--r-- 1 _graphite _graphite 124 Jan 8 19:59 console.log.8 -rw-r--r-- 1 _graphite _graphite 112 Jan 8 11:16 console.log.9 This looks like the twisted log class used in carbon is still doing internal logrotation when the file exceeds 1M in size. http://anonscm.debian.org/gitweb/?p=pkg-graphite/packages/graphite-carbon.git;a=commitdiff;h=5d747bfa7006ce1fab18b0299dcf21ae069acfa6 No problems with hangs during the last updates with this patch, so this is fixed. Thanks, greetings Hermann -- Netzwerkadministration/Zentrale Dienste, Interdiziplinaeres Zentrum fuer wissenschaftliches Rechnen der Universitaet Heidelberg IWR; INF 368; 69120 Heidelberg; Tel: (06221)54-8236 Fax: -5224 Email: hermann.la...@iwr.uni-heidelberg.de -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#696228: graphite-carbon: logrotate and internal carbon log rotation interact badly
Hello Jonas, On Thu, Dec 20, 2012 at 10:23:48PM +0100, Jonas Genannt wrote: many thanks for the quick patch, which looks like doing the right thing. Our customized package contains it now and is running, will report when I'm shure it works like expected. # ls -l listener.log* -rw-r- 1 _graphite _graphite 683725 Jan 3 12:27 listener.log -rw-r- 1 _graphite root 123 Jan 1 18:03 listener.log.1 -rw-r- 1 _graphite _graphite 835812 Dec 30 06:35 listener.log.2 -rw-r- 1 _graphite _graphite 60477 Dec 28 05:02 listener.log.2.gz -rw-r- 1 _graphite root 174 Dec 25 17:45 listener.log.3 -rw-r- 1 _graphite _graphite 62090 Dec 21 11:16 listener.log.3.gz -rw-r- 1 _graphite _graphite 728923 Dec 23 06:28 listener.log.4 -rw-r- 1 _graphite _graphite8300 Dec 9 06:40 listener.log.4.gz -rw-r- 1 _graphite _graphite 128840 Dec 16 06:38 listener.log.5 -rw-r- 1 _graphite _graphite9611 Dec 2 06:47 listener.log.5.gz -rw-r- 1 _graphite _graphite 10068 Nov 25 06:47 listener.log.6.gz -rw-r- 1 _graphite _graphite5931 Nov 18 06:32 listener.log.7.gz looks still somehow messy, but deleted now all console.log* files as you said and will watch if a new mess builds up during the next days. could you please apply on your package the following patch: http://anonscm.debian.org/gitweb/?p=pkg-graphite/packages/graphite-carbon.git;a=commitdiff;h=5d747bfa7006ce1fab18b0299dcf21ae069acfa6 Please test if the process hangs again on update. Did a dpkg -i /tmp/graphite-carbon_0.9.10-1.3_all.deb which replaced the last package without a hang. BTW, on https://bitbucket.org/hlauer/shinken2rrd we have a small patch to extend the udp receiver to a minimalistic protocol we use here a long time. Feel free to include or send upstream, if you think it's worth it. Many thanks, Hermann -- Netzwerkadministration/Zentrale Dienste, Interdiziplinaeres Zentrum fuer wissenschaftliches Rechnen der Universitaet Heidelberg IWR; INF 368; 69120 Heidelberg; Tel: (06221)54-8236 Fax: -5224 Email: hermann.la...@iwr.uni-heidelberg.de -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#696228: graphite-carbon: logrotate and internal carbon log rotation interact badly
Hello Hermann, many thanks for the quick patch, which looks like doing the right thing. Our customized package contains it now and is running, will report when I'm shure it works like expected. thanks for testing. | | `-dpkg,18859 --status-fd 15 --configure graphite-carbon:all | | `-frontend,18860 -w /usr/share/debconf/frontend /var/lib/dpkg/info/graphite-carbon.postinst configure 0.9.10-1.1 | | `-(graphite-carbon,18868) could you please apply on your package the following patch: http://anonscm.debian.org/gitweb/?p=pkg-graphite/packages/graphite-carbon.git;a=commitdiff;h=5d747bfa7006ce1fab18b0299dcf21ae069acfa6 Please test if the process hangs again on update. Greets, Jonas -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#696228: graphite-carbon: logrotate and internal carbon log rotation interact badly
Hello Jonas, On Tue, Dec 18, 2012 at 06:46:55PM +0100, Jonas Genannt wrote: you are correct. Carbon rotates also. I have created an patch against that problem. ... If it works on your side, I will submit it to upstream. many thanks for the quick patch, which looks like doing the right thing. Our customized package contains it now and is running, will report when I'm shure it works like expected. Btw., the upgrade script in the deb seems to contain a bug: | | `-apt-get,18775 upgrade | | `-dpkg,18859 --status-fd 15 --configure graphite-carbon:all | | `-frontend,18860 -w /usr/share/debconf/frontend /var/lib/dpkg/info/graphite-carbon.postinst configure 0.9.10-1.1 | | `-(graphite-carbon,18868) The graphite-carbon process is the 'defunct' state and the frontend must be killed. Any idea how to debug this ? Thanks, Hermann -- Netzwerkadministration/Zentrale Dienste, Interdiziplinaeres Zentrum fuer wissenschaftliches Rechnen der Universitaet Heidelberg IWR; INF 368; 69120 Heidelberg; Tel: (06221)54-8236 Fax: -5224 Email: hermann.la...@iwr.uni-heidelberg.de -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#696228: graphite-carbon: logrotate and internal carbon log rotation interact badly
Package: graphite-carbon Version: 0.9.10-1.1 Severity: normal Dear Maintainer, the usage of both logrotate and the carbon internal logrotation facility yields to the logs splitted, see content of directory /var/log/carbon below. For the moment, disabling /etc/logrotate.d/graphite-carbon would be avoid the mess, but in the long run carbon's internal logrotation should probably switched off (to be done upstream). Thanks, Hermann # ls -l console.log* -rw-r- 1 _graphite _graphite 1452684 Dec 18 12:29 console.log -rw-r- 1 _graphite _graphite 786274 Dec 16 06:38 console.log.1 -rw-r- 1 _graphite _graphite 2679760 Nov 10 23:59 console.log.2012_11_10 -rw-r- 1 _graphite root 1937967 Nov 11 23:59 console.log.2012_11_11 -rw-r- 1 _graphite _graphite 2635685 Nov 12 23:59 console.log.2012_11_12 -rw-r- 1 _graphite _graphite 2635677 Nov 13 23:59 console.log.2012_11_13 -rw-r- 1 _graphite _graphite 1769472 Nov 14 17:13 console.log.2012_11_14 -rw-r- 1 _graphite _graphite 45358953 Nov 15 23:59 console.log.2012_11_15 -rw-r- 1 _graphite _graphite 2800940 Nov 16 23:59 console.log.2012_11_16 -rw-r- 1 _graphite _graphite 2804795 Nov 17 23:59 console.log.2012_11_17 -rw-r- 1 _graphite root 2051073 Nov 18 23:59 console.log.2012_11_18 -rw-r- 1 _graphite _graphite 2796348 Nov 19 23:59 console.log.2012_11_19 -rw-r- 1 _graphite _graphite 2817940 Nov 20 23:59 console.log.2012_11_20 -rw-r- 1 _graphite _graphite 2798447 Nov 21 23:59 console.log.2012_11_21 -rw-r- 1 _graphite _graphite 2815762 Nov 22 23:59 console.log.2012_11_22 -rw-r- 1 _graphite _graphite 2867643 Nov 23 23:59 console.log.2012_11_23 -rw-r- 1 _graphite _graphite 2855280 Nov 24 23:59 console.log.2012_11_24 -rw-r- 1 _graphite root 2053204 Nov 25 23:59 console.log.2012_11_25 -rw-r- 1 _graphite _graphite 2861636 Nov 26 23:59 console.log.2012_11_26 -rw-r- 1 _graphite _graphite 2865025 Nov 27 23:59 console.log.2012_11_27 -rw-r- 1 _graphite _graphite 2862691 Nov 28 23:59 console.log.2012_11_28 -rw-r- 1 _graphite _graphite 2840902 Nov 29 23:59 console.log.2012_11_29 -rw-r- 1 _graphite _graphite 2884030 Nov 30 23:59 console.log.2012_11_30 -rw-r- 1 _graphite _graphite 2898732 Dec 1 23:59 console.log.2012_12_1 -rw-r- 1 _graphite _graphite 2871502 Dec 10 23:59 console.log.2012_12_10 -rw-r- 1 _graphite _graphite 2882910 Dec 11 23:59 console.log.2012_12_11 -rw-r- 1 _graphite _graphite 2866346 Dec 12 23:59 console.log.2012_12_12 -rw-r- 1 _graphite _graphite 2843207 Dec 13 23:59 console.log.2012_12_13 -rw-r- 1 _graphite _graphite 2855640 Dec 14 23:59 console.log.2012_12_14 -rw-r- 1 _graphite _graphite 2847990 Dec 15 23:59 console.log.2012_12_15 -rw-r- 1 _graphite root 2069418 Dec 16 23:59 console.log.2012_12_16 -rw-r- 1 _graphite _graphite 2823094 Dec 17 23:59 console.log.2012_12_17 -rw-r- 1 _graphite root 2099626 Dec 2 23:59 console.log.2012_12_2 -rw-r- 1 _graphite _graphite 2891764 Dec 3 23:59 console.log.2012_12_3 -rw-r- 1 _graphite _graphite 2906945 Dec 4 23:59 console.log.2012_12_4 -rw-r- 1 _graphite _graphite 2913170 Dec 5 23:59 console.log.2012_12_5 -rw-r- 1 _graphite _graphite 2901500 Dec 6 23:59 console.log.2012_12_6 -rw-r- 1 _graphite _graphite 2868494 Dec 7 23:59 console.log.2012_12_7 -rw-r- 1 _graphite _graphite 2902303 Dec 8 23:59 console.log.2012_12_8 -rw-r- 1 _graphite root 2079404 Dec 9 23:59 console.log.2012_12_9 -rw-r- 1 _graphite _graphite71779 Dec 9 06:40 console.log.2.gz -rw-r- 1 _graphite _graphite73977 Dec 2 06:47 console.log.3.gz -rw-r- 1 _graphite _graphite73076 Nov 25 06:47 console.log.4.gz -rw-r- 1 _graphite _graphite67372 Nov 18 06:33 console.log.5.gz -rw-r- 1 _graphite _graphite64762 Nov 11 06:32 console.log.6.gz -rw-r- 1 _graphite _graphite66894 Nov 4 06:49 console.log.7.gz -- System Information: Debian Release: wheezy/sid APT prefers testing APT policy: (500, 'testing') Architecture: sparc (sparc64) Kernel: Linux 3.2.0-4-sparc64-smp (SMP w/1 CPU core) Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968) Shell: /bin/sh linked to /bin/dash Versions of packages graphite-carbon depends on: ii adduser3.113+nmu3 ii debconf [debconf-2.0] 1.5.46 ii python 2.7.3~rc2-1 ii python-twisted-core12.0.0-1 ii python-whisper 0.9.10-1 ii python2.6 2.6.8-0.2 ii python2.7 2.7.3~rc2-2.1 graphite-carbon recommends no packages. graphite-carbon suggests no packages. -- Configuration Files: /etc/carbon/carbon.conf changed: [cache] STORAGE_DIR= /opt/graphite/storage/ CONF_DIR = /opt/graphite/conf/ LOG_DIR= /var/log/carbon/ PID_DIR= /var/run/ LOCAL_DATA_DIR =/opt/graphite/storage/whisper USER = _graphite MAX_CACHE_SIZE = inf MAX_UPDATES_PER_SECOND = 500 MAX_CREATES_PER_MINUTE = 50
Bug#696228: graphite-carbon: logrotate and internal carbon log rotation interact badly
Hello Hermann, the usage of both logrotate and the carbon internal logrotation facility yields to the logs splitted, see content of directory /var/log/carbon below. For the moment, disabling /etc/logrotate.d/graphite-carbon would be avoid the mess, but in the long run carbon's internal logrotation should probably switched off (to be done upstream). you are correct. Carbon rotates also. I have created an patch against that problem. Could you please apply the following patch against your installed log.py (/usr/share/pyshared/carbon/log.py): http://anonscm.debian.org/gitweb/?p=pkg-graphite/packages/graphite-carbon.git;a=blob;f=debian/patches/disable_internal_logroate.patch;h=eb9bb1f1c125b3d6662cd687aab071870f1149ce;hb=b9bb0232108dd8eececc4c6ec73a54a62122c248 Steps: - stop carbon - apply the patch - remove all logs from var/log/carbon/ - start carbon - activate logrotate again - wait one day - restart carbon - force run logroate If it works on your side, I will submit it to upstream. Thanks, Jonas -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
