Re: too much logging
Am 29.04.2015 um 20:24 schrieb Dan White: On 04/29/15 18:35 +0200, hw wrote: Am 29.04.2015 um 18:15 schrieb Dan White: Does this mean that before 2.5.1, the database is being opened and closed all the time, yielding a log message? Correct. But that was an error produced if the user_deny.db file didn't exist, and ended up flooding syslog. That's a different syslog entry from what you're seeing. You might have to modify your syslog config to get rid of them: http://cyrusimap.org/docs/cyrus-imapd/2.5.1/install-configure.php That is precisely what I do not want to do. There is no point in generating useless log messages, and it should be possible to turn these messages off. They are usually somewhat harmless, though. When there are thousands or millions of them generated, it might indicate that there is some issue that needs to be fixed. Not all imaps processes create so many messages. In this sense, the messages are not useless. But what might cause so many of these messages to be generated within a short time, all day long? user_deny is used to selectively deny access to services for certain users. It's called within the main cmdloop within the imapd code, among other places (pop3, nntpd, lmtpd, and httpd). Each imapd connection could result in many syslog entries depending on what activity the client is performing. The syslog entry is generated with this code: if (!denydb) denydb_open(/*create*/0); if (!denydb) return 0; memset(tok, 0, sizeof(tok)); /* fetch entry for user */ syslog(LOG_DEBUG, fetching user_deny.db entry for '%s', user); If cyrusdb_open cannot successfully open or create the entry (such as a permissions error), then that would effectively stop the syslog entries from being generated. So, setting a bogus value for 'userdeny_db' and/or 'userdeny_db_path' should do what you want in a round about way. That's assuming you're not actually using userdeny of course. In the process of setting up cyrus, I created the file with 'touch' because I got error messages about it in the log file. Now when I remove it, I'm getting even more log messages than I get when the file exists: imaps[28382]: IOERROR: stating flat /var/imap/user_deny.db: No such file or directory imaps[28382]: DENYDB_ERROR: error reading entry 'fax': cyrusdb error So I guess I'm not using it; I don't even know what this feature is or does. The manpage of imapd doesn't show any way to turn this feature off. This kind of logging is actually a bug because it's a security issue by supporting DOS attacks. Just imagine 1000 users doing something that causes the message to be logged 20 times per second and the system logger having to deal with 20k log messages per second. It seems I'd have to comment out the line and recompile cyrus because that's the only way to disable the message. I'll have to figure out how to do that on Gentoo. Thank you very much for your help! Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: too much logging
You can't remove the db file from under a rubbing instance, only while it is shut down. On Thu, Apr 30, 2015, at 06:37 PM, hw wrote: Am 29.04.2015 um 20:24 schrieb Dan White: On 04/29/15 18:35 +0200, hw wrote: Am 29.04.2015 um 18:15 schrieb Dan White: Does this mean that before 2.5.1, the database is being opened and closed all the time, yielding a log message? Correct. But that was an error produced if the user_deny.db file didn't exist, and ended up flooding syslog. That's a different syslog entry from what you're seeing. You might have to modify your syslog config to get rid of them: http://cyrusimap.org/docs/cyrus-imapd/2.5.1/install-configure.php That is precisely what I do not want to do. There is no point in generating useless log messages, and it should be possible to turn these messages off. They are usually somewhat harmless, though. When there are thousands or millions of them generated, it might indicate that there is some issue that needs to be fixed. Not all imaps processes create so many messages. In this sense, the messages are not useless. But what might cause so many of these messages to be generated within a short time, all day long? user_deny is used to selectively deny access to services for certain users. It's called within the main cmdloop within the imapd code, among other places (pop3, nntpd, lmtpd, and httpd). Each imapd connection could result in many syslog entries depending on what activity the client is performing. The syslog entry is generated with this code: if (!denydb) denydb_open(/*create*/0); if (!denydb) return 0; memset(tok, 0, sizeof(tok)); /* fetch entry for user */ syslog(LOG_DEBUG, fetching user_deny.db entry for '%s', user); If cyrusdb_open cannot successfully open or create the entry (such as a permissions error), then that would effectively stop the syslog entries from being generated. So, setting a bogus value for 'userdeny_db' and/or 'userdeny_db_path' should do what you want in a round about way. That's assuming you're not actually using userdeny of course. In the process of setting up cyrus, I created the file with 'touch' because I got error messages about it in the log file. Now when I remove it, I'm getting even more log messages than I get when the file exists: imaps[28382]: IOERROR: stating flat /var/imap/user_deny.db: No such file or directory imaps[28382]: DENYDB_ERROR: error reading entry 'fax': cyrusdb error So I guess I'm not using it; I don't even know what this feature is or does. The manpage of imapd doesn't show any way to turn this feature off. This kind of logging is actually a bug because it's a security issue by supporting DOS attacks. Just imagine 1000 users doing something that causes the message to be logged 20 times per second and the system logger having to deal with 20k log messages per second. It seems I'd have to comment out the line and recompile cyrus because that's the only way to disable the message. I'll have to figure out how to do that on Gentoo. Thank you very much for your help! Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -- Bron Gondwana br...@fastmail.fm Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: too much logging
Am 30.04.2015 um 15:10 schrieb Bron Gondwana: You can't remove the db file from under a rubbing instance, only while it is shut down. I could remove it just fine. Did I create issues doing that which might show up later? Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: too much logging
On Thu, Apr 30, 2015, at 11:10 PM, Bron Gondwana wrote: You can't remove the db file from under a rubbing instance, only while it is shut down. Er, a running instance. Great work autocomplete. I'm at decent internet again now. You need to shut down Cyrus and restart it for the existing processes to be closed. Once they have opened the file once, they'll keep expecting it to be there forever. Bron. -- Bron Gondwana br...@fastmail.fm Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: too much logging
On 04/30/15 18:41 +0200, hw wrote: Am 30.04.2015 um 16:35 schrieb Bron Gondwana: On Thu, Apr 30, 2015, at 11:10 PM, Bron Gondwana wrote: You can't remove the db file from under a rubbing instance, only while it is shut down. Er, a running instance. Great work autocomplete. I'm at decent internet again now. You need to shut down Cyrus and restart it for the existing processes to be closed. Once they have opened the file once, they'll keep expecting it to be there forever. Oh, hm, I think I might not have restarted Cyrus after I created the file. Now I restarted, and I'm still getting the 'fetching user_deny.db entry for ...' message. The code I specified was for 2.5.1, which is functionally different from 2.4.17. You'll need to trigger a failure for 'DENYDB-open', which I'm not clear how you would do. Try specifying a non-existant path for your user_deny database or a invalid database format. -- Dan White Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: too much logging
Am 30.04.2015 um 16:35 schrieb Bron Gondwana: On Thu, Apr 30, 2015, at 11:10 PM, Bron Gondwana wrote: You can't remove the db file from under a rubbing instance, only while it is shut down. Er, a running instance. Great work autocomplete. I'm at decent internet again now. You need to shut down Cyrus and restart it for the existing processes to be closed. Once they have opened the file once, they'll keep expecting it to be there forever. Oh, hm, I think I might not have restarted Cyrus after I created the file. Now I restarted, and I'm still getting the 'fetching user_deny.db entry for ...' message. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: too much logging
On 04/29/15 17:21 +0200, hw wrote: Am 29.04.2015 um 16:14 schrieb Dan White: On 04/29/15 16:07 +0200, hw wrote: Hi, is there a way to reduce the log output from cyrus? A lot, if not most, entries say 'imaps[20670]: fetching user_deny.db entry for ...', which seems to be a rather useless information. Which version are you running? 2.4.x changed the behavior of when the user_deny database is opened (at service startup time): http://cyrusimap.org/docs/cyrus-imapd/2.5.1/changes.php 2.4.17 The URL says Modified user_deny.db code to open database once at service startup time. Does this mean that before 2.5.1, the database is being opened and closed all the time, yielding a log message? Correct. But that was an error produced if the user_deny.db file didn't exist, and ended up flooding syslog. That's a different syslog entry from what you're seeing. You might have to modify your syslog config to get rid of them: http://cyrusimap.org/docs/cyrus-imapd/2.5.1/install-configure.php -- Dan White Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: too much logging
On 04/29/15 18:35 +0200, hw wrote: Am 29.04.2015 um 18:15 schrieb Dan White: Does this mean that before 2.5.1, the database is being opened and closed all the time, yielding a log message? Correct. But that was an error produced if the user_deny.db file didn't exist, and ended up flooding syslog. That's a different syslog entry from what you're seeing. You might have to modify your syslog config to get rid of them: http://cyrusimap.org/docs/cyrus-imapd/2.5.1/install-configure.php That is precisely what I do not want to do. There is no point in generating useless log messages, and it should be possible to turn these messages off. They are usually somewhat harmless, though. When there are thousands or millions of them generated, it might indicate that there is some issue that needs to be fixed. Not all imaps processes create so many messages. In this sense, the messages are not useless. But what might cause so many of these messages to be generated within a short time, all day long? user_deny is used to selectively deny access to services for certain users. It's called within the main cmdloop within the imapd code, among other places (pop3, nntpd, lmtpd, and httpd). Each imapd connection could result in many syslog entries depending on what activity the client is performing. The syslog entry is generated with this code: if (!denydb) denydb_open(/*create*/0); if (!denydb) return 0; memset(tok, 0, sizeof(tok)); /* fetch entry for user */ syslog(LOG_DEBUG, fetching user_deny.db entry for '%s', user); If cyrusdb_open cannot successfully open or create the entry (such as a permissions error), then that would effectively stop the syslog entries from being generated. So, setting a bogus value for 'userdeny_db' and/or 'userdeny_db_path' should do what you want in a round about way. That's assuming you're not actually using userdeny of course. -- Dan White Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: too much logging
Am 29.04.2015 um 16:14 schrieb Dan White: On 04/29/15 16:07 +0200, hw wrote: Hi, is there a way to reduce the log output from cyrus? A lot, if not most, entries say 'imaps[20670]: fetching user_deny.db entry for ...', which seems to be a rather useless information. Which version are you running? 2.4.x changed the behavior of when the user_deny database is opened (at service startup time): http://cyrusimap.org/docs/cyrus-imapd/2.5.1/changes.php 2.4.17 The URL says Modified user_deny.db code to open database once at service startup time. Does this mean that before 2.5.1, the database is being opened and closed all the time, yielding a log message? The log file has become 3GB in size now from all these useless messages, and there's only a handfull of users. I don't want to make syslog-ng filter these messages out as they shouldn't be generated in the first place. Some imaps processes actually flood the log file with this message, and I haven't found out yet what causes this. These processes cause relatively high CPU load, and since syslog-ng needs to process all the messages, it in turn also generates relatively high CPU load. What might be going wrong here? Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
