Re: user_deny.db, very high load and Apple-Spotlight

2010-10-19 Thread Patrick Goetz
On 10/14/2010 06:30 AM, Marc Patermann wrote:

 I created user_deny.db with touch to make the one error message go away.
 Now I have lots of fetching ... messages in the log (2.3.16).

 Is there anything to do about this?


I think the suggested solution is don't use bdb for the /var/lib/cyrus 
databases?  The default in 2.4.x is skiplists.

I know this has been mentioned or alluded to several times, but I'm 
still a bit unclear on how to convert an existing set of cyrus bdb 
databases to skiplists on a production system.


-- 
Patrick Goetz

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


Re: user_deny.db, very high load and Apple-Spotlight

2010-10-19 Thread Patrick Goetz
Correction:  I think you have to compile cyrus with no bdb support for 
the user-deny.db error message to go away:

configure --without-bdb


On 10/19/2010 01:20 PM, Patrick Goetz wrote:
 On 10/14/2010 06:30 AM, Marc Patermann wrote:

 I created user_deny.db with touch to make the one error message go away.
 Now I have lots of fetching ... messages in the log (2.3.16).

 Is there anything to do about this?


 I think the suggested solution is don't use bdb for the /var/lib/cyrus
 databases?  The default in 2.4.x is skiplists.




-- 
Patrick Goetz

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


Re: user_deny.db, very high load and Apple-Spotlight

2010-10-15 Thread Ralf Haferkamp
On Thursday 14 October 2010 13:30:22 Marc Patermann wrote:
 Hi,
 
 Mark Heisterkamp schrieb am 12.04.2010 09:03 Uhr:
[..]
  I think we shouldn't advise 5000 users not to use Spotlight, we
  should deactivate user_deny.db. By the way, what is this database
  really good for? If we want someone not to use cyrus-service we
  deny this person by ldap for example. Kenneth Murchison stated in
  some mail on this list that user_deny.db is used once per login,
  that's definitely not true, it is used every time the client 'uses'
  an IMAP-folder and that can be pretty often! Maybe we can change
  this behaviour by some config?
  
  Is it possible to deactivate fetching user_deny.db-entries by some
  config-option or do we have to patch the sources?
 
 I have this issue, too.
 But there is no Mac involved. It is just Thunderbird on the client
 (Win*/Linux) side.
 
 I created user_deny.db with touch to make the one error message go
 away. Now I have lots of fetching ... messages in the log (2.3.16).
 
 Is there anything to do about this?
You could either upgrade to 2.4.0 as the user_deny.db code has been
changed there to only try to open the database once. Or I guess you
could just backport these two patches to your 2.3.16 release:

http://git.cyrusimap.org/cyrus-imapd/commit/?id=e82c657e2f6a3d304d19d737104cec4782da15c0
http://git.cyrusimap.org/cyrus-imapd/commit/?id=3a755fac0d4b43071774af2a646ac4fa51aba8f3

regards,
Ralf
-- 
SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nuernberg)

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


Re: user_deny.db, very high load and Apple-Spotlight

2010-10-15 Thread Jeroen van Meeuwen (Kolab Systems)
Ralf Haferkamp wrote:
 On Thursday 14 October 2010 13:30:22 Marc Patermann wrote:
  Hi,
 
  Mark Heisterkamp schrieb am 12.04.2010 09:03 Uhr:
 [..]
 
   I think we shouldn't advise 5000 users not to use Spotlight, we
   should deactivate user_deny.db. By the way, what is this database
   really good for? If we want someone not to use cyrus-service we
   deny this person by ldap for example. Kenneth Murchison stated in
   some mail on this list that user_deny.db is used once per login,
   that's definitely not true, it is used every time the client 'uses'
   an IMAP-folder and that can be pretty often! Maybe we can change
   this behaviour by some config?
   
   Is it possible to deactivate fetching user_deny.db-entries by some
   config-option or do we have to patch the sources?
  
  I have this issue, too.
  But there is no Mac involved. It is just Thunderbird on the client
  (Win*/Linux) side.
  
  I created user_deny.db with touch to make the one error message go
  away. Now I have lots of fetching ... messages in the log (2.3.16).
  
  Is there anything to do about this?
 
 You could either upgrade to 2.4.0 as the user_deny.db code has been
 changed there to only try to open the database once. Or I guess you
 could just backport these two patches to your 2.3.16 release:
 
 http://git.cyrusimap.org/cyrus-imapd/commit/?id=e82c657e2f6a3d304d19d737104
 cec4782da15c0
 http://git.cyrusimap.org/cyrus-imapd/commit/?id=3a755fac0d4b43071774af2a64
 6ac4fa51aba8f3
 

Could any of you please submit a bug-report in our Bugzilla?

I can then go through the motions of backporting those to the 2.3 branch if 
appropriate (e.g. if Bron and/or Ken agree). You may just get a 2.3.17 out of 
it.

Kind regards,

-- 
Jeroen van Meeuwen
Senior Engineer, Kolab Systems AG

e: vanmeeu...@kolabsys.com
t: +316 42 801 403
w: http://www.kolabsys.com

pgp: 9342 BF08

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


Re: user_deny.db, very high load and Apple-Spotlight

2010-10-15 Thread Ralf Haferkamp
On Friday 15 October 2010 11:47:15 Jeroen van Meeuwen (Kolab Systems) 
wrote:
 Ralf Haferkamp wrote:
  On Thursday 14 October 2010 13:30:22 Marc Patermann wrote:
   Hi,
  
   Mark Heisterkamp schrieb am 12.04.2010 09:03 Uhr:
[..]
   
   I created user_deny.db with touch to make the one error message go
   away. Now I have lots of fetching ... messages in the log
   (2.3.16).
   
   Is there anything to do about this?
  
  You could either upgrade to 2.4.0 as the user_deny.db code has been
  changed there to only try to open the database once. Or I guess you
  could just backport these two patches to your 2.3.16 release:
  
  http://git.cyrusimap.org/cyrus-imapd/commit/?id=e82c657e2f6a3d304d19
  d737104 cec4782da15c0
  http://git.cyrusimap.org/cyrus-imapd/commit/?id=3a755fac0d4b43071774
  af2a64 6ac4fa51aba8f3
 
 Could any of you please submit a bug-report in our Bugzilla?
Acutally there is one already :), that's want resulted in the 2.4 change:

http://bugzilla.cyrusimap.org/bugzilla3/show_bug.cgi?id=3206

 I can then go through the motions of backporting those to the 2.3
 branch if appropriate (e.g. if Bron and/or Ken agree). You may just
 get a 2.3.17 out of it.

thanks,
Ralf
-- 
SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nuernberg)

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


Re: user_deny.db, very high load and Apple-Spotlight

2010-10-15 Thread Jeroen van Meeuwen (Kolab Systems)
Ralf Haferkamp wrote:
 On Friday 15 October 2010 11:47:15 Jeroen van Meeuwen (Kolab Systems)
   You could either upgrade to 2.4.0 as the user_deny.db code has been
   changed there to only try to open the database once. Or I guess you
   could just backport these two patches to your 2.3.16 release:
   
   http://git.cyrusimap.org/cyrus-imapd/commit/?id=e82c657e2f6a3d304d19
   d737104 cec4782da15c0
   http://git.cyrusimap.org/cyrus-imapd/commit/?id=3a755fac0d4b43071774
   af2a64 6ac4fa51aba8f3
  
  Could any of you please submit a bug-report in our Bugzilla?
 
 Acutally there is one already :), that's want resulted in the 2.4 change:
 
 http://bugzilla.cyrusimap.org/bugzilla3/show_bug.cgi?id=3206
 

Sorry, let me rephrase:

Should there be a bug-report already, would you please comment on it with the 
extra information you provided?

Anyway, I've done so accordingly. Thanks for the references!

Kind regards,

-- 
Jeroen van Meeuwen
Senior Engineer, Kolab Systems AG

e: vanmeeu...@kolabsys.com
t: +316 42 801 403
w: http://www.kolabsys.com

pgp: 9342 BF08

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


Re: user_deny.db, very high load and Apple-Spotlight

2010-10-14 Thread Marc Patermann
Hi,

Mark Heisterkamp schrieb am 12.04.2010 09:03 Uhr:

 we graded up to cyrus 2.3.16 a few weeks ago and since then the load 
 average showed values from 200 to 300 a few times per day. The server 
 has 16 cores, 64GB RAM an is attached to a SAN. This machine is quite 
 powerfull. It serves about 5000 mailboxes.
 
 First we touched user_deny.db to get rid of these annoying 
 IOERROR-messages. These messages where replaced by (annoying) 'fetching 
 user_deny.db'-entries. A normal IMAP-user causes about 500 to 1500 of 
 such messages in eight hours.
 
 But we found two users who 'generated' 500 (!) and 25 of such 
 messages in this period. After phoning them we found out, that they 
 where using Mac OS X and Thunderbird 3 (the one with 5 Mio messages) and 
 Mail.app (the other one). Turning off the spotlight-search on 
 IMAP-folders immediately turned the load average down to a normal value 
 (about 0.2).
 
 I think we shouldn't advise 5000 users not to use Spotlight, we should 
 deactivate user_deny.db. By the way, what is this database really good 
 for? If we want someone not to use cyrus-service we deny this person by 
 ldap for example. Kenneth Murchison stated in some mail on this list 
 that user_deny.db is used once per login, that's definitely not true, it 
 is used every time the client 'uses' an IMAP-folder and that can be 
 pretty often! Maybe we can change this behaviour by some config?
 
 Is it possible to deactivate fetching user_deny.db-entries by some 
 config-option or do we have to patch the sources?
I have this issue, too.
But there is no Mac involved. It is just Thunderbird on the client 
(Win*/Linux) side.

I created user_deny.db with touch to make the one error message go away. 
Now I have lots of fetching ... messages in the log (2.3.16).

Is there anything to do about this?


Marc

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


Re: user_deny.db, very high load and Apple-Spotlight

2010-04-12 Thread Joseph Brennan

Mark Heisterkamp heisterk...@rrzn.uni-hannover.de wrote:

 Kenneth Murchison stated in some mail on this list that user_deny.db is
 used once per login, that's definitely not true, it is used every time
 the client 'uses' an IMAP-folder and that can be pretty often!


Some clients open a new login session every time they open a new folder.
Enable telemetry if you want to check what these clients do.


Joseph Brennan
Columbia University Information Technology


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html