Re: autocreate.sieve doesn't work correct. temporary files

2015-10-05 Thread signaldeveloper 
Where are the default sieve rules at? If I go into round cube there is a sieve 
group called round cube. Where is that at?

- Paul

> On Oct 5, 2015, at 8:51 PM, ellie timoney  wrote:
> 
> Hi Artyom,
>  
> Yeah wow, this is really gross.
>  
> I'm pretty sure the gibberish in those "??Default.script.bc" filenames is 
> just whatever junk was in the (uninitialised) sieve_script_dir variable.
>  
> I've had a rummage around, and there's a user_sieve_path() function in 
> imap/user.c that does the heavy lifting of finding a user's sieve script 
> directory.  Looks like sieve_script_dir wanted to be the result of that.  
> Your fix is on the right track. :)
>  
> I've attached a patch for 2.5.x that fixes these paths using 
> user_sieve_path().  It also fixes the assumption that the sievedir value will 
> end in a "/", which it doesn't by default, and shouldn't need to.  Can you 
> try it out and see how it goes?  (The patch should apply cleanly on any 
> version of 2.5, this file has barely changed since it was created.)
>  
> The rest of the autocreate_sieve() function is pretty awful too -- there's 
> almost certainly more bugs in there, and fixing the paths might just shake 
> them out.  I'd like to tidy this up significantly (and make some test cases 
> for it), but in the meantime hopefully this will get you moving forward.
>  
> Cheers,
>  
> ellie
>  
>> On Tue, Oct 6, 2015, at 12:56 AM, Artyom Aleksandrov wrote:
>> Guys I don't understand hot it can work.
>>  
>> I added additional logging and found that sieve_script_dir is not defined.
>> After adding this definition the problem gone.
>>  
>>  
>> 147 /* Check if autocreate_sieve_compiledscript is defined in imapd.conf 
>> */
>> 148 if(!(compiled_source_script = 
>> config_getstring(IMAPOPT_AUTOCREATE_SIEVE_SCRIPT_COMPILED))) {
>> 149 syslog(LOG_WARNING, "autocreate_sieve: 
>> autocreate_sieve_compiledscript option is not defined. Compiling it");
>> 150 do_compile = 1;
>> 151 }
>> 152 
>> 153char userletter[1];
>> 154userletter[0]=userid[0];
>> 155snprintf(sieve_script_dir, MAX_FILENAME, 
>> "%s%s/%s/",sieve_dir,userletter,userid);
>>  
>>  
>>  
>> On Thu, Oct 1, 2015 at 8:49 PM, Artyom Aleksandrov  
>> wrote:
>> Is it works? Which version do you use?
>> Could you guest the reason of the problem? How I can troubleshoot it? 
>>  
>> :/var/lib/cyrus# ls -la
>> total 2176
>> -rw---  1 cyrus mail 124 Sep 25 16:04 ??Default.script.bc
>> -rw---  1 cyrus mail 231 Sep 25 16:04 ??Default.script.script
>> lrwxrwxrwx  1 cyrus mail  17 Sep 25 16:04 ??defaultbc -> 
>> Default.script.bc
>> -rw---  1 cyrus mail 124 Jul  2 12:38 ??N???Default.script.bc
>> -rw---  1 cyrus mail 231 Jul  2 12:38 ??N???Default.script.script
>> lrwxrwxrwx  1 cyrus mail  17 Jul  2 12:38 ??N???defaultbc -> 
>> Default.script.bc
>> -rw---  1 cyrus mail 124 Sep 22 15:10 0#?>??Default.script.bc
>> -rw---  1 cyrus mail 231 Sep 22 15:10 0#?>??Default.script.script
>> lrwxrwxrwx  1 cyrus mail  17 Sep 22 15:10 0#?>??defaultbc -> 
>> Default.script.bc
>>  
>>  
>> On Thu, Oct 1, 2015 at 7:55 PM, Alvin Starr  wrote:
>> I use autocreate.
>>  
>> So there is at least one.
>>  
>>  
>> On 10/01/2015 12:18 PM, Artyom Aleksandrov wrote:
>>> Does anybody use autocreate_sieve?
>>>  
>>> On Sat, Sep 26, 2015 at 1:30 AM, Artyom Aleksandrov 
>>>  wrote:
>>>  
>>> Hello,I want to create default sieve scipt for all my users but I stuck 
>>> with strange problem that looks like the bug. Unfortunately I've never 
>>> wrote on C so it's difficult for me to find it.
>>> When Cyrus (2.5.3 or 2.5.6) create default sieve script it doesn't put file 
>>> in sieve_dir/?/user folder. It jist creates tmp files in configdirectory 
>>> with names like this
>>> -rw---  1 cyrus mail   124 Sep 26 00:41 ?&?P??default.script.bc
>>> -rw---  1 cyrus mail   231 Sep 26 00:41 ?&?P??default.script.script
>>> lrwxrwxrwx  1 cyrus mail17 Sep 26 00:41 ?&?P??defaultbc -> 
>>> default.script.bc
>>>  
>>> There are not checks in this stage so my syslog is clean of error.
>>> Everything seems fine.
>>> 
>>> Sep 26 00:41:34 imapsync cyrus/imap[26117]: autocreate_sieve: Problem 
>>> opening compiled script file: default.script.bc. Compiling it
>>> Sep 26 00:41:34 imapsync cyrus/imap[26117]: autocreate_sieve: Compiled 
>>> sieve script was successfully saved in default.script.bc
>>> Sep 26 00:41:34 imapsync cyrus/imap[26117]: autocreate_sieve: User , 
>>> default sieve script creation succeeded
>>>  
>>> My setting:
>>> autocreate_sieve_script: /var/spool/sieve/global/default.script
>>> autocreate_sieve_script_compile: yes
>>> autocreate_sieve_script_compiled: default.script.bc
>>> sievedir: /var/spool/sieve/
>>> Distributive: Ubuntu 14.04.3
>>>  
>>>  
>>> I'll be glad for any help. )
>>>  
>>> Best regards, Artyom
>>>  
>>>

Re: Kolab 3.4 on CentOS 6.6 (ptload completely failed)

2015-09-25 Thread signaldeveloper 
Guys,

Can you shed any light on the below? Andrea I copied in the Cyrus list guys. 

- Paul

> On Sep 25, 2015, at 11:28 AM, Soliva, Andrea  wrote:
> 
> Hi Paul
> 
> here another issue I found...nothing to do with the issue. If you read in 
> cyrus-imap docu there is recommended to point for performance reason some 
> files to tmpfs files System like /dev/shm. This is specially also noted for 
> cyrus-imapd v2.5. Actually the config would be:
> 
>   # vi /etc/impad.conf
> 
>   --- /etc/impad.conf ---
> 
>   configdirectory: /var/lib/imap
>   # $confdir/proc/ will be used if not defined "proc_path"
>   proc_path: /dev/shm
>   # $confdir/lock will be used if not defined "mboxname_lockpath"
>   mboxname_lockpath: /dev/shm
># $confdir/statuscache.db will be used if not defined 
> "statuscache_db_path"
>   statuscache_db_path: /dev/shm/statuscache.db
># $confdir/tls_sessions.db will be used if not defined 
> "tls_sessions_db_path"
>   tls_sessions_db_path: /dev/shm/tls_sessions.db
># $confdir/deliver.db will be used if not defined "duplicate_db_path"
>   duplicate_db_path: /dev/shm/deliver.db
> 
>   --- /etc/impad.conf ---
> 
> This does not work...which means because the Kolab guys patched the stuff 
> from my Point of view there is a mistake in the code because if you use this 
> config it works as Long as cyurs-imapd does not any maintenance or how every 
> they call this because as soon as this happens you will have following error:
> 
>Aug 24 08:50:25 kolab imap/imaps[7782]: Deleted mailbox 
> comcept.ch!user.andrea^soliva
>Aug 24 08:50:25 kolab imap/imaps[7782]: IOERROR: bogus filename 
> "/dev/shm//tls_sessions.db" in proc_foreach
>Aug 24 08:50:25 kolab imap/imaps[7782]: IOERROR: bogus filename 
> "/dev/shm//deliver.db" in proc_foreach
>Aug 24 08:50:25 kolab imap/imaps[7782]: IOERROR: bogus filename 
> "/dev/shm//sem.slapd-kolab.stats" in proc_foreach
>Aug 24 08:50:25 kolab imap/imaps[7782]: IOERROR: bogus filename 
> "/dev/shm//q" in proc_foreach
>Aug 24 08:50:25 kolab imap/imaps[7782]: IOERROR: bogus filename 
> "/dev/shm//n" in proc_foreach
>Aug 24 08:50:25 kolab imap/imaps[7782]: IOERROR: bogus filename 
> "/dev/shm//statuscache.db" in proc_foreach
>Aug 24 08:50:25 kolab imap/imaps[7782]: IOERROR: bogus filename 
> "/dev/shm//domain" in proc_foreach
> 
> The Point to look at is "/dev/shm//tls_sessions.db" which has one / too much 
> it must be "/dev/shm/tls_sessions.db/". The same happens if you link the 
> files/dir to /dev/shm.
> 
> This only for your info :-)
> 
> ---
> Kind regards
> 
> Andrea Soliva
> 
> Email: andrea.sol...@comcept.ch
> 
> Am 25-09-2015 16:00, schrieb signaldevelo...@gmail.com:
>> The mailboxes show kolab lm?
>> - Paul
>>> On Sep 24, 2015, at 5:49 AM, Soliva, Andrea  
>>> wrote:
>>> Hi Paul
>>> from my Point of view absolutly yes because I also restart on daily base in 
>>> the night all Services because from my Point of view some services are 
>>> eating Memory which means in some parts they have to clean up things. 
>>> Anyway following services are restarted on daily base:
>>> # service rsyslog status ;\
 service dirsrv status ;\
 service mysqld status ;\
 service cyrus-imapd status ;\
 service kolab-saslauthd status ;\
 service wallace status ;\
 service clamd.amavisd status ;\
 service spamassassin status ;\
 service amavisd status ;\
 service postfix status ;\
 service httpd status ;\
 service memcached status ;\
 service kolabd status
>>> rsyslogd (pid  7032) is running...
>>> dirsrv kolab (pid 460) is running...
>>> mysqld (pid  780) is running...
>>> cyrus-imapd (pid  825) is running...
>>> kolab-saslauthd (pid  3111) is running...
>>> wallaced (pid  897) is running...
>>> clamd.amavisd (pid  6886) is running...
>>> spamd (pid  992) is running...
>>> amavisd (pid 6842 6841 6840 6839 6787) is running...
>>> master (pid  1131) is running...
>>> httpd (pid  6928) is running...
>>> memcached (pid  1194) is running...
>>> kolabd (pid  3423) is running...
>>> # ps -ef
>>> UIDPID  PPID  C STIME TTY  TIME CMD
>>> root 1 0  0 Jul15 ?00:00:02 /sbin/init
>>> root 2 0  0 Jul15 ?00:00:00 [kthreadd]
>>> root 3 2  0 Jul15 ?00:00:35 [migration/0]
>>> root 4 2  0 Jul15 ?00:00:06 [ksoftirqd/0]
>>> root 5 2  0 Jul15 ?00:00:00 [stopper/0]
>>> root 6 2  0 Jul15 ?00:00:23 [watchdog/0]
>>> root 7 2  0 Jul15 ?00:00:37 [migration/1]
>>> root 8 2  0 Jul15 ?00:00:00 [stopper/1]
>>> root 9 2  0 Jul15 ?00:00:06 [ksoftirqd/1]
>>> root10 2  0 Jul15 ?00:00:22 [watchdog/1]
>>> root11 2  0 Jul15 ?

Re: IMAP processes out of control

2015-09-23 Thread signaldeveloper 
Again this is active sync devices that are connecting with a ton of pushed 
folders. The more you tell it to sync (folders) the more processes it's going 
to fork for each user folder. Is this affecting performance that bad? What's 
your hardware? 

- Paul

> On Sep 22, 2015, at 7:43 PM, Moby  wrote:
> 
> 
> 
> On 9/22/2015 18:12, Shaheen Bakhtiar wrote:
>>> On Sep 22, 2015, at 2:17 PM, Andrew Morgan  wrote:
>>> 
 On Tue, 22 Sep 2015, Shaheen Bakhtiar wrote:
 
 It happened again….. although it took longer for it to happen, this has 
 been happening only since the upgrade in Jun.
 
 The number of imap processes continues to increase until the server is 
 completely OOM. the increase is drastic and all of a sudden.
>>> You should probably set maxchild to a value that won't run your server out 
>>> of memory.  :)
>>> 
>>> Have you looked at the processes to see what they have in common?  For 
>>> example, sometimes an IMAP client will run amok and make hundreds or 
>>> thousands of connections.  Or perhaps the processes are all stuck waiting 
>>> on a lock, etc.
>>> 
>>> lsof, strace, netstat, and your Cyrus logs can help a lot.
>>> 
>>>Andy
>> 
>> 
>> [shawn@postoffice ~]$ ps aux | grep imapd | wc -l
>> 255
>> [shawn@postoffice ~]# ps aux | grep imapds | wc -l
>> 1
>> [shawn@postoffice ~]# ps aux | grep pop3d | wc -l
>> 9
>> [shawn@postoffice ~]# ps aux | grep timseived | wc -l
>> 1
>> [shawn@postoffice ~]# ps aux | grep lmtpunix | wc -l
>> 1
>> 
>> Based on that output I changed the configuration file (below) adding 
>> maxchild. Most likely all my users have their clients open, and from 
>> previous monitoring I average about 200 instances of imapd:
>> 
>> # standard standalone server implementation
>> 
>> START {
>>   # do not delete this entry!
>>   recover   cmd="ctl_cyrusdb -r"
>> 
>>   # this is only necessary if using idled for IMAP IDLE
>>   idled cmd="idled"
>> }
>> 
>> # UNIX sockets start with a slash and are put into /var/lib/imap/sockets
>> SERVICES {
>>   # add or remove based on preferences
>>   imap  cmd="imapd" listen="imap" prefork=5 maxchild=300
>>   imaps cmd="imapd -s" listen="imaps" prefork=1 maxchild=100
>>   pop3  cmd="pop3d" listen="pop3" prefork=3 maxchild=5
>>   pop3s cmd="pop3d -s" listen="pop3s" prefork=1 maxchild=5
>>   sieve cmd="timsieved" listen="sieve" prefork=0
>> 
>>   # these are only necessary if receiving/exporting usenet via NNTP
>> #  nntp cmd="nntpd" listen="nntp" prefork=3
>> #  nntpscmd="nntpd -s" listen="nntps" prefork=1
>> 
>>   # at least one LMTP is required for delivery
>> #  lmtp cmd="lmtpd" listen="lmtp" prefork=0
>>   lmtpunix  cmd="lmtpd" listen="/var/lib/imap/socket/lmtp" prefork=1
>> 
>>   # this is only necessary if using notifications
>> #  notify   cmd="notifyd" listen="/var/lib/imap/socket/notify" 
>> proto="udp" prefork=1
>> }
>> 
>> EVENTS {
>>   # this is required
>>   checkpointcmd="ctl_cyrusdb -c" period=30
>> 
>>   # this is only necessary if using duplicate delivery suppression,
>>   # Sieve or NNTP
>>   delprune  cmd="cyr_expire -E 3" at=0400
>> 
>>   # this is only necessary if caching TLS sessions
>>   tlsprune  cmd="tls_prune" at=0400
>> }
>> 
>> 
>> 
>> Comments, concerns??? I’m going to keep observium open on a separate screen 
>> and watch when it starts going up, than do the lsof,netstat, and watch logs 
>> to see if I can tell why the sudden upticks.
>> 
>> Cyrus Home Page: http://www.cyrusimap.org/
>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>> To Unsubscribe:
>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> I have seen that when some of my users fiddle around on their iphone - 
> usually the complaints start with "I cannot get mail on my phone" and 
> around the same time the process count starts going up.  Very 
> intermittent though, and has not occurred since all users upgraded to 
> IOS 9.  Just my USD 0.02 worth...
> 
> -- 
> --Moby
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: IMAP processes out of control

2015-09-23 Thread signaldeveloper 
Agreed with Andy. Possibly an infected machine and a bad outlook hook in that's 
trying to send spam over your server. You can also try to setup message 
thresholds to combat this IF that is the case. 

- Paul

> On Sep 23, 2015, at 4:52 PM, Andrew Morgan  wrote:
> 
> You should be able to have a LOT of imapd processes with that much RAM. On a 
> server with 8GB of RAM, I have maxchild=4000 for imap and maxchild=1000 for 
> imaps.
> 
> However, it is good to leave lots of RAM for caching, so those limits are 
> mainly in place to prevent a bad client from causing a low-memory condition 
> on the server.
> 
> When you see the process count increasing, you need to identify what the 
> "extra" processes are doing.  You will probably be able to identify a pattern 
> by looking at the cyrus proc files.  Try this:
> 
>  cat ${configdir}/proc/* | sort
> 
> The format of the proc file is:
> 
>  hostname [IP-address] authenticated-username SELECTed-mailbox
> 
> I bet you'll see a lot of connections from one host or user.
> 
> You can also use lsof and netstat if things are hanging before the proc file 
> is created.
> 
>Andy
> 
>> On Wed, 23 Sep 2015, Shaheen Bakhtiar wrote:
>> 
>> 2 x AMD quad Core 64bit 4G RAM
>> 
>> This morning I woke up to a plethora of complaints that people were not able 
>> to access their emails. I remove the aforementioned maxchild from the 
>> configurations and restart to server. Once I did that people were able to 
>> re-connect with no problems.
>> 
>> I did not have these types of problems with the older version (I believe was 
>> 2.3.19). Just since I upgraded to the latest version of Cyrus. 
>> Current version is:
>> [root@postoffice ~]# dnf info cyrus-imapd
>> Last metadata expiration check performed 1:06:02 ago on Wed Sep 23 07:12:41 
>> 2015.
>> Installed Packages
>> Name: cyrus-imapd
>> Arch: x86_64
>> Epoch   : 0
>> Version : 2.4.17
>> Release : 9.fc22
>> 
>> Running on Fedora Core 22 64bit
>> 
 On Sep 23, 2015, at 7:44 AM, signaldevelo...@gmail.com wrote:
 Again this is active sync devices that are connecting with a ton of pushed 
 folders. The more you tell it to sync (folders) the more processes it's 
 going to fork for each user folder. Is this affecting performance that 
 bad? What's your hardware? - Paul
 On Sep 22, 2015, at 7:43 PM, Moby  wrote:
 On 9/22/2015 18:12, Shaheen Bakhtiar wrote:
>>> On Sep 22, 2015, at 2:17 PM, Andrew Morgan  wrote:
>>> On Tue, 22 Sep 2015, Shaheen Bakhtiar wrote:
>>> It happened again….. although it took longer for it to happen, this has 
>>> been happening only since the upgrade in Jun.
>>> The number of imap processes continues to increase until the server is 
>>> completely OOM. the increase is drastic and all of a sudden.
>> You should probably set maxchild to a value that won't run your server 
>> out of memory.  :)
>> Have you looked at the processes to see what they have in common?  For 
>> example, sometimes an IMAP client will run amok and make hundreds or 
>> thousands of connections.  Or perhaps the processes are all stuck 
>> waiting on a lock, etc.
>> lsof, strace, netstat, and your Cyrus logs can help a lot.
>> 
>>  Andy
> [shawn@postoffice ~]$ ps aux | grep imapd | wc -l
> 255
> [shawn@postoffice ~]# ps aux | grep imapds | wc -l
> 1
> [shawn@postoffice ~]# ps aux | grep pop3d | wc -l
> 9
> [shawn@postoffice ~]# ps aux | grep timseived | wc -l
> 1
> [shawn@postoffice ~]# ps aux | grep lmtpunix | wc -l
> 1
> Based on that output I changed the configuration file (below) adding 
> maxchild. Most likely all my users have their clients open, and from 
> previous monitoring I average about 200 instances of imapd:
> # standard standalone server implementation
> START {
> # do not delete this entry!
> recover   cmd="ctl_cyrusdb -r"
> 
> # this is only necessary if using idled for IMAP IDLE
> idled cmd="idled"
> }
> # UNIX sockets start with a slash and are put into /var/lib/imap/sockets
> SERVICES {
> # add or remove based on preferences
> imap  cmd="imapd" listen="imap" prefork=5 maxchild=300
> imaps cmd="imapd -s" listen="imaps" prefork=1 maxchild=100
> pop3  cmd="pop3d" listen="pop3" prefork=3 maxchild=5
> pop3s cmd="pop3d -s" listen="pop3s" prefork=1 maxchild=5
> sieve cmd="timsieved" listen="sieve" prefork=0
> 
> # these are only necessary if receiving/exporting usenet via NNTP
> #  nntp cmd="nntpd" listen="nntp" prefork=3
> #  nntpscmd="nntpd -s" listen="nntps" prefork=1
> 
> # at least one LMTP is required for delivery
> #  lmtp cmd="lmtpd" listen="lmtp" prefork=0
> lmtpunix  cmd="lmtpd" listen="/var/lib/imap/socket/lmtp"

Re: number of imapd processes never decrease

2015-09-17 Thread signaldeveloper 
You have IMAP with max children set at 3k. 

- Paul

> On Sep 17, 2015, at 1:51 AM, Konrad Mauz  wrote:
> 
> Hi,
> 
> on monday I updated our cyrus mailsystem to version 2.5.6 ( coming from 2.3.x 
> ). The upgrade process was successfull.
> 
> 
> Problem 1: The number of imapd processes increases during the day ( normal! ) 
> but keep increasing during the night ( not normal ).
> I have a graph attached.
> Cyrus is running an a recently updated Ubuntu 14.04. lts server. Is there 
> anything I can do?
> 
> Problem 2: In my start-stop script i sent SIGTERM to the master process. The 
> master exists, but sometimes most imapd process lists as "defunc" with "ps 
> -ef". They never exits an must be killed with kill -9 or by a server reboot. 
> Is this the wrong way?
> 
> Regards,
> 
> Konrad
> 
> cyrus.conf:
> 
> START {
>  # do not delete this entry!
>  recover   cmd="ctl_cyrusdb -r"
> 
>  # this is only necessary if using idled for IMAP IDLE
>  idled cmd="idled"
> }
> 
> # UNIX sockets start with a slash and are put into /var/imap/socket
> SERVICES {
>  # add or remove based on preferences
>  imap  cmd="imapd" listen="imap" prefork=10 maxchild=3000
>  imaps cmd="imapd -s" listen="imaps" prefork=10 maxchild=3000
>  pop3  cmd="pop3d" listen="pop3" prefork=5 maxchild=100
>  pop3s cmd="pop3d -s" listen="pop3s" prefork=5 maxchild=100
>  sieve cmd="timsieved" listen="sieve" prefork=1
> 
>  # these are only necessary if receiving/exporting usenet via NNTP
> #  nntp cmd="nntpd" listen="nntp" prefork=0
> #  nntpscmd="nntpd -s" listen="nntps" prefork=0
> 
>  # at least one LMTP is required for delivery
>  lmtpunix  cmd="lmtpd" listen="/srv/cyrus/imap/socket/lmtp" prefork=1 
> maxchild=5
> 
> # this is only necessary if using notifications
>  notifycmd="notifyd" listen="/srv/cyrus/imap/socket/notify" 
> proto="udp" prefork=0
> # syncserver   cmd="sync_server" listen="csync"
> }
> 
> EVENTS {
>  # this is required
>  checkpointcmd="ctl_cyrusdb -c" period=30
> 
>  # this is only necessary if using duplicate delivery suppression,
>  # Sieve or NNTP
>  # -X 60 -> expunged mails are unexpungable for 60 days
>  # -D 60 -> deleted forlders are undeleable for 60 days
>  # -E 7 -> delete entries in des duplicate database older then 7 days
>  delprune  cmd="cyr_expire -D 60 -X 60 -E 7" at=0400
> 
>  # this is only necessary if caching TLS sessions
>  tlsprune  cmd="tls_prune" at=0400
>  squatindexcmd="squatter -i -v -s" at=0500
> }
> 
> 
> 
> -- 
> Hochschule Konstanz - Rechenzentrum
> Konrad Mauz km...@htwg-konstanz.de
> Tel: +497531206-472, Fax: -153
> http://www.htwg-konstanz.de/rz.html
> 
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: number of imapd processes never decrease

2015-09-17 Thread signaldeveloper 
I would consider how many users you have connected as well as each type of 
device. If you are running some type of AS setup, each device will maintain an 
open connection and cause a process to happen. How many users?


- Paul

> On Sep 17, 2015, at 9:23 AM, Vladislav Kurz  
> wrote:
> 
>> On Thursday 17 of September 2015 Niels Dettenbach  wrote:
>> 
>> Am Donnerstag, 17. September 2015, 14:19:42 schrieb Sebastian Hagedorn:
 Cyrus is running an a recently updated Ubuntu 14.04. lts server. Is
 there anything I can do?
>>> 
>>> we are still on 2.4.x, but we had a similar problem that we resolved this
>>> 
>>> way:
>>>  imap  cmd="imapd -U 10" listen="imap.uni-koeln.de:imap"  ...
>> 
>> ...interesting to see that this happens on other Ubuntu systems as well - i
>> saw this on some Ubuntu systems (only on Ubuntu setups) in the past too,
>> without havin the option to dig in deeper what's happen there.
>> 
>> If someone has further investigation results or experiences about this,
>> this would be interesting...
> 
> I have set "imapd -U 30", maxchild=400, and I see gradual grow up of process 
> numbers. Some instances are very long lived, (almost 3 months). In 
> /var/lib/cyrus/procs I can see that some users have about a dozen of 
> connections to the same mailbox/folder (trash is a favorite). And also quite 
> a 
> lot connections with no user associated. Thus from time to time (when I get 
> above 350 processes), I kill imapd processes with no user, or with duplicates 
> (same IP, same user). I just feel as if the -U option does not work well.
> 
> Just to note, the number of processes is decreasing during night, but the max 
> and min values grow each day. I have a graph if someone is interested.
> 
> -- 
> Best Regards
>Vladislav Kurz
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus tweaks (slow on roundcube)

2015-09-11 Thread signaldeveloper 
I tried imapproxy. It is the same speed. And again, definitely not hardware 
related. 

I see in the logs in queries the proxy and that works fine but not sure why 
it's still the same speed. 


- Paul

> On Sep 11, 2015, at 2:47 PM, Andrew Morgan  wrote:
> 
>> On Thu, 10 Sep 2015, signaldevelo...@gmail.com wrote:
>> 
>> Is there some type of log I can provide from Cyrus / sasl to help diagnose 
>> this better to the kolab guys? Other kolab guys I know say their entropy is 
>> right where I'm at and they aren't experiencing these slowness issues.
>> 
>> Are their sasl or Cyrus logs I can provide?
> 
> Maybe I missed this detail earlier in the thread, but why not run an IMAP 
> proxy to reduce the rate of new connections to Cyrus?  Making a new IMAP 
> connection with every click seems abusive! :)
> 
>Andy

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus tweaks (slow on roundcube)

2015-09-11 Thread signaldeveloper 
Andre,

Thanks for the info!! Two questions since sasl is still new to me:

1) How many processes should I have running? Is there an option somewhere to 
adjust this or see it?

2) I installed havaged, but the process instantly crashes and tells me a sub 
system is locked when I try to restart it. Any ideas on that? (On centos 6)


Thanks again!

- Paul

> On Sep 11, 2015, at 2:59 PM, Andre Felipe Machado 
>  wrote:
> 
> Hello,
> 
> By your numbers it seems that your machine is able to generate random numbers 
> at good speed. But the problem is WHEN and HOW OFTEN.
> 
> Afaik, the linux kernel waits too long to trigger the process to generate 
> random numbers and fast paced process spawning or ssl connections could 
> deplete pool before the process is triggered again.
> 
> This is the problem that haveged could solve. Triggering a random numbers 
> generation at a higher threshold and at higher frequency.
> 
> http://blog-ftweedal.rhcloud.com/2014/05/more-entropy-with-haveged/
> 
> Well, it is only ONE of possible causes of your problem. Unfortunately one 
> obscure and difficult to identify because it does not generate errors, 
> crashes or logs. Simply slowness.
> 
> Had you checked disk latency? Does your servers have enough sasl processes?
> 
> We use Debian and did not find haveged installation issues, so you will have 
> to search a bit more about your running errors.
> 
> Regards.
> 
> Andre Felipe
> 
> http://www.techforce.com.br
> 
>  
> 
> Paul Bronson  wrote ..
> 
> Guys,
>  
> I ran cat /dev/urandom | rngtest -c 1000
>  
> and got:
>  
> rngtest: starting FIPS tests...
> rngtest: bits received from input: 2032
> rngtest: FIPS 140-2 successes: 998
> rngtest: FIPS 140-2 failures: 2
> rngtest: FIPS 140-2(2001-10-10) Monobit: 0
> rngtest: FIPS 140-2(2001-10-10) Poker: 0
> rngtest: FIPS 140-2(2001-10-10) Runs: 1
> rngtest: FIPS 140-2(2001-10-10) Long run: 1
> rngtest: FIPS 140-2(2001-10-10) Continuous run: 0
> rngtest: input channel speed: (min=22.980; avg=501.129; max=19073.486)Mibits/s
> rngtest: FIPS tests speed: (min=98.317; avg=121.603; max=131.541)Mibits/s
> rngtest: Program run time: 198018 microseconds
>  
>  
> Does this look bad to you considering all of my slow SASL auths? (no haveged 
> is on at this point.. available entropy is between 131 - 160... pool size is 
> default 4096.
>  
> I also tried installing haveged, which worked fine, but as soon as I started 
> the service it said something like process dead, sub sys locked... ? Sorry, 
> entropy is fairly new to me.
>  
>  
> 
>> On Thu, Sep 10, 2015 at 5:24 PM,  wrote:
>> Andre,
>> 
>> Really? What should it be? I was curious and checked.. Entropy on some of my 
>> other big time production servers for email is only about 200) and its 
>> lightning fast?
>> 
>> - Paul
>> 
>> > On Sep 10, 2015, at 5:00 PM, Andre Felipe Machado 
>> >  wrote:
>> >
>> > Hello,
>> > Entropy of 158 is way too low for production servers. And this *MAY* cause 
>> > weird
>> > slowness without logging any  errors.
>> > You could install "haveged" and configure for max threshold levels on 
>> > production
>> > servers.
>> > https://packages.debian.org/search?keywords=haveged
>> >
>> > Regards.
>> >
>> > Andre Felipe
>> > http://www.techforce.com.br
>> >
>> >
>> >
>> > signaldevelo...@gmail.com wrote ..
>> ! t;> Ru dy,
>> >>
>> >> Entropy is 158 I just looked. And as far as compiling against urandom, to 
>> >> be
>> > honest
>> >> I'm
>> >> not sure.
>> >>
>> >> - Paul
>> >>
>> >>
>> >>
>> >>
>> >>> On Sep 6, 2015, at 9:50 PM, Rudy Gevaert  wrote:
>> >>>
>> >>>
>> >>> Quoting signaldevelo...@gmail.com, Mon, 07 Sep 2015:
>> >>>
>>  Hosts file is fine I checked that, thanks. Kolab uses 389 to
>>  authenticate for everything, so Cyrus is using LDAP as you can see
>>  above. I think the problem lies in the constant TLS logins into
>>  Cyrus for every click:
>> 
>>  imap[2281]: login: localhost [::1] john...@domain.com PLAIN+TLS User
>>  logged in
>>  SESSIONID=
>>  Sep  5 20:54:51 es1 imap[2281]: USAGE john...@domain.com user:
>>  0.009998 sys: 0.006999
>> 
>> 
>>  Again its only one user, on roundcube... I am afraid to put any more
>>  users on it. There doesn't seem to be much of performance tweaks
>>  with Cyrus around the web either...
>> >>>
>> >>> does your system have enough entropy?
>> >>>
>> >>> Is saslauthd compiled against /dev/urandom?
>> >>>
>> >>> Rudy
>> >>>
>> >>> --
>> >>> -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- ! -- -- -- -- -- --
>> >>> Rudy Gevaert e-mail: rudy.geva...@ugent.be
>> >>> Directie ICT, Afdeling Infrastructuur
>> >>> Groep Systemen  tel: +32 9 264 4750
>> >>> Universiteit Gent   fax: +32 9 264 4994
>> >>> Krijgslaan 281,

Re: Cyrus tweaks (slow on roundcube)

2015-09-11 Thread signaldeveloper 
Hi Patrick,

Then do what with it?

- Paul

> On Sep 11, 2015, at 3:53 PM, Patrick Boutilier  wrote:
> 
>> On 09/11/2015 04:12 PM, signaldevelo...@gmail.com wrote:
>> Andre,
>> 
>> Thanks for the info!! Two questions since sasl is still new to me:
>> 
>> 1) How many processes should I have running? Is there an option
>> somewhere to adjust this or see it?
>> 
>> 2) I installed havaged, but the process instantly crashes and tells me a
>> sub system is locked when I try to restart it. Any ideas on that? (On
>> centos 6)
> 
> 
> For the lock part look in /var/lock/subsys/ for a file called havaged or 
> similar.
> 
>> 
>> 
>> Thanks again!
>> 
>> - Paul
> 
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus tweaks (slow on roundcube)

2015-09-11 Thread signaldeveloper 
When I installed havaged the process died instantly and gives me a locked sub 
system. If I restart it again, instantly dies again. Im on centos. Any ideas 
why this is happening?
 Anyone else experienced this?

- Paul

> On Sep 11, 2015, at 1:54 PM, Andre Felipe Machado 
>  wrote:
> 
> Hello, 
> We setup haveged threshold at 2048 (its max pool size is 4096 , afaik) for our
> high load cyrus imap servers.
> At our cyrus imap servers the depletion bursts are amazing.
> Watch the entropy available during your peak ours and you will get an overview
> of your needs.
> Regards.
> Andre Felipe
> 
> 
> signaldevelo...@gmail.com wrote ..
>> Andre,
>> 
>> Really? What should it be? I was curious and checked.. Entropy on some of my 
>> other
>> big time production servers for email is only about 200) and its lightning 
>> fast?
>> 
>> - Paul
>> 
>>> On Sep 10, 2015, at 5:00 PM, Andre Felipe Machado
> 
>> wrote:
>>> 
>>> Hello,
>>> Entropy of 158 is way too low for production servers. And this *MAY* cause 
>>> weird
>>> slowness without logging any  errors.
>>> You could install "haveged" and configure for max threshold levels on 
>>> production
>>> servers.
>>> https://packages.debian.org/search?keywords=haveged
>>> 
>>> Regards.
>>> 
>>> Andre Felipe
>>> http://www.techforce.com.br
>>> 
>>> 
>>> 
>>> signaldevelo...@gmail.com wrote ..
 Rudy,
 
 Entropy is 158 I just looked. And as far as compiling against urandom, to 
 be
>>> honest
 I'm
 not sure. 
 
 - Paul
 
 
 
 
> On Sep 6, 2015, at 9:50 PM, Rudy Gevaert  wrote:
> 
> 
> Quoting signaldevelo...@gmail.com, Mon, 07 Sep 2015:
> 
>> Hosts file is fine I checked that, thanks. Kolab uses 389 to  
>> authenticate for everything, so Cyrus is using LDAP as you can see  
>> above. I think the problem lies in the constant TLS logins into  
>> Cyrus for every click:
>> 
>> imap[2281]: login: localhost [::1] john...@domain.com PLAIN+TLS User  
>> logged in  
>> SESSIONID=
>> Sep  5 20:54:51 es1 imap[2281]: USAGE john...@domain.com user:  
>> 0.009998 sys: 0.006999
>> 
>> 
>> Again its only one user, on roundcube... I am afraid to put any more  
>> users on it. There doesn't seem to be much of performance tweaks  
>> with Cyrus around the web either...
> 
> does your system have enough entropy?
> 
> Is saslauthd compiled against /dev/urandom?
> 
> Rudy
> 
> -- 
> -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
> Rudy Gevaert e-mail: rudy.geva...@ugent.be
> Directie ICT, Afdeling Infrastructuur
> Groep Systemen  tel: +32 9 264 4750
> Universiteit Gent   fax: +32 9 264 4994
> Krijgslaan 281, gebouw S9, 9000 Gent, Belgie   www.UGent.be
> -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
> 
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
 
 Cyrus Home Page: http://www.cyrusimap.org/
 List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
 To Unsubscribe:
 https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>>> 
>>> Cyrus Home Page: http://www.cyrusimap.org/
>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>> To Unsubscribe:
>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus tweaks (slow on roundcube)

2015-09-11 Thread signaldeveloper 
Okay so I figured out since this is a container and not a VM I can't install 
haveged on it. Awesome I learned something. Okay now... Next question to solve 
this insanity. 

Can I point everything Cyrus/SASL and TLS Related to use urandom instead of 
random?

I found this:

http://security.stackexchange.com/a/14399/86596

But I don't want to do something to jeapordize my whole system. 

I think this is the last piece of the puzzle. 

Thanks again guys. 

- Paul

> On Sep 11, 2015, at 8:03 PM, Patrick Boutilier  wrote:
> 
> Delete it. Then you can try to start havaged and see if it crashes again.
> 
> 
>> On 09/11/2015 08:30 PM, signaldevelo...@gmail.com wrote:
>> Hi Patrick,
>> 
>> Then do what with it?
>> 
>> - Paul
>> 
 On Sep 11, 2015, at 3:53 PM, Patrick Boutilier  
 wrote:
 
 On 09/11/2015 04:12 PM, signaldevelo...@gmail.com wrote:
 Andre,
 
 Thanks for the info!! Two questions since sasl is still new to me:
 
 1) How many processes should I have running? Is there an option
 somewhere to adjust this or see it?
 
 2) I installed havaged, but the process instantly crashes and tells me a
 sub system is locked when I try to restart it. Any ideas on that? (On
 centos 6)
>>> 
>>> 
>>> For the lock part look in /var/lock/subsys/ for a file called havaged or 
>>> similar.
>>> 
 
 
 Thanks again!
 
 - Paul
>>> 
>>> 
>>> 
>>> Cyrus Home Page: http://www.cyrusimap.org/
>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>> To Unsubscribe:
>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus tweaks (slow on roundcube)

2015-09-11 Thread signaldeveloper 
Nope, dies instantly and locks it again. "Haveged dead but subsys locked"


- Paul

> On Sep 11, 2015, at 8:03 PM, Patrick Boutilier  wrote:
> 
> Delete it. Then you can try to start havaged and see if it crashes again.
> 
> 
>> On 09/11/2015 08:30 PM, signaldevelo...@gmail.com wrote:
>> Hi Patrick,
>> 
>> Then do what with it?
>> 
>> - Paul
>> 
 On Sep 11, 2015, at 3:53 PM, Patrick Boutilier  
 wrote:
 
 On 09/11/2015 04:12 PM, signaldevelo...@gmail.com wrote:
 Andre,
 
 Thanks for the info!! Two questions since sasl is still new to me:
 
 1) How many processes should I have running? Is there an option
 somewhere to adjust this or see it?
 
 2) I installed havaged, but the process instantly crashes and tells me a
 sub system is locked when I try to restart it. Any ideas on that? (On
 centos 6)
>>> 
>>> 
>>> For the lock part look in /var/lock/subsys/ for a file called havaged or 
>>> similar.
>>> 
 
 
 Thanks again!
 
 - Paul
>>> 
>>> 
>>> 
>>> Cyrus Home Page: http://www.cyrusimap.org/
>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>> To Unsubscribe:
>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus tweaks (slow on roundcube)

2015-09-11 Thread signaldeveloper 
So I tried:

haveged -r 0 

and the service now works but entropy is still 129

- Paul

> On Sep 11, 2015, at 8:03 PM, Patrick Boutilier  wrote:
> 
> Delete it. Then you can try to start havaged and see if it crashes again.
> 
> 
>> On 09/11/2015 08:30 PM, signaldevelo...@gmail.com wrote:
>> Hi Patrick,
>> 
>> Then do what with it?
>> 
>> - Paul
>> 
 On Sep 11, 2015, at 3:53 PM, Patrick Boutilier  
 wrote:
 
 On 09/11/2015 04:12 PM, signaldevelo...@gmail.com wrote:
 Andre,
 
 Thanks for the info!! Two questions since sasl is still new to me:
 
 1) How many processes should I have running? Is there an option
 somewhere to adjust this or see it?
 
 2) I installed havaged, but the process instantly crashes and tells me a
 sub system is locked when I try to restart it. Any ideas on that? (On
 centos 6)
>>> 
>>> 
>>> For the lock part look in /var/lock/subsys/ for a file called havaged or 
>>> similar.
>>> 
 
 
 Thanks again!
 
 - Paul
>>> 
>>> 
>>> 
>>> Cyrus Home Page: http://www.cyrusimap.org/
>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>> To Unsubscribe:
>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus tweaks (slow on roundcube)

2015-09-10 Thread signaldeveloper 
Is there some type of log I can provide from Cyrus / sasl to help diagnose this 
better to the kolab guys? Other kolab guys I know say their entropy is right 
where I'm at and they aren't experiencing these slowness issues. 

Are their sasl or Cyrus logs I can provide?

- Paul

> On Sep 10, 2015, at 5:00 PM, Andre Felipe Machado 
>  wrote:
> 
> Hello,
> Entropy of 158 is way too low for production servers. And this *MAY* cause 
> weird
> slowness without logging any  errors.
> You could install "haveged" and configure for max threshold levels on 
> production
> servers.
> https://packages.debian.org/search?keywords=haveged
> 
> Regards.
> 
> Andre Felipe
> http://www.techforce.com.br
> 
> 
> 
> signaldevelo...@gmail.com wrote ..
>> Rudy,
>> 
>> Entropy is 158 I just looked. And as far as compiling against urandom, to be
> honest
>> I'm
>> not sure. 
>> 
>> - Paul
>> 
>> 
>> 
>> 
>>> On Sep 6, 2015, at 9:50 PM, Rudy Gevaert  wrote:
>>> 
>>> 
>>> Quoting signaldevelo...@gmail.com, Mon, 07 Sep 2015:
>>> 
 Hosts file is fine I checked that, thanks. Kolab uses 389 to  
 authenticate for everything, so Cyrus is using LDAP as you can see  
 above. I think the problem lies in the constant TLS logins into  
 Cyrus for every click:
 
 imap[2281]: login: localhost [::1] john...@domain.com PLAIN+TLS User  
 logged in  
 SESSIONID=
 Sep  5 20:54:51 es1 imap[2281]: USAGE john...@domain.com user:  
 0.009998 sys: 0.006999
 
 
 Again its only one user, on roundcube... I am afraid to put any more  
 users on it. There doesn't seem to be much of performance tweaks  
 with Cyrus around the web either...
>>> 
>>> does your system have enough entropy?
>>> 
>>> Is saslauthd compiled against /dev/urandom?
>>> 
>>> Rudy
>>> 
>>> -- 
>>> -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
>>> Rudy Gevaert e-mail: rudy.geva...@ugent.be
>>> Directie ICT, Afdeling Infrastructuur
>>> Groep Systemen  tel: +32 9 264 4750
>>> Universiteit Gent   fax: +32 9 264 4994
>>> Krijgslaan 281, gebouw S9, 9000 Gent, Belgie   www.UGent.be
>>> -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
>>> 
>>> 
>>> 
>>> Cyrus Home Page: http://www.cyrusimap.org/
>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>> To Unsubscribe:
>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>> 
>> Cyrus Home Page: http://www.cyrusimap.org/
>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>> To Unsubscribe:
>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus tweaks (slow on roundcube)

2015-09-10 Thread signaldeveloper 
Andre,

Really? What should it be? I was curious and checked.. Entropy on some of my 
other big time production servers for email is only about 200) and its 
lightning fast?

- Paul

> On Sep 10, 2015, at 5:00 PM, Andre Felipe Machado 
>  wrote:
> 
> Hello,
> Entropy of 158 is way too low for production servers. And this *MAY* cause 
> weird
> slowness without logging any  errors.
> You could install "haveged" and configure for max threshold levels on 
> production
> servers.
> https://packages.debian.org/search?keywords=haveged
> 
> Regards.
> 
> Andre Felipe
> http://www.techforce.com.br
> 
> 
> 
> signaldevelo...@gmail.com wrote ..
>> Rudy,
>> 
>> Entropy is 158 I just looked. And as far as compiling against urandom, to be
> honest
>> I'm
>> not sure. 
>> 
>> - Paul
>> 
>> 
>> 
>> 
>>> On Sep 6, 2015, at 9:50 PM, Rudy Gevaert  wrote:
>>> 
>>> 
>>> Quoting signaldevelo...@gmail.com, Mon, 07 Sep 2015:
>>> 
 Hosts file is fine I checked that, thanks. Kolab uses 389 to  
 authenticate for everything, so Cyrus is using LDAP as you can see  
 above. I think the problem lies in the constant TLS logins into  
 Cyrus for every click:
 
 imap[2281]: login: localhost [::1] john...@domain.com PLAIN+TLS User  
 logged in  
 SESSIONID=
 Sep  5 20:54:51 es1 imap[2281]: USAGE john...@domain.com user:  
 0.009998 sys: 0.006999
 
 
 Again its only one user, on roundcube... I am afraid to put any more  
 users on it. There doesn't seem to be much of performance tweaks  
 with Cyrus around the web either...
>>> 
>>> does your system have enough entropy?
>>> 
>>> Is saslauthd compiled against /dev/urandom?
>>> 
>>> Rudy
>>> 
>>> -- 
>>> -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
>>> Rudy Gevaert e-mail: rudy.geva...@ugent.be
>>> Directie ICT, Afdeling Infrastructuur
>>> Groep Systemen  tel: +32 9 264 4750
>>> Universiteit Gent   fax: +32 9 264 4994
>>> Krijgslaan 281, gebouw S9, 9000 Gent, Belgie   www.UGent.be
>>> -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
>>> 
>>> 
>>> 
>>> Cyrus Home Page: http://www.cyrusimap.org/
>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>> To Unsubscribe:
>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>> 
>> Cyrus Home Page: http://www.cyrusimap.org/
>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>> To Unsubscribe:
>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Disappearing Mailbox Content

2015-09-09 Thread signaldeveloper 
Is the user in the RC db still? Are the folders displaying properly in 
Roundxube and they are just empty?



> On Sep 9, 2015, at 11:03 AM, Robert T. Covell  wrote:
> 
> We have an odd situation that I cannot track down regarding all content in a 
> mailbox disappearing (minus sub mailboxes). 
>  
> CentOS release 6.5
> Cyrus IMAP v2.4.17-Invoca-RPM-2.4.17-6.el6
>  
> A while back we setup a Cyrus install for a client to use as a mail 
> repository for customer contact.  They have one account which is shared 
> across all users in the office.  Approximate size of the account is 85GB.
>  
> At times when a user tries to move email into a Customers mailbox the action 
> is denied.  On the client side they see do see email in the mailbox (cached). 
>  Upon further review everything is gone for that mailbox including: 
> cyrus.cache, cyrus.header, cyrus.index.  Backups contain everything including 
> past emails and the core Cyrus files.  If the mailbox contained other 
> mailboxes they are not affected.  Reconstructing it corrects the issue, 
> luckily our backups do not propagate deletes.
>  
> Problem is that we can’t find any record of the mailbox being deleted.  The 
> content just disappears.  We have been running Cyrus for years and have never 
> seen anything like this.  I am leaning towards user error but I can’t 
> identify what that would be.
>  
> Any insight would be appreciated.
>  
> ___
> Robert Covell
> Rolet Hosting & Interactive
> www.rolet.com
>  
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Disappearing Mailbox Content

2015-09-09 Thread signaldeveloper 
And if you connect some type of IMAP client up to the account do the messages 
show? 

Sent from my iPhone

> On Sep 9, 2015, at 1:18 PM, Robert T. Covell  wrote:
> 
> Is the user in the RC db still? Are the folders displaying properly in 
> Roundxube and they are just empty?
> 
> 
> On Sep 9, 2015, at 11:03 AM, Robert T. Covell  wrote:
> We have an odd situation that I cannot track down regarding all content in a 
> mailbox disappearing (minus sub mailboxes).  
>  
> CentOS release 6.5
> Cyrus IMAP v2.4.17-Invoca-RPM-2.4.17-6.el6
>  
> A while back we setup a Cyrus install for a client to use as a mail 
> repository for customer contact.  They have one account which is shared 
> across all users in the office.  Approximate size of the account is 85GB.
>  
> At times when a user tries to move email into a Customers mailbox the action 
> is denied.  On the client side they see do see email in the mailbox (cached). 
>  Upon further review everything is gone for that mailbox including: 
> cyrus.cache, cyrus.header, cyrus.index.  Backups contain everything including 
> past emails and the core Cyrus files.  If the mailbox contained other 
> mailboxes they are not affected.  Reconstructing it corrects the issue, 
> luckily our backups do not propagate deletes.
>  
> Problem is that we can’t find any record of the mailbox being deleted.  The 
> content just disappears.  We have been running Cyrus for years and have never 
> seen anything like this.  I am leaning towards user error but I can’t 
> identify what that would be.
>  
> Any insight would be appreciated.
>  
> ___
> Robert Covell
> Rolet Hosting & Interactive
> www.rolet.com 
> 
> 
> 
> Yes, the folders do still display but are empty.  If the folder had children 
> they are not affected, just the primary contents of the folder in question.

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus tweaks (slow on roundcube)

2015-09-08 Thread signaldeveloper 
Bron,

So the kolab guys got back to me and said this is done purposely to check 
against cache. I am CCing the kolab user list and fwith many active users who 
are watching over this scenario and well as a few friends too. 

Can you think of any other reasons that this auth process would be slow? Again 
it's not horribly slow just it's noticeable for those that use roundcube with 
less of this "check ins" from multiple roundcube plugins during each message 
select. I suspect that the four plugins mentioned in the bug I reported are 
checking in with IMAP with each and EVERY message view (just viewing messages 
without changing folders)

They're wondering why my installation is so slow. Again, don't think it's the 
IO, it's running on a pretty high tech environment. 

Any logs or anything I can provide for further analysis on the Cyrus end of 
things?

Thanks again man!

- Paul



> On Sep 7, 2015, at 12:26 AM, Bron Gondwana  wrote:
> 
> Yeah, so tls to localhost is dumb.  That's security theatre at its silliest.  
> Best to turn that off.
> 
> Here's some possibilities to make it not required:
> 
> imapd.conf:
> allowplaintext: yes
> sasl_mech_list: PLAIN LOGIN
> 
> There used to be a sasl layer thing we did too... "-p 1" in cyrus.conf for 
> the imapd line that listens on localhost will tell sasl that you already have 
> a protection layer.
> 
> Bron.
> 
> 
>> On Mon, Sep 7, 2015, at 12:15, signaldevelo...@gmail.com wrote:
>> Hey Rudy!
>> 
>> As far as entropy: Probably not, it's brand new. One user (me.. Testing) is 
>> playing on it. This is something I've never touched and know very little 
>> about, can you explain? 
>> 
>> And can you explain: Is saslauthd compiled against /dev/urandom?
>> 
>> Thanks again guys..
>> 
>> - Paul
>> 
>> 
>> 
>> 
>> Sent from my iPhone
>> 
>>> On Sep 6, 2015, at 9:50 PM, Rudy Gevaert  wrote:
>>> 
>>> 
>>> Quoting signaldevelo...@gmail.com, Mon, 07 Sep 2015:
>>> 
 Hosts file is fine I checked that, thanks. Kolab uses 389 to  
 authenticate for everything, so Cyrus is using LDAP as you can see  
 above. I think the problem lies in the constant TLS logins into  
 Cyrus for every click:
 
 imap[2281]: login: localhost [::1] john...@domain.com PLAIN+TLS User  
 logged in  
 SESSIONID=
 Sep  5 20:54:51 es1 imap[2281]: USAGE john...@domain.com user:  
 0.009998 sys: 0.006999
 
 
 Again its only one user, on roundcube... I am afraid to put any more  
 users on it. There doesn't seem to be much of performance tweaks  
 with Cyrus around the web either...
>>> 
>>> does your system have enough entropy?
>>> 
>>> Is saslauthd compiled against /dev/urandom?
>>> 
>>> Rudy
>>> 
>>> -- 
>>> -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
>>> Rudy Gevaert e-mail: rudy.geva...@ugent.be
>>> Directie ICT, Afdeling Infrastructuur
>>> Groep Systemen  tel: +32 9 264 4750
>>> Universiteit Gent   fax: +32 9 264 4994
>>> Krijgslaan 281, gebouw S9, 9000 Gent, Belgie   www.UGent.be
>>> -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
>>> 
>>> 
>>> 
>>> Cyrus Home Page: http://www.cyrusimap.org/
>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>> To Unsubscribe:
>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>> 
>> Cyrus Home Page: http://www.cyrusimap.org/
>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>> To Unsubscribe:
>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> 
> 
> -- 
>  Bron Gondwana
>  br...@fastmail.fm
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus tweaks (slow on roundcube)

2015-09-06 Thread signaldeveloper 
Rudy,

Entropy is 158 I just looked. And as far as compiling against urandom, to be 
honest I'm
not sure. 

- Paul




> On Sep 6, 2015, at 9:50 PM, Rudy Gevaert  wrote:
> 
> 
> Quoting signaldevelo...@gmail.com, Mon, 07 Sep 2015:
> 
>> Hosts file is fine I checked that, thanks. Kolab uses 389 to  
>> authenticate for everything, so Cyrus is using LDAP as you can see  
>> above. I think the problem lies in the constant TLS logins into  
>> Cyrus for every click:
>> 
>> imap[2281]: login: localhost [::1] john...@domain.com PLAIN+TLS User  
>> logged in  
>> SESSIONID=
>> Sep  5 20:54:51 es1 imap[2281]: USAGE john...@domain.com user:  
>> 0.009998 sys: 0.006999
>> 
>> 
>> Again its only one user, on roundcube... I am afraid to put any more  
>> users on it. There doesn't seem to be much of performance tweaks  
>> with Cyrus around the web either...
> 
> does your system have enough entropy?
> 
> Is saslauthd compiled against /dev/urandom?
> 
> Rudy
> 
> -- 
>  -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
>  Rudy Gevaert e-mail: rudy.geva...@ugent.be
>  Directie ICT, Afdeling Infrastructuur
>  Groep Systemen  tel: +32 9 264 4750
>  Universiteit Gent   fax: +32 9 264 4994
>  Krijgslaan 281, gebouw S9, 9000 Gent, Belgie   www.UGent.be
>  -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
> 
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus tweaks (slow on roundcube)

2015-09-06 Thread signaldeveloper 
Hey Rudy!

As far as entropy: Probably not, it's brand new. One user (me.. Testing) is 
playing on it. This is something I've never touched and know very little about, 
can you explain? 

And can you explain: Is saslauthd compiled against /dev/urandom?

Thanks again guys..

- Paul




Sent from my iPhone

> On Sep 6, 2015, at 9:50 PM, Rudy Gevaert  wrote:
> 
> 
> Quoting signaldevelo...@gmail.com, Mon, 07 Sep 2015:
> 
>> Hosts file is fine I checked that, thanks. Kolab uses 389 to  
>> authenticate for everything, so Cyrus is using LDAP as you can see  
>> above. I think the problem lies in the constant TLS logins into  
>> Cyrus for every click:
>> 
>> imap[2281]: login: localhost [::1] john...@domain.com PLAIN+TLS User  
>> logged in  
>> SESSIONID=
>> Sep  5 20:54:51 es1 imap[2281]: USAGE john...@domain.com user:  
>> 0.009998 sys: 0.006999
>> 
>> 
>> Again its only one user, on roundcube... I am afraid to put any more  
>> users on it. There doesn't seem to be much of performance tweaks  
>> with Cyrus around the web either...
> 
> does your system have enough entropy?
> 
> Is saslauthd compiled against /dev/urandom?
> 
> Rudy
> 
> -- 
>  -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
>  Rudy Gevaert e-mail: rudy.geva...@ugent.be
>  Directie ICT, Afdeling Infrastructuur
>  Groep Systemen  tel: +32 9 264 4750
>  Universiteit Gent   fax: +32 9 264 4994
>  Krijgslaan 281, gebouw S9, 9000 Gent, Belgie   www.UGent.be
>  -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
> 
> 
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Cyrus tweaks (slow on roundcube)

2015-09-06 Thread signaldeveloper 
Haha hey bron. Sorry, I'm a 24/7 workaholic. :)

I don't think it's IO. It's a VM on a HA cluster. The node its on is 64GB of 
RAM, I have 1 user and assigned everything to it for now. Everything else runs 
beautifully on it the identical nodes. I have similar setups (non kolab) that 
run off of dovecot and runs fantastic. Other file intensive apps also work 
well. 

Some of the cyrus.conf

configdirectory: /var/lib/imap
partition-default: /var/spool/imap
admins: cyrus-admin
sievedir: /var/lib/imap/sieve
sendmail: /usr/sbin/sendmail
sasl_pwcheck_method: auxprop saslauthd
sasl_mech_list: PLAIN LOGIN
allowplaintext: no
tls_server_cert: /etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_server_key: /etc/pki/cyrus-imapd/cyrus-imapd.pem
# uncomment this if you're operating in a DSCP environment (RFC-4594)
# qosmarking: af13
auth_mech: pts
pts_module: ldap
ldap_servers: ldap://localhost:389
ldap_sasl: 0


Hosts file is fine I checked that, thanks. Kolab uses 389 to authenticate for 
everything, so Cyrus is using LDAP as you can see above. I think the problem 
lies in the constant TLS logins into Cyrus for every click:

imap[2281]: login: localhost [::1] john...@domain.com PLAIN+TLS User logged in 
SESSIONID=
Sep  5 20:54:51 es1 imap[2281]: USAGE john...@domain.com user: 0.009998 sys: 
0.006999


Again its only one user, on roundcube... I am afraid to put any more users on 
it. There doesn't seem to be much of performance tweaks with Cyrus around the 
web either...






> On Sep 6, 2015, at 7:57 PM, Bron Gondwana  wrote:
> 
> On a Sunday night?  Not really.
>  
> So Cyrus itself isn't this slow unless you have a horrible IO system under it 
> or so little RAM that you can't fit the entire index into memory and it goes 
> swapping.
>  
> A possible issue would be failed DNS resolution on every connect, so you 
> could check your hosts file to make sure localhost is mentioned (or change 
> the default_host to 127.0.0.1 to avoid the name lookup at all).
>  
> I don't know WTF Kolab are doing with chattrsync.  Cyrus uses fdatasync 
> correctly to avoid losing data already - if it doesn't that's a really 
> serious bug, but I'm pretty sure I've audited everything.
>  
> So basically - check your IO speeds, and check your DNS.  Anything more than 
> that, I'd need to see the system or know more about how Kolab itself is set 
> up.
>  
> Bron.
>  
>> On Mon, Sep 7, 2015, at 02:26, Paul Bronson wrote:
>> Anyone have any ideas on this?
>>  
>> On Sat, Sep 5, 2015 at 10:22 PM, Paul Bronson  
>> wrote:
>> Okay guys, I have put a lot of research into this so bear with me. 
>>  
>> Here's what I am up against. I have been working with RC (roundcube) for a 
>> long time and I know some awesome mySQL tweaks that work wonders, etc. My 
>> tweaks haven't done a single thing for the kolab install. This the first 
>> time I've worked with memcache. So here's my config before I proceed:
>>  
>> // IMAP Server Settings
>> $config['default_host'] = 'localhost';
>> $config['default_port'] = 143;
>> $config['imap_delimiter'] = '/';
>> $config['imap_force_lsub'] = true;
>>  
>> // Caching and storage settings
>> $config['imap_cache'] = 'memcache';
>> $config['imap_cache_ttl'] = '10d';
>> $config['messages_cache'] = 'db';
>> $config['message_cache_ttl'] = '10d';
>> $config['session_storage'] = 'db';
>>  
>> // SMTP Server Settings
>> $config['smtp_server'] = 'tls://localhost';
>> $config['smtp_port'] = 587;
>> $config['smtp_user'] = '%u';
>> $config['smtp_pass'] = '%p';
>> $config['smtp_helo_host'] = $_SERVER["HTTP_HOST"];
>>  
>>  
>> I did notice that when I turn off tls on:
>>  
>> $config['smtp_server'] = 'tls://localhost';
>>  
>> that I can't log into RC, it seems to want to force me to use TLS, which I 
>> read somewhere else. Not sure if this is the culprit here.
>>  
>> CHATTRSYNC is off on cyrus already (by default I guess?)
>>  
>> I've installed imapproxy which seems to be working as far as the logs are 
>> concerned, but I STILL get about 3-4-5 second delay when opening through 
>> email. I have since uninstalled the imapproxy... Memcache is working 
>> properly according to the simply ps aux command or memcache-tool command but 
>> it still seems slow on RC, doesn't seem to have made a difference. I get 
>> these constant logs:
>>  
>> Sep  5 20:54:51 es1 imap[2281]: login: localhost [::1] john...@domain.com 
>> PLAIN+TLS User logged in 
>> SESSIONID=
>> Sep  5 20:54:51 es1 imap[2281]: USAGE john...@domain.com user: 0.009998 sys: 
>> 0.006999
>>  
>> It logs everytime the user makes a click or opens an email. Again, imapproxy 
>> didn't work much.. I tried disabling TLS, tried changing the cache methods.. 
>> Have been doing research pretty much all day trying to tweak this. Some have 
>> said comment out the kolab files plugin, that didn't make any difference at 
>> all. Cyrus just seems to take forever for some