Re: [SOGo] Slow web performance

[Anders Wallin]

I have a lot of errors in my sogo-log for caching.. But dont know exactly caching for web, imap or what? Anyone have any ideas? onvm00102.on.corp - - [28/Aug/2012:11:30:37 GMT] "POST /SOGo/so/anders.wallin/Mail/0/mailboxes HTTP/1.1" 200 1824/0 3.559 - - 3M 2012-08-28 11:30:37.836 sogod[25838] Note: Using UTF-8 as URL encoding in NGExtensions. 2012-08-28 11:30:37.836 sogod[25843] Note: Using UTF-8 as URL encoding in NGExtensions. Aug 28 11:30:37 sogod [25838]: <0x0x14fa51e8[SOGoCache]> Cache cleanup interval set every 300.000000 seconds Aug 28 11:30:37 sogod [25838]: <0x0x14fa51e8[SOGoCache]> Using host(s) 'localhost' as server(s) 2012-08-28 11:30:37.836 sogod[25894] Note: Using UTF-8 as URL encoding in NGExtensions. Aug 28 11:30:37 sogod [25843]: <0x0x14fa51c8[SOGoCache]> Cache cleanup interval set every 300.000000 seconds Aug 28 11:30:37 sogod [25843]: <0x0x14fa51c8[SOGoCache]> Using host(s) 'localhost' as server(s) Aug 28 11:30:37 sogod [25894]: <0x0x14fa54c8[SOGoCache]> Cache cleanup interval set every 300.000000 seconds Aug 28 11:30:37 sogod [25894]: <0x0x14fa54c8[SOGoCache]> Using host(s) 'localhost' as server(s) 2012-08-28 11:30:37.843 sogod[25877] Note: Using UTF-8 as URL encoding in NGExtensions. Aug 28 11:30:37 sogod [25877]: <0x0x14fa53b8[SOGoCache]> Cache cleanup interval set every 300.000000 seconds Aug 28 11:30:37 sogod [25877]: <0x0x14fa53b8[SOGoCache]> Using host(s) 'localhost' as server(s) Aug 28 11:30:38 sogod [25761]: <0x0x14fb9ca8[SOGoCache]> an error occurred when caching value for key 'session:TuFujl98kXrNgwj9dtN2Yw==': "SYSTEM ERROR" Aug 28 11:30:38 sogod [25778]: <0x0x14fa5128[SOGoCache]> an error occurred when caching value for key 'session:TuFujl98kXrNgwj9dtN2Yw==': "SYSTEM ERROR" Aug 28 11:30:38 sogod [25877]: <0x0x14fa53b8[SOGoCache]> an error occurred when caching value for key 'session:TuFujl98kXrNgwj9dtN2Yw==': "SYSTEM ERROR" Aug 28 11:30:38 sogod [25843]: <0x0x14fa51c8[SOGoCache]> an error occurred when caching value for key 'session:TuFujl98kXrNgwj9dtN2Yw==': "SYSTEM ERROR" Aug 28 11:30:38 sogod [25838]: <0x0x14fa51e8[SOGoCache]> an error occurred when caching value for key 'session:TuFujl98kXrNgwj9dtN2Yw==': "SYSTEM ERROR" Aug 28 11:30:38 sogod [25894]: <0x0x14fa54c8[SOGoCache]> an error occurred when caching value for key 'session:TuFujl98kXrNgwj9dtN2Yw==': "SYSTEM ERROR" Aug 28 11:30:39 sogod [25843]: <0x0x150060f8[LDAPSource]> WARNING: using old bindFields format - please update it Aug 28 11:30:39 sogod [25894]: <0x0x150063f8[LDAPSource]> WARNING: using old bindFields format - please update it Aug 28 11:30:39 sogod [25778]: <0x0x14fa5128[SOGoCache]> an error occurred when caching value for key 'anders.wallin+attributes': "SYSTEM ERROR" Aug 28 11:30:39 sogod [25761]: <0x0x14fb9ca8[SOGoCache]> an error occurred when caching value for key 'anders.wallin+attributes': "SYSTEM ERROR" Aug 28 11:30:39 sogod [25761]: [ERROR] <0x0x15144608[NGLdapAttribute]> cound not convert value of objectGUID to string Aug 28 11:30:39 sogod [25761]: [ERROR] <0x0x150579d8[NGLdapAttribute]> cound not convert value of objectSid to string Aug 28 11:30:39 sogod [25778]: [ERROR] <0x0x150594b8[NGLdapAttribute]> cound not convert value of objectGUID to string Aug 28 11:30:39 sogod [25778]: [ERROR] <0x0x15089698[NGLdapAttribute]> cound not convert value of objectSid to string Aug 28 11:30:39 sogod [25761]: <0x0x14fb9ca8[SOGoCache]> an error occurred when caching value for key 'anders.wallin+attributes': "SYSTEM ERROR" Aug 28 11:30:39 sogod [25761]: <0x0x14fb9ca8[SOGoCache]> an error occurred when caching value for key 'anders.wal...@officenetwork.se+attributes': "SYSTEM ERROR" Aug 28 11:30:39 sogod [25778]: <0x0x14fa5128[SOGoCache]> an error occurred when caching value for key 'anders.wallin+attributes': "SYSTEM ERROR" Aug 28 11:30:39 sogod [25761]: <0x0x14fb9ca8[SOGoCache]> an error occurred when caching value for key 'anders.wal...@officenetwork.se+attributes': "SYSTEM ERROR" Aug 28 11:30:39 sogod [25778]: <0x0x14fa5128[SOGoCache]> an error occurred when caching value for key 'anders.wal...@officenetwork.se+attributes': "SYSTEM ERROR" Aug 28 11:30:39 sogod [25778]: <0x0x14fa5128[SOGoCache]> an error occurred when caching value for key 'anders.wal...@officenetwork.se+attributes': "SYSTEM ERROR" Aug 28 11:30:40 sogod [25894]: <0x0x14fa54c8[SOGoCache]> an error occurred when caching value for key 'anders.wallin+attributes': "SYSTEM ERROR" 2012-08-28 11:30:40.011 sogod[25894] Note(SoObject): SoDebugKeyLookup is enabled! 2012-08-28 11:30:40.011 sogod[25894] Note(SoObject): SoDebugBaseURL is enabled! 2012-08-28 11:30:40.011 sogod[25894] Note(SoObject): relative base URLs are enabled. Aug 28 11:30:40 sogod [25843]: <0x0x14fa51c8[SOGoCache]> an error occurred when caching value for key 'anders.wallin+attributes': "SYSTEM ERROR" 2012-08-28 11:30:40.013 sogod[25843] Note(SoObject): SoDebugKeyLookup is enabled! 2012-08-28 11:30:40.013 sogod[25843] Note(SoObject): SoDebugBaseURL is enabled! 2012-08-28 11:30:40.013 sogod[25843] Note(SoObject): relative base URLs are enabled. Aug 28 11:30:40 sogod [25894]: [ERROR] <0x0x15117cc8[NGLdapAttribute]> cound not convert value of objectGUID to string Aug 28 11:30:40 sogod [25894]: [ERROR] <0x0x15118628[NGLdapAttribute]> cound not convert value of objectSid to string Aug 28 11:30:40 sogod [25894]: <0x0x14fa54c8[SOGoCache]> an error occurred when caching value for key 'anders.wallin+attributes': "SYSTEM ERROR" Aug 28 11:30:40 sogod [25843]: [ERROR] <0x0x151179c8[NGLdapAttribute]> cound not convert value of objectGUID to string Aug 28 11:30:40 sogod [25894]: <0x0x14fa54c8[SOGoCache]> an error occurred when caching value for key 'anders.wal...@officenetwork.se+attributes': "SYSTEM ERROR" Aug 28 11:30:40 sogod [25843]: [ERROR] <0x0x15118328[NGLdapAttribute]> cound not convert value of objectSid to string Aug 28 11:30:40 sogod [25894]: <0x0x14fa54c8[SOGoCache]> an error occurred when caching value for key 'anders.wal...@officenetwork.se+attributes': "SYSTEM ERROR" Aug 28 11:30:40 sogod [25843]: <0x0x14fa51c8[SOGoCache]> an error occurred when caching value for key 'anders.wallin+attributes': "SYSTEM ERROR" Aug 28 11:30:40 sogod [25843]: <0x0x14fa51c8[SOGoCache]> an error occurred when caching value for key 'anders.wal...@officenetwork.se+attributes': "SYSTEM ERROR" Aug 28 11:30:40 sogod [25843]: <0x0x14fa51c8[SOGoCache]> an error occurred when caching value for key 'anders.wal...@officenetwork.se+attributes': "SYSTEM ERROR" Aug 28 11:30:40 sogod [25877]: <0x0x150062d8[LDAPSource]> WARNING: using old bindFields format - please update it Aug 28 11:30:40 sogod [25838]: <0x0x15006108[LDAPSource]> WARNING: using old bindFields format - please update it Aug 28 11:30:40 sogod [25877]: <0x0x14fa53b8[SOGoCache]> an error occurred when caching value for key 'anders.wallin+attributes': "SYSTEM ERROR" 2012-08-28 11:30:40.097 sogod[25877] Note(SoObject): SoDebugKeyLookup is enabled! 2012-08-28 11:30:40.097 sogod[25877] Note(SoObject): SoDebugBaseURL is enabled! 2012-08-28 11:30:40.097 sogod[25877] Note(SoObject): relative base URLs are enabled. Aug 28 11:30:40 sogod [25877]: [ERROR] <0x0x15117b28[NGLdapAttribute]> cound not convert value of objectGUID to string Aug 28 11:30:40 sogod [25877]: [ERROR] <0x0x15118488[NGLdapAttribute]> cound not convert value of objectSid to string Aug 28 11:30:40 sogod [25877]: <0x0x14fa53b8[SOGoCache]> an error occurred when caching value for key 'anders.wallin+attributes': "SYSTEM ERROR" Aug 28 11:30:40 sogod [25877]: <0x0x14fa53b8[SOGoCache]> an error occurred when caching value for key 'anders.wal...@officenetwork.se+attributes': "SYSTEM ERROR" Aug 28 11:30:40 sogod [25877]: <0x0x14fa53b8[SOGoCache]> an error occurred when caching value for key 'anders.wal...@officenetwork.se+attributes': "SYSTEM ERROR" Aug 28 11:30:40 sogod [25838]: <0x0x14fa51e8[SOGoCache]> an error occurred when caching value for key 'anders.wallin+attributes': "SYSTEM ERROR" 2012-08-28 11:30:40.119 sogod[25838] Note(SoObject): SoDebugKeyLookup is enabled! 2012-08-28 11:30:40.119 sogod[25838] Note(SoObject): SoDebugBaseURL is enabled! 2012-08-28 11:30:40.119 sogod[25838] Note(SoObject): relative base URLs are enabled. Aug 28 11:30:40 sogod [25838]: [ERROR] <0x0x151179d8[NGLdapAttribute]> cound not convert value of objectGUID to string Aug 28 11:30:40 sogod [25838]: [ERROR] <0x0x15118338[NGLdapAttribute]> cound not convert value of objectSid to string Aug 28 11:30:40 sogod [25838]: <0x0x14fa51e8[SOGoCache]> an error occurred when caching value for key 'anders.wallin+attributes': "SYSTEM ERROR" Aug 28 11:30:40 sogod [25838]: <0x0x14fa51e8[SOGoCache]> an error occurred when caching value for key 'anders.wal...@officenetwork.se+attributes': "SYSTEM ERROR" Aug 28 11:30:40 sogod [25838]: <0x0x14fa51e8[SOGoCache]> an error occurred when caching value for key 'anders.wal...@officenetwork.se+attributes': "SYSTEM ERROR" Aug 28 11:30:40 sogod [25761]: <0x0x14fb9ca8[SOGoCache]> an error occurred when caching value for key 'session:TuFujl98kXrNgwj9dtN2Yw==': "SYSTEM ERROR" Aug 28 11:30:40 sogod [25778]: <0x0x14fa5128[SOGoCache]> an error occurred when caching value for key 'session:TuFujl98kXrNgwj9dtN2Yw==': "SYSTEM ERROR" Aug 28 11:30:40 sogod [25894]: <0x0x14fa54c8[SOGoCache]> an error occurred when caching value for key 'session:TuFujl98kXrNgwj9dtN2Yw==': "SYSTEM ERROR" Aug 28 11:30:40 sogod [25843]: <0x0x14fa51c8[SOGoCache]> an error occurred when caching value for key 'session:TuFujl98kXrNgwj9dtN2Yw==': "SYSTEM ERROR" Aug 28 11:30:40 sogod [25877]: <0x0x14fa53b8[SOGoCache]> an error occurred when caching value for key 'session:TuFujl98kXrNgwj9dtN2Yw==': "SYSTEM ERROR" Aug 28 11:30:40 sogod [25838]: <0x0x14fa51e8[SOGoCache]> an error occurred when caching value for key 'session:TuFujl98kXrNgwj9dtN2Yw==': "SYSTEM ERROR" Best Regards Anders Wallin Anders Wallin Systemutvecklare anders.wal...@officenetwork.se   Direktnummer: Växelnummer: Fax: 0855519271 0855519200 0855519299   Adress: Hemsida: Storsätragränd 12 SE-12739 SKÄRHOLMEN www.officenetwork.se Philipp v. Strobl-Albeg skrev 2012-08-28 10:13: Hi, the options to improve performance for me was: 1. Pooling IMAP (in sogo conf) 2. Tune Caching of IMAP (e.g. dovecot-lda ...) 3. Tune SQL Parameter (SHMMIN,SHMMAX) 4. reduce LDAP logging The rest i don't test ;-) For VMs i guess - having enough RAM - it's the best solution to put as much as possible from disk to RAM. Hope this helps. If so, please let me know, which one helps. Best Regards Philipp v. Strobl.-Albeg Dipl.-Ing. Zellerstr. 19 70180 Stuttgart Tel. +49 711 1215 8269 Fax. +49 711 6583 089 Mobil +49 151 27039 710 Am 28.08.2012 07:29, schrieb Anders Wallin: Hi Christian, The HDD setup of our VM storage is not optimal, since we use image-file as file system devices instead of LVM. But I don't think this will be the issue as well, since none of the VM servers are very "file intensive". So a quick little test shows these values: hdparm -t /dev/mapper/VolGroup00-LogVol00 Timing buffered disk reads:  956 MB in  3.00 seconds = 318.51 MB/sec seeker /dev/mapper/VolGroup00-LogVol00 Benchmarking /dev/mapper/VolGroup00-LogVol00 [293760MB], wait 30 seconds.............................. Results: 186 seeks/second, 5.37 ms random access time I haven't actually touched any of the caching settings in apache.. Im not an apache wiz ;) But these are the current settings: #<IfModule mod_disk_cache.c> #   CacheEnable disk / #   CacheRoot "/var/cache/mod_proxy" #</IfModule> I removed the comments on this, but i dont really see any difference in the performance. The thing is that it looks like it's "Connecting", and when it finally starts to download it goes fast. Just a little visual thing I noticed.. Like the loading "wheel" in chrome moves slowly counter clockwise. Then when it starts to download content it moves faster clockwise? When it finally starts to download everything goes at a rather more "quick" speed. Best Regards Anders Wallin --   Anders Wallin Systemutvecklare   Direktnummer: Växelnummer: E-mail: 0855519271 0855519200 anders.wal...@officenetwork.se On Monday, August 27, 2012 16:07 CEST, Christian Mack <christian.m...@uni-konstanz.de> wrote: Hello Anders Wallin The below given values are looking good to me. If it would be a VM problem, then you would have it from Thunderbird too. So it's not the culprit. How fast is the storage for the constant content (pictures, css and _javascript_ files) ? These are a lot of small files to get. How are your caching settings in Apache? (I assume you didn't turn off caching in browsers.) Kind regards, Christian Mack On 2012-08-27 09:38, Anders Wallin wrote: > Hello. > > I would say that probably only 1-2 users are connected at any one time > to the webmail. > And the machine is a dedicated webserver. So it only serves sogo, > cyrus-imap, postfix (and apache for webmail). > > It's auth through LDAP (windows 2008), but i dont think auth is the > issue for slow performance. Since it's fast as hell through thunderbird. > > > And all clients, including myself when i use the webmail have this > issue.. And it's regardless of computer performance and/or browser (most > use chrome or firefox). > > > The box isa virtual machine, but it has 4GB of memory dedicated to it. > > > And performance is the same whenever Im on LAN or via Internet. And it's > also on 100/100mbits line serving just 20-25 people. So should be > allright :) > > > And I dont use any proxy servers for this setup. > > > Sogo prefork is set at 70 > Postgresql max con is 250 > > Cyrus: > imap cmd="imapd -U 30 -p 256 -D" listen="imap" prefork=5 > maxchild=-1 > imaps cmd="imapd -s" listen="imaps" prefork=1 > > Apache: > > StartServers 8 > MinSpareServers 5 > MaxSpareServers 20 > ServerLimit 256 > MaxClients 256 > MaxRequestsPerChild 4000 > > > > Best Regards > Anders Wallin > Office Network Delimiter > > Anders Wallin > Systemutvecklare > anders.wal...@officenetwork.se > > > > Direktnummer: > Växelnummer: > Fax: > > > > 0855519271 > 0855519200 > 0855519299 > > > > Adress: > > Hemsida: > > > > Storsätragränd 12 > SE-12739 SKÄRHOLMEN > www.officenetwork.se > > > Christian Mack skrev 2012-08-24 15:28: >> Hello Anders Wallin >> >> >> On 2012-08-24 14:35, Anders Wallin wrote: >>> Im running sogo on a Centos6 box. >>> I have always had very slow sogo web interface for some reason. >>> >>> Loading the e-mail windows/calendar windows takes about 15-20s. >>> Opening an e-mail takes about 5-20 seconds. And the same with pressing >>> reply and so on. >>> >>> In thunderbird with the sogo connectors all is almost instant. >>> And the system only serves around 20-25 users.. >>> >>> And it never seems to be under any load. >>> >>> Does anyone know what might be wrong? >>> >>> >>> I will post my apache sogo config: >> < cut > >> >> You only showed settings which don't say anything about performance. >> >> >> Do you use prefork workers in Apache? >> How many workers do you use in Apache? >> By what count do you increase and decrease those workers? >> How many Memory does Apache have? >> How fast is the storage for the constant content (pictures, css and >> _javascript_ files) ? >> Is there a transparent Proxy in between (eg. squid)? >> Do you see any error messages in the SOGo log? >> >> What about your client machines? >> Do they have enough memory? >> Did the user in question open a lot of other websites (with _javascript_ >> animations) at the same time? >> Which browser do you use? >> >> How about the Network link? >> Had the Thunderbird machine and the Web one the same connection speed? >> >> >> Kind regards, >> Christian Mack >> -- Christian Mack Gruppe Informationsdienste Rechenzentrum Universität Konstanz