[xmail] Re: Userlist hugging hard drive
Dear Davide, Is this maybe something we can patch ourselves in the code? I would be very much interested in starting to use this since updates to accounts are very regular for us. Thanks! Sincerely, Bart Mortelmans Davide Libenzi schreef: On Mon, 16 Jun 2008, Bart Mortelmans wrote: I did some testing with strace. Running it on CtrlClnt didn't tell me much, so I started XMail in debug through strace. As I understand the result correctly, then XMail forks to handle the CTRL-request. The first files it opens in that fork seem reasonable: ctrl.ipmap.tab, ctrlaccounts.tab, the ctrl-logfile, a temp-file and mailusers.tab. After copying the contents of mailusers.tab into the temp-file (I think), it however start opening each user.tab-file of every single user in the system. I'm not sure why it's doing that, but I'm rather sure that it's this that is causing the delay. It ended up opening almost 16000 files. Any ideas of why it would be doing this? Yep, you're right. There's a common function to load the user info, and this one fetches the user.tab file too. I fixed it in the next version by adding a parameter to tell if the caller wants user.tab info too, when loading it. - Davide - To unsubscribe from this list: send the line unsubscribe xmail in the body of a message to [EMAIL PROTECTED] For general help: send the line help in the body of a message to [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe xmail in the body of a message to [EMAIL PROTECTED] For general help: send the line help in the body of a message to [EMAIL PROTECTED]
[xmail] Re: Userlist hugging hard drive
Davide Libenzi wrote: Yep, you're right. There's a common function to load the user info, and this one fetches the user.tab file too. I fixed it in the next version by adding a parameter to tell if the caller wants user.tab info too, when loading it. Thanks! Looking forward to this new version. - To unsubscribe from this list: send the line unsubscribe xmail in the body of a message to [EMAIL PROTECTED] For general help: send the line help in the body of a message to [EMAIL PROTECTED]
[xmail] Re: Userlist hugging hard drive
I did some testing with strace. Running it on CtrlClnt didn't tell me much, so I started XMail in debug through strace. As I understand the result correctly, then XMail forks to handle the CTRL-request. The first files it opens in that fork seem reasonable: ctrl.ipmap.tab, ctrlaccounts.tab, the ctrl-logfile, a temp-file and mailusers.tab. After copying the contents of mailusers.tab into the temp-file (I think), it however start opening each user.tab-file of every single user in the system. I'm not sure why it's doing that, but I'm rather sure that it's this that is causing the delay. It ended up opening almost 16000 files. Any ideas of why it would be doing this? decker schreef: Hey, ../CtrlClnt -s localhost -u *** -p ** userlist test.domain.tld Just a shot in the dark, is it the same when using 127.0.0.1 rather than localhost ? Can you strace the CtrlClnt process to see where it's spending the most time at ? ~Darren - To unsubscribe from this list: send the line unsubscribe xmail in the body of a message to [EMAIL PROTECTED] For general help: send the line help in the body of a message to [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe xmail in the body of a message to [EMAIL PROTECTED] For general help: send the line help in the body of a message to [EMAIL PROTECTED]
[xmail] Re: Userlist hugging hard drive
On Mon, 16 Jun 2008, Bart Mortelmans wrote: I did some testing with strace. Running it on CtrlClnt didn't tell me much, so I started XMail in debug through strace. As I understand the result correctly, then XMail forks to handle the CTRL-request. The first files it opens in that fork seem reasonable: ctrl.ipmap.tab, ctrlaccounts.tab, the ctrl-logfile, a temp-file and mailusers.tab. After copying the contents of mailusers.tab into the temp-file (I think), it however start opening each user.tab-file of every single user in the system. I'm not sure why it's doing that, but I'm rather sure that it's this that is causing the delay. It ended up opening almost 16000 files. Any ideas of why it would be doing this? Yep, you're right. There's a common function to load the user info, and this one fetches the user.tab file too. I fixed it in the next version by adding a parameter to tell if the caller wants user.tab info too, when loading it. - Davide - To unsubscribe from this list: send the line unsubscribe xmail in the body of a message to [EMAIL PROTECTED] For general help: send the line help in the body of a message to [EMAIL PROTECTED]
[xmail] Re: Userlist hugging hard drive
Dear Davide, Thanks for your input! To check that I'm certainly not doing anything else time-consuming and that it's not a faulty hard drive with a bad cluster in mailusers.tab, I made this little shell script: === date grep \test\.domain\.tld\ ../mailusers.tab date ../CtrlClnt -s localhost -u *** -p ** userlist test.domain.tld date grep \test\.domain\.tld\ ../mailusers.tab date === This is the result: === Sun Jun 15 15:06:15 CEST 2008 test.domain.tld 1 5421574 1 U test.domain.tld 2 5721575 2 U test.domain.tld 3 5621576 3 U test.domain.tld 4 5121577 4 U test.domain.tld 5 5021578 5 U Sun Jun 15 15:06:15 CEST 2008 test.domain.tld 1 1 U test.domain.tld 2 2 U test.domain.tld 3 3 U test.domain.tld 4 4 U test.domain.tld 5 5 U Sun Jun 15 15:06:30 CEST 2008 test.domain.tld 1 5421574 1 U test.domain.tld 2 5721575 2 U test.domain.tld 3 5621576 3 U test.domain.tld 4 5121577 4 U test.domain.tld 5 5021578 5 U Sun Jun 15 15:06:30 CEST 2008 === Getting the userlist via CtrlClnt took about 15 seconds while grep needed less then a second. When getting a full list of all users (not only for one domain), the processing seems to stall now and again. It however stalls in different locations every next time I run it. Any ideas of what might be going wrong? Could this be caused by some faulty setting somewhere? Sincerely, Bart Mortelmans Op 15-jun-08, om 01:13 heeft Davide Libenzi het volgende geschreven: On Fri, 13 Jun 2008, Bart Mortelmans wrote: Hi all, When I run the CTRL-command userlist requesting users for one domain, it seems to take an unreasonable amount of time before it returns (20 to 30 seconds). In between, when I monitor the server with the Linux command top, I notice that wa goes up to over 90%. Normal load average on this machine is around 0, but when running userlist this will go up to over 2. This is the same even if the userlist which is returned, should contain no users or only a small amount. The total size of mailusers.tab is reasonably large (over 1Mb), but I would expect CTRL to at least be able to find the users in the list as quicly as a simple grep for example? For the record: I'm not requesting the full userlist, I use the command like this: userlist[TAB]domainCRLF The complete configuration of XMailserver is a copy/paste from an other server. I'm seeing this both with XMailserver 1.24 and 1.25. The hard drive of this machine seems to be okay. I also seem to notice something similar on an other machine, although not as extreme. I looked into many different possibilities to find the cause of this (including deleting tabindex), but am unable to find it. Or is this normal behaviour? Userlist should be very fast. The time of copying out the mailusers file (that for 1MB size should be like 0.1s) and the time of reading each line. Are you sure you're not issuing other commands, like, for example, the ones to query the mailbox/spool? Disk activity should be almost NIL for userlist. - Davide - To unsubscribe from this list: send the line unsubscribe xmail in the body of a message to [EMAIL PROTECTED] For general help: send the line help in the body of a message to [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe xmail in the body of a message to [EMAIL PROTECTED] For general help: send the line help in the body of a message to [EMAIL PROTECTED]
[xmail] Re: Userlist hugging hard drive
Delays do indeed make me think of some sort of timeout. But using 127.0.0.1 didn't make a difference... Op 15-jun-08, om 19:04 heeft decker het volgende geschreven: Hey, ../CtrlClnt -s localhost -u *** -p ** userlist test.domain.tld Just a shot in the dark, is it the same when using 127.0.0.1 rather than localhost ? Can you strace the CtrlClnt process to see where it's spending the most time at ? ~Darren - To unsubscribe from this list: send the line unsubscribe xmail in the body of a message to [EMAIL PROTECTED] For general help: send the line help in the body of a message to [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe xmail in the body of a message to [EMAIL PROTECTED] For general help: send the line help in the body of a message to [EMAIL PROTECTED]
[xmail] Re: Userlist hugging hard drive
On Fri, 13 Jun 2008, Bart Mortelmans wrote: Hi all, When I run the CTRL-command userlist requesting users for one domain, it seems to take an unreasonable amount of time before it returns (20 to 30 seconds). In between, when I monitor the server with the Linux command top, I notice that wa goes up to over 90%. Normal load average on this machine is around 0, but when running userlist this will go up to over 2. This is the same even if the userlist which is returned, should contain no users or only a small amount. The total size of mailusers.tab is reasonably large (over 1Mb), but I would expect CTRL to at least be able to find the users in the list as quicly as a simple grep for example? For the record: I'm not requesting the full userlist, I use the command like this: userlist[TAB]domainCRLF The complete configuration of XMailserver is a copy/paste from an other server. I'm seeing this both with XMailserver 1.24 and 1.25. The hard drive of this machine seems to be okay. I also seem to notice something similar on an other machine, although not as extreme. I looked into many different possibilities to find the cause of this (including deleting tabindex), but am unable to find it. Or is this normal behaviour? Userlist should be very fast. The time of copying out the mailusers file (that for 1MB size should be like 0.1s) and the time of reading each line. Are you sure you're not issuing other commands, like, for example, the ones to query the mailbox/spool? Disk activity should be almost NIL for userlist. - Davide - To unsubscribe from this list: send the line unsubscribe xmail in the body of a message to [EMAIL PROTECTED] For general help: send the line help in the body of a message to [EMAIL PROTECTED]
