Re: cyrus-imapd with lmtpd + postfix slow delivery with group email ids

2015-05-18 Thread Stephen Ulmer 
I mostly waned top make sure that your delivery concurrency wasn’t limited by 
the number of processes that could access a named pipe. That doesn’t seem to be 
your problem, though.

-- 
Stephen



> On May 18, 2015, at 4:21 AM, jayesh shinde  
> wrote:
> 
> Hi Stephen , 
> 
> I am handling via TCP  i.e from smtp server to mailbox server. 
> 
> on smtp server postfix's main.cf  transport_maps is maintain , which deliver 
> the emails to lmtp. 
> 
> Example like below. 
> /@mydomain.com/lmtp:[192.168.1.1]
> 
> Request you to please guide here or any fine tuning etc 
> 
> Regards
> Jayesh Shinde
> 
> 
> On 05/15/2015 10:11 PM, Stephen Ulmer wrote:
>> Are you handing off to LMTP via TCP or a pipe?
>> 
>> -- 
>> Stephen
>> 
>> 
>> 
>>> On May 15, 2015, at 2:18 AM, jayesh shinde >> > wrote:
>>> 
>>> Thanks Dan for your feedback. 
>>> I verified the suggested points , its working as require.  Please find 
>>> below inline reply for each. 
>>> 
>>> I found that  when big group email is getting deliver ( i.e more than 500 - 
>>> 4K members ) at that time other emails are not getting deliver 
>>> simultaneously to user's mailbox. 
>>> I verified this from live maillog  i.e  by doing grep key words like "to 
>>> mailbox:" /  "Delivered:"  etc ...
>>> 
>>> My mail queue issue is getting critical in peak hours !!  Kindly suggest. 
>>> 
>>> On 05/14/2015 08:22 PM, Dan White wrote:
 On 05/14/15 05:42 +, jayesh.shinde wrote: 
> Please suggest , for faster delivery  what could be the best config  in 
> postfix + cyrus-imapd . 
> 
> I am using the cyrus-imapd-2.4.17-6.el6.x86_64 for mailbox server and for 
> smtp server postfix-2.10.0-1.el6.x86_64 Both server have SAS hdd , 16 
> core 
> , 16 GB RAM .  There is no i/o issues 
 
> Mailbox server have  21K email ids.  Cyrus running with  tcp socket on 
> port 24 i.e lmtpd .  cyrus singleinstance is maintained 
 
> For group id mapping is under virtual_alias_maps of smtp's main.cf  i.e 
> no 
> /etc/aliases. From smtp server emails are getting deliver by 
> transport_maps to mailbox server . 
 
> My problem observe :-- 
> -- 
> 1) When HOD send email to 3-5 big group  email ids then , then its 
> postfix 
> delivery get slow. What I observe is postfix getting delivered the 
> traffic 
> per email and not parallel.   Each group contain 3k or 7k email ids. 
 
 Verify that single instance store is working. If not, you may have a 
 Postfix configuration issue. Find one of these mass emails in a mailbox 
 and 
 stat it to find out: 
 
 stat -c %h  
 
 A returned value of 1 means single instance store isn't working. 
 
 If that's not the issue, you can determine if this is a postfix issue or a 
 cyrus issue by sending a mass email directly to lmtpd, with lmtptest. 
>>> Cyrus Singleinstance is working , I have verified like below. In imapd.conf 
>>> man page its define as default 1 ( i.e default enable ) 
>>> 
>>> [root@Mailbox ~]# ls -lirth /var/spool/imap/domain/v/vdomain/m/user/muser1/ 
>>> | tail -n 20  
>>> 
>>> 163326970 -rw---  198 cyrus mail 413K May 15 10:07 64828.
>>> 163396232 -rw---  145 cyrus mail  92K May 15 10:13 64829.
>>> 163379367 -rw---  151 cyrus mail  96K May 15 10:29 64830.
>>> 163323993 -rw---  496 cyrus mail  13K May 15 10:30 64831.
>>> 163398621 -rw---  516 cyrus mail 850K May 15 10:42 64832.
>>> 163398568 -rw---  523 cyrus mail 455K May 15 10:43 64833.
>>> 163398933 -rw---  617 cyrus mail 3.7M May 15 10:47 64834.
>>> 163398962 -rw---  522 cyrus mail 1.3M May 15 10:48 64835.
>>> 163398990 -rw---  617 cyrus mail  58K May 15 10:49 64836.
>>> 163396678 -rw---  465 cyrus mail 207K May 15 10:56 64837.
>>> 163399529 -rw---  576 cyrus mail 332K May 15 10:57 64838.
>>> 163399554 -rw---2 cyrus mail  54K May 15 10:57 64839.
>>> 163346523 -rw---   36 cyrus mail 767K May 15 11:01 64840.
>>> 
>>> [root@Mailbox muser1]# stat -c %h 64829.
>>> 145
>>> [root@Mailbox muser1]# stat -c %h 64828.
>>> 198
>>> 
 
> In peak hours the queue on postfix get high and other normal emails also 
> getting stuck in queue. Once the group email get clear after that other 
> emails also get clear. 
 
> in mailbox cyrus.conf :-- lmtp  cmd="lmtpd -a" listen="lmtp" prefork=0 
> in main.cf of smtp server :-- 
> lmtp_destination_concurrency_limit = 100 
> lmtp_destination_recipient_limit = 0 
 
 How many lmtp processes do you see spawned in this scenario? 
 
>>> [root@Mailbox ~]# ps auxwww| grep lmtp 
>>> 
>>> cyrus 3224  0.0  0.0  73376  4648 ?S10:55   0:00 lmtpd
>>> cyrus 4043  0.0  0.0  73376  4156 ?S10:59   0:00 lmtpd
>>> postfix   6972  0.0  0.0  75708  4336 ?S11:10   0:00 lmtp -t 
>>> unix -u
>>> postfix   6974  0.0  0.0  75708  4

Re: cyrus-imapd with lmtpd + postfix slow delivery with group email ids

2015-05-18 Thread jayesh shinde 

  
  
Hi Stephen , 
  
  I am handling via TCP  i.e from smtp server to mailbox server.
  
  
  on smtp server postfix's main.cf  transport_maps is maintain ,
  which deliver the emails to lmtp. 
  
  Example like below. 
  /@mydomain.com/    lmtp:[192.168.1.1]
  
  Request you to please guide here or any fine tuning etc 
  
  Regards
  Jayesh Shinde

  
  On 05/15/2015 10:11 PM, Stephen Ulmer wrote:


  
  Are you handing off to LMTP via TCP or a pipe?
  

  
-- 
Stephen


  



  
On May 15, 2015, at 2:18 AM, jayesh shinde
  
  wrote:


  
  
Thanks Dan for your
feedback. 
I verified the suggested points , its working as
require.  Please find below inline reply for each. 

I found that  when big group email is getting
deliver ( i.e more than 500 - 4K members ) at that
time other emails are not getting deliver
simultaneously to user's mailbox. 
I verified this from live maillog  i.e  by doing
grep key words like "to mailbox:" /  "Delivered:" 
etc ...

My mail queue issue is getting critical in peak
hours !!  Kindly suggest. 

On 05/14/2015 08:22 PM, Dan White wrote:
  
On 05/14/15 05:42 +,
jayesh.shinde wrote: 
  
  Please
  suggest , for faster delivery  what could be the
  best config  in 
  postfix + cyrus-imapd . 
 
  I am using the cyrus-imapd-2.4.17-6.el6.x86_64 for
  mailbox server and for 
  smtp server postfix-2.10.0-1.el6.x86_64 Both
  server have SAS hdd , 16 core 
  , 16 GB RAM .  There is no i/o issues 

  
  
  Mailbox
  server have  21K email ids.  Cyrus running with 
  tcp socket on 
  port 24 i.e lmtpd .  cyrus singleinstance is
  maintained 

  
  
  For group id
  mapping is under virtual_alias_maps of smtp's
  main.cf  i.e no 
  /etc/aliases. From smtp server emails are getting
  deliver by 
  transport_maps to mailbox server . 

  
  
  My problem
  observe :-- 
  -- 
  1) When HOD send email to 3-5 big group  email ids
  then , then its postfix 
  delivery get slow. What I observe is postfix
  getting delivered the traffic 
  per email and not parallel.   Each group contain
  3k or 7k email ids. 

  
Verify that single instance store is working. If
not, you may have a 
Postfix configuration issue. Find one of these mass
emails in a mailbox and 
stat it to find out: 
   
stat -c %h  
   
A returned value of 1 means single instance store
isn't working. 
   
If that's not the issue, you can determine if this
is a postfix issue or a 
cyrus issue by sending a mass email directly to
lmtpd, with lmtptest. 
  
Cyrus
  Singleinstance is working , I have verified like
  below. In imapd.conf man page its define as default 1
  ( i.e default enable ) 
  
  [root@Mailbox ~]# ls -lirth
  /var/spool/imap/domain/v/vdomain/m/user/muser1/ | tail
  -n 20  
  
  163326970 -rw---  198 cyrus mail
  413K May 15 10:07 64828.
  163396232 -rw---  145 cyrus mail 
  92K May 15 10:13 64829.
  163379367 -rw---  151 cyrus mail 

Re: cyrus-imapd with lmtpd + postfix slow delivery with group email ids

2015-05-14 Thread jayesh shinde 

  
  
Thanks Dan for your feedback. 
I verified the suggested points , its working as require. 
Please find below inline reply for each. 

I found that  when big group email is getting deliver ( i.e more
than 500 - 4K members ) at that time other emails are not
getting deliver simultaneously to user's mailbox. 
I verified this from live maillog  i.e  by doing grep key words
like "to mailbox:" /  "Delivered:"  etc ...

My mail queue issue is getting critical in peak hours !!  Kindly
suggest. 

On 05/14/2015 08:22 PM, Dan White wrote:
  
On 05/14/15 05:42 +,
jayesh.shinde wrote:

  
  Please
  suggest , for faster delivery  what could be the best config 
  in
  
  postfix + cyrus-imapd .
  


  I am using the cyrus-imapd-2.4.17-6.el6.x86_64 for mailbox
  server and for
  
  smtp server postfix-2.10.0-1.el6.x86_64 Both server have SAS
  hdd , 16 core
  
  , 16 GB RAM .  There is no i/o issues
  

  
  
  Mailbox
  server have  21K email ids.  Cyrus running with  tcp socket on
  
  port 24 i.e lmtpd .  cyrus singleinstance is maintained
  

  
  
  For
  group id mapping is under virtual_alias_maps of smtp's
  main.cf  i.e no
  
  /etc/aliases. From smtp server emails are getting deliver by
  
  transport_maps to mailbox server .
  

  
  
  My
  problem observe :--
  
  --
  
  1) When HOD send email to 3-5 big group  email ids then , then
  its postfix
  
  delivery get slow. What I observe is postfix getting delivered
  the traffic
  
  per email and not parallel.   Each group contain 3k or 7k
  email ids.
  

  
Verify that single instance store is working. If not, you may
have a

Postfix configuration issue. Find one of these mass emails in a
mailbox and

stat it to find out:

  
  
stat -c %h 

  
  
A returned value of 1 means single instance store isn't working.

  
  
If that's not the issue, you can determine if this is a postfix
issue or a

cyrus issue by sending a mass email directly to lmtpd, with
lmtptest.

  
Cyrus Singleinstance is
  working , I have verified like below. In imapd.conf man page its
  define as default 1 ( i.e default enable ) 
  
  [root@Mailbox ~]# ls -lirth
  /var/spool/imap/domain/v/vdomain/m/user/muser1/ | tail -n 20  
  
  163326970 -rw---  198 cyrus mail 413K May 15 10:07
  64828.
  163396232 -rw---  145 cyrus mail  92K May 15 10:13
  64829.
  163379367 -rw---  151 cyrus mail  96K May 15 10:29
  64830.
  163323993 -rw---  496 cyrus mail  13K May 15 10:30
  64831.
  163398621 -rw---  516 cyrus mail 850K May 15 10:42
  64832.
  163398568 -rw---  523 cyrus mail 455K May 15 10:43
  64833.
  163398933 -rw---  617 cyrus mail 3.7M May 15 10:47
  64834.
  163398962 -rw---  522 cyrus mail 1.3M May 15 10:48
  64835.
  163398990 -rw---  617 cyrus mail  58K May 15 10:49
  64836.
  163396678 -rw---  465 cyrus mail 207K May 15 10:56
  64837.
  163399529 -rw---  576 cyrus mail 332K May 15 10:57
  64838.
  163399554 -rw---    2 cyrus mail  54K May 15 10:57
  64839.
  163346523 -rw---   36 cyrus mail 767K May 15 11:01
  64840.
  
  [root@Mailbox muser1]# stat -c %h 64829.
  145
  [root@Mailbox muser1]#
  stat -c %h 64828.
  198
  


  
  
  In
  peak hours the queue on postfix get high and other normal
  emails also
  
  getting stuck in queue. Once the group email get clear after
  that other
  
  emails also get clear.
  

  
  
  in
  mailbox cyrus.conf :-- lmtp  cmd="lmtpd -a" listen="lmtp"
  prefork=0
  
  in main.cf of smtp server :--
  
  lmtp_destination_concurrency_limit = 100
  
  lmtp_destination_recipient_limit = 0
  

  
How many lmtp processes do you see spawned in this scenario?

  
  
  
[root@Mailbox ~]# ps
  auxwww| grep lmtp 
  
  cyrus 3224  0.0  0.0  73376  4648 ?    S    10:55   0:00
  lmtpd
  cyrus 4043  0.0  0.0  73376  4156 ?    S    10:59   0:00
  lmtpd
  postfix   6972  0.0  0.0  75708

Re: cyrus-imapd with lmtpd + postfix slow delivery with group email ids

2015-05-14 Thread Dan White 
On 05/14/15 05:42 +, jayesh.shinde wrote:
>Please suggest , for faster delivery  what could be the best config  in
>postfix + cyrus-imapd .
>
>I am using the cyrus-imapd-2.4.17-6.el6.x86_64 for mailbox server and for
>smtp server postfix-2.10.0-1.el6.x86_64 Both server have SAS hdd , 16 core
>, 16 GB RAM .  There is no i/o issues

>Mailbox server have  21K email ids.  Cyrus running with  tcp socket on
>port 24 i.e lmtpd .  cyrus singleinstance is maintained

>For group id mapping is under virtual_alias_maps of smtp's main.cf  i.e no
>/etc/aliases. From smtp server emails are getting deliver by
>transport_maps to mailbox server .

>My problem observe :--
>--
>1) When HOD send email to 3-5 big group  email ids then , then its postfix
>delivery get slow. What I observe is postfix getting delivered the traffic
>per email and not parallel.   Each group contain 3k or 7k email ids.

Verify that single instance store is working. If not, you may have a
Postfix configuration issue. Find one of these mass emails in a mailbox and
stat it to find out:

stat -c %h 

A returned value of 1 means single instance store isn't working.

If that's not the issue, you can determine if this is a postfix issue or a
cyrus issue by sending a mass email directly to lmtpd, with lmtptest.

>In peak hours the queue on postfix get high and other normal emails also
>getting stuck in queue. Once the group email get clear after that other
>emails also get clear.

>in mailbox cyrus.conf :-- lmtp  cmd="lmtpd -a" listen="lmtp" prefork=0
>in main.cf of smtp server :--
>lmtp_destination_concurrency_limit = 100
>lmtp_destination_recipient_limit = 0

How many lmtp processes do you see spawned in this scenario?

-- 
Dan White

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

cyrus-imapd with lmtpd + postfix slow delivery with group email ids

2015-05-13 Thread jayesh.shinde 
Hi , 
Please suggest , for faster delivery  what could be the best config  in postfix 
+ cyrus-imapd . 
I am using the cyrus-imapd-2.4.17-6.el6.x86_64 for mailbox server and for smtp 
server postfix-2.10.0-1.el6.x86_64
 Both server have SAS hdd , 16 core , 16 GB RAM .  There is no i/o issues
Mailbox server have  21K email ids.  Cyrus running with  tcp socket on port 24 
i.e lmtpd .  cyrus singleinstance is maintained
For group id mapping is under virtual_alias_maps of smtp's main.cf  i.e no 
/etc/aliases. From smtp server emails are getting deliver by transport_maps to 
mailbox server .
My problem observe :-- 
--
1) When HOD send email to 3-5 big group  email ids then , then its postfix 
delivery get slow. What I observe is postfix getting delivered the traffic per 
email and not parallel.   Each group contain 3k or 7k email ids.   
In peak hours the queue on postfix get high and other normal emails also 
getting stuck in queue. Once the group email get clear after that other emails 
also get clear. 
With the help previous post and google I have made few changes as follows , I 
current config as follows 
in mailbox cyrus.conf :-- lmtp  cmd="lmtpd -a" listen="lmtp" prefork=0
in main.cf of smtp server :--
lmtp_destination_concurrency_limit = 100
lmtp_destination_recipient_limit = 0
Kindly suggest whether its correct or any other changes require. 
Regards
Jayesh Shinde

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus