Re: [xmail] problem with mx ip selection on retries
-Message d'origine- De : xmail-boun...@xmailserver.org [mailto:xmail-boun...@xmailserver.org]de la part de Davide Libenzi Envoye : dimanche 7 novembre 2010 21:27 A : XMail Users Mailing List Objet : Re: [xmail] problem with mx ip selection on retries On Wed, 3 Nov 2010, fcxm...@aquinet.net wrote: -Message d'origine- De : xmail-boun...@xmailserver.org [mailto:xmail-boun...@xmailserver.org]de la part de Davide Libenzi Envoye : mercredi 3 novembre 2010 03:46 A : XMail Users Mailing List Objet : Re: [xmail] problem with mx ip selection on retries On Tue, 2 Nov 2010, fcxm...@aquinet.net wrote: Hello Davide I found an problem in xmail when re-trying to connect to mx with multiple ips Here is a sample : Assuming domain XX.com have this dns setup xx.commx 10 mx10.xx.com xx.com mx 20 mx20.xx.com mx10.xx.com A 10.10.10.1 mx10.xx.com A 10.10.10.2 mx10.xx.com A 10.10.10.3 mx20.xx.com A 20.20.20.1 mx20.xx.com A 20.20.20.2 supposing xmail have now to send a mail @xx.com on first try it use mx10.xx.com at 10.10.10.1 : now the tcp connection don't work then os same first try xmail use mx20 at 20.20.20.2 : suppose tcp connection don't work too ! What i see in a trace is that for ALL the others retries for this mail, xmail retries ONLY on SAME ips, 10.10.10.1 and 20.20.20.2 It never retry on others mx's ips !! And because there was no response from these two ips, mail bounced back to sender after all possible retries :-/ (i checked the others ips, they responded correctly :-/ but xmail never tried them ...) I think this is a major bug :( Does not look like. XMail would cache (in the MX cache) the *names*, which are mx10.xx.com and mx20.xx.com (and, for the duration of the TTL). Then the names are resolved to IP addresses using OS specific library calls (getaddrinfo()). So, I don't understand why using nslookup on the xmail server itself resolving mx10.xx.com i get all the mx10 ips values round robined : nslookup mx10.xx.com 10.10.10.2 10.10.10.3 10.10.10.1 another immediate nslookup mx10.xx.com 10.10.10.3 10.10.10.1 10.10.10.2 another 10.10.10.1 10.10.10.2 10.10.10.3 Notice that in the real domain case the dns records default ttl was 2 hours, no ttl specified in the mx records (so default 2 hours) and none in the A records too (so default 2 hours) With xmail retry schedule configured with Qt 300 Qi 1 Qr 10 the retry schedule was : 01 send-time = 0 (00:00:00) next-try = 300(00:05:00) 02 send-time = 300(00:05:00) next-try = 600(00:10:00) 03 send-time = 900(00:15:00) next-try = 1200 (00:20:00) 04 send-time = 2100 (00:35:00) next-try = 2400 (00:40:00) 05 send-time = 4500 (01:15:00) next-try = 4800 (01:20:00) 06 send-time = 9300 (02:35:00) next-try = 9600 (02:40:00) 07 send-time = 18900 (05:15:00) next-try = 19200 (05:20:00) 08 send-time = 38100 (10:35:00) next-try = 38400 (10:40:00) 09 send-time = 76500 (21:15:00) next-try = 76800 (21:20:00) 10 send-time = 153300 (42:35:00) next-try = 153600 (42:40:00) So after retry 05 the next retry is more than 2 hours later, so chance to get same lookups in same order for mx10 and simutanenously the same ip for mx20 seems minimal or very 'bad' coincidence. And notice that exact same think occured for all mails for this domain. One day more than 20 mails was in xmail queue for retries to this domain, and all was blocked by same mx's not responding 'bad' ip ! very bad 'chance' to get same ips :/ The dns server is a bind 9 server on same machine, the xmail use it with smartdnshost entry in server.tab Trying with no smartdnshost didn't change anythink, nor changing local dns address to another 'external' dns server with or without smartdnshost entry in xmail. In all configurations, nslookups allways returned the mx ips in round robin manner even if done at less than the dns entry ttl (here 2 hours) so how to explain xmail 'use' the same IP ? Do you think OS getaddrinfo (here win32 windows 2000 sp4 patched) return the same think in same order (sorting them and returning only the first ip) ? Supposing nslookup and dig don't use getaddrinfo but use direct dns connections (and i think it is), how to test OS getaddrinfo ? I think I found it. Friggen getaddrinfo() sort results returned by the system DNS servers, instead of returning them as in response order. Duh! Need to look into fixing this ... - Davide Good news :) Waiting for an xmail 'workaround' to getaddrinfo sorted returns :) PS : If I understood, xmail actualy directly acquire MX entries doing direct dns connections queries to dns servers (using or not smartdnshost), so why not use same method to acquire A entries (finaly completely bypass any OS dns
Re: [xmail] problem with mx ip selection on retries
On Wed, 3 Nov 2010, fcxm...@aquinet.net wrote: -Message d'origine- De : xmail-boun...@xmailserver.org [mailto:xmail-boun...@xmailserver.org]de la part de Davide Libenzi Envoye : mercredi 3 novembre 2010 03:46 A : XMail Users Mailing List Objet : Re: [xmail] problem with mx ip selection on retries On Tue, 2 Nov 2010, fcxm...@aquinet.net wrote: Hello Davide I found an problem in xmail when re-trying to connect to mx with multiple ips Here is a sample : Assuming domain XX.com have this dns setup xx.com mx 10 mx10.xx.com xx.com mx 20 mx20.xx.com mx10.xx.comA 10.10.10.1 mx10.xx.comA 10.10.10.2 mx10.xx.comA 10.10.10.3 mx20.xx.comA 20.20.20.1 mx20.xx.comA 20.20.20.2 supposing xmail have now to send a mail @xx.com on first try it use mx10.xx.com at 10.10.10.1 : now the tcp connection don't work then os same first try xmail use mx20 at 20.20.20.2 : suppose tcp connection don't work too ! What i see in a trace is that for ALL the others retries for this mail, xmail retries ONLY on SAME ips, 10.10.10.1 and 20.20.20.2 It never retry on others mx's ips !! And because there was no response from these two ips, mail bounced back to sender after all possible retries :-/ (i checked the others ips, they responded correctly :-/ but xmail never tried them ...) I think this is a major bug :( Does not look like. XMail would cache (in the MX cache) the *names*, which are mx10.xx.com and mx20.xx.com (and, for the duration of the TTL). Then the names are resolved to IP addresses using OS specific library calls (getaddrinfo()). So, I don't understand why using nslookup on the xmail server itself resolving mx10.xx.com i get all the mx10 ips values round robined : nslookup mx10.xx.com 10.10.10.2 10.10.10.3 10.10.10.1 another immediate nslookup mx10.xx.com 10.10.10.3 10.10.10.1 10.10.10.2 another 10.10.10.1 10.10.10.2 10.10.10.3 Notice that in the real domain case the dns records default ttl was 2 hours, no ttl specified in the mx records (so default 2 hours) and none in the A records too (so default 2 hours) With xmail retry schedule configured with Qt 300 Qi 1 Qr 10 the retry schedule was : 01 send-time = 0 (00:00:00) next-try = 300(00:05:00) 02 send-time = 300(00:05:00) next-try = 600(00:10:00) 03 send-time = 900(00:15:00) next-try = 1200 (00:20:00) 04 send-time = 2100 (00:35:00) next-try = 2400 (00:40:00) 05 send-time = 4500 (01:15:00) next-try = 4800 (01:20:00) 06 send-time = 9300 (02:35:00) next-try = 9600 (02:40:00) 07 send-time = 18900 (05:15:00) next-try = 19200 (05:20:00) 08 send-time = 38100 (10:35:00) next-try = 38400 (10:40:00) 09 send-time = 76500 (21:15:00) next-try = 76800 (21:20:00) 10 send-time = 153300 (42:35:00) next-try = 153600 (42:40:00) So after retry 05 the next retry is more than 2 hours later, so chance to get same lookups in same order for mx10 and simutanenously the same ip for mx20 seems minimal or very 'bad' coincidence. And notice that exact same think occured for all mails for this domain. One day more than 20 mails was in xmail queue for retries to this domain, and all was blocked by same mx's not responding 'bad' ip ! very bad 'chance' to get same ips :/ The dns server is a bind 9 server on same machine, the xmail use it with smartdnshost entry in server.tab Trying with no smartdnshost didn't change anythink, nor changing local dns address to another 'external' dns server with or without smartdnshost entry in xmail. In all configurations, nslookups allways returned the mx ips in round robin manner even if done at less than the dns entry ttl (here 2 hours) so how to explain xmail 'use' the same IP ? Do you think OS getaddrinfo (here win32 windows 2000 sp4 patched) return the same think in same order (sorting them and returning only the first ip) ? Supposing nslookup and dig don't use getaddrinfo but use direct dns connections (and i think it is), how to test OS getaddrinfo ? I think I found it. Friggen getaddrinfo() sort results returned by the system DNS servers, instead of returning them as in response order. Duh! Need to look into fixing this ... - Davide ___ xmail mailing list xmail@xmailserver.org http://xmailserver.org/mailman/listinfo/xmail
Re: [xmail] problem with mx ip selection on retries
-Message d'origine- De : xmail-boun...@xmailserver.org [mailto:xmail-boun...@xmailserver.org]de la part de Davide Libenzi Envoye : mercredi 3 novembre 2010 03:46 A : XMail Users Mailing List Objet : Re: [xmail] problem with mx ip selection on retries On Tue, 2 Nov 2010, fcxm...@aquinet.net wrote: Hello Davide I found an problem in xmail when re-trying to connect to mx with multiple ips Here is a sample : Assuming domain XX.com have this dns setup xx.com mx 10 mx10.xx.com xx.com mx 20 mx20.xx.com mx10.xx.com A 10.10.10.1 mx10.xx.com A 10.10.10.2 mx10.xx.com A 10.10.10.3 mx20.xx.com A 20.20.20.1 mx20.xx.com A 20.20.20.2 supposing xmail have now to send a mail @xx.com on first try it use mx10.xx.com at 10.10.10.1 : now the tcp connection don't work then os same first try xmail use mx20 at 20.20.20.2 : suppose tcp connection don't work too ! What i see in a trace is that for ALL the others retries for this mail, xmail retries ONLY on SAME ips, 10.10.10.1 and 20.20.20.2 It never retry on others mx's ips !! And because there was no response from these two ips, mail bounced back to sender after all possible retries :-/ (i checked the others ips, they responded correctly :-/ but xmail never tried them ...) I think this is a major bug :( Does not look like. XMail would cache (in the MX cache) the *names*, which are mx10.xx.com and mx20.xx.com (and, for the duration of the TTL). Then the names are resolved to IP addresses using OS specific library calls (getaddrinfo()). So, I don't understand why using nslookup on the xmail server itself resolving mx10.xx.com i get all the mx10 ips values round robined : nslookup mx10.xx.com 10.10.10.2 10.10.10.3 10.10.10.1 another immediate nslookup mx10.xx.com 10.10.10.3 10.10.10.1 10.10.10.2 another 10.10.10.1 10.10.10.2 10.10.10.3 Notice that in the real domain case the dns records default ttl was 2 hours, no ttl specified in the mx records (so default 2 hours) and none in the A records too (so default 2 hours) With xmail retry schedule configured with Qt 300 Qi 1 Qr 10 the retry schedule was : 01 send-time = 0 (00:00:00) next-try = 300(00:05:00) 02 send-time = 300(00:05:00) next-try = 600(00:10:00) 03 send-time = 900(00:15:00) next-try = 1200 (00:20:00) 04 send-time = 2100 (00:35:00) next-try = 2400 (00:40:00) 05 send-time = 4500 (01:15:00) next-try = 4800 (01:20:00) 06 send-time = 9300 (02:35:00) next-try = 9600 (02:40:00) 07 send-time = 18900 (05:15:00) next-try = 19200 (05:20:00) 08 send-time = 38100 (10:35:00) next-try = 38400 (10:40:00) 09 send-time = 76500 (21:15:00) next-try = 76800 (21:20:00) 10 send-time = 153300 (42:35:00) next-try = 153600 (42:40:00) So after retry 05 the next retry is more than 2 hours later, so chance to get same lookups in same order for mx10 and simutanenously the same ip for mx20 seems minimal or very 'bad' coincidence. And notice that exact same think occured for all mails for this domain. One day more than 20 mails was in xmail queue for retries to this domain, and all was blocked by same mx's not responding 'bad' ip ! very bad 'chance' to get same ips :/ The dns server is a bind 9 server on same machine, the xmail use it with smartdnshost entry in server.tab Trying with no smartdnshost didn't change anythink, nor changing local dns address to another 'external' dns server with or without smartdnshost entry in xmail. In all configurations, nslookups allways returned the mx ips in round robin manner even if done at less than the dns entry ttl (here 2 hours) so how to explain xmail 'use' the same IP ? Do you think OS getaddrinfo (here win32 windows 2000 sp4 patched) return the same think in same order (sorting them and returning only the first ip) ? Supposing nslookup and dig don't use getaddrinfo but use direct dns connections (and i think it is), how to test OS getaddrinfo ? Any help to find were is the problem will be appreciated :) Francis ___ xmail mailing list xmail@xmailserver.org http://xmailserver.org/mailman/listinfo/xmail
Re: [xmail] problem with mx ip selection on retries
-Message d'origine- De : xmail-boun...@xmailserver.org [mailto:xmail-boun...@xmailserver.org]de la part de Davide Libenzi Envoye : mercredi 3 novembre 2010 03:57 A : XMail Users Mailing List Objet : Re: [xmail] problem with mx ip selection on retries On Tue, 2 Nov 2010, Sabahattin Gucukoglu wrote: On 2 Nov 2010, at 11:25, fcxm...@aquinet.net fcxm...@aquinet.net wrote: I found an problem in xmail when re-trying to connect to mx with multiple ips Here is a sample : Assuming domain XX.com have this dns setup xx.com mx 10 mx10.xx.com xx.com mx 20 mx20.xx.com mx10.xx.comA 10.10.10.1 mx10.xx.comA 10.10.10.2 mx10.xx.comA 10.10.10.3 mx20.xx.comA 20.20.20.1 mx20.xx.comA 20.20.20.2 supposing xmail have now to send a mail @xx.com on first try it use mx10.xx.com at 10.10.10.1 : now the tcp connection don't work then os same first try xmail use mx20 at 20.20.20.2 : suppose tcp connection don't work too ! What i see in a trace is that for ALL the others retries for this mail, xmail retries ONLY on SAME ips, 10.10.10.1 and 20.20.20.2 It never retry on others mx's ips !! And because there was no response from these two ips, mail bounced back to sender after all possible retries :-/ (i checked the others ips, they responded correctly :-/ but xmail never tried them ...) I think this is a major bug :( It's not violating the standard, but in the interests of robustness, I agree that it is a problem. See: http://tools.ietf.org/html/rfc5321#section-5 Another peculiar XMail behaviour is that even if the hostname in an MX record is unknown, XMail logs an error but then tries again. This only makes sense if the recipient fixes his MX records, which I think is more likely if the mail is permanently failed rather than temporarily. In Postfix, it's user-configurable which method is used. Again, no violation of the spec, but my preference is for a behaviour that is somewhat more robust, especially today with spam-filled queues everywhere. Note that if the remote domain would properly implement RR DNS, the issue would not arise, as the IP list would be permuted at every lookup. Of course, people does RR DNS with TTL of one day, which kinda defeats the purpose. XMail could do its own random-pickup in the supplied list, but this is really not its own task. This was exactly the case here, RR DNS worked (see my previous response) Francis ___ xmail mailing list xmail@xmailserver.org http://xmailserver.org/mailman/listinfo/xmail
Re: [xmail] problem with mx ip selection on retries
On 2 Nov 2010, at 11:25, fcxm...@aquinet.net fcxm...@aquinet.net wrote: I found an problem in xmail when re-trying to connect to mx with multiple ips Here is a sample : Assuming domain XX.com have this dns setup xx.commx 10 mx10.xx.com xx.com mx 20 mx20.xx.com mx10.xx.com A 10.10.10.1 mx10.xx.com A 10.10.10.2 mx10.xx.com A 10.10.10.3 mx20.xx.com A 20.20.20.1 mx20.xx.com A 20.20.20.2 supposing xmail have now to send a mail @xx.com on first try it use mx10.xx.com at 10.10.10.1 : now the tcp connection don't work then os same first try xmail use mx20 at 20.20.20.2 : suppose tcp connection don't work too ! What i see in a trace is that for ALL the others retries for this mail, xmail retries ONLY on SAME ips, 10.10.10.1 and 20.20.20.2 It never retry on others mx's ips !! And because there was no response from these two ips, mail bounced back to sender after all possible retries :-/ (i checked the others ips, they responded correctly :-/ but xmail never tried them ...) I think this is a major bug :( It's not violating the standard, but in the interests of robustness, I agree that it is a problem. See: http://tools.ietf.org/html/rfc5321#section-5 Another peculiar XMail behaviour is that even if the hostname in an MX record is unknown, XMail logs an error but then tries again. This only makes sense if the recipient fixes his MX records, which I think is more likely if the mail is permanently failed rather than temporarily. In Postfix, it's user-configurable which method is used. Again, no violation of the spec, but my preference is for a behaviour that is somewhat more robust, especially today with spam-filled queues everywhere. Cheers, Sabahattin ___ xmail mailing list xmail@xmailserver.org http://xmailserver.org/mailman/listinfo/xmail
Re: [xmail] problem with mx ip selection on retries
On Tue, 2 Nov 2010, fcxm...@aquinet.net wrote: Hello Davide I found an problem in xmail when re-trying to connect to mx with multiple ips Here is a sample : Assuming domain XX.com have this dns setup xx.commx 10 mx10.xx.com xx.com mx 20 mx20.xx.com mx10.xx.com A 10.10.10.1 mx10.xx.com A 10.10.10.2 mx10.xx.com A 10.10.10.3 mx20.xx.com A 20.20.20.1 mx20.xx.com A 20.20.20.2 supposing xmail have now to send a mail @xx.com on first try it use mx10.xx.com at 10.10.10.1 : now the tcp connection don't work then os same first try xmail use mx20 at 20.20.20.2 : suppose tcp connection don't work too ! What i see in a trace is that for ALL the others retries for this mail, xmail retries ONLY on SAME ips, 10.10.10.1 and 20.20.20.2 It never retry on others mx's ips !! And because there was no response from these two ips, mail bounced back to sender after all possible retries :-/ (i checked the others ips, they responded correctly :-/ but xmail never tried them ...) I think this is a major bug :( Does not look like. XMail would cache (in the MX cache) the *names*, which are mx10.xx.com and mx20.xx.com (and, for the duration of the TTL). Then the names are resolved to IP addresses using OS specific library calls (getaddrinfo()). - Davide ___ xmail mailing list xmail@xmailserver.org http://xmailserver.org/mailman/listinfo/xmail
Re: [xmail] problem with mx ip selection on retries
On Tue, 2 Nov 2010, Sabahattin Gucukoglu wrote: On 2 Nov 2010, at 11:25, fcxm...@aquinet.net fcxm...@aquinet.net wrote: I found an problem in xmail when re-trying to connect to mx with multiple ips Here is a sample : Assuming domain XX.com have this dns setup xx.com mx 10 mx10.xx.com xx.com mx 20mx20.xx.com mx10.xx.com A 10.10.10.1 mx10.xx.com A 10.10.10.2 mx10.xx.com A 10.10.10.3 mx20.xx.com A 20.20.20.1 mx20.xx.com A 20.20.20.2 supposing xmail have now to send a mail @xx.com on first try it use mx10.xx.com at 10.10.10.1 : now the tcp connection don't work then os same first try xmail use mx20 at 20.20.20.2 : suppose tcp connection don't work too ! What i see in a trace is that for ALL the others retries for this mail, xmail retries ONLY on SAME ips, 10.10.10.1 and 20.20.20.2 It never retry on others mx's ips !! And because there was no response from these two ips, mail bounced back to sender after all possible retries :-/ (i checked the others ips, they responded correctly :-/ but xmail never tried them ...) I think this is a major bug :( It's not violating the standard, but in the interests of robustness, I agree that it is a problem. See: http://tools.ietf.org/html/rfc5321#section-5 Another peculiar XMail behaviour is that even if the hostname in an MX record is unknown, XMail logs an error but then tries again. This only makes sense if the recipient fixes his MX records, which I think is more likely if the mail is permanently failed rather than temporarily. In Postfix, it's user-configurable which method is used. Again, no violation of the spec, but my preference is for a behaviour that is somewhat more robust, especially today with spam-filled queues everywhere. Note that if the remote domain would properly implement RR DNS, the issue would not arise, as the IP list would be permuted at every lookup. Of course, people does RR DNS with TTL of one day, which kinda defeats the purpose. XMail could do its own random-pickup in the supplied list, but this is really not its own task. - Davide ___ xmail mailing list xmail@xmailserver.org http://xmailserver.org/mailman/listinfo/xmail
