Re: several messages
"Mark J. Taylor" wrote: You may want to set the transmit and recieve low-water marks as well. Look at the man page for "setsockopt". Amd tcp(4). -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC http://softweyr.com/ [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: several messages
Mark J. Taylor wrote: You may want to set the transmit and recieve low-water marks as well. Look at the man page for setsockopt. Amd tcp(4). -- Where am I, and what am I doing in this handbasket? Wes Peters Softweyr LLC http://softweyr.com/ w...@softweyr.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: several messages
Thank you for your reply. At what point should I set this socket option? I am assuming right after the socket is allocated?? I will try this and post my results tomorrow night. For those wondering, I cannot just execute Sendmail directly, there are many architectural reasons for this design... Thanks again, Wayne On Tue, 24 Aug 1999, Daniel O'Connor wrote: Date: Tue, 24 Aug 1999 13:41:37 +0930 (CST) From: Daniel O'Connor [EMAIL PROTECTED] To: Wayne Cuddy [EMAIL PROTECTED] Cc: FreeBSD Hackers List [EMAIL PROTECTED] Subject: RE: network performance vs. linux on small transfers On 24-Aug-99 Wayne Cuddy wrote: I REALLY want to use FreeBSD over Linux on this one and need some major help to get the performance out of FreeBSD. Tried setsockopt and TCP_NODELAY? From netinet/tcp.h #define TCP_NODELAY 0x01/* don't delay send to coalesce packets */ --- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum On Mon, 23 Aug 1999, David Greenman wrote: Date: Mon, 23 Aug 1999 21:17:06 -0700 From: David Greenman [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: FreeBSD Hackers List [EMAIL PROTECTED] Subject: Re: network performance vs. linux on small transfers I am involved in a messaging system at work in which we need to send/receive large amounts of small (one line messages) SMTP messages. We are currently using Sendmail 8.9.3 on HPUX. Our application sends messages down a FIFO to a daemon process that is reading from the FIFO. This process then connects to port 25 of the destination system and delivers the mail via SMTP. Currently the destination system is the local system so everything is done on one machine. Using HPUX we typically pass 5 messages a second. This system is a dual 180Mhz K class server so this is surprisingly low performance for this system. When testing on FreeBSD 3.1 we also got 5 messages a second. This system is a 500Mhz P3, this is also unacceptable performance. When we tested with Linux (kernel 2.2.5) we passed 15 messages a second consistently using the exact same P3 described above. Since the HPUX and FreeBSD numbers are so close I am wondering there is some performance tuning that I do not know about. Do you think the number might change if multiple hosts were used? The daemon that reads from the FIFO makes only one connection to the local Sendmail to deliver multiple messages in sequence. I REALLY want to use FreeBSD over Linux on this one and need some major help to get the performance out of FreeBSD. Are you setting the TCP_NODELAY socket option on the SMTP connection? If not, then please do that and let me know if it fixes the problem or not. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: several messages
As an (former) implementer of fast TCP/IP peer-peer communications, I'd have to agree with Dave, and say that it is definitely the TCP_NODELAY option. You'll find that disabling the TCP-ACK delay will greatly increase your performace. The reason that it is so "slow" is because the TCP/IP stack is trying to wait to send a TCP "ACK" piggy-backed with data that you MAY BE SENDING SOON. So it waits for 1/5 of a second for you to send SOMETHING, then shrugs its shoulders when you don't, and sends the TCP ACK without further delay. This is a "standard" that most TCP/IP stacks implement. You'll find the same "problem" under Windows. In fact, when I was doing the peer-peer communications, from a Unix to a Windows '95 machine (and NT), the TCP_NODELAY was not yet implemented in the WinSock library. The workaround was to send garbage back as fast as possible, so the ACK could piggy-back itself on SOMETHING. You may want to set the transmit and recieve low-water marks as well. Look at the man page for "setsockopt". -Mark Taylor NetMAX Developer [EMAIL PROTECTED] http://www.netmax.com/ Wayne Cuddy wrote: Thank you for your reply. At what point should I set this socket option? I am assuming right after the socket is allocated?? I will try this and post my results tomorrow night. For those wondering, I cannot just execute Sendmail directly, there are many architectural reasons for this design... Thanks again, Wayne On Tue, 24 Aug 1999, Daniel O'Connor wrote: Date: Tue, 24 Aug 1999 13:41:37 +0930 (CST) From: Daniel O'Connor [EMAIL PROTECTED] To: Wayne Cuddy [EMAIL PROTECTED] Cc: FreeBSD Hackers List [EMAIL PROTECTED] Subject: RE: network performance vs. linux on small transfers On 24-Aug-99 Wayne Cuddy wrote: I REALLY want to use FreeBSD over Linux on this one and need some major help to get the performance out of FreeBSD. Tried setsockopt and TCP_NODELAY? From netinet/tcp.h #define TCP_NODELAY 0x01/* don't delay send to coalesce packets */ --- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum On Mon, 23 Aug 1999, David Greenman wrote: Date: Mon, 23 Aug 1999 21:17:06 -0700 From: David Greenman [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: FreeBSD Hackers List [EMAIL PROTECTED] Subject: Re: network performance vs. linux on small transfers I am involved in a messaging system at work in which we need to send/receive large amounts of small (one line messages) SMTP messages. We are currently using Sendmail 8.9.3 on HPUX. Our application sends messages down a FIFO to a daemon process that is reading from the FIFO. This process then connects to port 25 of the destination system and delivers the mail via SMTP. Currently the destination system is the local system so everything is done on one machine. Using HPUX we typically pass 5 messages a second. This system is a dual 180Mhz K class server so this is surprisingly low performance for this system. When testing on FreeBSD 3.1 we also got 5 messages a second. This system is a 500Mhz P3, this is also unacceptable performance. When we tested with Linux (kernel 2.2.5) we passed 15 messages a second consistently using the exact same P3 described above. Since the HPUX and FreeBSD numbers are so close I am wondering there is some performance tuning that I do not know about. Do you think the number might change if multiple hosts were used? The daemon that reads from the FIFO makes only one connection to the local Sendmail to deliver multiple messages in sequence. I REALLY want to use FreeBSD over Linux on this one and need some major help to get the performance out of FreeBSD. Are you setting the TCP_NODELAY socket option on the SMTP connection? If not, then please do that and let me know if it fixes the problem or not. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: several messages
Wayne Cuddy scribbled this message on Aug 24: Thank you for your reply. At what point should I set this socket option? I am assuming right after the socket is allocated?? I will try this and post my results tomorrow night. For those wondering, I cannot just execute Sendmail directly, there are many architectural reasons for this design... sounds like you need to fork upon recieption of the message and send the message in a child process... don't forget to reap your children though... you don't want a lot of zombies laying around... if you do this, you really don't need to set the TCP_NODELAY option.. but you might want to anyways... -- John-Mark Gurney Voice: +1 541 684 8449 Cu Networking P.O. Box 5693, 97405 "The soul contains in itself the event that shall presently befall it. The event is only the actualizing of its thought." -- Ralph Waldo Emerson To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: several messages
Thank you for your reply. At what point should I set this socket option? I am assuming right after the socket is allocated?? I will try this and post my results tomorrow night. For those wondering, I cannot just execute Sendmail directly, there are many architectural reasons for this design... Thanks again, Wayne On Tue, 24 Aug 1999, Daniel O'Connor wrote: Date: Tue, 24 Aug 1999 13:41:37 +0930 (CST) From: Daniel O'Connor docon...@gsoft.com.au To: Wayne Cuddy wa...@crb-web.com Cc: FreeBSD Hackers List freebsd-hackers@FreeBSD.ORG Subject: RE: network performance vs. linux on small transfers On 24-Aug-99 Wayne Cuddy wrote: I REALLY want to use FreeBSD over Linux on this one and need some major help to get the performance out of FreeBSD. Tried setsockopt and TCP_NODELAY? From netinet/tcp.h #define TCP_NODELAY 0x01/* don't delay send to coalesce packets */ --- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au The nice thing about standards is that there are so many of them to choose from. -- Andrew Tanenbaum On Mon, 23 Aug 1999, David Greenman wrote: Date: Mon, 23 Aug 1999 21:17:06 -0700 From: David Greenman d...@root.com To: wa...@crb-web.com Cc: FreeBSD Hackers List freebsd-hackers@FreeBSD.ORG Subject: Re: network performance vs. linux on small transfers I am involved in a messaging system at work in which we need to send/receive large amounts of small (one line messages) SMTP messages. We are currently using Sendmail 8.9.3 on HPUX. Our application sends messages down a FIFO to a daemon process that is reading from the FIFO. This process then connects to port 25 of the destination system and delivers the mail via SMTP. Currently the destination system is the local system so everything is done on one machine. Using HPUX we typically pass 5 messages a second. This system is a dual 180Mhz K class server so this is surprisingly low performance for this system. When testing on FreeBSD 3.1 we also got 5 messages a second. This system is a 500Mhz P3, this is also unacceptable performance. When we tested with Linux (kernel 2.2.5) we passed 15 messages a second consistently using the exact same P3 described above. Since the HPUX and FreeBSD numbers are so close I am wondering there is some performance tuning that I do not know about. Do you think the number might change if multiple hosts were used? The daemon that reads from the FIFO makes only one connection to the local Sendmail to deliver multiple messages in sequence. I REALLY want to use FreeBSD over Linux on this one and need some major help to get the performance out of FreeBSD. Are you setting the TCP_NODELAY socket option on the SMTP connection? If not, then please do that and let me know if it fixes the problem or not. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: several messages
As an (former) implementer of fast TCP/IP peer-peer communications, I'd have to agree with Dave, and say that it is definitely the TCP_NODELAY option. You'll find that disabling the TCP-ACK delay will greatly increase your performace. The reason that it is so slow is because the TCP/IP stack is trying to wait to send a TCP ACK piggy-backed with data that you MAY BE SENDING SOON. So it waits for 1/5 of a second for you to send SOMETHING, then shrugs its shoulders when you don't, and sends the TCP ACK without further delay. This is a standard that most TCP/IP stacks implement. You'll find the same problem under Windows. In fact, when I was doing the peer-peer communications, from a Unix to a Windows '95 machine (and NT), the TCP_NODELAY was not yet implemented in the WinSock library. The workaround was to send garbage back as fast as possible, so the ACK could piggy-back itself on SOMETHING. You may want to set the transmit and recieve low-water marks as well. Look at the man page for setsockopt. -Mark Taylor NetMAX Developer mtay...@cybernet.com http://www.netmax.com/ Wayne Cuddy wrote: Thank you for your reply. At what point should I set this socket option? I am assuming right after the socket is allocated?? I will try this and post my results tomorrow night. For those wondering, I cannot just execute Sendmail directly, there are many architectural reasons for this design... Thanks again, Wayne On Tue, 24 Aug 1999, Daniel O'Connor wrote: Date: Tue, 24 Aug 1999 13:41:37 +0930 (CST) From: Daniel O'Connor docon...@gsoft.com.au To: Wayne Cuddy wa...@crb-web.com Cc: FreeBSD Hackers List freebsd-hackers@FreeBSD.ORG Subject: RE: network performance vs. linux on small transfers On 24-Aug-99 Wayne Cuddy wrote: I REALLY want to use FreeBSD over Linux on this one and need some major help to get the performance out of FreeBSD. Tried setsockopt and TCP_NODELAY? From netinet/tcp.h #define TCP_NODELAY 0x01/* don't delay send to coalesce packets */ --- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au The nice thing about standards is that there are so many of them to choose from. -- Andrew Tanenbaum On Mon, 23 Aug 1999, David Greenman wrote: Date: Mon, 23 Aug 1999 21:17:06 -0700 From: David Greenman d...@root.com To: wa...@crb-web.com Cc: FreeBSD Hackers List freebsd-hackers@FreeBSD.ORG Subject: Re: network performance vs. linux on small transfers I am involved in a messaging system at work in which we need to send/receive large amounts of small (one line messages) SMTP messages. We are currently using Sendmail 8.9.3 on HPUX. Our application sends messages down a FIFO to a daemon process that is reading from the FIFO. This process then connects to port 25 of the destination system and delivers the mail via SMTP. Currently the destination system is the local system so everything is done on one machine. Using HPUX we typically pass 5 messages a second. This system is a dual 180Mhz K class server so this is surprisingly low performance for this system. When testing on FreeBSD 3.1 we also got 5 messages a second. This system is a 500Mhz P3, this is also unacceptable performance. When we tested with Linux (kernel 2.2.5) we passed 15 messages a second consistently using the exact same P3 described above. Since the HPUX and FreeBSD numbers are so close I am wondering there is some performance tuning that I do not know about. Do you think the number might change if multiple hosts were used? The daemon that reads from the FIFO makes only one connection to the local Sendmail to deliver multiple messages in sequence. I REALLY want to use FreeBSD over Linux on this one and need some major help to get the performance out of FreeBSD. Are you setting the TCP_NODELAY socket option on the SMTP connection? If not, then please do that and let me know if it fixes the problem or not. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: several messages
Wayne Cuddy scribbled this message on Aug 24: Thank you for your reply. At what point should I set this socket option? I am assuming right after the socket is allocated?? I will try this and post my results tomorrow night. For those wondering, I cannot just execute Sendmail directly, there are many architectural reasons for this design... sounds like you need to fork upon recieption of the message and send the message in a child process... don't forget to reap your children though... you don't want a lot of zombies laying around... if you do this, you really don't need to set the TCP_NODELAY option.. but you might want to anyways... -- John-Mark Gurney Voice: +1 541 684 8449 Cu Networking P.O. Box 5693, 97405 The soul contains in itself the event that shall presently befall it. The event is only the actualizing of its thought. -- Ralph Waldo Emerson To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
