Solved (Re: -current vs. -stable network performance)
In case you are interested, I found why CURRENT performed so badly. It turns out that CURRENT still does not have the fix to M_LEADINGSPACE that permits writing into non-shared mbufs. This caused the header of forwarded packets to be pulled up in a separate buffer, and triggered a known (to me at least!) performance bug in the 21143 which wastes about 5us when a packet is split across two descriptor. With the one-line change shown below, forwarding speed goes up to the same values as STABLE. The change below has been committed to STABLE 7 weeks ago, but did not go into CURRENT because there was some disagreement on the semantics of M_LEADINGSPACE. However I would strongly vote for committing this change to CURRENT as well, given the huge performance implications (even if the 21143 were not buggy, not being able to write into clusters hurts a lot of pieces of the networking stack). cheers luigi Index: mbuf.h === RCS file: /home/xorpc/u2/freebsd/src/sys/sys/mbuf.h,v retrieving revision 1.85 diff -u -r1.85 mbuf.h --- mbuf.h 2001/09/30 01:58:35 1.85 +++ mbuf.h 2001/12/14 09:46:28 @@ -360,7 +360,7 @@ */ #defineM_LEADINGSPACE(m) \ ((m)-m_flags M_EXT ? \ - /* (m)-m_data - (m)-m_ext.ext_buf */ 0 : \ + (M_WRITABLE(m) ? (m)-m_data - (m)-m_ext.ext_buf : 0): \ (m)-m_flags M_PKTHDR ? (m)-m_data - (m)-m_pktdat : \ (m)-m_data - (m)-m_dat) On Wed, Dec 12, 2001 at 10:42:06PM -0800, Luigi Rizzo wrote: Hi, I am testing the forwarding performance of CURRENT vs. STABLE (both more or less up to date, unmodified, with the latest performance patches to the dc driver, which I am using) and I am having some surprises. STABLE can forward approx 125Kpps, whereas CURRENT tops at approx 80Kpps. This is on the same hardware, 750MHz Athlon, fastforwarding enabled, a 4-port 21143 card, one input driven with a stream of up to 148Kpps (64 bytes each). Ability to transmit seems roughly the same (in both cases 138Kpps), and lack of CPU does not seem to be the problem (at least for CURRENT), so I am suspecting some difference in the initialization of PCI parameters, such as burst size etc, but I am unclear on where to look at. Any ideas ? cheers luigi --+- Luigi RIZZO, [EMAIL PROTECTED] . ACIRI/ICSI (on leave from Univ. di Pisa) http://www.iet.unipi.it/~luigi/ . 1947 Center St, Berkeley CA 94704 Phone: (510) 666 2927 --+- To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Solved (Re: -current vs. -stable network performance)
Luigi Rizzo wrote: [..] The change below has been committed to STABLE 7 weeks ago, but did not go into CURRENT because there was some disagreement on the semantics of M_LEADINGSPACE. However I would strongly vote for committing this change to CURRENT as well, given the huge performance implications (even if the 21143 were not buggy, not being able to write into clusters hurts a lot of pieces of the networking stack). Incidently, this is a poster-child example of why fixes are not to go to -stable first. It leads to exactly this sort of lossage. rev 1.44.2.11: ... This does not go in CURRENT as is: as discussed in -net, M_LEADINGSPACE should not check for writability, just report available space, leaving the check to some other piece of code. Unfortunately, some code in the tree depends on M_LEADINGSPACE checking for writability, and so implementing M_LEADINGSPACE in the correct way also requires to fix the incorrect usage. This is what will be done in CURRENT, but for STABLE, this is probably more than we want, and so we are happy (more or less) with this simple fix. ... How about fixing it for real as described in the commit message? Cheers, -Peter -- Peter Wemm - [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] All of this is for nothing if we don't go to the stars - JMS/B5 To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Solved (Re: -current vs. -stable network performance)
Peter Wemm wrote: Luigi Rizzo wrote: [..] The change below has been committed to STABLE 7 weeks ago, but did not go into CURRENT because there was some disagreement on the semantics of M_LEADINGSPACE. However I would strongly vote for committing this change to CURRENT as well, given the huge performance implications (even if the 21143 were not buggy, not being able to write into clusters hurts a lot of pieces of the networking stack). Incidently, this is a poster-child example of why fixes are not to go to -stable first. It leads to exactly this sort of lossage. If we waited for all disagreement about semantics in -current to be resolved, we would all be tripping over our beards before some things would be committed. Is the semantic complain going to be overridden by the performance advantages of not caring about the semantics, only the speed? rev 1.44.2.11: ... This does not go in CURRENT as is: as discussed in -net, M_LEADINGSPACE should not check for writability, just report available space, leaving the check to some other piece of code. Unfortunately, some code in the tree depends on M_LEADINGSPACE checking for writability, and so implementing M_LEADINGSPACE in the correct way also requires to fix the incorrect usage. This is what will be done in CURRENT, but for STABLE, this is probably more than we want, and so we are happy (more or less) with this simple fix. ... How about fixing it for real as described in the commit message? Uh... patches welcome from the people who complained about the writability check? 8^) 8^)... -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Solved (Re: -current vs. -stable network performance)
On Fri, Dec 14, 2001 at 02:34:36AM -0800, Peter Wemm wrote: Luigi Rizzo wrote: [..] The change below has been committed to STABLE 7 weeks ago, but did not go into CURRENT because there was some disagreement on the ... Incidently, this is a poster-child example of why fixes are not to go to -stable first. It leads to exactly this sort of lossage. I guess what you are trying to say is Thank you for taking the time to track down this performance bug and having it fixed in -stable so that we can ship 4.5 with the fix, and now for looking at the same problem in -current even if you don't really use it Please, (re)read the commit log and the discussion on -net, I did not want to step over the toes of the author of the code in -current, but he did not have such strong objections for having -stable fixed. This fix has not been lost _only_ because the code was in stable and it was documented in the commit logs. If I had not committed it there (and I could not do it on -current for the reasons above), I would have just forgotten about it. How about fixing it for real as described in the commit message? The real fix, for me, is the one-line change to M_LEADINGSPACE. The one described in the commit message was just Bosko's point of view, with which I and many others disagree, and which requires an extensive scrutiny of the code and a change of the commonly understood semantics in M_LEADINGSPACE. cheers luigi To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Solved (Re: -current vs. -stable network performance)
On Fri, Dec 14, 2001 at 03:16:57AM -0800, Luigi Rizzo wrote: How about fixing it for real as described in the commit message? The real fix, for me, is the one-line change to M_LEADINGSPACE. The one described in the commit message was just Bosko's point of view, with which I and many others disagree, and which requires an extensive scrutiny of the code and a change of the commonly understood semantics in M_LEADINGSPACE. I think we should make this change for the moment as it doesn't change the current meaning of M_LEADINGSPACE and seems to improve performance a bit. Later, we can decide if changing the meaning of M_LEADINGSPACE is a useful thing to do. David. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
