Dave, I'm not suprised by 10:1 on the inet driver. STREAMS over sockets is upside down from usual and is horribly inefficient. On the other hand, benchmark tests between our SCTP STREAMS driver running on LiS and our SCTP NET4 socket driver running inside the kernel indicate that the LiS version (wihtout performance enhancements) of SVR 4.2 STREAMS has superior performance over the Linux NET4 version of BSD Sockets.
But I think everyone knew that 14 years ago. The socket's *interface* is kinda nice, but the guts of a BSD sockets stack sucks in performance in comparison to streams. So the 10:1 in the inet driver is just an impeadance mismatch (a lot of extra copying, queueing, locking, state tracking, error checking, bounds checking, sheduling, wait queues, ....) As for Linux SVR3 style non-STREAMS pipes vs. LiS SVR 4.2 bidirectional STREAMS-based pipies, Linux code uses 1 memory page a a buffer and 1 page of source for the write routine. But it won't pass a file descriptor, and fattach and connld is out of the question. But the throughput should still be better on LiS. Check the data path. I see several things. lis_strputpmsg() should check for flow control before allocating a buffer and copying data. A lot of cpu work time is destroyed whenever EAGAIN is returned. lis_strwrite() checks flow control before allocating the message block. But then it uses PUTNEXT() instead of lis_putnext() as well. I don't know why running queues is correct for lis_strwrite() but not for lis_strputpmsg() or visa versa. With lis_strwrite() the queues will get marked to run. But look at stuff like lis_head_get_fcn() and lis_head_put_fcn() (head.c about line 500 to line 720). In lis_head_get_fcn() an lis_atomic_inc is peformed on a freshly allocated stream head structure that noone else knows about. It is necessary to lock the bus to initialize the value? lis_head_put_fcn() is the deepest and most contorted atomic_dec_and_test that I have seen. IRQ disabling spin locks are taken around conditional atomic_dec(). Also there is a race in the function: when called by two threads, the first thread to exit can return NULL and the second one return non-NULL. So the thing in the data path that slows things down is the paraphenalia of pointer checking and rechecking (walk through each function and count how many time q and hd are checked against NULL or QMAGIC) and the horrendous lis_ locks and abuse of kernel lock functions like the two functions above. Profiling won't help you much when lis_ functions waste time everywhere. You will not see the hot spot because it is hot everywhere. Well, I'm back to why I started LfS in the first place. So I'll shut up now.. --brian On Fri, 09 Apr 2004, Dave Grothe wrote: > > It has always been known (I think) that there is a huge difference in > speed between Linux native pipes and streams based pipes via LiS. > I have seen similar results sending UDP datagrams through the Linux > loopback driver using native sockets vs the inet driver that Brian > publishes. I get about a 10:1 ratio between sockets and streams. > Nothing has ever jumped out at me when I run oprofile during these > tests -- except lock contention. Which is what I am working on now. > -- Dave > At 11:58 AM 4/9/2004, Eugene LiS User wrote: > > To exclude my module from the picture I have decided to compare > data pumping rates for the pipe interface. > I have googled pipespeed2 program, downloaded it and with a minor > changes > got it compiled in 2 versions with and without LiS. > The results are as following: > # ./ps2 2000k 4k > ps2 -x 1 2048000 4096 9.163 Seconds -- 915.493 MB/sec > # ./ps2lis 2000k 4k > ps2lis -x 1 2048000 4096 45.602 Seconds -- 183.953 MB/sec > Clearly there is some overhead in the LiS version of a pipe. > [Hopefuly [for my module] that overhead is not only pipe related] > Attached is a programm. > Compiling with LiS: > # cc -I/usr/src/LiS/include -L /usr/src/LiS/libc -lLiS -o ps2lis > pipespeed2.c > Compiling without LiS: > # cc -o ps2 pipespeed2.c > __________________________________________________________________ > Introducing the New Netscape Internet Service. > Only $9.95 a month -- Sign up today at > [1]http://isp.netscape.com/register > Netscape. Just the Net You Need. > New! Netscape Toolbar for Internet Explorer > Search from anywhere on the Web and block those annoying pop-ups. > Download now at > [2]http://channels.netscape.com/ns/search/install.jsp > --- > Incoming mail is certified Virus Free. > Checked by AVG anti-virus system ([3]http://www.grisoft.com). > Version: 6.0.655 / Virus Database: 420 - Release Date: 4/8/2004 > > References > > 1. http://isp.netscape.com/register > 2. http://channels.netscape.com/ns/search/install.jsp > 3. http://www.grisoft.com/ > > --- > Outgoing mail is certified Virus Free. > Checked by AVG anti-virus system (http://www.grisoft.com). > Version: 6.0.655 / Virus Database: 420 - Release Date: 4/8/2004 -- Brian F. G. Bidulock � The reasonable man adapts himself to the � [EMAIL PROTECTED] � world; the unreasonable one persists in � http://www.openss7.org/ � trying to adapt the world to himself. � � Therefore all progress depends on the � � unreasonable man. -- George Bernard Shaw � _______________________________________________ Linux-streams mailing list [EMAIL PROTECTED] http://gsyc.escet.urjc.es/mailman/listinfo/linux-streams
