Quoting r. zhu shi song <[EMAIL PROTECTED]>: > Subject: why sdp connections cost so much memory > > (1) ibv_devinfo > HCA: MHES18-XTC > FW: 1.1.0 > OFED: OFED-1.1-rc1 > (2) Test Bed > On Client: > ib0: 193.12.10.24 > test command: > LD_PRELOAD=/usr/local/ofed/lib64/libsdp.so > SIMPLE_LIBSDP=1 ab -c m -n m -X 193.12.10.14:3129 > http://www.sse.com.cn/sseportal/ps/zhs/home.shtml > The web page is about 68K. > On Server: > ib0: 193.12.10.14 > squid.sdp -d 10 -f squid2.conf (I have changed > squid-cache to support listening on SDP port 3129) > > The test result is : > Concurrent Conns(=m) Free Memory Requests > completed > 0 926980 0 > 100 712508 100 > 200 497372 200 > 300 282636 256 > 400 52868 256 > 500 kernel crashed because of > "out of memory" > > >From above, every about 100 concurrent SDP connections > will cost 210M memory. It's too vast for large scale > applications. TCP costs very lower memory than SDP. > The max concurrent connections completed successfully > is 256. it is some bad limit. Who knows how and when > will solve the problem? > I'll test the performance of sdp connection and > compare it with TCP further. > tks > zhu
Most memory in SDP goes into pre-posted receive buffers. Currently SDP pre-posts a fixed 64 32K buffers per connection, that is 2M per connection. To verify that's the issue, try opening drivers/infiniband/ulp/sdp/sdp.h and changing SDP_RX_SIZE from 0x40 to a smaller value. If this helps, as a quick work-around I can make this value globally configurable. TCP on the other hand scales down more gracefully, and so should SDP longer-term. > --- [EMAIL PROTECTED] wrote: > > > Send openib-general mailing list submissions to > > [email protected] > > > > To subscribe or unsubscribe via the World Wide Web, > > visit > > http://openib.org/mailman/listinfo/openib-general > > or, via email, send a message with subject or body > > 'help' to > > [EMAIL PROTECTED] > > > > You can reach the person managing the list at > > [EMAIL PROTECTED] > > > > When replying, please edit your Subject line so it > > is more specific > > than "Re: Contents of openib-general digest..." Is this relevant somehow? -- MST _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
