Hi Yaroslav, I noticed te 104 error only when the server can't handle the large amount of connections correctly.
Did you check your ulimit settings? You can refer to the excellent G-WAN doc explaining how to tweek your seetings at: http://gwan.ch/source/ab.c.txt Let me give httpress a try, I'll be back with more info. Keep up the good work man. Regards, Zabrane On Dec 4, 2011, at 1:25 AM, Yaroslav wrote: > Hi, everybody! > > I tried to use weighttp for benchmarking but encountered some problems when > using large number of concurrent connections in multi-threaded mode. For > example: > > weighttp -c 10000 -n 100000 -t 4 -k http://... > > This almost always produces quite a number of '104 Connection reset by peer' > errors, many connections do not close, which means the test runs long, and > measurements get poor. This is not specific to nxweb, same situation with all > tested servers. I even tried it on different hardware with CentOS. Same > situation. The interesting part is that when used in single thread mode (-t > 1) there are no such errors even with large number of connections. > > I've spent whole day digging the source of weighttp, even made some patches, > but could not figure out the source of the problem. > > Finally I've decided to write my own benchmarking tool, and successfully did > it. With the same result though... As I also used libev. > > Anyway I made some improvements over weighttp, namely: > - load is evenly distributed among threads, in weighttp one thread could > finish much earlier than others > - I forcibly cut timed out connections, not allowing them to spoil the test > > Otherwise it is the same efficient as weighttp. Need to mention that I have > not implemented (yet) chunked encoding, custom headers and IPv6, which > weighttp implements. > > Here is the project page if anyone interested: > https://bitbucket.org/yarosla/httpress/ > > I plan to redo all benchmarks for NXWEB in a few days. With the new tool > results are much better now. > > I wonder if anybody experienced same issues with hanging/reset connections > with libev under high concurrency? > > Regards, > > Yaroslav > > On Fri, Dec 2, 2011 at 3:14 PM, Yaroslav <[email protected]> wrote: > Hi Mickael, > > I mentioned in limitations that it has only been tested on Linux. I can't > test it on OSX unfortunately, and frankly speaking portability was not my > primary goal. At least not at this stage. Hosting providers mostly offer > Linux, never heard about OSX. > > But still I think it should be possible to port it to OSX, it's not Windows > anyway. > > What causing compile errors is the absense of argp and obstack in OSX's > library. argp is not so critical for NXWEB - this is just command line > parsing. While obstack is used extensively by the core. > > Thanks for pointing to benchmarking tool. I will take a look at it. I wasn't > satisfied by ab, but couldn't find anything better. httperf happened to be > too complicated for my brain. > > Yaroslav > > > On Fri, Dec 2, 2011 at 2:56 PM, Zabrane Mickael <[email protected]> wrote: > Hi Yaroslav, > > My first try to compile NxWeb (OSX 10.6.8 + libev4): > > nxweb $ make > mkdir -p bin/Release > mkdir -p obj/Release/nxweb > gcc -c -o obj/Release/nxweb/main.o nxweb/main.c -pthread -Wno-strict-aliasing > -O2 -s > nxweb/main.c:29:18: error: argp.h: No such file or directory > In file included from nxweb/nxweb_internal.h:30, > from nxweb/main.c:35: > nxweb/nxweb.h:30:21: error: obstack.h: No such file or directory > In file included from nxweb/nxweb_internal.h:30, > from nxweb/main.c:35: > nxweb/nxweb.h:121: error: field ‘data’ has incomplete type > nxweb/nxweb.h:122: error: field ‘user_data’ has incomplete type > nxweb/main.c:123: error: array type has incomplete element type > nxweb/main.c:140: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ > before ‘parse_opt’ > nxweb/main.c:165: error: variable ‘argp’ has initializer but incomplete type > nxweb/main.c:165: warning: excess elements in struct initializer > nxweb/main.c:165: warning: (near initialization for ‘argp’) > nxweb/main.c:165: error: ‘parse_opt’ undeclared here (not in a function) > nxweb/main.c:165: warning: excess elements in struct initializer > nxweb/main.c:165: warning: (near initialization for ‘argp’) > nxweb/main.c:165: warning: excess elements in struct initializer > nxweb/main.c:165: warning: (near initialization for ‘argp’) > nxweb/main.c:165: warning: excess elements in struct initializer > nxweb/main.c:165: warning: (near initialization for ‘argp’) > make: *** [obj/Release/nxweb/main.o] Error 1 > > > I think you should update your benchmark to use weighttp: > http://redmine.lighttpd.net/projects/weighttp/wiki > As pointed by G-WAN's author, ab performs badly on multi-cores. > > Regards, > Zabrane > > On Dec 2, 2011, at 9:58 AM, Yaroslav wrote: > >> What a shame. Sorry about that. It is my first public repo at Bitbucket, I >> turned 'private' flag off, but did not check that I have to make wiki and >> issue tracker public as well. >> >> It should be all OK now. >> >> Sorry for the inconvenience. >> >> On Fri, Dec 2, 2011 at 12:13 PM, Chris Brody <[email protected]> wrote: >> Yeah I created an account with Bitbucket and still could not get access to >> the Wiki. >> >> Can you please make this project public if you want to announce it? >> >> >> On Fri, Dec 2, 2011 at 3:04 AM, Rodrigo Campos <[email protected]> wrote: >> On Fri, Dec 02, 2011 at 03:55:27AM +0400, Yaroslav wrote: >> > Hi, >> > >> > I'd like to thank the authors of libev for very good product. Based on it I >> > have developed NXWEB web server, which I'd like to present to community. >> > Here >> > are the highlights: >> > >> > NXWEB project page: https://bitbucket.org/yarosla/nxweb/ >> >> Sounds interesting, but the project page asks for a login. It seems I can not >> see anything without an account :( >> >> >> >> >> >> Thanks, >> Rodrigo >> >> _______________________________________________ >> libev mailing list >> [email protected] >> http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev >> >> >> _______________________________________________ >> libev mailing list >> [email protected] >> http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev >> >> _______________________________________________ >> libev mailing list >> [email protected] >> http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev > > > > > > _______________________________________________ > libev mailing list > [email protected] > http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
_______________________________________________ libev mailing list [email protected] http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
