My ulimit I think is high enough. It was 65536, now I checked with 131072. Same results. And with all three servers: nxweb, g-wan, nginx.
But after applying magic sysctl script error rate actually reduced. It looks frightening though. On Sun, Dec 4, 2011 at 5:26 AM, Zabrane Mickael <[email protected]> wrote: > Hi Yaroslav, > > I noticed te 104 error only when the server can't handle the large amount > of connections correctly. > > Did you check your *ulimit* settings? > You can refer to the excellent G-WAN doc explaining how to tweek your > seetings at: > http://gwan.ch/source/ab.c.txt > > Let me give *httpress* a try, I'll be back with more info. > > Keep up the good work man. > > Regards, > Zabrane > > On Dec 4, 2011, at 1:25 AM, Yaroslav wrote: > > Hi, everybody! > > I tried to use weighttp for benchmarking but encountered some problems > when using large number of concurrent connections in multi-threaded mode. > For example: > > weighttp -c 10000 -n 100000 -t 4 -k http://... > > This almost always produces quite a number of '104 Connection reset by > peer' errors, many connections do not close, which means the test runs > long, and measurements get poor. This is not specific to nxweb, same > situation with all tested servers. I even tried it on different hardware > with CentOS. Same situation. The interesting part is that when used in > single thread mode (-t 1) there are no such errors even with large number > of connections. > > I've spent whole day digging the source of weighttp, even made some > patches, but could not figure out the source of the problem. > > Finally I've decided to write my own benchmarking tool, and successfully > did it. With the same result though... As I also used libev. > > Anyway I made some improvements over weighttp, namely: > - load is evenly distributed among threads, in weighttp one thread could > finish much earlier than others > - I forcibly cut timed out connections, not allowing them to spoil the test > > Otherwise it is the same efficient as weighttp. Need to mention that I > have not implemented (yet) chunked encoding, custom headers and IPv6, which > weighttp implements. > > Here is the project page if anyone interested: > https://bitbucket.org/yarosla/httpress/ > > I plan to redo all benchmarks for NXWEB in a few days. With the new tool > results are much better now. > > I wonder if anybody experienced same issues with hanging/reset connections > with libev under high concurrency? > > Regards, > > Yaroslav > > On Fri, Dec 2, 2011 at 3:14 PM, Yaroslav <[email protected]> wrote: > >> Hi Mickael, >> >> I mentioned in limitations that it has only been tested on Linux. I can't >> test it on OSX unfortunately, and frankly speaking portability was not my >> primary goal. At least not at this stage. Hosting providers mostly offer >> Linux, never heard about OSX. >> >> But still I think it should be possible to port it to OSX, it's not >> Windows anyway. >> >> What causing compile errors is the absense of argp and obstack in OSX's >> library. argp is not so critical for NXWEB - this is just command line >> parsing. While obstack is used extensively by the core. >> >> Thanks for pointing to benchmarking tool. I will take a look at it. I >> wasn't satisfied by ab, but couldn't find anything better. httperf happened >> to be too complicated for my brain. >> >> Yaroslav >> >> >> On Fri, Dec 2, 2011 at 2:56 PM, Zabrane Mickael <[email protected]>wrote: >> >>> Hi Yaroslav, >>> >>> My first try to compile NxWeb (OSX 10.6.8 + libev4): >>> >>> *nxweb* $ make >>> mkdir -p bin/Release >>> mkdir -p obj/Release/nxweb >>> gcc -c -o obj/Release/nxweb/main.o nxweb/main.c -pthread >>> -Wno-strict-aliasing -O2 -s >>> nxweb/main.c:29:18: error: argp.h: No such file or directory >>> In file included from nxweb/nxweb_internal.h:30, >>> from nxweb/main.c:35: >>> nxweb/nxweb.h:30:21: error: obstack.h: No such file or directory >>> In file included from nxweb/nxweb_internal.h:30, >>> from nxweb/main.c:35: >>> nxweb/nxweb.h:121: error: field ‘data’ has incomplete type >>> nxweb/nxweb.h:122: error: field ‘user_data’ has incomplete type >>> nxweb/main.c:123: error: array type has incomplete element type >>> nxweb/main.c:140: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or >>> ‘__attribute__’ before ‘parse_opt’ >>> nxweb/main.c:165: error: variable ‘argp’ has initializer but incomplete >>> type >>> nxweb/main.c:165: warning: excess elements in struct initializer >>> nxweb/main.c:165: warning: (near initialization for ‘argp’) >>> nxweb/main.c:165: error: ‘parse_opt’ undeclared here (not in a function) >>> nxweb/main.c:165: warning: excess elements in struct initializer >>> nxweb/main.c:165: warning: (near initialization for ‘argp’) >>> nxweb/main.c:165: warning: excess elements in struct initializer >>> nxweb/main.c:165: warning: (near initialization for ‘argp’) >>> nxweb/main.c:165: warning: excess elements in struct initializer >>> nxweb/main.c:165: warning: (near initialization for ‘argp’) >>> make: *** [obj/Release/nxweb/main.o] Error 1 >>> >>> >>> I think you should update your *benchmark* to use *weighttp*: >>> http://redmine.lighttpd.net/projects/weighttp/wiki >>> As pointed by G-WAN's author, *ab* performs badly on multi-cores. >>> >>> Regards, >>> Zabrane >>> >>> On Dec 2, 2011, at 9:58 AM, Yaroslav wrote: >>> >>> What a shame. Sorry about that. It is my first public repo at Bitbucket, >>> I turned 'private' flag off, but did not check that I have to make wiki and >>> issue tracker public as well. >>> >>> It should be all OK now. >>> >>> Sorry for the inconvenience. >>> >>> On Fri, Dec 2, 2011 at 12:13 PM, Chris Brody <[email protected]>wrote: >>> >>>> Yeah I created an account with Bitbucket and still could not get access >>>> to the Wiki. >>>> >>>> Can you please make this project public if you want to announce it? >>>> >>>> >>>> On Fri, Dec 2, 2011 at 3:04 AM, Rodrigo Campos <[email protected]>wrote: >>>> >>>>> On Fri, Dec 02, 2011 at 03:55:27AM +0400, Yaroslav wrote: >>>>> > Hi, >>>>> > >>>>> > I'd like to thank the authors of libev for very good product. Based >>>>> on it I >>>>> > have developed NXWEB web server, which I'd like to present to >>>>> community. Here >>>>> > are the highlights: >>>>> > >>>>> > NXWEB project page: https://bitbucket.org/yarosla/nxweb/ >>>>> >>>>> Sounds interesting, but the project page asks for a login. It seems I >>>>> can not >>>>> see anything without an account :( >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Thanks, >>>>> Rodrigo >>>>> >>>>> _______________________________________________ >>>>> libev mailing list >>>>> [email protected] >>>>> http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev >>>> >>>> >>>> >>>> _______________________________________________ >>>> libev mailing list >>>> [email protected] >>>> http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev >>>> >>> >>> _______________________________________________ >>> libev mailing list >>> [email protected] >>> http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev >>> >>> >>> >>> >> > _______________________________________________ > libev mailing list > [email protected] > http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev > > > > >
_______________________________________________ libev mailing list [email protected] http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
