That is right. The reasoning is similar: scheduling threads has some
overhead in boost::asio and adding more threads adds to the
contention. To minimize this adverse effect in this particular
experiment, one can change the code to have read/write handlers be
only invoked from the same thread. Here is how to do that in
dev/destiny-fast:
* Open src/lib/openflow.c.
* Find the lines with async_{read,write}*.
* Add strand.wrap() around boost::bind.
If you are interested in a more in-depth analysis, let me know. We
have a draft that we can share with you.
Hope it helps,
Amin
On Thu, Oct 27, 2011 at 10:07 PM, Andreas Voellmy
<andreas.voel...@gmail.com> wrote:
> Thanks. I noticed that when I run cbench with 1 switch against the learning
> switch controller in destiny, I get much worse throughput if I run with 32
> OS threads than with 1 OS thread. With 1 OS thread I get 470k requests per
> second, while with 32 OS threads I get 22k requests per second. With even 2
> OS threads, the throughput drops to 49k requests per second. Have you
> noticed this behavior? Do you know why this is happening? Is this something
> that will be fixed in your upcoming release of this branch?
> Andreas
>
> On Thu, Oct 27, 2011 at 12:15 PM, Amin Tootoonchian <a...@cs.toronto.edu>
> wrote:
>>
>> On Wed, Oct 26, 2011 at 7:32 PM, Andreas Voellmy
>> <andreas.voel...@gmail.com> wrote:
>> > On Wed, Oct 26, 2011 at 8:42 PM, Amin Tootoonchian <a...@cs.toronto.edu>
>> > wrote:
>> >>
>> >> I only updated the 'switch' app in that code base, and I never looked
>> >> at 'hub'. My guess is that the hub app is doing so little that locking
>> >> within boost::asio scheduler outweights the actual work done by the
>> >> hub app. We need to make sure that the amount of work done by each
>> >> thread upon its invocation is significantly more than the locking
>> >> overhead in boost::asio's internal job queue.
>> >>
>> >
>> > I'm unclear about how components in the destiny branch work. Do the
>> > handlers
>> > run concurrently by default, or is there something extra that one has to
>> > write to get them to execute concurrently? If something extra is needed,
>> > what is it in switch.cc that makes it execute concurrently? Or are you
>> > saying that the event handlers in 'hub' are indeed running concurrently,
>> > but
>> > they aren't doing enough work to get much performance gain? (By the way,
>> > I
>> > was looking at /src/nox/coreapps/switch/switch.cc
>> > and /src/nox/coreapps/hub/hub.cc)
>> >
>> > Thanks,
>> > Andreas
>>
>> They run concurrently by default. They should be indeed running
>> concurrently, but I am guessing locking overhead within boost::asio
>> significantly outweights the actual work done by each thread. It
>> shouldn't be hard to fix, but not worth it since we consider that code
>> base to be just a proof of concept.
>>
>> Thanks,
>> Amin
>>
>> >>
>> >> Cheers,
>> >> Amin
>> >>
>> >> P.S.: Btw, passing '--enable-ndebug' to configure should boost the
>> >> performace.
>> >>
>> >> On Wed, Oct 26, 2011 at 2:08 PM, Andreas Voellmy
>> >> <andreas.voel...@gmail.com> wrote:
>> >> > Thanks. The code compiled after configuring without python.
>> >> > I was able to get roughly the same kind of performance out of the
>> >> > 'switch'
>> >> > application that is mentioned on the performance page
>> >> >
>> >> >
>> >> > (http://www.openflow.org/wk/index.php/Controller_Performance_Comparisons).
>> >> > However, the 'hub' controller doesn't have much speedup when running
>> >> > with
>> >> > more threads. For example, when running with one thread I get a
>> >> > throughput
>> >> > of 213868.81 and when I run it with 8 threads I get a throughput
>> >> > of 264017.35. (To run with 8 threads, I am starting the controller
>> >> > like
>> >> > this: "./nox_core -i ptcp:9999 hub -t 8"; I am testing with cbench in
>> >> > throughput mode "cbench -p 9999 -t")
>> >> > Is this - that 'hub' gets not much speedup while 'switch' gets lots
>> >> > of
>> >> > speedup - expected with this branch of NOX? Is there something that
>> >> > needs to
>> >> > be done to hub in order to enable the framework to run it
>> >> > concurrently?
>> >> > Regards,
>> >> > Andreas
>> >> >
>> >> > On Wed, Oct 26, 2011 at 5:53 AM, Murphy McCauley <jam...@nau.edu>
>> >> > wrote:
>> >> >>
>> >> >> This branch is quite a bit behind the actual development. We're
>> >> >> preparing
>> >> >> to release the updated codebase in the near future.
>> >> >> But for one thing, Python doesn't work in it. So you probably need
>> >> >> to
>> >> >> do
>> >> >> --with-python=no when you run configure.
>> >> >> Hope that helps.
>> >> >> -- Murphy
>> >> >> On Oct 25, 2011, at 8:49 PM, Andreas Voellmy wrote:
>> >> >>
>> >> >> Thanks. I tried editing the conflict marker out in a couple ways
>> >> >> that
>> >> >> seemed reasonable to me, but I got other compile errors. Does anyone
>> >> >> know if
>> >> >> there is a known working version of this branch in the repository,
>> >> >> and
>> >> >> how I
>> >> >> can get back to it?
>> >> >> Thanks,
>> >> >> Andreas
>> >> >>
>> >> >> 2011/10/25 Zoltán Lajos Kis <zoltan.lajos....@ericsson.com>
>> >> >>>
>> >> >>> Seems like someone checked in a conflict marker to that file:
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> http://noxrepo.org/cgi-bin/gitweb.cgi?p=nox;a=blob;f=src/nox/coreapps/pyrt/context.i;h=cb8641d72feb3a1f0543e97830a2addd55d502b9;hb=dev/destiny-fast#l83
>> >> >>>
>> >> >>> Z.
>> >> >>>
>> >> >>> ________________________________________
>> >> >>> From: nox-dev-boun...@noxrepo.org [nox-dev-boun...@noxrepo.org] On
>> >> >>> Behalf
>> >> >>> Of Andreas Voellmy [andreas.voel...@gmail.com]
>> >> >>> Sent: Wednesday, October 26, 2011 4:40 AM
>> >> >>> To: nox-dev@noxrepo.org
>> >> >>> Subject: [nox-dev] Error building dev/destiny-fast branch
>> >> >>>
>> >> >>> Hi,
>> >> >>>
>> >> >>> I'd like to try the destiny-fast branch (I saw it mentioned here:
>> >> >>>
>> >> >>>
>> >> >>> http://www.openflow.org/wk/index.php/Controller_Performance_Comparisons),
>> >> >>> so
>> >> >>> I did the following
>> >> >>>
>> >> >>> git clone git://noxrepo.org/nox<http://noxrepo.org/nox>
>> >> >>> cd nox
>> >> >>> git checkout dev/destiny-fast
>> >> >>>
>> >> >>> Is that the right way to get this branch? After that I ran
>> >> >>> ./boot.sh
>> >> >>> mkdir build
>> >> >>> cd build
>> >> >>> ../configure
>> >> >>> make
>> >> >>>
>> >> >>> and got the following error:
>> >> >>>
>> >> >>> Making all in pyrt
>> >> >>> make[8]: Entering directory
>> >> >>> `/home/av/Download/nox-destiny/nox/build/src/nox/coreapps/pyrt'
>> >> >>> /usr/bin/swig -c++ -python -DSWIGWORDSIZE64
>> >> >>> -I../../../../../src/include/openflow -I../../../../../src/nox/lib/
>> >> >>> -outdir
>> >> >>> ./. -o oxidereactor_wrap.cc -module oxidereactor
>> >> >>> ../../../../../src/nox/coreapps/pyrt/oxidereactor.i
>> >> >>> /usr/bin/swig -c++ -python -DSWIGWORDSIZE64 -outdir ./. -o
>> >> >>> deferredcallback_wrap.cc -module deferredcallback
>> >> >>> ../../../../../src/nox/coreapps/pyrt/deferredcallback.i
>> >> >>> /usr/bin/swig -c++ -python -DSWIGWORDSIZE64
>> >> >>> -I../../../../../src/include/openflow -I../../../../../src/nox/lib/
>> >> >>> -outdir
>> >> >>> ./. -o pycomponent_wrap.cc -module pycomponent
>> >> >>> ../../../../../src/nox/coreapps/pyrt/component.i
>> >> >>> ../../../../../src/nox/coreapps/pyrt/context.i:79: Error: Syntax
>> >> >>> error
>> >> >>> in
>> >> >>> input(3).
>> >> >>> make[8]: *** [pycomponent.py] Error 1
>> >> >>>
>> >> >>> Does anyone know what went wrong and how to fix this?
>> >> >>>
>> >> >>> Thanks,
>> >> >>> Andreas
>> >> >>>
>> >> >>
>> >> >> _______________________________________________
>> >> >> nox-dev mailing list
>> >> >> nox-dev@noxrepo.org
>> >> >> http://noxrepo.org/mailman/listinfo/nox-dev
>> >> >>
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > nox-dev mailing list
>> >> > nox-dev@noxrepo.org
>> >> > http://noxrepo.org/mailman/listinfo/nox-dev
>> >> >
>> >> >
>> >
>> >
>
>
_______________________________________________
nox-dev mailing list
nox-dev@noxrepo.org
http://noxrepo.org/mailman/listinfo/nox-dev
