On Wed, Aug 29, 2012 at 12:13 AM, Stefan Fuhrmann <stefan.fuhrm...@wandisco.com> wrote: > > > On Wed, Aug 29, 2012 at 12:00 AM, Johan Corveleyn <jcor...@gmail.com> wrote: >> >> On Mon, Aug 27, 2012 at 10:45 AM, Johan Corveleyn <jcor...@gmail.com> >> wrote: >> > On Mon, Aug 27, 2012 at 2:09 AM, Bert Huijben <b...@qqmail.nl> wrote: >> >>> -----Original Message----- >> >>> From: Johan Corveleyn [mailto:jcor...@gmail.com] >> >>> Sent: maandag 27 augustus 2012 00:39 >> >>> To: Subversion Development >> >>> Subject: notification output bottleneck >> >>> >> >>> I've never noticed this before (slow server, but now I'm testing a >> >>> new, >> >> faster >> >>> server), but it seems that the writing of notification output on >> >>> stdout is >> >> a >> >>> bottleneck for checkout, update or export on Windows (cmd.exe). With a >> >>> fast server, hot caches, and everything on a Gb LAN, checkout of a >> >>> large >> >>> working copy is twice as fast (from 79 minutes down to 35 minutes) >> >>> when I >> >>> redirect output to NUL. I tested with both 1.7.5 (SlikSVN) and trunk, >> >>> the >> >>> results were about the same. >> >>> >> >>> Is there anything that could be done in svn to remove that bottleneck >> >>> (buffering, ...)? >> >>> >> >>> (there might have been some discussion about this before, but I can't >> >>> find >> >> it) >> >> >> >> I found this problem too while profiling. >> >> >> >> The console apis on Windows in the Microsoft libraries are slow because >> >> they >> >> try to be safe for multithreaded applications. There are some defines >> >> that >> >> can be used to speed this up (which will avoid the most costly thread >> >> synchronisations), but we can't just apply these to our code as that >> >> would >> >> make libsvn_subr no longer multithread safe. >> > >> > Interesting. >> > >> > Am I correct in saying that this is really only a problem for the >> > commandline 'svn' client (and perhaps 'svnadmin' and 'svnlook' etc >> > ...)? Other clients can do their own fast notification handling, >> > right? >> > >> > And Isn't 'svn' always single-threaded? >> > >> > In that case: couldn't we make the necessary multithread-unsafe, but >> > fast, output functions for 'svn' only? >> >> Is the above idea possible / sane / desirable? >> >> Bert, what are those defines to speed up console access at the expense >> of thread safety? I'd like to give them a quick try just to see how >> much it would impact. >> >> ISTR from before we migrated to SVN that CVS did not have this >> problem: console output would just fly by very quickly. At least >> that's how I remember it :-), I haven't run CVS for some time now >> (it's also possible that I am misremembering, or that it is because I >> was running the client on WinXP back then). > > > 2D (XP) vs. 3D (Vista+) text rendering? > > If that is the root cause, redirecting to some > text file should eliminate the bottleneck - just > for testing purposes. From a client perspective, > the same API would be used and that also > gives us a clearer idea of what part of the > system is to blame.
Yep, redirecting to a file eliminates the bottleneck (almost the same as redirecting to NUL) (I ran it a couple of times to make sure the server cache was hot): [[[ C:\Temp\svnperf>timeit svn export https://playsvn/repos export1 ... snip output ... start: 2012-29-08 00:48:49,06 end : 2012-29-08 00:53:29,07 total runtime: 280010 ms C:\Temp\svnperf>timeit svn export https://playsvn/repos export2 >output.txt start: 2012-29-08 00:55:14,14 end : 2012-29-08 00:57:31,83 total runtime: 137690 ms C:\Temp\svnperf>timeit svn export https://playsvn/repos export3 >NUL start: 2012-29-08 00:57:48,51 end : 2012-29-08 00:59:56,35 total runtime: 127840 ms ]]] -- Johan
