Re: Looking to improve performance of svn annotate

2010-08-10 Thread Johan Corveleyn
Hi all, Other priorities have unfortunately kept me from focusing on the project of speeding up blame. But recently I've spent some time thinking about it, reading the other mail threads, studying the code and profiling a little bit. I hope I can still do something useful for blame, whether it be

Re: Looking to improve performance of svn annotate

2010-08-11 Thread Johan Corveleyn
Thanks for the input/explanations, Julian and Greg. Some reactions below ... On Wed, Aug 11, 2010 at 12:04 PM, Julian Foad wrote: [...] > I hadn't really appreciated this difficulty. By noticing the copy > source positions when interpreting the delta, we might be able to do > better: > > cp fro

Re: Looking to improve performance of svn annotate

2010-08-17 Thread Johan Corveleyn
On Thu, Aug 12, 2010 at 5:30 PM, Greg Hudson wrote: > On Thu, 2010-08-12 at 10:57 -0400, Julian Foad wrote: >> I'm wary of embedding any client functionality in the server, but I >> guess it's worth considering if it would be that useful.  If so, let's >> take great care to ensure it's only lightl

svn diff optimization to make blame faster?

2010-08-17 Thread Johan Corveleyn
Hi devs, While "Looking to improve performance of svn annotate" [1], I found that the current blame algorithm is mainly client-side bound, and that most of its time is spent on "svn diff" (calls to svn_diff_file_diff_2 from add_file_blame in blame.c). Apart from avoiding to build full-texts and di

Re: svn diff optimization to make blame faster?

2010-08-18 Thread Johan Corveleyn
On Wed, Aug 18, 2010 at 12:49 PM, Stefan Sperling wrote: > On Wed, Aug 18, 2010 at 12:59:21AM +0200, Johan Corveleyn wrote: >> Hi devs, >> >> While "Looking to improve performance of svn annotate" [1], I found >> that the current blame algorithm is mainly c

Re: svn commit: r986521 - in /subversion/branches/performance/subversion: libsvn_fs_fs/id.c libsvn_fs_fs/temp_serializer.c libsvn_subr/svn_temp_serializer.c

2010-08-18 Thread Johan Corveleyn
On Wed, Aug 18, 2010 at 1:11 AM, wrote: > Author: stefan2 > Date: Tue Aug 17 23:11:13 2010 > New Revision: 986521 > > URL: http://svn.apache.org/viewvc?rev=986521&view=rev > Log: > Change serialized representation of pointers: instead of storing offsets > relative to the whole buffer, store the o

Re: Performance branch ready for review

2010-08-18 Thread Johan Corveleyn
On Wed, Aug 18, 2010 at 9:14 PM, Stefan Fuhrmann wrote: > Hi @all, > > I just finished my porting work; the performance branch > is now fully synchronized with my prototype code. > From my point of view, review can start now. > > According to my measurements, the code is now faster > than the orig

Re: svn diff optimization to make blame faster?

2010-08-19 Thread Johan Corveleyn
On Wed, Aug 18, 2010 at 3:44 PM, Johan Corveleyn wrote: > On Wed, Aug 18, 2010 at 12:49 PM, Stefan Sperling wrote: >> Can you show a profiler run that illustrates where the client is >> spending most of its time during diff? That would probably help with >> getting opinions

Re: svn diff optimization to make blame faster?

2010-08-19 Thread Johan Corveleyn
On Thu, Aug 19, 2010 at 1:54 PM, Johan Corveleyn wrote: > On Wed, Aug 18, 2010 at 3:44 PM, Johan Corveleyn wrote: >> FWIW, below is the output of a recent run with my instrumented trunk >> build. I've instrumented svn_diff_diff in diff.c and >> svn_diff__get_tokens

Re: svn diff optimization to make blame faster?

2010-08-19 Thread Johan Corveleyn
On Thu, Aug 19, 2010 at 5:13 PM, Bert Huijben wrote: > > >> -Original Message----- >> From: Johan Corveleyn [mailto:jcor...@gmail.com] >> Sent: donderdag 19 augustus 2010 4:55 >> To: Subversion Development >> Subject: Re: svn diff optimization to make

Re: svn diff optimization to make blame faster?

2010-08-20 Thread Johan Corveleyn
On Thu, Aug 19, 2010 at 11:38 PM, Johan Corveleyn wrote: > Feh, I just redid my apr_time_now+printf profiling with a release > build (of trunk), instead of a debug build, and that makes a *big* > difference. Total time of the svn_diff_diff call is now down to ~300 > ms. Still a lot

Re: Looking to improve performance of svn annotate

2010-08-20 Thread Johan Corveleyn
On Tue, Aug 17, 2010 at 3:26 PM, Johan Corveleyn wrote: > Another thing that occurred to me: since most time of the current > blame implementation is spent on "diff" (svn_diff_file_diff_2), maybe > a quick win could be to simply (?) optimize the diff code? Or write a > spec

Re: Looking to improve performance of svn annotate

2010-08-20 Thread Johan Corveleyn
On Fri, Aug 20, 2010 at 1:40 PM, Johan Corveleyn wrote: > After eliminating antivirus, and running with a release build instead > of a debug build, svn diff is just about on par with GNU diff. So this > eliminates the option of optimizing diff ... Unless ... For every diff dur

Re: Performance branch ready for review

2010-08-23 Thread Johan Corveleyn
On Sun, Aug 22, 2010 at 2:56 PM, Stefan Fuhrmann wrote: > Johan Corveleyn wrote: >> >> On Wed, Aug 18, 2010 at 9:14 PM, Stefan Fuhrmann >> wrote: >> >>> >>> Hi @all, >>> >>> I just finished my porting work; the performance branch &g

Performance branch - svnserve crash in fs_history_prev

2010-08-23 Thread Johan Corveleyn
Hi, I've taken the performance branch for a spin. Some of performance increases are awesome (svn log is ~4 times faster on my machine (tested with a file with 300 revisions)). However, I also experienced a crash of svnserve, for both "svn log" and "svn blame" of a big file with >2000 revisions (s

Re: Performance branch - svnserve crash in fs_history_prev

2010-08-23 Thread Johan Corveleyn
Trying again with .txt extension added. Johan On Mon, Aug 23, 2010 at 11:40 AM, Julian Foad wrote: > On Mon, 2010-08-23, Lieven Govaerts wrote: >> Either you forgot the attachments, or they were dropped by our mailing >> sw. Try adding a .txt extension. > > AFAIK, the mailing list strips attachm

Re: Performance branch ready for review

2010-08-23 Thread Johan Corveleyn
On Tue, Aug 24, 2010 at 12:22 AM, Stefan Fuhrmann wrote: > Daniel Shahaf wrote: >> >> Stefan, you did mention "Patch by" for Johan's patches which you >> committed, did you intend to mention "Found by" or "Suggested by" for >> the other two (quoted below)? >> >> >> http://subversion.apache.org/doc

Re: svn diff optimization to make blame faster?

2010-08-24 Thread Johan Corveleyn
On Sun, Aug 22, 2010 at 4:02 PM, Branko Čibej wrote: > On 18.08.2010 00:59, Johan Corveleyn wrote: >> Hi devs, >> >> While "Looking to improve performance of svn annotate" [1], I found >> that the current blame algorithm is mainly client-side bound, and that

Re: Looking to improve performance of svn annotate

2010-08-24 Thread Johan Corveleyn
On Fri, Aug 20, 2010 at 9:11 PM, Johan Corveleyn wrote: > On Fri, Aug 20, 2010 at 1:40 PM, Johan Corveleyn wrote: >> After eliminating antivirus, and running with a release build instead >> of a debug build, svn diff is just about on par with GNU diff. So this >> eli

Re: Performance branch - svnserve crash in fs_history_prev

2010-08-25 Thread Johan Corveleyn
On Wed, Aug 25, 2010 at 11:34 AM, Stefan Fuhrmann wrote: > Johan Corveleyn wrote: >> However, I also experienced a crash of svnserve, for both "svn log" >> and "svn blame" of a big file with >2000 revisions (so this is quite >> an extreme case). See b

Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_serializer.c

2010-08-29 Thread Johan Corveleyn
Hi Stefan, On Sun, Aug 29, 2010 at 12:32 PM, wrote: > Author: stefan2 > Date: Sun Aug 29 10:32:08 2010 > New Revision: 990537 > > URL: http://svn.apache.org/viewvc?rev=990537&view=rev > Log: > Looking for the cause of Johan Corveleyn's crash (see > http://svn.haxx.se/dev/archive-2010-08/0652.sht

Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_ser ializer.c

2010-08-29 Thread Johan Corveleyn
On Sun, Aug 29, 2010 at 4:36 PM, Daniel Shahaf wrote: > Johan Corveleyn wrote on Sun, Aug 29, 2010 at 16:24:24 +0200: > > Hi Stefan, > > > > On Sun, Aug 29, 2010 at 12:32 PM, wrote: > > > Author: stefan2 > > > Date: Sun Aug 29 10:32:08 2010 > >

Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_serializer.c

2010-08-30 Thread Johan Corveleyn
On Mon, Aug 30, 2010 at 12:05 PM, Stefan Fuhrmann < stefanfuhrm...@alice-dsl.de> wrote: > Johan Corveleyn wrote: > >> On Sun, Aug 29, 2010 at 12:32 PM, wrote: >> /> Author: stefan2 / >> /> Date: Sun Aug 29 10:32:08 2010 / >> /> New Revision: 9905

Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_serializer.c

2010-08-30 Thread Johan Corveleyn
On Mon, Aug 30, 2010 at 9:32 PM, Johan Corveleyn wrote: > On Mon, Aug 30, 2010 at 12:05 PM, Stefan Fuhrmann < > stefanfuhrm...@alice-dsl.de> wrote: > >> I thought so ;) To narrow down the nature of the problem, >> I added some checks that should be able to discern pl

Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_serializer.c

2010-08-31 Thread Johan Corveleyn
On Tue, Aug 31, 2010 at 12:37 AM, Stefan Fuhrmann < stefanfuhrm...@alice-dsl.de> wrote: > Johan Corveleyn wrote: > >> On Mon, Aug 30, 2010 at 9:32 PM, Johan Corveleyn > jcor...@gmail.com>> wrote: >> >>On Mon, Aug 30, 2010 at 12:05 PM, Stefan Fuhrmann &

Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_serializer.c

2010-08-31 Thread Johan Corveleyn
Stefan, On Tue, Aug 31, 2010 at 10:07 PM, Johan Corveleyn wrote: > Some additional info: > - I couldn't reproduce the crash with a narrow range. Not even 9:0 would > crash it (right after startup). > - BUT: if after 9:0 I run log without -r arguments, I get an error on

Worried about single-db performance

2010-09-03 Thread Johan Corveleyn
Hi devs, >From what I understand about the performance problems of WC-1 vs. WC-NG, and what I'm reading on this list, I expect(ed) a huge performance boost from WC-NG for certain client operations (especially on Windows, where the locking of WC-1 is quite problematic). Also, I knew I had to wait f

Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_serializer.c

2010-09-04 Thread Johan Corveleyn
On Tue, Aug 31, 2010 at 11:35 PM, Johan Corveleyn wrote: > Stefan, > > On Tue, Aug 31, 2010 at 10:07 PM, Johan Corveleyn wrote: >> Some additional info: >> - I couldn't reproduce the crash with a narrow range. Not even 9:0 would >> crash it (right after startu

Re: svn diff optimization to make blame faster?

2010-09-04 Thread Johan Corveleyn
On Tue, Aug 24, 2010 at 11:11 AM, Philip Martin wrote: > Johan Corveleyn writes: >> On Sun, Aug 22, 2010 at 4:02 PM, Branko Čibej wrote: >>> svn_diff uses basically the same algorithm as GNU diff but implemented >>> slightly differently and IIRC it does

Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_serializer.c

2010-09-08 Thread Johan Corveleyn
On Wed, Sep 8, 2010 at 12:00 PM, Stefan Fuhrmann wrote: > Johan Corveleyn wrote: >> >> On Mon, Sep 6, 2010 at 2:10 AM, Stefan Fuhrmann >> wrote: >> >>> >>> The only way to get closer to understanding the nature of the >>> problem, I'v

Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_serializer.c

2010-09-08 Thread Johan Corveleyn
On Wed, Sep 8, 2010 at 12:41 PM, Johan Corveleyn wrote: > On Wed, Sep 8, 2010 at 12:00 PM, Stefan Fuhrmann > wrote: >> Johan Corveleyn wrote: >>> >>> On Mon, Sep 6, 2010 at 2:10 AM, Stefan Fuhrmann >>> wrote: >>> >>>> >&g

Re: [Issue 3474] making a new subdir, moving files into it and then renaming the subdir, breaks history of the moved files

2010-09-10 Thread Johan Corveleyn
On Fri, Sep 10, 2010 at 11:45 PM, wrote: > http://subversion.tigris.org/issues/show_bug.cgi?id=3474 > > --- Additional comments from joha...@tigris.org Fri Sep 10 14:45:17 -0700 > 2010 --- > This issue seems to be fixed on trunk. The described scenario now goes as > follows: > > (assumi

Wrong reply-to address in mails from issue tracker

2010-09-10 Thread Johan Corveleyn
Hi, The mails coming from the SVN issue tracker (on the issues mailing list) still contain the old d...@s.t.o address as reply-to. Could someone change this to the d...@s.a.o address? Cheers, -- Johan

Re: svn diff optimization to make blame faster?

2010-09-15 Thread Johan Corveleyn
On Sun, Sep 5, 2010 at 1:53 AM, Johan Corveleyn wrote: > On Tue, Aug 24, 2010 at 11:11 AM, Philip Martin > wrote: >> Johan Corveleyn writes: >>> On Sun, Aug 22, 2010 at 4:02 PM, Branko Čibej wrote: >>>> svn_diff uses basically the same algorithm as GNU d

Re: svn commit: r995478 [1/2] - in /subversion/branches/performance: ./ subversion/bindings/javahl/native/ subversion/bindings/javahl/src/org/apache/subversion/javahl/ subversion/bindings/javahl/src/o

2010-09-16 Thread Johan Corveleyn
On Thu, Sep 9, 2010 at 5:59 PM, wrote: > Author: hwright > Date: Thu Sep  9 15:59:00 2010 > New Revision: 995478 > > URL: http://svn.apache.org/viewvc?rev=995478&view=rev > Log: > On the performance branch: > Bring up-to-date with trunk. The performance branch doesn't compile anymore on my machi

Re: svn diff optimization to make blame faster?

2010-09-20 Thread Johan Corveleyn
On Mon, Sep 20, 2010 at 11:52 AM, Branko Čibej wrote: >  On 15.09.2010 14:20, Johan Corveleyn wrote: >> Some update on this: I have implemented this for svn_diff (excluding >> the identical prefix and suffix of both files, and only then starting >> to fill up the token

[WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-09-26 Thread Johan Corveleyn
Hi devs, As discussed in [1], here is a patch that makes svn_diff_diff (libsvn_diff/diff.c) skip the identical prefix and suffix of the original and modified files, before starting the LCS (longest common subsequence) algorithm on the "non-matching" part. This makes diff a lot faster (especially

Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-09-28 Thread Johan Corveleyn
Hi Daniel, Thanks for the feedback. On Tue, Sep 28, 2010 at 4:11 PM, Daniel Shahaf wrote: >> Index: subversion/include/svn_diff.h >> === >> --- subversion/include/svn_diff.h     (revision 1001548) >> +++ subversion/include/svn_diff.

Re: Interrupting an update after change of externals causes corrupt working copy

2010-09-30 Thread Johan Corveleyn
Hi devs, As per Daniel Shahaf's suggestion, I'm continuing this discussion on the dev list. See the previous mails in this thread on the users list for some context (or http://svn.haxx.se/users/archive-2010-09/0406.shtml). I'll summarize below. This issue reproduces with 1.6.12 as well as with tr

trunk failing tests on Windows XP (32 bit): prop-tests.py 33, stat-tests.py 5, upgrade-tests.py 11

2010-09-30 Thread Johan Corveleyn
Hi devs, The following tests fail on my machine (Windows XP 32-bit, built with VCE 2008, ra_local): - prop-tests.py 33: test properties of obstructed subdirectories svn: Can't open directory 'C:\research\svn\client_build\svn_trunk\Release\subversion\tests\cmdline\svn-test-work\working_copies\prop

Re: trunk failing tests on Windows XP (32 bit): prop-tests.py 33, stat-tests.py 5, upgrade-tests.py 11

2010-10-01 Thread Johan Corveleyn
On Fri, Oct 1, 2010 at 3:45 PM, Paul Burba wrote: > On Thu, Sep 30, 2010 at 8:10 PM, Bert Huijben wrote: >>> -Original Message- >>> From: Johan Corveleyn [mailto:jcor...@gmail.com] >>> Sent: vrijdag 1 oktober 2010 1:51 >>> To: Subversion Developm

Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-10-02 Thread Johan Corveleyn
Hi, Here is a second iteration of the patch. It now passes make check. Differences from the previous version are: - Support for \r eol-style (\n and \r\n was already ok). - The number of prefix_lines is now passed to svn_diff__lcs, so it can use that value to set the position offset of the "EOF"

Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-10-03 Thread Johan Corveleyn
On Sun, Oct 3, 2010 at 1:46 AM, Johan Corveleyn wrote: > Hi, > > Here is a second iteration of the patch. It now passes make check. > > Differences from the previous version are: > - Support for \r eol-style (\n and \r\n was already ok). > - The number of prefix_l

[RFC] Diff (blame) optimization: how to go forward

2010-10-07 Thread Johan Corveleyn
Hi all, This is a follow-up to the WIP-patches I posted last week [1], which are about improving performance of svn_diff (and therefor also blame on the client-side), especially for large files. To summarize: the idea was (is) to strip off the identical prefix and suffix, and then letting the exi

Re: [RFC] caret notation enhancement

2010-10-08 Thread Johan Corveleyn
On Fri, Oct 8, 2010 at 2:35 PM, Karl Heinz Marbaise wrote: > Hi Tino, > >>>     svn cp ^^/trunk ^^/tags/RELEASE-1.0.0 -m"- Tagging" >>> >>> The usage of the doubled ^ is just as an example, cause i know >>> on Windows you already have to type the doubled ^ because of the shell. >> >> So what shoul

Re: [RFC] Diff (blame) optimization: how to go forward

2010-10-08 Thread Johan Corveleyn
On Fri, Oct 8, 2010 at 3:08 PM, Daniel Shahaf wrote: > Johan Corveleyn wrote on Fri, 8 Oct 2010 at 01:44 -: >> With suffix-lines-to-keep=50, you'd need to insert a >> block of text that has its last 50 lines identical to the 50 lines >> preceding the insertion

Re: [RFC] Diff (blame) optimization: how to go forward

2010-10-08 Thread Johan Corveleyn
On Fri, Oct 8, 2010 at 10:35 PM, Hyrum K. Wright wrote: > On Fri, Oct 8, 2010 at 3:24 PM, Johan Corveleyn wrote: >> ... >> >> Just one more thing: as I mentioned in my rather long mail, blame >> would benefit the most from my optimization if the server were fast &g

Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-10-08 Thread Johan Corveleyn
Ok, third iteration of the patch in attachment. It passes make check. As discussed in [1], this version keeps 50 lines of the identical suffix around, to give the algorithm a good chance to generate a diff output of good quality (in all but the most extreme cases, this will be the same as with the

Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-10-09 Thread Johan Corveleyn
On Sat, Oct 9, 2010 at 2:57 AM, Julian Foad wrote: > On Sat, 2010-10-09, Johan Corveleyn wrote: >> Ok, third iteration of the patch in attachment. It passes make check. >> >> As discussed in [1], this version keeps 50 lines of the identical >> suffix around, to give the

Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-10-09 Thread Johan Corveleyn
On Sat, Oct 9, 2010 at 5:19 PM, Daniel Shahaf wrote: > Johan Corveleyn wrote on Sat, Oct 09, 2010 at 14:21:09 +0200: >> (side-note: I considered first doing suffix scanning, then prefix >> scanning, so I could reuse the buffers/pointers from diff_baton all >> the time, and

Re: trunk failing tests on Windows XP (32 bit): prop-tests.py 33, stat-tests.py 5, upgrade-tests.py 11

2010-10-10 Thread Johan Corveleyn
On Sat, Oct 2, 2010 at 11:24 AM, Bert Huijben wrote: >> -Original Message- >> From: Paul Burba [mailto:ptbu...@gmail.com] >> Sent: vrijdag 1 oktober 2010 15:46 >> To: Bert Huijben >> Cc: Johan Corveleyn; Subversion Development >> Subject: Re: trunk

Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-10-10 Thread Johan Corveleyn
On Sat, Oct 9, 2010 at 2:21 PM, Johan Corveleyn wrote: > On Sat, Oct 9, 2010 at 2:57 AM, Julian Foad wrote: >> But this makes me think, it looks to me like this whole >> prefix-suffix-skipping functionality would fit better inside the >> lower-level diff algorithm r

Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-10-11 Thread Johan Corveleyn
On Mon, Oct 11, 2010 at 11:53 AM, Julian Foad wrote: > On Sat, 2010-10-09, Johan Corveleyn wrote: >> On Sat, Oct 9, 2010 at 2:57 AM, Julian Foad wrote: >> > So I wrote a patch - attached - that refactors this into an array of 4 >> > sub-structures, and simplifies

Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-10-12 Thread Johan Corveleyn
On Tue, Oct 12, 2010 at 12:10 PM, Julian Foad wrote: > On Tue, 2010-10-12 at 00:31 +0200, Johan Corveleyn wrote: >> On Mon, Oct 11, 2010 at 11:53 AM, Julian Foad >> wrote: >> > On Sat, 2010-10-09, Johan Corveleyn wrote: >> >> On Sat, Oct 9, 2010 at 2:57 AM, J

Re: trunk failing tests on Windows XP (32 bit): prop-tests.py 33, stat-tests.py 5, upgrade-tests.py 11

2010-10-15 Thread Johan Corveleyn
On Sun, Oct 10, 2010 at 3:07 PM, Johan Corveleyn wrote: > On Sat, Oct 2, 2010 at 11:24 AM, Bert Huijben wrote: >>> -Original Message- >>> From: Paul Burba [mailto:ptbu...@gmail.com] >>> Sent: vrijdag 1 oktober 2010 15:46 >>> To: Bert Huij

Re: trunk failing tests on Windows XP (32 bit): prop-tests.py 33, stat-tests.py 5, upgrade-tests.py 11

2010-10-15 Thread Johan Corveleyn
On Fri, Oct 15, 2010 at 12:20 PM, Johan Corveleyn wrote: > I saw that the problem was fixed in r102170 by Bert s/r102170/r1021760/ I really should install that "Undo send" google labs extension :) -- Johan

Re: svn commit: r1022931 - in /subversion/trunk/subversion/libsvn_wc: status.c wc-queries.sql wc_db.c wc_db.h

2010-10-15 Thread Johan Corveleyn
On Fri, Oct 15, 2010 at 4:19 PM, wrote: > Author: philip > Date: Fri Oct 15 14:19:36 2010 > New Revision: 1022931 > > URL: http://svn.apache.org/viewvc?rev=1022931&view=rev > Log: > Implement status using per-dir queries.  On my machine (Linux, local > disk) this improves the speed of status on a

Re: massive memory leak

2010-10-19 Thread Johan Corveleyn
On Tue, Oct 19, 2010 at 8:14 PM, Stefan Küng wrote: > On 19.10.2010 20:13, Lieven Govaerts wrote: > >> This has been fixed in serf trunk r1408 for a while, but hasn't shown >> up in a serf patch release yet. > > Sorry, I should have checked the serf commits first. > > Thanks for the update on this

diff4: is it actually used?

2010-10-19 Thread Johan Corveleyn
Hi devs, In the context of the diff optimization patch I'm working on ([1]), I'm wondering if diff4 is actually used in svn. If I look for usages of subversion/libsvn_diff/diff4.c#svn_diff_diff4, I only come up with tools/diff/diff4.c#main. So: this code isn't used in the svn core itself? What's

Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-10-19 Thread Johan Corveleyn
On Tue, Oct 12, 2010 at 12:10 PM, Julian Foad wrote: > On Tue, 2010-10-12 at 00:31 +0200, Johan Corveleyn wrote: >> On Mon, Oct 11, 2010 at 11:53 AM, Julian Foad >> wrote: >> > On Sat, 2010-10-09, Johan Corveleyn wrote: >> >> On Sat, Oct 9, 2010 at 2:57 AM, J

Diff optimization: implement prefix/suffix-skipping in token-handling code (was: Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster)

2010-10-19 Thread Johan Corveleyn
On Tue, Oct 12, 2010 at 12:35 PM, Julian Foad wrote: > On Sun, 2010-10-10 at 23:43 +0200, Johan Corveleyn wrote: >> On Sat, Oct 9, 2010 at 2:21 PM, Johan Corveleyn wrote: >> > On Sat, Oct 9, 2010 at 2:57 AM, Julian Foad >> > wrote: >> >> But this makes m

Issue triage: issues 3429 and 3474

2010-11-08 Thread Johan Corveleyn
Hi, I recently verified that the following two issues are fixed on trunk: - #3429: "svn mv A B; svn mv B A" generates replace without history - #3474: making a new subdir, moving files into it and then renaming the subdir, breaks history of the moved files Should I mark these as fixed in the is

Re: Interrupting an update after change of externals causes corrupt working copy

2010-11-15 Thread Johan Corveleyn
Almost forgot about this. It's now filed in the issue tracker: http://subversion.tigris.org/issues/show_bug.cgi?id=3751 Cheers, Johan On Thu, Sep 30, 2010 at 10:03 PM, Johan Corveleyn wrote: > Hi devs, > > As per Daniel Shahaf's suggestion, I'm continuing this discussio

Re: Issue triage: issues 3429 and 3474

2010-11-16 Thread Johan Corveleyn
On Mon, Nov 8, 2010 at 1:02 PM, Philip Martin wrote: > Johan Corveleyn writes: > >> Or, maybe the best approach: I could add a regression test for these >> issues, so we can all be sure that they are fixed (and remain fixed), >> after which they can be marked as fixe

[PATCH] copy_tests.py - expand move_file_back_and_forth, to verify issue #3429

2010-11-16 Thread Johan Corveleyn
The attached patch expands the test "move_file_back_and_forth" (copy_tests.py 45) as a regression test for issue#3429. [[[ Expand move_file_back_and_forth test to verify issue #3429 ("svn mv A B; svn mv B A" generates replace without history). * subversion/tests/cmdline/copy_tests.py (move_file

Re: [PATCH] copy_tests.py - expand move_file_back_and_forth, to verify issue #3429

2010-11-16 Thread Johan Corveleyn
On Wed, Nov 17, 2010 at 2:14 AM, Johan Corveleyn wrote: > The attached patch expands the test "move_file_back_and_forth" > (copy_tests.py 45) as a regression test for issue#3429. > > [[[ > Expand move_file_back_and_forth test to verify issue #3429 > ("svn mv A

Re: Issue triage: issues 3429 and 3474

2010-11-17 Thread Johan Corveleyn
On Wed, Nov 17, 2010 at 7:14 PM, Daniel Shahaf wrote: > Johan Corveleyn wrote on Wed, Nov 17, 2010 at 01:25:24 +0100: >> On Mon, Nov 8, 2010 at 1:02 PM, Philip Martin >> wrote: >> > Johan Corveleyn writes: >> > >> >> Or, maybe the best appr

Diff optimizations progress (or not)

2010-11-20 Thread Johan Corveleyn
[ adding dev@ to discussion between danielsh and me about (non-)progress of the diff-optimizations work. ] On Wed, Nov 17, 2010 at 3:21 AM, Daniel Shahaf wrote: > [ if you want to, you can add dev@ to the CC on replies. ] > > Johan Corveleyn wrote on Wed, Nov 17, 2010 at 00:57:53 +0

Re: AW: How to find out the rev number where a file was deleted?

2010-11-29 Thread Johan Corveleyn
discussion continued about why this is so hard right now, and if there are alternative approaches. See below... On Mon, Nov 29, 2010 at 3:51 AM, Daniel Shahaf wrote: > Johan Corveleyn wrote on Sun, Nov 28, 2010 at 21:20:28 +0100: >> On Sun, Nov 28, 2010 at 6:35 PM, Daniel Shahaf >> wro

diff-optimizations-tokens branch: I think I'm going to abandon it

2010-11-30 Thread Johan Corveleyn
Hi devs, As mentioned in [1], I've created two branches to try out two different approaches for the diff optimizations of prefix/suffix scanning. The first one, diff-optimizations-bytes, has a working implementation of the optimization. It still has some open todo items, but it basically works.

Re: diff-optimizations-tokens branch: I think I'm going to abandon it

2010-12-01 Thread Johan Corveleyn
On Wed, Dec 1, 2010 at 3:38 AM, Daniel Shahaf wrote: > Johan Corveleyn wrote on Wed, Dec 01, 2010 at 00:25:27 +0100: >> I am now considering to abandon the tokens-approach, for the following >> reasons: > ... >> So, unless someone can convince me otherwise, I'm proba

Re: diff-optimizations-tokens branch: I think I'm going to abandon it

2010-12-01 Thread Johan Corveleyn
On Wed, Dec 1, 2010 at 11:44 AM, Daniel Shahaf wrote: > Johan Corveleyn wrote on Wed, Dec 01, 2010 at 10:05:29 +0100: >> On Wed, Dec 1, 2010 at 3:38 AM, Daniel Shahaf >> wrote: >> > Johan Corveleyn wrote on Wed, Dec 01, 2010 at 00:25:27 +0100: >> >> I am n

Re: diff-optimizations-tokens branch: I think I'm going to abandon it

2010-12-02 Thread Johan Corveleyn
On Thu, Dec 2, 2010 at 6:43 AM, Daniel Shahaf wrote: > [ finally getting back to this mail; having slept on it, etc. ] > > Johan Corveleyn wrote on Wed, Dec 01, 2010 at 13:34:48 +0100: >> On Wed, Dec 1, 2010 at 11:44 AM, Daniel Shahaf >> wrote: >> > Johan Corveleyn

Re: diff-optimizations-tokens branch: I think I'm going to abandon it

2010-12-02 Thread Johan Corveleyn
On Thu, Dec 2, 2010 at 4:21 PM, Julian Foad wrote: > Hi Johan. > > I've just read the whole of this thread. > > I didn't quite understand your original point (2) that "token-based > suffix scanning will not be as fast as byte-based suffix scanning". > Sure it won't, but is there any reason you men

Re: diff-optimizations-tokens branch: I think I'm going to abandon it

2010-12-02 Thread Johan Corveleyn
On Thu, Dec 2, 2010 at 6:18 PM, Bill Tutt wrote: > Note: This email only tangentially relates to svn diff and more about > reverse token scanning in general: > > As someone who has implemented suffix reverse token scanning before: Thanks for the input. It's nice to see other people have also stru

Implementations of svn_diff_fns_t?

2010-12-02 Thread Johan Corveleyn
Hi, This question came up during recent discussion about the diff-optimizations-tokens branch [1]: What are the known implementors of svn_diff_fns_t, the vtable of svn_diff callback functions in subversion/include/svn_diff.h? Besides the internal diff_memory.c and diff_file.c that is. Are there

Re: [PATCH] copy_tests.py - expand move_file_back_and_forth, to verify issue #3429

2010-12-08 Thread Johan Corveleyn
On Wed, Dec 8, 2010 at 3:17 PM, Julian Foad wrote: > On Wed, 2010-11-17, Johan Corveleyn wrote: >> On Wed, Nov 17, 2010 at 2:14 AM, Johan Corveleyn wrote: >> > The attached patch expands the test "move_file_back_and_forth" >> > (copy_tests.py 45) as a regress

Re: Missing SVN_ERR() wraps

2010-12-13 Thread Johan Corveleyn
On Mon, Dec 13, 2010 at 5:21 PM, Daniel Shahaf wrote: > danie...@apache.org wrote on Fri, Dec 10, 2010 at 18:18:43 -: >> Author: danielsh >> Date: Fri Dec 10 18:18:42 2010 >> New Revision: 1044467 >> >> URL: http://svn.apache.org/viewvc?rev=1044467&view=rev >> Log: >> Fix error leaks. >> >> *

[RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-14 Thread Johan Corveleyn
Hi all, On the diff-optimizations-bytes branch, in diff_file.c, there are two functions which are called for every byte of the identical prefix and suffix: increment_pointers and decrement_pointers. These functions are actually equivalents of curp++ or curp--, reading the next/previous byte, but t

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-15 Thread Johan Corveleyn
On Wed, Dec 15, 2010 at 10:30 AM, Branko Čibej wrote: > On 15.12.2010 02:30, Stefan Fuhrmann wrote: >> On 14.12.2010 23:35, Johan Corveleyn wrote: >>> Some considerations: >>> - Maybe I can use APR_INLINE, with similar results? >>> >>> - Maybe I can

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-15 Thread Johan Corveleyn
On Wed, Dec 15, 2010 at 2:30 AM, Stefan Fuhrmann wrote: > On 14.12.2010 23:35, Johan Corveleyn wrote: >> >> Hi all, > > Hi Johan ;) Hi Stefan, thanks for the input :). I suspected that you might have some ideas about this ... >> On the diff-optimizations-bytes branch

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-15 Thread Johan Corveleyn
On Wed, Dec 15, 2010 at 11:50 AM, Johan Corveleyn wrote: > On Wed, Dec 15, 2010 at 2:30 AM, Stefan Fuhrmann wrote: >> On 14.12.2010 23:35, Johan Corveleyn wrote: >>> >>> Hi all, >> >> Hi Johan ;) > > Hi Stefan, thanks for the input :). I suspecte

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-15 Thread Johan Corveleyn
On Wed, Dec 15, 2010 at 11:53 AM, Johan Corveleyn wrote: > On Wed, Dec 15, 2010 at 11:50 AM, Johan Corveleyn wrote: >> Also, I did my measurements with a blame operation on this very large >> file, which has ~2500 revisions. So that's 2500 diffs of a ~1,5 Mb >> file, w

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-19 Thread Johan Corveleyn
On Wed, Dec 15, 2010 at 10:58 AM, Stefan Fuhrmann wrote: > On 15.12.2010 02:30, Stefan Fuhrmann wrote: >> >> On 14.12.2010 23:35, Johan Corveleyn wrote: >> >>> Thoughts? >> >> Two things you might try: >> * introduce a local variable for afile[i]

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-20 Thread Johan Corveleyn
On Mon, Dec 20, 2010 at 11:03 AM, Julian Foad wrote: > On Mon, 2010-12-20, Johan Corveleyn wrote: >> New macro version (increment only, decrement is similar): >> [[[ >> /* For all files in the FILE array, increment the curp pointer.  If a file >>  * points before th

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-20 Thread Johan Corveleyn
On Mon, Dec 20, 2010 at 2:16 PM, Johan Corveleyn wrote: > On Mon, Dec 20, 2010 at 11:03 AM, Julian Foad > wrote: >> On Mon, 2010-12-20, Johan Corveleyn wrote: >>> New macro version (increment only, decrement is similar): >>> [[[ >>> /* For all files in the

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-20 Thread Johan Corveleyn
On Mon, Dec 20, 2010 at 11:19 AM, Philip Martin wrote: > Johan Corveleyn writes: > >> This makes the diff algorithm another 10% - 15% >> faster (granted, this was measured with my "extreme" testcase of a 1,5 >> Mb file (6 lines), of which most lines are id

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-22 Thread Johan Corveleyn
On Wed, Dec 22, 2010 at 11:50 AM, Philip Martin wrote: > Johan Corveleyn writes: > >> On Mon, Dec 20, 2010 at 11:19 AM, Philip Martin >> wrote: >>> Johan Corveleyn writes: >>> >>>> This makes the diff algorithm another 10% - 15% >>

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-23 Thread Johan Corveleyn
On Thu, Dec 23, 2010 at 1:05 PM, Julian Foad wrote: > On Thu, 2010-12-23, Daniel Shahaf wrote: >> Daniel Shahaf wrote on Thu, Dec 23, 2010 at 13:25:40 +0200: >> > Johan Corveleyn wrote on Thu, Dec 23, 2010 at 01:51:08 +0100: >> > > Yes, that's a good idea. I

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-23 Thread Johan Corveleyn
out in the end. That's also an option, but I think it's better to write it out every time it's doubled (to avoid using up too much memory). But good suggestion anyway. I think I will go with some variation of this (starting with a large enough chunk of representative conten

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-23 Thread Johan Corveleyn
On Thu, Dec 23, 2010 at 11:43 PM, Gavin Beau Baumanis wrote: > On 23/12/2010, at 11:36 PM, Johan Corveleyn wrote: > >> On Thu, Dec 23, 2010 at 3:44 AM, Gavin Beau Baumanis >> wrote: >>> Hi Johan, >> >> Hi Gavin. Thanks for your interest. It's a nice p

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-28 Thread Johan Corveleyn
On Fri, Dec 24, 2010 at 3:40 PM, Stefan Fuhrmann wrote: > On 20.12.2010 02:43, Johan Corveleyn wrote: >> >> On Wed, Dec 15, 2010 at 10:58 AM, Stefan Fuhrmann  wrote: >>> >>> On 15.12.2010 02:30, Stefan Fuhrmann wrote: >>>> >>>> On 14.

Re: FSFS format 6 (was: Re: FSv2)

2010-12-28 Thread Johan Corveleyn
On Sun, Dec 12, 2010 at 4:23 PM, Stefan Fuhrmann wrote: > On 19.10.2010 15:10, Daniel Shahaf wrote: >> >> Greg Stein wrote on Tue, Oct 19, 2010 at 04:31:42 -0400: >>> >>> Personally, I see [FSv2] as a broad swath of API changes to align our >>> needs with the underlying storage. Trowbridge noted t

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-30 Thread Johan Corveleyn
On Tue, Dec 28, 2010 at 7:31 PM, Johan Corveleyn wrote: > On Fri, Dec 24, 2010 at 3:40 PM, Stefan Fuhrmann wrote: >> On 20.12.2010 02:43, Johan Corveleyn wrote: >>> >>> On Wed, Dec 15, 2010 at 10:58 AM, Stefan Fuhrmann  wrote: >>>> >>>> On 15.1

diff-optimizations-bytes: how to make diff3 work with prefix/suffix scanning

2011-01-01 Thread Johan Corveleyn
[ Taking a privately-started discussion with danielsh to the list, in case others have inspiration/insight about this. Question at hand: I'm having trouble making diff3 work with prefix/suffix scanning of the diff-optimizations-bytes branch. Any feedback is highly appreciated :-). ] On Fri, Dec 31

Re: svn commit: r1054286 - /subversion/trunk/subversion/libsvn_delta/xdelta.c

2011-01-02 Thread Johan Corveleyn
On Sat, Jan 1, 2011 at 9:33 PM, wrote: > Author: stefan2 > Date: Sat Jan  1 20:33:22 2011 > New Revision: 1054286 > > URL: http://svn.apache.org/viewvc?rev=1054286&view=rev > Log: > XDelta calculation is major part of svn_txdelta_send_txstream. > Therefore, speed up string matching code and the r

Re: svn commit: r1054286 - /subversion/trunk/subversion/libsvn_delta/xdelta.c

2011-01-02 Thread Johan Corveleyn
On Sun, Jan 2, 2011 at 9:20 AM, Johan Corveleyn wrote: > On Sat, Jan 1, 2011 at 9:33 PM,   wrote: >> Author: stefan2 >> Date: Sat Jan  1 20:33:22 2011 >> New Revision: 1054286 >> >> URL: http://svn.apache.org/viewvc?rev=1054286&view=rev >>

Re: svn commit: r1054286 - /subversion/trunk/subversion/libsvn_delta/xdelta.c

2011-01-02 Thread Johan Corveleyn
On Sat, Jan 1, 2011 at 9:33 PM, wrote: > Author: stefan2 > Date: Sat Jan  1 20:33:22 2011 > New Revision: 1054286 > > URL: http://svn.apache.org/viewvc?rev=1054286&view=rev > Log: > XDelta calculation is major part of svn_txdelta_send_txstream. > Therefore, speed up string matching code and the r

Re: svn commit: r1054286 - /subversion/trunk/subversion/libsvn_delta/xdelta.c

2011-01-02 Thread Johan Corveleyn
On Sun, Jan 2, 2011 at 4:45 PM, Stefan Fuhrmann wrote: > On 02.01.2011 16:38, Johan Corveleyn wrote: >> >> On Sun, Jan 2, 2011 at 9:20 AM, Johan Corveleyn  wrote: >>> >>> On Sat, Jan 1, 2011 at 9:33 PM,  wrote: >>>> >>>> Author: stef

Re: -bytes branch reports bogus conflict that trunk doesn't

2011-01-02 Thread Johan Corveleyn
On Sun, Jan 2, 2011 at 7:36 PM, Daniel Shahaf wrote: > Johan, > > I saw your sync merge, so I ran 'svn up' with intent to rebuild.  When > I ran that 'up' with the branch build, I got a conflict, which didn't > appear when I ran it with a trunk build.  (See attached transcript.) Did you do that w

  1   2   3   4   5   6   7   8   9   10   >