Re: svn commit: r982057 - in /subversion/branches/performance/subversion/svnserve: main.c server.h

2010-08-04 Thread Johan Corveleyn
On Wed, Aug 4, 2010 at 12:46 AM, stef...@apache.org wrote: Author: stefan2 Date: Tue Aug  3 22:46:10 2010 New Revision: 982057 URL: http://svn.apache.org/viewvc?rev=982057view=rev Log: Add compression, memory-cache-size and open-file-count command line parameters to svnserve. The latter

Re: Looking to improve performance of svn annotate

2010-08-10 Thread Johan Corveleyn
Hi all, Other priorities have unfortunately kept me from focusing on the project of speeding up blame. But recently I've spent some time thinking about it, reading the other mail threads, studying the code and profiling a little bit. I hope I can still do something useful for blame, whether it be

Re: Looking to improve performance of svn annotate

2010-08-17 Thread Johan Corveleyn
On Thu, Aug 12, 2010 at 5:30 PM, Greg Hudson ghud...@mit.edu wrote: On Thu, 2010-08-12 at 10:57 -0400, Julian Foad wrote: I'm wary of embedding any client functionality in the server, but I guess it's worth considering if it would be that useful.  If so, let's take great care to ensure it's

svn diff optimization to make blame faster?

2010-08-17 Thread Johan Corveleyn
Hi devs, While Looking to improve performance of svn annotate [1], I found that the current blame algorithm is mainly client-side bound, and that most of its time is spent on svn diff (calls to svn_diff_file_diff_2 from add_file_blame in blame.c). Apart from avoiding to build full-texts and

Re: Performance branch ready for review

2010-08-18 Thread Johan Corveleyn
On Wed, Aug 18, 2010 at 9:14 PM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: Hi @all, I just finished my porting work; the performance branch is now fully synchronized with my prototype code. From my point of view, review can start now. According to my measurements, the code is now

Re: svn diff optimization to make blame faster?

2010-08-19 Thread Johan Corveleyn
On Wed, Aug 18, 2010 at 3:44 PM, Johan Corveleyn jcor...@gmail.com wrote: On Wed, Aug 18, 2010 at 12:49 PM, Stefan Sperling s...@elego.de wrote: Can you show a profiler run that illustrates where the client is spending most of its time during diff? That would probably help with getting

Re: svn diff optimization to make blame faster?

2010-08-20 Thread Johan Corveleyn
On Thu, Aug 19, 2010 at 11:38 PM, Johan Corveleyn jcor...@gmail.com wrote: Feh, I just redid my apr_time_now+printf profiling with a release build (of trunk), instead of a debug build, and that makes a *big* difference. Total time of the svn_diff_diff call is now down to ~300 ms. Still a lot

Re: Looking to improve performance of svn annotate

2010-08-20 Thread Johan Corveleyn
On Tue, Aug 17, 2010 at 3:26 PM, Johan Corveleyn jcor...@gmail.com wrote: Another thing that occurred to me: since most time of the current blame implementation is spent on diff (svn_diff_file_diff_2), maybe a quick win could be to simply (?) optimize the diff code? Or write a specialized

Re: Looking to improve performance of svn annotate

2010-08-20 Thread Johan Corveleyn
On Fri, Aug 20, 2010 at 1:40 PM, Johan Corveleyn jcor...@gmail.com wrote: After eliminating antivirus, and running with a release build instead of a debug build, svn diff is just about on par with GNU diff. So this eliminates the option of optimizing diff ... Unless ... For every diff during

Re: Performance branch ready for review

2010-08-23 Thread Johan Corveleyn
On Sun, Aug 22, 2010 at 2:56 PM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: Johan Corveleyn wrote: On Wed, Aug 18, 2010 at 9:14 PM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: Hi @all, I just finished my porting work; the performance branch is now fully synchronized

Performance branch - svnserve crash in fs_history_prev

2010-08-23 Thread Johan Corveleyn
Hi, I've taken the performance branch for a spin. Some of performance increases are awesome (svn log is ~4 times faster on my machine (tested with a file with 300 revisions)). However, I also experienced a crash of svnserve, for both svn log and svn blame of a big file with 2000 revisions (so

Re: Performance branch - svnserve crash in fs_history_prev

2010-08-23 Thread Johan Corveleyn
Trying again with .txt extension added. Johan On Mon, Aug 23, 2010 at 11:40 AM, Julian Foad julian.f...@wandisco.com wrote: On Mon, 2010-08-23, Lieven Govaerts wrote: Either you forgot the attachments, or they were dropped by our mailing sw. Try adding a .txt extension. AFAIK, the mailing

Re: Performance branch ready for review

2010-08-23 Thread Johan Corveleyn
On Tue, Aug 24, 2010 at 12:22 AM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: Daniel Shahaf wrote: Stefan, you did mention Patch by for Johan's patches which you committed, did you intend to mention Found by or Suggested by for the other two (quoted below)?

Re: svn diff optimization to make blame faster?

2010-08-24 Thread Johan Corveleyn
On Sun, Aug 22, 2010 at 4:02 PM, Branko Čibej br...@xbc.nu wrote: On 18.08.2010 00:59, Johan Corveleyn wrote: Hi devs, While Looking to improve performance of svn annotate [1], I found that the current blame algorithm is mainly client-side bound, and that most of its time is spent on svn

Re: Looking to improve performance of svn annotate

2010-08-24 Thread Johan Corveleyn
On Fri, Aug 20, 2010 at 9:11 PM, Johan Corveleyn jcor...@gmail.com wrote: On Fri, Aug 20, 2010 at 1:40 PM, Johan Corveleyn jcor...@gmail.com wrote: After eliminating antivirus, and running with a release build instead of a debug build, svn diff is just about on par with GNU diff. So

Re: Performance branch - svnserve crash in fs_history_prev

2010-08-25 Thread Johan Corveleyn
On Wed, Aug 25, 2010 at 11:34 AM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: Johan Corveleyn wrote: However, I also experienced a crash of svnserve, for both svn log and svn blame of a big file with 2000 revisions (so this is quite an extreme case). See both .log files in attachment

Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_serializer.c

2010-08-29 Thread Johan Corveleyn
Hi Stefan, On Sun, Aug 29, 2010 at 12:32 PM, stef...@apache.org wrote: Author: stefan2 Date: Sun Aug 29 10:32:08 2010 New Revision: 990537 URL: http://svn.apache.org/viewvc?rev=990537view=rev Log: Looking for the cause of Johan Corveleyn's crash (see

Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_ser ializer.c

2010-08-29 Thread Johan Corveleyn
On Sun, Aug 29, 2010 at 4:36 PM, Daniel Shahaf d...@daniel.shahaf.namewrote: Johan Corveleyn wrote on Sun, Aug 29, 2010 at 16:24:24 +0200: Hi Stefan, On Sun, Aug 29, 2010 at 12:32 PM, stef...@apache.org wrote: Author: stefan2 Date: Sun Aug 29 10:32:08 2010 New Revision: 990537

Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_serializer.c

2010-08-30 Thread Johan Corveleyn
On Mon, Aug 30, 2010 at 12:05 PM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: Johan Corveleyn wrote: On Sun, Aug 29, 2010 at 12:32 PM, stefan2_at_apache.org wrote: / Author: stefan2 / / Date: Sun Aug 29 10:32:08 2010 / / New Revision: 990537 / / / / URL: http://svn.apache.org

Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_serializer.c

2010-08-30 Thread Johan Corveleyn
On Mon, Aug 30, 2010 at 9:32 PM, Johan Corveleyn jcor...@gmail.com wrote: On Mon, Aug 30, 2010 at 12:05 PM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: I thought so ;) To narrow down the nature of the problem, I added some checks that should be able to discern plain data corruption

Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_serializer.c

2010-08-31 Thread Johan Corveleyn
On Tue, Aug 31, 2010 at 12:37 AM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: Johan Corveleyn wrote: On Mon, Aug 30, 2010 at 9:32 PM, Johan Corveleyn jcor...@gmail.commailto: jcor...@gmail.com wrote: On Mon, Aug 30, 2010 at 12:05 PM, Stefan Fuhrmann stefanfuhrm...@alice

Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_serializer.c

2010-08-31 Thread Johan Corveleyn
Stefan, On Tue, Aug 31, 2010 at 10:07 PM, Johan Corveleyn jcor...@gmail.com wrote: Some additional info: - I couldn't reproduce the crash with a narrow range. Not even 9:0 would crash it (right after startup). - BUT: if after 9:0 I run log without -r arguments, I get an error

Worried about single-db performance

2010-09-03 Thread Johan Corveleyn
Hi devs, From what I understand about the performance problems of WC-1 vs. WC-NG, and what I'm reading on this list, I expect(ed) a huge performance boost from WC-NG for certain client operations (especially on Windows, where the locking of WC-1 is quite problematic). Also, I knew I had to wait

Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_serializer.c

2010-09-04 Thread Johan Corveleyn
On Tue, Aug 31, 2010 at 11:35 PM, Johan Corveleyn jcor...@gmail.com wrote: Stefan, On Tue, Aug 31, 2010 at 10:07 PM, Johan Corveleyn jcor...@gmail.com wrote: Some additional info: - I couldn't reproduce the crash with a narrow range. Not even 9:0 would crash it (right after startup

Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_serializer.c

2010-09-08 Thread Johan Corveleyn
On Wed, Sep 8, 2010 at 12:41 PM, Johan Corveleyn jcor...@gmail.com wrote: On Wed, Sep 8, 2010 at 12:00 PM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: Johan Corveleyn wrote: On Mon, Sep 6, 2010 at 2:10 AM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: The only way to get

Re: [Issue 3474] making a new subdir, moving files into it and then renaming the subdir, breaks history of the moved files

2010-09-10 Thread Johan Corveleyn
On Fri, Sep 10, 2010 at 11:45 PM, joha...@tigris.org wrote: http://subversion.tigris.org/issues/show_bug.cgi?id=3474 --- Additional comments from joha...@tigris.org Fri Sep 10 14:45:17 -0700 2010 --- This issue seems to be fixed on trunk. The described scenario now goes as

Wrong reply-to address in mails from issue tracker

2010-09-10 Thread Johan Corveleyn
Hi, The mails coming from the SVN issue tracker (on the issues mailing list) still contain the old d...@s.t.o address as reply-to. Could someone change this to the d...@s.a.o address? Cheers, -- Johan

Re: svn diff optimization to make blame faster?

2010-09-15 Thread Johan Corveleyn
On Sun, Sep 5, 2010 at 1:53 AM, Johan Corveleyn jcor...@gmail.com wrote: On Tue, Aug 24, 2010 at 11:11 AM, Philip Martin philip.mar...@wandisco.com wrote: Johan Corveleyn jcor...@gmail.com writes: On Sun, Aug 22, 2010 at 4:02 PM, Branko Čibej br...@xbc.nu wrote: svn_diff uses basically

Re: svn diff optimization to make blame faster?

2010-09-20 Thread Johan Corveleyn
On Mon, Sep 20, 2010 at 11:52 AM, Branko Čibej br...@xbc.nu wrote:  On 15.09.2010 14:20, Johan Corveleyn wrote: Some update on this: I have implemented this for svn_diff (excluding the identical prefix and suffix of both files, and only then starting to fill up the token tree and let the lcs

[WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-09-26 Thread Johan Corveleyn
Hi devs, As discussed in [1], here is a patch that makes svn_diff_diff (libsvn_diff/diff.c) skip the identical prefix and suffix of the original and modified files, before starting the LCS (longest common subsequence) algorithm on the non-matching part. This makes diff a lot faster (especially

Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-09-28 Thread Johan Corveleyn
Hi Daniel, Thanks for the feedback. On Tue, Sep 28, 2010 at 4:11 PM, Daniel Shahaf d...@daniel.shahaf.name wrote: Index: subversion/include/svn_diff.h === --- subversion/include/svn_diff.h     (revision 1001548) +++

Re: Interrupting an update after change of externals causes corrupt working copy

2010-09-30 Thread Johan Corveleyn
Hi devs, As per Daniel Shahaf's suggestion, I'm continuing this discussion on the dev list. See the previous mails in this thread on the users list for some context (or http://svn.haxx.se/users/archive-2010-09/0406.shtml). I'll summarize below. This issue reproduces with 1.6.12 as well as with

trunk failing tests on Windows XP (32 bit): prop-tests.py 33, stat-tests.py 5, upgrade-tests.py 11

2010-09-30 Thread Johan Corveleyn
Hi devs, The following tests fail on my machine (Windows XP 32-bit, built with VCE 2008, ra_local): - prop-tests.py 33: test properties of obstructed subdirectories svn: Can't open directory

Re: trunk failing tests on Windows XP (32 bit): prop-tests.py 33, stat-tests.py 5, upgrade-tests.py 11

2010-10-01 Thread Johan Corveleyn
On Fri, Oct 1, 2010 at 3:45 PM, Paul Burba ptbu...@gmail.com wrote: On Thu, Sep 30, 2010 at 8:10 PM, Bert Huijben b...@qqmail.nl wrote: -Original Message- From: Johan Corveleyn [mailto:jcor...@gmail.com] Sent: vrijdag 1 oktober 2010 1:51 To: Subversion Development Subject: trunk

Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-10-02 Thread Johan Corveleyn
Hi, Here is a second iteration of the patch. It now passes make check. Differences from the previous version are: - Support for \r eol-style (\n and \r\n was already ok). - The number of prefix_lines is now passed to svn_diff__lcs, so it can use that value to set the position offset of the EOF

Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-10-03 Thread Johan Corveleyn
On Sun, Oct 3, 2010 at 1:46 AM, Johan Corveleyn jcor...@gmail.com wrote: Hi, Here is a second iteration of the patch. It now passes make check. Differences from the previous version are: - Support for \r eol-style (\n and \r\n was already ok). - The number of prefix_lines is now passed

[RFC] Diff (blame) optimization: how to go forward

2010-10-07 Thread Johan Corveleyn
Hi all, This is a follow-up to the WIP-patches I posted last week [1], which are about improving performance of svn_diff (and therefor also blame on the client-side), especially for large files. To summarize: the idea was (is) to strip off the identical prefix and suffix, and then letting the

Re: [RFC] caret notation enhancement

2010-10-08 Thread Johan Corveleyn
On Fri, Oct 8, 2010 at 2:35 PM, Karl Heinz Marbaise khmarba...@gmx.de wrote: Hi Tino,     svn cp ^^/trunk ^^/tags/RELEASE-1.0.0 -m- Tagging The usage of the doubled ^ is just as an example, cause i know on Windows you already have to type the doubled ^ because of the shell. So what should

Re: [RFC] Diff (blame) optimization: how to go forward

2010-10-08 Thread Johan Corveleyn
On Fri, Oct 8, 2010 at 3:08 PM, Daniel Shahaf d...@daniel.shahaf.name wrote: Johan Corveleyn wrote on Fri, 8 Oct 2010 at 01:44 -: With suffix-lines-to-keep=50, you'd need to insert a block of text that has its last 50 lines identical to the 50 lines preceding the insertion point, to mess

Re: [RFC] Diff (blame) optimization: how to go forward

2010-10-08 Thread Johan Corveleyn
On Fri, Oct 8, 2010 at 10:35 PM, Hyrum K. Wright hyrum_wri...@mail.utexas.edu wrote: On Fri, Oct 8, 2010 at 3:24 PM, Johan Corveleyn jcor...@gmail.com wrote: ... Just one more thing: as I mentioned in my rather long mail, blame would benefit the most from my optimization if the server were

Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-10-08 Thread Johan Corveleyn
Ok, third iteration of the patch in attachment. It passes make check. As discussed in [1], this version keeps 50 lines of the identical suffix around, to give the algorithm a good chance to generate a diff output of good quality (in all but the most extreme cases, this will be the same as with

Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-10-09 Thread Johan Corveleyn
On Sat, Oct 9, 2010 at 2:57 AM, Julian Foad julian.f...@wandisco.com wrote: On Sat, 2010-10-09, Johan Corveleyn wrote: Ok, third iteration of the patch in attachment. It passes make check. As discussed in [1], this version keeps 50 lines of the identical suffix around, to give the algorithm

Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-10-09 Thread Johan Corveleyn
On Sat, Oct 9, 2010 at 5:19 PM, Daniel Shahaf d...@daniel.shahaf.name wrote: Johan Corveleyn wrote on Sat, Oct 09, 2010 at 14:21:09 +0200: (side-note: I considered first doing suffix scanning, then prefix scanning, so I could reuse the buffers/pointers from diff_baton all the time, and still

Re: trunk failing tests on Windows XP (32 bit): prop-tests.py 33, stat-tests.py 5, upgrade-tests.py 11

2010-10-10 Thread Johan Corveleyn
On Sat, Oct 2, 2010 at 11:24 AM, Bert Huijben b...@qqmail.nl wrote: -Original Message- From: Paul Burba [mailto:ptbu...@gmail.com] Sent: vrijdag 1 oktober 2010 15:46 To: Bert Huijben Cc: Johan Corveleyn; Subversion Development Subject: Re: trunk failing tests on Windows XP (32 bit

Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-10-10 Thread Johan Corveleyn
On Sat, Oct 9, 2010 at 2:21 PM, Johan Corveleyn jcor...@gmail.com wrote: On Sat, Oct 9, 2010 at 2:57 AM, Julian Foad julian.f...@wandisco.com wrote: But this makes me think, it looks to me like this whole prefix-suffix-skipping functionality would fit better inside the lower-level diff

Re: trunk failing tests on Windows XP (32 bit): prop-tests.py 33, stat-tests.py 5, upgrade-tests.py 11

2010-10-15 Thread Johan Corveleyn
On Sun, Oct 10, 2010 at 3:07 PM, Johan Corveleyn jcor...@gmail.com wrote: On Sat, Oct 2, 2010 at 11:24 AM, Bert Huijben b...@qqmail.nl wrote: -Original Message- From: Paul Burba [mailto:ptbu...@gmail.com] Sent: vrijdag 1 oktober 2010 15:46 To: Bert Huijben Cc: Johan Corveleyn

Re: trunk failing tests on Windows XP (32 bit): prop-tests.py 33, stat-tests.py 5, upgrade-tests.py 11

2010-10-15 Thread Johan Corveleyn
On Fri, Oct 15, 2010 at 12:20 PM, Johan Corveleyn jcor...@gmail.com wrote: I saw that the problem was fixed in r102170 by Bert s/r102170/r1021760/ I really should install that Undo send google labs extension :) -- Johan

Re: svn commit: r1022931 - in /subversion/trunk/subversion/libsvn_wc: status.c wc-queries.sql wc_db.c wc_db.h

2010-10-15 Thread Johan Corveleyn
On Fri, Oct 15, 2010 at 4:19 PM, phi...@apache.org wrote: Author: philip Date: Fri Oct 15 14:19:36 2010 New Revision: 1022931 URL: http://svn.apache.org/viewvc?rev=1022931view=rev Log: Implement status using per-dir queries.  On my machine (Linux, local disk) this improves the speed of

Re: massive memory leak

2010-10-19 Thread Johan Corveleyn
On Tue, Oct 19, 2010 at 8:14 PM, Stefan Küng tortoise...@gmail.com wrote: On 19.10.2010 20:13, Lieven Govaerts wrote: This has been fixed in serf trunk r1408 for a while, but hasn't shown up in a serf patch release yet. Sorry, I should have checked the serf commits first. Thanks for the

diff4: is it actually used?

2010-10-19 Thread Johan Corveleyn
Hi devs, In the context of the diff optimization patch I'm working on ([1]), I'm wondering if diff4 is actually used in svn. If I look for usages of subversion/libsvn_diff/diff4.c#svn_diff_diff4, I only come up with tools/diff/diff4.c#main. So: this code isn't used in the svn core itself? What's

Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster

2010-10-19 Thread Johan Corveleyn
On Tue, Oct 12, 2010 at 12:10 PM, Julian Foad julian.f...@wandisco.com wrote: On Tue, 2010-10-12 at 00:31 +0200, Johan Corveleyn wrote: On Mon, Oct 11, 2010 at 11:53 AM, Julian Foad julian.f...@wandisco.com wrote: On Sat, 2010-10-09, Johan Corveleyn wrote: On Sat, Oct 9, 2010 at 2:57 AM

Diff optimization: implement prefix/suffix-skipping in token-handling code (was: Re: [WIP PATCH] Make svn_diff_diff skip identical prefix and suffix to make diff and blame faster)

2010-10-19 Thread Johan Corveleyn
On Tue, Oct 12, 2010 at 12:35 PM, Julian Foad julian.f...@wandisco.com wrote: On Sun, 2010-10-10 at 23:43 +0200, Johan Corveleyn wrote: On Sat, Oct 9, 2010 at 2:21 PM, Johan Corveleyn jcor...@gmail.com wrote: On Sat, Oct 9, 2010 at 2:57 AM, Julian Foad julian.f...@wandisco.com wrote

Issue triage: issues 3429 and 3474

2010-11-08 Thread Johan Corveleyn
Hi, I recently verified that the following two issues are fixed on trunk: - #3429: svn mv A B; svn mv B A generates replace without history - #3474: making a new subdir, moving files into it and then renaming the subdir, breaks history of the moved files Should I mark these as fixed in the

Re: Interrupting an update after change of externals causes corrupt working copy

2010-11-15 Thread Johan Corveleyn
Almost forgot about this. It's now filed in the issue tracker: http://subversion.tigris.org/issues/show_bug.cgi?id=3751 Cheers, Johan On Thu, Sep 30, 2010 at 10:03 PM, Johan Corveleyn jcor...@gmail.com wrote: Hi devs, As per Daniel Shahaf's suggestion, I'm continuing this discussion

Re: Issue triage: issues 3429 and 3474

2010-11-16 Thread Johan Corveleyn
On Mon, Nov 8, 2010 at 1:02 PM, Philip Martin philip.mar...@wandisco.com wrote: Johan Corveleyn jcor...@gmail.com writes: Or, maybe the best approach: I could add a regression test for these issues, so we can all be sure that they are fixed (and remain fixed), after which they can be marked

[PATCH] copy_tests.py - expand move_file_back_and_forth, to verify issue #3429

2010-11-16 Thread Johan Corveleyn
The attached patch expands the test move_file_back_and_forth (copy_tests.py 45) as a regression test for issue#3429. [[[ Expand move_file_back_and_forth test to verify issue #3429 (svn mv A B; svn mv B A generates replace without history). * subversion/tests/cmdline/copy_tests.py

Re: [PATCH] copy_tests.py - expand move_file_back_and_forth, to verify issue #3429

2010-11-16 Thread Johan Corveleyn
On Wed, Nov 17, 2010 at 2:14 AM, Johan Corveleyn jcor...@gmail.com wrote: The attached patch expands the test move_file_back_and_forth (copy_tests.py 45) as a regression test for issue#3429. [[[ Expand move_file_back_and_forth test to verify issue #3429 (svn mv A B; svn mv B A generates

Re: Issue triage: issues 3429 and 3474

2010-11-17 Thread Johan Corveleyn
On Wed, Nov 17, 2010 at 7:14 PM, Daniel Shahaf d...@daniel.shahaf.name wrote: Johan Corveleyn wrote on Wed, Nov 17, 2010 at 01:25:24 +0100: On Mon, Nov 8, 2010 at 1:02 PM, Philip Martin philip.mar...@wandisco.com wrote: Johan Corveleyn jcor...@gmail.com writes: Or, maybe the best approach

Diff optimizations progress (or not)

2010-11-20 Thread Johan Corveleyn
[ adding dev@ to discussion between danielsh and me about (non-)progress of the diff-optimizations work. ] On Wed, Nov 17, 2010 at 3:21 AM, Daniel Shahaf d...@daniel.shahaf.name wrote: [ if you want to, you can add dev@ to the CC on replies. ] Johan Corveleyn wrote on Wed, Nov 17, 2010 at 00

Re: AW: How to find out the rev number where a file was deleted?

2010-11-29 Thread Johan Corveleyn
. But the discussion continued about why this is so hard right now, and if there are alternative approaches. See below... On Mon, Nov 29, 2010 at 3:51 AM, Daniel Shahaf d...@daniel.shahaf.name wrote: Johan Corveleyn wrote on Sun, Nov 28, 2010 at 21:20:28 +0100: On Sun, Nov 28, 2010 at 6:35 PM, Daniel Shahaf d

diff-optimizations-tokens branch: I think I'm going to abandon it

2010-11-30 Thread Johan Corveleyn
Hi devs, As mentioned in [1], I've created two branches to try out two different approaches for the diff optimizations of prefix/suffix scanning. The first one, diff-optimizations-bytes, has a working implementation of the optimization. It still has some open todo items, but it basically works.

Re: diff-optimizations-tokens branch: I think I'm going to abandon it

2010-12-01 Thread Johan Corveleyn
On Wed, Dec 1, 2010 at 3:38 AM, Daniel Shahaf d...@daniel.shahaf.name wrote: Johan Corveleyn wrote on Wed, Dec 01, 2010 at 00:25:27 +0100: I am now considering to abandon the tokens-approach, for the following reasons: ... So, unless someone can convince me otherwise, I'm probably going

Re: diff-optimizations-tokens branch: I think I'm going to abandon it

2010-12-01 Thread Johan Corveleyn
On Wed, Dec 1, 2010 at 11:44 AM, Daniel Shahaf d...@daniel.shahaf.name wrote: Johan Corveleyn wrote on Wed, Dec 01, 2010 at 10:05:29 +0100: On Wed, Dec 1, 2010 at 3:38 AM, Daniel Shahaf d...@daniel.shahaf.name wrote: Johan Corveleyn wrote on Wed, Dec 01, 2010 at 00:25:27 +0100: I am now

Re: diff-optimizations-tokens branch: I think I'm going to abandon it

2010-12-02 Thread Johan Corveleyn
On Thu, Dec 2, 2010 at 6:43 AM, Daniel Shahaf d...@daniel.shahaf.name wrote: [ finally getting back to this mail; having slept on it, etc. ] Johan Corveleyn wrote on Wed, Dec 01, 2010 at 13:34:48 +0100: On Wed, Dec 1, 2010 at 11:44 AM, Daniel Shahaf d...@daniel.shahaf.name wrote: Johan

Re: diff-optimizations-tokens branch: I think I'm going to abandon it

2010-12-02 Thread Johan Corveleyn
On Thu, Dec 2, 2010 at 4:21 PM, Julian Foad julian.f...@wandisco.com wrote: Hi Johan. I've just read the whole of this thread. I didn't quite understand your original point (2) that token-based suffix scanning will not be as fast as byte-based suffix scanning. Sure it won't, but is there

Re: diff-optimizations-tokens branch: I think I'm going to abandon it

2010-12-02 Thread Johan Corveleyn
On Thu, Dec 2, 2010 at 6:18 PM, Bill Tutt b...@tutts.org wrote: Note: This email only tangentially relates to svn diff and more about reverse token scanning in general: As someone who has implemented suffix reverse token scanning before: Thanks for the input. It's nice to see other people

Implementations of svn_diff_fns_t?

2010-12-02 Thread Johan Corveleyn
Hi, This question came up during recent discussion about the diff-optimizations-tokens branch [1]: What are the known implementors of svn_diff_fns_t, the vtable of svn_diff callback functions in subversion/include/svn_diff.h? Besides the internal diff_memory.c and diff_file.c that is. Are there

Re: [PATCH] copy_tests.py - expand move_file_back_and_forth, to verify issue #3429

2010-12-08 Thread Johan Corveleyn
On Wed, Dec 8, 2010 at 3:17 PM, Julian Foad julian.f...@wandisco.com wrote: On Wed, 2010-11-17, Johan Corveleyn wrote: On Wed, Nov 17, 2010 at 2:14 AM, Johan Corveleyn jcor...@gmail.com wrote: The attached patch expands the test move_file_back_and_forth (copy_tests.py 45) as a regression

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-15 Thread Johan Corveleyn
On Wed, Dec 15, 2010 at 10:30 AM, Branko Čibej br...@xbc.nu wrote: On 15.12.2010 02:30, Stefan Fuhrmann wrote: On 14.12.2010 23:35, Johan Corveleyn wrote: Some considerations: - Maybe I can use APR_INLINE, with similar results? - Maybe I can put just the critical section into a macro

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-15 Thread Johan Corveleyn
On Wed, Dec 15, 2010 at 2:30 AM, Stefan Fuhrmann eq...@web.de wrote: On 14.12.2010 23:35, Johan Corveleyn wrote: Hi all, Hi Johan ;) Hi Stefan, thanks for the input :). I suspected that you might have some ideas about this ... On the diff-optimizations-bytes branch, in diff_file.c

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-15 Thread Johan Corveleyn
On Wed, Dec 15, 2010 at 11:53 AM, Johan Corveleyn jcor...@gmail.com wrote: On Wed, Dec 15, 2010 at 11:50 AM, Johan Corveleyn jcor...@gmail.com wrote: Also, I did my measurements with a blame operation on this very large file, which has ~2500 revisions. So that's 2500 diffs of a ~1,5 Mb file

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-20 Thread Johan Corveleyn
On Mon, Dec 20, 2010 at 11:03 AM, Julian Foad julian.f...@wandisco.com wrote: On Mon, 2010-12-20, Johan Corveleyn wrote: New macro version (increment only, decrement is similar): [[[ /* For all files in the FILE array, increment the curp pointer.  If a file  * points before the beginning

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-20 Thread Johan Corveleyn
On Mon, Dec 20, 2010 at 2:16 PM, Johan Corveleyn jcor...@gmail.com wrote: On Mon, Dec 20, 2010 at 11:03 AM, Julian Foad julian.f...@wandisco.com wrote: On Mon, 2010-12-20, Johan Corveleyn wrote: New macro version (increment only, decrement is similar): [[[ /* For all files in the FILE array

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-28 Thread Johan Corveleyn
On Fri, Dec 24, 2010 at 3:40 PM, Stefan Fuhrmann eq...@web.de wrote: On 20.12.2010 02:43, Johan Corveleyn wrote: On Wed, Dec 15, 2010 at 10:58 AM, Stefan Fuhrmanneq...@web.de  wrote: On 15.12.2010 02:30, Stefan Fuhrmann wrote: On 14.12.2010 23:35, Johan Corveleyn wrote: Thoughts? Two

Re: FSFS format 6 (was: Re: FSv2)

2010-12-28 Thread Johan Corveleyn
On Sun, Dec 12, 2010 at 4:23 PM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: On 19.10.2010 15:10, Daniel Shahaf wrote: Greg Stein wrote on Tue, Oct 19, 2010 at 04:31:42 -0400: Personally, I see [FSv2] as a broad swath of API changes to align our needs with the underlying storage.

Re: [RFC] diff-optimizations-bytes branch: avoiding function call overhead (?)

2010-12-30 Thread Johan Corveleyn
On Tue, Dec 28, 2010 at 7:31 PM, Johan Corveleyn jcor...@gmail.com wrote: On Fri, Dec 24, 2010 at 3:40 PM, Stefan Fuhrmann eq...@web.de wrote: On 20.12.2010 02:43, Johan Corveleyn wrote: On Wed, Dec 15, 2010 at 10:58 AM, Stefan Fuhrmanneq...@web.de  wrote: On 15.12.2010 02:30, Stefan

diff-optimizations-bytes: how to make diff3 work with prefix/suffix scanning

2011-01-01 Thread Johan Corveleyn
[ Taking a privately-started discussion with danielsh to the list, in case others have inspiration/insight about this. Question at hand: I'm having trouble making diff3 work with prefix/suffix scanning of the diff-optimizations-bytes branch. Any feedback is highly appreciated :-). ] On Fri, Dec

Re: svn commit: r1054286 - /subversion/trunk/subversion/libsvn_delta/xdelta.c

2011-01-02 Thread Johan Corveleyn
On Sat, Jan 1, 2011 at 9:33 PM, stef...@apache.org wrote: Author: stefan2 Date: Sat Jan  1 20:33:22 2011 New Revision: 1054286 URL: http://svn.apache.org/viewvc?rev=1054286view=rev Log: XDelta calculation is major part of svn_txdelta_send_txstream. Therefore, speed up string matching code

Re: svn commit: r1054286 - /subversion/trunk/subversion/libsvn_delta/xdelta.c

2011-01-02 Thread Johan Corveleyn
On Sun, Jan 2, 2011 at 9:20 AM, Johan Corveleyn jcor...@gmail.com wrote: On Sat, Jan 1, 2011 at 9:33 PM,  stef...@apache.org wrote: Author: stefan2 Date: Sat Jan  1 20:33:22 2011 New Revision: 1054286 URL: http://svn.apache.org/viewvc?rev=1054286view=rev Log: XDelta calculation is major

Re: svn commit: r1054286 - /subversion/trunk/subversion/libsvn_delta/xdelta.c

2011-01-02 Thread Johan Corveleyn
On Sat, Jan 1, 2011 at 9:33 PM, stef...@apache.org wrote: Author: stefan2 Date: Sat Jan  1 20:33:22 2011 New Revision: 1054286 URL: http://svn.apache.org/viewvc?rev=1054286view=rev Log: XDelta calculation is major part of svn_txdelta_send_txstream. Therefore, speed up string matching code

Re: svn commit: r1054286 - /subversion/trunk/subversion/libsvn_delta/xdelta.c

2011-01-02 Thread Johan Corveleyn
On Sun, Jan 2, 2011 at 4:45 PM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: On 02.01.2011 16:38, Johan Corveleyn wrote: On Sun, Jan 2, 2011 at 9:20 AM, Johan Corveleynjcor...@gmail.com  wrote: On Sat, Jan 1, 2011 at 9:33 PM,stef...@apache.org  wrote: Author: stefan2 Date: Sat Jan  1

Re: -bytes branch reports bogus conflict that trunk doesn't

2011-01-02 Thread Johan Corveleyn
On Sun, Jan 2, 2011 at 7:36 PM, Daniel Shahaf danie...@apache.org wrote: Johan, I saw your sync merge, so I ran 'svn up' with intent to rebuild.  When I ran that 'up' with the branch build, I got a conflict, which didn't appear when I ran it with a trunk build.  (See attached transcript.)

Re: -bytes branch reports bogus conflict that trunk doesn't

2011-01-02 Thread Johan Corveleyn
On Sun, Jan 2, 2011 at 8:19 PM, Daniel Shahaf danie...@apache.org wrote: Johan Corveleyn wrote on Sun, Jan 02, 2011 at 20:13:59 +0100: On Sun, Jan 2, 2011 at 7:36 PM, Daniel Shahaf danie...@apache.org wrote: Johan, I saw your sync merge, so I ran 'svn up' with intent to rebuild.  When I

Re: My take on the diff-optimizations-bytes branch

2011-01-02 Thread Johan Corveleyn
On Sat, Jan 1, 2011 at 11:25 PM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: Hi Johan, Thursday night I did something stupid and had a look at  how svn blame could be made faster based on the HEAD code in your branch. One night and most of the following day later, I think I made it

Re: My take on the diff-optimizations-bytes branch

2011-01-02 Thread Johan Corveleyn
On Sun, Jan 2, 2011 at 10:04 PM, Johan Corveleyn jcor...@gmail.com wrote: On Sat, Jan 1, 2011 at 11:25 PM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: Hi Johan, Thursday night I did something stupid and had a look at  how svn blame could be made faster based on the HEAD code in your

Re: My take on the diff-optimizations-bytes branch

2011-01-02 Thread Johan Corveleyn
On Mon, Jan 3, 2011 at 12:13 AM, Johan Corveleyn jcor...@gmail.com wrote: On Sun, Jan 2, 2011 at 10:04 PM, Johan Corveleyn jcor...@gmail.com wrote: On Sat, Jan 1, 2011 at 11:25 PM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: Hi Johan, Thursday night I did something stupid and had

Re: My take on the diff-optimizations-bytes branch

2011-01-03 Thread Johan Corveleyn
On Mon, Jan 3, 2011 at 12:03 PM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: On 03.01.2011 02:14, Johan Corveleyn wrote: On Mon, Jan 3, 2011 at 12:13 AM, Johan Corveleynjcor...@gmail.com  wrote: [... snip ...] And it's fast too! It's taking only 58 seconds in diff, vs. 72

Re: diff-optimizations-bytes: how to make diff3 work with prefix/suffix scanning

2011-01-05 Thread Johan Corveleyn
On Wed, Jan 5, 2011 at 12:11 PM, Julian Foad julian.f...@wandisco.com wrote: On Sat, 2011-01-01, Johan Corveleyn wrote: [ Taking a privately-started discussion with danielsh to the list, in case others have inspiration/insight about this. Question at hand: I'm having trouble making diff3 work

Re: svn commit: r1054701 - in /subversion/trunk/subversion/bindings/javahl/native: CopySources.cpp CreateJ.cpp EnumMapper.cpp ListCallback.cpp Revision.cpp RevisionRange.cpp StatusCallback.cpp org_apa

2011-01-10 Thread Johan Corveleyn
On Mon, Jan 3, 2011 at 7:34 PM, hwri...@apache.org wrote: Author: hwright Date: Mon Jan  3 18:34:35 2011 New Revision: 1054701 URL: http://svn.apache.org/viewvc?rev=1054701view=rev Log: Fix JavaHL build and test failures introduced in r1054680. *

Re: Diff optimizations and generating big test files

2011-01-16 Thread Johan Corveleyn
On Wed, Jan 5, 2011 at 4:33 PM, Philip Martin philip.mar...@wandisco.com wrote: Johan Corveleyn jcor...@gmail.com writes: Thanks for the script, it gives me some good inspiration. However, it doesn't fit well with the optimization that's currently being done on the diff-optimizations-bytes

Re: My take on the diff-optimizations-bytes branch

2011-01-16 Thread Johan Corveleyn
On Sat, Jan 8, 2011 at 6:50 AM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: On 03.01.2011 02:14, Johan Corveleyn wrote: On Mon, Jan 3, 2011 at 12:13 AM, Johan Corveleynjcor...@gmail.com  wrote: On Sun, Jan 2, 2011 at 10:04 PM, Johan Corveleynjcor...@gmail.com  wrote: For now, some

Re: Coding goals / requirements.

2011-01-18 Thread Johan Corveleyn
Hi, As the guy responsible for the quote that started this thread ([1]): Actually, what's a little bit troubling is that there are currently only 3 possible file_len's, of which only 2 are used in practice: diff2 and diff3 (diff4 is not used in svn core, only in tools/diff/diff4). So, if we

Re: My take on the diff-optimizations-bytes branch

2011-01-18 Thread Johan Corveleyn
On Mon, Jan 17, 2011 at 3:12 AM, Johan Corveleyn jcor...@gmail.com wrote: On Sat, Jan 8, 2011 at 6:50 AM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: [ ... snip ... ] But I think, the stack variable is certainly helpful and easy to do. Ok, I've done this (locally, still have to clean

Re: My take on the diff-optimizations-bytes branch

2011-01-22 Thread Johan Corveleyn
On Wed, Jan 19, 2011 at 1:19 AM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: On 18.01.2011 12:56, Johan Corveleyn wrote: On Mon, Jan 17, 2011 at 3:12 AM, Johan Corveleynjcor...@gmail.com  wrote: On Sat, Jan 8, 2011 at 6:50 AM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de  wrote

Re: My take on the diff-optimizations-bytes branch

2011-01-23 Thread Johan Corveleyn
On Sat, Jan 8, 2011 at 6:50 AM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: On 03.01.2011 02:14, Johan Corveleyn wrote: It would be interesting to see where the biggest gains are coming from (I'm guessing from the per-machine-word reading/comparing; I'd like to try that first, maybe

Re: My take on the diff-optimizations-bytes branch

2011-01-23 Thread Johan Corveleyn
On Sun, Jan 23, 2011 at 10:46 PM, Johan Corveleyn jcor...@gmail.com wrote: On Sat, Jan 8, 2011 at 6:50 AM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de wrote: On 03.01.2011 02:14, Johan Corveleyn wrote: It would be interesting to see where the biggest gains are coming from (I'm guessing from

Assertion failure during update_tests.py 58 (XFAIL: update a nonexistent child of a copied dir)

2011-01-23 Thread Johan Corveleyn
Hi, Already for some time now, update_tests.py 58 (XFAIL: update a nonexistent child of a copied dir) crashes on my machine: svn: In file '..\..\..\subversion\libsvn_wc\update_editor.c' line 4877: assertion failed (repos_root != NULL repos_uuid != NULL) I understand that this test is

Re: FSFS format 6

2011-01-23 Thread Johan Corveleyn
On Wed, Dec 29, 2010 at 8:37 PM, Stefan Fuhrmann eq...@web.de wrote: On 29.12.2010 01:58, Johan Corveleyn wrote: On Sun, Dec 12, 2010 at 4:23 PM, Stefan Fuhrmann stefanfuhrm...@alice-dsl.de  wrote: On 19.10.2010 15:10, Daniel Shahaf wrote: Greg Stein wrote on Tue, Oct 19, 2010 at 04:31:42

Re: My take on the diff-optimizations-bytes branch

2011-01-25 Thread Johan Corveleyn
On Tue, Jan 25, 2011 at 1:37 AM, Stefan Fuhrmann eq...@web.de wrote: [ ... snip ...] And, as promised, here some ideas how to get more speed from the generic code. Your latest commit: +#if SVN_UNALIGNED_ACCESS_IS_OK + +      /* Skip quickly over the stuff between EOLs. */ +      for (i = 0,

Re: Status of the branch diff-optimizations-bytes branch

2011-01-25 Thread Johan Corveleyn
On Tue, Jan 25, 2011 at 4:58 PM, Hyrum K Wright hy...@hyrumwright.org wrote: Johan (and other interested parties), I've been following some of the commits to the diff-optimizations-branch with interest.  While I've not reviewed them for technical merit, it appears that others have, and that

  1   2   3   4   5   6   7   8   9   10   >