Re: [fossil-users] git incremental import speedup.

2016-10-26 Thread Karel Gardas 
On Wed, Oct 26, 2016 at 11:53 PM, Adam Jensen  wrote:
> On 10/26/2016 05:37 PM, Karel Gardas wrote:
>> I'm now able to import OpenBSD
>> source tree from OpenBSD src git mirror to fossil.
>
> This might be a silly question since I am terribly uniformed of the
> issues but could you have imported the OpenBSD source directly from
> their CVS repository?
>
> http://www.fossil-scm.org/index.html/wiki?name=Import+CVS+Repositories
> http://www.openbsd.org/anoncvs.html

For this you would need to have working cvs -> svn transition tool but
the problem is that cvs2svn was buggy on OpenBSD src. I've tested that
in the past and reported even a proper bugreport but I'm not able to
find it now...
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] git incremental import speedup.

2016-10-26 Thread Karel Gardas 
On Wed, Oct 26, 2016 at 11:44 PM, jungle Boogie  wrote:
> On 26 October 2016 at 14:37, Karel Gardas  wrote:
>> Anyway, there is small nitpick. While using incremental import on such
>> repo, fossil is horribly slow. The pstack command reveals that
>> majority of time is spent in import_cmd -> export_marks ->
>> mark_name_from_rid call chain. I've solved this by patch below which
>> basically just adds index on xmark's trid field. Speedup is from 50-60
>> minutes (fossil head) to 30-40 seconds (fossil head + patch) so please
>> consider for inclusion if possible -- and if it is correct of course.
>> I think the same change may also be added to export.c but I'm not able
>> to test it now as I'm not using export for two-way sync yet.
>
>
> You're saying openbsd import from git to fossil with fossil head is 60
> minutes and with your patch and with the same repo on the same
> machine, it's not 60 seconds??

No, I'm not talking about whole import but about incremental import.
If you like to know all numbers then

1) import OpenBSD src git -> fossil from 1995 till let say 2016-10-24
took 42 hours when run with --no-rebuild option

2) *incremental* import of 2016-10-25 changes (few patches) takes 56
minutes on fossil head and if I do the same with the patch I takes 34
seconds.

Honestly (2) is still not correct since patches into git are pushed
constantly so I do not test exactly the same situation. I just seen
few patches added into git so I tested another run, OK?
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] git incremental import speedup.

2016-10-26 Thread Adam Jensen 
On 10/26/2016 05:37 PM, Karel Gardas wrote:
> I'm now able to import OpenBSD
> source tree from OpenBSD src git mirror to fossil.

This might be a silly question since I am terribly uniformed of the
issues but could you have imported the OpenBSD source directly from
their CVS repository?

http://www.fossil-scm.org/index.html/wiki?name=Import+CVS+Repositories
http://www.openbsd.org/anoncvs.html
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] git incremental import speedup.

2016-10-26 Thread jungle Boogie 
On 26 October 2016 at 14:37, Karel Gardas  wrote:
> Anyway, there is small nitpick. While using incremental import on such
> repo, fossil is horribly slow. The pstack command reveals that
> majority of time is spent in import_cmd -> export_marks ->
> mark_name_from_rid call chain. I've solved this by patch below which
> basically just adds index on xmark's trid field. Speedup is from 50-60
> minutes (fossil head) to 30-40 seconds (fossil head + patch) so please
> consider for inclusion if possible -- and if it is correct of course.
> I think the same change may also be added to export.c but I'm not able
> to test it now as I'm not using export for two-way sync yet.


You're saying openbsd import from git to fossil with fossil head is 60
minutes and with your patch and with the same repo on the same
machine, it's not 60 seconds??



-- 
---
inum: 883510009027723
sip: jungleboo...@sip2sip.info
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users