Re: [fossil-users] git incremental import speedup.
On Wed, Oct 26, 2016 at 11:53 PM, Adam Jensenwrote: > On 10/26/2016 05:37 PM, Karel Gardas wrote: >> I'm now able to import OpenBSD >> source tree from OpenBSD src git mirror to fossil. > > This might be a silly question since I am terribly uniformed of the > issues but could you have imported the OpenBSD source directly from > their CVS repository? > > http://www.fossil-scm.org/index.html/wiki?name=Import+CVS+Repositories > http://www.openbsd.org/anoncvs.html For this you would need to have working cvs -> svn transition tool but the problem is that cvs2svn was buggy on OpenBSD src. I've tested that in the past and reported even a proper bugreport but I'm not able to find it now... ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] git incremental import speedup.
On Wed, Oct 26, 2016 at 11:44 PM, jungle Boogiewrote: > On 26 October 2016 at 14:37, Karel Gardas wrote: >> Anyway, there is small nitpick. While using incremental import on such >> repo, fossil is horribly slow. The pstack command reveals that >> majority of time is spent in import_cmd -> export_marks -> >> mark_name_from_rid call chain. I've solved this by patch below which >> basically just adds index on xmark's trid field. Speedup is from 50-60 >> minutes (fossil head) to 30-40 seconds (fossil head + patch) so please >> consider for inclusion if possible -- and if it is correct of course. >> I think the same change may also be added to export.c but I'm not able >> to test it now as I'm not using export for two-way sync yet. > > > You're saying openbsd import from git to fossil with fossil head is 60 > minutes and with your patch and with the same repo on the same > machine, it's not 60 seconds?? No, I'm not talking about whole import but about incremental import. If you like to know all numbers then 1) import OpenBSD src git -> fossil from 1995 till let say 2016-10-24 took 42 hours when run with --no-rebuild option 2) *incremental* import of 2016-10-25 changes (few patches) takes 56 minutes on fossil head and if I do the same with the patch I takes 34 seconds. Honestly (2) is still not correct since patches into git are pushed constantly so I do not test exactly the same situation. I just seen few patches added into git so I tested another run, OK? ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] git incremental import speedup.
On 10/26/2016 05:37 PM, Karel Gardas wrote: > I'm now able to import OpenBSD > source tree from OpenBSD src git mirror to fossil. This might be a silly question since I am terribly uniformed of the issues but could you have imported the OpenBSD source directly from their CVS repository? http://www.fossil-scm.org/index.html/wiki?name=Import+CVS+Repositories http://www.openbsd.org/anoncvs.html ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] git incremental import speedup.
On 26 October 2016 at 14:37, Karel Gardaswrote: > Anyway, there is small nitpick. While using incremental import on such > repo, fossil is horribly slow. The pstack command reveals that > majority of time is spent in import_cmd -> export_marks -> > mark_name_from_rid call chain. I've solved this by patch below which > basically just adds index on xmark's trid field. Speedup is from 50-60 > minutes (fossil head) to 30-40 seconds (fossil head + patch) so please > consider for inclusion if possible -- and if it is correct of course. > I think the same change may also be added to export.c but I'm not able > to test it now as I'm not using export for two-way sync yet. You're saying openbsd import from git to fossil with fossil head is 60 minutes and with your patch and with the same repo on the same machine, it's not 60 seconds?? -- --- inum: 883510009027723 sip: jungleboo...@sip2sip.info ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users