On Fri, 8 Mar 2013 09:09:56 -0500 Martin Gagnon <[email protected]> wrote:
> Hi, > > I found a git repository of the full openbsd src tree here: > http://anoncvs.estpak.ee/cgi-bin/cgit/openbsd-src/ > > I think this repository is good to test scalability of fossil, since it's > contain not only the kernel source of openbsd, but also all the userland of > the base system. > > I know someone recently test with the NetBSD port tree, but port tree is a > bit less realistic since it contain a incredible huge number of small files > with an incredible number of commits. > > I've clone the repo and start an fossil import of it.. I Start the import 2 > days ago and now it is at the final step, rebuilding of meta-data is > finished and it's doing Vacuuming since a while.. > > Here some statistics: > - Gzipped git fast-export file: > 2.58 GB > - Time for the import (before rebuilding meta-data): > 22 hours > - rebuilding meta-data + Vacuuming: > > 18 hours > - Fossil memory usage during import: > Always increasing, up to: approx: 730MB during vacuuming. > > Actually, right now during vacuuming, CPU usage of the fossil process is > about 3% and there's no progress printout during vacuuming, so I don't know > if it will finish soon... > > I'll give more news when I'll try to use the repository under fossil > compared with git for the speed. I'm doing the same experiment with it now. I clone the openbsd repository and fast export to a file. It's 12 GB size and wc -l shows 430.083.609 lines. I have modified fossil to know in what line is it working. I have no a diff but its trivial, edit import.c and add the lines marked with + in function git_fast_import: static void git_fast_import(FILE *pIn){ ImportFile *pFile, *pNew; int i, mx; char *z; char *zUuid; char *zName; char *zPerm; char *zFrom; char *zTo; char zLine[1000]; + sqlite3_int64 zLineNumber; gg.xFinish = finish_noop; + zLineNumber = 0; while( fgets(zLine, sizeof(zLine), pIn) ){ + zLineNumber++; + if (zLineNumber%1000==0){ + fossil_print(" Processing Line Number: %d \r", zLineNumber); + } Perhaps another change can show the number of lines per minute. I will try some tweaks and see which ones makes the import faster. > > -- > Martin G. --- --- Eduardo Morras <[email protected]> _______________________________________________ fossil-users mailing list [email protected] http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
