Dan Price wrote: > On Tue 09 Dec 2008 at 02:33PM, Danek Duvall wrote: >> I discovered that when importing a 7z'ed package, we're reading data from >> the 7z pipe one character at a time, which slows things down a bit. I >> filed >> >> 5724 seriously suboptimal bufsize in cpiofile.py >> >> and have fixed this globally with >> >> http://cr.opensolaris.org/~dduvall/pkg-bufsize/ >> >> by making the default buffer size 128k for pkgsubprocess, and confirmed >> that it does speed up an import (just slurping the data out of the packages >> and writing directly to a filesystem takes 11 minutes on my system now, >> where I'd stopped it after 20+ minutes before, when it seemed maybe a third >> to a half of the way through). > > Nice find. Fix looks fine-- I presume we're not committing the same sin > anywhere else in our code, correct?
There are two other places where experiments with bufsize might prove fruitful: * filelist_0 in repository.py -- tarfile defaults to a bufsize of 10,240 bytes. * bufsz in add_content in server/transaction.py; but johansen has already set this to 128K. Cheers, -- Shawn Walker _______________________________________________ pkg-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
