On Wednesday 22 October 2008 23:49, NextGen$ wrote: > * Matthew Toseland <[EMAIL PROTECTED]> [2008-10-22 20:48:24]: > > > On Wednesday 22 October 2008 01:09, NextGen$ wrote: > > > * Matthew Toseland <[EMAIL PROTECTED]> [2008-10-21 20:53:51]: > > > > > > > On Tuesday 21 October 2008 16:24, [EMAIL PROTECTED] wrote: > > > > > Author: nextgens > > > > > Date: 2008-10-21 15:24:47 +0000 (Tue, 21 Oct 2008) > > > > > New Revision: 23014 > > > > > > > > > > Modified: > > > > > trunk/freenet/src/freenet/client/ArchiveManager.java > > > > > trunk/freenet/src/freenet/client/ArchiveStoreContext.java > > > > > trunk/freenet/src/freenet/client/ClientMetadata.java > > > > > trunk/freenet/src/freenet/client/HighLevelSimpleClientImpl.java > > > > > trunk/freenet/src/freenet/client/Metadata.java > > > > > trunk/freenet/src/freenet/client/async/ClientPutter.java > > > > > trunk/freenet/src/freenet/client/async/SimpleManifestPutter.java > > > > > trunk/freenet/src/freenet/client/async/SingleFileFetcher.java > > > > > trunk/freenet/src/freenet/client/async/SingleFileInserter.java > > > > > trunk/freenet/src/freenet/client/async/SplitFileInserter.java > > > > > trunk/freenet/src/freenet/clients/http/WelcomeToadlet.java > > > > > trunk/freenet/src/freenet/frost/message/FrostMessage.java > > > > > trunk/freenet/src/freenet/node/NodeARKInserter.java > > > > > trunk/freenet/src/freenet/node/TextModeClientInterface.java > > > > > trunk/freenet/src/freenet/node/fcp/ClientPut.java > > > > > trunk/freenet/src/freenet/node/fcp/DirPutFile.java > > > > > trunk/freenet/src/freenet/node/simulator/BootstrapPushPullTest.java > > > > > Log: > > > > > more work on bug #71: *** IT NEEDS TESTING! *** > > > > > It's still not backward compatible with stable but should be > > > > forward-compatible ;) > > > > > > > > > [...] see r23023 > > > > > > > Do we attempt to compress all files with bzip2 as well as gzip now? > > Shouldn't > > > > there be a max size configuration above which we don't try bzip2, perhaps > > > > unless asked to via FCP? bzip2'ing ISOs could take a really long time ... > > > > > > I don't think we need one. Big files will take long to compress but will > > take > > > long to insert too. I think it's worth spending a few more CPU cycles to > > > spare the insertion of a few blocks (plus their FEC blocks). > > > > I'm not convinced that this is acceptable from a usability point of view. > > Maybe we can provide a progress bar within the compression phase? On the new > > UI it is proposed to separate downloads which are not yet finalised (i.e. > > haven't fetched the last lot of metadata) from downloads that are... we could > > do something similar with inserts in compression. > > > > Have a look to what I have commited. From now on the compression is fully > serialized... We have one mutex, and only one compression job (just like we > do for FEC encoding in fact) which means a even higher latency.
For FEC we can run up to min(3, # cores) jobs. The limit of 3 can be gotten rid of if/when we solve the too many fd's problem (each encode uses 256 fd's at the moment); I have a partial fix on the db4o branch but it needs optimising and extending to non-persistent ...
pgpgZ2tYQ4lWj.pgp
Description: PGP signature
_______________________________________________ Devl mailing list [email protected] http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
