Oh, I forgot to mention - once all this autosplit stuff fully stabilises,
fcpputsplit will be deleted, since it will no longer been needed.
----- Original Message -----
From: David McNab
To: Freenet Chat ; Freenet Development
Sent: Friday, October 05, 2001 1:11 AM
Subject: [freenet-chat] FCPtools now with AutoSplit support
Hi all,
I've checked into CVS a new version of the fcptools (freenet CVS, in
Contrib/fcptools), which now supports auto-splitting of all inserted files.
Since auto-split is implemented completely in the ezFCPlib library layer, all
client software which uses fcpPutKeyFromFile() will inherit the autosplit
functionality with no changes needed (except a new parameter to fcpStartup()).
Summary of changes
1) fcpStartup() now launches a 'splitfile insert manager' thread, which
manages and dispatches all splitfile insertions. This manager thread maintains
a queue of all file insert jobs, and spawns further insert threads as needed.
2) All files are split if they exceed the default chunksize (256k). This can
not be overridden.
3) Default thread limit for chunk insertions is 8 for the whole process
4) New command line arguments recognised by fcpput and fcpputsite:
-ss size of splitfile chunks in bytes ('k','m','g' at end recognised,
eg '256K')
-st maximum number of splitfile threads, default 8
5) Modules fcpSetHost.c and fcpSetHtl.c removed, consolidated into
fcpSetParam.c
6) Splitting is *not* done with fcpPutKeyFromMem()
7) Splitting is *not* done from streamed key insert
(fcpOpenKey()/fcpWriteKey()/fcpCloseKey())
8) Splitting *is* done when key is inserted via fcpPutKeyFromFile()
9) The global variable 'int fcpSplitChunkSize', if set before calling
fcpStartup(), governs the splitfile chunk size (default 256k).
10) fcpStartup() parameters change:
int fcpStartup(char *host, int port, int defaultHtl, int raw, int
maxSplitThreads)
11) Obviously, fcpput and fcpputsite now insert splitfiles, automatically and
transparently. There is no way to override filesplitting for large files.
12) Lots of debug messages available if tools are run with '-v 4'.
Status - I've spent the last day and a half weeding out bugs and
stress-testing. The whole thing seems to be holding up well on Linux and
Windows, even with big jobs.
Bugs?
Probably infinite. If you find bugs, please find the simplest possible way to
consistently reproduce them, run the applicable tool at full verbosity ('-v
4'), capture the output to a file, and send the file to me, along with your OS
information etc. Feel free to 'censor' out any SSK private keys if necessary.
Cheers
David
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://emu.freenetproject.org/pipermail/devl/attachments/20011005/85a2b2ad/attachment.html>
