On Thu, Sep 12, 2002 at 07:37:32PM +1000, fish wrote: > > Most of this sounds pretty good. > > Firstly, a stupid question - is there any reason to seperate "data blocks" > and "check blocks"? As far as the FEC encoder/decoder knows, they're just > blocks, right? I mean, that's the whole *point* of it (you need any k > blocks of n in order to decode a file). Would make things simpler, > conceptually, I think. This of course is based on assumptions from FEC > implentations that I have seen, where the block size is > constant... obviously, if your FEC implentation makes the distinction, > then I guess it makes sense (translation from fishish: yeah, we need to > support checkblocks for certain algorythms, but they don't make sense for > onion's) > > Anyhow, the other point I wished to make, is that from looking at your > information, it seems like it would be far more convinent still to just > call the onionnetworks library direclty - okay, yeah, I see the usefulness > of this for providing access to people who don't have/don't want to have > bindings to this, but it just seems like an unnessesary layer of > abstraction to me. But perhaps I am on crack. Hmmm, maybe. It's not well documented afaics? Anyway, higher level commands (with reasonable status reporting, and some sort of keepalives or a way to reconnect to a running command, and it only terminate when explicitly told to), would make clients easier to write, but this looks useful. > > > III. Changes to SplitFile metadata format. > > > > 0) Deprecate the BlockSize field, since check blocks are not necessarily the > > same size as data blocks and blocks may be different sizes across segments. > > I strongly disagree with this - if we want to support this case, it is > better to then have a seperate set of metadata for each segment and > specify the block/check size in each one. This information is very useful > to have for reasons of memory allocation and the like. > > Other than that, however, All of that being said, this all looks okay to > me on my initial reading. > > - fish > > p.s. I included the following in the original draft of this email, but > considered it offtopic and hence seperated it out from the main > email. However, I include it here because it is interesting and > semi-relevent: > > > For a given maximum block size, some FEC algorithms can only > > practically handle files up to a certain maximum size. The design > > uses segmentation to handle this case. Large files are divided into > > smaller segments and FEC is only done on a per segment basis. This > > compromise provides a least limited redundancy for large files. > > Should we be perhaps looking for a library which doens't suffer from these > problems as much as onion's library, however? The thing is, the whole > usefulness of FEC is for big files, you know... I know that we do have to > deal with reality instead of would-be-nice's, however, but it is something > to think about. > > The other problem is, that as you stripe like this, the amount of > redundancy is, of course, reduced significantly, however you already knew > that. However, people writing bad algorythms for downloading files (block > 1,2,3,4.... etc in order) could make this problem even worse. (As a side > note to this, I have been wondering if an AWT based freenet download > manager would be a useful thing to code/have... any thoughts on > this? Heh, of course, this would require me to learn how to communicate > with freenet from java, but i can't be that hard, can it? :-p) Please do. Do NOT use swing. And include more detail on the status of the individual blocks, eg bytes downloaded/total progress bar, time this block idle for etc. Would be very cool. > > Anyhow, I'm sure you knew all of this... just restating it for my own > benefit, don't mind me :). I'll look into the alternative libraries > myself over the next few days... there's nothing to stop us having two > encoders, given that the facilities are there, and let the best codec win > :-p. > > >
-- Matthew Toseland mtoseland at blueyonder.co.uk amphibian at sourceforge.net Freenet/Coldstore open source hacker. Employed full time by Freenet Project Inc. from 11/9/02 to 11/11/02. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20020912/fa75bf71/attachment.pgp>
