On Thursday 12 September 2002 05:37, you wrote: > Most of this sounds pretty good. > > Firstly, a stupid question - is there any reason to seperate "data blocks" > and "check blocks"? As far as the FEC encoder/decoder knows, they're just > blocks, right? I mean, that's the whole *point* of it (you need any k > blocks of n in order to decode a file). Would make things simpler, > conceptually, I think. This of course is based on assumptions from FEC > implentations that I have seen, where the block size is > constant... obviously, if your FEC implentation makes the distinction, > then I guess it makes sense (translation from fishish: yeah, we need to > support checkblocks for certain algorythms, but they don't make sense for > onion's) It's good to make the distinction because if you manage to get all the data blocks you don't need to decode at all.
> > Anyhow, the other point I wished to make, is that from looking at your > information, it seems like it would be far more convinent still to just > call the onionnetworks library direclty - okay, yeah, I see the usefulness > of this for providing access to people who don't have/don't want to have > bindings to this, but it just seems like an unnessesary layer of > abstraction to me. But perhaps I am on crack. Client writers just don't want to deal with it. FEC support has been in CVS for almost a year and you are the first person who has attempted to write a client besides me. > > > III. Changes to SplitFile metadata format. > > > > 0) Deprecate the BlockSize field, since check blocks are not necessarily > > the same size as data blocks and blocks may be different sizes across > > segments. > > I strongly disagree with this - if we want to support this case, it is > better to then have a seperate set of metadata for each segment and > specify the block/check size in each one. This information is very useful > to have for reasons of memory allocation and the like. You can still get this info out of the SegmentHeader messages if you want it. I am leery of bloating the SplitFile metadata any more. > > Other than that, however, All of that being said, this all looks okay to > me on my initial reading. > > - fish > > p.s. I included the following in the original draft of this email, but > considered it offtopic and hence seperated it out from the main > email. However, I include it here because it is interesting and > > semi-relevent: > > For a given maximum block size, some FEC algorithms can only > > practically handle files up to a certain maximum size. The design > > uses segmentation to handle this case. Large files are divided into > > smaller segments and FEC is only done on a per segment basis. This > > compromise provides a least limited redundancy for large files. > > Should we be perhaps looking for a library which doens't suffer from these > problems as much as onion's library, however? The glass is half full man. The math works for files out to about 1G no problem even with segmentation for the current implementation. The OnionNetworks code works well for moderately sized files now! We should make it easily available. I will keep a similar plugin architecture so that new encoder/decoder implementations can be used if/when they are written. > The thing is, the whole > usefulness of FEC is for big files, you know... I know that we do have to > deal with reality instead of would-be-nice's, however, but it is something > to think about. > Segmenting reduces redundancy not striping. (see above) > The other problem is, that as you stripe like this, the amount of > redundancy is, of course, reduced significantly, however you already knew > that. However, people writing bad algorythms for downloading files (block > 1,2,3,4.... etc in order) could make this problem even worse. (As a side > note to this, I have been wondering if an AWT based freenet download > manager would be a useful thing to code/have... any thoughts on > this? Heh, of course, this would require me to learn how to communicate > with freenet from java, but i can't be that hard, can it? :-p) > > Anyhow, I'm sure you knew all of this... just restating it for my own > benefit, don't mind me :). I'll look into the alternative libraries > myself over the next few days... there's nothing to stop us having two > encoders, given that the facilities are there, and let the best codec win. Cool. > > :-p. > > _______________________________________________ > devl mailing list > devl at freenetproject.org > http://hawk.freenetproject.org/cgi-bin/mailman/listinfo/devl _______________________________________________ devl mailing list devl at freenetproject.org http://hawk.freenetproject.org/cgi-bin/mailman/listinfo/devl
