On Wed, Oct 10, 2001 at 11:00:19AM +0200, Sebastian Sp?th wrote:
> thelema wrote:
>
> > I'm just suprised that everyone now seems to agree that we need
> > redundancy when before everyone seemed to be saying "hell no, keep that
> > redundancy away", and I had to compromise with a system that allowed
> > both redundant and non-redundant usage.
>
>
> Not everyone agrees on the splitting plans with redunancy yet. I don't,
> for example, but gave up resistance as I seem to be a tiny minority. I
> am against too eagerly splitting of files in the first place (There were
> discussions where a 64kb/128kb split size was proposed, which I consider
> ridiculous).
The reason that we need to use splitting is performance. As Freenet
grows, it can only be expected that download times for single files
going through 5-10 hops will be quite deplorable - with splitting,
however, you can be pretty sure that all searches will diverge therefore
causing your nodes Internet connection to be the total bottleneck. It
effectively gives us swarming and potentially makes Freenet a high
performance system at little cost.
> a) splitting increases the likelihood of a retrieval failure (more
> potential possibilities of one or more missing piece(s))
> p_tot = p_retr^splitparts
> (file=1,01 MB, splitsize=256kb, splitparts=5, without redundancy)
> p_retr=99% -> p_tot = 95%
> p_retr=90% -> p_tot = 59%
As I said, this is why redundancy is necessary.
> b) splitting might require redundancy in the split files to be able to
> compensate a) -> more data in Freenet -> more data drops out ->
> decreased reliability and storage capacity
We are not talking about large increases here, about 10% redundancy or
so will be enough most of the time. The decrease in success do to
Freenet filling will (unless the network is degenerating) be linear,
whereas the benefit given by redundancy is exponential - the former
cannot compete and therefore there is little or no risk of an evil
circle.
> c) splitfiles lead to more overhead as the last part has to be filled
> with mumbo-jumbo to reach the std size -> more data in Freenet... see b)
To start with your thinking is wrong (more parts, means smaller parts,
means that the difference between 2^n and 2^n+1 such that the size falls
between them is smaller - it should compensate exactly), and it doesn't
matter anyways since you would obviously choose the size of the parts so
as to give little or no overhead.
> I see there a vicious circle coming up, which I don't really like.
Your argument is completely huristic and mostly incorrect - there is no
mathematical argument for why this should be the case.
> Mind, I am not against splitting per se, it might be appropriate for
> *big* files, but why should my 386kb .gif be splitted in two parts?
Because you would rather retrieve it in 193 then 386 seconds.
>
>
--
Though here at journey's end I lie
In darkness buried deep, above all shadows rides the Sun
beyond all towers strong and high, and the Stars forever dwell:
beyond all mountains steep, I will not say the Day is done,
nor bid the Stars farewell.
(JRRT)
Oskar Sandberg
oskar at freenetproject.org
_______________________________________________
Devl mailing list
Devl at freenetproject.org
http://lists.freenetproject.org/mailman/listinfo/devl
