Freenet thoughts: FEC vs. HTL (breadth vs depth)
----------------------------------------------------
Originally I thought Freenet did not need Forward
Error Correction in the least. But now I think FEC
could be the key (no pun intended) to an excellent
Freenet. Let me explain.
When inserting data, you presently have three options
to ensure that they stick to the network: (1) use a
high enough HTL, (2) use FEC splitfiles, or (3) do
both. Now the argument *against* FEC is very
compelling: assuming all conditions are good (your
data are popular enough; nodes aren't deleting their
stores; there is enough space on the network), you
don't need FEC. You should be able to retrieve all
your blocks, unless they are never used.
When it comes down to it, FEC implies breadth (width),
and HTL implies depth. What I mean is that when you
insert with a high HTL (i.e. 4), each piece of the
same hash is upload to 4 nodes (4 "deep"). If you
insert with a FEC redundancy of 4x (to match with the
HTL), you're inserting the same amount of data, but
spreading it across a wider variety of nodes
("breadth"). I know that 400% is absurd; it is chosen
to correlate with the HTL of 4 so both are about the
same amount of data. *I'm proposing that breadth is
preferable to depth.*
We know that the network is unstable because most data
is lost a few days after insertion. You simply cannot
find files that have many blocks. Let's assume for
the sake of argument that *data is lost due to nodes
temporarily dissapearing from the network.* Let's
also assume that freenet uploads any block to the
perfect node (the most specialized for a given hash)
first, then the next most perfect, the next, etc. So
you get efficient routing that way (key closeness) as
described in Ian's paper.
Now lets imagine that our perfect node is offline, and
we're searching for the key on it. Effectively the
route to this key is destroyed now. We must now
backtrack the route to search similar nodes to see if
they have the block (this is like searching deeper).
On the other hand, if we're missing a block with FEC,
we can search wider, across more nodes, for another
block that has a *different hash*.
Now the argument still stands that inserting FEC
doubles the amount of data you need to insert. *But
every increase in HTL multiplies the amount of data
you insert, without as much benefit as FEC!* Lets say
I want to insert a 250MB file. If I use an HTL of 10
(which is probably necessary for anyone to get the
whole thing) and no FEC, I have to insert 2,500 MB.
On the other hand, if I insert with 100% redundancy
and an HTL of 2, that's only 1,000 MB I have to
insert. And wouldn't that be much more reliable?
I apologize for not presenting a more mathematical
argument, but I felt that this might be an interesting
idea so I wanted to express it as quickly as possible.
__________________________________________________
Do You Yahoo!?
Yahoo! Sports - Coverage of the 2002 Olympic Games
http://sports.yahoo.com
_______________________________________________
freenet-tech mailing list
[EMAIL PROTECTED]
http://lists.freenetproject.org/mailman/listinfo/tech
