I have been going through my logs; trying to validate the assertion that I
saw once on this list that the data actually requested through freenet was
small; and that the overhead of the requests was where much of the bandwidth
went.
While I'm not going to dispute that the overhead needs to be limited /
controlled; I do think that the assertion that the data sought is small
isn't totally valid;
Grepping and otherwise mangling my logs; I found that the DataLength had the
following distribution:
Length,Occurances
41,1
82,8
94,5
123,1
188,1
1025,204
2049,26
4097,20
8193,55
16385,19
32790,36
65600,38
131220,30
262460,171
524940,80
1049900,168
2099820,5
4196972,28
To be noted; I tossed out where datalength was the first element in the set,
eg {DataLength rather than ,DataLength as in those cases the value appeared
to be hex; I also uniquely sorted to get rid of double counting where my
node accepted the data from one node to pass it onto the next.
I therefore (re)pose the question of shortening the data path between the
requester and the recipient when the data load is large.
I expect any attempt to bypass nodes on small keys (eg < 10k of data load)
would likely be pointless; as the overhead/time taken to establish the skip
a node would likely be longer than just transferring the data. For the
larger keys; my node (likely) would be uninterested in storing many 4 meg
keys unless they were really close to it's focus, so it would make sense to
offer it the possibility of opting out of the transfer; I am suggesting
something like request.status large data, datasize = #, request direction
type of message; to which the node (if it didn't wish to store the key in
its datastore) could ask the requester of it something like request status,
large data, suggest node (ident src node) transfer directly, request
direction -- that node could either indicate that it will accept the direct
transfer; or indicate that it cannot accept it (for whatever reason) from
that node; possibly because it is transient and cannot accept incoming
connections; and it is unable to establish a connection to the offering
node. If the node is willing to accept an attempt at a direct transfer; let
the offering node know the node's ident of where to send the data; if
successful the offering node would let the requester know that the data was
transfered successfully so that it can close the request; or if unsuccessful
(eg communication cannot be established) fall back to sending the data
through the intermediary node. I am suggesting that only every second node
would be able to step asside from the data transfer, to maintain the
anonymitity of the requester & provider; but that it could reduce bandwidth
transfering data up and down through nodes that (due to the present
overload) don't really specialize in that area of the keyspace.
Less data flowing would hopefully mean that the network could backoff from
its overloaded state, and requests would become more successful with shorter
data paths; thereby reducing the flow of data more -- the long data paths
could be saved for when the data is difficult to come by.
Thoughts / comments?
Trevor
----- Original Message -----
From: "Gianni Johansson" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Monday, May 20, 2002 9:44 PM
Subject: [freenet-devl] Lets use the Wiki to try to come up with ideas on
what to do next...
> I started a "whats wrong and how do we fix it" brainstorming page in the
> wiki.
>
> http://freenetproject.org/cgi-bin/twiki/view/Pub/ThoughtsAndAnalysis
>
> Testable hypotheses with proposed experiments / observations to validate
them
> are preferred. It's kind of paradoxical that a Freenet node is capable of
> producing more data than any human could ever hope to analyze but few of
the
> diagnoses of what's wrong with the network are actually backed up by
data....
>
> -- gj
>
>
> --
> Freesite
> (0.4) freenet:SSK@npfV5XQijFkF6sXZvuO0o~kG4wEPAgM/homepage//
>
> _______________________________________________
> Devl mailing list
> [EMAIL PROTECTED]
> http://lists.freenetproject.org/mailman/listinfo/devl
>
_______________________________________________
devl mailing list
[EMAIL PROTECTED]
http://hawk.freenetproject.org/cgi-bin/mailman/listinfo/devl
