Ian wrote: > This is extremely interesting. Do you think that these predictions > (either RTE's or SVM's) are statistically significant (I will need > to look more closely at the data).
I do believe the results are significant in that they are not just a lucky run or fluke; I tried many different parameter combinations and have seen the SVM perform better in most cases. As to whether or not the differences matter, I tend to doubt the estimation accuracy improvement would be much help if the rest of the routing algorithm did not change. I think that a predictor that improved accuracy only 1%, but was used as part of the forwarding decision algorithm could still cause a dramatic improvement in Freenet's performace as a whole, as the final output is not an estimate but a routing decision. So really the question to me is, how do these algorithms perform as part of a larger network? My suspicion is that the simple version of routing that I have heard about involving forwarding to "closest keys" may be creating some congestion bottlenecks or file "masking" effects, analogous to how a hash table using linear probing will create more clustered distributions than a hash table using a secondary hash function. I think it's clear that there will be some sort of "self-organizing complexity" as Wolfram would phrase it, but am still trying to decide how we might characterize and measure different large-scale network behaviors when under the control of different routing schemes. Perhaps it will be useful to add some optional request-tracking to the protocol so that we can look at actual traces of a query flowing through the Freenet, and see where the actual holdup occurs, usually. I have no idea how feasible this might be; but if we can't temporarily disable anonymity then I am not sure how else we can even measure the effects of experimental adjustments. Rudi > Matthew is working hard to get the NGR infrastructure in-place -=20 > then we will be able to do a more realistic test that will include > factors such as CPU and memory requirements which are important > to consider prior to an actual deployment. > > Ian. > > On Sat, Jul 26, 2003 at 06:08:05AM -0700, Rudi Cilibrasi wrote: > > Hi again everybody, > >=20 > > I've just finished running "experiment 2" based on the real-life > > sampled data Ian recently posted. Short summary: SVM is about 10% > > more accurate (in terms of RMS) on this sample, as compared to > > RTE set on max accuracy. There's a lot of tweaking and experimentation > > that we can do, but I think it's wasted until we make a better > > description of how it is supposed to interface in the real > > freenet code. You can see the results and code yourself at: > >=20 > > http://homepages.cwi.nl/~cilibrar/ngrouting/exp2/ > >=20 > > This one is pretty slow to run, as I'm doing a lot of N^2 type loops > > for easy implementation -- I think we can probably adjust the > > time/space/accuracy tradeoff better when we have a clearer idea of > > what the real interface in Freenet will look like. > >=20 > > I don't think the 10% number is very significant by itself, but I do > > expect that if we wind up using the SVM as part of a route > > choosing scheme, then it may evolve quite different global network > > behavior than the RTE scheme. In combination with adding more > > independent input data, I think we may get very different behavior. > > Better or worse, I don't know, but I do think it will be interesting > > to find out. =20 > >=20 > > Rudi > >=20 > > _______________________________________________ > > devl mailing list > > [EMAIL PROTECTED] > > http://hawk.freenetproject.org:8080/cgi-bin/mailman/listinfo/devl > > --=20 > Ian Clarke [EMAIL PROTECTED] > Coordinator, The Freenet Project http://freenetproject.org/ > Founder, Locutus http://locut.us/ > Personal Homepage http://locut.us/ian/ > > --HcAYCG3uE/tztfnV > Content-Type: application/pgp-signature > Content-Disposition: inline > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.2.1 (GNU/Linux) > > iD8DBQE/JWzQQtgxRWSmsqwRAnYAAJ0Usm/J7eMzc5geG8+jPKzJFi5vkwCffNdg > hVFPDJGqTbz2dFn+gRlEhrE= > =syXJ > -----END PGP SIGNATURE----- > > --HcAYCG3uE/tztfnV-- > > --__--__-- > > Message: 10 > Date: Mon, 28 Jul 2003 20:38:26 +0100 > To: [EMAIL PROTECTED] > From: Roger Hayter <[EMAIL PROTECTED]> > Subject: [freenet-dev] Possible security problem with experimental build and/or Sun > Java > Reply-To: [EMAIL PROTECTED] > > I have been running build 6508, snapshot of about 48 hours ago, with Sun > 1.4.2. I noticed today that isproxy would not start. I found a process > which was only identified as "java2", which is the name of the symlink > to ..../jre/bin/java, had a tcp connection from a high port on my > machine to port 6667 on a host in attbi.com. I can think of no > legitimate explanation of this, but would be glad to hear of one! > -- > Roger Hayter > > > --__--__-- > > _______________________________________________ > devl mailing list > [EMAIL PROTECTED] > http://hawk.freenetproject.org:8080/cgi-bin/mailman/listinfo/devl > > > End of devl Digest _______________________________________________ devl mailing list [EMAIL PROTECTED] http://hawk.freenetproject.org:8080/cgi-bin/mailman/listinfo/devl
