Sorry people... are you actually proposing inserting the same files over and over? Because you are! You are actually polluting freenet this way by introducing redundancy of the worst kind: blind redundancy
Freenet is efficient storage because every unique piece of data (in the file level) has a related CHK key so that the system knows what is already in there. Packaging stuff into compressed files, for whatever reason, hides the data, introducing useless new CHK keys that claim space for the compressed file, which is actually partly, if not mostly, the same data. Imagine a "typical", let's say 1MB, DBR site re-inserted every day, even though only part of the content is altered (i.e. graphics, which should be the larger part in size, stay the same, but the html changes). Poor usage of the capabilities of freenet, wouldn't you say; There is indeed a need to improve speed (try saying that 10 times, fast), but let's not break freenet, OK? Doc -----Original Message----- From: Reuben Balik [mailto:[email protected]] Sent: Thursday, November 21, 2002 3:58 AM To: devl at freenetproject.org Subject: Re: [freenet-dev] tying freesites together with jars I think this is a very good idea, but might not work for very large freesites. Maybe there could be some metadata that says whether the files are individual or inside of a jar file. That way a freesite with a 1gb of content won't have to insert one huge file, (and visitors won't have to download it), but a smaller freesite could. What does everybody else think? --Reuben --- Tyler Riddle <triddle_1999 at yahoo.com> wrote: > This was posted to tech at freenetproject.org but I did > not see any word of here or any activity regarding > it > on the tech list so I'm posting it here. This is > something I have been advocating for a while (not > entirely in this manor, but the functionality is the > same) and I also posted to tech@ some time ago and > got > ignored as well. This would be a great feature but I > see a huge potential for abuse as well. I think it > would be worth it to discuss how to pull this > particular feature off if it is at all possible with > out bringing the network to it's knees. > > Tyler > > Begin original message: > > The problem: > Freenet is slow. Loading a freesite from an > activelink > takes a long > time, and when you finally get the HTML file, it > takes > even longer > before you see the graphics (assuming you get to see > all of them). If > you then navigate to a secondary page of that > freesite, odds are that > it won't even load. > The problem isn't one of bandwidth, but one of > latency. For each file > that is needed, one or more key(s) must be > retrieved, > and for each key > that is needed, a little "roamer" is sent out on the > net to visit each > and every node (within a radius) one by one, hoping > to > run into the > sought after key and come home with it. This takes a > long time. I > therefore submit that, because of this, it takes > much > longer to > retrieve ten 10KB files than it takes to retrieve > one > 100KB file, it's > the nature of the beast. Furthermore, since it takes > ten little roamers > to retrieve the ten keys and only one to retrieve > the > one key, it > causes ten times the traffic on the net, compounding > the problem. > A solution: > JAR files. Take the few HTML files and the few > graphic > files that make > a freesite, pack them in a .jar file and insert it > at > a "/site//" type > URI. URI's like "/site//index.html" and > "/site//images/activelink.gif" > would retrieve the .jar file and extract the target > file. > The advantages: > 1. you only need to retrieve one file to render a > complete freesite > (including secondary pages). > 2. by doing so, you replicate the whole site, not > just > the page(s) you > visited. > 3. activelinks replicate the whole site as well. > 4. the .jar files are compressed, improving DS usage > and transmission > times. > 5. insertions are faster and more efficient. > 6. chatter on the net is reduced dramatically. > 7. using real .jar files leverages existing > technology > in the JDK. > 8. users could even use the jar tool to create them > (until insertion > tools get smarter). > A few things to keep in mind: > 1. the .jar files should be kept under 1MB to keep > them from splitting > (and for practical reasons). > 2. freesites should not include files for download > in > the .jar files, > instead they should be inserted separately and > linked > to by CHK. > 3. .jar files not accessed through a "//" URI should > simply be > retrieved. > 4. users can still use other methods if they prefer > (maps/manifests). > This simple feature has the potential to greatly > improve Freenet's > worst perceived shortcoming. > Yves Lempereur > > ===== > AIM:rllybites Y! Messenger:triddle_1999 > > __________________________________________________ > Do you Yahoo!? > Yahoo! Web Hosting - Let the expert host your site > http://webhosting.yahoo.com > > _______________________________________________ > devl mailing list > devl at freenetproject.org > http://hawk.freenetproject.org/cgi-bin/mailman/listinfo/devl __________________________________________________ Do you Yahoo!? Yahoo! Web Hosting - Let the expert host your site http://webhosting.yahoo.com _______________________________________________ devl mailing list devl at freenetproject.org http://hawk.freenetproject.org/cgi-bin/mailman/listinfo/devl _______________________________________________ devl mailing list devl at freenetproject.org http://hawk.freenetproject.org/cgi-bin/mailman/listinfo/devl
