On Sat, Oct 05, 2002 at 11:57:23AM -0400, Gianni Johansson wrote: > On Wednesday 02 October 2002 19:23, ian wrote: > > > > Shouldn't the datastore be used for this kind of thing? > > > There are four issues: Now that the native datastore is the default, why can't we default the temporary bucket dir, for both FEC and fproxy, to store/temp/, and create this dir? > > 1) The BucketFactory interface doesn't allow you to create a bucket of > unknown size. As a result, you have to either massively overallocate or rely > on unspecified behavior of the underlying implementation. The FEC > inserting/downloading code in fproxy does latter. > > i.e. bf.makeBucket(-1) // Works only because the BucketFactory implementation > is making FileBuckets. > > 2) The FSBucketFactory.FSBucket.size returns the requested bucket size when > the bucket was created, *not* the number of bytes that have been written into > the bucket. This is a violation of the Bucket interface semantics and is > just wrong, but I guess it's hard to fix (maybe it's easier with the new > native datastore implementation?) > > I argued with Tavin over this for a long time last year and finally just gave > up. You have to know how to pick your battles :-( > > The effect is that even if you massively overallocate, hoping that you will > have enough space for the data for the key you are requesting, there's no > what to know how much data you actually got. > > 3) FEC plugin implementation assumes FileBucket derived Buckets. > > This is a hack. My bad. But without it striping would work orders of > magnitude slower. > > The issue is that you need to be able to map a Bucket to a File (containing > unencrypted data) so that you can seek(). > > 4) The shear size of FEC temp files. > Do you really want users to flush their data store every time they insert an > ISO? > > I haven't been keeping up with the list. Can the new datastore grow without > bound? If not, the datastore size would impose an upper limit on the maximum > FEC splitfile size. > > I guess it wouldn't be that hard to hack your way around 1 and 2, and 4 > (separate datastore for fec temp files. yuck). 3 is much tougher. > > > Just how are these settings intended to be used? > > > > # Make sure you use a trailing slash. I guess I should make it optional. > > #Set this parameter to a directory with a lot of free space. > FECTempDir=/home/giannij/freenet_module/fectmp/ > > #Set this parameter to a directory with a lot of free space > mainport.params.servlet.1.params.tempDir=/usr2/giannij_space/fproxy_tmp/ > > These can point to the same directory. Ideally we would have only one > parameter, but you would have to do some sort of rude hack, given the > different manner in which fred and servlet parameters are handled. i.e. > Ideally servlets should know nothing about fred parameters so they can run in > an abitrary Container. We have violated that principle in so many other > places, I guess one more hack woudn't hurt. > > Also, I am somewhat reluctant to do anything that requires changes to > parameter handling after the fiasco of the week before last. > > --gj > > p.s. > Your README.paranoid file is a good idea. > > People who are really bent out of shape about unencrypted temp files should > consider whether their JVM is using locked memory... > > > Ian. > > > > On Wed, Oct 02, 2002 at 04:20:34PM -0700, Reuben Balik wrote: > > > What would be the recommended entries into the > > > freenet.ini to fix the following errors: > > > > > > WARNING: FECTempDir not set in freenet.conf/ini! > > > Oct 2, 2002 5:31:55 PM (freenet.node.Main, main): > > > WARNING: FECTempDir not set in freenet.conf/ini! > > > Oct 2, 2002 6:15:44 PM > > > (freenet.client.http.FproxyServlet, QThread-10): > > > WARNING: fproxy tempDir not set. > > > Oct 2, 2002 6:15:44 PM > > > (freenet.client.http.FproxyServlet, QThread-10): > > > Set mainport.params.servlet.1.params.tempDir in > > > freenet.conf/ini. > > > WARNING: fproxy tempDir not set. > > > > > > --Reuben Balik > > > > > > __________________________________________________ > > > Do you Yahoo!? > > > New DSL Internet Access from SBC & Yahoo! > > > http://sbc.yahoo.com > > > > > > _______________________________________________ > > > devl mailing list > > > devl at freenetproject.org > > > http://hawk.freenetproject.org/cgi-bin/mailman/listinfo/devl > > ---------------------------------------- > Content-Type: application/pgp-signature; charset="us-ascii"; > name="Attachment: 1" > Content-Transfer-Encoding: 7bit > Content-Description: > ---------------------------------------- > > _______________________________________________ > devl mailing list > devl at freenetproject.org > http://hawk.freenetproject.org/cgi-bin/mailman/listinfo/devl >
-- Matthew Toseland toad at amphibian.dyndns.org amphibian at users.sourceforge.net Freenet/Coldstore open source hacker. Employed full time by Freenet Project Inc. from 11/9/02 to 11/11/02. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20021015/6967ee00/attachment.pgp>
