Send Devl mailing list submissions to
devl at freenetproject.org
To subscribe or unsubscribe via the World Wide Web, visit
http://www.uprizer.com/mailman/listinfo/devl
or, via email, send a message with subject or body 'help' to
devl-request at freenetproject.org
You can reach the person managing the list at
devl-admin at freenetproject.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Devl digest..."
Today's Topics:
1. Re: Protozilla (Mark J. Roberts)
2. Re: Proposal: algorithm for forgetting documents in datastore (Ian Clarke)
3. Re: Suggestion for solving the small vs. large file debate (Ian Clarke)
4. Re: Protozilla (Steven Hazel)
5. Re: Proposal: algorithm for forgetting documents in datastore (Mr.Bad)
6. Re: Proposal: algorithm for forgetting documents in datastore (Oskar
Sandberg)
7. Re: Proposal: algorithm for forgetting documents in datastore (Travis
Bemann)
8. Re: Proposal: algorithm for forgetting documents in datastore (Travis
Bemann)
9. Re: Proposal: algorithm for forgetting documents in datastore (Scott G.
Miller)
10. Re: Proposal: algorithm for forgetting documents in datastore (Tavin Cole)
11. Re: Proposal: algorithm for forgetting documents in datastore (Tavin Cole)
--__--__--
Message: 1
Date: Sun, 28 Jan 2001 14:05:30 -0600 (CST)
From: "Mark J. Roberts" <m...@statesmean.com>
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Protozilla
Reply-To: devl at freenetproject.org
On Sun, 28 Jan 2001, Adam Langley wrote:
> On Sun, Jan 28, 2001 at 12:58:03PM -0500, Scott G. Miller wrote:
> > Damn you! FIRST POST!
>
> Yea, ok. Your post hadn't come thru when I sent mine. You win ;)
This list SUCKS! They posted the SAME STORY a couple weeks ago! Whoever
runs this thing must be an IDIOT! ;-)
--
Mark Roberts
mjr at statesmean.com
--__--__--
Message: 2
Date: Sun, 28 Jan 2001 12:17:11 -0800
From: Ian Clarke <i...@octayne.com>
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Proposal: algorithm for forgetting documents in
datastore
Reply-To: devl at freenetproject.org
--fUYQa+Pmc3FrFX/N
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
On Sun, Jan 28, 2001 at 04:34:51PM +0000, Theodore Hong wrote:
> What's intrinsically wrong with byte-seconds? It has quite a clear meaning
> -- it is a volume in spacetime. Over the lifetime of a node, its datastore
> has a certain total volume, so this is just a packing problem: how do we
> allocate space in this volume so that the total value stored is maximized?
It is not what is wrong with byte-seconds, it is what is right with it.
Why not kilobyte-seconds, or kilobyte-minutes etc etc.
The choice of units is essentially arbitrary but *will* effect the
behaviour of the system. The need to make arbitrary decisions are a
symptom of a flawed design. This was one of the advantages of using a
LRU cache.
Before we start proposing replacements for the current mechanism,
someone should actually figure out if there is anything wrong with it!
If it isn't broken, don't fix it.
Ian.
--fUYQa+Pmc3FrFX/N
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQE6dH5GQtgxRWSmsqwRAtmMAJ9T6v0w7XTLktvj0YlEu2Y/MFZnNACfersJ
SjymExGf5RPMTG8DsU5LSp0=
=Bcnk
-----END PGP SIGNATURE-----
--fUYQa+Pmc3FrFX/N--
--__--__--
Message: 3
Date: Sun, 28 Jan 2001 12:21:57 -0800
From: Ian Clarke <i...@octayne.com>
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Suggestion for solving the small vs. large file
debate
Reply-To: devl at freenetproject.org
--V0207lvV8h4k8FAm
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
On Sun, Jan 28, 2001 at 04:44:16PM +0100, Oskar Sandberg wrote:
> Yeah, because the oldest file in seconds will so not be the oldest file in
> minutes, and a file twice as large in bytes will so not be twice as
> large in kB...
Ah yes, misfired on that one.
Ian.
--V0207lvV8h4k8FAm
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQE6dH9lQtgxRWSmsqwRAnK9AJ91MeVPirScVgixELpprFmiNb2mZgCeMy/S
hSSA5rU8A/Qa4+DDKlGwh/s=
=PGJZ
-----END PGP SIGNATURE-----
--V0207lvV8h4k8FAm--
--__--__--
Message: 4
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Protozilla
From: Steven Hazel <s...@thalassocracy.org>
Date: 28 Jan 2001 14:24:29 -0600
Reply-To: devl at freenetproject.org
"Mark J. Roberts" <mjr at statesmean.com> writes:
> On Sun, 28 Jan 2001, Adam Langley wrote:
>
> > On Sun, Jan 28, 2001 at 12:58:03PM -0500, Scott G. Miller wrote:
> > > Damn you! FIRST POST!
> >
> > Yea, ok. Your post hadn't come thru when I sent mine. You win ;)
>
> This list SUCKS! They posted the SAME STORY a couple weeks ago! Whoever
> runs this thing must be an IDIOT! ;-)
meept
-S
--__--__--
Message: 5
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Proposal: algorithm for forgetting documents in
datastore
From: Mr.Bad <mr....@pigdog.org>
Organization: Pigdog Journal
Date: 28 Jan 2001 13:04:23 -0800
Reply-To: devl at freenetproject.org
>>>>> "IC" == Ian Clarke <ian at octayne.com> writes:
IC> It is not what is wrong with byte-seconds, it is what is right
IC> with it. Why not kilobyte-seconds, or kilobyte-minutes etc
IC> etc.
IC> The choice of units is essentially arbitrary but *will* effect
IC> the behaviour of the system.
I'm not sure I understand why. Isn't it a relative measurement? It's
not really important that what the value of a particular file is --
only what its magnitude is relative to other files in the cache.
So if file A has 6, file B has 7, and file D has 10, it doesn't really
matter if we multiply them all by a scaling factor of 2 or 10 or 1000.
One thing I don't get is why kb-sec are important. It seems like
IC> Before we start proposing replacements for the current
IC> mechanism, someone should actually figure out if there is
IC> anything wrong with it! If it isn't broken, don't fix it.
Well, it seems that size is -too- much of a factor, and that it's hard
to get large files (like, MP3-size files) to stay in the system for
any reasonable length of time.
Does anyone else have the same experience? Or am I once again smoking
crack?
~Mr. Bad
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/\____/\ Mr. Bad <mr.bad at pigdog.org>
\ / Pigdog Journal | http://pigdog.org/ | *Stay*Real*Bad*
| (X \x)
( ((**) "If it's not bad, don't do it.
\ <vvv> If it's not crazy, don't say it." - Ben Franklin
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
--__--__--
Message: 6
Date: Sun, 28 Jan 2001 22:24:42 +0100
From: Oskar Sandberg <md98-...@nada.kth.se>
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Proposal: algorithm for forgetting documents in
datastore
Reply-To: devl at freenetproject.org
On Sun, Jan 28, 2001 at 12:08:04PM -0500, Scott G. Miller wrote:
> On Sun, Jan 28, 2001 at 04:38:49PM +0000, Theodore Hong wrote:
< >
> > Limiting the past time interval would require picking an arbitrary cutoff.
> > The best way is to apply a decay factor to old requests, something like:
> >
> > P = #requests in the past hour
> > + (#requests in the previous 2 hrs)/2
> > + (#requests in the previous 4 hrs)/4
> > + (#requests in the previous 8 hrs)/8
> > + ...
>
> Every two hours you halve P. When a request comes in you add a constant.
That is the same thing, just Theo put it as a formula. I think a sense a
cultural conflict here...
--
'DeCSS would be fine. Where is it?'
'Here,' Montag touched his head.
'Ah,' Granger smiled and nodded.
Oskar Sandberg
md98-osa at nada.kth.se
--__--__--
Message: 7
Date: Sun, 28 Jan 2001 16:12:58 -0500
From: Travis Bemann <bem...@bemann.uprizer.com>
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Proposal: algorithm for forgetting documents in
datastore
Reply-To: devl at freenetproject.org
--n8g4imXOkfNTN/H1
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Sun, Jan 28, 2001 at 02:25:01AM -0500, Tavin Cole wrote:
> On Sat, Jan 27, 2001 at 11:53:24PM +0100, Oskar Sandberg wrote:
> > On Sat, Jan 27, 2001 at 04:17:30PM -0600, Mark J. Roberts wrote:
> > > Of course. What prompted me to ask is that dataStoreSize is set quite=
low
> > > by default (1000). A more efficient system could raise that value, wh=
ich
> > > would presumably improve routing.
> >=20
> > The current DataStore is quite lousy (linear search), so I don't really
> > feel comfortable making it much bigger. Tavin is working on a more
> > efficient routing table for 0.4, so when that is in we may wish to make=
it
> > larger, but there are also memory limits (a NodeReference may be around
> > half a kB, but not every entry will be to a unique NodeReference of
> > course).
> >=20
> > It is probably not as simple as bigger =3D better though...
>=20
> What I'm planning to do for 0.4 is logically separate the routing table
> from the document store. The document store will have an upper limit in
> disk space but no (practical) limit on the number of files stored.
>=20
> I'm not sure how the routing table will be constrained -- as Oskar was
> saying it's not so simple as just restricting the number of entries b/c
> the memory and disk space usage will be determined by the number of
> unique NodeReferences as well as the number of keys in the routing
> table. However it will be a red-black BST so it can definitely be
> larger than 1000.
One idea would be to have a non-constrained routing table that is
stored on disk as just a bunch of unsorted entries, and in memory as a
hash which can have multiple items per bucket. That is what is in
(currently paused development) nfreenetd, and I think that is a better
idea than a fixed size routing table or a tree (hashes are faster).
Loading the routing table into memory is still fast, due to the fact
that you are just putting things in a hash. As for closeness, this is
done through a doubly linked list both in memory and on disk (on disk
this is done with file offsets). This is something that I just
thought up now (it isn't yet used in nfreenetd).
--=20
Travis Bemann
Sendmail is still screwed up on my box.
My email address is really bemann at execpc.com.
--n8g4imXOkfNTN/H1
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v0.9.9 (GNU/Linux)
Comment: For info see http://www.gnupg.org
owF1Vk2PG0UQzYdysWRE/kHd1lZsZ3aTEDDaQDZBykZsErIrkHJrz/R4Gs90T6a7
d9b5ARFSJIQ4ciEnDiAkOHDnwC0XxCXiyCGnFRK/APGq2961AxxsWZ6uV6/eq6qe
L7rnz567ePjRq1/Kj3/4/OyLY3dm7J49umW0k9oND+a1HJOTR+5yXQql36W0EI2V
btvbobCpUt3O8uxtZWtjlVNGj0npUml5+vCgEdrmshl+oFOTKT0d02NvnMyGdaO0
E5MSh7ud+5r2vR7QXaFp6+0BbSXJJglHydZ469o42by5R8PkWpIM6EAcKk23TCmp
bQA07nZuEIcLtwi/fhq+uTm+dmW8dfXBHl1KNjn8vp2JBod1NpHNdAXi/0GSq+PN
6+MryQPm8BaD7IlmRndH9NAAxNk1FODklBoPsUb0SYH4ujFVjYqpkuQMCTsjZcnx
o0w4se9MI/fVE8n/QmHoo5zc7naoNO0SczKnTObCl456qCPpj+gmVQgkmecqVZCa
7Nw6WXHuMqNGKCtjkkNRejmgtgCmSoslZBvO1Y20voINc1IVmB5Kaox3MGoUDm5v
JTHgoJCU+qbhTLeXtJlyoAuu3s6px+ZDX4uvtOgPyBrapczoDUeNFGU5j2C5lCWI
VrlpQgtQJWZIScpR5dOCJmo6lc1oYTaStKYJB4wmEeqOOKfFL0hThAMuJaOrIX9b
SB2FAA7QWs42p1bZgu1A5iC2chGyFA1SD2jiHaIkahT8KYFUSWSeU6kqBdN7gu6Z
TD6U6G6p04g64ePG6yyCFaLMQXi2E/G0cSQPJTBAuWEOZckh3BXktXrs5WuYJo9A
saH6r3myG2qCa5NgIMML9BCMhAb4FWWk7Su3kcU5/HSF8dNiNGKgBUzo0d2NijDo
WgcNDSxbasgZSjNVKbsHY2vRCMedJdc1Z6gcnR6eZCb1VehJ7pIRhe5Z/zMWXwg0
HObN1zXYBWXhEWNlCnNiawERonTUqxuROibSX5w0OmTTvsIcQivKVSltxM9WSuTq
WBzrkbcw7b/Zn1iRGm0dhkdjXodDFjFujFZYRrJiHvt0w0ZEsyL3p95ymyNepRF6
jRx7rkBvcjllKH646Chso9V6vRXTU0aZhHFVIIQtsAbJMP/VNpbZtBLxwr5GYibn
YQpWFAhkWAT4dMe03KA8iMv8AiVlw0kp0hnt7B9wyXiawjRsJKUx/CX3PaPE2eFh
08RbahRXO9ZEJsVi4cR2j75DQj1cVXzdk8XQdjvRULY7qLSUWqAzNLYFyvLaYpXg
zELkQdAUdS4VhiTdTiEw822BJRj4BxIVdqpiA1FIhWEC/4lPZ9KFro1boz3ZHt1O
b7EEUXQtvEXKDIqVpubW7pPOGym1dFlksIsilJ6d7B+xGERsYpYkKCXQtkfAsXwH
rCuAEcQhIFKPubOx6OBcYNE3vAw+NCJbNtp6JO7Wk4XF94pjNzlwQJkPFnBMjoHq
dgK5n7+bGx/gg7a1d4sWxnfoGEFMAarctGE3pKWxqNRCaxyywShserat4SWDgMx4
3ku4FGaor1TAnRhXrPqiT23tLX4woYBHAa5ViODBhs857kfbD87EE9ZUMlCMCu8G
8gzADBzWClqspR5vFcuX0Bz3azANFE6s6oc+HQ7DtsALyyGAd2SFbdjt7EudVUKV
pyLatJEtEIANwhV63xwBYA9LPRwUWYYdENjFKw+WM9b78kimdTrCtTf67L3zF87y
+9byBeziua++P/P8t6fnzI3jl0++/OneN29efvrH3xs7z848f/mO+PPCnePf//o1
+fbHS68evXHvxdf/AA==
=ENEu
-----END PGP SIGNATURE-----
--n8g4imXOkfNTN/H1--
--__--__--
Message: 8
Date: Sun, 28 Jan 2001 16:15:03 -0500
From: Travis Bemann <bem...@bemann.uprizer.com>
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Proposal: algorithm for forgetting documents in
datastore
Reply-To: devl at freenetproject.org
--dc+cDN39EJAMEtIO
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Sun, Jan 28, 2001 at 02:16:21PM +0000, Theodore Hong wrote:
> "Mark J. Roberts" <mjr at statesmean.com> wrote:
> > On Sat, 27 Jan 2001, Oskar Sandberg wrote:
> > > Either way is just as screwed up, and at least the former is the natu=
ral
> > > default and doesn't allow strategic techniques for making files live
> > > longer.
> > <>
> > > Perhaps a compromise that sets VAD =3D 1/f(size) where f is some less=
than
> > > linear function is a possible compromise.
> >=20
> > I tend to favor Ian's model because I'm a proponent of segmentation (ev=
en
> > slightly redundant segmentation), and Ian's model encourages such
> > segmentation. However, I certainly wouldn't want to see it pushed to
> > extremes, where data must be inserted as tiny segments to survive. But I
> > think we should discourage large (>1M) files. Looks like a compromise is
> > necessary.
>=20
> Historically, this tension between large and small resulted in the Great
> Compromise of 1787 in the United States. The rights of small states were
> preserved by equal representation by state in the Senate, while those of
> large states were preserved by proportional representation by size in the
> House. Perhaps we should adopt something similar.
That's a different matter, and does government work at all?
--=20
Travis Bemann
Sendmail is still screwed up on my box.
My email address is really bemann at execpc.com.
--dc+cDN39EJAMEtIO
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v0.9.9 (GNU/Linux)
Comment: For info see http://www.gnupg.org
owFtVUFvG0UUTlohoUgW9IgQ0lMuSYTt2kEilWnc0qaiqYgaSKh6Qox3n71T785s
5s16YyQO3JAQhQvn0DsHTvTCkQMS4swJCQmQoP8BiW92vY0j4YO1nn3v+9573/fG
X7Uur166Mnvvrx/TB989Xv35mV8Z+M9/vW2NZ+M7x/OcB+T51F/NU6XNWxQlygn7
3UI6SiKtW2tN7J6W3Ir22poBaZNqw+cvj50yMmbXuWMiG2szGdBJYT3Hndxp49Uo
RXBr7b6ho8K06Z4ytH2tTdu9Xp+Up972oP/mYLt/eECv9/Bp03HCNraO6a41Eyod
sAattSGtHyg3pXtdet+O2HlZp+vZI3dTvPIsGSvTjWw2XEoYUiBVHmQ7NS8423Rf
psrh3MSAmVyIH9Id7RN2VKo5aaFHhXhSQhI5LjmmIm8T8kLdKSu8QzCNrcuQgvDw
yyhf7LbWnEobyJjHqkh9lRhbFrOB5zS1JYl3qH2iI+gQJUafFCwBjjI1xSBprFMc
pHrGDVaKkbDr1j+vD5vjQ3aJyoUUYQS5s5kWRjUoE4IKPXh7j3bf2KP+1fGm6I95
i0o0icpD0WIzRjciqDrkmOdckBmDGhcmCsqHWEXwgWhIukRUV7O73asT99EMOvWW
xmqGXvaV2RDKbMwpjThSBUrb38gClrO5NfAQ2TEKnWR4VBXVJs9QDS9KkVRPEp/O
yXFcmFgZfyF6q9ZkmYdhxcKpCaYnRZQsYJZyujBXyTN2bdQbwU3YABCUtkjjoE8Z
SNCCMJP2lBeScOipRsLSOM5Y2os5xsoryoJXRgg3AjxEwzdem3nDKxVe4WaQs0u3
Ck/7NZpPtJlSySRJoKdYy6J6SpXD9+awf7BVm6FL71o7DZ6Y8kW1tdRohiNoqdw8
yLIQ5a4Wb90/OoLt5u1AKEEkCaMesS+ZzYIqDFIyhGHWAtOiDW0qX7/jWPkAdvuc
E7L1d67tNCEfGB0SjqqF7FJYZHJBOqkErmDrbUW3rvJ0Dhp2M2SN5sQnhQrE1WFj
BZxXOQ3JEWPDOEwe88CJreoIWHUHSwQX0Su3uYD5/yTYiwVHNTILn6KHZrPO5VGx
zX21NUG4CRIzDepuuOSOsXIbYU1iPcaVGKydKe+DzZrtp4mF7UxwBOyGC01Vt8GN
kN7pVILhRp1BoVucKYMVQMdxpnRaLavXYYjPryNC8dmcRvYU/AcYYRWo4hjdSUiA
atAcKgOrZW7yKUd5FC7K7mc3Lr+wGv4Rmr+IK5de/GHl7OEv33/09fD3w2//zB+e
lf9uvJSsP1s5+/DL1z45fPXTJ3//9vSnl195/OSLb57+8R8=
=mZNX
-----END PGP SIGNATURE-----
--dc+cDN39EJAMEtIO--
--__--__--
Message: 9
Date: Sun, 28 Jan 2001 17:23:12 -0500
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Proposal: algorithm for forgetting documents in
datastore
From: "Scott G. Miller" <scgmi...@indiana.edu>
Reply-To: devl at freenetproject.org
--0F1p//8PRICkK4MW
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Sun, Jan 28, 2001 at 10:24:42PM +0100, Oskar Sandberg wrote:
> On Sun, Jan 28, 2001 at 12:08:04PM -0500, Scott G. Miller wrote:
> > On Sun, Jan 28, 2001 at 04:38:49PM +0000, Theodore Hong wrote:
> < >=20
> > > Limiting the past time interval would require picking an arbitrary cu=
toff.
> > > The best way is to apply a decay factor to old requests, something li=
ke:
> > >=20
> > > P =3D #requests in the past hour
> > > + (#requests in the previous 2 hrs)/2
> > > + (#requests in the previous 4 hrs)/4
> > > + (#requests in the previous 8 hrs)/8
> > > + ...
> >=20
> > Every two hours you halve P. When a request comes in you add a constan=
t.
>=20
> That is the same thing, just Theo put it as a formula. I think a sense a
> cultural conflict here...
I was suggesting an implementation of his formula. Not disagreeing in
any way.
--0F1p//8PRICkK4MW
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQE6dJvPr9IW4v3mHtQRAhkNAJ0U/tUFDHb/xY/64NV7lIzIZTKuCgCfdyep
InIJ3464timAtiY6x/EarmI=
=8voF
-----END PGP SIGNATURE-----
--0F1p//8PRICkK4MW--
--__--__--
Message: 10
Date: Sun, 28 Jan 2001 18:51:38 -0500
From: Tavin Cole <ta...@mailandnews.com>
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Proposal: algorithm for forgetting documents in
datastore
Reply-To: devl at freenetproject.org
On Sun, Jan 28, 2001 at 04:12:58PM -0500, Travis Bemann wrote:
> > What I'm planning to do for 0.4 is logically separate the routing table
> > from the document store. The document store will have an upper limit in
> > disk space but no (practical) limit on the number of files stored.
> >
> > I'm not sure how the routing table will be constrained -- as Oskar was
> > saying it's not so simple as just restricting the number of entries b/c
> > the memory and disk space usage will be determined by the number of
> > unique NodeReferences as well as the number of keys in the routing
> > table. However it will be a red-black BST so it can definitely be
> > larger than 1000.
>
> One idea would be to have a non-constrained routing table that is
> stored on disk as just a bunch of unsorted entries, and in memory as a
> hash which can have multiple items per bucket. That is what is in
> (currently paused development) nfreenetd, and I think that is a better
> idea than a fixed size routing table or a tree (hashes are faster).
> Loading the routing table into memory is still fast, due to the fact
> that you are just putting things in a hash. As for closeness, this is
> done through a doubly linked list both in memory and on disk (on disk
> this is done with file offsets). This is something that I just
> thought up now (it isn't yet used in nfreenetd).
Well, dude, the constraint we have to live under is memory & disk usage..
naturally the routing table should grow as big as possible with that in mind.
Plus, using a hashtable + d.l.l. for closeness won't work because you need
to be able to find the closest keys for a key that doesn't exist in the
hashtable.
--
// Tavin Cole
--__--__--
Message: 11
Date: Sun, 28 Jan 2001 19:03:57 -0500
From: Tavin Cole <ta...@mailandnews.com>
To: devl at freenetproject.org
Subject: Re: [freenet-devl] Proposal: algorithm for forgetting documents in
datastore
Reply-To: devl at freenetproject.org
On Sun, Jan 28, 2001 at 04:22:53PM +0100, Oskar Sandberg wrote:
> > > I haven't really been able to make up my mind on this. Should I reply
> > > with a RequestFailed on Inserts for data larger than whatever the nodes
> > > limit is?
> >
> > But even if the node can't store the data b/c it is too large, it could
> > stream it to an upstream node which can store the data, right?
>
> On a request we have to way to know the size of the data until it is
> really too late to reject the request, so for that we would simply have
> the node tunnel the data but not cache it. We could do the same for
> Inserts, but people are concerned that someone would insert a large
> document and not know that all the nodes on the route failed to cache it,
> and on Inserts we do have opportunity reject the InsertRequest right away
> if the data is to big (we would have to add a InsertSize field to it and
> then make sure that is >= the actual DataLength, but that is trivial).
I'm no expert on the details of the Freenet protocol, but suppose node/client
A inserts a file and the insert goes in a chain all the way to node Z. AFAIK
you are going to make node Z responsible for sending back a StoreData message
all the way to A instructing each node to commit the file. Can't you also
send back info indicating whether the file was actually stored? E.g., suppose
node M receives the StoreData message and it includes a status indicator that
the file has not been stored by node N or any node upstream of N. M can say,
I've got room for it, store it, and change the status indicator to positive
before passing the StoreData to node L. Node/client A will learn whether any
node stored the file, but not who of course.
Have I missed some crucial detail?
--
// Tavin Cole
--__--__--
_______________________________________________
Devl mailing list
Devl at freenetproject.org
http://www.uprizer.com/mailman/listinfo/devl
End of Devl Digest
