Send Freenet-dev mailing list submissions to
freenet-dev at lists.sourceforge.net
To subscribe or unsubscribe via the web, visit
http://lists.sourceforge.net/mailman/listinfo/freenet-dev
or, via email, send a message with subject or body 'help' to
freenet-dev-request at lists.sourceforge.net
You can reach the person managing the list at
freenet-dev-admin at lists.sourceforge.net
When replying, please edit your Subject line so it is more specific than
"Re: Contents of Freenet-dev digest..."
Today's Topics:
1. Re: KHK authentication (Greg Titus)
2. This is NOT SPAM (Wicked Night)
3. Re: Promoting metadata (Daniel Phillips)
4. Re: Promoting metadata (Oskar Sandberg)
5. Re: KHK authentication (Michael Wiktowy)
6. Re: Promoting metadata (Daniel Phillips)
7. Re: KHK authentication (Oskar Sandberg)
8. Re: Meaningless hits on metadata (Alex Barnell)
9. Re: 0.3 Todo List (Alex Barnell)
10. Re: URI form? (Scott G. Miller)
11. Re: Proposed URI spec (was Re: URI form?) (Scott G. Miller)
--__--__--
Message: 1
To: freenet-dev at lists.sourceforge.net
Subject: Re: [Freenet-dev] KHK authentication
Date: Mon, 26 Jun 2000 01:41:54 -0700
From: Greg Titus <t...@omnigroup.com>
Reply-To: freenet-dev at lists.sourceforge.net
Oskar wrote:
> I don't think either version helps against attacks of generating a
> close key - after all a brute force attack to create a key closer
> then any other on freenet to a given key can never be higher in order
> then the number of documents on freenet, which we cannot count on
> being cryptographically safe.
Actually you need to generate enough close keys to saturate the
target nodes' caches (which throws in another reasonably large
constant factor for the attacker to overcome).
Just to see what this looked like, I wrote a little program to (very
naively) brute force attack Hal's system. If
KHK=hash(hash(keystring)), and the data includes a Verifier =
hash(keystring), then you can try random Verifier values, hash them,
and see if the result is close to the KHK that you are trying to
crowd out. (It doesn't matter whether the Verifier is the hash of any
real keystring or not - the data you are inserting is going to be
bogus anyway, but your Verifier and KHK has to match up or presumably
no node will do your insertion.)
My test starts with a random 160 bit KHK to compare against, then in
a loop, generates a random Verifier, hashes it, and finds the bit
difference between the hash and that original KHK. If it is closer
than anything currently found so far, I print out a message to that
effect. I'm using SHA1 for the hashing, and with the totally
unoptimized code on my pretty slow G3/233 laptop, I'm going through a
little under 2000 keys/second.
1 tries, 77 bits: 0.000557 seconds
3 tries, 74 bits: 0.001785 seconds
4 tries, 68 bits: 0.002030 seconds
118 tries, 60 bits: 0.024733 seconds
349 tries, 57 bits: 0.056413 seconds
10270 tries, 55 bits: 1.500579 seconds
79662 tries, 53 bits: 11.475965 seconds
144997 tries, 51 bits: 20.875274 seconds
1547879 tries, 50 bits: 226.920472 seconds
1990936 tries, 42 bits: 313.353716 seconds
It's been about half an hour now without finding a key closer than
42 bits difference, which makes this sort of attack more difficult
than I had first expected using Hal's version of KHK's. With the
original version of non-verified KHK's, of course, it is trivial to
generate close keys because there is no verification step at all.
This sort of attack is very dangerous because it uses the strengths
of Freenet (the automatic routing) against it. It is a way to use
Freenet itself to seek out and destroy the information you want to
censor. If we allow key types like unverified KHKs (and really, even
if we don't allow them) we will probably have to come up with some
kind of node policy on insertions that keeps a prolific publisher
from drowning out other nearby keys.
--Greg
--__--__--
Message: 2
Reply-To: "Wicked Night" <w2n at mail.com>
From: "Wicked Night" <w...@mail.com>
To: <freenet-dev at lists.sourceforge.net>
Date: Mon, 26 Jun 2000 19:20:52 +0800
boundary="----=_NextPart_000_0008_01BFDFA3.A4D2E090"
Subject: [Freenet-dev] This is NOT SPAM
Reply-To: freenet-dev at lists.sourceforge.net
This is a multi-part message in MIME format.
------=_NextPart_000_0008_01BFDFA3.A4D2E090
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Greetings caught an article about your product & was interested in =
finding a PC version to run on Windows 98, NT4.0 or 2000
I too believe in the freedom of information & would like to host a =
server for your network...=20
I am currently located in Southern Cali & will soon be acquiring Cable =
access hopefully before August of this year...
I've a strong background in Networking Essentials & NT Admin I would =
further like to offer any assistance I might be able to provide you... =20
I have a network of experts on my end with various other areas of =
experience including Linux, Web Design, & PC Construction...
I can also provide translations between English & the following:=20
Spanish, French, Portuguese, Italian, Arabic & Chinese through the use =
of my various contacts worldwide...
I look forward to your response...=20
Laterz,
WN
------=_NextPart_000_0008_01BFDFA3.A4D2E090
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content=3D"text/html; charset=3Diso-8859-1" =
http-equiv=3DContent-Type>
<META content=3D"MSHTML 5.00.2920.0" name=3DGENERATOR>
<STYLE></STYLE>
</HEAD>
<BODY bgColor=3D#ffffff>
<DIV><FONT face=3DArial size=3D2>Greetings caught an article about your =
product=20
& was interested in finding a PC version to run on Windows 98, NT4.0 =
or=20
2000</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>I too believe in the freedom of =
information &=20
would like to host a server for your network... </FONT></DIV>
<DIV><FONT face=3DArial size=3D2>I am currently located in Southern Cali =
& will=20
soon be acquiring Cable access hopefully before August of this=20
year...</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>I've a strong background in Networking =
Essentials=20
& NT Admin I would further like to offer any assistance I might be =
able to=20
provide you... </FONT></DIV>
<DIV><FONT face=3DArial size=3D2>I have a network of experts on my =
end=20
with various other areas of experience including Linux, Web Design, =
&=20
PC Construction...</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>I can also provide =
translations between=20
English & the following: </FONT></DIV>
<DIV><FONT face=3DArial size=3D2>Spanish, French, Portuguese, =
Italian, Arabic=20
& Chinese through the use of my various contacts=20
worldwide...</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>I look forward to your response... =
</FONT></DIV>
<DIV> </DIV>
<DIV><FONT face=3DArial size=3D2>Laterz,</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>WN</FONT></DIV></BODY></HTML>
------=_NextPart_000_0008_01BFDFA3.A4D2E090--
--__--__--
Message: 3
From: Daniel Phillips <phill...@bonn-fries.net>
To: freenet-dev at lists.sourceforge.net
Subject: Re: [Freenet-dev] Promoting metadata
Date: Mon, 26 Jun 2000 12:47:50 +0200
Reply-To: freenet-dev at lists.sourceforge.net
On Mon, 26 Jun 2000, hal at finney.org wrote:
> Daniel writes:
> > Design Goal: To allow a search hit on metadata item to promote that item in
> > a
> > server's LRU list, but only if the underlying data is actually requested.
[...]
> The other mistake you are making, which is more fundamental, is that in
> most cases the client and the server will be on the same computer, and
> controlled by the same person.
I didn't miss that, I just didn't note it as an assumption. Though it's an
interesting point, It's not really relevant to the discussion. Even when
running on the same computer the server is still more trustworthy than the
client in the same way that inetd is more trustworthy than <pickyourclient>.
There will be fewer variations on servers than clients, so the behavior of the
server is much more predictable than the client. Further, it's likely that the
clients will be running as normal user programs and thus be more vulnerable to
attack. For these reasons anything that looks like routing or policy should
run in the server, not the client.
> [...]
> This means that you cannot trust a random node any more than a random
> client, and therefore you might as well do this in the client if you
> are going to do it at all. Nodes are no more trustworthy than clients.
There is not much in this case that you have to worry about trusting. The
worst that can happen is a node learns that you accessed an item you
previously searched for - not very informative. Even if trust was an issue, the
node you are trusting is running on *your* machine.
[begin actual content]
> So here is the meat of the idea: create a message which boosts metadata.
> And perhaps, don't boost metadata when it gets returned as part of
> a search.
Yes, and also the search result needs to include metadata keys or some other
means of retracing the route (see below).
It is also tempting to boost metadata *a little* simply because it was found
in a search. Right now Freenet doesn't have any concept of "a little boost" -
it's all the way to the top of the list or nothing. So I left that out for
now; it's easy enough to add later and doesn't address the main issue of how to
boost metadata that actually gets used to find real data.
> We had a long debate earlier about a very similar proposal, although that
> was in the context of ordinary data. It was suggested that rather than
> automatically promoting fetched data, it would be better to separate the
> promotion from the fetch. Only after you had fetched it and verified
> that the data was what you wanted would you send the promotion message.
> This is to protect against the Popescu attack where he posts messages
> under invalid labels.
>
> I'm not familiar enough with the various proposals to say whether this
> is a good idea in the case of metadata. The one thing I'd add is that
> you don't need a metadata key to do it, and in fact that would be the
> wrong way. You probably want to retrace the path you followed when you
> did the search. For that the metadata-boost message should be routed
> in exactly the same way as the search was, using the original search
> terms and the same fuzzy closeness relationships.
Whatever way works. I considered the strategy of trying to replicate the
original search path but indentified a number of annoying issues, not the
least of which is code complexity. For example, how do you prune the search
path so it goes only towards the one piece of metadata you actually used?
Scanning through the (many) postings on metadata, it seems to me that metadata
has keys, and that these keys can be used in the normal Freenet way. In fact,
it would require extra work to make an exception. So simply returning the key
shouldn't be a lot of trouble.
There is another way of retracing the metadata route that would have zero
impact on message size: every node remembers enough routing data when a search
result passes through it so that the route can be followed in reverse, as with
data requests but in the opposite direction. The disadvantage of this is
that the work is multiplied by the number of search results, and most of that
routing data will never be used. An expiry policy would be needed for the
routing data. Probably just using the metadata key is simpler.
--
Daniel
--__--__--
Message: 4
From: Oskar Sandberg <md98-...@nada.kth.se>
To: freenet-dev at lists.sourceforge.net
Subject: Re: [Freenet-dev] Promoting metadata
Date: Mon, 26 Jun 2000 15:49:09 +0200
Reply-To: freenet-dev at lists.sourceforge.net
On Mon, 26 Jun 2000, Daniel Phillips wrote:
<>
> I didn't miss that, I just didn't note it as an assumption. Though it's an
> interesting point, It's not really relevant to the discussion. Even when
> running on the same computer the server is still more trustworthy than the
> client in the same way that inetd is more trustworthy than <pickyourclient>.
Daniel, I believe we already discussed your difference with the rest of the
security engineering world on this point. As much as you may not accept
everybody elses definition of security, don't think that the rest of the world
is going to accept yours (which as far as I can tell is "most people don't mess
with their daemons").
<>
--
Oskar Sandberg
md98-osa at nada.kth.se
--__--__--
Message: 5
Date: Mon, 26 Jun 2000 10:13:27 -0400
From: Michael Wiktowy <s...@mindless.com>
To: freenet-dev at lists.sourceforge.net
Subject: Re: [Freenet-dev] KHK authentication
Reply-To: freenet-dev at lists.sourceforge.net
> Date: Mon, 26 Jun 2000 08:38:16 +0200 (MET DST)
> From: Oskar Sandberg <md98-osa at nada.kth.se>
> Subject: Re: [Freenet-dev] KHK authentication
>
> But with JJs version (we have to make up an acronym here :-)) ...
Knowing the group's love for TLAs, I chose one for that scheme
when writing up the beastiary of keys located at:
http://freenet.sourceforge.net/index.php?page=keys
JJs version is under SKK (Signed Keyword Key) at the end of the
document. If you hate SKK, make up something better and I'll change
it.
Looking at it now, it looks like I have some corrections anyways.
The SKK reference should be free:SKK at User Defined Keyword
(same with the KHK ref come to think about it) along the lines of
free:SKK@(find stuff here).
Mike
--__--__--
Message: 6
From: Daniel Phillips <phill...@bonn-fries.net>
To: freenet-dev at lists.sourceforge.net
Subject: Re: [Freenet-dev] Promoting metadata
Date: Mon, 26 Jun 2000 16:41:37 +0200
Reply-To: freenet-dev at lists.sourceforge.net
On Mon, 26 Jun 2000, Oskar Sandberg wrote:
> On Mon, 26 Jun 2000, Daniel Phillips wrote:
> <>
> > I didn't miss that, I just didn't note it as an assumption. Though it's an
> > interesting point, It's not really relevant to the discussion. Even when
> > running on the same computer the server is still more trustworthy than the
> > client in the same way that inetd is more trustworthy than <pickyourclient>.
>
> Daniel, I believe we already discussed your difference with the rest of the
> security engineering world on this point. As much as you may not accept
> everybody elses definition of security, don't think that the rest of the world
> is going to accept yours (which as far as I can tell is "most people don't
> mess
> with their daemons").
Goodness, where did that come from? 1) This is not the central point of the
discussion 2) Daemons *are* more secure than normal user programs, especially
when normal users can't get root 3) This is far from the only means of
maintaining security, 4) far from the only means I've discussed and 5) we
discussed no such thing.
Feel free to flame me by email if you must, I don't mind, but I don't think it
belongs on the list.
If you've found a real security hole in what I've proposed, please point it out.
--
Daniel
--__--__--
Message: 7
From: Oskar Sandberg <md98-...@nada.kth.se>
To: freenet-dev at lists.sourceforge.net
Subject: Re: [Freenet-dev] KHK authentication
Date: Mon, 26 Jun 2000 16:54:08 +0200
Reply-To: freenet-dev at lists.sourceforge.net
I think KSK, Keyword Signed Key. It is very vague, but anything more precise
will require more letters. Another option is of course to leave the functional
acronyms and name it an FFK (Furlong-Finney Key, since JJ proposed it and Hal
defined the DSA algorithm).
I want the KSK to be the default form of Keyword generated key, so the old KHKs
should be quickly deprecated (this where Ian, not having followed the
discussion, raises hellfire).
On Mon, 26 Jun 2000, Michael Wiktowy wrote:
> > Date: Mon, 26 Jun 2000 08:38:16 +0200 (MET DST)
> > From: Oskar Sandberg <md98-osa at nada.kth.se>
> > Subject: Re: [Freenet-dev] KHK authentication
> >
> > But with JJs version (we have to make up an acronym here :-)) ...
>
> Knowing the group's love for TLAs, I chose one for that scheme
> when writing up the beastiary of keys located at:
>
> http://freenet.sourceforge.net/index.php?page=keys
>
> JJs version is under SKK (Signed Keyword Key) at the end of the
> document. If you hate SKK, make up something better and I'll change
> it.
>
> Looking at it now, it looks like I have some corrections anyways.
> The SKK reference should be free:SKK at User Defined Keyword
> (same with the KHK ref come to think about it) along the lines of
> free:SKK@(find stuff here).
>
> Mike
>
>
> _______________________________________________
> Freenet-dev mailing list
> Freenet-dev at lists.sourceforge.net
> http://lists.sourceforge.net/mailman/listinfo/freenet-dev
--
Oskar Sandberg
md98-osa at nada.kth.se
--__--__--
Message: 8
Date: Mon, 26 Jun 2000 16:07:01 +0100
From: Alex Barnell <ae...@doc.ic.ac.uk>
To: freenet-dev at lists.sourceforge.net
Subject: Re: [Freenet-dev] Meaningless hits on metadata
Reply-To: freenet-dev at lists.sourceforge.net
Brandon wrote:
>
> > I don't see how. Nodes need the plain-text of values to perform fuzzy
> > matching (and fuzzy routing). Nodes need the plain-text of fields to allow
> > matches to be performed in the case where the request includes just fields
> > A and B, but the node has something matching in its store with fields A, B
> > and C. If the fields were encrypted, there would be no way of knowing it
> > was a match.
>
> If we do either the one insert per field or else the powerset insert then
> it's basically just a KHK. You convert the name, field pairs into strings
> and then hash. It wouldn't work if we fill in missing fields. I'm not
> saying we can encrypt, just that we maybe can encrypt.
>
Yes, we can encrypt if we only allow exact matches. However, I wanted fuzzy
matching, so that a search for (Title=Freenet) will (hopefully!)
return metadata including fields like:
Title=The Art of Freenet
Title=An Introduction to Freenet
Title=Why freenet sucks
Title=FreenetII
in them. This will prevent us using encryption since we need the plaintext.
I think a simulator will show that fuzzy matching allow cases like the
example above to work.
--__--__--
Message: 9
Date: Mon, 26 Jun 2000 16:12:31 +0100
From: Alex Barnell <ae...@doc.ic.ac.uk>
To: freenet-dev at lists.sourceforge.net
Subject: Re: [Freenet-dev] 0.3 Todo List
Reply-To: freenet-dev at lists.sourceforge.net
Brandon wrote:
>
> > I agree, and propose that a preliminary-metadata search facility should
> > be setup in a similar way to the web-key-informing we have now. This will
> > allow us to start developing clients with search facilities, will
> > please end-users, all without us committing ourselves to a potential
> > flawed search mechanism. When we do implement proper search, the change
> > will be transparent to end-users, since the client will look the same.
>
> I'm not sure what you're suggesting. Is this a client or server feature
> which you want to add? Or both?
>
It's just a client feature - sorry for being vague.
I'm suggesting an extension to the key-inform servers we have now. In fact,
we don't even need to change the key-inform servers. For example, if a user
creates metadata like:
Title=Foo
Author=Fred
KHK=text/foo
Then they post a key to the keyinform server under the name
"Metadata:Title=Foo,Author=Fred,KHK=text/foo". Then clients could include
a preliminary search feature, by simply downloading the key lists from the
key servers, and searching through them for Metadata. This will show users,
and the press, that Freenet is aiming for searchability. It will also
help us determine the kinds/length of metadata people are likely to insert.
All this without changing the node.
--__--__--
Message: 10
Date: Mon, 26 Jun 2000 12:13:30 -0500
To: freenet-dev at lists.sourceforge.net
Subject: Re: [Freenet-dev] URI form?
protocol="application/pgp-signature"; boundary="6c2NcOVqGQ03X4Wi"
From: "Scott G. Miller" <scgmi...@indiana.edu>
Reply-To: freenet-dev at lists.sourceforge.net
--6c2NcOVqGQ03X4Wi
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
> No, it's ceiling(160/6) =3D 27 characters (possibly with some '=3D' paddi=
ng,
> but I don't think we need that).
Yes, quite a long time ago we realized that my math sucked there. Thanks=20
for bringing it up again. :)
> This can also be done in 27 characters for both CHKs and SVKs, by deriving
> the encryption key (and IMHO a MAC key is also required for CHKs) using
> something like:
>=20
> key =3D SHA-1("make CHK encryption key:" || CHK)
>=20
> and then using a different hash for routing:
>=20
> locator =3D SHA-1("make CHK locator:" || CHK)
Yes, but that shares too much entropy. I believe the method currently is
to use the hash of the document as the encryption key, then the hash of
the encrypted document as the routing key (which also allows verification)
> iQEVAwUBOVaXDTkCAxeYt5gVAQGbpwgAqjOrIqcPgox6JCEBv+lkPLBiyQ6BQPhJ
> bla+6IQ9banAQZmIIDLLzQ3nVDyNMp3LJUGkQh3OIR4aIrU/NduCYc3waDgOfcl7
> l91wF+SGrBaT3xhyJy5KbjPr+CApJ5voH59eMsbAaXuKdorxRkD3h5Y8fGKHmZ6H
> J5uQC7T0M6L2f00EXiwA2mikpxD5gsugvIKbZpOR90S/krMALwPAzEq/wYSeDrPp
> W4ZgGo5J92g1/LE00+ujwM6RmeYRjg50x02ZuWGwCQd6JG/+adS3zrv7W/M5AYDu
> jK6/Jw3d89RuRTWabUSrGQTQEmgQfML++fyhKzJEakMj54VJA6Wyjg=3D=3D
> =3Djm0Q
> -----END PGP SIGNATURE-----
>=20
> _______________________________________________
> Freenet-dev mailing list
> Freenet-dev at lists.sourceforge.net
> http://lists.sourceforge.net/mailman/listinfo/freenet-dev
>=20
--6c2NcOVqGQ03X4Wi
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.1 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQE5V486pXyM95IyRhURAutzAKDRxnHAmcHGqWTqUfqfYsysP5u3wACcDHR2
X6rHhXDzC4MWqw79/aGlImU=
=pu83
-----END PGP SIGNATURE-----
--6c2NcOVqGQ03X4Wi--
--__--__--
Message: 11
Date: Mon, 26 Jun 2000 12:14:18 -0500
To: freenet-dev at lists.sourceforge.net
Subject: Re: [Freenet-dev] Proposed URI spec (was Re: URI form?)
protocol="application/pgp-signature"; boundary="Pd0ReVV5GZGQvF3a"
From: "Scott G. Miller" <scgmi...@indiana.edu>
Reply-To: freenet-dev at lists.sourceforge.net
--Pd0ReVV5GZGQvF3a
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
I'd appreciate it if you would go ahead and finish reading the archives
(skimming them at least). This has already been discussed, and I'm
starting to get major deja-vu.
> > >
> > > freenet:<Keytype>/<Key>
>=20
> '/' probably isn't a good idea for the separator, because it occurs in
> base-64 output.
>=20
> > > Where Key is dependent on the keytype:
> > >
> > > KHK/<Base 64 key>
> > > CHK/<Base 64 locator-key>/<Base 64 crypto key>
> > > SVK/<Base 64 key>
> >=20
> > I agree, except the SVK requires a crypto key as well, just like the
> > CHK...
>=20
> Both SVKs and CHKs can be represented in only the length of a hash result,
> by deriving the crypto key from the hash.
>=20
> > > I propose using base-64 only because it keeps keys shorter.
> > > Are base-64 numbers valid URLs?
>=20
> Yes (with some caveats; see below). The relevant specs are RFC 2396
> and RFC 2718. RFC 2718 is particularly worth reading; it's effectively
> a "how to define your own URI scheme" cookbook.
>=20
> Re the character issue, here are some extracts from RFC 2396:
>=20
> # reserved =3D ";" | "/" | "?" | ":" | "@" | "&" | "=3D" | "+" |
> # "$" | ","
> #
> # The "reserved" syntax class above refers to those characters that are
> # allowed within a URI, but which may not be allowed within a
> # particular component of the generic URI syntax; ...
>=20
> I.e. it would be fine to use '@' to separate the key type from the rest
> of the URI, and also to disallow '@' (unless escaped as %2c) within a
> KHK keystring. ':' as the separator is another possibility, but I'll
> assume '@' below (ideally it should be something that is likely to occur
> only rarely in KHK keystrings).
>=20
> [I think people have been expanding KHK as "keyword hash key", but
> since it's an arbitrary Unicode string rather than a word, I prefer
> "keystring hash key".]
>=20
> The "reserved" characters '+', '=3D', and '/' are used in base-64, but
> as long as the URI scheme definition says that they can be used in a
> particular URI component, that does not present a problem as far as
> RFC 2396 is concerned.
>=20
> However, I think it would be useful to remove the '=3D' characters from
> the end of the encoded hash, because they are liable to cause nasty
> interactions with quoted-printable content-transfer-encoding in some
> mail clients, if Freenet URIs are pasted into e-mails. The '=3D'
> characters aren't necessary in order to unambiguously decode base-64,
> anyway. I'll call a version of base-64 that does not use '=3D' characters
> "modified-base-64".
>=20
> More from RFC 2396:
>=20
> # unreserved =3D alphanum | mark
> # mark =3D "-" | "_" | "." | "!" | "~" | "*" | "'" | "(" | ")"
> #
> # Data must be escaped if it does not have a representation using an
> # unreserved character; ...
> #
> # [Excluded characters:]
> #
> # control =3D <US-ASCII coded characters 00-1F and 7F hexadecimal>
> # space =3D <US-ASCII coded character 20 hexadecimal>
> # delims =3D "<" | ">" | "#" | "%" | <">
> ...
> # Other characters are excluded because gateways and other transport
> # agents are known to sometimes modify such characters, or they are
> # used as delimiters.
> #
> # unwise =3D "{" | "}" | "|" | "\" | "^" | "[" | "]" | "`"
> #
> # Data corresponding to excluded characters must be escaped in order to
> # be properly represented within a URI.
>=20
> As far as internationalisation is concerned,
> http://www.w3.org/International/O-URL-and-ident.html says that
> UTF-8 should be used for new URI schemes, which seems very reasonable.
>=20
> URI references are sequences of ISO 10646 characters, so the grammar
> below is expressed in terms of characters. However, it could just as
> easily be viewed in terms of octets of the UTF-8 encoding (in that
> case, the 'non-ascii' term only includes sequences of octets that
> are valid UTF-8 encodings of a character with code point >=3D 0x80).
>=20
> Parsing the URI reference can be done either by converting to UTF-8
> first and parsing octet-sequences, or by parsing character sequences
> in some other encoding of ISO 10646. These two methods will give the
> same result, *provided* that % escapes of values >=3D 0x80 are treated
> as if they were encoded as UTF-8. It is not valid to represent a
> single non-ASCII character partly using a % escape and partly using
> octets >=3D 0x80. At least characters with code points between 0 and
> 0x10FFFF inclusive should be representable.
>=20
> I've included room in the grammar for queries and fragments, as used
> in HTTP URLs. Designing a query mechanism would obviously require a
> lot of discussion first (not least security issues with server-side
> scripts), however reserving the characters that would be needed to
> support this is harmless in itself. Fragments are very straightforward;
> they behave exactly as in HTTP/HTML, are only interpreted by the client,
> and never appear in any protocol messages (hashed or otherwise).
>=20
> freenet-uri-reference
> =3D f r e e n e t ":" stem ["?" query] ["#" fragment]
> f =3D "f" | "F"
> r =3D "r" | "R"
> e =3D "e" | "E"
> n =3D "n" | "N"
> t =3D "t" | "T"
> stem =3D keystring | chk | svk
> query =3D uristring
> fragment =3D uristring
> keystring =3D uristring
> chk =3D c h k "@" hash
> svk =3D s v k "@" hash
> c =3D "c" | "C" | escaped<"c" | "C">
> h =3D "h" | "H" | escaped<"h" | "H">
> k =3D "k" | "K" | escaped<"k" | "K">
> s =3D "s" | "S" | escaped<"s" | "S">
> v =3D "v" | "V" | escaped<"v" | "V">
> hash =3D *(base64char | escaped<base64char>)
> base64char =3D alphanum | "+" | "/"
> uristring =3D *(urichar | escaped<any>)
> any =3D <any ISO 10646 character>
> urichar =3D "!" | "$" | "&" | "'" | "(" | ")" | "*" | "+" | "," |
> "-" | "." | "/" | ":" | ";" | "=3D" | "_" | "~" |=20
> alphanum | non-compliant | non-ascii
> non-compliant =3D "{" | "}" | "|" | "\" | "^" | "[" | "]" | "`" | space
> excluded =3D control | "<" | ">" | doublequote | "?" | "#" | "%" =
| "@"
> space =3D <code point 0x20>
> doublequote =3D <code point 0x22>
> non-ascii =3D <code points >=3D 0x80>
> control =3D <code points 0 to 0x1F and 0x7F>
> hex =3D digit | ("A"-"F") | ("a"-"f")
> digit =3D "0"-"9"
> alphanum =3D alpha | digit
> alpha =3D ("A"-"Z") | ("a"-"z")
> escaped<term> =3D 1*("%" hex hex)
> where the hex digit-pairs must expand to a valid
> UTF-8 encoding of a character from 'term'.
>=20
> (This looks more complicated than it is; ignoring queries and
> fragments, the format is basically just "freenet:keytype at hash" or
> "freenet:keystring".)
>=20
> The strings "freenet:", "chk" and "svk" are case-insensitive. It
> is valid (but not recommended) for the letters in "chk" and "svk"
> to be %-encoded, but "freenet:" and '@' MUST NOT be %-encoded.
> Case MUST be preserved in the keystring and the hashes.
>=20
> Technically, the categories of characters labelled 'non-compliant'
> and 'non-ascii' are not allowed in URIs without %-escaping,
> according to RFC 2396. However, there is no reason why these
> characters shouldn't be used in cases where the URI transport or
> file format is known to be able to represent them (for example, in
> the Freenet protocol, or HTML with the charset specified as UTF-8
> using a META tag). This is worthwhile because it is likely that
> KHK keystrings will regularly contain spaces and non-ASCII
> characters, and always %-encoding those characters would result in
> an expansion of up to 3 times.
>=20
> In any case, characters from the 'non-compliant' and 'non-ascii'
> categories MUST be accepted in freenet: URI references, even if
> they are not generated.
>=20
>=20
> To interpret a freenet: URI reference,
>=20
> 1. Convert to a character encoding suitable for parsing, e.g.
> UTF-8. If the URI reference was found in an HTML/XML/SGML
> document, this includes expanding '&' entity references.
>=20
> 2. Extract the scheme-specific part (i.e. after "freenet:").
>=20
> 3. Parse the scheme-specific part into a stem, an optional query
> string, and an optional fragment, according to the syntax
>=20
> stem ["?" query] ["#" fragment]
>=20
> 4. The query string will at some point need to be unescaped; however,
> that should only be done by whatever software interprets the
> query, *not* by the client. Within a query string, the characters
> ";", "/", "?", ":", "@", "&", "=3D", "+", ",", and "$" are reserved
> (this is the same as in HTTP).
>=20
> 5. Unescape all % escape sequences in the fragment (there are no
> reserved characters for fragments).
>=20
> 6. Test whether the stem contains an unescaped '@' character.
> If it does,
> 6a. Parse the stem into a keytype and a keytype-specific string,
> splitting at the first '@'.
> 6b. Unescape all % escape sequences in the keytype and in the
> keytype-specific string.
> 6c. If the keytype is a case-insensitive match for UTF-8 "chk",
> the URI represents a CHK. Decode the keytype-specific string
> as modified-base-64, and finish.
> 6d. If the keytype is a case-insensitive match for UTF-8 "svk",
> the URI represents an SVK. Decode the keytype-specific string
> as modified-base-64, and finish.
> 6e. The keytype is unrecognised; finish with an error.
>=20
> 7. The stem does not contain a '@' character, therefore the URI
> represents a KHK keystring. Unescape all % escape sequences in
> the keystring (there are no reserved characters), and finish.
>=20
>=20
> The following are errors:
> - a '%' not followed by two hex digits,
> - a sequence of %-escapes that includes values >=3D 0x80, but does
> not form a complete UTF-8 encoding of a character,
> - an unrecognised key type (this probably indicates that the URI
> is for a future protocol version that the client doesn't
> recognise),
> - a keystring that is not valid UTF-8,
> - a CHK or SVK hash that cannot be decoded according to
> modified-base-64.
>=20
> Characters in a KHK keystring from the 'excluded' category MUST
> always be %-escaped. '/' and '+' in modified-base-64 encodings
> SHOULD NOT be %-escaped.
>=20
> These rules achieve several desirable goals:
> - all possible keystrings can be represented,
> - most keystrings can be written without any escaping,
> - KHK URIs are easy to type and have no extraneous characters,
> - there is an unambiguous procedure for distinguishing KHKs, CHKs,
> SVKs, and any future key types.
>=20
> When encoding a KHK URI for transmission through some hostile medium,
> for example e-mail, characters from the 'non-compliant' category MUST
> be %-escaped, and it may also be beneficial to escape other
> characters that could cause problems with some mail clients (e.g. '=3D').
>=20
> Implementations can impose a limit on the maximum length of a
> keystring, but this MUST be at least 1024 bytes.
>=20
> Only the "GET" operation is defined on freenet: URIs.
>=20
>=20
> Examples:
>=20
> "freenet:chk at 8HX6+0gOWem7/K26XapVwjM9/us"
> a typical CHK URI, with a 160-bit SHA-1 hash.
>=20
> "freenet:chk at 8HX6%2b0gOWem7/K26XapVwjM9/us"
> same as above (the '+' has been encoded as "%2b"; this is valid
> but not recommended).
>=20
> "freenet:svk%2c123"
> a KHK URI for "svk at 123".
>=20
> "freenet:abc%xyz"
> invalid, because "xy" are not hex digits.
>=20
> "freenet:abc%25xyz"
> a KHK URI for "abc%xyz".
>=20
> "freenet:%53vk at 8HX6+0gOWem7/K26XapVwjM9/us"
> an SVK URI ("%53" is unescaped to 'S' before parsing the keytype,
> and the match with "svk" is case-insensitive).
>=20
> "freenet:%E6%97%A5%E6%9C%AC%E8%AA%9E"
> a KHK URI for the Japanese word that transliterates as "nihongo"
> (example from RFC 2279). A client should ideally display this as
> three Japanese characters, and allow it to be typed using a Japanese
> input method.
>=20
> "freenet:" <E6 97 A5 E6 9C AC E8 AA 9E> (in UTF-8)
> same as above, but represented in only 17 bytes rather than 35.
>=20
> "freenet:%E6" <97 A5 E6 9C AC E8 AA 9E>
> invalid, because a non-ASCII character cannot be encoded partly
> using %-escaping and partly as literal octets.
>=20
> "freenet:some-document#section1"
> a KHK URI with a fragment identifier, "section1". The fragment is
> not part of the keystring, and is interpreted by the client only
> after it has received the document, in a media-type-specific way.
> For example, if the document is HTML and contains
> <a name=3D"section1">...</a>, the client will render it starting
> at that point.
>=20
> "freenet:svk at 8HX6+0gOWem7/K26XapVwjM9/us?foo=3Dbar#section1"
> a SVK URI with both a query string and a fragment identifier.
>=20
> "freenet:a keystring with spaces"
> a KHK URI, not suitable for e-mail, etc. transmission, but in a
> form that could be typed into a client's location field.
>=20
> "freenet:a%20keystring%20with%20spaces"
> the same KHK URI as above, suitable for e-mail transmission.
>=20
> - --=20
> David Hopwood <hopwood at zetnet.co.uk>
> PGP public key: http://www.users.zetnet.co.uk/hopwood/public.asc
> RSA 2048-bit; fingerprint 71 8E A6 23 0E D3 4C E5 0F 69 8C D4 FA 66 15 01
>=20
>=20
> -----BEGIN PGP SIGNATURE-----
> Version: 2.6.3i
> Charset: noconv
>=20
> iQEVAwUBOVaTSTkCAxeYt5gVAQEpKgf/V4JC6lLxI42tyhapbEtwS5Ko8CABP/ps
> vxRqVCPWdMIderkyWK5lSou+zfI8wRSoeLh2/PBxihJixlFWPrCAK5p6BfGM6n+j
> YA0L1p4+PSuvv/oh7AKgGkD9azGqQFPX7W9RW/2gA+V5IQ+F0Dfqbd3lRpQluvXK
> Xl58uAtoM08T5ws93d1opT0H9TpjILAnhNNSrY6054kpRgClZoCU/kgF+neaG408
> 58DmphdRzw9g03rV3HNq+3g3Tz4P6Nu56rbaI5hlJ7y9UHm6/SQQ9O/y7xZQrYm2
> kiA8IMF3yF5OubH/kf1hc8t6DISS312mvKXDhvufxc5ZuJcL3NTVqg=3D=3D
> =3Dq/Zn
> -----END PGP SIGNATURE-----
>=20
>=20
> _______________________________________________
> Freenet-dev mailing list
> Freenet-dev at lists.sourceforge.net
> http://lists.sourceforge.net/mailman/listinfo/freenet-dev
>=20
--Pd0ReVV5GZGQvF3a
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.1 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQE5V49qpXyM95IyRhURAvQ7AJ9aADtwtGVhLDhdukCE25R8XZyBmgCfca7Z
nSjf+6jBVjfA5a1LutTbHCs=
=XcQw
-----END PGP SIGNATURE-----
--Pd0ReVV5GZGQvF3a--
--__--__--
_______________________________________________
Freenet-dev mailing list
Freenet-dev at lists.sourceforge.net
http://lists.sourceforge.net/mailman/listinfo/freenet-dev
End of Freenet-dev Digest
