Re: Bringing Tahoe ideas to HTTP

2009-09-18 Thread Peter Gutmann
Alexandre Dulaunoy adu...@gmail.com writes:

On the same idea, there is an expired Internet-Draft called Link
Fingerprints :
http://www.potaroo.net/ietf/idref/draft-lee-uri-linkfingerprints/

Although the draft has expired, the concept lives on in various tools.  For
example DownThemAll for Firefox supports this.  There was some discussion
about including it into FF3, but then the draft was dropped and the FF support
never appeared, does anyone know what happened?

(The cynic in me would say it's such a simple, straightforward, easy-to-
implement idea, of course it'll never be adopted when there are things like EV
certs to be pushed instead, but who knows...).

Peter.

-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Re: Bringing Tahoe ideas to HTTP

2009-09-18 Thread Alexandre Dulaunoy
On Fri, Sep 18, 2009 at 6:27 AM, Peter Gutmann
pgut...@cs.auckland.ac.nz wrote:

 Although the draft has expired, the concept lives on in various tools.  For
 example DownThemAll for Firefox supports this.  There was some discussion
 about including it into FF3, but then the draft was dropped and the FF support
 never appeared, does anyone know what happened?

It was a SoC project within Mozilla but seeing the following discussion :

https://bugzilla.mozilla.org/show_bug.cgi?id=377245

The state looks a bit unclear to me...

 (The cynic in me would say it's such a simple, straightforward, easy-to-
 implement idea, of course it'll never be adopted when there are things like EV
 certs to be pushed instead, but who knows...).

mode type=cynic
Right... When I see the EV audit process[1], it looks to me like PCI DSS without
the technical aspect...
/mode

[1] http://cabforum.org/WebTrustAuditGuidelines.pdf

-- 
--   Alexandre Dulaunoy (adulau) -- http://www.foo.be/
-- http://www.foo.be/cgi-bin/wiki.pl/Diary
-- Knowledge can create problems, it is not through ignorance
--that we can solve them Isaac Asimov

-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Re: [tahoe-dev] Bringing Tahoe ideas to HTTP

2009-09-18 Thread Peter Gutmann
Brian Warner war...@lothar.com writes:

From what I can tell, the Sparkle update framework (for OS-X)[1] is doing
something like what I want for firefox: the Sparkle-enabled application will
only accept update bundles which are signed by a DSA privkey that matches a
pubkey embedded in the app.

You can extend this further to make it tolerant of key loss by embedding
multiple public keys and allowing a quorum of them to replace an existing key.
So say you have five keys, you can decide that any three of them can vote out
an existing key, allowing compromised keys to be replaced and existing keys to
be rolled over.  This creates a kind of fault-tolerant PKI which does away
with the need to have a CA vouch for key replacements, once you've got the
initial keys established (for example on first install) you can recover from
anything short of a total compromise, upgrade to larger keys sizes and hashes,
and so on.

It'd be nice if Firefox could do the same. And if Firefox were to establish a
quietly-backwards-compatible convention (i.e. the hash-mark trick) for strong
URL-based authentication of HTTP resources, then other applications could
start using it too, and a significant class of current web security problems
(like the mixed-content one where an HTTPS page loads a javascript library via
HTTP) could be fixed.

See my previous post, there was an attempt made to do this in the past but it
never got anywhere.  It'd be interesting to hear the reasons why.

Peter.

-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Re: Bringing Tahoe ideas to HTTP

2009-09-17 Thread Alexandre Dulaunoy
On Thu, Aug 27, 2009 at 11:57 PM, Brian Warner war...@lothar.com wrote:

 == Integrity ==

 To start with integrity-checking, we could imagine a firefox plugin that
 validated a PyPI-style #md5= annotation on everything it loads. The rule
 would be that no action would be taken on the downloaded content until
 the hash was verified, and that a hash failure would be treated like a
 404. Or maybe a slightly different error code, to indicate that the
 correct resource is unavailable and that it's a server-side problem, but
 it's because you got the wrong version of the document, rather than the
 document being missing altogether.

On the same idea,
there is an expired Internet-Draft called Link Fingerprints :

http://www.potaroo.net/ietf/idref/draft-lee-uri-linkfingerprints/

I made some experiments around while used as Machine Tag/Triple Tag[1] :

http://www.foo.be/cgi-bin/wiki.pl/MachineTagLinkFingerprint

to have an extension with OpenPGP detached signature.

Another potential use, it's to benefit from the number of users
checking the integrity and contribute back the computed
value into a tagging system like del.icio.us or any other
collaborative bookmarking.

I especially like the Firefox (or wget,curl) extension that
could compute the hash value and check it against various
contributed hashes. That could give a kind of confidence level
regarding the integrity of the file and its corresponding URL/URI.

Just some ideas,

adulau

[1] http://www.foo.be/cgi-bin/wiki.pl/MachineTag

-- 
--   Alexandre Dulaunoy (adulau) -- http://www.foo.be/
-- http://www.foo.be/cgi-bin/wiki.pl/Diary
-- Knowledge can create problems, it is not through ignorance
--that we can solve them Isaac Asimov

-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Re: Bringing Tahoe ideas to HTTP

2009-09-16 Thread Ivan Krstić

On Sep 15, 2009, at 4:12 PM, James A. Donald wrote:
The ideas used in Tahoe are useful tools that can be used to solve  
important problems.



Yes, and I'd be happy to opine on that as soon as someone told me what  
those important problems are.


--
Ivan Krstić krs...@solarsail.hcs.harvard.edu | http://radian.org

-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Re: [tahoe-dev] Bringing Tahoe ideas to HTTP

2009-09-16 Thread Zooko Wilcox-O'Hearn

On Wednesday,2009-09-16, at 14:44 , Ivan Krstić wrote:

Yes, and I'd be happy to opine on that as soon as someone told me  
what those important problems are.


The message that you quoted from Brian Warner, which ended with him  
wondering aloud what new applications could be enabled by such  
features, began with him mentioning a specific use case that he cares  
about and sees how to improve: authentication of Mozilla plugins.   
Brian has an admirable habit of documenting the use cases that  
motivate his engineering decisions.  I think in this case he omitted  
some explanation of why he finds the current solution unsatisfactory,  
perhaps because he assumed the audience already shared his view.  (I  
think he mentioned something in his letter like the well-known  
failures of the SSL/CA approach to this problem.)


Regards,

Zooko
-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Re: Bringing Tahoe ideas to HTTP

2009-09-15 Thread Ivan Krstić

On Aug 27, 2009, at 2:57 PM, Brian Warner wrote:
I've no idea how hard it would be to write this sort of plugin. But  
I'm

pretty sure it's feasible, as would be the site-building tools. If
firefox had this built-in, and web authors used it, what sorts of
vulnerabilities would go away? What sorts of new applications could we
build that would take advantage of this kind of security?


What you're proposing amounts to a great deal of complex and  
complicated cryptography. If it were implemented tomorrow, it would  
take years for the most serious of implementation errors to get weeded  
out, and some years thereafter for proper interoperability in corner  
cases. In the meantime, mobile device makers would track you down for  
the express purpose of breaking into your house at night to pee in  
your Cheerios, as retaliation for making them explain to their  
customers why their mobile web browsing is either half the speed it  
used to be, or not as secure as on the desktop, with no particularly  
explicable upside.


It bugs the hell out of me when smart, technical people spend time and  
effort devising solutions in search of problems. You need to *start*  
with the sorts of vulnerabilities you want to do away with, or the  
kinds of new applications you can build that current security systems  
don't address, and *then* work your way to solutions that enable those  
use cases.


It's okay to do it in reverse order in the academia, but you seem to  
be talking about real-world systems. And in real-world systems, you  
don't get to play Jeopardy with cryptography.


Cheers,

--
Ivan Krstić krs...@solarsail.hcs.harvard.edu | http://radian.org

-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Re: Bringing Tahoe ideas to HTTP

2009-09-15 Thread James A. Donald

Ivan Krsti  wrote:
What you're proposing amounts to a great deal of complex and complicated 
cryptography. If it were implemented tomorrow, it would take years for 
the most serious of implementation errors to get weeded out, and some 
years thereafter for proper interoperability in corner cases. In the 
meantime, mobile device makers would track you down for the express 
purpose of breaking into your house at night to pee in your Cheerios, as 
retaliation for making them explain to their customers why their mobile 
web browsing is either half the speed it used to be, or not as secure as 
on the desktop, with no particularly explicable upside.


The ideas used in Tahoe are useful tools that can be used to solve 
important problems.


It is true that just dumping them on end users and hoping that end users 
will use them correctly to solve important problems will fail


It is our job to apply these tools, not the end user's job, the hard 
part being user interface architecture, rather than cryptography protocols.


Yurls are one example of an idea for a user interface wrapping
Tahoe like methods to solve useful problems.

-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Re: [tahoe-dev] Bringing Tahoe ideas to HTTP

2009-09-08 Thread James A. Donald

Nicolas Williams wrote:
  One possible problem: streaming [real-time] content.

Brian Warner wrote:
 Yeah, that's a very different problem space. You need
 the low-alacrity stuff from Tahoe, but also you don't
 generally know the full contents in advance. So you're
 talking about a mutable stream rather than an
 immutable file.

Not mutable, just incomplete.

Immutable streaming content needs a tiger hash or a
patricia hash, which can handle the fact that some of
the stream will be lost in transmission, and that one
needs to validate the small part of the stream that one
has already received rather than waiting for the end.

 upgrade bundles are produced by a very strict process,
 and are rigidly immutable [...] For software upgrades,
 it would reduce the attack surface significantly.

But how does one know which immutable file is the one
that has been blessed by proper authority?  Although
Version 3.5.0 is immutable, what makes it Version 3.5.0,
rather than haxx350, is that some authority says so -
the same authority that said that your current version
is 3.4.7 - and it should not even be possible to get a
file named by some different authority as an upgrade.

Of course, one would like a protocol that committed the
authority to say the same thing for everyone, and the
same thing for all time, saying new things in future,
but never being able to retroactively adjust past things
it has said.

In other words, upgrades should be rooted in an append
only capability.

I suppose this could be implemented as a signed dag of
hashes, in which whenever one upgraded, one checked that
the signed dag that validates the upgrade is consistent
with the signed dag that validated the current version.

-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Re: [tahoe-dev] Bringing Tahoe ideas to HTTP

2009-09-08 Thread Brian Warner
James A. Donald wrote:
 Nicolas Williams wrote:
One possible problem: streaming [real-time] content.
 
 Brian Warner wrote:
   Yeah, that's a very different problem space. You need
   the low-alacrity stuff from Tahoe, but also you don't
   generally know the full contents in advance. So you're
   talking about a mutable stream rather than an
   immutable file.
 
 Not mutable, just incomplete.
 
 Immutable streaming content needs a tiger hash or a
 patricia hash, which can handle the fact that some of
 the stream will be lost in transmission, and that one
 needs to validate the small part of the stream that one
 has already received rather than waiting for the end.

I was assuming a real-time stream, so the goal would be to provide a
filecap before the source had finished generating all the bits. This
would necessarily be a mutable filecap, unless you've got some way of
predicting the future :-).

If instead, you just have a large file that a client wants to fetch one
piece at a time, well, Tahoe's immutable-file merkle trees already
handle the goal of quickly validating a small part of a large byte
sequence. You could use this in a non-real-time stream, in which you
process the entire input stream, produce and publish the filecap, then a
client fetches pieces of that stream at their own pace.


   upgrade bundles are produced by a very strict process,
   and are rigidly immutable [...] For software upgrades,
   it would reduce the attack surface significantly.
 
 But how does one know which immutable file is the one
 that has been blessed by proper authority?

You're right, I was assuming a pre-existing secure what to upgrade to
channel which could send down an integrity-enhanced download URL. To
actually implement such a channel would require further integrity
guarantees over mutable data.

As I understand it, Firefox actually has a fairly complex upgrade path,
because only certain combinations of from-version/to-version are fully
tested by QA. Sometimes moving from e.g. 3.0.0.8 to 3.5.0.3 requires
going through a 3.0.0.8-3.0.0.9-3.5.0.0-3.5.0.2-3.5.0.3 sort of
path. The upgrade channel basically provides instructions to each older
version, telling them which new version they should move to.

The best sort of integrity guarantees I could think of would be a rule
that says the new version must be signed by my baked-in DSA key and
have a version number that is greater than mine, or maybe just the
upgrade instructions must be signed by that DSA key. It's probably not
a robust idea to make the rules too strict.

cheers,
 -Brian

-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Bringing Tahoe ideas to HTTP

2009-08-31 Thread Brian Warner

[sent once to tahoe-dev, now copying to cryptography too, sorry for the
duplicate]

At lunch yesterday, Nathan mentioned that he is interested in seeing how
Tahoe's ideas and techniques could trickle outwards and influence the
design of other security systems. And I was complaining about how the
Firefox upgrade process doesn't provide the integrity checks that I want
(it turns out they rely upon the CA infrastructure and SSL alone, no
end-to-end checking; the updates and releases are GPG-signed, but
firefox doesn't check that, only humans might). And PyPI has this nice
habit of appending #md5=XYZ.. to the URLs of the release tarballs that
they publish, which is (I think) automatically used by tools like
easy_install to guard against corrupted downloads (and which I always
use, as a human, to do the same). And Nathan mentioned a class of web
attacks in which a page, loaded over SSL, imports something (JS, CSS,
JPG) via a regular http: URL, and becomes vulnerable to third-parties
who can take over the page by controlling what arrives over
unauthenticated HTTP.

So, setting aside the reliability-via-distributedness properties for a
moment, what could we bring from Tahoe into regular HTTP and regular
webservers that could improve the state of security on the web?

== Integrity ==

To start with integrity-checking, we could imagine a firefox plugin that
validated a PyPI-style #md5= annotation on everything it loads. The rule
would be that no action would be taken on the downloaded content until
the hash was verified, and that a hash failure would be treated like a
404. Or maybe a slightly different error code, to indicate that the
correct resource is unavailable and that it's a server-side problem, but
it's because you got the wrong version of the document, rather than the
document being missing altogether.

This would work just fine for a flat hash: the original file remains
untouched, only the referencing URLs change to get the new hash
annotation. Non-enhanced browsers are unaffected: the #-prefixed
fragment identifier is never sent to the server, and the a name= tag
is fairly rare these days (and would still mostly work). Container files
(the HTML which references the hashed documents) could be updated to
benefit at leisure. Automation (see below) could be used to update the
URLs in the containers whenever the referenced objects were modified.

To improve alacrity on larger files, Tahoe uses a Merkle tree over
segments of the file. This tree has to be stored somewhere (Tahoe stores
it along with the shares, but it would be more convenient for a web site
to not modify the source files). We could use an annotation like
#hashtree=ROOTXYZ;http://otherplace; to reference an external hash tree
(with root hash XYZ). The plugin would start pulling from the source
file and the hash tree at the same time, and not deliver any source data
until it had been validated. The hashtree object would need to start
with the segment size and filesize, so the tree could be computed
properly. For very large files, you could read those parameters and then
pull down (via a Range: header) just the parts of the Merkle tree that
were necessary. In this case, the automation would need to create the
hash tree file and put it in a known place each time the source file
changes, and then updated the references.

(note that ROOTXYZ provides the identification properties of this
annotation, and http://otherplace; provides the location properties,
where identification means the ability to recognize the correct document
if someone gives it to you, and location means the ability to retrieve a
possibly-correct document. URIs provide identification, URLs are
supposed to provide both.)

We could compress this by establishing an (overriable) convention that
http://example.com/foo.mp3 always has a hashtree at
http://example.com/foo.mp3.hashtree, resulting in a URL that looked like
http://example.com/foo.mp3#hashtree=ROOTXYZ;. If you needed to store it
elsewhere, you could use #hashtree=ROOTXYZ;WHERE, and define WHERE to
be a relative URL (with a default value of NAME.hashtree).

== Mutable Integrity ==

Zooko and I have both run HTML presentations out of a Tahoe grid (which
makes for a great demo), and the first thing you learn there is that
immutability, while a great property in some cases, is a hassle for
authoring. You need mutability somewhere, and the more places you have
it, the fewer URLs you have to update every time you change something.
In technical terms, you frequently want to cut down the diameter of the
immutable domains of the object DAG, by splitting those domains with
mutable boundary nodes. In practical terms, it means you might want to
publish *everything* via a mutable file. At the very least, if your web
site has any internal cycles in it, you'll need a mutable node to break
the cycle.

Again, this requires data beyond the contents of the source file. We
could use a #sigkey=XYZ annotation with a base62'ed ECDSA pubkey (this

Re: [tahoe-dev] Bringing Tahoe ideas to HTTP

2009-08-31 Thread Michael Walsh
Hi Brian, all;

I'm all for including merkle trees with HTTP GETs, two items that
spring to mind:
 - Appending the location of the hash as you suggest in
#hashtree=ROOTXYZ;http://otherplace which requires no changes to the
webserver.
 - Adding a HTTP header with this data but requires something like a
server module or output script. It also doesn't ugly up the URL (but
then again, we have url shortner services for manual typing).

One merkle hash tree over HTTP that interested me was the Tiger Tree
Hash Exchange/THEX [1] that's already in use in some P2P systems, and
would be interesting reading for other hash tree over HTTP systems.

Google Wave appears to use hash trees also, but it seems to be
under-speced [2]. I guess once that becomes fleshed out there would be
more content systems outputting data along with tree hashes.


I do like straightforwardness of using the
file.ext#hashtree=root;location or file.ext.sig conventions and an
added benefit is the .sig request can be HTTP/1.1 pipelined rather
than parsing the returned headers before sending on the additional
request.

 I've no idea how hard it would be to write this sort of plugin. But I'm
 pretty sure it's feasible, as would be the site-building tools. If
 firefox had this built-in, and web authors used it, what sorts of
 vulnerabilities would go away? What sorts of new applications could we
 build that would take advantage of this kind of security?

My thoughts purely turn to verifying files and all webpage resources
integrity in a transparent and backward compatible way. Who has not
encountered unstable connections where images get corrupted and css
files don't fully load? Solving that problem would make me very happy!

[1] http://open-content.net/specs/draft-jchapweske-thex-02.html
[2] 
http://groups.google.com/group/wave-protocol/browse_thread/thread/74dbbd2c0f586ec9

-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Re: [tahoe-dev] Bringing Tahoe ideas to HTTP

2009-08-31 Thread Brian Warner
Michael Walsh wrote:

  - Adding a HTTP header with this data but requires something like a
 server module or output script. It also doesn't ugly up the URL (but
 then again, we have url shortner services for manual typing).

Ah, but see, that loses the security. If the URL doesn't contain the
root hash, then you're depending upon somebody else for your
authentication, and then it's not end-to-end anymore. URL shortener
services are great for the location properties, but lousy for the
identification properties. Not only are you relying upon your DNS, and
your network, and every other network between you and the server, and
the server who's providing you with the data.. now you're also dependent
upon the operator of the shortener service, and their database, and
anyone who's managed to break into their database, etc.


I guess there are three new things to add:

 * secure identifier in the URL
 * an upstream request, to say what additional integrity information you
   want
 * a downstream response, to provide that additional integrity
   information

The stuff I proposed used extra HTTP requests for those last two.

But, if you use the HTTP request headers to ask for the extra integrity
metadata, you could use the HTTP response headers to convey it. The only
place where this would make sense would be to fetch the merkle tree, or
the signature. (if the file was small and you only check the flat hash,
then there's nothing else to fetch; if the file is encrypted, then you
can define its data layout to be whatever you like, and just include the
integrity information in it directly).

Oh, and that would make the partial-range request a lot simpler: the
client does a GET with a Range: 123-456 header, and the server looks
at the associated merkle tree, figures out which chain they'll need to
validate those bytes, and returns a header that includes all of those
hash tree nodes (and the segment size, filesize). And returns enough
file data to cover those segments (i.e. the Content-Range: response
would be for a larger range than the Range: request, basically rounded
up to a segment size). The client would hash the segments it receives,
build and verify the merkle chain, then compare the root of the chain
against the roothash in the URL and make sure they match.

The response headers might get a bit large:
log2(filesize/segsize)*hashsize . And you have to fetch at least a full
segsize. But that's predictable and fairly well bounded, so maybe it
isn't that big of a deal.

Doing this with HTTP headers instead of a separate GET would avoid a
roundtrip, since the server (which now takes an active role in the
process) can decide these things (like which hash tree nodes are needed)
on behalf of the client. Instead of the client pulling one file for the
data, then pulling part of another file to get the segment size (so it
can figure out which segments it wants), then pulling a different part
of that file to get the hash nodes... the client just does a GET Range:,
and the server figures out the rest.

As you said, it requires a server module. But compared to a client-side
plugin, that's positively easy :). It'd probably be a good idea to think
about a scheme that would take advantage of a server which had
additional capabilities like this. Maybe the client could try the header
thing first, and if the response didn't indicate that the server is able
to provide that data, go and fetch the .hashtree file.


 My thoughts purely turn to verifying files and all webpage resources
 integrity in a transparent and backward compatible way. Who has not
 encountered unstable connections where images get corrupted and css
 files don't fully load? Solving that problem would make me very happy!

Yeah. We've been having an interesting thread on tahoe-dev recently
about the backwards-compatible question, what sorts of failure modes are
best to aim for when you're faced with an old server or whatever.

You can't make this stuff totally transparent and backwards compatible
(in the sense that existing webpages and users should start benefiting
from it without doing any work).. I think that's part of the brokenness
of the SSL+CA model, where they wanted the only change to be starting
from https instead of http. But you can certainly create a framework
that lets people get better control over what they're loading. Now, if
only distributed programs could be written in languages with those sorts
of properties...

cheers,
 -Brian

-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com