[tor-dev] Proposal xxx: Consensus Hash Chaining
Here's a proposal Nick Mathewson and I just wrote for ticket #11157. --- Begin proposal body --- Filename: xxx-consensus-hash-chaining.txt Title: Consensus Hash Chaining Author: Nick Mathewson, Andrea Shepard Created: 06-Jan-2015 Status: Draft 1. Introduction and overview To avoid some categories of attacks against directory authorities and their keys, it would be handy to have an explicit hash chain in consensuses. 2. Directory authority operation We add the following field to votes and consensuses: previous-consensus ISOTIME [SP HashName = Base16]* NL where HashName is any keyword. This field may occur any number of times. The date in a previous-consensus line in a vote is the valid-after date of the consensus the line refers to. The hash should be computed over the signed portion of the consensus document. A directory authority should include a previous-consensus line for a consensus using all hashes it supports for all consensuses it knows which are still valid, together with the two most recently expired ones. When this proposal is implemented, a new consensus method should be allocated for adding previous-consensus lines to the consensus. A previous-consensus line is included in the consensus if and only if a line with that date was listed by more than half of the authorities whose votes are under consideration. A hash is included in that line if the hash was listed by more than half of the authorities whose votes are under consideration. Hashes are sorted lexically with a line by hashname; dates are sorted in temporal order. If, when computing a consensus, the authorities find that any previous-consensus line is *incompatible* with another, they must issue a loud warning. Two lines are incompatible if they have the same ISOTIME, but different values for the the same HashName. The hash sha256 is mandatory. 3. Client and cache operation All parties receiving consensus documents should validate previous-consensus lines, and complain loudly if a hash fails to match. When a party receives a consensus document, it SHOULD check all previous-consensus lines against any previous consensuses it has retained, and if a hash fails to match it SHOULD warn loudly in the log mentioning the specific hashes and valid-after times in question, and store both the new consensus containing the mismatching hashes and the old consensus being checked for later analysis. An option SHOULD be provided to disable operation as a client or as a hidden service if this occurs. All relying parties SHOULD by default retain all valid consensuses they download plus two; but see Security considerations below. If a hash is not mismatched, the relying party may nonetheless be unable to validate the chain: either because there is a gap in the chain itself, or because the relying party does not have any of the consensuses that the latest consensus mentions. If this happens, the relying party should log a warning stating the specific cause, the hashes and valid-after time of both the consensus containing the unverifiable previous-consensus line and the hashes and valid-after time of the line for each such line, and retain a copy of the consensus document in question. A relying party MAY provide an option to disable operation as a client or hidden service in this event, but due to the risk that breaks in the chain may occur accidentally, such an option SHOULD be disabled by default if provided. If a relying party starts up and finds only very old consensuses such that no previous-consensus lines can be verified, it should log a notice of the gap along the lines of consensus (date, hash) is quite new. Can't chain back to old consensus (date, hash). If it has no old consensuses at all, it should log an info-level message of the form we got consensus (date, hash). We haven't got any older consensuses, so we won't do any hash chain verification 4. Security Considerations: * Retaining consensus documents on clients might leak information about when the client was active if a disk is later stolen or the client compromised. This should be documented somewhere and an option to disable (but thereby also disable verifying previous-consensus hashes) should be provided. * Clients MAY offer the option to retain previous consensuses in memory only to allow for validation without the potential disk leak. --- End proposal body --- -- Andrea Shepard and...@torproject.org PGP fingerprint (ECC): BDF5 F867 8A52 4E4A BECF DE79 A4FF BC34 F01D D536 PGP fingerprint (RSA): 3611 95A4 0740 ED1B 7EA5 DF7E 4191 13D9 D0CF BDA5 pgpvtC4IJQ9ju.pgp Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Hidden Service authorization UI
On Sun, Nov 09, 2014 at 12:50:00PM +, George Kadianakis wrote: I suspect that HS authorization is very rare in the current network, and if we believe it's a useful tool, it might be worthwhile to make it more useable by people. Yes, HS authoritzation is rare. It's rare enough that it was broken for a whole series of releases and no one noticed or complained. That sucks and it should be used more because it probably does help resist attacks for a large category of use cases. For example, it would be interesting if TBB would allow people to input a password/pubkey upon visiting a protected HS. Protected HSes can be recognized by looking at the authentication-required field of the HS descriptor. Typing your password on the browser is much more useable than editing a config file. How would Tor Browser learn about this reason for not being able to connect/ tell Tor the authentication info? This is starting to sound like wanting SOCKS5 extensions to indicate different causes for connection failures in #6031 did. -- Andrea Shepard and...@torproject.org PGP fingerprint (ECC): BDF5 F867 8A52 4E4A BECF DE79 A4FF BC34 F01D D536 PGP fingerprint (RSA): 3611 95A4 0740 ED1B 7EA5 DF7E 4191 13D9 D0CF BDA5 pgpzJ7kEbdqKF.pgp Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Hidden Service authorization UI
On Sun, Nov 09, 2014 at 09:16:40PM -0500, Griffin Boyce wrote: On 2014-11-09 15:30, Fabio Pietrosanti - lists wrote: On 11/9/14 8:58 PM, Jacob Appelbaum wrote: For example, it would be interesting if TBB would allow people to input a password/pubkey upon visiting a protected HS. Protected HSes can be recognized by looking at the authentication-required field of the HS descriptor. Typing your password on the browser is much more useable than editing a config file. That sounds interesting. Also i love this idea but i would suggest to preserve the copypaste self-authenticated URL property of TorHS, also in presence of authorization. I'm conflicted about this idea. Much better for usability ~but~ there should be an option for authenticated hidden services that want to *not* prompt and instead fail silently if the key isn't in the torrc (or x.y.onion url, depending on the design). Use case: if someone finds my hidden service url written in my planner while traveling across the border, they might visit it to see what it contains. If it offers a prompt, then they know it exists and can press me for the auth key (perhaps with an M4 carbine). If there's no prompt and the request fails, then perhaps it used to exist a long time ago, or I wrote down an example URL. best, Griffin I believe it's verifiable whether an authenticated HS exists anyway; you can get the descriptor, but the list of intro points is encrypted. -- Andrea Shepard and...@torproject.org PGP fingerprint (ECC): BDF5 F867 8A52 4E4A BECF DE79 A4FF BC34 F01D D536 PGP fingerprint (RSA): 3611 95A4 0740 ED1B 7EA5 DF7E 4191 13D9 D0CF BDA5 pgprRfXrwBan3.pgp Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Sybil attack detection (was: Karsten's July 2014)
On Tue, Aug 05, 2014 at 11:24:32AM -0400, Philipp Winter wrote: On Tue, Aug 05, 2014 at 11:37:45AM +0200, Karsten Loesing wrote: Started looking into better algorithms to detect Sybil attacks on the Tor network. Current thinking is that we should define relay similarity metrics like common IP address prefix length or time between first seen in a consensus, go throw the consensus archives, and see which relays look similar but are not defined to be in the same family. Do you already have some code or more details on that? I'm quite interested in this topic and I'm wondering if it wouldn't be best to start with something simple like cosine similarity [0]. We would have to transform a relay descriptor into a numerical vector and then calculate the cosine similarity to other relay vectors. However, this might scale poorly as we would need (n^2)/2 comparisons. We might also want to weigh the vector's elements differently as some of them are easy to control for an attacker (advertised bandwidth, uptime, flags, exit policy) and others require more effort (IP address, ASN, observed bandwidth). Like you mentioned, the key thing to look at might be time, i.e., uptime and derived features such as total online time in last month etc. [0] https://en.wikipedia.org/wiki/Cosine_similarity You only need O(n*log(n)) if you can define any similarity metric that respects the triangle inequality. There's a lot of research on data structures for this: https://en.wikipedia.org/wiki/Metric_tree -- Andrea Shepard and...@torproject.org PGP fingerprint (ECC): BDF5 F867 8A52 4E4A BECF DE79 A4FF BC34 F01D D536 PGP fingerprint (RSA): 3611 95A4 0740 ED1B 7EA5 DF7E 4191 13D9 D0CF BDA5 pgpbb3vBkhc9W.pgp Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] channel_t to IP Address
On Thu, Jul 24, 2014 at 12:20:52PM +0100, Gareth Owen wrote: Hi all Quick question, how does one turn channel_t into an IP address, e.g. given a circuit_t how do I get the IP address of the previous and next hop (noting that they might not be in the consensus so I cant use the identity hash). I can see the information is in the connection_t struct but I don't know how to get that from the channel/circuit. The IP address (and the fact that the channel_t even is over IP) are specific to particular transport; you'll have to downcast it to a channel_tls_t using the BASE_CHAN_TO_TLS macro and then look at the or_connection_t pointer in that structure. -- Andrea Shepard and...@torproject.org PGP fingerprint (ECC): BDF5 F867 8A52 4E4A BECF DE79 A4FF BC34 F01D D536 PGP fingerprint (RSA): 3611 95A4 0740 ED1B 7EA5 DF7E 4191 13D9 D0CF BDA5 pgpD4i2EOymVx.pgp Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] channel_t to IP Address
On Thu, Jul 24, 2014 at 01:55:43PM +0100, Gareth Owen wrote: Andrea Thanks for taking the time to reply and for the advice. I have just this second discovered this method: TO_OR_CIRCUIT(circ)-p_chan-get_remote_descr(TO_OR_CIRCUIT(circ)-p_chan, 0) which returns the endpoint as a string. I wonder, is this safer/future proof or should this approach be discouraged? Don't call channel methods directly like that. Do call something like channel_get_actual_remote_descr(), channel_get_actual_remote_address(), or channel_get_canonical_remote_descr() (see code in channel.c for differences among them) on TO_OR_CIRCUIT(circ)-p_chan. Note: only do this if you want a string to show the user or something like that. If you're going to parse it, it's very much brittle and not future- proof. -- Andrea Shepard and...@torproject.org PGP fingerprint (ECC): BDF5 F867 8A52 4E4A BECF DE79 A4FF BC34 F01D D536 PGP fingerprint (RSA): 3611 95A4 0740 ED1B 7EA5 DF7E 4191 13D9 D0CF BDA5 pgp8LTosxltAG.pgp Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] hidden service involvement in external program
On Sun, Jul 20, 2014 at 04:34:16PM +0200, Tomas Bortoli wrote: Hi all, I want to develop a program that connects to a hidden server through tor. This is my problem: how can i establish a connection to the server from the client? I read the hidden service protocol, but i need a usable implementation of it. So i was looking in stem but seems it doesn't provide this functionality, i'm wrong? Anyone have an idea of how can i achieve a solution?? any library already implemented? You would have to implement most of a Tor client to do it the way you're thinking, including all the path selection and circuit building and everything. You should just connect through Tor's SOCKS5 port. -- Andrea Shepard and...@torproject.org PGP fingerprint (ECC): BDF5 F867 8A52 4E4A BECF DE79 A4FF BC34 F01D D536 PGP fingerprint (RSA): 3611 95A4 0740 ED1B 7EA5 DF7E 4191 13D9 D0CF BDA5 pgpyg_Gc8xzqG.pgp Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Crippled intro points
A recent thread has advocated crippling relays to selectively refuse to act as intro points for hidden services. Unfortunately, with the current HS design it would be implementable. I state my opposition to adding any such censorship feature now, and look forward to an improved HS protocol that will make it impossible. Also, note that it would be possible to patch Tor to do this without any official protocol changes, but this would have the effect of leaving the targeted HSes no way to discover intro points that will accept them other than trial and error. This strikes me as something that would comprise an attack on the network and be good grounds for a !reject. -- Andrea Shepard and...@torproject.org PGP fingerprint (ECC): BDF5 F867 8A52 4E4A BECF DE79 A4FF BC34 F01D D536 PGP fingerprint (RSA): 3611 95A4 0740 ED1B 7EA5 DF7E 4191 13D9 D0CF BDA5 pgpxvFp86pD6r.pgp Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] IRC meeting to discuss sponsor F progress on Wed July 3, 16:00 to 17:00 UTC in #tor-dev
On Thu, Jun 20, 2013 at 07:44:21PM +0200, Karsten Loesing wrote: Hi all, I'd like to schedule an IRC meeting to discuss what progress we made on sponsor F deliverables in June. Suggested time and place are: Wed July 3, 16:00 to 17:00 UTC in #tor-dev That time in other timezones is: 9:00 in San Francisco 12:00 in Boston 18:00 in Berlin 19:00 in Athens 21:30 in New Delhi People who should attend are: Roger, Nick, Andrea, George, David, Sathya, Moritz, Linus, Andrew, Tom, and anyone else who wants to attend. If your name is on that list and the suggested date or time doesn't work for you, please let me know, either here or in private mail. An alternative date would be: Tue July 2, 16:00 to 17:00 UTC in #tor-dev I'm going to send out another reminder by the end of next week that will contain the definite date. Uh, sorry, but the odds of 9 AM happening in my time zone are not too good unless it's on the tail end of the previous day's period of consciousness. -- Andrea Shepard and...@torproject.org PGP fingerprint: 3611 95A4 0740 ED1B 7EA5 DF7E 4191 13D9 D0CF BDA5 pgpXXyTHOTD6W.pgp Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] building from source in a 64-bit windows environment..
On Sat, May 18, 2013 at 11:55:48AM -0400, Zack Weinberg wrote: There's nothing wrong with sizeof(long) == sizeof(int), but I assure you that C89 does require sizeof(long) = sizeof(void *) [more precisely, that a valid value of type 'void*' can be cast to 'unsigned long' and back without loss of information] provided also that the memory space is flat. It is not itself a spelled-out requirement in the standard, but it follows from two requirements which are explicitly stated. First, 'size_t' is required to be able to represent the size of any object; when the memory space is flat, this entails that 'void*' can be cast to 'size_t' and back without loss of information. Second, 'size_t' is required to be no larger than 'unsigned long'. No, just no. It requires that sizeof(void *) can be cast to size_t. There are plenty of archs where the virtual address space is larger than any single object can be; lots and lots of old real-mode x86 compilers, for example. There are explicitly standards-conforming archs where pointer types can have sizes (a) dependent on the target type of the pointer and (b) larger than any integer type. For examples of weird pointers: http://c-faq.com/null/machexamp.html -- Andrea Shepard and...@torproject.org PGP fingerprint: 3611 95A4 0740 ED1B 7EA5 DF7E 4191 13D9 D0CF BDA5 pgpO1aU6Ly4h5.pgp Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Channel incoming queue
On Thu, May 02, 2013 at 10:58:54AM -0400, MF Nowlan wrote:
Hi all,
I am working on integrating uTCP and uTLS (
http://arxiv.org/abs/1103.0463) into Tor to see if we can lower the
latency due to head of line blocking across circuits. This is
Tracker Item #7679 (
https://trac.torproject.org/projects/tor/ticket/7679). I want to
first test the Tor code's ability to process cells from different
circuits out of order with respect to how TCP delivered them. To
test this, I made some changes to the file named src/or/channel.c.
1. I forced queueing of cells into the channel's incoming_queue with:
channel_queue_cell(...) {
...
need_to_queue = 1; // Force all cells into the queue, so they do NOT
go directly to the handler.
...
2. Now, look for case when two cells from different circuits are
present in the incoming_queue and process the second cell before the
first cell.
channel_process_cells(...) {
...
while (NULL != (q = TOR_SIMPLEQ_FIRST(chan-incoming_queue))) {
// Remove q from the incoming_queue.
// if the queue is still not empty, get the next one,
// if the circuit ids do not match, Swap the cells.
}
...
}
My problem is that number 2. above never occurs. I have not observed
a case where the incoming_queue ever has two cells from different
circuits. In fact, I don't think I've ever had a time when the
incoming_queue has more than 1 cell in it. I am hammering a small
tor test network with 30+ curl requests at once. I have two proxies,
each of them uses the same entry node, and the same exit node, and
there is only one other node in the network, so the circuit that is
set up should be the same for every single request. Am I missing
something? Will this function (channel_process_cells) ever process
more than one cell at a time? I've checked the logs to verify that
different circuits are actually set up for the different requests
(rather than the Proxies just reusing the existing circuit and
giving each new request a new stream id).
[I wrote the channel code, so I'll explain it]
Under normal operation with the current channeltls.c implementation, those
queues should rarely, if ever have more than one cell. The queue exists
because the channel abstraction includes possibilities like going temporarily
into CHANNEL_STATE_MAINT and not processing new traffic, in which case those
queues will accumulate temporarily unprocessable cells. I believe some of
the opening and closing cases might cause them to fill. None of these normally
occur with channeltls.c, which is currently the only channel implementation
layer, and at least the CHANNEL_STATE_MAINT one never occurs.
Short version: you're seeing normal behavior, don't worry about it.
--
Andrea Shepard
and...@torproject.org
PGP fingerprint: 3611 95A4 0740 ED1B 7EA5 DF7E 4191 13D9 D0CF BDA5
pgpwnUEcL_iQq.pgp
Description: PGP signature
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Iran
On Sun, May 05, 2013 at 10:40:52AM +, Nima wrote: Iran is actively dropping connections to *any* unknown port right after *60secs*. Pluggable Transport successfully connects to Tor network, Although it can not make a circuit in many ISPs including Mobin. -- Nima 0x1C92A77B I disapprove of what you say, but I will defend to the death your right to say it --Evelyn Beatrice Hall Hmmm... what does it behave like during those 60 seconds? Is it throttled, or can we get data through by cycling through a series of fresh TCP connections? What does it do with UDP packets? Could a datagram-based protocol defeat this? If they're interfering there, what about using TCP-looking packets to fake it? I.e., send SYNs with the data we really want to get through in the body and let them waste resources on their routers tracking connections that don't even really exist. -- Andrea Shepard and...@torproject.org PGP fingerprint: 3611 95A4 0740 ED1B 7EA5 DF7E 4191 13D9 D0CF BDA5 pgpUmFVkGYq97.pgp Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Parallel relaycrypt data structures for review
On Fri, Nov 30, 2012 at 08:12:10AM -0500, Nick Mathewson wrote: Hi! Here are some initial thoughts: * If we're going to do it like this, maybe we need to make cell_t packed or something eventually. It's got a fair amount of padding overhead right now. Yeah - but relay_crypt() wants a cell_t, so we'd have to unpack and repack then. * Maybe we'll need a next pointer in cells if we're queueing them? Hmmm - yeah, I think that could be made to work. I'm mostly concerned with avoiding having to have the main thread/worker thread either hold a lock for the entire time it takes to crypt a bunch of cells/pull them off the out queue * Why is there only an rc_job for outgoing cells on a circuit? It seems for symmetry we'd need to have one for inbound cells and one for outbound cells. It looks like that code isn't there right now? I was trying to figure out if we can simplify it by just using the queue for the other circuit, but yeah, think it is necessary to make rc_jobs (circuit_t, cell_direction_t) tuples. * Maybe I'm confused by these queues. The system of cell queues is going to get a little confusing, maybe. Putting cells on the outgoing queue isn't always right, since some cells (e.g., relay_data cells at an exit node) need to be handled locally rather than relaying them. So we need more new queues? The outgoing queue is the queue of crypted cells for the main thread to pick up; it isn't the same as the circuit outgoing queue because a circuit might get closed while a worker is active, and because the thing to do after the cell is crypted is a bit complicated and I wanted to minimize the number of other things that would end up being called from the worker thread. See circuit_receive_relay_cell() in relay.c; if the cell is 'recognized', it gets special handling, or else it goes on the queue for the circuit. That logic would move to the second half of the main-thread processing, after the cell is removed from the rc_job output queue. * Should the jobs be in some data structure other than an smartlist_t? A queue would seem to make more sense, since jobs are getting added and pulled off. (Yes, protecting the data structure there with a lock makes sense.) Yeah, I just said that as a placeholder - what's really best surely depends on the selection policy. * If you're going to have separate locks, it's important to document how they nest, to prevent deadlock conditions. Yeah - I specified always lock the relaycrypt_dispatcher_t, then the relaycrypt_job_t in the comments, I believe. * Presumably relaycrypt_job_t would need to have a pointer to the actual circuit that needs work, and a note about whether it's a job for outbound or inbound cells. Well, it shouldn't be messing with the circuit too much because then a lot of other stuff that touches the circuit also needs to worry about being thread-safe, and the case of a circuit being shut down while a worker is active gets hairy. The worker will only be touching the queues in the rc_job, but it will need the circuit for the call to relay_crypt() - hmm, we need a way to make sure the crypto keys don't get freed out from under it if a circuit goes away. * In the non-threaded-relaycrypt case, presumably the intention is that there's a function that would otherwise queue a cell for crypto but instead just put it directly on the appropriate circuit queue? Yeah - the non-thready version would just call whatever (possibly refactored) relay_crypt() that the worker thread calls and then the handler for crypted cells, all in the main thread, rather than queueing anything. Thanks again! I'll let you know if I think of anything else. Okay, thanks. I'm trying to get some code started this weekend, I think. -- Andrea Shepard and...@torproject.org PGP fingerprint: 3611 95A4 0740 ED1B 7EA5 DF7E 4191 13D9 D0CF BDA5 pgpgbw7JbBvGe.pgp Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
