Re: [tor-dev] Proposal XXX: FlashFlow: A Secure Speed Test for Tor (Parent Proposal)
On 4/23/20 1:48 PM, Matt Traudt wrote: > 5.4 Other Changes/Investigations/Ideas > > - How can FlashFlow data be used in a way that doesn't lead to poor > load balancing given the following items that lead to non-uniform > client behavior: > - Guards that high-traffic HSs choose (for 3 months at a time) > - Guard vs middle flag allocation issues > - New Guard nodes (Guardfraction) > - Exit policies other than default/all > - Directory activity > - Total onion service activity > - Super long-lived circuits > - What is the explanation for dennis.jackson's scary graphs in this [2] > ticket? Was it because of the speed test? Why? Will FlashFlow produce > the same behavior? It will also be wise to provide a way for relays to signify that they are on the same machine. I bet concurrent machine deployments are one of the top contributors to the long tail of bad perf we saw caused by the Flashflow experiment[2]. If flashflow measures each such relay as having the full link capacity instead of a shared fraction, this is obviously going to result in overload on those relays, leading to a long tail of bad perf when they are chosen and are also overloaded. It is unlikely that we can deploy a FlashFlow that has this long tail perf problem without fixing this and related balancing issues (though hopefully most will be smoothed over by sbws). This is a little tricky, because we might not want rogue relays joining each others "machines" (similar to the Family problem), but for testing something as simple as how MyFamily works would be great. Ideally, though, relays would ask or detect that they are concurrently running in nearby IP space and either warn the operator to set the flag, or set it automatically. We actually have this work included in a future performance funding proposal, but the timeline on that getting approved (or even rejected) is so far out that we should figure out a way to do this before that, especially if Flashflow development is going to begin soon. > [2] Mike Perry: Graph onionperf and consensus information from Rob's > experiments https://trac.torproject.org/projects/tor/ticket/33076 -- Mike Perry ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Support for full DNS resolution and DNSSEC validation
> Before we go further, can you walk me through the reasons (if you had thought > of it of course) why you didn't use something like libunbound? > > There are side effects of adding DNSSEC client support (with our own > implementation) that we, people maintaining tor, have to become DNSSEC expert > in some ways just to be able to understand what is happening in that code, fix > issues but also possibly implement new features if any. That is where a third > part library like unbound becomes very nice because they are the experts at > providing such features. > > Of course, everytime we have to link to an external library we do it carefully > and with considerations because of the "yet another attack" vector problem. > But adding that much code to support a well known feature like DNSSEC also has > huge implications for tor maintainability and security. > > Finally, something I noticed and made me itch a bit. You hardcoded a lot of > .onions where one appears to be Cloudflare (?) resolver. What are the other > addresses? I worry here because default options are always the one used the > most so I'm concerned here about shipping hardcoded addresses _within_ our C > code. +1 for "don't roll your own DNS(SEC) implementation". There are people that do DNS(SEC) for years, should the torproject really implement and maintain its own custom DNS stub resolver and DNSSEC validator? Also consider that DNS is moving towards DNS encryption protocols (DoT and DoH) and firefox has DoH support that could be made stream isolation aware. Using open protocols will increase the availability of multiple public resolvers speaking that protocol. signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Support for full DNS resolution and DNSSEC validation
> To me, extra round-trips over the Tor network in the critical path of > "user clicks and waits for the website to load" are really bad, and > need a really good argument for being there. Given that DNS is only one > piece of the connection -- after all, the exit relay can still route you > somewhere else -- it's hard to see how this case brings enough benefit > to justify the extra round trip(s). > > Ok, with that as a preface, here is an alternative design: the Tor > client sends the hostname to the exit relay, along with a request for > dnssec proofs. The exit relay uses its own dnssec server to convince > itself that its answer is right. Then it returns the IP address in > the CONNECTED cell as it does now, and also it returns a set of dnssec > answers that the client can use to reconstruct the dnssec interaction > and convince itself of the result too. > > This design adds no extra round trips (yay), but it requires a notion of > "dnssec chain stapling" that I think doesn't entirely exist yet. Alex > points me to a long-expired IETF draft from agl on the topic: > https://tools.ietf.org/html/draft-agl-dane-serializechain-01 > and I don't know if there is newer progress. also expired but newer https://datatracker.ietf.org/doc/draft-ietf-tls-dnssec-chain-extension/ > I would also worry about the overall size of the stapled answers -- if we > add another 50KBytes to every stream interaction in Tor, that's probably > not worth it. Whereas adding another 1KByte could be a great tradeoff. > > Yet another twist here is that it's hard for the client to cache answers, > or to cache intermediate certs in the chain, because changing behavior > based on cached answers can reveal the client's past browsing history. I share your concerns on performance (additional round trips) and caching and I find it also important to note that in the context of tor browser (probably the main use case of tor) encrypted DNS (confidentiality) is more relevant than DNSSEC (integrity), especially as soon as ESNI becomes reality. https://datatracker.ietf.org/doc/draft-ietf-tls-esni/ That will allow for using tor exits without disclosing the visited domain to the exit relay and no more issues with exits with broken DNS. DNSSEC would still be valuable for TLS verification but DANE for https does not appear to be a thing. signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Support for full DNS resolution and DNSSEC validation
Alexander Færøy: > I wonder if it would make more sense to have an onion-aware > DNSSEC-enabled resolver *outside* of the Tor binary and have a way for > Tor to query an external tool for DNS lookups. I'm also in favor of this approach, and you can do this today with no code changes to tor at all. CF demonstrated it even before DoH/RFC8484 was finalized: https://blog.cloudflare.com/welcome-hidden-resolver/ > Such tool should be > allowed to use Tor itself for transport of the actual queries. One of > the best parts of Tor (in my opinion) is the Pluggable Transport > subsystem. This subsystem allows external developers, researchers, and > hackers to build new technology that benefits users in censored areas > *without* having to alter a single line of C code in tor.git. > > Let's say we had a "Pluggable DNS" layer in Tor. Users would be able to > configure their Tor process to *never* use the built-in DNS subsystem in > Tor, but instead outsource it to an external process that Tor spawns on > startup. This process could use .onion's to reach a > DNS-over-(TLS|HTTPS|TCP) server as onions themselves aren't looked up > via DNS. + 1 for DoT and DoH over tor, especially due to the DoH implementation that is available in firefox (it would still require work on stream isolation and caching risks to ensure the usual first party isolation). In terms of achieving a big improvement on tor browser users in the context of DNS this would be the most effective path to spend coding resources on in my opinion. signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Support for full DNS resolution and DNSSEC validation
> I can not really say anything about how this design compares to other > approaches, since I don't know how I can setup meaningful test > scenarios to compare them. Do we really need test setups to discuss protocol designs and compare protocols with a common threat model if specs for the protocols are available? > However, I would appreciate if you could > share how to setup such test environments. take your preferred DoT client implementation that supports the strict profile (RFC8310) or your preferred DoH implementation and route it over tor to your resolver of choice. signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Support for full DNS resolution and DNSSEC validation
On Fri, May 15, 2020 at 05:39:23PM +0200, Christian Hofer wrote: > Final remarks. When I started, I didn't expect it to get this big, and > frankly, if I had known before, I might not have even started. However, > I learned a lot about DNS, DNSSEC, SOCKS, and Tor. So even if you > decide not to merge it, it was not a waste :-) Hi Christian! Given the above learning, let me ask you a related question. Or maybe rather, a question about an alternative design. As I understand it, the design in your patch is basically to let the Tor client talk via Tor to an authenticated dns server that the client chooses, and do the DNS interaction to satisfy itself that the DNS answer is correct. And because the DNS server is a different destination than the original stream destination, this requires another round-trip through Tor, for doing the resolve? And maybe it's way worse than that, because to do DNSSEC properly, you need to go up the chain, with a new round-trip over Tor for each link in the chain? Whereas in the current Tor design, we have no extra round-trips for the DNS component, because the client sends the hostname in the BEGIN cell, and the exit relay resolves it and connects, and then sends the IP address back in the CONNECTED cell in case the client wanted to know it. To me, extra round-trips over the Tor network in the critical path of "user clicks and waits for the website to load" are really bad, and need a really good argument for being there. Given that DNS is only one piece of the connection -- after all, the exit relay can still route you somewhere else -- it's hard to see how this case brings enough benefit to justify the extra round trip(s). Ok, with that as a preface, here is an alternative design: the Tor client sends the hostname to the exit relay, along with a request for dnssec proofs. The exit relay uses its own dnssec server to convince itself that its answer is right. Then it returns the IP address in the CONNECTED cell as it does now, and also it returns a set of dnssec answers that the client can use to reconstruct the dnssec interaction and convince itself of the result too. This design adds no extra round trips (yay), but it requires a notion of "dnssec chain stapling" that I think doesn't entirely exist yet. Alex points me to a long-expired IETF draft from agl on the topic: https://tools.ietf.org/html/draft-agl-dane-serializechain-01 and I don't know if there is newer progress. I would also worry about the overall size of the stapled answers -- if we add another 50KBytes to every stream interaction in Tor, that's probably not worth it. Whereas adding another 1KByte could be a great tradeoff. Yet another twist here is that it's hard for the client to cache answers, or to cache intermediate certs in the chain, because changing behavior based on cached answers can reveal the client's past browsing history. Do these goals make sense? :) Thanks! --Roger ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Support for full DNS resolution and DNSSEC validation
Alexander Færøy: > Hey, > > On 2020/05/15 16:36, Jeremy Rand wrote: >> The Prop279 spec text is ambiguous about whether the target is required >> to be a .onion domain, but the implementations (TorNS and StemNS) do not >> have that restriction. TorNS and StemNS allow a Prop279 plugin to >> advertise acceptance of any domain suffix (haven't explicitly tried the >> root zone as an suffix, but if that doesn't work, it's a bug that should >> be easy to fix) and can resolve them to any result (e.g. an IP address, >> a .onion domain, or another DNS name a la CNAME). > > In proposal #279 the subprocess passes the `RESOLVED` message to Tor > once it is has completed a name look up. The `RESOLVED` message is > defined as follows: > > ``When the name plugin completes the name resolution, it prints the > following line in its stdout: > > RESOLVED > > where QUERY_ID is the corresponding query ID and STATUS_CODE is an integer > status code. RESULT is the resolution result (an onion address) or an > error > message if the resolution was not succesful.'' > > Here the `` must be an onion address. We would have to change > that, such that an IP address can be returned as well :-) Hi Alex, The ambiguity I was referring to is that while the section you quote does require that the result be a .onion domain, below it is this note: > Tor MUST validate that the resolution result is a valid .onion name. > XXX should we also accept IPs and regular domain results??? Prop279 is clearly labeled as a draft, so this makes it appear that no decision was reached on whether the result should be required to be a .onion domain. My opinion is that accepting non-.onion addresses as results is desirable (both because it's useful for the Namecoin use case and because it's useful for the DNSSEC use case that we're discussing). Cheers, -- -Jeremy Rand Lead Application Engineer at Namecoin Mobile email: jeremyrandmob...@airmail.cc Mobile OpenPGP: 2158 0643 C13B B40F B0FD 5854 B007 A32D AB44 3D9C Send non-security-critical things to my Mobile with OpenPGP. Please don't send me unencrypted messages. My business email jer...@veclabs.net is having technical issues at the moment. signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Support for full DNS resolution and DNSSEC validation
Hey, On 2020/05/15 16:36, Jeremy Rand wrote: > The Prop279 spec text is ambiguous about whether the target is required > to be a .onion domain, but the implementations (TorNS and StemNS) do not > have that restriction. TorNS and StemNS allow a Prop279 plugin to > advertise acceptance of any domain suffix (haven't explicitly tried the > root zone as an suffix, but if that doesn't work, it's a bug that should > be easy to fix) and can resolve them to any result (e.g. an IP address, > a .onion domain, or another DNS name a la CNAME). In proposal #279 the subprocess passes the `RESOLVED` message to Tor once it is has completed a name look up. The `RESOLVED` message is defined as follows: ``When the name plugin completes the name resolution, it prints the following line in its stdout: RESOLVED where QUERY_ID is the corresponding query ID and STATUS_CODE is an integer status code. RESULT is the resolution result (an onion address) or an error message if the resolution was not succesful.'' Here the `` must be an onion address. We would have to change that, such that an IP address can be returned as well :-) All the best, Alex. -- Alexander Færøy ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Support for full DNS resolution and DNSSEC validation
Alexander Færøy: > Hey Jeremy, > > On 2020/05/15 15:53, Jeremy Rand wrote: >> FYI I already wrote a Prop279 provider that looks up the names via DNS >> (it's aptly named "dns-prop279"); it does pretty much exactly what you >> describe. It doesn't handle DNSSEC validation itself (it assumes that >> you've specified a DNS server that you trust -- most likely one running >> on localhost). Stream isolation can be handled via an EDNS0 field (and >> I'm guessing it would not be difficult to patch an existing DNS server >> to respect that EDNS0 field). I wouldn't be surprised if it's easy to >> make dns-prop279 do DNSSEC validation itself (and not use a >> localhost-based DNS server) if that's desired -- the library it uses >> (miekg/dns) does claim to support DNSSEC validation, though I've never >> tried testing that feature. > > Very interesting. > > I think proposal #279 only tries to solve the subset of name look ups, > which is about looking up onions from a human name. The work in this > thread is to replace all name lookups *except* for Onions. It could very > well be that it would be easier to extend proposal #279 by having it > handle all lookups and not just for .onion's, but I think my intuition > says that it should be two different systems as onion lookups is still a > much more open question whereas Tor will need to support ordinary DNS > for many years into the future. > > If `OnionNamePlugin` allowed you to specify `.*` for "everything" as the > TLD specifier, then it might be possible to implement such system using > proposal #279 :-) Hi Alex, The Prop279 spec text is ambiguous about whether the target is required to be a .onion domain, but the implementations (TorNS and StemNS) do not have that restriction. TorNS and StemNS allow a Prop279 plugin to advertise acceptance of any domain suffix (haven't explicitly tried the root zone as an suffix, but if that doesn't work, it's a bug that should be easy to fix) and can resolve them to any result (e.g. an IP address, a .onion domain, or another DNS name a la CNAME). Cheers, -- -Jeremy Rand Lead Application Engineer at Namecoin Mobile email: jeremyrandmob...@airmail.cc Mobile OpenPGP: 2158 0643 C13B B40F B0FD 5854 B007 A32D AB44 3D9C Send non-security-critical things to my Mobile with OpenPGP. Please don't send me unencrypted messages. My business email jer...@veclabs.net is having technical issues at the moment. signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Support for full DNS resolution and DNSSEC validation
Hey Jeremy, On 2020/05/15 15:53, Jeremy Rand wrote: > FYI I already wrote a Prop279 provider that looks up the names via DNS > (it's aptly named "dns-prop279"); it does pretty much exactly what you > describe. It doesn't handle DNSSEC validation itself (it assumes that > you've specified a DNS server that you trust -- most likely one running > on localhost). Stream isolation can be handled via an EDNS0 field (and > I'm guessing it would not be difficult to patch an existing DNS server > to respect that EDNS0 field). I wouldn't be surprised if it's easy to > make dns-prop279 do DNSSEC validation itself (and not use a > localhost-based DNS server) if that's desired -- the library it uses > (miekg/dns) does claim to support DNSSEC validation, though I've never > tried testing that feature. Very interesting. I think proposal #279 only tries to solve the subset of name look ups, which is about looking up onions from a human name. The work in this thread is to replace all name lookups *except* for Onions. It could very well be that it would be easier to extend proposal #279 by having it handle all lookups and not just for .onion's, but I think my intuition says that it should be two different systems as onion lookups is still a much more open question whereas Tor will need to support ordinary DNS for many years into the future. If `OnionNamePlugin` allowed you to specify `.*` for "everything" as the TLD specifier, then it might be possible to implement such system using proposal #279 :-) All the best, Alex. -- Alexander Færøy ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Support for full DNS resolution and DNSSEC validation
On Fri, 2020-05-15 at 15:29 +, Alexander Færøy wrote: > Hello Christian, > Hi Alex! > On 2020/04/26 19:37, Christian Hofer wrote: > > I have a proposal regarding DNS name resolution. > > > > Ticket: https://trac.torproject.org/projects/tor/ticket/34004 > > Proposal: > > https://trac.torproject.org/projects/tor/attachment/ticket/34004/317-secure-dns-name-resolution.txt > > Implementation: https://github.com/torproject/tor/pull/1869 > > > > All functioniality is behind the DNSResolver feature flag, so don't > > forget to activate it before you start testing. > > > > Please let me know what you think. > > Thanks for doing this work. I think our DNS subsystem has been > lacking > behind for a while. This work is exciting. > > Generally, after having done one pass over your code, I think the > source > code is good quality, especially if this is your first contribution > to > Tor! However, I think this is going to be a bit problematic for us to > import. > > It will be hard, if not impossible, for Tor's Network Team to adopt > 27k > LOC's in one pull-request. We will have to have multiple people going > over each line repeatedly and try to build up some confidence in this > code. If we are to go down this path, with having a complete DNS > subsystem in Tor, we need to add some capacity from our side to take > this in and maintain it. I think that with the recent layoffs in Tor, > it > will be hard to achieve in a time-frame that is fair towards you. > There are not many changes to the existing code, but most of the code is new. How could I prepare the changes to simplify the review? Since most parts depend on other parts within this change, I don't think it would be a good approach to split them up in multiple PRs? > One of the goals with our specification process is to have a set of > documents, which allows other people to understand how Tor is working > to > the point where they should be able to implement Tor from scratch if > they found that useful. This isn't always possible today, but this is > a > goal we should have in mind. Your proposal is mostly a specification > of > the *implementation* of the DNS resolver patches and doesn't contain > any > information on any changes to the network layer of Tor. Instead, > those > seem to be referenced as the various DNS related RFCs from the IETF. > Configuration options of the Tor binary is largely an implementation > detail. > There is only one entry and one exit point, apart from this there are no further changes to the network layer if you consider these changes even as a change to the network layer. See the sections "SocksPort related changes" and "DNSPort related changes" in the proposal. The DNS resolver implementation should of course comply to the DNS RFCs. I would like to try to improve the specification. So, you suggest to remove the section about configuration options and add more details about network layer changes? > I wonder if it would make more sense to have an onion-aware > DNSSEC-enabled resolver *outside* of the Tor binary and have a way > for > Tor to query an external tool for DNS lookups. Such tool should be > allowed to use Tor itself for transport of the actual queries. One of > the best parts of Tor (in my opinion) is the Pluggable Transport > subsystem. This subsystem allows external developers, researchers, > and > hackers to build new technology that benefits users in censored areas > *without* having to alter a single line of C code in tor.git. > > Let's say we had a "Pluggable DNS" layer in Tor. Users would be able > to > configure their Tor process to *never* use the built-in DNS subsystem > in > Tor, but instead outsource it to an external process that Tor spawns > on > startup. This process could use .onion's to reach a > DNS-over-(TLS|HTTPS|TCP) server as onions themselves aren't looked up > via DNS. > Sounds great. Let's start? > A "Pluggable DNS" subsystem would be much less code, I believe, and > it > wouldn't require us to have a DNS+DNSSEC implementation in the heart > of > Tor to maintain in the future. Such a system would be similar to the > proposed design for Name => Onion lookups defined in proposal #279 by > asn, yawning, and dgoulet. > > Lastly, I assume it's just for testing purpose, but I don't think we > could ship with CloudFlare's DNS-over-Onion services as the default > servers for a feature like this without having a discussion in the > community about it first :-) > You are right. There is even the DNSResolver flag that defaults to off, which completely disables this feature. > All the best, > Alex. > BR Christian ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Support for full DNS resolution and DNSSEC validation
Alexander Færøy: > I wonder if it would make more sense to have an onion-aware > DNSSEC-enabled resolver *outside* of the Tor binary and have a way for > Tor to query an external tool for DNS lookups. Such tool should be > allowed to use Tor itself for transport of the actual queries. One of > the best parts of Tor (in my opinion) is the Pluggable Transport > subsystem. This subsystem allows external developers, researchers, and > hackers to build new technology that benefits users in censored areas > *without* having to alter a single line of C code in tor.git. > > Let's say we had a "Pluggable DNS" layer in Tor. Users would be able to > configure their Tor process to *never* use the built-in DNS subsystem in > Tor, but instead outsource it to an external process that Tor spawns on > startup. This process could use .onion's to reach a > DNS-over-(TLS|HTTPS|TCP) server as onions themselves aren't looked up > via DNS. > > A "Pluggable DNS" subsystem would be much less code, I believe, and it > wouldn't require us to have a DNS+DNSSEC implementation in the heart of > Tor to maintain in the future. Such a system would be similar to the > proposed design for Name => Onion lookups defined in proposal #279 by > asn, yawning, and dgoulet. Hi Alex, FYI I already wrote a Prop279 provider that looks up the names via DNS (it's aptly named "dns-prop279"); it does pretty much exactly what you describe. It doesn't handle DNSSEC validation itself (it assumes that you've specified a DNS server that you trust -- most likely one running on localhost). Stream isolation can be handled via an EDNS0 field (and I'm guessing it would not be difficult to patch an existing DNS server to respect that EDNS0 field). I wouldn't be surprised if it's easy to make dns-prop279 do DNSSEC validation itself (and not use a localhost-based DNS server) if that's desired -- the library it uses (miekg/dns) does claim to support DNSSEC validation, though I've never tried testing that feature. I originally wrote dns-prop279 for Namecoin purposes, but I see no reason it couldn't be used to achieve DNSSEC support in Tor. If there's interest in pursuing this, let me know, I'm happy to contribute. Code is at https://github.com/namecoin/dns-prop279 Cheers, -- -Jeremy Rand Lead Application Engineer at Namecoin Mobile email: jeremyrandmob...@airmail.cc Mobile OpenPGP: 2158 0643 C13B B40F B0FD 5854 B007 A32D AB44 3D9C Send non-security-critical things to my Mobile with OpenPGP. Please don't send me unencrypted messages. My business email jer...@veclabs.net is having technical issues at the moment. signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Support for full DNS resolution and DNSSEC validation
On Thu, 2020-05-14 at 15:56 -0400, David Goulet wrote: > On 26 Apr (19:37:56), Christian Hofer wrote: > > Hi there, > > Greetings Christian! > Hi David! > > I have a proposal regarding DNS name resolution. > > > > Ticket: https://trac.torproject.org/projects/tor/ticket/34004 > > Proposal: > > https://trac.torproject.org/projects/tor/attachment/ticket/34004/317-secure-dns-name-resolution.txt > > Implementation: https://github.com/torproject/tor/pull/1869 > > First, this is quite impressive piece of work. I was _NOT_ expecting > a 27k > line diff ;). > Yeah this might look scary at first, but if you take a closer look it is not that bad. More than half of the lines account for tests: 15,437 3,584 test_crypto_dnssec.c 1,456 test_crypto_dnssec_openssl.c 726 test_dns_message.c 5,105 test_dns_resolver.c 1,389 test_dns_string.c 3,177 test_dns_wireformat.c Another big part accounts for containers and translators: 6,531 They don't contain much logic and their main purpose is to translate DNS message <-> wireformat and DNS message <-> string. So this area shouldn't change very often, except you want to add support for new DNS types or the specification for an existing DNS type changes, which shouldn't actually happen. 1,150 dns_message.c 173 dns_message.h 457 dns_message_st.h 199 dns_types.h 1,564 dns_string.c 80 dns_string.h 2,586 dns_wireformat.c 322 dns_wireformat.h The most interesting part accounts (only) for 4,970 lines (2,066 relevant lines). It contains all the DNS resolution and DNSSEC validation logic. * The dns_lookup_st contains state related types that lives in AP connections (entry_connection_t). * The dns_resolver is responsible for sending and receiving DNS messages, validating DNS messages using crypto_dnssec, and caching DNS messages. * The crypto_dnssec contains all required DNSSEC algorithms. lines / relevant / coverage / filename 57 /0 /100.0 / dns_lookup_st.h 2,564 /1,197 /98.75 / dns_resolver.c 179 /0 /100.0 / dns_resolver.h 1,478 / 684 /98.83 / crypto_dnssec.c 495 / 185 /99.46 / crypto_dnssec_openssl.c 197 /0 /100.0 / crypto_dnssec.h These numbers were taken from the coveralls report: https://coveralls.io/builds/30352518 > So the proposal looks very good. I like the idea very much. I > honestly thought > that you were about to propose a way for Tor to talk to an *external* > DNS > resolver client application (third part) but I see that client DNSSEC > is > basically implemented in tor with your patch which is... interesting? > > Before we go further, can you walk me through the reasons (if you had > thought > of it of course) why you didn't use something like libunbound? > I might be wrong, please correct me if so. The scenario I was aiming for is when the target address that is received within the socks handshake in connection_ap_handshake_process_socks is a hostname. As far as I understand it, in this case the name resolution is done at the exit relay that is also responsible for retrieving the contents from the target. This means that the user has to trust the exit relay in terms of name resolution? However, I am not sure if this is a valid use case or if you can cover this using libunbound. So please let me know. Using the DNS resolver would resolve the hostname upfront in a separate connection that is not related to the connection that is responsible for retrieving the contents from the target. Additionally, the user has control over the nameserver that should be used for name resolution and is also able to verify the response using DNSSEC. So this should reduces the number of entities that must be trusted. > There are side effects of adding DNSSEC client support (with our own > implementation) that we, people maintaining tor, have to become > DNSSEC expert > in some ways just to be able to understand what is happening in that > code, fix > issues but also possibly implement new features if any. That is where > a third > part library like unbound becomes very nice because they are the > experts at > providing such features. > I can understand this. > Of course, everytime we have to link to an external library we do it > carefully > and with considerations because of the "yet another attack" vector > problem. > But adding that much code to support a well known feature like DNSSEC > also has > huge implications for tor maintainability and security. > > Finally, something I noticed and made me itch a bit. You hardcoded a > lot of > .onions where one appears to be Cloudflare (?) resolver. What are the > other > addresses? I worry here because default options are always the one > used the > most so I'm concerned here about shipping hardcoded addresses > _within_ our C > code. > Regarding the the default configuration. There is a configuration called DNSResolver which defaults to off. As long as this flag is turned off nothing changes related to DNS resolution. Yes, the first
Re: [tor-dev] Support for full DNS resolution and DNSSEC validation
Hello Christian, On 2020/04/26 19:37, Christian Hofer wrote: > I have a proposal regarding DNS name resolution. > > Ticket: https://trac.torproject.org/projects/tor/ticket/34004 > Proposal: > https://trac.torproject.org/projects/tor/attachment/ticket/34004/317-secure-dns-name-resolution.txt > Implementation: https://github.com/torproject/tor/pull/1869 > > All functioniality is behind the DNSResolver feature flag, so don't > forget to activate it before you start testing. > > Please let me know what you think. Thanks for doing this work. I think our DNS subsystem has been lacking behind for a while. This work is exciting. Generally, after having done one pass over your code, I think the source code is good quality, especially if this is your first contribution to Tor! However, I think this is going to be a bit problematic for us to import. It will be hard, if not impossible, for Tor's Network Team to adopt 27k LOC's in one pull-request. We will have to have multiple people going over each line repeatedly and try to build up some confidence in this code. If we are to go down this path, with having a complete DNS subsystem in Tor, we need to add some capacity from our side to take this in and maintain it. I think that with the recent layoffs in Tor, it will be hard to achieve in a time-frame that is fair towards you. One of the goals with our specification process is to have a set of documents, which allows other people to understand how Tor is working to the point where they should be able to implement Tor from scratch if they found that useful. This isn't always possible today, but this is a goal we should have in mind. Your proposal is mostly a specification of the *implementation* of the DNS resolver patches and doesn't contain any information on any changes to the network layer of Tor. Instead, those seem to be referenced as the various DNS related RFCs from the IETF. Configuration options of the Tor binary is largely an implementation detail. I wonder if it would make more sense to have an onion-aware DNSSEC-enabled resolver *outside* of the Tor binary and have a way for Tor to query an external tool for DNS lookups. Such tool should be allowed to use Tor itself for transport of the actual queries. One of the best parts of Tor (in my opinion) is the Pluggable Transport subsystem. This subsystem allows external developers, researchers, and hackers to build new technology that benefits users in censored areas *without* having to alter a single line of C code in tor.git. Let's say we had a "Pluggable DNS" layer in Tor. Users would be able to configure their Tor process to *never* use the built-in DNS subsystem in Tor, but instead outsource it to an external process that Tor spawns on startup. This process could use .onion's to reach a DNS-over-(TLS|HTTPS|TCP) server as onions themselves aren't looked up via DNS. A "Pluggable DNS" subsystem would be much less code, I believe, and it wouldn't require us to have a DNS+DNSSEC implementation in the heart of Tor to maintain in the future. Such a system would be similar to the proposed design for Name => Onion lookups defined in proposal #279 by asn, yawning, and dgoulet. Lastly, I assume it's just for testing purpose, but I don't think we could ship with CloudFlare's DNS-over-Onion services as the default servers for a feature like this without having a discussion in the community about it first :-) All the best, Alex. -- Alexander Færøy ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Deprecating Tor Protocol Versions
Hi David, > On 15 May 2020, at 20:53, David Goulet wrote: > > On 15 May (13:58:06), teor wrote: >> >> Nick and I were talking about how we remove legacy features in tor, >> and their corresponding subprotocol versions. >> >> Here is a list of the current subprotocol versions: >> https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt#n2049 >> >> Here's a recent protocol version proposal, which deals with >> recommending and requiring new features: >> https://gitweb.torproject.org/torspec.git/tree/proposals/303-protover-removal-policy.txt >> >> But we don't have a similar proposal for removing support for older >> protocol versions from the tor codebase. >> >> For an example of what that proposal could look like, see our proposal >> for deprecating consensus methods: >> https://gitweb.torproject.org/torspec.git/tree/proposals/290-deprecate-consensus-methods.txt >> >> Here's the original conversation Nick and I had: >> https://github.com/torproject/tor/pull/1874#discussion_r423713579 >> >> But after reading our consensus methods deprecation proposal, I've >> changed my mind. I think that we should check for "protocol N, and >> any later version" by default. > > I agree that this is the right approach imo as well. > >> That's what we do for consensus methods, and it seems to work well. >> We can drop the earliest consensus methods, and recent tor versions >> just keep working. >> >> If we need an incompatible change, we can make it another protocol >> version, and recommend then require support for it. >> >> So here's an edited version of my notes on that ticket: >> >> There are a few instances of ">=" and "=" confusion in protocol >> versions. We should try to fix them all. >> >> It only matters when we remove protocol versions. We haven't >> really specified, tested, or exercised this functionality in >> practice. And so our reviewers lack experience. (And when we did >> discover a need for it with NSS and LinkAuth, it was more complicated >> than we expected.) >> >> I'd like to see a proposal that tells us how to check future protocol >> versions as they are added. Along with a migration plan for disabling >> protocol versions. >> >> So let's also open a ticket to check for "any future version". >> We should replace all "=" checks with ">=". Let's make sure we check >> all the places where we use protocol versions, even if they don't >> have a summary flag. >> >> Overall, I think it would be helpful if future protocol versions were >> orthogonal. Or if they depend on earlier features, that dependency >> should be clearly documented. (For example, Relay=3 IPv6 extends >> depends on Relay=2 EXTEND2 cells. So if we were checking EXTEND2 cell >> support, it would be Relay=2 or Relay=3.) > > At the moment, they do depend between each other last time I had that > discussion with Nick. As in the later in your example. > > Which means that supporting protocol version with a ">=" is consistent with > our "non-written expectations" that we have now. That's how most protocol versions work right now. The following protocol versions are exceptions: * Linkall Link protocols introduce incompatible changes, but a shared Link protocol is dynamically negotiated at runtime * LinkAuth=3 might not support LinkAuth=1 (RSA link authentication) * Padding=1 accidentally enabled for relays that don't support circuit-level padding Link is a special case because it's the lowest-level protocol, and negotiated at runtime. (And using an unknown feature terminates the connection.) LinkAuth became incompatible when we introduced NSS support. But a better design might have been: * LinkAuth=1 RSA only * LinkAuth=3 ed25519 and RSA * LinkAuth=4 ed25519 only Padding was a mistake :-( > So if I understand correctly, we'll need to add a new protocol version let say > N+1 in order to deprecate anything <= N ? > > As an example, Relay=4 could mean "deprecate Relay=2 and use only Relay=3" > > I'm +1 on this if iiuc. Yes, I think that's what we should do in future. So we should: * write a proposal or spec update, and * change the current code to use "protocol_list_supports_protocol_or_later() in most cases. I'm going to wait for asn or nickm to weigh in, before opening tickets. T signature.asc Description: Message signed with OpenPGP ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Deprecating Tor Protocol Versions
On 15 May (13:58:06), teor wrote: > Hi all, > > Nick and I were talking about how we remove legacy features in tor, > and their corresponding subprotocol versions. > > Here is a list of the current subprotocol versions: > https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt#n2049 > > Here's a recent protocol version proposal, which deals with > recommending and requiring new features: > https://gitweb.torproject.org/torspec.git/tree/proposals/303-protover-removal-policy.txt > > But we don't have a similar proposal for removing support for older > protocol versions from the tor codebase. > > For an example of what that proposal could look like, see our proposal > for deprecating consensus methods: > https://gitweb.torproject.org/torspec.git/tree/proposals/290-deprecate-consensus-methods.txt > > Here's the original conversation Nick and I had: > https://github.com/torproject/tor/pull/1874#discussion_r423713579 > > But after reading our consensus methods deprecation proposal, I've > changed my mind. I think that we should check for "protocol N, and > any later version" by default. I agree that this is the right approach imo as well. > > That's what we do for consensus methods, and it seems to work well. > We can drop the earliest consensus methods, and recent tor versions > just keep working. > > If we need an incompatible change, we can make it another protocol > version, and recommend then require support for it. > > So here's an edited version of my notes on that ticket: > > There are a few instances of ">=" and "=" confusion in protocol > versions. We should try to fix them all. > > It only matters when we remove protocol versions. We haven't > really specified, tested, or exercised this functionality in > practice. And so our reviewers lack experience. (And when we did > discover a need for it with NSS and LinkAuth, it was more complicated > than we expected.) > > I'd like to see a proposal that tells us how to check future protocol > versions as they are added. Along with a migration plan for disabling > protocol versions. > > So let's also open a ticket to check for "any future version". > We should replace all "=" checks with ">=". Let's make sure we check > all the places where we use protocol versions, even if they don't > have a summary flag. > > Overall, I think it would be helpful if future protocol versions were > orthogonal. Or if they depend on earlier features, that dependency > should be clearly documented. (For example, Relay=3 IPv6 extends > depends on Relay=2 EXTEND2 cells. So if we were checking EXTEND2 cell > support, it would be Relay=2 or Relay=3.) At the moment, they do depend between each other last time I had that discussion with Nick. As in the later in your example. Which means that supporting protocol version with a ">=" is consistent with our "non-written expectations" that we have now. So if I understand correctly, we'll need to add a new protocol version let say N+1 in order to deprecate anything <= N ? As an example, Relay=4 could mean "deprecate Relay=2 and use only Relay=3" I'm +1 on this if iiuc. Cheers! David -- dApigzB8NtOQEAlKqhqbshxjxOMakjiX9LGU9wvhFqs= signature.asc Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
