Re: [tor-dev] When RFC 7686 and transparent proxies collide

[kaizushi]

The thing with this issue, and their ignorant attitude to it, is that it 
is this easy to patch. The if statement that does this could simply be 
nested in another that checks for an environment variable, giving users 
an option to enable .onion resolution.


diff --git a/lib/hostip.c b/lib/hostip.c
index e7c318a..c0e2583 100644
--- a/lib/hostip.c
+++ b/lib/hostip.c
@@ -693,12 +693,12 @@ enum resolve_t Curl_resolv(struct Curl_easy *data,
   struct connectdata *conn = data->conn;
   /* We should intentionally error and not resolve .onion TLDs */
   size_t hostname_len = strlen(hostname);
-  if(hostname_len >= 7 &&
- (curl_strequal([hostname_len - 6], ".onion") ||
-  curl_strequal([hostname_len - 7], ".onion."))) {
-failf(data, "Not resolving .onion address (RFC 7686)");
-return CURLRESOLV_ERROR;
-  }
+//   if(hostname_len >= 7 &&
+// (curl_strequal([hostname_len - 6], ".onion") ||
+//  curl_strequal([hostname_len - 7], ".onion."))) {
+//failf(data, "Not resolving .onion address (RFC 7686)");
+//return CURLRESOLV_ERROR;
+//  }
   *entry = NULL;
 #ifndef CURL_DISABLE_DOH
   conn->bits.doh = FALSE; /* default is not */

On 2023-11-13 16:34, Alec Muffett wrote:

Hi Shawn!

On Mon, 13 Nov 2023 at 15:54, Shawn Webb 
wrote:


I agree that infoleaks, especially of .onion DNS requests, is
problematic. However, I disagree that prohibiting it in broadly
monocultured libraries (libcurl) is an advisable approach.


If Curl is outright banning ".onion" at the level of the Curl source
code, I would not support that on the grounds that are described in
bullet point 2 of section 2, here, which I will requote in full:

https://datatracker.ietf.org/doc/html/rfc7686#section-2


_2. Application Software: Applications (including proxies) that
implement the Tor protocol MUST recognize .onion names as special by
either accessing them directly or using a proxy (e.g., SOCKS
[RFC1928]) to do so. Applications that do not implement the Tor
protocol SHOULD generate an error upon the use of .onion and SHOULD
NOT perform a DNS lookup._


...but I will also note that I have not (maybe I missed it?) seen
bullet point 3 being referenced in this thread:


_3. Name Resolution APIs and Libraries: Resolvers MUST either
respond to requests for .onion names by resolving them according to
[tor-rendezvous] or by responding with NXDOMAIN [RFC1035]._


I see Curl/LibCurl in the context of an application (§2) which makes
calls into name resolution apis (§3).  I regret that the text of §2
("...that do not implement the Tor protocol...") is ambiguous in its
scope, and would prefer something about the app being incapable of
dealing with and unaware of the existence of multiple possible
name-resolution namespaces, instead.

Likewise I feel that _"Applications that do not implement the Tor
protocol SHOULD generate an error"_ would benefit from being rewritten
to acknowledge that the desirable error _may_ come passively as a
consequence of the name resolution libraries that are called, rather
than via some manner of "policing" invocation of the .onion domain.

tldr: I feel it should not be up to curl/libcurl to be policing the
use of ".onion" ... but I am very content for its chosen DNS-based
name resolution backends to be doing do so.

However convenient it may be to attempt to bolt ".onion" onto the DNS
for mobile or Whonix or whatever development, there's no avoiding that
in several ways it is both risky and unspecified to do that. I can't
fix that for anyone, but I also cannot deny that it's pushing water
uphill to attempt it.

My personal sense has always been that at some point in the future
systems-level Tor onion access might need to be provided via a network
interface that presents and routes AF_ONION addresses; but until then
(and per the linked video) new directions in DNS provide us with a
secondary possible solution: Those (mobile?) people who cannot get the
benefit of a solution via /etc/nsswitch.conf should probably have
their handsets reconfigured to do "DNS" lookup via DNS-over-HTTPS[1]
to a local HTTPS service that both understands and
treats-in-isolation, all ".onion" lookups.

Of course this does not solve apps which do their own DNS resolution,
yadda yadda, but then there is no way no NSS to solve them, either;
also this points to the importance of a TCB being curated with a
"systems" perspective (including NSS integration?) rather than trying
to bolt stuff together to get to a merely "functional" solution.

Overall: long-term continuing to shoehorn Onions into DNS for
transparent-proxy name resolution is relentlessly moving towards being
actively painful. I feel that now would be a good time to embrace a
different, ideally standards-compliant / more-futureproof approach.

-a

[1] Fun reading on a related topic:
https://github.com/alecmuffett/dohot


While I can appreciate and understand the many nuances of this
particular problem, it is one that is indeed difficult to solve.

Are there other 

Re: [tor-dev] When RFC 7686 and transparent proxies collide

[Alec Muffett]

Hi Shawn!

On Mon, 13 Nov 2023 at 15:54, Shawn Webb 
wrote:

> I agree that infoleaks, especially of .onion DNS requests, is
> problematic. However, I disagree that prohibiting it in broadly
> monocultured libraries (libcurl) is an advisable approach.
>

If Curl is outright banning ".onion" at the level of the Curl source code,
I would not support that on the grounds that are described in bullet point
2 of section 2, here, which I will requote in full:


*https://datatracker.ietf.org/doc/html/rfc7686#section-2
*

*2. Application Software: Applications (including proxies) that implement
> the Tor protocol MUST recognize .onion names as special by either accessing
> them directly or using a proxy (e.g., SOCKS [RFC1928]) to do so.
> Applications that do not implement the Tor protocol SHOULD generate an
> error upon the use of .onion and SHOULD NOT perform a DNS lookup.*


...but I will also note that I have not (maybe I missed it?) seen bullet
point 3 being referenced in this thread:

*3. Name Resolution APIs and Libraries: Resolvers MUST either respond to
> requests for .onion names by resolving them according to [tor-rendezvous]
> or by responding with NXDOMAIN [RFC1035].*


I see Curl/LibCurl in the context of an application (§2) which makes calls
into name resolution apis (§3).  I regret that the text of §2 ("...that do
not implement the Tor protocol...") is ambiguous in its scope, and would
prefer something about the app being incapable of dealing with and unaware
of the existence of multiple possible name-resolution namespaces, instead.

Likewise I feel that *"Applications that do not implement the Tor protocol
SHOULD generate an error"* would benefit from being rewritten to
acknowledge that the desirable error *may* come passively as a consequence
of the name resolution libraries that are called, rather than via some
manner of "policing" invocation of the .onion domain.

tldr: I feel it should not be up to curl/libcurl to be policing the use of
".onion" ... but I am very content for its chosen DNS-based name resolution
backends to be doing do so.

However convenient it may be to attempt to bolt ".onion" onto the DNS for
mobile or Whonix or whatever development, there's no avoiding that in
several ways it is both risky and unspecified to do that. I can't fix that
for anyone, but I also cannot deny that it's pushing water uphill to
attempt it.

My personal sense has always been that at some point in the future
systems-level Tor onion access might need to be provided via a network
interface that presents and routes AF_ONION addresses; but until then (and
per the linked video) new directions in DNS provide us with a secondary
possible solution: Those (mobile?) people who cannot get the benefit of a
solution via /etc/nsswitch.conf should probably have their handsets
reconfigured to do "DNS" lookup via DNS-over-HTTPS[1] to a local HTTPS
service that both understands and treats-in-isolation, all ".onion" lookups.

Of course this does not solve apps which do their own DNS resolution, yadda
yadda, but then there is no way no NSS to solve them, either; also this
points to the importance of a TCB being curated with a "systems"
perspective (including NSS integration?) rather than trying to bolt stuff
together to get to a merely "functional" solution.

Overall: long-term continuing to shoehorn Onions into DNS for
transparent-proxy name resolution is relentlessly moving towards being
actively painful. I feel that now would be a good time to embrace a
different, ideally standards-compliant / more-futureproof approach.

-a


[1] Fun reading on a related topic: https://github.com/alecmuffett/dohot









>
> While I can appreciate and understand the many nuances of this
> particular problem, it is one that is indeed difficult to solve.
>
> Are there other commonalities between "infoleaky" deployments that
> could be improved? It seems to me that outright prohibition should be
> a method of last resort. Are we already there?
>
> Thanks,
>
> --
> Shawn Webb
> Cofounder / Security Engineer
> HardenedBSD
>
>
> https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
> ___
> tor-dev mailing list
> tor-dev@lists.torproject.org
> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
>


-- 
https://alecmuffett.com/about
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] When RFC 7686 and transparent proxies collide

[Shawn Webb]

On Mon, Nov 13, 2023 at 03:01:15PM +, Alec Muffett wrote:
> Hi! I'm one of the authors of RFC 7686.
> 
> Although myself and Appelbaum[1] are cited on it, the document is the
> result of a huge amount of argument and input from many people (shout out
> to Mark Nottingham most especially, whom I feel should have gotten an
> author credit) on various IETF maillists, and against considerable pressure
> from some in the DNS, ICANN and other naming communities who didn't want to
> set a precedent nor open a gateway to "more exceptions for new TLDs".
> 
> Reading this thread I can confirm that transparent proxies (of several
> implementations) were considered to be outside of the scope of the
> specification, not by intention but rather as a consequence of the DNS
> community being **extremely motivated** to prevent the existence or
> official sanction for anything that could be construed as extending DNS by
> default, without going through the full DNS standards processes.
> 
> There was (and, I suspect, remains) very much an attitude of DNS being
> practically sacred and unamendable unless overseen by an elite priesthood
> of experts, and as such all the "Tor" stuff was presented to them and to
> the standards committees as being *"something utterly different from DNS,
> really, we swear, honest, this is more of a namespace bookkeeping issue
> than anything else"* — in order to prevent the standard being shot to death
> by DNS zealots.
> 
> Also: the ".onion" resolutions which "leak" into the global DNS network
> were at the time — and probably remain — both a nuisance and a huge
> information leak that gets mined by various state security agencies; those
> participants who perceived that as an issue saw the RFC as an issue to
> address both the noise and the leak by drawing a very firm line between DNS
> and Onion addressing, hence the text which is under discussion here.
> 
> Myself? I am looking at the application wants sympathetically, but with a
> perspective of 36 years of Unix. To be frank I believe that the process we
> went through and the points that were made during the RFC are prettymuch
> valid, and I believe that using DNS as transport for a hack to resolve
> Onion addresses is ill-advised, even massively dangerous, for the reasons
> described.
> 
> I have sympathy for the DNS resolver community being explicit about banning
> ".onion" and I think that doing so would be good for Onion privacy; but
> that doesn't mean that I find the *need* to resolve .onion addresses to a
> virtual IP address to be illegitimate.
> 
> Back in the 1990s we used to deal with their being different namespaces for
> different networks[2] using the /etc/nsswitch.conf service which was
> literally designed[3] to address this kind of problem; the acronym stands
> for "Name Service Switch"[4] and it's how local naming in huge intranets
> used to be implemented.
> 
> As such, why not just build a small service to perform this mapping lookup
> properly and splice it into nsswitch.conf, and save yourself from having to
> police the DNS clients for data leakage re: "This IP address just tried to
> look up `supersecretleakysite234567abcde.onion`"?

Hey Alec,

Thank you for taking the time to write such an informative response.
I, too, come from a hihgly UNIX-centric background and can understand
the direction you are going in. Though I don't have 36 years of UNIX
experience, I've now spent more than half my age in UNIX (or
UNIX-like) environments, hacking both on the kernel and userland.
We're speaking the same language. :-)

The problem with relying on system modifications, like nsswitch.conf,
is that the switch to everything-on-smartphone makes solutions like
that impossible or impractical. I'm unaware of even a single major
smartphone vendor that permits that level of customization. Granted,
my own lack of awareness does not mean there isn't a phone out there
that permits modifying system configuration files--just that my
knowledge is incomplete.

I agree that infoleaks, especially of .onion DNS requests, is
problematic. However, I disagree that prohibiting it in broadly
monocultured libraries (libcurl) is an advisable approach.

While I can appreciate and understand the many nuances of this
particular problem, it is one that is indeed difficult to solve.

Are there other commonalities between "infoleaky" deployments that
could be improved? It seems to me that outright prohibition should be
a method of last resort. Are we already there?

Thanks,

-- 
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc


signature.asc
Description: PGP signature
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] When RFC 7686 and transparent proxies collide

[Alec Muffett]

Hi! I'm one of the authors of RFC 7686.

Although myself and Appelbaum[1] are cited on it, the document is the
result of a huge amount of argument and input from many people (shout out
to Mark Nottingham most especially, whom I feel should have gotten an
author credit) on various IETF maillists, and against considerable pressure
from some in the DNS, ICANN and other naming communities who didn't want to
set a precedent nor open a gateway to "more exceptions for new TLDs".

Reading this thread I can confirm that transparent proxies (of several
implementations) were considered to be outside of the scope of the
specification, not by intention but rather as a consequence of the DNS
community being **extremely motivated** to prevent the existence or
official sanction for anything that could be construed as extending DNS by
default, without going through the full DNS standards processes.

There was (and, I suspect, remains) very much an attitude of DNS being
practically sacred and unamendable unless overseen by an elite priesthood
of experts, and as such all the "Tor" stuff was presented to them and to
the standards committees as being *"something utterly different from DNS,
really, we swear, honest, this is more of a namespace bookkeeping issue
than anything else"* — in order to prevent the standard being shot to death
by DNS zealots.

Also: the ".onion" resolutions which "leak" into the global DNS network
were at the time — and probably remain — both a nuisance and a huge
information leak that gets mined by various state security agencies; those
participants who perceived that as an issue saw the RFC as an issue to
address both the noise and the leak by drawing a very firm line between DNS
and Onion addressing, hence the text which is under discussion here.

Myself? I am looking at the application wants sympathetically, but with a
perspective of 36 years of Unix. To be frank I believe that the process we
went through and the points that were made during the RFC are prettymuch
valid, and I believe that using DNS as transport for a hack to resolve
Onion addresses is ill-advised, even massively dangerous, for the reasons
described.

I have sympathy for the DNS resolver community being explicit about banning
".onion" and I think that doing so would be good for Onion privacy; but
that doesn't mean that I find the *need* to resolve .onion addresses to a
virtual IP address to be illegitimate.

Back in the 1990s we used to deal with their being different namespaces for
different networks[2] using the /etc/nsswitch.conf service which was
literally designed[3] to address this kind of problem; the acronym stands
for "Name Service Switch"[4] and it's how local naming in huge intranets
used to be implemented.

As such, why not just build a small service to perform this mapping lookup
properly and splice it into nsswitch.conf, and save yourself from having to
police the DNS clients for data leakage re: "This IP address just tried to
look up `supersecretleakysite234567abcde.onion`"?

 - alec

[1] yes, I know
[2] see this https://www.youtube.com/watch?v=pebRZyg_bh8
[3] https://man7.org/linux/man-pages/man5/nsswitch.conf.5.html
[4] https://man7.org/linux/man-pages/man5/nss.5.html
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] When RFC 7686 and transparent proxies collide

[Shawn Webb]

On Sun, Nov 05, 2023 at 12:57:40AM +, kaizu...@cock.li wrote:
> They blatantly don't care and deny the issue, and don't care that it breaks
> a feature Tor has had for over a decade. They respond with fallacies and
> irrational garbage. They don't acknowledge that an environment variable
> would fix the issue and tacitly admit not to care.
> 
> Fortunately I run Gentoo and this stupid bit of code is very easy to fix on
> that distribution. I made this patch and used /etc/portage/patches to fix
> it, which is a page on the Gentoo Wiki: https://bpa.st/Q4VQc - I think this
> illustrates how simple this issue is as well as fixing it for all users.
> 
> However, the transproxy feature allows for people to idiot proof their
> systems and share with people which is supposed to be what open source is
> about. I hope as many people call out this kind of ignorant behavior for
> what it clearly is. You can see the depths of their stupidity and denial
> here...

The prohibition of .onion in libcurl presents a problem for existing
production HardenedBSD deployments in certain environments where
transparent proxying is used rather than SOCKS. Systems living in such
environments now cannot be updated, even to patch security
vulnerabilities.

HardenedBSD provides Tor Onion Service endpoints for all its
publicly-facing infrastructure. Users can go from zero to dev to prod
completely behind a fully Tor-ified network (via transparent
proxying).

But now that libcurl, which the HardenedBSD package manager uses,
prohibits .onion DNS record resolutions, our users are not able to
access our Tor Onion Endpoints, leaving them insecure and vulnerable
to exploitation by bad actors.

With the curl project's unwillingness to compromise, HardenedBSD is
now forced to maintain a patch to revert the prohibition. But that
only applies to HardenedBSD users. What about users of other operating
systems with similar transparent proxy deployments?

I would strongly suggest to officially clarify the RFC or strike the
relevant section altogether. Without doing so, the curl project seems
unwilling to make the prohibition a choice that can be made at runtime
by the user, even.

Please remember that these kinds of things have real and tangible
human consequences. Some of our users remain in harms way to this day
because of this.

Thanks,

-- 
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc


signature.asc
Description: PGP signature
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] When RFC 7686 and transparent proxies collide

[kaizushi]

On 2023-11-03 16:29, rhatto wrote:

On Thu, Sep 14, 2023 at 08:22:22PM +1000, Matt Jolly wrote:

I recently inadvertently opened a much larger can of worms than I'd
intended when fixing a bug reported downstream where cURL would,
when configured with certain DNS backends, fail to resolve .onion
addresses.

[...]


Hi Matt,

Thanks for the detailed problem description and references, and I'm 
sorry to

hear about the trouble you're getting when trying to fix it.


I've asked in the Tor Specifications issue (inspired by Silvio's
suggestions), and in the cURL issue, but I seem to be getting nowhere
and the impacted users are clamouring for a quick band-aid solution,
which I feel will work out worse for everyone in the long run:
> How can client applications (safely):
>
> 1.discover that they're in a Tor-enabled environment
> 2.resolve onion services only via Tor in that circumstance
> 3.not leak .onion resolution attempts at all
>
> Right now, not making these requests in the first place is the
> safest (and correct) thing to do, however inconvenient it may be.
> Rather than immediately trying to come up with a band-aid approach
> to this problem, a sane mechanism needs to be implemented to:
>
> 1.prevent each application from coming up with their own solution
> 2.prevent inconsistency in .onion resolution (i.e. no "oh it only
> leaks if DO_ONION_RESOLUTION is set")
> 3.provide a standardised mechanism for applications that want to be Tor
> aware to discover that they're in a Tor-enabled environment.
I'm not particularly attached to that last point, but it's worth 
discussing.


On a related note:
-is the use of a transparent proxy recommended?


I consider it a valid use case. The only concern I'm aware is about
proxy leaks. I don't see any reason not to recommend transparent Tor
proxies if using a configuration that avoids such leakage.


-is there a sane alternative that involves as minimal configuration
as possible for these users?


Can't think of anything right now besides what was already covered in
length in the threads you mentioned. Instead, I'll focus in the overall
problem.


I'm not sure what the best way forward is here, but I'm hoping that
actual Tor developers might have a useful opinion on the matter, or
at least be able to point me in the right direction.


The long analysis below does not necessarily represents an "official 
Tor

position". It may be frustrating that there is no "official" position
right now, maybe because this still need further discussion, and also
because it may be a hard topic.

# Interpreting RFC 7686

Let's start with an hermeneutical exercise, by reading carefully what
RFC 7686 says:


2.  Application Software: Applications (including proxies) that
implement the Tor protocol MUST recognize .onion names as special
by either accessing them directly or using a proxy (e.g., SOCKS
[RFC1928]) to do so.  Applications that do not implement the Tor
protocol SHOULD generate an error upon the use of .onion and
SHOULD NOT perform a DNS lookup.


Applications MUST support .onion either directly or through a proxy. 
The RFC
gives SOCKS as an example, but it could as well have mentioned 
transparent
proxies. So applications could easily allow environment variables, 
command line
flags, configuration parameters or "environment detection" to indicate 
that

they support .onion domains.

I don't see why an upstream shouldn't allow users and operating systems
toggle the behavior regarding RFC 7686. Seems like the best default
approach so far is to generate an error, but nothing is blocking
applications to expose ways users can indicate that their systems acts
as proxies for .onion.

I suggest we take a corollary from this RFC portion: applications 
implementing
the Tor protocol are those which either access .onion addresses 
directly or use
a proxy that supports Tor. If the application does not support any way 
to

properly resolve .onion addresses, then it must make sure to refuse any
attempts to resolve .onion addresses. Nothing is blocking an 
application to
operate in two modes: rejecting .onion resolution attempts by default, 
except

if it's told to do so.

I agree that "implement the Tor protocol" is rather vague. The "Tor 
protocol"
might mean a lot of stuff, but I'm assuming here the _intention_ behind 
this
statement is this: in order to build a Tor-enabled application, one has 
to

interface with Tor somehow (being a Tor client or using a proxy).

As for curl, it seems to me that upstream wants uniform behavior with 
all DNS
backend libraries. And c-ares is rejecting .onion resolution, so for 
the sake

of uniformity curl is just rejecting it for all backends.

But the RFC explicitly mentions "Applications" and not "Libraries". 
Shall we

assume then that the RFC is referring to curl, but not to c-ares?

Would c-ares maintainers be free do follow whatever they prefer?

Or this is not a concern for c-ares, since it's a library and not an
application? In 

Re: [tor-dev] When RFC 7686 and transparent proxies collide

[rhatto]

On Thu, Sep 14, 2023 at 08:22:22PM +1000, Matt Jolly wrote:
> I recently inadvertently opened a much larger can of worms than I'd
> intended when fixing a bug reported downstream where cURL would,
> when configured with certain DNS backends, fail to resolve .onion
> addresses.
>
> [...]

Hi Matt,

Thanks for the detailed problem description and references, and I'm sorry to
hear about the trouble you're getting when trying to fix it.

> I've asked in the Tor Specifications issue (inspired by Silvio's
> suggestions), and in the cURL issue, but I seem to be getting nowhere
> and the impacted users are clamouring for a quick band-aid solution,
> which I feel will work out worse for everyone in the long run:
> > How can client applications (safely):
> > 
> > 1.discover that they're in a Tor-enabled environment
> > 2.resolve onion services only via Tor in that circumstance
> > 3.not leak .onion resolution attempts at all
> > 
> > Right now, not making these requests in the first place is the
> > safest (and correct) thing to do, however inconvenient it may be.
> > Rather than immediately trying to come up with a band-aid approach
> > to this problem, a sane mechanism needs to be implemented to:
> > 
> > 1.prevent each application from coming up with their own solution
> > 2.prevent inconsistency in .onion resolution (i.e. no "oh it only
> > leaks if DO_ONION_RESOLUTION is set")
> > 3.provide a standardised mechanism for applications that want to be Tor
> > aware to discover that they're in a Tor-enabled environment.
> I'm not particularly attached to that last point, but it's worth discussing.
>
> On a related note:
> -is the use of a transparent proxy recommended?

I consider it a valid use case. The only concern I'm aware is about
proxy leaks. I don't see any reason not to recommend transparent Tor
proxies if using a configuration that avoids such leakage.

> -is there a sane alternative that involves as minimal configuration
> as possible for these users?

Can't think of anything right now besides what was already covered in
length in the threads you mentioned. Instead, I'll focus in the overall
problem.

> I'm not sure what the best way forward is here, but I'm hoping that
> actual Tor developers might have a useful opinion on the matter, or
> at least be able to point me in the right direction.

The long analysis below does not necessarily represents an "official Tor
position". It may be frustrating that there is no "official" position
right now, maybe because this still need further discussion, and also
because it may be a hard topic.

# Interpreting RFC 7686

Let's start with an hermeneutical exercise, by reading carefully what
RFC 7686 says:

> 2.  Application Software: Applications (including proxies) that
> implement the Tor protocol MUST recognize .onion names as special
> by either accessing them directly or using a proxy (e.g., SOCKS
> [RFC1928]) to do so.  Applications that do not implement the Tor
> protocol SHOULD generate an error upon the use of .onion and
> SHOULD NOT perform a DNS lookup.

Applications MUST support .onion either directly or through a proxy. The RFC
gives SOCKS as an example, but it could as well have mentioned transparent
proxies. So applications could easily allow environment variables, command line
flags, configuration parameters or "environment detection" to indicate that
they support .onion domains.

I don't see why an upstream shouldn't allow users and operating systems
toggle the behavior regarding RFC 7686. Seems like the best default
approach so far is to generate an error, but nothing is blocking
applications to expose ways users can indicate that their systems acts
as proxies for .onion.

I suggest we take a corollary from this RFC portion: applications implementing
the Tor protocol are those which either access .onion addresses directly or use
a proxy that supports Tor. If the application does not support any way to
properly resolve .onion addresses, then it must make sure to refuse any
attempts to resolve .onion addresses. Nothing is blocking an application to
operate in two modes: rejecting .onion resolution attempts by default, except
if it's told to do so.

I agree that "implement the Tor protocol" is rather vague. The "Tor protocol"
might mean a lot of stuff, but I'm assuming here the _intention_ behind this
statement is this: in order to build a Tor-enabled application, one has to
interface with Tor somehow (being a Tor client or using a proxy).

As for curl, it seems to me that upstream wants uniform behavior with all DNS
backend libraries. And c-ares is rejecting .onion resolution, so for the sake
of uniformity curl is just rejecting it for all backends.

But the RFC explicitly mentions "Applications" and not "Libraries". Shall we
assume then that the RFC is referring to curl, but not to c-ares?

Would c-ares maintainers be free do follow whatever they prefer?

Or this is not a concern for c-ares, since it's a library and not an

[tor-dev] When RFC 7686 and transparent proxies collide

[Matt Jolly]

Hello Everyone,
I recently inadvertently opened a much larger can of worms than I'd
intended when fixing a bug reported downstream where cURL would,
when configured with certain DNS backends, fail to resolve .onion
addresses.
https://bugs.gentoo.org/887287
After doing some digging I discovered that the c-ares library was
updated in 2018 to intentionally fail to resolve .onion addresses
in line with RFC 7686, and another reported 'Bug' in cURL for
leaking .onion DNS requests:
https://github.com/c-ares/c-ares/issues/196
https://github.com/curl/curl/issues/543
I took the obviously sane and uncontroversial approach of making
sure that cURL would always behave the same way regardless of the
DNS backend in use, and that it would output a useful error message
when it failed to resolve a .onion address.
Unfortunately, this has made a lot of people very angry and been
~~widely regarded as a bad move~~panned by a small subset of
downstream cURL users:
https://github.com/curl/curl/discussions/11125
https://infosec.exchange/@harrysintonen/110977751446379372
https://gitlab.torproject.org/tpo/core/torspec/-/issues/202
I accept that, in particular, transproxy users are being inconvenienced,
but I also don't want to go back to 'cURL leaks .onion DNS requests
_sometimes_'. As a career sysadmin and downstream bug triager: this
is the stuff that keeps me up late at night. Quite literally, far too
often.
I have found, however that the downstreams that I expected to be 
inconvenienced

most (Whonix and Tails) simply use socks:
https://github.com/Kicksecure/sdwdate/commit/5724d83b258a469b7a9a7bbc65153987f986369b
https://github.com/Kicksecure/tb-updater/commit/d040c12085a527f4d39cb1751f2e514642c7ebaa
https://github.com/Kicksecure/usability-misc/blob/8f722bbbc7b7f2f3a35619a5a1cdeee337a68a78/usr/bin/dist-installer-cli#L2459C22-L2459C22
https://gitlab.tails.boum.org/tails/tails/-/issues/19488
https://gitlab.tails.boum.org/tails/tails/-/merge_requests/1123
I've asked in the Tor Specifications issue (inspired by Silvio's
suggestions), and in the cURL issue, but I seem to be getting nowhere
and the impacted users are clamouring for a quick band-aid solution,
which I feel will work out worse for everyone in the long run:

How can client applications (safely):

1.discover that they're in a Tor-enabled environment
2.resolve onion services only via Tor in that circumstance
3.not leak .onion resolution attempts at all

Right now, not making these requests in the first place is the
safest (and correct) thing to do, however inconvenient it may be.
Rather than immediately trying to come up with a band-aid approach
to this problem, a sane mechanism needs to be implemented to:

1.prevent each application from coming up with their own solution
2.prevent inconsistency in .onion resolution (i.e. no "oh it only
leaks if DO_ONION_RESOLUTION is set")
3.provide a standardised mechanism for applications that want to be Tor
aware to discover that they're in a Tor-enabled environment.

I'm not particularly attached to that last point, but it's worth discussing.
On a related note:
-is the use of a transparent proxy recommended?
-is there a sane alternative that involves as minimal configuration
as possible for these users?
I'm not sure what the best way forward is here, but I'm hoping that
actual Tor developers might have a useful opinion on the matter, or
at least be able to point me in the right direction.
Thanks for your time,
Cheers,
Matt
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev