Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-12-18 Thread David Fifield
On Fri, Dec 16, 2022 at 04:27:06AM +, Gary C. New via tor-relays wrote: > On Tuesday, December 13, 2022, 07:35:23 PM MST, David Fifield > wrote: > > On Tue, Dec 13, 2022 at 07:29:45PM +, Gary C. New via tor-relays wrote: > >> On Tuesday, December 13, 2022, 10:11:41 AM PST, David Fifield

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-12-16 Thread Gary C. New via tor-relays
On Tuesday, December 13, 2022, 07:35:23 PM MST, David Fifield wrote: On Tue, Dec 13, 2022 at 07:29:45PM +, Gary C. New via tor-relays wrote: >> On Tuesday, December 13, 2022, 10:11:41 AM PST, David Fifield >> wrote: >> >> Am I correct in assuming extor-static-cookie is only useful within

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-12-13 Thread David Fifield
On Tue, Dec 13, 2022 at 07:29:45PM +, Gary C. New via tor-relays wrote: > On Tuesday, December 13, 2022, 10:11:41 AM PST, David Fifield > wrote: > > > The Snowflake proxy is not a pluggable transport. You just > run it as a > > normal command-line program. There is no torrc involved, and the

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-12-13 Thread Gary C. New via tor-relays
On Tuesday, December 13, 2022, 10:11:41 AM PST, David Fifield wrote: > The Snowflake proxy is not a pluggable transport. You just > run it as a > normal command-line program. There is no torrc involved, and the proxy > does not interact with a tor process at all. Thank you for the

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-12-13 Thread David Fifield
On Mon, Dec 12, 2022 at 08:19:53PM +, Gary C. New via tor-relays wrote: > I am having some issues or misunderstandings with implementing Snowflake Proxy > within Tor. I assumed that implementing Snowflake Proxy within Tor would be > similar to OBFS4Bridge in that Tor would initialize Snowflake

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-12-13 Thread Gary C. New via tor-relays
On Monday, December 12, 2022, 08:31:43 AM MST, David Fifield wrote: On Sun, Dec 11, 2022 at 04:25:06AM +, Gary C. New via tor-relays wrote: >> I was successfully able to get Snowflake cross-compiled and installed for >> OpenWRT and Entware as a package. > Thanks, nice work. >> # opkg

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-12-12 Thread David Fifield
On Sun, Dec 11, 2022 at 04:25:06AM +, Gary C. New via tor-relays wrote: > I was successfully able to get Snowflake cross-compiled and installed for > OpenWRT and Entware as a package. Thanks, nice work. > # opkg files snowflake > Package snowflake (2.4.1-1) is installed on root and has the

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-12-11 Thread Gary C. New via tor-relays
David, I was successfully able to get Snowflake cross-compiled and installed for OpenWRT and Entware as a package. # opkg install ./snowflake_2.4.1-1_armv7-2.6.ipk Installing snowflake (2.4.1-1) to root... Configuring snowflake. # opkg info snowflake Package: snowflake Version: 2.4.1-1

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-12-11 Thread Gary C. New via tor-relays
On Saturday, December 10, 2022, 8:01:15 AM MST, David Fifield wrote: On Sat, Dec 10, 2022 at 05:19:43AM +, Gary C. New via tor-relays wrote: >> I'm in the process of trying to cross-compile snowflake for OpenWRT and >> Entware. Are there any other dependencies to compile snowflake other

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-12-10 Thread David Fifield
On Sat, Dec 10, 2022 at 05:19:43AM +, Gary C. New via tor-relays wrote: > I'm in the process of trying to cross-compile snowflake for OpenWRT and > Entware. Are there any other dependencies to compile snowflake other than Go? The README should list dependencies. Setting GOOS and GOARCH should

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-12-10 Thread Gary C. New via tor-relays
David, I'm in the process of trying to cross-compile snowflake for OpenWRT and Entware. Are there any other dependencies to compile snowflake other than Go? Do you know if it's possible to configure multiple pluggable transports with different listeners within a single torrc? Thanks, again.

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-12-09 Thread David Fifield
On Fri, Dec 09, 2022 at 08:43:26AM +, Gary C. New wrote: > In my implementation of the loadbalanced OBFS4 configuration, it appears that > BridgeDB still tests the ORPort for availability and without it marks the > OBFS4 bridge as being down. I see. Then yes, I suppose it is still necessary

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-12-09 Thread David Fifield
On Fri, Dec 09, 2022 at 10:16:47AM +0100, Toralf Förster wrote: > On 12/9/22 07:02, David Fifield wrote: > > But now there is rdsys and bridgestrap, which may have the ability to > > test the obfs4 port rather than the ORPort. I cannot say whether that > > removes the requirement to expose the

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-12-09 Thread Gary C. New via tor-relays
David, In my implementation of the loadbalanced OBFS4 configuration, it appears that BridgeDB still tests the ORPort for availability and without it marks the OBFS4 bridge as being down. I gather that default bridges don't require a DistributionMethod as your loadbalanced Snowflake

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-12-09 Thread Toralf Förster
On 12/9/22 07:02, David Fifield wrote: But now there is rdsys and bridgestrap, which may have the ability to test the obfs4 port rather than the ORPort. I cannot say whether that removes the requirement to expose the ORPort. Would be a step toward to make scanning for bridges harder IMO, if

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-12-08 Thread Gary C. New via tor-relays
David, I finally have time to migrate my loadbalanced Tor relay to a loadbalanced Tor obfs4proxy configuration. In the process, I've been reviewing this thread and was hoping you could help with one clarification regarding your loadbalanced Tor snowflake configuration? I noticed that you are

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-12-08 Thread David Fifield
On Fri, Dec 09, 2022 at 01:09:05AM +, Gary C. New wrote: > Is it truly necessary to expose the ORPort to the World in a pluggable > transport configuration? I don't know if it is necessary for ordinary bridges to expose the ORPort. For a long time, it was necessary, because BridgeDB used the

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-03-07 Thread Georg Koppen
Gary C. New via tor-relays: Georg, Are there any "Issues" submitted for a similar change to Concensus Weight and Relay Probability to Tor Metrics on Onionoo? It appears these values are still only being reported for a Single Tor Node.  Hrm, good question. I don't think so and I am not

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-03-05 Thread Gary C. New via tor-relays
David, > When I made my own combined graphs, I relied on different instances having different nicknames. I don't know an easy way to distinguish the descriptors of different instances otherwise. Please let me know what the Tor Metrics Team decides, if/when they reimplement the change. > You

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-03-05 Thread Gary C. New via tor-relays
Georg, >> Are there any "Issues" submitted for a similar change to Concensus Weight >> and Relay Probability to Tor Metrics on Onionoo? It appears these values are >> still only being reported for a Single Tor Node. > Hrm, good question. I don't think so and I am not sure yet, whether we 

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-03-05 Thread Gary C. New via tor-relays
David, I see that the metrics change has been reverted. If/When the metrics change is implemented, will loadbalanced Tor Relay Nodes need to be uniquely named or will they all be able to use the same nickname? I'm glad to hear your loadbalanced Snowflake Relay continues to work well. Thanks,

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-03-04 Thread David Fifield
On Fri, Mar 04, 2022 at 09:40:01PM +, Gary C. New wrote: > I see that the metrics change has been reverted. > > If/When the metrics change is implemented, will loadbalanced Tor Relay Nodes > need to be uniquely named or will they all be able to use the same nickname? When I made my own

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-03-03 Thread Georg Koppen
Gary C. New via tor-relays: Georg, Yes! That is precisely it! Please know that the change appears to be working with my loadbalanced Tor Relay deployment as well. Are there any "Issues" submitted for a similar change to Concensus Weight and Relay Probability to Tor Metrics on Onionoo? It

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-03-03 Thread Gary C. New via tor-relays
Georg, Yes! That is precisely it! Please know that the change appears to be working with my loadbalanced Tor Relay deployment as well. Are there any "Issues" submitted for a similar change to Concensus Weight and Relay Probability to Tor Metrics on Onionoo? It appears these values are still

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-03-03 Thread David Fifield
On Thu, Mar 03, 2022 at 08:13:34PM +, Gary C. New wrote: > Has Tor Metrics implemented your RFC related to Written Bytes per Second and > Read Bytes per Second on Onionoo? > > As of the 27th of February, I've noticed a change in reporting that accurately > reflects the aggregate of my Tor

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-03-03 Thread Georg Koppen
Gary C. New via tor-relays: David, Has Tor Metrics implemented your RFC related to Written Bytes per Second and Read Bytes per Second on Onionoo? That's probably https://gitlab.torproject.org/tpo/network-health/metrics/onionoo/-/issues/40022 , no? Georg As of the 27th of February, I've

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-03-03 Thread Gary C. New via tor-relays
David, Has Tor Metrics implemented your RFC related to Written Bytes per Second and Read Bytes per Second on Onionoo? As of the 27th of February, I've noticed a change in reporting that accurately reflects the aggregate of my Tor Relay Nodes opposed to the previously reported Single Tor Node.

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-02-08 Thread Gary C. New via tor-relays
David, Excellent Documentation and References! I hope the proposed RFC's (auth, key, and metrics) for loadbalanced Tor topologies are seriously considered and implemented by Tor Core and Tor Metrics. Great Work! Respectfully, Gary— This Message Originated by the Sun. iBigBlue 63W Solar Array

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-02-08 Thread David Fifield
The load-balanced Snowflake bridge is running in production since 2022-01-31. Thanks Roger, Gary, Roman for your input. Hopefully reproducible installation instructions:

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-01-31 Thread Gary C. New via tor-relays
On Sunday, January 30, 2022, 2:26:08 AM PST, Roman Mamedov wrote: On Fri, 28 Jan 2022 19:58:49 -0700 David Fifield wrote: >> But a slight variation does work: make secret_onion_key.old and >> secret_onion_key_ntor.old *directories*, so that tor_rename cannot rename a >> file over them. It

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-01-30 Thread Gary C. New via tor-relays
On Saturday, January 29, 2022, 9:46:59 PM PST, David Fifield wrote: >> > I'm not using nyx. I'm just looking at the bandwidth on the network >> > interface. >> If you have time, would you mind installing nyx to validate observed >> similarities/differences between our loadbalanced

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-01-30 Thread Roman Mamedov
On Fri, 28 Jan 2022 19:58:49 -0700 David Fifield wrote: > > On the matter of onion key rotation, I had the idea of making the onion key > > files read-only. Roger did some source code investigation and said that it > > might work to prevent onion key rotation, with some minor side effects. I

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-01-29 Thread David Fifield
On Sat, Jan 29, 2022 at 02:54:40AM +, Gary C. New via tor-relays wrote: > > > From your documentation, it sounds like you're running everything on the > > > same machine? When expanding to additional machines, similar to the file > > > limit issue, you'll have to expand the usable ports as

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-01-29 Thread Gary C. New via tor-relays
David, > Making secret_onion_key and secret_onion_key_ntor read-only does not quite >work, because tor first renames them to secret_onion_key.old and >secret_onion_key_ntor.old before writing new files. (Making the *.old files >read-only does not work either, because the `tor_rename` function

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-01-29 Thread Gary C. New via tor-relays
David, On Thursday, January 27, 2022, 1:03:25 AM MST, David Fifield wrote: >> It's nice to see that the Snowflake daemon offers a native configuration >> option for LimitNOFile. I ran into a similar issue with my initial >> loadbalanced Tor Relay Nodes that was solved at the O/S level

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-01-28 Thread David Fifield
> On the matter of onion key rotation, I had the idea of making the onion key > files read-only. Roger did some source code investigation and said that it > might work to prevent onion key rotation, with some minor side effects. I > plan to give the idea a try on a different bridge. The

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-01-27 Thread David Fifield
> With regard to loadbalanced Snowflake sessions, I'm curious to know what > connections (i.e., inbound, outbound, directory, control, etc) are being > displayed within nyx? I'm not using nyx. I'm just looking at the bandwidth on the network interface. > Your Heartbeat logs continue to appear

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-01-27 Thread Gary C. New via tor-relays
David, I've been following your progress in the "Add load balancing to bridge (#40095)" issue. > The apparent decrease has to be spurious, since even at the beginning the >bridge was moving more than 10 MB/s in both directions. A couple of hypotheses >about what might be happening: -

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-01-27 Thread Gary C. New via tor-relays
David, > Snowflake sessions are now using the staging bridge, except for those that >started before the change happened and haven't finished yet, and perhaps some >proxies that still have the IP address of the production bridge in their DNS >cache. I am not sure yet what will happen with

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-01-27 Thread David Fifield
On Tue, Jan 25, 2022 at 11:21:10PM +, Gary C. New via tor-relays wrote: > It's nice to see that the Snowflake daemon offers a native configuration > option for LimitNOFile. I ran into a similar issue with my initial > loadbalanced Tor Relay Nodes that was solved at the O/S level using

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-01-26 Thread Gary C. New via tor-relays
David, > I'd like to see more of your HAProxy configuration. Do you not have to use >transparent proxy mode with Snowflake instances as you do with Tor Relay >instances? I hadn't realized HAProxy had a client timeout. Thank you for that >tidbit. And thank you for referencing my comments as

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-01-26 Thread Gary C. New via tor-relays
David, Excellent documentation of your loadbalanced Snowflake endeavors! > The DNS record for the Snowflake bridge was switched to a temporary staging >server, running the load balancing setup, at 2022-01-25 17:41:00. We were >debugging some initial problems until 2022-01-25 18:47:00. You

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-01-25 Thread David Fifield
The DNS record for the Snowflake bridge was switched to a temporary staging server, running the load balancing setup, at 2022-01-25 17:41:00. We were debugging some initial problems until 2022-01-25 18:47:00. You can read about it here:

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-01-17 Thread Gary C. New via tor-relays
On Monday, January 17, 2022, 11:47:11 AM MST, David Fifield wrote: > Gary, I was wondering how you are dealing with the changing onion key issue, >and I suppose it is >[this](https://forum.torproject.net/t/tor-relays-how-to-reduce-tor-cpu-load-on-a-single-bridge/1483/13): >> use Source

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-01-17 Thread David Fifield
On Tue, Jan 04, 2022 at 11:57:36PM -0500, Roger Dingledine wrote: > Hm. It looks promising! But we might still have a Tor-side problem remaining. > I think it boils down to how long the KCP sessions last. > > The details on how exactly these bridge instances will diverge over time: > > The keys

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-01-09 Thread Gary C. New via tor-relays
David, Roger, et al., I just got back from holidays and really enjoyed this thread! I run my Loadbalanced Tor Relay as a Guard/Middle Relay, very similar to David's topology diagram, without the Snoflake-Server proxy. I'm using Nginx (which forks a child process per core) instead of HAProxy.

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-01-09 Thread Gary C. New via tor-relays
Kristian, > I am not really concerned about my IPs being blacklisted as these are normal > relays, not bridges. I suppose if you have the address space and are running your relays in a server environment--it's your prerogative. In my case, I'm running my super relay, from home, with limited

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2022-01-04 Thread Roger Dingledine
[I'm about to go off-line for some days, so I am sending my current suboptimally-organized reply, which I hope is better than waiting another week to respond :)] On Thu, Dec 30, 2021 at 10:42:51PM -0700, David Fifield wrote: > Let's make a distinction between the "frontend" snowflake-server >

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2021-12-31 Thread David Fifield
On Thu, Dec 30, 2021 at 10:42:51PM -0700, David Fifield wrote: > One complication we'll have to work out is that ptadapter doesn't have a > setting for ExtORPort forwarding. ptadapter absorbs any ExtORPort > information and forwards an unadorned connection onward. The idea I had > to to work

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2021-12-30 Thread David Fifield
On Mon, Dec 27, 2021 at 04:00:34PM -0500, Roger Dingledine wrote: > On Mon, Dec 27, 2021 at 12:05:26PM -0700, David Fifield wrote: > > I have the impression that tor cannot use more than one CPU core???is that > > correct? If so, what can be done to permit a bridge to scale beyond > > 1×100% CPU?

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2021-12-29 Thread Roman Mamedov
On Tue, 28 Dec 2021 21:39:27 +0100 (CET) abuse--- via tor-relays wrote: > why would that be needed? Linux has a pretty good thread scheduler imo and > will shuffle loads around as needed. To improve cache locality, as in modern CPUs L1/L2/L3 cache is partitioned into various schemes per core or

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2021-12-29 Thread abuse--- via tor-relays
Hi Gary, thanks! > As an aside... Presently, are you using a single, public address with many > ports or many, public addresses with a single port for your Tor deployments? > Have you ever considered putting all those Tor instances behind a single, > public address:port (fingerprint) to

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2021-12-28 Thread Gary C. New via tor-relays
Hi Kristian, Thanks for the screenshot. Nice Machine! Not everyone is as fortunate as you when it comes to resources for their Tor deployments. While a cpu affinity option isn't high on the priority list, as you point out, many operating systems do a decent job of load management and there are

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2021-12-28 Thread Gary C. New via tor-relays
BTW... I just fact-checked my post-script and the cpu affinity configuration I was thinking of is for Nginx (not Tor). Tor should consider adding a cpu affinity configuration option. What happens if you configure additional Tor instances on the same machine (my Tor instances are on different

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2021-12-27 Thread Gary C. New via tor-relays
David/Roger: Search the tor-relay mail archive for my previous responses on loadbalancing Tor Relays, which I've been successfully doing for the past 6 months with Nginx (it's possible to do with HAProxy as well). I haven't had time to implement it with a Tor Bridge, but I assume it will be

Re: [tor-relays] How to reduce tor CPU load on a single bridge?

2021-12-27 Thread Roger Dingledine
On Mon, Dec 27, 2021 at 12:05:26PM -0700, David Fifield wrote: > I have the impression that tor cannot use more than one CPU core???is that > correct? If so, what can be done to permit a bridge to scale beyond > 1×100% CPU? We can fairly easily scale the Snowflake-specific components > around the