Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+---
 Reporter:  arma |  Owner:  cohosh
 Type:  enhancement  | Status:  closed
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:  implemented
 Keywords:  anti-censorship-roadmap  |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor28-can
-+---
Changes (by cohosh):

 * status:  merge_ready => closed
 * resolution:   => implemented


Comment:

 Merged to master and deployed.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+---
 Reporter:  arma |  Owner:  cohosh
 Type:  enhancement  | Status:  merge_ready
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  anti-censorship-roadmap  |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor28-can
-+---
Changes (by phw):

 * status:  needs_review => merge_ready


Comment:

 The patch looks good to me!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+---
 Reporter:  arma |  Owner:  cohosh
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  anti-censorship-roadmap  |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor28-can
-+---

Comment (by cohosh):

 Replying to [comment:33 cohosh]:
 > Replying to [comment:32 cohosh]:
 > > Replying to [comment:31 irl]:
 > > Thanks irl!
 > > > Was there a reason for removing "snowflake-available-count"? This
 number is going to be the same as the sum of all country codes in
 "snowflake-ips", but it would probably be nice to have this in addition to
 be able to see at a glance.
 > > >
 > > I opted for `snowflake-idle-count` and `snowflake-client-match-count`
 instead, since I think this gives us the information we'd want to use
 `snowflake-available-count` for anyway. I'm not opposed to exporting
 another stat on the available snowflakes, I'll add the code for that back
 in shortly.
 >
 > Here's a commit that adds this metric:
 
https://github.com/cohosh/snowflake/commit/8f2dc3563b1922b285f406a48da85a5a94ee86f9

 Whoops, forgot the tests:
 
https://github.com/cohosh/snowflake/commit/908cf3fc6413930dacdf8a29b5834a5dcf5eab92

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+---
 Reporter:  arma |  Owner:  cohosh
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  anti-censorship-roadmap  |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor28-can
-+---

Comment (by irl):

 Having the total count is also a good way to make sure the GeoIP code
 isn't doing something strange. Looking at UpdateCountryStats I don't think
 there will be any issues here, because you've got good error checking and
 a fallback option. Separating things that depend on GeoIP databases from
 things that don't is sometimes a good idea though.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+---
 Reporter:  arma |  Owner:  cohosh
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  anti-censorship-roadmap  |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor28-can
-+---

Comment (by cohosh):

 Replying to [comment:32 cohosh]:
 >
 > I'm putting this back into needs_revision to add the total available
 snowflake stats. I'll get a code review on that once I complete it, and
 then I'm tempted to close out this ticket and open a new one for the next
 steps in hooking these metrics outputs to whatever the metrics team needs
 to publish these.

 I created #30978 for this

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+---
 Reporter:  arma |  Owner:  cohosh
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  anti-censorship-roadmap  |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor28-can
-+---
Changes (by cohosh):

 * status:  needs_revision => needs_review
 * reviewer:  irl => phw


Comment:

 Replying to [comment:32 cohosh]:
 > Replying to [comment:31 irl]:
 > Thanks irl!
 > > Was there a reason for removing "snowflake-available-count"? This
 number is going to be the same as the sum of all country codes in
 "snowflake-ips", but it would probably be nice to have this in addition to
 be able to see at a glance.
 > >
 > I opted for `snowflake-idle-count` and `snowflake-client-match-count`
 instead, since I think this gives us the information we'd want to use
 `snowflake-available-count` for anyway. I'm not opposed to exporting
 another stat on the available snowflakes, I'll add the code for that back
 in shortly.

 Here's a commit that adds this metric:
 
https://github.com/cohosh/snowflake/commit/8f2dc3563b1922b285f406a48da85a5a94ee86f9

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+
 Reporter:  arma |  Owner:  cohosh
 Type:  enhancement  | Status:  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  anti-censorship-roadmap  |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  irl  |Sponsor:  Sponsor28-can
-+
Changes (by cohosh):

 * status:  needs_review => needs_revision


Comment:

 Replying to [comment:31 irl]:
 Thanks irl!
 > Was there a reason for removing "snowflake-available-count"? This number
 is going to be the same as the sum of all country codes in "snowflake-
 ips", but it would probably be nice to have this in addition to be able to
 see at a glance.
 >
 I opted for `snowflake-idle-count` and `snowflake-client-match-count`
 instead, since I think this gives us the information we'd want to use
 `snowflake-available-count` for anyway. I'm not opposed to exporting
 another stat on the available snowflakes, I'll add the code for that back
 in shortly.

 > I can follow your thought processes and I think that these metrics
 described in comment:19, and also snowflake-available-count from
 comment:14 would be OK to make public. Nothing is jumping out as
 particularly sensitive.
 >
 > Is it possible to run two snowflake proxies from the same IP address?
 There does seem to be an implied limit of 1 proxy per IP address in your
 metrics descriptions. Maybe from a perspective of whether a bridge is
 burned or not, the fact that two processes may be running on the same IP
 doesn't matter because they would both be burned together?
 It is possible to run multiple snowflakes on a single IP. Only the country
 codes stats (and the total available snowflakes which I'll add back in)
 are unique by IP. The `snowflake-idle-count` and `snowflake-client-match-
 count` are not unique by IP and would reflect one IP address running
 multiple snowflakes. I think splitting the metrics in this way makes
 sense. The unique-by-IP ones will tell us information that's useful for
 censorship or blocking by IP and the ones that aren't unique by IP will
 tell us useful information about load on the system.


 I'm putting this back into needs_revision to add the total available
 snowflake stats. I'll get a code review on that once I complete it, and
 then I'm tempted to close out this ticket and open a new one for the next
 steps in hooking these metrics outputs to whatever the metrics team needs
 to publish these.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+---
 Reporter:  arma |  Owner:  cohosh
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  anti-censorship-roadmap  |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  irl  |Sponsor:  Sponsor28-can
-+---

Comment (by irl):

 Was there a reason for removing "snowflake-available-count"? This number
 is going to be the same as the sum of all country codes in "snowflake-
 ips", but it would probably be nice to have this in addition to be able to
 see at a glance.

 I can follow your thought processes and I think that these metrics
 described in comment:19, and also snowflake-available-count from
 comment:14 would be OK to make public. Nothing is jumping out as
 particularly sensitive.

 Is it possible to run two snowflake proxies from the same IP address?
 There does seem to be an implied limit of 1 proxy per IP address in your
 metrics descriptions. Maybe from a perspective of whether a bridge is
 burned or not, the fact that two processes may be running on the same IP
 doesn't matter because they would both be burned together?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+---
 Reporter:  arma |  Owner:  cohosh
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  anti-censorship-roadmap  |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  irl  |Sponsor:  Sponsor28-can
-+---

Comment (by cohosh):

 While we're waiting for review, I propose we deploy these changes to the
 broker and start collecting the metrics data locally so we can take a look
 at it.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+---
 Reporter:  arma |  Owner:  cohosh
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  anti-censorship-roadmap  |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  irl  |Sponsor:  Sponsor28-can
-+---
Changes (by cohosh):

 * reviewer:   => irl


Comment:

 Thanks! I put irl as a reviewer for the metrics feedback

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+---
 Reporter:  arma |  Owner:  cohosh
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  anti-censorship-roadmap  |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:   |Sponsor:  Sponsor28-can
-+---
Changes (by phw):

 * reviewer:  phw =>


Comment:

 Replying to [comment:26 cohosh]:
 > Replying to [comment:25 phw]:
 > > I left a number of minor comments in the GitHub pull request.
 > Thanks! Pushed some commits to address these comments:
 https://github.com/cohosh/snowflake/pull/3
 [[br]]
 Looks good to me! We're still waiting for the metrics team to review the
 statistics, right? I'll just remove myself as reviewer and keep the ticket
 in needs_review.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+
 Reporter:  arma |  Owner:  cohosh
 Type:  enhancement  | Status:  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  anti-censorship-roadmap  |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor28-can
-+

Comment (by cohosh):

 Replying to [comment:25 phw]:
 > I left a number of minor comments in the GitHub pull request.
 Thanks! Pushed some commits to address these comments:
 https://github.com/cohosh/snowflake/pull/3

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+---
 Reporter:  arma |  Owner:  cohosh
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  anti-censorship-roadmap  |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor28-can
-+---
Changes (by cohosh):

 * status:  needs_revision => needs_review


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+
 Reporter:  arma |  Owner:  cohosh
 Type:  enhancement  | Status:  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  anti-censorship-roadmap  |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor28-can
-+
Changes (by phw):

 * status:  needs_review => needs_revision


Comment:

 I left a number of minor comments in the GitHub pull request.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+---
 Reporter:  arma |  Owner:  cohosh
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  anti-censorship-roadmap  |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor28-can
-+---
Changes (by gaba):

 * keywords:   => anti-censorship-roadmap


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+---
 Reporter:  arma |  Owner:  cohosh
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor28-can
-+---
Changes (by phw):

 * reviewer:  irl => phw


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+---
 Reporter:  arma |  Owner:  cohosh
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  irl  |Sponsor:  Sponsor28-can
-+---
Changes (by cohosh):

 * status:  assigned => needs_review


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+---
 Reporter:  arma |  Owner:  cohosh
 Type:  enhancement  | Status:  assigned
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  irl  |Sponsor:  Sponsor28-can
-+---
Changes (by cohosh):

 * owner:  (none) => cohosh
 * status:  needs_review => assigned


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+---
 Reporter:  arma |  Owner:  (none)
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  irl  |Sponsor:  Sponsor28-can
-+---

Comment (by cohosh):

 Here's a PR that implements the metrics specified above:
 https://github.com/cohosh/snowflake/pull/3

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+---
 Reporter:  arma |  Owner:  (none)
 Type:  enhancement  | Status:  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  irl  |Sponsor:  Sponsor28-can
-+---
Changes (by cohosh):

 * status:  needs_revision => needs_review


Comment:

 After looking at #30731, I want to change the proposed collected metrics
 to match the data shown in
 [https://trac.torproject.org/projects/tor/ticket/30693#comment:3 these
 graphs]. I modified the Rscript in those tickets slightly to bin to the
 nearest multiple of 8 and the results are almost identical (due to how
 often proxies are polling). We can get away with even coarser binning but
 I think the multiple of 8 method does enough to disguise individual client
 traffic.

 In addition to this I think it would be interesting to collect geoip data
 on the '''available proxies'''. I don't want to bin snowflake proxies at
 this time because we have too few for this count to be useful at the
 moment, maybe we'd want to add it later but I don't think we need to from
 a client safety perspective. The overall proxy count won't leak client
 information because it only measures whether a proxy has polled at all in
 a 24 hour period and '''not''' whether it was given out (which would leak
 client usage data).

 So I'll propose the following metrics (gathered at a granularity of every
 24 hours):

 {{{
 "snowflake-stats-end" -MM-DD HH:MM:SS (NSEC s) NL
 [At most once.]

 -MM-DD HH:MM:SS defines the end of the included measurement
 interval of length NSEC seconds (86400 seconds by default).

 "snowflake-ips" CC=NUM,CC=NUM,... NL
 [At most once.]

 List of mappings from two-letter country codes to the number of
 unique IP addresses of snowflake proxies that have polled.

 "proxy-idle-count" NUM
 [At most once.]

 A count of the number of times a proxy has polled but received
 no client offer, rounded up to the nearest multiple of 8.

 "client-denied-count" NUM
 [At most once.]

 A count of the number of times a client has requested a proxy
 from the broker but no proxies were available, rounded up to
 the nearest multiple of 8.

 "client-proxy-match-count" NUM
 [At most once.]

 A count of the number of times a client successfully received a
 proxy from the broker, rounded up to the nearest multiple of 8.
 }}}

 I'm going to start implementing these metrics and meanwhile put this in
 needs_review for the metrics team to look at.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+
 Reporter:  arma |  Owner:  (none)
 Type:  enhancement  | Status:  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  irl  |Sponsor:  Sponsor19
-+

Comment (by dcf):

 Replying to [comment:16 cohosh]:
 > Also noting that mapping unique proxies to IP addresses isn't quite how
 snowflake is supposed to work: we'll probably want some kind of persistent
 identifier later as mentioned in the comments above, but I think it will
 work fine for our purposes now.

 Identifying by IP address is interesting by itself, though. If IP address
 is the expected basis of blocking, then it's interesting to consider a
 proxy's "identity" as IP address because that's the information available
 to a censor. A long-term identity independent of IP address is also
 interesting, but less informative for measuring blocking resistance IMO.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+
 Reporter:  arma |  Owner:  (none)
 Type:  enhancement  | Status:  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  irl  |Sponsor:  Sponsor19
-+
Changes (by cohosh):

 * cc: dcf, arlolra (added)


Comment:

 I've cc'd dcf and arlolra to see if they have thoughts on this.

 I just thought of another thing that makes snowflake quite different from
 relays or bridges is the expected amount of time that each proxy will be
 online. It might make more sense to shorten the collection period from 24
 hours in this case. Another thing we can do is have the broker export
 another metric that includes something like the average amount of time a
 unique proxy was available.

 Also noting that mapping unique proxies to IP addresses isn't quite how
 snowflake is supposed to work: we'll probably want some kind of persistent
 identifier later as mentioned in the comments above, but I think it will
 work fine for our purposes now.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+
 Reporter:  arma |  Owner:  (none)
 Type:  enhancement  | Status:  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  irl  |Sponsor:  Sponsor19
-+

Comment (by cohosh):

 Replying to [comment:14 irl]:
 Thanks for this feedback. This was very helpful. What makes snowflake
 statistics a little more complex than bridge or relay stats is that, while
 stats about how many times a bridge was used doesn't closely reflect
 client usage, proxies handle either a single client or a small, fixed
 number of clients as determined by the individual proxy and so there's a
 greater possibility for data leakage there.
 > Number of currently available snowflake proxies is not sensitive. We do
 not make any efforts to hide the numbers of relays or bridges, and so this
 can be an exact count. The question here is not the count resolution but
 the time resolution. (Sorry to answer your question with a question.)
 >
 > If I'm an attacker, can I learn anything about a client if I can observe
 the client's traffic and the exact count of snowflakes. For example, what
 do I learn if a snowflake that a client is using disappears? I'm not sure
 what the snowflake protocol does in this case.
 Possibly, as stated above, it depends on what type of proxy you are and
 how it's set up. I think we're better off doing binning in this case.
 However, as stated below, if we collect at a granularity of every 24 hours
 this shouldn't leak client usage.
 >
 > I'm not sure what you mean with the GeoIP stats. If these are stats
 regarding the locations of proxies, again exact counts would be fine and
 would be in line with what we do for relays and bridges at the moment. If
 this is for clients, we should aim to provide differential privacy. I fear
 that at the moment, we are not seeing enough users that we can safely
 report GeoIP stats (usefully) for clients at all. With relays and bridges,
 we round the counts up to the nearest multiple of 8.
 We're absolutely not collecting geoip stats of clients. These are only of
 snowflake proxies. I originally thought to include geoip stats of proxies
 that are actually handed out but it's safer to do stats for available
 proxies since this shouldn't leak client usage if collected over a period
 of 24 hours.
 >
 > Round trip time of snowflake rendezvous sounds like a really useful
 metric for engineering work, but a dangerous one for safety. This would be
 a good candidate for PrivCount but without such a technique I wouldn't do
 this one. We currently measure performance of relays using active
 measurement, such that we are only analyzing our own traffic. We have
 extended that tool, OnionPerf, to also work for pluggable transports but
 it will do the end-to-end performance not just client->snowflake.
 >
 That's fair, this is really only available for debug purposes. We don't
 need to export it as a metric and I'd argue that this should only be
 logged locally.
 > Can you lay out in detail exactly what metrics you'd want, what
 resolution data you want (both in counts and in time) and what you might
 consider an attacker could learn, assuming they are in a position to
 monitor, or are running, a point in the network?
 >
 It looks like bridge stats default to 24 hours, that seems reasonable for
 snowflake as well.
 > Section 2.1.2 of dir-spec contains some examples of descriptions of
 metrics.

 To summarize, and be more precise about what we want to collect, I've put
 our proposed exported metrics in the Tor Directory Protocol Format:
 {{{
 "snowflake-stats-end" -MM-DD HH:MM:SS (NSEC s) NL
 [At most once.]

 -MM-DD HH:MM:SS defines the end of the included measurement
 interval of length NSEC seconds (86400 seconds by default).

 "snowflake-ips" CC=NUM,CC=NUM,... NL
 [At most once.]

 List of mappings from two-letter country codes to the number of
 unique IP addresses of available snowflake proxies, rounded up
 to the nearest multiple of 8.

 "snowflake-available-count" NUM
 [At most once.]

 A count of the number of unique IP addresses corresponding
 to currently available snowflake proxies, rounded up to
 the nearest multiple of 8.

 "snowflake-usage-count" NUM
 [At most once.]

 A count of the number of snowflake proxies that have been
 handed out by the broker to clients, rounded up to the
 nearest multiple of 8.

 }}}

 So in short, we'd collect 

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+
 Reporter:  arma |  Owner:  (none)
 Type:  enhancement  | Status:  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  irl  |Sponsor:  Sponsor19
-+
Changes (by irl):

 * status:  needs_information => needs_revision


Comment:

 Number of currently available snowflake proxies is not sensitive. We do
 not make any efforts to hide the numbers of relays or bridges, and so this
 can be an exact count. The question here is not the count resolution but
 the time resolution. (Sorry to answer your question with a question.)

 If I'm an attacker, can I learn anything about a client if I can observe
 the client's traffic and the exact count of snowflakes. For example, what
 do I learn if a snowflake that a client is using disappears? I'm not sure
 what the snowflake protocol does in this case.

 I'm not sure what you mean with the GeoIP stats. If these are stats
 regarding the locations of proxies, again exact counts would be fine and
 would be in line with what we do for relays and bridges at the moment. If
 this is for clients, we should aim to provide differential privacy. I fear
 that at the moment, we are not seeing enough users that we can safely
 report GeoIP stats (usefully) for clients at all. With relays and bridges,
 we round the counts up to the nearest multiple of 8.

 Round trip time of snowflake rendezvous sounds like a really useful metric
 for engineering work, but a dangerous one for safety. This would be a good
 candidate for PrivCount but without such a technique I wouldn't do this
 one. We currently measure performance of relays using active measurement,
 such that we are only analyzing our own traffic. We have extended that
 tool, OnionPerf, to also work for pluggable transports but it will do the
 end-to-end performance not just client->snowflake.

 Can you lay out in detail exactly what metrics you'd want, what resolution
 data you want (both in counts and in time) and what you might consider an
 attacker could learn, assuming they are in a position to monitor, or are
 running, a point in the network?

 Section 2.1.2 of dir-spec contains some examples of descriptions of
 metrics.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-+---
 Reporter:  arma |  Owner:  (none)
 Type:  enhancement  | Status:  needs_information
 Priority:  Medium   |  Milestone:
Component:  Circumvention/Snowflake  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:  #29461   | Points:
 Reviewer:  irl  |Sponsor:  Sponsor19
-+---
Changes (by gaba):

 * reviewer:   => irl


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs