Re: [tor-bugs] #9316 [Circumvention/BridgeDB]: BridgeDB should export statistics

[Tor Bug Tracker & Wiki]

#9316: BridgeDB should export statistics
-+-
 Reporter:  asn  |  Owner:  phw
 Type:  task | Status:  closed
 Priority:  Medium   |  Milestone:
Component:  Circumvention/BridgeDB   |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics, bridgedb, prometheus,   |  implemented
  anti-censorship-roadmap-september, s30-o21a1   |  Actual Points:  2.3
Parent ID:  #31274   | Points:  3
 Reviewer:  cohosh   |Sponsor:
 |  Sponsor30-must
-+-
Changes (by gaba):

 * keywords:
 metrics, bridgedb, prometheus, anti-censorship-roadmap-september,
 s30-a1
 =>
 metrics, bridgedb, prometheus, anti-censorship-roadmap-september,
 s30-o21a1


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #9316 [Circumvention/BridgeDB]: BridgeDB should export statistics

[Tor Bug Tracker & Wiki]

#9316: BridgeDB should export statistics
-+-
 Reporter:  asn  |  Owner:  phw
 Type:  task | Status:  closed
 Priority:  Medium   |  Milestone:
Component:  Circumvention/BridgeDB   |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics, bridgedb, prometheus,   |  implemented
  anti-censorship-roadmap-september, s30-a1  |  Actual Points:  2.3
Parent ID:  #31274   | Points:  3
 Reviewer:  cohosh   |Sponsor:
 |  Sponsor30-must
-+-
Changes (by gaba):

 * keywords:  metrics, bridgedb, prometheus, anti-censorship-roadmap-
 september =>
 metrics, bridgedb, prometheus, anti-censorship-roadmap-september,
 s30-a1


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #9316 [Circumvention/BridgeDB]: BridgeDB should export statistics

[Tor Bug Tracker & Wiki]

#9316: BridgeDB should export statistics
-+-
 Reporter:  asn  |  Owner:  phw
 Type:  task | Status:  closed
 Priority:  Medium   |  Milestone:
Component:  Circumvention/BridgeDB   |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics, bridgedb, prometheus,   |  implemented
  anti-censorship-roadmap-september  |  Actual Points:  2.3
Parent ID:  #31274   | Points:  3
 Reviewer:  cohosh   |Sponsor:
 |  Sponsor30-must
-+-
Changes (by phw):

 * actualpoints:   => 2.3


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #9316 [Circumvention/BridgeDB]: BridgeDB should export statistics

[Tor Bug Tracker & Wiki]

#9316: BridgeDB should export statistics
-+-
 Reporter:  asn  |  Owner:  phw
 Type:  task | Status:  closed
 Priority:  Medium   |  Milestone:
Component:  Circumvention/BridgeDB   |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics, bridgedb, prometheus,   |  implemented
  anti-censorship-roadmap-september  |  Actual Points:
Parent ID:  #31274   | Points:  3
 Reviewer:  cohosh   |Sponsor:
 |  Sponsor30-must
-+-
Changes (by phw):

 * status:  merge_ready => closed
 * resolution:   => implemented


Comment:

 Replying to [comment:35 cohosh]:
 > Thanks, this looks good to me!
 [[br]]
 Merged and deployed, thanks!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #9316 [Circumvention/BridgeDB]: BridgeDB should export statistics

[Tor Bug Tracker & Wiki]

#9316: BridgeDB should export statistics
-+-
 Reporter:  asn  |  Owner:  phw
 Type:  task | Status:
 |  merge_ready
 Priority:  Medium   |  Milestone:
Component:  Circumvention/BridgeDB   |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics, bridgedb, prometheus,   |  Actual Points:
  anti-censorship-roadmap-september  |
Parent ID:  #31274   | Points:  3
 Reviewer:  cohosh   |Sponsor:
 |  Sponsor30-must
-+-

Comment (by phw):

 Replying to [comment:34 phw]:
 > We probably also want BridgeDB-centric metrics such as "number of
 bridges per ring" and "number of requests that were answered with 0
 bridges". I suggest that we discuss these in a separate ticket, ok?
 [[br]]
 I filed #31422 for this enhancement.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #9316 [Circumvention/BridgeDB]: BridgeDB should export statistics

[Tor Bug Tracker & Wiki]

#9316: BridgeDB should export statistics
-+-
 Reporter:  asn  |  Owner:  phw
 Type:  task | Status:
 |  merge_ready
 Priority:  Medium   |  Milestone:
Component:  Circumvention/BridgeDB   |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics, bridgedb, prometheus,   |  Actual Points:
  anti-censorship-roadmap-september  |
Parent ID:  #31274   | Points:  3
 Reviewer:  cohosh   |Sponsor:
 |  Sponsor30-must
-+-
Changes (by cohosh):

 * status:  needs_review => merge_ready


Comment:

 Thanks, this looks good to me!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #9316 [Circumvention/BridgeDB]: BridgeDB should export statistics

[Tor Bug Tracker & Wiki]

#9316: BridgeDB should export statistics
-+-
 Reporter:  asn  |  Owner:  phw
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/BridgeDB   |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics, bridgedb, prometheus,   |  Actual Points:
  anti-censorship-roadmap-september  |
Parent ID:  #31274   | Points:  3
 Reviewer:  cohosh   |Sponsor:
 |  Sponsor30-must
-+-
Changes (by phw):

 * status:  needs_information => needs_review


Comment:

 Replying to [comment:32 cohosh]:
 > This looks good to me, I added some comments to this commit:
 
https://github.com/NullHypothesis/bridgedb/commit/d6fa8e18dd764cbc612f834338cb71c4ab322a9b
 [[br]]
 Thanks. My latest branch should include all your feedback:
 https://github.com/NullHypothesis/bridgedb/commits/fix/9316
 [[br]]
 > There were some changes that I couldn't track how they relate to the
 metrics feature, perhaps they snuck in from some other changes being made
 to bridgedb? Otherwise it looks really good!
 [[br]]
 Right, I split the branch into three commits, to make it less confusing.
 
[https://github.com/NullHypothesis/bridgedb/commit/0d5ed52e5906260e142ef9cfa12752810fd1ffa2
 0d5ed52e] fixes the broken download of Tor exit relays,
 
[https://github.com/NullHypothesis/bridgedb/commit/85a69d1be3f14a8ba0c5ba8e4840e40214217941
 85a69d1b] updates a comment, and
 
[https://github.com/NullHypothesis/bridgedb/commit/91d12652065f7912a6a8fe12bee27fc9dcaeb842
 91d12652] implements the metrics feature.
 [[br]]
 > There might be an unchecked failure case
 
[https://github.com/NullHypothesis/bridgedb/blob/d1a70dee2ca476b0819a1cb24212215e31c755d6/bridgedb/distributors/moat/server.py#L741
 here] with the moat reporting for when we don't have any bridge lines to
 return. It's not really a failure of the system though so much as a lack
 of bridges so I'm not sure how we'd want to count that.
 [[br]]
 The current metrics implementation is user-centric, meaning that requests
 are classified as "success" if the user did everything right and "fail" if
 the user did something wrong (e.g., used an email account other than Gmail
 or Riseup, or failed to solve the CAPTCHA).

 We probably also want BridgeDB-centric metrics such as "number of bridges
 per ring" and "number of requests that were answered with 0 bridges". I
 suggest that we discuss these in a separate ticket, ok?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #9316 [Circumvention/BridgeDB]: BridgeDB should export statistics

[Tor Bug Tracker & Wiki]

#9316: BridgeDB should export statistics
-+-
 Reporter:  asn  |  Owner:  phw
 Type:  task | Status:
 |  needs_information
 Priority:  Medium   |  Milestone:
Component:  Circumvention/BridgeDB   |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics, bridgedb, prometheus,   |  Actual Points:
  anti-censorship-roadmap-september  |
Parent ID:  #31274   | Points:  3
 Reviewer:  cohosh   |Sponsor:
 |  Sponsor30-must
-+-
Changes (by cohosh):

 * status:  needs_review => needs_information


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #9316 [Circumvention/BridgeDB]: BridgeDB should export statistics

[Tor Bug Tracker & Wiki]

#9316: BridgeDB should export statistics
-+-
 Reporter:  asn  |  Owner:  phw
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/BridgeDB   |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics, bridgedb, prometheus,   |  Actual Points:
  anti-censorship-roadmap-september  |
Parent ID:  #31274   | Points:  3
 Reviewer:  cohosh   |Sponsor:
 |  Sponsor30-must
-+-

Comment (by cohosh):

 This looks good to me, I added some comments to this commit:
 
https://github.com/NullHypothesis/bridgedb/commit/d6fa8e18dd764cbc612f834338cb71c4ab322a9b

 There were some changes that I couldn't track how they relate to the
 metrics feature, perhaps they snuck in from some other changes being made
 to bridgedb? Otherwise it looks really good!

 There might be an unchecked failure case
 
[https://github.com/NullHypothesis/bridgedb/blob/d1a70dee2ca476b0819a1cb24212215e31c755d6/bridgedb/distributors/moat/server.py#L741
 here] with the moat reporting for when we don't have any bridge lines to
 return. It's not really a failure of the system though so much as a lack
 of bridges so I'm not sure how we'd want to count that.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #9316 [Circumvention/BridgeDB]: BridgeDB should export statistics

[Tor Bug Tracker & Wiki]

#9316: BridgeDB should export statistics
-+-
 Reporter:  asn  |  Owner:  phw
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/BridgeDB   |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics, bridgedb, prometheus,   |  Actual Points:
  anti-censorship-roadmap-september  |
Parent ID:  #31274   | Points:  3
 Reviewer:  cohosh   |Sponsor:
 |  Sponsor30-must
-+-
Changes (by phw):

 * reviewer:  sysrqb => cohosh


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #9316 [Circumvention/BridgeDB]: BridgeDB should export statistics

[Tor Bug Tracker & Wiki]

#9316: BridgeDB should export statistics
-+-
 Reporter:  asn  |  Owner:  phw
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/BridgeDB   |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics, bridgedb, prometheus,   |  Actual Points:
  anti-censorship-roadmap-september  |
Parent ID:  #31274   | Points:  3
 Reviewer:  sysrqb   |Sponsor:
 |  Sponsor30-must
-+-
Changes (by phw):

 * parent:  #31268 => #31274


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #9316 [Circumvention/BridgeDB]: BridgeDB should export statistics

[Tor Bug Tracker & Wiki]

#9316: BridgeDB should export statistics
-+-
 Reporter:  asn  |  Owner:  phw
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/BridgeDB   |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics, bridgedb, prometheus,   |  Actual Points:
  anti-censorship-roadmap-september  |
Parent ID:  #31268   | Points:  3
 Reviewer:  sysrqb   |Sponsor:
 |  Sponsor30-must
-+-
Changes (by phw):

 * parent:  #19332 => #31268


Old description:

> BridgeDB should export statistics on its usage. Stuff like distributor
> usage, number of clients served, etc.

New description:

 BridgeDB should export statistics on its usage. Stuff like distributor
 usage, number of clients served, etc.

 Ticket #19332 tracks our follow-up task: the inclusion of BridgeDB's
 metrics into CollecTor.

--

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #9316 [Circumvention/BridgeDB]: BridgeDB should export statistics

[Tor Bug Tracker & Wiki]

#9316: BridgeDB should export statistics
-+-
 Reporter:  asn  |  Owner:  phw
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/BridgeDB   |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics, bridgedb, prometheus,   |  Actual Points:
  anti-censorship-roadmap-september  |
Parent ID:  #19332   | Points:  3
 Reviewer:  sysrqb   |Sponsor:
 |  Sponsor30-must
-+-

Comment (by phw):

 I sent an email with a summary of the first batch of metrics to
 [https://lists.torproject.org/pipermail/tor-dev/2019-July/013953.html tor-
 dev@]. [https://lists.torproject.org/pipermail/tor-
 dev/2019-July/013954.html Rick Huebner suggested] to add metrics on
 BridgeDB's hash ring allocations. This would be a useful albeit not
 critical addition to our existing metrics.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #9316 [Circumvention/BridgeDB]: BridgeDB should export statistics

[Tor Bug Tracker & Wiki]

#9316: BridgeDB should export statistics
-+-
 Reporter:  asn  |  Owner:  phw
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/BridgeDB   |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics, bridgedb, prometheus,   |  Actual Points:
  anti-censorship-roadmap-september  |
Parent ID:  #19332   | Points:  3
 Reviewer:  sysrqb   |Sponsor:
 |  Sponsor30-must
-+-
Changes (by gaba):

 * keywords:  metrics, bridgedb, prometheus, ex-sponsor-19, anti-censorship-
 roadmap => metrics, bridgedb, prometheus, anti-censorship-roadmap-
 september


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #9316 [Circumvention/BridgeDB]: BridgeDB should export statistics

[Tor Bug Tracker & Wiki]

#9316: BridgeDB should export statistics
-+-
 Reporter:  asn  |  Owner:  phw
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Circumvention/BridgeDB   |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics, bridgedb, prometheus, ex-   |  Actual Points:
  sponsor-19, anti-censorship-roadmap|
Parent ID:  #19332   | Points:  3
 Reviewer:  sysrqb   |Sponsor:
 |  Sponsor30-must
-+-
Changes (by phw):

 * status:  assigned => needs_review
 * reviewer:   => sysrqb


Comment:

 I pushed a patch to my
 
[https://github.com/NullHypothesis/bridgedb/commit/50d991585adc128d1e557ae32001211066520dac
 fix/9316 branch]. The commit message provides an overview of what the
 patch seeks to accomplish.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #9316 [Circumvention/BridgeDB]: BridgeDB should export statistics

[Tor Bug Tracker & Wiki]

#9316: BridgeDB should export statistics
-+-
 Reporter:  asn  |  Owner:  phw
 Type:  task | Status:
 |  assigned
 Priority:  Medium   |  Milestone:
Component:  Circumvention/BridgeDB   |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics, bridgedb, prometheus, ex-   |  Actual Points:
  sponsor-19, anti-censorship-roadmap|
Parent ID:  #19332   | Points:  3
 Reviewer:   |Sponsor:
 |  Sponsor30-must
-+-

Comment (by phw):

 We just heard back from Tor's Research Safety Board. You can find the
 response below. The reviewer writes that our proposal wouldn't be an issue
 in a one-off setting but could be problematic in the long run. I think a
 reasonable way forward would be to implement the proposal, run it in a
 one-off setting for, say, a week, and then evaluate if we should change
 data collection. In the long run, we should also transition to PrivCount
 as the reviewer mentions.

 {{{
 Tor Research Safety Board Paper #20 Reviews and Comments
 ===
 Paper #20 Collecting BridgeDB usage statistics


 Review #20A
 ===
 * Updated: 11 Jun 2019 6:02:53pm EDT

 Overall merit
 -
 4. Accept

 Reviewer expertise
 --
 3. Knowledgeable

 Paper summary
 -
 The document proposing collecting a new set of usage statistics through
 data
 available from the operation of BridgeDB. The statistics would be useful
 for
 better prioritizing development tasks, to improve reaction time to bridge
 enumeration attacks and blockages, to reduce failure rates, and to help
 promote
 censorship circumvention research.

 Comments for author
 ---
 If this was a short term study, I would say go for it, no questions asked.
 The
 benefits are clear and I agree that they outweigh the risks.

 However, I think it was implied (although not explicitly stated) that the
 new
 statistics would be regularly collected and published on an ongoing basis.
 I
 think there are more risks associated with such an ongoing collection as
 opposed
 to a one-off or short term study, so we should carefully consider the
 trade-offs
 between cost/effort of safer collection methods with the privacy benefits
 of
 such methods.

 The most concerning statistics to me are the per-country statistics and
 the
 per-service (gmail, yahoo, etc.) statistics. I think it is clear from
 Sections 3
 and 4 that you understand the risks associated with collecting these
 statistics:
 a single user from an unpopular country could be identified because the
 1-10
 bucket suddenly changed from a 0 count to a 1 count. This issue might also
 exist
 if unpopular email service providers are selected. This issue is already
 present
 in Tor's per-country user statistics, and I believe there is a plan to
 transition away from these statistics because of the safety concerns. The
 bucketing proposal (round to the nearest 10) does provide some
 uncertainty, but
 it's hard to reason about what protection it is providing.

 In an ideal world, we would collect these statistics with a privacy-
 preserving
 statistics collection tool. In fact, I think most if not all of these
 could be
 collected with PrivCount (assuming it was extended to support the new
 event
 types).

 One useful thing about PrivCount is secure aggregation, meaning that if
 you have
 multiple data collectors, you can securely count a total across all of
 them
 without leaking individual inputs. In this case, it seems like there is
 only one
 BridgeDB data source, so we woud not benefit from PrivCount's secure
 aggregation.

 The other useful thing that PrivCount provides is differential privacy.
 This is
 where you could get most of the benefit. Rather than rounding to 10 and
 not
 knowing how much privacy that provides, you instead start by defining how
 much
 privacy each statistic should achieve based on your operational
 environment
 (these are called action bounds), and then PrivCount will add noise to the
 statistics in a way that will guarantee differential privacy under those
 constraints. If these constraints add too much noise for the resulting
 statistics to be useful, then you have to consider if the measurement is
 too
 privacy-invasive for the given actions you are trying to protect and
 therefore
 you possibly shouldn't collect them.

 Tor has PrivCount on the roadmap (I believe), so one option could be to
 implement the 

Re: [tor-bugs] #9316 [Circumvention/BridgeDB]: BridgeDB should export statistics

[Tor Bug Tracker & Wiki]

#9316: BridgeDB should export statistics
-+-
 Reporter:  asn  |  Owner:  phw
 Type:  task | Status:
 |  assigned
 Priority:  Medium   |  Milestone:
Component:  Circumvention/BridgeDB   |Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics, bridgedb, prometheus, ex-   |  Actual Points:
  sponsor-19, anti-censorship-roadmap|
Parent ID:  #19332   | Points:  3
 Reviewer:   |Sponsor:
 |  Sponsor30-must
-+-
Changes (by gaba):

 * keywords:  metrics, bridgedb, prometheus, ex-sponsor-19 => metrics,
 bridgedb, prometheus, ex-sponsor-19, anti-censorship-roadmap


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs