Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:  closed
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:  fixed
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-
Changes (by karsten):

 * status:  merge_ready => closed
 * resolution:   => fixed


Comment:

 I just upgraded our CollecTor mirror and will take a look how that goes.
 Closing, because deployment shouldn't be part of this ticket anymore.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  merge_ready
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by karsten):

 Replying to [comment:40 karsten]:
 > All changes are merged, released, and deployed. Next steps are that we
 >  - wait for descriptors to show up on CollecTor and Tor Metrics,

 Done.

 >  - try out syncing from CollecTor host, and

 Done.

 >  - update the CollecTor mirror to sync from the main CollecTor instance.

 irl, do you want to do that, or shall I?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  merge_ready
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by karsten):

 All changes are merged, released, and deployed. Next steps are that we
  - wait for descriptors to show up on CollecTor and Tor Metrics,
  - try out syncing from CollecTor host, and
  - update the CollecTor mirror to sync from the main CollecTor instance.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  merge_ready
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by irl):

 Yes, I had thought that was already merged. I guess not.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  merge_ready
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by karsten):

 Replying to [comment:37 irl]:
 > This looks good to me now.

 Did you also review [https://gitweb.torproject.org/user/karsten/metrics-
 lib.git/commit/?h=task-29461=c8f832160a1eaf5c1c57b2cb81fc849bd20cd69e
 metrics-lib commit c8f8321]?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  merge_ready
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-
Changes (by irl):

 * status:  needs_review => merge_ready


Comment:

 This looks good to me now.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-
Changes (by karsten):

 * status:  needs_revision => needs_review


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by karsten):

 Replying to [comment:33 notirl]:
 > Again the startProcessing() method in the Downloader is huge. Testing
 and review would be easier if this were broken up. This one is easier to
 follow though.

 Thanks for the review! I broke up the method to take out the HTTP GET and
 writing to disk.

 > We should file a ticket to refactor HTTP GET request code into a util
 function.

 Agreed. Filed #31599 for this.

 > I think we already had a ticket to refactor cleaning up directory after
 X days? If we don't then we should file that ticket.

 Yes, that's #20546.

 > The create-tarballs script was updated, but the PROTOCOL was not
 updated, for the monthly tarballs.

 Oops, fixed.

 Please take another look:
 
[https://gitweb.torproject.org/user/karsten/collector.git/commit/?h=task-29461=eeb9f2018c217fb56f1d08f2389319a5a08cf165
 commit eeb9f20 in my task-29461 branch].

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by notirl):

 Status: -> needs_revision

 Again the startProcessing() method in the Downloader is huge. Testing and
 review would be easier if this were broken up. This one is easier to
 follow though.

 We should file a ticket to refactor HTTP GET request code into a util
 function.

 I think we already had a ticket to refactor cleaning up directory after X
 days? If we don't then we should file that ticket.

 The create-tarballs script was updated, but the PROTOCOL was not updated,
 for the monthly tarballs.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-
Changes (by karsten):

 * status:  needs_information => needs_review


Comment:

 irl, please review
 
[https://gitweb.torproject.org/user/karsten/collector.git/commit/?h=task-29461=9a095a63022444c2121fb8b668c092c46f8de7fe
 my task-29461 branch until 9a095a6].

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_information
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by karsten):

 So, it seems that we're not doing #31493. That means that the remaining
 tasks on my list are to update the default URL, do another test round, and
 hand the branch over to irl for review.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_information
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by karsten):

 Commented on #31493.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_information
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by cohosh):

 Okay I've updated #31493 to use the following spec:
 {{{
 We export metrics in the following (version 1.0) format:

 "snowflake-stats-end" -MM-DD HH:MM:SS (NSEC s) NL
 [At start, exactly once.]

 -MM-DD HH:MM:SS defines the end of the included measurement
 interval of length NSEC seconds (86400 seconds by default).

 "@type snowflake-stats 1.0"
 [Exactly once.]

 Defines the version of snowflake stats output being used
 (currently 1.0)

 "snowflake-ips" CC=NUM,CC=NUM,... NL
 [At most once.]

 List of mappings from two-letter country codes to the number of
 unique IP addresses of snowflake proxies that have polled.

 [rest omitted for brevity]
 }}}

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_information
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by cohosh):

 Ah I see, should every entry have the `@type` format? I based the spec off
 of how I gathered relays were exporting stats by reading the source code:
 https://gitweb.torproject.org/tor.git/tree/src/feature/stats/geoip_stats.c#n994

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_information
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by dcf):

 Replying to [comment:26 cohosh]:
 > I can do that, yes. dcf just commented that it might be easier to do the
 version on a separate line (see
 https://trac.torproject.org/projects/tor/ticket/31493#comment:3). Do you
 have an opinion on that? We're looking for ease of use now and in the
 future if/when the version bumps.

 I was picturing a `@type snowflake-stats 1.0` so that the format is the
 same as other CollecTor formats.
 [https://metrics.torproject.org/collector.html#data-formats Data Formats]:
 "Each descriptor provided here contains an `@type` annotation using the
 format `@type $descriptortype $major.$minor`." See for example in
 
[https://github.com/NullHypothesis/zoossh/blob/7ce3729b60488c3418d9ab01bd89126d7dd4b21d/util.go#L48-L50
 zoossh].

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_information
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by cohosh):

 Replying to [comment:25 karsten]:
 > Replying to [comment:24 cohosh]:
 > > Replying to [comment:22 cohosh]:
 > > > Replying to [comment:19 karsten]:
 > > > > cohosh, are you still planning to revise the spec and code to
 include such a `snowflake-stats $version` line?
 > > >
 > > > Yes, I just got back from vacation and I'm going to implement it
 this afternoon.
 > >
 > > Opened #31493 and implemented
 [https://github.com/cohosh/snowflake/compare/bug31493 here]. How does that
 look to you?
 >
 > Well, that would work, though I think that introducing a separate line
 for descriptor type and version might be clearer:
 >
 > {{{
 > snowflake-stats 1.0
 > snowflake-stats-end 2019-08-07 19:52:11 (86400 s)
 > snowflake-ips VN=5,NL=26,AU=30,GT=2,NO=5,EG=3,NI=1,AT=22,[...]
 > snowflake-ips-total 937
 > snowflake-idle-count 660976
 > client-denied-count 0
 > client-snowflake-match-count 864
 > }}}
 >
 > If this looks okay to you and if you make this change, do you think you
 could somehow edit the existing stats file on the snowflake broker, maybe
 using some `sed`/`awk` magic?

 I can do that, yes. dcf just commented that it might be easier to do the
 version on a separate line (see
 https://trac.torproject.org/projects/tor/ticket/31493#comment:3). Do you
 have an opinion on that? We're looking for ease of use now and in the
 future if/when the version bumps.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_information
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by karsten):

 Replying to [comment:24 cohosh]:
 > Replying to [comment:22 cohosh]:
 > > Replying to [comment:19 karsten]:
 > > > cohosh, are you still planning to revise the spec and code to
 include such a `snowflake-stats $version` line?
 > >
 > > Yes, I just got back from vacation and I'm going to implement it this
 afternoon.
 >
 > Opened #31493 and implemented
 [https://github.com/cohosh/snowflake/compare/bug31493 here]. How does that
 look to you?

 Well, that would work, though I think that introducing a separate line for
 descriptor type and version might be clearer:

 {{{
 snowflake-stats 1.0
 snowflake-stats-end 2019-08-07 19:52:11 (86400 s)
 snowflake-ips VN=5,NL=26,AU=30,GT=2,NO=5,EG=3,NI=1,AT=22,[...]
 snowflake-ips-total 937
 snowflake-idle-count 660976
 client-denied-count 0
 client-snowflake-match-count 864
 }}}

 If this looks okay to you and if you make this change, do you think you
 could somehow edit the existing stats file on the snowflake broker, maybe
 using some `sed`/`awk` magic?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_information
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by cohosh):

 Replying to [comment:22 cohosh]:
 > Replying to [comment:19 karsten]:
 > > cohosh, are you still planning to revise the spec and code to include
 such a `snowflake-stats $version` line?
 >
 > Yes, I just got back from vacation and I'm going to implement it this
 afternoon.

 Opened #31493 and implemented
 [https://github.com/cohosh/snowflake/compare/bug31493 here]. How does that
 look to you?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_information
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by cohosh):

 Replying to [comment:21 cohosh]:
 > Replying to [comment:20 karsten]:
 > > Replying to [comment:14 karsten]:
 > > > Replying to [comment:13 irl]:
 > > > > The SnowflakeStatsUrl doesn't look right at all. There's an
 ampersat in it?
 > > >
 > > > You mean the default URL in `collector.properties`? That's known, I
 need to put in the actual URL once that's available (#31376).
 > >
 > > cohosh, what URL did you come up with in that ticket?
 >
 > I just deployed this and you can now fetch it from https://snowflake-
 broker.bamsoftware.com/metrics
 Ooops, actually we should use https://snowflake-
 broker.torproject.net/metrics

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_information
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by cohosh):

 Replying to [comment:19 karsten]:
 > Replying to [comment:16 karsten]:
 > > Replying to [comment:12 cohosh]:
 > > > Replying to [comment:9 karsten]:
 > > > Thanks karsten!
 > > > >  - As indicated before, I'm using `snowflake-stats-end` as
 descriptor type identifier, which means that future data format versions
 will have to keep that line as their first line. A better choice would
 have been to use something like `snowflake-stats $version` or similar. (If
 it's any relief, we're forced to use `[0-9]{10}` as descriptor type
 identifier for bandwidth files, so there would have been plenty of room to
 do worse.)
 > > > Oh this is a good point. I can make this change pretty easily now.
 > >
 > > Okay, in that case please make this change and let me know where to
 find the new spec and a new sample.
 >
 > cohosh, are you still planning to revise the spec and code to include
 such a `snowflake-stats $version` line?

 Yes, I just got back from vacation and I'm going to implement it this
 afternoon.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_information
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by cohosh):

 Replying to [comment:20 karsten]:
 > Replying to [comment:14 karsten]:
 > > Replying to [comment:13 irl]:
 > > > The SnowflakeStatsUrl doesn't look right at all. There's an ampersat
 in it?
 > >
 > > You mean the default URL in `collector.properties`? That's known, I
 need to put in the actual URL once that's available (#31376).
 >
 > cohosh, what URL did you come up with in that ticket?

 I just deployed this and you can now fetch it from https://snowflake-
 broker.bamsoftware.com/metrics

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_information
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by karsten):

 Replying to [comment:14 karsten]:
 > Replying to [comment:13 irl]:
 > > The SnowflakeStatsUrl doesn't look right at all. There's an ampersat
 in it?
 >
 > You mean the default URL in `collector.properties`? That's known, I need
 to put in the actual URL once that's available (#31376).

 cohosh, what URL did you come up with in that ticket?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_information
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-
Changes (by karsten):

 * status:  needs_revision => needs_information


Comment:

 Replying to [comment:16 karsten]:
 > Replying to [comment:12 cohosh]:
 > > Replying to [comment:9 karsten]:
 > > Thanks karsten!
 > > >  - As indicated before, I'm using `snowflake-stats-end` as
 descriptor type identifier, which means that future data format versions
 will have to keep that line as their first line. A better choice would
 have been to use something like `snowflake-stats $version` or similar. (If
 it's any relief, we're forced to use `[0-9]{10}` as descriptor type
 identifier for bandwidth files, so there would have been plenty of room to
 do worse.)
 > > Oh this is a good point. I can make this change pretty easily now.
 >
 > Okay, in that case please make this change and let me know where to find
 the new spec and a new sample.

 cohosh, are you still planning to revise the spec and code to include such
 a `snowflake-stats $version` line?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by karsten):

 Replying to [comment:17 karsten]:
 > Replying to [comment:15 irl]:
 > > I would like to have things becoming immutable after we think they
 will not receive more data, as it makes it easier to manage archives in
 the longer term and do things like monitor for bitrot.
 >
 > Works for me. Will revise this.

 Revised in
 
[https://gitweb.torproject.org/user/karsten/collector.git/commit/?h=task-29461=ac00e57ec95fe1d5092165f129ddde147513e751
 ac00e57].

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by karsten):

 Replying to [comment:15 irl]:
 > I would like to have things becoming immutable after we think they will
 not receive more data, as it makes it easier to manage archives in the
 longer term and do things like monitor for bitrot.

 Works for me. Will revise this.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by karsten):

 Replying to [comment:12 cohosh]:
 > Replying to [comment:9 karsten]:
 > Thanks karsten!
 > >  - As indicated before, I'm using `snowflake-stats-end` as descriptor
 type identifier, which means that future data format versions will have to
 keep that line as their first line. A better choice would have been to use
 something like `snowflake-stats $version` or similar. (If it's any relief,
 we're forced to use `[0-9]{10}` as descriptor type identifier for
 bandwidth files, so there would have been plenty of room to do worse.)
 > Oh this is a good point. I can make this change pretty easily now.

 Okay, in that case please make this change and let me know where to find
 the new spec and a new sample.

 > >  - The current format only supports a single snowflake broker. Maybe
 this is acceptable for the snowflake design. But just in case that you'll
 one day want to add a second broker, you'll have to include some sort of
 broker identifier in the format.
 > This will definitely be the case for quite a while, I think a spec
 change (and version bump of the spec) will be a good way to handle this.
 How difficult will it be to have data with different spec versions on your
 end?

 Not difficult at all. Okay, let's keep this unchanged then.

 > >  - The current format is not signed, which is somewhat related to not
 having a broker identifier in the format.
 > >  - As a consequence of the above, CollecTor needs to make a decision
 whether it wants to archive a newly downloaded snowflake-stats snippet, if
 it already has another snippet with the same timestamp and different
 contents. Possible strategies for this specific case are to a) never
 overwrite, b) always overwrite, c) keep all versions by including a digest
 in the file name, d) maybe something else. I implemented a) for now.
 > I didn't think about signing, that would take while to implement I
 think. I agree that the best decision here is (a) for now.
 > >
 > > I think we can start with what we have, without changing anything of
 the above. Of course, if you want to change something with regard to
 future maintenance effort, now's the time!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by irl):

 I would like to have things becoming immutable after we think they will
 not receive more data, as it makes it easier to manage archives in the
 longer term and do things like monitor for bitrot.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by karsten):

 Replying to [comment:13 irl]:
 > The SnowflakeStatsUrl doesn't look right at all. There's an ampersat in
 it?

 You mean the default URL in `collector.properties`? That's known, I need
 to put in the actual URL once that's available (#31376).

 > > 'snowflakes' contains a single compressed tarball with snowflake
 statistics:
 >
 > It is nice that we can have archives that after a while become read-
 only. Why are we not splitting this by month?

 The reason is that these statistics are tiny and that processing these
 files would be simpler with a single tarball. But changing this is
 trivial, if you think that doing so has more advantages.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_revision
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-
Changes (by irl):

 * status:  needs_review => needs_revision


Comment:

 The SnowflakeStatsUrl doesn't look right at all. There's an ampersat in
 it?

 > 'snowflakes' contains a single compressed tarball with snowflake
 statistics:

 It is nice that we can have archives that after a while become read-only.
 Why are we not splitting this by month?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-

Comment (by cohosh):

 Replying to [comment:9 karsten]:
 Thanks karsten!
 >  - As indicated before, I'm using `snowflake-stats-end` as descriptor
 type identifier, which means that future data format versions will have to
 keep that line as their first line. A better choice would have been to use
 something like `snowflake-stats $version` or similar. (If it's any relief,
 we're forced to use `[0-9]{10}` as descriptor type identifier for
 bandwidth files, so there would have been plenty of room to do worse.)
 Oh this is a good point. I can make this change pretty easily now.
 >  - The current format only supports a single snowflake broker. Maybe
 this is acceptable for the snowflake design. But just in case that you'll
 one day want to add a second broker, you'll have to include some sort of
 broker identifier in the format.
 This will definitely be the case for quite a while, I think a spec change
 (and version bump of the spec) will be a good way to handle this. How
 difficult will it be to have data with different spec versions on your
 end?
 >  - The current format is not signed, which is somewhat related to not
 having a broker identifier in the format.
 >  - As a consequence of the above, CollecTor needs to make a decision
 whether it wants to archive a newly downloaded snowflake-stats snippet, if
 it already has another snippet with the same timestamp and different
 contents. Possible strategies for this specific case are to a) never
 overwrite, b) always overwrite, c) keep all versions by including a digest
 in the file name, d) maybe something else. I implemented a) for now.
 I didn't think about signing, that would take while to implement I think.
 I agree that the best decision here is (a) for now.
 >
 > I think we can start with what we have, without changing anything of the
 above. Of course, if you want to change something with regard to future
 maintenance effort, now's the time!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:  irl  |Sponsor:
 |  Sponsor28
-+-
Changes (by irl):

 * reviewer:   => irl


Comment:

 I have seen this, and plan to do the review tomorrow morning.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:   |Sponsor:
 |  Sponsor28
-+-

Comment (by karsten):

 Note to self: pending on #31376 for including the correct, working URL as
 default configuration option.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:   |Sponsor:
 |  Sponsor28
-+-
Changes (by karsten):

 * status:  new => needs_review


Comment:

 Couple ideas after implementing the CollecTor and metrics-web parts:
  - As indicated before, I'm using `snowflake-stats-end` as descriptor type
 identifier, which means that future data format versions will have to keep
 that line as their first line. A better choice would have been to use
 something like `snowflake-stats $version` or similar. (If it's any relief,
 we're forced to use `[0-9]{10}` as descriptor type identifier for
 bandwidth files, so there would have been plenty of room to do worse.)
  - The current format only supports a single snowflake broker. Maybe this
 is acceptable for the snowflake design. But just in case that you'll one
 day want to add a second broker, you'll have to include some sort of
 broker identifier in the format.
  - The current format is not signed, which is somewhat related to not
 having a broker identifier in the format.
  - As a consequence of the above, CollecTor needs to make a decision
 whether it wants to archive a newly downloaded snowflake-stats snippet, if
 it already has another snippet with the same timestamp and different
 contents. Possible strategies for this specific case are to a) never
 overwrite, b) always overwrite, c) keep all versions by including a digest
 in the file name, d) maybe something else. I implemented a) for now.

 I think we can start with what we have, without changing anything of the
 above. Of course, if you want to change something with regard to future
 maintenance effort, now's the time!

 irl, please review [https://gitweb.torproject.org/user/karsten/metrics-
 lib.git/commit/?h=task-29461=c8f832160a1eaf5c1c57b2cb81fc849bd20cd69e
 metrics-lib commit c8f8321],
 
[https://gitweb.torproject.org/user/karsten/collector.git/commit/?h=task-29461=cd5a27d3495f2cd8fd7c693e20a406f571d36a2f
 CollecTor commit cd5a27d], and [https://gitweb.torproject.org/user/karsten
 /metrics-
 web.git/commit/?h=task-29461=5958454a3d3bb6274367bd696b8cac52cde4f05a
 metrics-web commit 5958454]. If there are more changes, I'll try to add
 those as fixup/squash commits.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:   |Sponsor:
 |  Sponsor28
-+-

Comment (by cohosh):

 Replying to [comment:7 karsten]:
 > One quick comment on the spec after writing the metrics-lib part: The
 `"snowflake-stats-end"` line should have multiplicity `"[At start, exactly
 once.]"` rather than `"[At most once.]"`. Other than that, everything
 seemed reasonable from a parsing perspective. (I didn't start with the
 CollecTor part, yet.)
 >
 Thanks! Filed a PR for torspec in #31407 with this change.
 > To answer your earlier question above: having just the last 24 hours of
 data might be problematic if the metrics host goes down for, say, a
 weekend. Ideally, there would be at least a week of statistics available.
 Or maybe just accumulate new statistics forever, given the tiny amount of
 statistics per day.
 Okay great, I have a implementation of this in #31376 that will respond
 with all logged metrics data.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:   |Sponsor:
 |  Sponsor28
-+-

Comment (by karsten):

 One quick comment on the spec after writing the metrics-lib part: The
 `"snowflake-stats-end"` line should have multiplicity `"[At start, exactly
 once.]"` rather than `"[At most once.]"`. Other than that, everything
 seemed reasonable from a parsing perspective. (I didn't start with the
 CollecTor part, yet.)

 To answer your earlier question above: having just the last 24 hours of
 data might be problematic if the metrics host goes down for, say, a
 weekend. Ideally, there would be at least a week of statistics available.
 Or maybe just accumulate new statistics forever, given the tiny amount of
 statistics per day.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:   |Sponsor:
 |  Sponsor28
-+-

Comment (by cohosh):

 Replying to [comment:5 karsten]:
 > Where would I find the spec? I found
 [https://github.com/cohosh/snowflake/pull/3/files#diff-
 c8ef7ba143d251e41b143b2ab02f3733 this comment] which is fine for me to
 start write some metrics-lib code. But we'll have to link to something
 more permanent once we start collecting these statistics; ideally tor-
 spec.git. Thanks!

 I'll work on adding it to tor-spec.git. Right now the source code is the
 ground truth on the spec.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:   |Sponsor:
 |  Sponsor28
-+-

Comment (by karsten):

 Where would I find the spec? I found
 [https://github.com/cohosh/snowflake/pull/3/files#diff-
 c8ef7ba143d251e41b143b2ab02f3733 this comment] which is fine for me to
 start write some metrics-lib code. But we'll have to link to something
 more permanent once we start collecting these statistics; ideally tor-
 spec.git. Thanks!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:   |Sponsor:
 |  Sponsor28
-+-
Changes (by cohosh):

 * Attachment "example_metrics.log" added.

 metrics log output from the last 2 measurement periods

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:   |Sponsor:
 |  Sponsor28
-+-

Comment (by cohosh):

 I started on implementing a handler to return metrics from `snowflake-
 bro...@bamsoftware.com/metrics` in #31376. I have more questions about
 what exactly you need here. Is it only the metrics from the last full 24
 hour measurement period?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+-
 Reporter:  irl  |  Owner:
 |  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-august, anti-|  Actual Points:
  censorship-roadmap-september   |
Parent ID:   | Points:  8
 Reviewer:   |Sponsor:
 |  Sponsor28
-+-
Changes (by gaba):

 * keywords:  metrics-roadmap-august => metrics-roadmap-august, anti-
 censorship-roadmap-september


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
+--
 Reporter:  irl |  Owner:  metrics-team
 Type:  enhancement | Status:  new
 Priority:  Medium  |  Milestone:
Component:  Metrics/CollecTor   |Version:
 Severity:  Normal  | Resolution:
 Keywords:  metrics-roadmap-august  |  Actual Points:
Parent ID:  | Points:  8
 Reviewer:  |Sponsor:  Sponsor28
+--
Changes (by gaba):

 * keywords:  metrics-roadmap-2019-q2 => metrics-roadmap-august


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-+--
 Reporter:  irl  |  Owner:  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/CollecTor|Version:
 Severity:  Normal   | Resolution:
 Keywords:  metrics-roadmap-2019-q2  |  Actual Points:
Parent ID:   | Points:  8
 Reviewer:   |Sponsor:  Sponsor28
-+--
Changes (by gaba):

 * sponsor:   => Sponsor28


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

[tor-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
---+-
 Reporter:  irl|  Owner:  metrics-team
 Type:  enhancement| Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/CollecTor  |Version:
 Severity:  Normal |   Keywords:  metrics-roadmap-2019-q2
Actual Points: |  Parent ID:
   Points:  8  |   Reviewer:
  Sponsor: |
---+-
 Add a module to CollecTor to archive the statistics produced in #21315.
 The actual statistics and format should be discussed in that ticket. The
 discussion in #29315 may help to inform that.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs