Re: [tor-bugs] #20224 [Metrics/CollecTor]: Fix `BridgeDescriptorMappingsLimit` config option

[Tor Bug Tracker & Wiki]

#20224: Fix `BridgeDescriptorMappingsLimit` config option
---+-
 Reporter:  karsten|  Owner:  karsten
 Type:  defect | Status:  reopened
 Priority:  Low|  Milestone:  CollecTor 1.7.0
Component:  Metrics/CollecTor  |Version:
 Severity:  Normal | Resolution:
 Keywords:  metrics-2018   |  Actual Points:
Parent ID: | Points:
 Reviewer:  irl|Sponsor:
---+-
Changes (by cypherpunks):

 * status:  closed => reopened
 * resolution:  fixed =>


Comment:

 (diff)
 11:08 AM Ticket #21377 (DirAuths should expose bwauth bandwidth files)
 updated by juga
 Status changed
 11:07 AM Ticket #21377 (DirAuths should expose bwauth bandwidth files)
 updated by juga
 Changes to dir-spec commented …
 10:30 AM Tickets #25242,25673,25861 batch updated by irl
 Owner, Status changed
 Working on this this week.
 10:23 AM Ticket #24153 (Make DescriptorCollector resume previously aborted
 downloads) updated by irl
 Status changed
 Looks good to me. For the new ticket: Files.createTempFile only …
 10:20 AM Ticket #3723 (Report version of bwscanners in votes) updated by
 juga
 Sorry, there were new commits on master touching parts of that code …
 10:14 AM Ticket #24546 (Use tor_addr_is_v4() rather than family, or reject
 all v6-mapped IPv4 ...) updated by ahf
 Status changed
 The patches looks good to me, but there is a big merge conflict right
 …
 10:09 AM Ticket #26374 (MacOS Sandbox) updated by ahf
 Started some early refactoring work for this to be possible in …
 10:07 AM Ticket #20224 (Fix `BridgeDescriptorMappingsLimit` config option)
 updated by irl
 Status changed
 Looks good to me.
 9:42 AM Ticket #26193 (Tarballs are not compressed in a run following an
 aborted run) updated by irl
 Status changed
 Looks good to me. I don't think there is a need to combine the tar …
 9:34 AM Ticket #24291 (Rename CollecTor packages) updated by irl
 Status changed
 Looks good to me.
 9:31 AM Ticket #18342 (Provide more accurate reverse DNS results) updated
 by irl
 Status changed

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20224 [Metrics/CollecTor]: Fix `BridgeDescriptorMappingsLimit` config option

[Tor Bug Tracker & Wiki]

#20224: Fix `BridgeDescriptorMappingsLimit` config option
---+-
 Reporter:  karsten|  Owner:  karsten
 Type:  defect | Status:  closed
 Priority:  Low|  Milestone:  CollecTor 1.7.0
Component:  Metrics/CollecTor  |Version:
 Severity:  Normal | Resolution:  fixed
 Keywords:  metrics-2018   |  Actual Points:
Parent ID: | Points:
 Reviewer:  irl|Sponsor:
---+-
Changes (by karsten):

 * status:  merge_ready => closed
 * resolution:   => fixed


Comment:

 Thanks for looking! Rebased and pushed to master. Closing.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20224 [Metrics/CollecTor]: Fix `BridgeDescriptorMappingsLimit` config option

[Tor Bug Tracker & Wiki]

#20224: Fix `BridgeDescriptorMappingsLimit` config option
---+-
 Reporter:  karsten|  Owner:  karsten
 Type:  defect | Status:  merge_ready
 Priority:  Low|  Milestone:  CollecTor 1.7.0
Component:  Metrics/CollecTor  |Version:
 Severity:  Normal | Resolution:
 Keywords:  metrics-2018   |  Actual Points:
Parent ID: | Points:
 Reviewer:  irl|Sponsor:
---+-
Changes (by irl):

 * status:  needs_review => merge_ready


Comment:

 Looks good to me.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20224 [Metrics/CollecTor]: Fix `BridgeDescriptorMappingsLimit` config option

[Tor Bug Tracker & Wiki]

#20224: Fix `BridgeDescriptorMappingsLimit` config option
---+-
 Reporter:  karsten|  Owner:  karsten
 Type:  defect | Status:  needs_review
 Priority:  Low|  Milestone:  CollecTor 1.7.0
Component:  Metrics/CollecTor  |Version:
 Severity:  Normal | Resolution:
 Keywords:  metrics-2018   |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+-
Changes (by karsten):

 * status:  accepted => needs_review


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20224 [Metrics/CollecTor]: Fix `BridgeDescriptorMappingsLimit` config option

[Tor Bug Tracker & Wiki]

#20224: Fix `BridgeDescriptorMappingsLimit` config option
---+-
 Reporter:  karsten|  Owner:  karsten
 Type:  defect | Status:  accepted
 Priority:  Low|  Milestone:  CollecTor 1.7.0
Component:  Metrics/CollecTor  |Version:
 Severity:  Normal | Resolution:
 Keywords:  metrics-2018   |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+-
Changes (by karsten):

 * owner:  metrics-team => karsten
 * status:  assigned => accepted


Comment:

 Taking over from iwakeh here. irl, please review
 [https://gitweb.torproject.org/karsten/metrics-db.git/log/?h=task-20224-2
 my task-20224-2 branch], which is a rebase of iwakeh's branch linked above
 plus a squash commit with two fixes.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20224 [Metrics/CollecTor]: Fix `BridgeDescriptorMappingsLimit` config option

[Tor Bug Tracker & Wiki]

#20224: Fix `BridgeDescriptorMappingsLimit` config option
---+-
 Reporter:  karsten|  Owner:  iwakeh
 Type:  defect | Status:  needs_revision
 Priority:  Low|  Milestone:  CollecTor 1.6.0
Component:  Metrics/CollecTor  |Version:
 Severity:  Normal | Resolution:
 Keywords:  metrics-2018   |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+-
Changes (by karsten):

 * status:  needs_review => needs_revision


Comment:

 Hmm, that doesn't fix the problem here. Sample log output:

 {{{
 2018-04-16 10:30:18,170 INFO o.t.c.b.SanitizedBridgesWriter:212 Using cut-
 off datetime '+5877594-10-07 10:30:18' for secrets.
 }}}

 And can you also include a change log entry? Thanks!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20224 [Metrics/CollecTor]: Fix `BridgeDescriptorMappingsLimit` config option

[Tor Bug Tracker & Wiki]

#20224: Fix `BridgeDescriptorMappingsLimit` config option
---+--
 Reporter:  karsten|  Owner:  iwakeh
 Type:  defect | Status:  needs_review
 Priority:  Low|  Milestone:
Component:  Metrics/CollecTor  |Version:
 Severity:  Normal | Resolution:
 Keywords:  metrics-2018   |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--
Changes (by iwakeh):

 * status:  accepted => needs_review


Comment:

 Please review
 [https://gitweb.torproject.org/user/iwakeh/collector.git/commit/?h=task-20224
 this patch commit].

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20224 [Metrics/CollecTor]: Fix `BridgeDescriptorMappingsLimit` config option

[Tor Bug Tracker & Wiki]

#20224: Fix `BridgeDescriptorMappingsLimit` config option
---+--
 Reporter:  karsten|  Owner:  iwakeh
 Type:  defect | Status:  accepted
 Priority:  Low|  Milestone:
Component:  Metrics/CollecTor  |Version:
 Severity:  Normal | Resolution:
 Keywords:  metrics-2018   |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--
Changes (by iwakeh):

 * owner:  metrics-team => iwakeh
 * status:  assigned => accepted


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20224 [Metrics/CollecTor]: Fix `BridgeDescriptorMappingsLimit` config option

[Tor Bug Tracker & Wiki]

#20224: Fix `BridgeDescriptorMappingsLimit` config option
---+--
 Reporter:  karsten|  Owner:  iwakeh
 Type:  defect | Status:  assigned
 Priority:  Low|  Milestone:
Component:  Metrics/CollecTor  |Version:
 Severity:  Normal | Resolution:
 Keywords: |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--
Changes (by iwakeh):

 * owner:   => iwakeh
 * status:  needs_review => assigned


Comment:

 Oh, those date formats must have caused many programming/troubleshooting
 years on earth, in total.

 The cut-off makes sense; there are no older descriptors.

 I'll add this to my list.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20224 [Metrics/CollecTor]: Fix `BridgeDescriptorMappingsLimit` config option

[Tor Bug Tracker & Wiki]

#20224: Fix `BridgeDescriptorMappingsLimit` config option
---+--
 Reporter:  karsten|  Owner:
 Type:  defect | Status:  needs_review
 Priority:  Low|  Milestone:
Component:  Metrics/CollecTor  |Version:
 Severity:  Normal | Resolution:
 Keywords: |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by karsten):

 Hmm, no, that seems wrong or at least confusing, and on second thought my
 patch above is wrong or at least confusing, too.

 Here's a table with some timestamps that seem to be relevant here:

 ||= Row=||= `BridgeDescriptorMappingsLimit`=||= ''Milliseconds since the
 epoch''=||=''ISO-8601 date/time'' =||
 || 1|| `0`|| `1475517323620`|| AD 2016-10-03 17:55:23.620 ||
 || 2|| `6120.7468`|| `94668479`|| AD 1999-12-31 23:59:59.999 ||
 || 3|| `17077.7468`|| `-1`|| AD 1969-12-31 23:59:59.999 ||
 || 4|| `17102.6019`|| `-Integer.MAX_VALUE = -2147483647`|| AD 1969-12-07
 03:28:36.353 ||
 || 5|| `-Integer.MAX_VALUE = -2147483647`|| `-18554583476380`|| BC
 5877475-12-05 17:55:23.620 ||

 Explanations for these rows:
  1. This is the current date/time, so if we had put in `0` as value for
 `BridgeDescriptorMappingsLimit`, that's what the cut off time would have
 been.
  2. This is what we had put in before using `Integer.MAX_VALUE` as
 parameter default value, and this date in 1999 is indeed small enough that
 no bridge descriptor could have been published earlier.
  3. This is what I had in mind in my patch above: pick `-1` as smallest
 possible time in milliseconds since the epoch, but only because I had
 misread 1999 as 1969.  Oops.
  4. That's what I think you're suggesting above, and I think it doesn't
 really make sense to use this threshold to decide whether or not to fall
 back to a date in 1999.
  5. Here's the explanation for this bug, which I found out today: we're
 not looking at an integer overflow/underflow, we're looking at BC dates
 rather than AD dates, and we're not including AD/BC in the output date
 format.  Ugh!

 My suggestion is to cut off at `1999-12-31 23:59:59` by hard-coding that
 date/time in milliseconds as threshold.  Untested code:

 {{{
SimpleDateFormat formatter = new SimpleDateFormat(
"-MM-dd HH:mm:ss");
formatter.setTimeZone(TimeZone.getTimeZone("UTC"));
long cutTime = System.currentTimeMillis() - 24L * 60L * 60L * 1000L
* limitBridgeSanitizingInterval);

final long MIN_CUT_TIME = 94668479L;
if (cutTime > MIN_CUT_TIME) {
  this.bridgeSanitizingCutOffTimestamp = formatter.format(cutTime);
} else {
  this.bridgeSanitizingCutOffTimestamp = "1999-12-31 23:59:59";
}
 }}}

 Does this make sense?  If so, want to submit another patch?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20224 [Metrics/CollecTor]: Fix `BridgeDescriptorMappingsLimit` config option

[Tor Bug Tracker & Wiki]

#20224: Fix `BridgeDescriptorMappingsLimit` config option
---+--
 Reporter:  karsten|  Owner:
 Type:  defect | Status:  needs_review
 Priority:  Low|  Milestone:
Component:  Metrics/CollecTor  |Version:
 Severity:  Normal | Resolution:
 Keywords: |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by iwakeh):

 I'd rather set the time explicitly in case of a too small value, i.e.

 {{{

SimpleDateFormat formatter = new SimpleDateFormat(
"-MM-dd HH:mm:ss");
formatter.setTimeZone(TimeZone.getTimeZone("UTC"));
long cutTime = System.currentTimeMillis() - 24L * 60L * 60L * 1000L
* limitBridgeSanitizingInterval);

if (cutTime > -Integer.MAX_VALUE)
  this.bridgeSanitizingCutOffTimestamp = formatter.format(cutTime);
} else {
  this.bridgeSanitizingCutOffTimestamp = "1999-12-31 23:59:59";
}
 }}}

 (Assuming 1999-12-31 23:59:59 is the smallest possible time stamp that
 makes sense.)

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20224 [Metrics/CollecTor]: Fix `BridgeDescriptorMappingsLimit` config option

[Tor Bug Tracker & Wiki]

#20224: Fix `BridgeDescriptorMappingsLimit` config option
---+--
 Reporter:  karsten|  Owner:
 Type:  defect | Status:  needs_review
 Priority:  Low|  Milestone:
Component:  Metrics/CollecTor  |Version:
 Severity:  Normal | Resolution:
 Keywords: |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--
Changes (by karsten):

 * status:  new => needs_review


Comment:

 Hmm, looks like I should have set this to needs_review before.  Doing that
 now.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs