Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
-+-
 Reporter:  teor |  Owner:  (none)
 Type:  enhancement  | Status:  closed
 Priority:  Low  |  Milestone:  Tor:
 |  unspecified
Component:  Core Tor/Tor |Version:  Tor:
 |  0.2.9.1-alpha
 Severity:  Normal   | Resolution:  wontfix
 Keywords:  regression, triaged-out-20170124,|  Actual Points:
  034-triage-20180328, 034-removed-20180328  |
Parent ID:   | Points:  0.5
 Reviewer:   |Sponsor:
-+-
Changes (by teor):

 * status:  new => closed
 * resolution:   => wontfix


Comment:

 These schedules use  exponential backoff with decorrelated jitter, with no
 maximums. That seems to be working well enough,

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
--+
 Reporter:  teor  |  Owner:  (none)
 Type:  defect| Status:  new
 Priority:  Low   |  Milestone:  Tor:
  |  0.3.3.x-final
Component:  Core Tor/Tor  |Version:  Tor:
  |  0.2.9.1-alpha
 Severity:  Normal| Resolution:
 Keywords:  regression, triaged-out-20170124  |  Actual Points:
Parent ID:| Points:  0.5
 Reviewer:|Sponsor:
--+
Changes (by nickm):

 * status:  assigned => new


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
--+
 Reporter:  teor  |  Owner:  (none)
 Type:  defect| Status:  assigned
 Priority:  Low   |  Milestone:  Tor:
  |  0.3.3.x-final
Component:  Core Tor/Tor  |Version:  Tor:
  |  0.2.9.1-alpha
 Severity:  Normal| Resolution:
 Keywords:  regression, triaged-out-20170124  |  Actual Points:
Parent ID:| Points:  0.5
 Reviewer:|Sponsor:
--+
Changes (by nickm):

 * owner:  nickm => (none)
 * status:  accepted => assigned
 * milestone:  Tor: 0.3.2.x-final => Tor: 0.3.3.x-final


Comment:

 Okay; then I'll unassign from myself (I can't write this documentation)
 and defer to 0.3.3.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
--+
 Reporter:  teor  |  Owner:  nickm
 Type:  defect| Status:  accepted
 Priority:  Low   |  Milestone:  Tor:
  |  0.3.2.x-final
Component:  Core Tor/Tor  |Version:  Tor:
  |  0.2.9.1-alpha
 Severity:  Normal| Resolution:
 Keywords:  regression, triaged-out-20170124  |  Actual Points:
Parent ID:| Points:  0.5
 Reviewer:|Sponsor:
--+

Comment (by teor):

 Replying to [comment:18 nickm]:
 > Merged, leaving open for documentation

 We should document and close this.
 Given that things are working, the other child tickets aren't a high
 priority.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
--+
 Reporter:  teor  |  Owner:  nickm
 Type:  defect| Status:  accepted
 Priority:  Low   |  Milestone:  Tor:
  |  0.3.2.x-final
Component:  Core Tor/Tor  |Version:  Tor:
  |  0.2.9.1-alpha
 Severity:  Normal| Resolution:
 Keywords:  regression, triaged-out-20170124  |  Actual Points:
Parent ID:| Points:  0.5
 Reviewer:|Sponsor:
--+

Comment (by nickm):

 What's our status here?  Should we merge something else in 0.3.2, or close
 this and document it, or some other thing?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
-+-
 Reporter:  teor |  Owner:  nickm
 Type:  defect   | Status:
 |  accepted
 Priority:  Low  |  Milestone:  Tor:
 |  0.3.2.x-final
Component:  Core Tor/Tor |Version:  Tor:
 |  0.2.9.1-alpha
 Severity:  Normal   | Resolution:
 Keywords:  regression, CoreTorTeam201611,   |  Actual Points:
  triaged-out-20170124, 031-reach|
Parent ID:   | Points:  0.5
 Reviewer:   |Sponsor:
-+-
Changes (by nickm):

 * milestone:  Tor: 0.3.1.x-final => Tor: 0.3.2.x-final


Comment:

 I'm not going to be able to get this right in 0.3.1

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
-+-
 Reporter:  teor |  Owner:  nickm
 Type:  defect   | Status:
 |  accepted
 Priority:  Low  |  Milestone:  Tor:
 |  0.3.1.x-final
Component:  Core Tor/Tor |Version:  Tor:
 |  0.2.9.1-alpha
 Severity:  Normal   | Resolution:
 Keywords:  regression, CoreTorTeam201611,   |  Actual Points:
  triaged-out-20170124, 031-reach|
Parent ID:   | Points:  0.5
 Reviewer:   |Sponsor:
-+-
Changes (by catalyst):

 * cc: catalyst (added)


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
---+---
 Reporter:  teor   |  Owner:  nickm
 Type:  defect | Status:  accepted
 Priority:  Medium |  Milestone:  Tor:
   |  0.3.0.x-final
Component:  Core Tor/Tor   |Version:  Tor:
   |  0.2.9.1-alpha
 Severity:  Normal | Resolution:
 Keywords:  regression, CoreTorTeam201611  |  Actual Points:
Parent ID: | Points:  0.5
 Reviewer: |Sponsor:
---+---
Changes (by teor):

 * parent:  #20499 =>


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
---+---
 Reporter:  teor   |  Owner:  nickm
 Type:  defect | Status:  accepted
 Priority:  Medium |  Milestone:  Tor:
   |  0.3.0.x-final
Component:  Core Tor/Tor   |Version:  Tor:
   |  0.2.9.1-alpha
 Severity:  Normal | Resolution:
 Keywords:  regression, CoreTorTeam201611  |  Actual Points:
Parent ID:  #20499 | Points:  0.5
 Reviewer: |Sponsor:
---+---

Comment (by teor):

 Turns out that sometimes I just can't do my sums right.
 Here are the actual figures for each case with exponent 4 (multiplier 3):

 Initial delay 0: (most schedules)
 {{{
 Attempt / Failure   Min Increment   Max Increment   Average Minimum
 Maximum
 1   0   0   0   0   0
 2   1   2   2   1   2
 3   1   8   5   2   10
 4   1   32  13  3   42
 5   1   128 33  4   170
 6   1   512 84  5   682
 7   1   2048211 6   2730
 }}}

 Initial delay 6: (client bootstrap from authorities)
 {{{
 Attempt / Failure   Min Increment   Max Increment   Average Minimum
 Maximum
 1   0   0   6   6   6
 2   1   20  11  7   26
 3   1   80  27  8   106
 4   1   320 69  9   426
 5   1   1280174 10  1706
 }}}

 Initial delay 1200: (bridge client bridge descriptors)
 {{{
 Attempt / Failure   Min Increment   Max Increment   Average Minimum
 Maximum
 1   0   0   120012001200
 2   1   3602180212014802
 3   1   14408   4505120219210
 4   1   57632   11263   120376842
 }}}

 And for test networks, with exponent 3 (multiplier 2):
 Initial delay 0: (most schedules)
 {{{
 Attempt / Failure   Min Increment   Max Increment   Average Minimum
 Maximum
 1   0   0   0   0   0
 2   1   2   2   1   2
 3   1   6   4   2   8
 4   1   18  9   3   26
 5   1   54  19  4   80
 6   1   162 39  5   242
 7   1   486 79  6   728
 }}}

 Initial delay 20 (bridge client bridge descriptors):
 {{{
 Attempt / Failure   Min Increment   Max Increment   Average Minimum
 Maximum
 1   0   0   20  20  20
 2   1   42  42  21  62
 3   1   126 84  22  188
 }}}

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
---+---
 Reporter:  teor   |  Owner:  nickm
 Type:  defect | Status:  accepted
 Priority:  Medium |  Milestone:  Tor:
   |  0.3.0.x-final
Component:  Core Tor/Tor   |Version:  Tor:
   |  0.2.9.1-alpha
 Severity:  Normal | Resolution:
 Keywords:  regression, CoreTorTeam201611  |  Actual Points:
Parent ID:  #20499 | Points:  0.5
 Reviewer: |Sponsor:
---+---
Changes (by nickm):

 * status:  merge_ready => accepted
 * owner:   => nickm
 * milestone:  Tor: 0.2.9.x-final => Tor: 0.3.0.x-final


Comment:

 Merged, leaving open for documentation

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
---+---
 Reporter:  teor   |  Owner:
 Type:  defect | Status:  merge_ready
 Priority:  Medium |  Milestone:  Tor:
   |  0.2.9.x-final
Component:  Core Tor/Tor   |Version:  Tor:
   |  0.2.9.1-alpha
 Severity:  Normal | Resolution:
 Keywords:  regression, CoreTorTeam201611  |  Actual Points:
Parent ID:  #20499 | Points:  0.5
 Reviewer: |Sponsor:
---+---

Comment (by teor):

 I wanted a multiplier of 2.5, but we settled on 3, because, integers.

 In 0.3.0 or 0.3.1, I think we should consider having the next delay in
 [delay*2, delay*3], rather than [delay, delay*3]. This would increase the
 average and lower bound, and decrease the upper bound (I don't like the
 range in the current model). The initial values might need some tweaking
 after this change.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
---+---
 Reporter:  teor   |  Owner:
 Type:  defect | Status:  merge_ready
 Priority:  Medium |  Milestone:  Tor:
   |  0.2.9.x-final
Component:  Core Tor/Tor   |Version:  Tor:
   |  0.2.9.1-alpha
 Severity:  Normal | Resolution:
 Keywords:  regression, CoreTorTeam201611  |  Actual Points:
Parent ID:  #20499 | Points:  0.5
 Reviewer: |Sponsor:
---+---

Comment (by teor):

 Here's how the latest version works, with the next delay in [1, delay*5]
 (multiplier 4), except when delay is 0, when the next delay is in [1,2].
 This time, I'm using ranges and averages:

 {{{
 0, 1.5 [1-2], 4.3 [2-10], 11 [3-50], 28 [4-250], 71 [5-1250], ...
 6, 19 [7-30], 47 [8-150], 117 [9-750], 294 [10-3750], ...
 1200, 3601 [1201-6000], 9002 [1202-3], 22505 [1203-15], ...
 }}}

 Ok, so that multiplier is going to be terrible (in rare cases) for
 clients, let's try delay*4 (multiplier 3):

 {{{
 0, 1.5 [1-2], 5 [2-8], 11 [3-32], 22 [4-128], 44 [5-256], 88 [6-512], ...
 6, 16 [7-24], 32 [8-96], 64 [9-384], 128 [10-1536], ...
 1200, 3001 [1201-4800], 6002 [1202-19200], 12004 [1203-76800], ...
 }}}

 And delay*3 (multiplier 2):

 {{{
 0, 1.5 [1-2], 4 [2-6], 6.5 [3-18], 10 [4-54], 16 [5-162], 24 [6-486], 37
 [7-1458], 56 [8-4374], 85 [9-13122], ...
 6, 13 [7-18], 19 [8-54], 29 [9-162], 45 [10-486], ...
 1200, 2400 [1201-3600], 3601 [1202-10800], 5402 [1203-32400], ...
 }}}

 So a multiplier of 2 is too fast, 3 seems just about right.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
---+---
 Reporter:  teor   |  Owner:
 Type:  defect | Status:  merge_ready
 Priority:  Medium |  Milestone:  Tor:
   |  0.2.9.x-final
Component:  Core Tor/Tor   |Version:  Tor:
   |  0.2.9.1-alpha
 Severity:  Normal | Resolution:
 Keywords:  regression, CoreTorTeam201611  |  Actual Points:
Parent ID:  #20499 | Points:  0.5
 Reviewer: |Sponsor:
---+---

Comment (by teor):

 Also, authority-only clients will try 6 authorities in the first minute.
 (5 in the first 30 seconds, 4 in the first 10 seconds). This isn't ideal,
 but it's also not the default, so I don't think it matters that much.

 One way to fix this would be to make
 ClientBootstrapConsensusAuthorityOnlyDownloadSchedule start with 1 or 2 or
 3, rather than 0. (What I really want is a way to say: "initial delay,
 then exponential on this other delay after that".)

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
---+---
 Reporter:  teor   |  Owner:
 Type:  defect | Status:  merge_ready
 Priority:  Medium |  Milestone:  Tor:
   |  0.2.9.x-final
Component:  Core Tor/Tor   |Version:  Tor:
   |  0.2.9.1-alpha
 Severity:  Normal | Resolution:
 Keywords:  regression, CoreTorTeam201611  |  Actual Points:
Parent ID:  #20499 | Points:  0.5
 Reviewer: |Sponsor:
---+---

Comment (by teor):

 Here's a relevant comment from the 0.2.8 #4483 implementation in config.c.
 We can contrast it with the 0.2.9 #15942 implementation:

 {{{
  /* With the ClientBootstrapConsensus*Download* below:
* Clients with only authorities will try:
*  - 3 authorities over 10 seconds, then wait 60 minutes.
* Clients with authorities and fallbacks will try:
*  - 2 authorities and 4 fallbacks over 21 seconds, then wait 60
 minutes.
* Clients will also retry when an application request arrives.
* After a number of failed reqests, clients retry every 3 days + 1
 hour.
*
* Clients used to try 2 authorities over 10 seconds, then wait for
* 60 minutes or an application request.
*
* When clients have authorities and fallbacks available, they use these
* schedules: (we stagger the times to avoid thundering herds) */
 }}}

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
---+---
 Reporter:  teor   |  Owner:
 Type:  defect | Status:  merge_ready
 Priority:  Medium |  Milestone:  Tor:
   |  0.2.9.x-final
Component:  Core Tor/Tor   |Version:  Tor:
   |  0.2.9.1-alpha
 Severity:  Normal | Resolution:
 Keywords:  regression, CoreTorTeam201611  |  Actual Points:
Parent ID:  #20499 | Points:  0.5
 Reviewer: |Sponsor:
---+---

Comment (by nickm):

 I'll fix and merge then, and leave the ticket open for a documentation
 spree.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
---+---
 Reporter:  teor   |  Owner:
 Type:  defect | Status:  merge_ready
 Priority:  Medium |  Milestone:  Tor:
   |  0.2.9.x-final
Component:  Core Tor/Tor   |Version:  Tor:
   |  0.2.9.1-alpha
 Severity:  Normal | Resolution:
 Keywords:  regression, CoreTorTeam201611  |  Actual Points:
Parent ID:  #20499 | Points:  0.5
 Reviewer: |Sponsor:
---+---
Changes (by teor):

 * status:  needs_review => merge_ready


Comment:

 Replying to [comment:10 nickm]:
 > Okay, I've taken a look here in my branch `bug20534_029`. Is that what
 you had in mind?  I've not been able to convince myself 5x is safe, so I
 went with 4x. Let's see if that works out.

 That's fine, I was concerned 5x would lead to too much variance.

 One nitpick: `no more than quadruple.` is wrong, it is `no more than
 quintuple.`, because the increment is added to the existing delay.

 Let me just do my sums for exponent 2.5 (max = 4*delay):

 0, 1, 2.5, 6.3, 15.6, 39.1, 97.7, ...
 6, 15, 37.5, 93.75, ...
 1200, 3000, 7500, 18750, ...

 This would mean:

 * In 0.2.9, almost every tor instance would try to download almost every
 document 6 times in the first minute. In 0.2.8, this was 3-4 times in the
 first minute.
 * In 0.2.9, clients would try authorities 3 times in the first minute (and
 twice in the first 21 seconds). In 0.2.8, this was 2 times in the first
 minute (and twice in the first 21 seconds).
 * In 0.2.9, bridge clients will try to download bridge descriptors 3 times
 in the first 3 hours (the first time after 20 minutes). In 0.2.8, this was
 4 times in the first 3 hours (the first time after 1 hour).

 I think we should document this somewhere, but that should be a separate
 task.
 (I'm sure arma would like to read this analysis, eventually.)

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
---+---
 Reporter:  teor   |  Owner:
 Type:  defect | Status:  needs_review
 Priority:  Medium |  Milestone:  Tor:
   |  0.2.9.x-final
Component:  Core Tor/Tor   |Version:  Tor:
   |  0.2.9.1-alpha
 Severity:  Normal | Resolution:
 Keywords:  regression, CoreTorTeam201611  |  Actual Points:
Parent ID:  #20499 | Points:  0.5
 Reviewer: |Sponsor:
---+---
Changes (by nickm):

 * status:  new => needs_review


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
---+---
 Reporter:  teor   |  Owner:
 Type:  defect | Status:  new
 Priority:  Medium |  Milestone:  Tor:
   |  0.2.9.x-final
Component:  Core Tor/Tor   |Version:  Tor:
   |  0.2.9.1-alpha
 Severity:  Normal | Resolution:
 Keywords:  regression, CoreTorTeam201611  |  Actual Points:
Parent ID:  #20499 | Points:  0.5
 Reviewer: |Sponsor:
---+---

Comment (by nickm):

 Okay, I've taken a look here in my branch `bug20534_029`. Is that what you
 had in mind?  I've not been able to convince myself 5x is safe, so I went
 with 4x. Let's see if that works out.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
---+---
 Reporter:  teor   |  Owner:
 Type:  defect | Status:  new
 Priority:  Medium |  Milestone:  Tor:
   |  0.2.9.x-final
Component:  Core Tor/Tor   |Version:  Tor:
   |  0.2.9.1-alpha
 Severity:  Normal | Resolution:
 Keywords:  regression, CoreTorTeam201611  |  Actual Points:
Parent ID:  #20499 | Points:  0.5
 Reviewer: |Sponsor:
---+---
Changes (by teor):

 * keywords:  regression => regression, CoreTorTeam201611
 * status:  needs_review => new
 * version:   => Tor: 0.2.9.1-alpha


Comment:

 So, in summary, here are the 3 changes I think resolve #20499, by making
 the exponential backoff schedules much more like the 0.2.8 schedules:
 * adjust max_increment to be `delay*5`, which makes the exponent `(1+5)/2
 = 3`
 * make the first entry in
 ClientBootstrapConsensusAuthorityDownloadSchedule 6
 * make the first entry in TestingBridgeDownloadSchedule 1200

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
--+
 Reporter:  teor  |  Owner:
 Type:  defect| Status:  needs_review
 Priority:  Medium|  Milestone:  Tor: 0.2.9.x-final
Component:  Core Tor/Tor  |Version:
 Severity:  Normal| Resolution:
 Keywords:  regression|  Actual Points:
Parent ID:  #20499| Points:  0.5
 Reviewer:|Sponsor:
--+

Comment (by teor):

 Here are the schedules we used to use before exponential backoff was
 implemented:
 {{{
 TestingServerDownloadSchedule "0, 0, 0, 60, 60, 120, 300, 900, 2147483647"
 TestingClientDownloadSchedule "0, 0, 60, 300, 600, 2147483647"
 TestingServerConsensusDownloadSchedule "0, 0, 60, 300, 600, 1800, 1800,
 1800, 1800, 1800, 3600, 7200"
 TestingClientConsensusDownloadSchedule "0, 0, 60, 300, 600, 1800, 3600,
 3600, 3600, 10800, 21600, 43200"
 ClientBootstrapConsensusFallbackDownloadSchedule "0, 1, 4, 11, 3600,
 10800, 25200, 54000, 111600, 262800"
 ClientBootstrapConsensusAuthorityOnlyDownloadSchedule "0, 3, 7, 3600,
 10800, 25200, 54000, 111600, 262800"
 ClientBootstrapConsensusAuthorityDownloadSchedule "10, 11, 3600, 10800,
 25200, 54000, 111600, 262800"
 TestingBridgeDownloadSchedule "3600, 900, 900, 3600"
 }}}

 And here are the average exponential backoff attempt times for each unique
 starting point above:
 {{{
 0, 1, 2, 3.5, 5.5, 8.5, 13, 19.5, 29, 43.5, 65.5, ...
 10, 15, 22.5, 34, 51, 76.5, ...
 3600, 5400, 8100, ...
 }}}

 This means:
 * In 0.2.9, almost every tor instance will try to download almost every
 document 11 times in the first minute. In 0.2.8, this was 3-4 times in the
 first minute.
 * In 0.2.9, clients will try authorities 5 times in the first minute. In
 0.2.8, this was 2 times in the first minute.
 * In 0.2.9, bridge clients will try to download bridge descriptors 2 times
 in the first 3 hours. In 0.2.8, this was 4 times in the first 3 hours.

 We can't fix this by modifying the minimum times. But we might be able to
 fix it by modifying the exponent. Or providing a failure count at which we
 increase the delay to hourly, rather than slowly increasing it in an
 exponential fashion.

 Here's what it would look like if the exponent were 3 (max = 5*delay)
 rather than 1.5 (max = 2*delay), and we adjusted the client to authority
 and bridge descriptor start times:
 {{{
 0, 1, 3, 9, 27, 81, ...
 6, 18, 54, ...
 1200, 3600, 10800, ...
 }}}

 This would mean:
 * In 0.2.9, almost every tor instance would try to download almost every
 document 5 times in the first minute. In 0.2.8, this was 3-4 times in the
 first minute.
 * In 0.2.9, clients would try authorities 3 times in the first minute (and
 twice in the first 24 seconds). In 0.2.8, this was 2 times in the first
 minute (and twice in the first 21 seconds).
 * In 0.2.9, bridge clients will try to download bridge descriptors 3 times
 in the first 3 hours (the first time after 20 minutes). In 0.2.8, this was
 4 times in the first 3 hours (the first time after 1 hour).

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
--+
 Reporter:  teor  |  Owner:
 Type:  defect| Status:  needs_review
 Priority:  Medium|  Milestone:  Tor: 0.2.9.x-final
Component:  Core Tor/Tor  |Version:
 Severity:  Normal| Resolution:
 Keywords:  regression|  Actual Points:
Parent ID:  #20499| Points:  0.5
 Reviewer:|Sponsor:
--+

Comment (by teor):

 I've reviewed `20499_part1_029` over in #20499.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
--+
 Reporter:  teor  |  Owner:
 Type:  defect| Status:  needs_review
 Priority:  Medium|  Milestone:  Tor: 0.2.9.x-final
Component:  Core Tor/Tor  |Version:
 Severity:  Normal| Resolution:
 Keywords:  regression|  Actual Points:
Parent ID:  #20499| Points:  0.5
 Reviewer:|Sponsor:
--+
Changes (by nickm):

 * status:  needs_information => needs_review


Comment:

 I think that the answer here is just to remove the final times entirely,
 or make them not count when the schedule is exponential.  I've done this
 as part of my branch `20499_part1_029`.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
--+
 Reporter:  teor  |  Owner:
 Type:  defect| Status:  needs_information
 Priority:  Medium|  Milestone:  Tor: 0.2.9.x-final
Component:  Core Tor/Tor  |Version:
 Severity:  Normal| Resolution:
 Keywords:  regression|  Actual Points:
Parent ID:  #20499| Points:  0.5
 Reviewer:|Sponsor:
--+

Comment (by teor):

 Replying to [comment:4 dgoulet]:
 > Replying to [comment:3 teor]:
 > > Actually, there's not much point in revising these schedules - the
 exponential backoff code only pays attention to the first and last value
 in the schedule.
 >
 > teor, should we keep this in 029 then? Or the ticket at all?

 If we revert the exponential backoff code as our solution to #20499, we
 will want to tweak the final times on some of these schedules, to achieve
 Roger's goal of "never have a relay stop trying entirely".

 If we don't, we will want to adjust the initial time as well.

 So I think my comment was a bit hasty - it's only the middle times that
 don't matter, and only if we keep exponential backoff.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
--+
 Reporter:  teor  |  Owner:
 Type:  defect| Status:  needs_information
 Priority:  Medium|  Milestone:  Tor: 0.2.9.x-final
Component:  Core Tor/Tor  |Version:
 Severity:  Normal| Resolution:
 Keywords:  regression|  Actual Points:
Parent ID:  #20499| Points:  0.5
 Reviewer:|Sponsor:
--+
Changes (by dgoulet):

 * status:  new => needs_information


Comment:

 Replying to [comment:3 teor]:
 > Actually, there's not much point in revising these schedules - the
 exponential backoff code only pays attention to the first and last value
 in the schedule.

 teor, should we keep this in 029 then? Or the ticket at all?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
--+
 Reporter:  teor  |  Owner:
 Type:  defect| Status:  new
 Priority:  Medium|  Milestone:  Tor: 0.2.9.x-final
Component:  Core Tor/Tor  |Version:
 Severity:  Normal| Resolution:
 Keywords:  regression|  Actual Points:
Parent ID:  #20499| Points:  0.5
 Reviewer:|Sponsor:
--+

Comment (by teor):

 Actually, there's not much point in revising these schedules - the
 exponential backoff code only pays attention to the first and last value
 in the schedule.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
--+
 Reporter:  teor  |  Owner:
 Type:  defect| Status:  new
 Priority:  Medium|  Milestone:  Tor: 0.2.9.x-final
Component:  Core Tor/Tor  |Version:
 Severity:  Normal| Resolution:
 Keywords:  regression|  Actual Points:
Parent ID:  #20499| Points:  0.5
 Reviewer:|Sponsor:
--+

Comment (by teor):

 Some of these schedules also affect #19969

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
--+
 Reporter:  teor  |  Owner:
 Type:  defect| Status:  new
 Priority:  Medium|  Milestone:  Tor: 0.2.9.x-final
Component:  Core Tor/Tor  |Version:
 Severity:  Normal| Resolution:
 Keywords:  regression|  Actual Points:
Parent ID:  #20499| Points:  0.5
 Reviewer:|Sponsor:
--+
Changes (by teor):

 * parent:   => #20499


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

[tor-bugs] #20534 [Core Tor/Tor]: Revise hard-coded download schedules

[Tor Bug Tracker & Wiki]

#20534: Revise hard-coded download schedules
--+
 Reporter:  teor  |  Owner:
 Type:  defect| Status:  new
 Priority:  Medium|  Milestone:  Tor: 0.2.9.x-final
Component:  Core Tor/Tor  |Version:
 Severity:  Normal|   Keywords:  regression
Actual Points:|  Parent ID:
   Points:  0.5   |   Reviewer:
  Sponsor:|
--+
 We should tweak the download schedules in config.c based on what we've
 learned in #20499.

 These schedules should retry sooner than never:
 TestingServerDownloadSchedule
 TestingClientDownloadSchedule

 These schedules retry at most every 2 hours, should that be higher?
 TestingServerConsensusDownloadSchedule

 These schedules retry at most every 12 hours, should that be higher?
 lower?
 TestingClientConsensusDownloadSchedule

 These schedules retry at most every 73 hours, should that be lower?
 Should we try more times before jumping to retrying after an hour?
 ClientBootstrapConsensusAuthorityDownloadSchedule
 ClientBootstrapConsensusFallbackDownloadSchedule
 ClientBootstrapConsensusAuthorityOnlyDownloadSchedule

 Should we try more than 7 or 8 times to get directory documents?
 TestingConsensusMaxDownloadTries
 ClientBootstrapConsensusMaxDownloadTries
 TestingDescriptorMaxDownloadTries
 TestingMicrodescMaxDownloadTries
 TestingCertMaxDownloadTries

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs