Re: [tor-bugs] #34023 [Metrics/Onionperf]: Reduce the number of 50 KiB downloads

2020-05-25 Thread Tor Bug Tracker & Wiki 
#34023: Reduce the number of 50 KiB downloads
---+
 Reporter:  karsten|  Owner:  karsten
 Type:  enhancement| Status:  closed
 Priority:  Medium |  Milestone:
Component:  Metrics/Onionperf  |Version:
 Severity:  Normal | Resolution:  implemented
 Keywords:  metrics-team-roadmap-2020  |  Actual Points:  0.3
Parent ID:  #33321 | Points:  1.0
 Reviewer: |Sponsor:  Sponsor59-must
---+
Changes (by karsten):

 * status:  needs_review => closed
 * points:  1 => 1.0
 * resolution:   => implemented
 * actualpoints:   => 0.3


Comment:

 I just merged this together with the other two commits in that branch. And
 in retrospect, this patch didn't need review as badly anyway. Closing.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
[email protected]
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #34023 [Metrics/Onionperf]: Reduce the number of 50 KiB downloads

2020-05-22 Thread Tor Bug Tracker & Wiki 
#34023: Reduce the number of 50 KiB downloads
---+
 Reporter:  karsten|  Owner:  karsten
 Type:  enhancement| Status:  needs_review
 Priority:  Medium |  Milestone:
Component:  Metrics/Onionperf  |Version:
 Severity:  Normal | Resolution:
 Keywords:  metrics-team-roadmap-2020  |  Actual Points:
Parent ID:  #33321 | Points:  1
 Reviewer: |Sponsor:  Sponsor59-must
---+
Changes (by karsten):

 * status:  accepted => needs_review


Comment:

 Please review
 
[https://gitweb.torproject.org/user/karsten/onionperf.git/commit/?h=tasks-33434-34024-and-34023&id=3252ddfaad76f6e940afd29285b5644a8ffafcee
 commit 3252ddf in my tasks-33434-34024-and-34023 branch].

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
[email protected]
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #34023 [Metrics/Onionperf]: Reduce the number of 50 KiB downloads

2020-05-22 Thread Tor Bug Tracker & Wiki 
#34023: Reduce the number of 50 KiB downloads
---+
 Reporter:  karsten|  Owner:  karsten
 Type:  enhancement| Status:  accepted
 Priority:  Medium |  Milestone:
Component:  Metrics/Onionperf  |Version:
 Severity:  Normal | Resolution:
 Keywords:  metrics-team-roadmap-2020  |  Actual Points:
Parent ID:  #33321 | Points:  1
 Reviewer: |Sponsor:  Sponsor59-must
---+
Changes (by karsten):

 * status:  needs_review => accepted
 * owner:  metrics-team => karsten


Comment:

 We discussed this at our weekly team meeting on Thursday and decided that
 there are no major concerns against making this change and that we're
 going to do it. I'm working on a patch.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
[email protected]
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #34023 [Metrics/Onionperf]: Reduce the number of 50 KiB downloads

2020-05-16 Thread Tor Bug Tracker & Wiki 
#34023: Reduce the number of 50 KiB downloads
---+
 Reporter:  karsten|  Owner:  metrics-team
 Type:  enhancement| Status:  needs_review
 Priority:  Medium |  Milestone:
Component:  Metrics/Onionperf  |Version:
 Severity:  Normal | Resolution:
 Keywords:  metrics-team-roadmap-2020  |  Actual Points:
Parent ID: | Points:  1
 Reviewer: |Sponsor:  Sponsor59-must
---+

Comment (by karsten):

 With #26673 being resolved we now have partial completion times available.
 That means that we have timestamps when a 5 MiB download completed the
 first 50 KiB or 1 MiB and when a 1 MiB download completed the first 50
 KiB.

 I reprocessed past measurement results from op-hk2, op-nl2, and op-us2
 from the first two weeks of May 2020 to see whether we can use partial
 completion times of larger downloads together with full download
 completion times.

 Here are some remarks on the
 [https://trac.torproject.org/projects/tor/attachment/ticket/34023
 /onionperf-partials-2020-05-16.pdf attached PDF]:

  1. The first plot shows number of measurements without (green) and with
 (purple) including partial downloads. The number of 50 KiB measurements
 increases by about 1/5, which includes both 1 MiB and 5 MiB downloads. The
 number of 1 MiB measurements increases by about 1/2 as compared to before,
 which only includes the 5 MiB downloads. The number of 5 MiB downloads
 stays the same, because there are no larger downloads than 5 MiB.

  2. The second plot shows ECDFs of time to download 50 KiB. Each of the
 subplots contains two lines, one in green and one in purple. They are just
 so similar that they're basically indistinguishable. The purple line
 contains 20% more data points than the green line, but that doesn't make
 any visible difference.

  3. The third plot shows ECDFs of time to download 1 MiB. Interestingly,
 there are some minor differences between the green and purple line. The
 reason is that the purple line contains 50% more data points than the
 green line. That's a larger difference than the additional 20% in the 50
 KiB case. Still, there doesn't seem to be systematically different
 measurements when including partial completion times or not.

  4. The fourth plot shows ECDFs of time to download 5 MiB. I only included
 this as a way to sanity check the plotting code. Green and purple lines
 are exactly the same here, because we have as many partial completion
 times for 5 MiB downloads as full download completion times.

 Let's discuss how to proceed with these results. My recommendation is that
 we:

  1. extend the [https://metrics.torproject.org/torperf.html "Time to
 download files over Tor" graph on the Tor Metrics website] to include
 partial completion times, if available, and

  2. modify deployed OnionPerf instances to ''only'' download 5 MiB files
 and no 50 KiB or 1 MiB files anymore.

 Leaving in needs_review to get feedback. I'll also bring this ticket up at
 the next weekly meeting on Thursday.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
[email protected]
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #34023 [Metrics/Onionperf]: Reduce the number of 50 KiB downloads

2020-05-16 Thread Tor Bug Tracker & Wiki 
#34023: Reduce the number of 50 KiB downloads
---+
 Reporter:  karsten|  Owner:  metrics-team
 Type:  enhancement| Status:  needs_review
 Priority:  Medium |  Milestone:
Component:  Metrics/Onionperf  |Version:
 Severity:  Normal | Resolution:
 Keywords:  metrics-team-roadmap-2020  |  Actual Points:
Parent ID: | Points:  1
 Reviewer: |Sponsor:  Sponsor59-must
---+
Changes (by karsten):

 * Attachment "onionperf-partials-2020-05-16.pdf" added.


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
[email protected]
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #34023 [Metrics/Onionperf]: Reduce the number of 50 KiB downloads

2020-05-05 Thread Tor Bug Tracker & Wiki 
#34023: Reduce the number of 50 KiB downloads
---+--
 Reporter:  karsten|  Owner:  metrics-team
 Type:  enhancement| Status:  needs_review
 Priority:  Medium |  Milestone:
Component:  Metrics/Onionperf  |Version:
 Severity:  Normal | Resolution:
 Keywords:  metrics-team-roadmap-2020  |  Actual Points:
Parent ID: | Points:  1
 Reviewer: |Sponsor:  Sponsor59
---+--
Changes (by gaba):

 * keywords:   => metrics-team-roadmap-2020
 * points:   => 1
 * sponsor:   => Sponsor59


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
[email protected]
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #34023 [Metrics/Onionperf]: Reduce the number of 50 KiB downloads

2020-04-27 Thread Tor Bug Tracker & Wiki 
#34023: Reduce the number of 50 KiB downloads
---+--
 Reporter:  karsten|  Owner:  metrics-team
 Type:  enhancement| Status:  needs_review
 Priority:  Medium |  Milestone:
Component:  Metrics/Onionperf  |Version:
 Severity:  Normal | Resolution:
 Keywords: |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by karsten):

 Even better plan: I'm going to deploy the three new OnionPerf instances
 without changing weights or timeouts and leave them running for a while to
 see whether results are different from currently running instances. If
 they are not, we'll learn that something in the setup was different, and
 it would be good to keep the number of changes as small as possible.
 Coming back to this in May. Thanks for all the great input so far!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
[email protected]
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #34023 [Metrics/Onionperf]: Reduce the number of 50 KiB downloads (was: Kill the 50 KiB downloads)

2020-04-27 Thread Tor Bug Tracker & Wiki 
#34023: Reduce the number of 50 KiB downloads
---+--
 Reporter:  karsten|  Owner:  metrics-team
 Type:  enhancement| Status:  needs_review
 Priority:  Medium |  Milestone:
Component:  Metrics/Onionperf  |Version:
 Severity:  Normal | Resolution:
 Keywords: |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by karsten):

 Replying to [comment:4 acute]:
 > We could also look at reprocessing past results to test how this fares
 against actual measurements.

 Yes, this makes a lot of sense! Adding this to my list.

 (Changing the summary back; apparently something about concurrent updates
 on the ticket.)

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
[email protected]
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #34023 [Metrics/Onionperf]: Reduce the number of 50 KiB downloads (was: Kill the 50 KiB downloads)

2020-04-27 Thread Tor Bug Tracker & Wiki 
#34023: Reduce the number of 50 KiB downloads
---+--
 Reporter:  karsten|  Owner:  metrics-team
 Type:  enhancement| Status:  needs_review
 Priority:  Medium |  Milestone:
Component:  Metrics/Onionperf  |Version:
 Severity:  Normal | Resolution:
 Keywords: |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by karsten):

 Replying to [comment:2 robgjansen]:
 > **If we remove 50KiB, also remove 1Mib?**
 >
 > My main concern isn't the added load on the network, but rather that you
 are removing a metric that we have consistently used as a benchmark for
 the last ~decade. It's useful to be able to compare against a consistent
 benchmark over time.
 >
 > I notice from [https://trac.torproject.org/projects/tor/ticket/33076
 #33076] the suggestion that we could use the 1MiB and/or 5MiB results to
 compute the 50KiB times (using the incremental DATAPERC timestamps). That
 seems reasonable to me. Following that logic though, why not remove the
 1MiB file too? I think both the 50KiB and the 1MiB times could be computed
 from the 5MiB results, since we have incremental timestamps for every 10%
 of the download.

 That's a good point. Here's the math for that suggestion:

  - With only 5 MiB downloads we'd be downloading on average 5 MiB = 5120
 KiB every 5 minutes, or 5120 * 8 * 1024 / (300 * 1000) = 140 kbps.

 And yes, we do have some code somewhere to compute partial completion
 timestamps from 5 MiB downloads.

 > **Reasons to keep all of the files.**
 >
 > I think there are two strong reasons to keep all 3 file sizes:
 > 1. You can specify a different timeout for each of the 3 sizes. That
 let's you cancel the smaller files much sooner if they are hanging. And if
 the timeouts are set realistically, it helps you get a better sense of how
 often we fail to meet a target completion time.

 In theory, we could retroactively apply timeouts by pretending that a
 partial 50 KiB or 1 MiB download taking longer than `x` would have timed
 out.

 > 1. Diversity of circuits. If you follow the suggestion above and remove
 50KiB and 1MiB and only keep 5MiB, and then you get a crappy circuit, data
 points for all 3 download times will be affected. Previously that only
 affected one data point.

 We wouldn't change the frequency of making downloads. We would just
 extract more than one time-to-last-byte timestamp from a given
 measurement. But I see how we would want to document this very clearly to
 help our users interpret our data.

 > **Adjust download weights instead?**
 >
 > If you would like more data points for 1MiB and 5MiB files and fewer for
 50KiB, have you considered adjusting the weights that are used in the TGen
 model file instead of completely removing a file size?
 [https://gitweb.torproject.org/onionperf.git/tree/onionperf/model.py#n90
 The weights are specified here.] For example, if you want all file sizes
 to have equal download probabilities, set the weight for each file size to
 `1.0`.)

 We did consider this to avoid increasing load on the measurement hosts and
 network too much but then figured we can kill these downloads altogether.
 But you raise some important points above that require more attention
 before killing 50 KiB downloads entirely.

 New plan: use a weight of 1.0 for all three download sizes until we figure
 out how to kill 50 KiB and 1 MiB downloads.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
[email protected]
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs