Re: [tor-dev] Brief state of sbws thoughts
Hi, > On 20 Jul 2018, at 01:16, juga wrote: > > Matt Traudt: >> Teor, Juga >> >> There's a lot of things fighting for my attention right now, so you >> might have noticed I've slowed way down on attending to sbws >> tickets/PRs/etc. I think time will free up in the next few days. >> >> I think sbws is in a very good place code-wise right now. I don't think >> much more **has** to be done to the code. Even though I enjoy adding >> things like the state file (GHPR#236 [2]), I don't think that was a good >> use of my time. >> >> It looks like there's a lot of check boxes Juga has made regarding >> making a Debian package[0]. Those should get checked. These are important. >> >> However, I think the absolute most important thing for us to be spending >> our time on right now is deciding what "good" results are and verifying >> sbws produces "good" results. You’re right - we need to know if we can switch to sbws, and we can’t use sbws unless it has reasonable results. If the results aren’t reasonable, we might need to: * do further processing on the sbws results (like scaling) * change the sbws measurement design The good news is that sbws ranks are approximately the same as torflow ranks. So the measurement design is probably ok. But torflow weights are larger (max 100,000) than sbws weights (max 4000), so we will need to scale the sbws results. torflow results are also steeper than sbws results: the ratio between high and low ranked relays is 1000:1 in torflow, but 10:1 in sbws. If we want to, we can make sbws match torflow by defining a scaling algorithm that scales large relays more than small relays. But we could also decide that the flatter sbws curve is better for the network, because high-weight relays are overloaded. Let’s do a few more experiments before we decide. >> To accomplish this, I think one of the two suggestions I made in a >> comment on GH#182 [1] (quoted here) is what we should be doing. >> >> 1. Run torflow and sbws side-by-side (but not at the same time) to >> remove more variables. This has the added benefit of us having access to >> the raw scanner results from torflow before it does whatever magic >> scaling it does. OR > > In that ticket you also mentioned that someone that already runs torflow > should also run sbws. > I said i can run both, and still the case if needed. Ok, so juga can run sbws and torflow at different times on the same machine. > On 20 Jul 2018, at 01:34, Tom Ritter wrote: > > I'm happy and prepared to run sbws and torflow side by side. I'm a > little less swamped than I was a month ago. I don't need a debian > package; I'd rather run it from a git clone. > > I think the only things I can't do are > a) give you access to the box directly (but I can make whatever > files/logs/raw results that you want available to you over HTTP) > b) stop running torflow. (Unless we're ready to switch a live bwauth > over to sbws.) And tom can run sbws and torflow at the same time on the same machine. I think we should run both comparisons, wait a week so they are in a stable state, and then check the results for a few weeks. T >> [0]: https://trac.torproject.org/projects/tor/ticket/26848 >> [1]: >> https://github.com/pastly/simple-bw-scanner/issues/182#issuecomment-404250053 >> [2]: https://github.com/pastly/simple-bw-scanner/pull/236 signature.asc Description: Message signed with OpenPGP ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Brief state of sbws thoughts
I'm happy and prepared to run sbws and torflow side by side. I'm a little less swamped than I was a month ago. I don't need a debian package; I'd rather run it from a git clone. I think the only things I can't do are a) give you access to the box directly (but I can make whatever files/logs/raw results that you want available to you over HTTP) b) stop running torflow. (Unless we're ready to switch a live bwauth over to sbws.) FWIW, I have the advantage of having archived my (semi-)raw bwauth data for a while: https://bwauth.ritter.vg/bwauth/ -tom On 19 July 2018 at 10:16, juga wrote: > Matt Traudt: >> Teor, Juga >> >> There's a lot of things fighting for my attention right now, so you >> might have noticed I've slowed way down on attending to sbws >> tickets/PRs/etc. I think time will free up in the next few days. >> >> I think sbws is in a very good place code-wise right now. I don't think >> much more **has** to be done to the code. Even though I enjoy adding >> things like the state file (GHPR#236 [2]), I don't think that was a good >> use of my time. >> >> It looks like there's a lot of check boxes Juga has made regarding >> making a Debian package[0]. Those should get checked. These are important. >> >> However, I think the absolute most important thing for us to be spending >> our time on right now is deciding what "good" results are and verifying >> sbws produces "good" results. >> >> To accomplish this, I think one of the two suggestions I made in a >> comment on GH#182 [1] (quoted here) is what we should be doing. >> >> 1. Run torflow and sbws side-by-side (but not at the same time) to >> remove more variables. This has the added benefit of us having access to >> the raw scanner results from torflow before it does whatever magic >> scaling it does. OR > > In that ticket you also mentioned that someone that already runs torflow > should also run sbws. > I said i can run both, and still the case if needed. > >> 2. Ask for access to raw scanner results from someone running torflow. >> >> I fear sbws is doomed to die the death of the new bandwidth scanners >> before it if we don't start seriously verifying sbws is "good" or if I >> personally slowly stop working/coordinating work on it. > > I don't think that's the case. I've not forget it... and i'm sure teor > neither. > Some of the last work we have done is regarding getting the bandwidth > files archived, what will also help to determine whether sbws results > are "good". > > If 1. would be run by someone else, getting [0] done is indeed important > and i'm currently working on it. > > And maybe we aren't able to determine how "good" sbws results are until > it actually starts being run by dirauths, for which [0] is still important. > > Thanks for sharing your thoughts, > juga. > >> Thanks >> >> Matt >> >> [0]: https://trac.torproject.org/projects/tor/ticket/26848 >> [1]: >> https://github.com/pastly/simple-bw-scanner/issues/182#issuecomment-404250053 >> [2]: https://github.com/pastly/simple-bw-scanner/pull/236 >> ___ >> tor-dev mailing list >> tor-dev@lists.torproject.org >> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev >> > > ___ > tor-dev mailing list > tor-dev@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Brief state of sbws thoughts
Teor, Juga There's a lot of things fighting for my attention right now, so you might have noticed I've slowed way down on attending to sbws tickets/PRs/etc. I think time will free up in the next few days. I think sbws is in a very good place code-wise right now. I don't think much more **has** to be done to the code. Even though I enjoy adding things like the state file (GHPR#236 [2]), I don't think that was a good use of my time. It looks like there's a lot of check boxes Juga has made regarding making a Debian package[0]. Those should get checked. These are important. However, I think the absolute most important thing for us to be spending our time on right now is deciding what "good" results are and verifying sbws produces "good" results. To accomplish this, I think one of the two suggestions I made in a comment on GH#182 [1] (quoted here) is what we should be doing. 1. Run torflow and sbws side-by-side (but not at the same time) to remove more variables. This has the added benefit of us having access to the raw scanner results from torflow before it does whatever magic scaling it does. OR 2. Ask for access to raw scanner results from someone running torflow. I fear sbws is doomed to die the death of the new bandwidth scanners before it if we don't start seriously verifying sbws is "good" or if I personally slowly stop working/coordinating work on it. Thanks Matt [0]: https://trac.torproject.org/projects/tor/ticket/26848 [1]: https://github.com/pastly/simple-bw-scanner/issues/182#issuecomment-404250053 [2]: https://github.com/pastly/simple-bw-scanner/pull/236 ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
