Re: [tor-dev] Trouble with onionperf visualize and S61 performance experiments

2020-12-06 Thread Karsten Loesing
On 2020-12-02 20:23, George Kadianakis wrote: > Karsten Loesing writes: > >> On 2020-11-03 15:01, George Kadianakis wrote: >>> Hello Karsten, >> >> Hi George! >> >>> hope you are doing well! >>> >>> I've been working on the S6

Re: [tor-dev] Trouble with onionperf visualize and S61 performance experiments

2020-11-24 Thread Karsten Loesing
On 2020-11-23 16:49, George Kadianakis wrote: > Karsten Loesing writes: > >> On 2020-11-03 17:16, Karsten Loesing wrote: >>> On 2020-11-03 15:01, George Kadianakis wrote: >>>> Hello Karsten, >>> >>> Hi George! >> >> Hi agai

Re: [tor-dev] Trouble with onionperf visualize and S61 performance experiments

2020-11-23 Thread Karsten Loesing
On 2020-11-03 17:16, Karsten Loesing wrote: > On 2020-11-03 15:01, George Kadianakis wrote: >> Hello Karsten, > > Hi George! Hi again! >> hope you are doing well! >> >> I've been working on the S61 performance experiments [0] and I would >> appreciate &g

Re: [tor-dev] Trouble with onionperf visualize and S61 performance experiments

2020-11-03 Thread Karsten Loesing
On 2020-11-03 15:01, George Kadianakis wrote: > Hello Karsten, Hi George! > hope you are doing well! > > I've been working on the S61 performance experiments [0] and I would > appreciate > some help with onionperf. > > I have done various onionperf measurements using something the following

[tor-dev] Is anybody using OnionPerf's measurement results in the .tpf format?

2020-05-10 Thread Karsten Loesing
Hello list, if the subject line doesn't make any sense to you, this email is very likely irrelevant for you. But if it does make sense to you and you did use OnionPerf's [0] measurement in the .tpf format [1] in the past and/or plan to use this data in the future, you should comment on ticket

Re: [tor-dev] Proposal 313: Relay IPv6 Statistics

2020-02-10 Thread Karsten Loesing
On 2020-02-10 07:36, teor wrote: > Hi, Hi teor, I'm including some comments below. > Here is an initial draft of Proposal 313: Relay IPv6 Statistics. > > This proposal includes: > * logging the number of IPv6 relays in the consensus, and > * relays publishing IPv6 connection and consumed

[tor-dev] Changes to accessing and using MaxMind's databases

2020-01-09 Thread Karsten Loesing
Hi! When trying to update tor's geoip databases the other day I found that MaxMind's GeoLite2 database is not available for download anymore. The reason is: https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-using-geolite2-databases/ We do not have a MaxMind account (yet).

Re: [tor-dev] 24 hours worth of BridgeDB usage metrics

2019-07-30 Thread Karsten Loesing
On 2019-07-30 00:01, Philipp Winter wrote: > [...] > > After a cursory look at the numbers, I would like to aggregate the data, > to make it easier to compare distributors, transports, and countries. > For example: how do moat, email, and HTTPS rank in popularity? I'll > improve the patch to

Re: [tor-dev] Anti-censorship discussion with Briar devs

2019-02-25 Thread Karsten Loesing
On 2019-02-22 17:05, Cecylia Bocovich wrote: > [...] Thanks for summarizing that conversation here! > The main thing they would like on the metrics side is up-to-date > information about which PT works in which country and where PTs are > needed at all in order to make quick and easy decisions

Re: [tor-dev] Proposal 300: Walking Onions: Scaling and Saving Bandwidth

2019-02-06 Thread Karsten Loesing
On 2019-02-05 18:44, Michael Rogers wrote: > Argh, I'm really sorry, I thought I'd reached the end of the proposal > but my questions were addressed further down. Sorry for the noise. Hang on. The issues you mentioned are indeed addressed in the proposal, and there are also solutions given in the

[tor-dev] Scheduled changes to Tor Metrics CSV files in the Performance and the Traffic category

2018-12-06 Thread Karsten Loesing
Hi, if you're not pulling CSV files from the Tor Metrics website in an automated fashion, you can stop reading now. We just scheduled some changes to the Tor Metrics CSV files in the Performance and the Traffic category: - December 20, 2018 (scheduled): Remove source parameters and output rows

[tor-dev] [release] CollecTor 1.8.0

2018-10-15 Thread Karsten Loesing
Hi, everyone! A new release is available: https://dist.torproject.org/collector/1.8.0/ From the change log: * Medium changes - Properly clean up sanitized web server logs in the recent/ directory when they turn older than three days. * Minor changes - Once more, fix the bug in

[tor-dev] [release] Tor Metrics Library 2.5.0 and Tor Metrics Website 1.2.0

2018-09-26 Thread Karsten Loesing
Hi, we just released new versions of the Tor Metrics Library, metrics-lib 2.5.0, and Tor Metrics Website, metrics-web 1.2.0. The primary reason for these new releases is that libraries contained in the tarball match the ones needed to build current master. Downloads are available at:

Re: [tor-dev] Upcoming changes to CSV files provided by Tor Metrics

2018-09-14 Thread Karsten Loesing
Hello again, this is a quick reminder about changes to CSV files provided by Tor Metrics. On 2018-07-31 16:50, Karsten Loesing wrote: > Hello tor-dev@, > > today we finally published specifications [0] for the per-graph CSV > files that we added to Tor Metrics earli

Re: [tor-dev] Tor Volunteer

2018-09-09 Thread Karsten Loesing
Hi Milan, On 2018-09-07 16:35, Milan Damjanovic wrote: > No meeting today? No, we had to move the meeting to tomorrow, Monday, September 10: https://lists.torproject.org/pipermail/metrics-team/2018-September/000895.html

Re: [tor-dev] Tor Volunteer

2018-08-27 Thread Karsten Loesing
Hi Esaul and Milan, thanks for reaching out! And yes, with the background you two mention you might indeed be great fits for the metrics team. Going through the lists: - statistics: Here is an idea that I wrote down three weeks ago for another volunteer who might already have started with it,

Re: [tor-dev] [release] Onionoo 7.0 will be released after September 3, 2018

2018-08-09 Thread Karsten Loesing
On 2018-08-09 12:26, nusenu wrote: > Hello Karsten, Hi nusenu, >> yesterday we announced Onionoo protocol version 6.2 that we released a >> few days before on August 3, 2018. >> >> One month later, after September 3, 2018, we're going to release Onionoo >> protocol version 7.0 > > > does this

[tor-dev] [release] Onionoo 7.0 will be released after September 3, 2018

2018-08-09 Thread Karsten Loesing
Hello tor-dev@, yesterday we announced Onionoo protocol version 6.2 that we released a few days before on August 3, 2018. One month later, after September 3, 2018, we're going to release Onionoo protocol version 7.0 which will make a couple backward-incompatible changes: - extend the "version"

[tor-dev] Upcoming changes to CSV files provided by Tor Metrics

2018-07-31 Thread Karsten Loesing
Hello tor-dev@, today we finally published specifications [0] for the per-graph CSV files that we added to Tor Metrics earlier this year. At the same time we're announcing two upcoming changes to CSV files provided by Tor Metrics: - August 15: We're going to make the first batch of changes to

Re: [tor-dev] Proposal: Tor bandwidth measurements document format

2018-05-01 Thread Karsten Loesing
Hi Juga, On 2018-05-01 14:36, Nick Mathewson wrote: > This is a review of the document from > https://raw.githubusercontent.com/juga0/torspec/c7f06023dd1d5d47adad128de541f8eba2a13bfb/bandwidth-file-spec.txt > , which I *think* is the same as the document you have below. I'd like to review this

Re: [tor-dev] Closing old Trac Milestones

2018-04-30 Thread Karsten Loesing
On 2018-04-30 06:02, teor wrote: > Hi, Hi teor, > Just following up on the old trac milestones. > >> On 23 Apr 2018, at 14:31, teor wrote: >> >> Hi, >> >> Tor's trac has a lot of milestones. >> Sometimes users select the wrong one, because there are so many. > > Does

[tor-dev] New link on Tor Metrics: download data as CSV

2018-02-28 Thread Karsten Loesing
Hi tor-dev! Today we added a small, but potentially quite useful feature to Tor Metrics. The data that is displayed in a graph can now be downloaded as CSV file by clicking on "download data as CSV". This is different from before, where we provided the larger CSV file that a graph is based on.

Re: [tor-dev] onionoo: varnish 503

2018-02-05 Thread Karsten Loesing
On 2018-02-05 16:42, nusenu wrote: > https://onionoo.torproject.org/details?search=123 > >> Error 503 Backend fetch failed >> >> Backend fetch failed >> Guru Meditation: >> Varnish cache server > > > thanks for having a look Hmm, works just fine for me. How about we move this issue (if you

Re: [tor-dev] onionoo.tpo stuck at 2018-02-02 19:00

2018-02-03 Thread Karsten Loesing
On 2018-02-03 12:53, nusenu wrote: > > > Karsten Loesing: >> On 2018-02-03 01:32, nusenu wrote: >>> thanks for looking into it >> >> Looks like the CollecTor host is down, along with several other hosts. I >> sent mail to the admins. > >

Re: [tor-dev] onionoo.tpo stuck at 2018-02-02 19:00

2018-02-03 Thread Karsten Loesing
On 2018-02-03 01:32, nusenu wrote: > thanks for looking into it Looks like the CollecTor host is down, along with several other hosts. I sent mail to the admins. All the best, Karsten ___ tor-dev mailing list tor-dev@lists.torproject.org

Re: [tor-dev] onionoo.tpo stuck at 2018-01-21 22:00

2018-01-26 Thread Karsten Loesing
On 2018-01-26 11:34, nusenu wrote: >> What we can do, though, is think about providing more history in >> Onionoo, so that you can give up on archiving Onionoo data. > > It is nice of you to consider that but it is not necessary (at least for me) > I can life with my current hacks and other

Re: [tor-dev] onionoo.tpo stuck at 2018-01-21 22:00

2018-01-24 Thread Karsten Loesing
Hi nusenu, On 2018-01-22 18:57, nusenu wrote: >> Looks like the primary CollecTor instance had a problem between 22:00 >> and 08:00 UTC. It works again now, as does Onionoo. > > Karsten, thanks for the fast reaction. > >> We didn't lose any data, because the primary CollecTor instance obtained

Re: [tor-dev] onionoo.tpo stuck at 2018-01-21 22:00

2018-01-22 Thread Karsten Loesing
On 2018-01-22 09:03, nusenu wrote: > Hi Karsten, Hi nusenu, > just wanted to let you know that the delta between > relays_published and current time is unusually high. > > > https://onionoo.torproject.org/details?limit=0 > > {"version":"5.0", > "build_revision":"0bce98a", >

Re: [tor-dev] Tor Metrics Roadmap 2017/18

2017-11-17 Thread Karsten Loesing
On 2017-11-17 21:29, Georg Koppen wrote: > Karsten Loesing: >> Hello everyone, >> >> we, the Tor Metrics Team, have finished writing our roadmap for the 12 >> months between October 2017 and September 2018: >> >> https://trac.torproject.org/projects/tor/ra

[tor-dev] Tor Metrics Roadmap 2017/18

2017-11-17 Thread Karsten Loesing
Hello everyone, we, the Tor Metrics Team, have finished writing our roadmap for the 12 months between October 2017 and September 2018: https://trac.torproject.org/projects/tor/raw-attachment/wiki/org/teams/MetricsTeam/metrics-team-roadmap-2017-11-17.pdf

Re: [tor-dev] onionoo.tpo stuck at 2017-10-25 04:00?

2017-10-26 Thread Karsten Loesing
On 2017-10-25 23:51, nusenu wrote: > Hi Karsten, Hi nusenu, > just wanted to let you know that the delta between > relays_published and current time is unusually high. > > > https://onionoo.torproject.org/details?limit=0 > > {"version":"4.2", > "build_revision":"bce585f", >

[tor-dev] Your input on the Tor Metrics Roadmap 2017/18

2017-10-06 Thread Karsten Loesing
Hello everyone, we, the Tor Metrics Team, are currently drafting a roadmap for the work we'd like to do on in the upcoming 12 months until September 2018. If you believe that you're affected by these plans and want your ideas to be included in this roadmap, please read our current draft and send

Re: [tor-dev] Tor and IP2Location LITE

2017-08-20 Thread Karsten Loesing
On 2017-08-16 21:19, Karsten Loesing wrote: > On 2017-08-16 05:38, KL Liew wrote: >> All, > > Hi Kim, > >> My name is Kim, the founder of IP2Location, a geolocation service >> provider since 2002. >> >> It looks like Tor is looking to review ot

Re: [tor-dev] Tor and IP2Location LITE

2017-08-16 Thread Karsten Loesing
On 2017-08-16 05:38, KL Liew wrote: > All, Hi Kim, > My name is Kim, the founder of IP2Location, a geolocation service > provider since 2002. > > It looks like Tor is looking to review other providers for GeoIP service > while I was reading one of a meeting minute for a meeting back in March >

Re: [tor-dev] onionoo.tpo stuck at 2017-06-12 12:00?

2017-06-13 Thread Karsten Loesing
On 2017-06-13 08:54, Karsten Loesing wrote: > On 2017-06-13 00:03, nusenu wrote: >> Hi, >> >> just wanted to let you know that the delta between >> >> relays_published and current time is unusually high. >> >> >> https://onionoo.t

Re: [tor-dev] onionoo.tpo stuck at 2017-06-12 12:00?

2017-06-13 Thread Karsten Loesing
On 2017-06-13 00:03, nusenu wrote: > Hi, > > just wanted to let you know that the delta between > > relays_published and current time is unusually high. > > > https://onionoo.torproject.org/details?limit=0 > > {"version":"4.0", > "relays_published":"2017-06-12 12:00:00", Right, I noticed the

Re: [tor-dev] [release] metrics-lib/DescripTor 1.8.0

2017-06-07 Thread Karsten Loesing
On 07.06.17 12:31, Alan S wrote: > Remove me from this list!! I get emails from here and I never signed up! There's a link at the bottom of each message that you can use to unsubscribe. > On Jun 7, 2017 4:28 AM, "iwakeh" wrote: > > Hi there! > > The first June release

Re: [tor-dev] onionoo.tpo hardly reachable

2017-05-24 Thread Karsten Loesing
On 23.05.17 21:48, nusenu wrote: >> we're working on resolving the load issues of the main server and its >> two caching frontends, which really get hammered since last week for no >> obvious reason. > > I wanted to compare atlas weblogs with onionoo data but didn't find any > onionoo data on

Re: [tor-dev] onionoo.tpo hardly reachable

2017-05-23 Thread Karsten Loesing
On 22.05.17 23:11, nusenu wrote: >>> Patrick O'Doherty: Is this something that an org such as noisetor could help out with by contributing a public mirror? We'd be more than happy to do so if it'd be useful to the project/community. >>> >>> Since the previous mirror is gone and

Re: [tor-dev] onionoo.tpo hardly reachable

2017-05-19 Thread Karsten Loesing
On 19.05.17 09:52, nusenu wrote: > Hi Karsten, Hi nusenu, > onionoo is hardly reachable since about 17 hours ago. Indeed. For some reason Onionoo saw an increase in requests from 700 to 4000 per second 24 hours or so ago. There's now a third host answering Onionoo requests from a local cache,

[tor-dev] Simulating how Laplace noise added to directory-request statistics would affect user number estimates

2017-05-19 Thread Karsten Loesing
Hi everyone, we'd like to improve directory-request statistics by obfuscating values on relays before they are reported to the directory authorities. A possible obfuscation method is to add Laplace noise to request counts for all ~250 countries, so that it's unclear whether a request was

Re: [tor-dev] Anonymous Local Count Statistics Using PCSA - GSoC

2017-04-02 Thread Karsten Loesing
On 02.04.17 15:22, Aaron Johnson wrote: > Also, I think that counting users by IP is still a fine way to do it (absent > the privacy issue that PCSA tries to address). I was just stating that my > understanding based on talking to the Tor Metrics people is that the plan is > to handle the

[tor-dev] Open tickets

2017-03-13 Thread Karsten Loesing
Hi everyone, I'm working on some statistics on open tickets of the metrics team in preparation of the Amsterdam meeting, and as part of that I made the following graph on open Trac tickets for all components (except Archived/*): https://people.torproject.org/~karsten/volatile/open-tickets.pdf

Re: [tor-dev] ExitPortStatistics interpretation

2017-01-30 Thread Karsten Loesing
On 29/01/17 17:54, nusenu wrote: > > > Karsten Loesing: >>> Oh thanks, so it is not possible to find out which is the most frequent >>> exit port by number of streams opened, that's a pity. >> Well, that one is easy: port 80. :) > > Ok, maybe I shou

Re: [tor-dev] ExitPortStatistics interpretation

2017-01-29 Thread Karsten Loesing
On 29/01/17 17:12, nusenu wrote: > Karsten Loesing: >> Those are the 10 ports with the highest number of (written and read) >> bytes, unrelated to the number of stream. And all lines below report >> statistics for these 10 ports plus "other". > > Oh than

Re: [tor-dev] ExitPortStatistics interpretation

2017-01-29 Thread Karsten Loesing
On 29/01/17 13:56, nusenu wrote: > Hi, Hi nusenu, > I'm looking at ExitPortStatistics. Since the spec [1] is not very > specific, I wanted to confirm that my assumption is correct: > > The current tor implementation includes the 10 most relevant ports, > correct? (highest number of bytes or

Re: [tor-dev] onionoo: understanding 'exit_policy_v6_summary'

2017-01-29 Thread Karsten Loesing
On 28/01/17 00:07, nusenu wrote: > Hi Karsten, Hi nusenu, > given this torrc configuration and exit policy: > https://lists.torproject.org/pipermail/tor-relays/2017-January/011806.html > > would you expect onionoo's 'exit_policy_v6_summary' to be not set? [1] > >

Re: [tor-dev] onionoo.tpo stuck at 2017-01-27 13:00

2017-01-29 Thread Karsten Loesing
On 28/01/17 10:23, Karsten Loesing wrote: > On 28/01/17 10:06, nusenu wrote: >> >> >> Karsten Loesing: >>> If you notice similar problems in the future, be sure to let us know! >>> We do have a few checks in place, but this issue slipped through >>

Re: [tor-dev] onionoo.tpo stuck at 2017-01-27 13:00

2017-01-28 Thread Karsten Loesing
On 28/01/17 10:06, nusenu wrote: > > > Karsten Loesing: >> If you notice similar problems in the future, be sure to let us know! >> We do have a few checks in place, but this issue slipped through >> somehow. > > > I assume you are already aware that onion

Re: [tor-dev] Removing Countries with Low Counts from Metrics

2016-11-07 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 02/11/16 07:38, teor wrote: > Hi, > > Have we considered removing countries with low counts from > metrics? (This probably means removing the metrics at the relays, > not just the graphs.) > > For example, two places that are close to home for

Re: [tor-dev] onionoo.tpo stuck at 2016-10-30 23:00:00

2016-11-01 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi nusenu, should be resolved now. Turns out one of the CollecTor modules silently died. It's back now. If you notice similar problems in the future, be sure to let us know! We do have a few checks in place, but this issue slipped through

[tor-dev] metrics-lib/DescripTor 1.5.0 is released

2016-10-20 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hello devs, we just released metrics-lib/DescripTor 1.5.0: https://dist.torproject.org/descriptor/1.5.0/ - From the change log: # Changes in version 1.5.0 - 2016-10-19 * Major changes - Make the DescriptorCollector implementation that uses

Re: [tor-dev] Tor Browser downloads and updates graphs

2016-10-09 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 22/09/16 01:48, Aaron Johnson wrote: Oops, this thread got lost in the Seattle preparations and only surfaced today while doing some housekeeping. Please find my response below. >> Log files are sorted as part of the sanitizing procedure, so

Re: [tor-dev] Tor Browser downloads and updates graphs

2016-09-22 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 20/09/16 17:46, Lunar wrote: > Karsten Loesing: >>> If you feel that's interesting enough, would it be possible to >>> also add the number of download of cryptographic signatures >>> to the graph?

Re: [tor-dev] Tor Browser downloads and updates graphs

2016-09-21 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Aaron, On 20/09/16 15:43, Aaron Johnson wrote: >> >> Good thinking! I summarized the methodology on the graph page >> as: The graph above is based on sanitized Tor web server logs >> [0]. These are a stripped-down version of Apache's "combined"

[tor-dev] More changes to CollecTor's sanitized bridge descriptors: sanitized TCP ports, tarballs per descriptor type

2016-09-19 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi devs, lots of changes to CollecTor's sanitized bridge descriptors these weeks. A few weeks ago there was the switch from Tonga to Bifroest change: https://lists.torproject.org/pipermail/tor-dev/2016-August/011336.html This message is about two

Re: [tor-dev] Tor Browser downloads and updates graphs

2016-09-19 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 17/09/16 18:28, Aaron Johnson wrote: >>> Here's >>> >>> the same graph with more data, more request types, and of >>> course a lot more shininess: >>> >>> https://tor-metrics.shinyapps.io/webstats/ >>>

Re: [tor-dev] Tor Browser downloads and updates graphs

2016-09-19 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 17/09/16 17:52, Lunar wrote: > Karsten Loesing: >> On 11/09/16 18:13, Georg Koppen wrote: >>> Here are the graphs showing initial downloads, update pings and >>> update requests over time: >> >>> https

Re: [tor-dev] onionoo: poor reverse DNS results

2016-09-02 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi nusenu, On 01/09/16 22:43, nusenu wrote: >> some time ago I reported onionoo's poor reverse DNS results >> https://trac.torproject.org/projects/tor/ticket/18342 and it >> didn't change since then. >> >> As of 2016-07-02 08:00 (tpo instance) 3144

[tor-dev] metrics-lib/DescripTor 1.4.0 is released

2016-08-31 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello devs, we just released metrics-lib/DescripTor 1.4.0: https://dist.torproject.org/descriptor/1.4.0/ - From the change log: # Changes in version 1.4.0 - 2016-08-31 * Major changes - Add the Simple Logging Facade for Java (slf4j) for

[tor-dev] Bifroest's sanitized bridge descriptors available from CollecTor in 48 hours from now

2016-08-29 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi devs, if you're not using CollecTor's sanitized bridge descriptors for anything, you can safely ignore this message. As you may know, there are currently two bridge directory authorities in the Tor network, Tonga and Bifroest, and Tonga will go

[tor-dev] CollecTor will soon serve descriptors that don't pass metrics-lib's parser

2016-08-05 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello CollecTor data consumers, note: if you don't know what CollecTor is or if you have never used CollecTor data before, you can safely stop reading now and enjoy your Friday. But in case you're now curious what CollecTor is, feel free to take a

[tor-dev] metrics-lib/DescripTor 1.3.1 is released

2016-08-01 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello devs, we just released metrics-lib/DescripTor 1.3.1: https://dist.torproject.org/descriptor/1.3.1/ - From the change log: # Changes in version 1.3.1 - 2016-08-01 * Medium changes - Adapt to CollecTor's new date format to make

[tor-dev] metrics-lib/DescripTor 1.3.0 is released

2016-07-07 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello devs, we just released metrics-lib/DescripTor 1.3.0: https://dist.torproject.org/descriptor/1.3.0/ - From the change log: # Changes in version 1.3.0 - 2016-07-06 * Medium changes - Parse "package" lines in consensuses and votes. -

Re: [tor-dev] DescripTor 1.2.0 is released

2016-06-06 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 06/06/16 14:28, tl wrote: > Hi, > >> On 31.05.2016, at 21:17, Karsten Loesing <kars...@torproject.org> >> wrote: >> >> Signed PGP part Hello devs, >> >> I just released DescripTor 1.2.0: >

Re: [tor-dev] onionoo.tc stuck at 2016-05-26 00:00

2016-06-05 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 31/05/16 14:21, Karsten Loesing wrote: > On 31/05/16 11:24, nusenu wrote: >> Hi Thomas, > > Hi nusenu, > >> I'm using your onionoo instance to fetch details documents. > > (Thomas is kindly providing the host, b

Re: [tor-dev] DescripTor 1.2.0 is released

2016-06-02 Thread Karsten Loesing
split up metrics-lib at this point. All the best, Karsten > Cheers! -Damian > > > On Tue, May 31, 2016 at 12:17 PM, Karsten Loesing > <kars...@torproject.org> wrote: Hello devs, > > I just released DescripTor 1.2.0: > > https://dist.torproject.org/descriptor/1.2.0/ >

Re: [tor-dev] onionoo.tc stuck at 2016-05-26 00:00

2016-05-31 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 31/05/16 11:24, nusenu wrote: > Hi Thomas, Hi nusenu, > I'm using your onionoo instance to fetch details documents. (Thomas is kindly providing the host, but I'm operating the Onionoo instance on that.) > Currently it is stuck at 2016-05-26

[tor-dev] CollecTor has a new Git home

2016-04-24 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi devs, the CollecTor Git repository just moved from metrics-db to collector: https://gitweb.torproject.org/collector.git/ The old name made sense when the Tor network data archive was still part of Metrics, but it's rather confusing since

Re: [tor-dev] Should we disable the collection of some stats published in extra-infos?

2016-03-28 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 25/03/16 16:24, Rob Jansen wrote: > >> On Feb 11, 2016, at 2:51 PM, Rob Jansen >> wrote: >> >>> These statistics are being collected for years now, and it >>> might take another year or so for relays to upgrade to stop

Re: [tor-dev] Two major improvements to the Tor Metrics back-end

2016-02-15 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/01/16 18:05, Karsten Loesing wrote: > Hello list, > > as some of you may have noticed, Tor Metrics had some trouble > updating its aggregate data lately, and the issues there are by no > means completely gone now. > > B

Re: [tor-dev] Monitoring Service

2016-02-02 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Tom, On 27/01/16 20:02, Tom Ritter wrote: > [feel free to reply adding tor-project or whomever] Sure, let me copy tor-dev@. > Remember a while ago I lamented that I wished there was some > monitoring service that could tell me when my metrics

Re: [tor-dev] Should we disable the collection of some stats published in extra-infos?

2016-01-19 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 15/01/16 23:00, Rob Jansen wrote: > Hello, Hi Rob, I'm moving this discussion from metrics-team@ to tor-dev@, because I think it's relevant for little-t-tor devs who are not subscribed to metrics-team@. Hope you don't mind. > I was recently

Re: [tor-dev] Upcoming Onionoo version 3.1 removes "family" field from details and stops returning code 500 when serving stale data

2016-01-18 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/01/16 10:25, Karsten Loesing wrote: > Hello Onionoo users, > > the upcoming Onionoo [0] version 3.1 will contain two minor > changes that are worth announcing in advance: > > - Details documents will not contain a &quo

Re: [tor-dev] Comparing Stem, metrics-lib, and zoossh

2016-01-15 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 14/01/16 17:22, Damian Johnson wrote: > Oh, forgot to talk about compression. You can run the stem script > against compressed tarballs but python didn't add lzma support > until python 3.3... > >

Re: [tor-dev] Comparing Stem, metrics-lib, and zoossh

2016-01-14 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 13/01/16 20:42, Damian Johnson wrote: >> This was Stem commit c01a9cda4e7699c7f4bd642c8e81ed45aab7a29b >> and Python version 2.7.10. > > Great, thanks! Also what was the metrics-lib and zoossh commits? metrics-lib:

Re: [tor-dev] Comparing Stem, metrics-lib, and zoossh

2016-01-14 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 13/01/16 21:01, Philipp Winter wrote: > On Wed, Jan 13, 2016 at 05:47:03PM +0100, Karsten Loesing wrote: >> Do the Zoossh results there look plausible? > > I'm surprised that descriptor parsing is so slow, but I think the > res

Re: [tor-dev] Comparing Stem, metrics-lib, and zoossh

2016-01-13 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/01/16 17:19, Damian Johnson wrote: > Thanks! Yup, those results look reasonable. I was expecting a > smaller delta with server/extrainfo descriptors and larger one > with microdescriptors due to the lazy loading but oh well. What > stem commit

Re: [tor-dev] Comparing Stem, metrics-lib, and zoossh

2016-01-13 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 13/01/16 16:28, Philipp Winter wrote: > On Tue, Jan 12, 2016 at 09:40:35AM +0100, Karsten Loesing wrote: >> Philipp, would you be able to write the Zoossh counterpart for >> the descriptor types supported by it? > > I

Re: [tor-dev] Comparing Stem, metrics-lib, and zoossh

2016-01-12 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/01/16 05:38, Damian Johnson wrote: > Hi Karsten, implemented Stem counterparts of these (see attached). > On one hand the code is delightfully simple, but on the other > measurements I got were quite a bit slower. Curious to see what > you get

[tor-dev] Upcoming Onionoo version 3.1 removes "family" field from details and stops returning code 500 when serving stale data

2016-01-11 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello Onionoo users, the upcoming Onionoo [0] version 3.1 will contain two minor changes that are worth announcing in advance: - Details documents will not contain a "family" field anymore but instead two fields "alleged_family" and

Re: [tor-dev] Tor metrics: Questionable relays-by-platform accounting

2016-01-09 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 08/01/16 17:31, Fabian Keil wrote: > Karsten Loesing <kars...@torproject.org> wrote: >> Should I proceed with my earlier suggestion to change "FreeBSD" >> to "BSD" in the legend? > > Yes, please. I ag

Re: [tor-dev] Tor metrics: Questionable relays-by-platform accounting

2016-01-08 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/01/16 18:48, Fabian Keil wrote: > Looking at "Tor metrics - Relays by platform" [0] one could think > that the number of "FreeBSD" relays is only slightly lower than the > number of "Windows" relays. > > It also looks like the number of

Re: [tor-dev] Tor metrics: Questionable relays-by-platform accounting

2016-01-08 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 08/01/16 16:54, Fabian Keil wrote: > Karsten Loesing <kars...@torproject.org> wrote: > >> On 07/01/16 18:48, Fabian Keil wrote: >>> Looking at "Tor metrics - Relays by platform" [0] one could >>> thi

Re: [tor-dev] Two major improvements to the Tor Metrics back-end

2016-01-07 Thread Karsten Loesing
> Aaron > >> On Jan 7, 2016, at 12:25 PM, David Goulet <dgou...@ev0ke.net> >> wrote: >> >> On 07 Jan (18:05:09), Karsten Loesing wrote: >>> Hello list, >>> >>> as some of you may have noticed, Tor Metrics had some trouble >>> updati

Re: [tor-dev] Comparing Stem, metrics-lib, and zoossh

2016-01-07 Thread Karsten Loesing
ults! All the best, Karsten > > Cheers! -Damian > > > On Sun, Jan 3, 2016 at 9:56 AM, Karsten Loesing > <kars...@torproject.org> wrote: Hi Damian, > > I'm digging out this old thread, because I think it's still > relevant. > > I started writing some pe

Re: [tor-dev] Comparing Stem, metrics-lib, and zoossh

2016-01-03 Thread Karsten Loesing
?) All the best, Karsten On 01/10/15 09:28, Karsten Loesing wrote: > Hello Philipp and iwakeh, hello list, > > Damian and I sat down yesterday at the dev meeting to talk about > doing a comparison of the various descriptor-parsing libraries with > respect to capabilities, run-t

[tor-dev] metrics-lib 1.1.0 is released

2015-12-28 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello devs, I just released metrics-lib 1.1.0: https://dist.torproject.org/descriptor/1.1.0/ - From the change log: # Changes in version 1.1.0 - 2015-12-28 * Medium changes - Parse flag thresholds in bridge network statuses, and parse the

Re: [tor-dev] Scaling Tor Metrics, Round 2

2015-12-12 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/12/15 18:18, Spencer wrote: > Hi, Hello Spencer, >> Karsten Loesing: I'm not really worried about server load. At >> least this hasn't been an issue with the current Metrics >> website. >> > > Word.

Re: [tor-dev] Scaling Tor Metrics, Round 2

2015-12-09 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/12/15 04:01, Spencer wrote: > Hi, > >> >> Karsten Loesing: We briefly discussed making a JavaScript-free >> Globe a while ago by using Node.js. I'm not sure whether this >> would also work for Metrics. It

Re: [tor-dev] Scaling Tor Metrics, Round 2

2015-12-09 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/12/15 17:31, David Goulet wrote: > On 06 Dec (16:52:45), Karsten Loesing wrote: >> Hi everyone, > > [snip] > >> One important and still somewhat low-hanging fruit is: #10 give >> external developers more support

Re: [tor-dev] Scaling Tor Metrics, Round 2

2015-12-07 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/12/15 01:07, Spencer wrote: > Hi, >> teor: Do David's visualizations already use JavaScript? We could >> make (another) part of the metrics site use JavaScript. >> > > Can the data be processed on the host server and sent to the > client

Re: [tor-dev] Scaling Tor Metrics, Round 2

2015-12-07 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/12/15 12:10, Tim Wilson-Brown - teor wrote: > >> On 7 Dec 2015, at 19:14, Karsten Loesing <kars...@torproject.org> >> wrote: >> >> On 07/12/15 01:07, Spencer wrote: >>> Hi, >>>> teor

[tor-dev] Scaling Tor Metrics, Round 2

2015-12-06 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi everyone, I posted some thoughts on Scaling Tor Metrics [0] almost two weeks ago and received very useful feedback from George, David, Thomas, and Letty. Thanks for that! In fact, this is such a big topic and this was so much feedback that I

[tor-dev] metrics-lib 1.0.0 is released

2015-12-05 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello devs, I just released metrics-lib 1.0.0: https://dist.torproject.org/descriptor/1.0.0/ metrics-lib, which also goes by the name DescripTor, is a Java API that fetches Tor descriptors from a variety of sources like cached descriptors and

[tor-dev] Scaling Tor Metrics

2015-11-25 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello devs, the Tor Metrics website [0] claims to be "the primary place to learn interesting facts about the Tor network" and invites its visitors who "come across something that is missing" to contact the website authors about it. That's a bold

Re: [tor-dev] Upcoming Onionoo version 3.0 will support searches by space-separated fingerprint

2015-11-15 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 14/10/15 12:46, Karsten Loesing wrote: > Hello Onionoo client developers, > > the upcoming Onionoo version 3.0 will support searches by > space-separated fingerprint. > > To give you an example, the following two searches

[tor-dev] NO 1-1-1 task exhange meeting on Thursday, Nov 12

2015-11-11 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello devs, I decided to stop the experiment of organizing 1-1-1 task exchange meetings for everyone in Tor, because there was too little interest in having these meetings. If you have no idea what I'm talking about, here's the wiki page:

[tor-dev] 1-1-1 task exchange meeting on Thursday, Nov 5, 15:00 UTC in #tor-dev

2015-11-04 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello devs, we'll have another 1-1-1 task exchange meeting on Thursday, November 5, 2015, 15:00 UTC in #tor-dev https://www.timeanddate.com/worldclock/fixedtime.html?iso=20151105T15 Quick reminder: The idea of the 1-1-1 task exchange meeting is

[tor-dev] An ANTLR 4 grammar for Tor bridge network statuses

2015-11-04 Thread Karsten Loesing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello developers, in the past few days I have been working on a grammar to parse Tor bridge network statuses and hopefully other Tor descriptors in the future. It's working, for some definition of working, but some issues remain and I need some

  1   2   3   4   5   >