Andrew Gallagher dijo [Thu, May 13, 2021 at 12:34:20PM +0100]: > > The first thing to understand what I'm plotting are the many graphs > > that are _not_ present in the summary page: Scroll down to the table > > that has many dates. Clicking on any such date will show you a walk of > > gossiping servers, starting from basically a random server in the > > network (I pick up whatever answers to pool.sks-keyservers.net - > > That's why you will see many entries with zero or very few nodes: it > > means I was unlucky with the resolution for a given day). The (quite > > ugly and ad-hoc) source for those graphs is at: > > > > http://sks-status.gwolf.org/walk_sks.rb > > It might be worth exploring whether starting with more than one initial node > would help your luck - usually ha.pool.sks-keyservers.net, na.pool, eu.pool > resolve to different servers and at least one of those should be in good > shape at any given time.
I guess it would! It is... an improvement I'll save for later, though. I don't want to put too much (more) work into it, as I only get bad connections' "noise" in about 1/10 of the tries. But yes, that would surely help! > > Do note that at the beginning I sampled the network much more often > > (hourly). I decided this was too much, and since June 2019, am now > > walking the network only every four hours. I do hope nobody sees this > > as excessive! > > Given that fastly and happy-eyeballs spider the network several times a > minute, I can't imagine anyone would have a problem with your hourly > extravagance. :-D That's very good to know :-) > > This shows the very large drop the SKS network had in mid-2019, as > > well as its behavior since then. I am happy, even hopeful, to note > > that it seems the network hit reliability minimums between October > > 2020 and February 2021, but it seems there is a slight trend for > > improvement, at least back to the late-2019 levels. > > I think we can attribute some of that to Casey's work on Hockeypuck which > has downgraded the poison key issue from a killer to an annoyance. Great! Well, it's very good to have a data point for this. > > Please do tell me if this data sounds interesting, and if you can > > thing of anything to improve on what I'm doing. Of course, I cannot > > apply any changes to already-collected data, but there are surely > > many other things that can be considered. > > This is very interesting work, thank you. I find the connectivity graphs > hard to make sense of though as the nodes tend to be drawn on top of each > other. A quick google hasn't thrown up any answers but I'm sure there's a > way to address this by imposing some minimum separation between nodes. The > graph browser we include in our software in $WORK has the ability to do > this, but I need to check the details before recommending it, as I'm not > sure if it has a noninteractive mode. Right. Well, I'm now also producing links to the source Graphviz files, it might be easier for you to produce something easier to view. > Some simple changes that I would otherwise suggest: > > 1. produce a connectivity graph with only working nodes Added, as "Success" graph. > 2. ignore localhost and private IPs, they will never work :-) Of course. But still, it documents a given server works in a clustered way. Of course they will fail, but who cares? ;- > 3. should it not default to port 11371 and fall back to 80? > 4. don't plot *.pool.sks-keyservers.net in the graph Right... I could add this bit later on, but then again, it does not clutter the graph much. AFAICT, It is only a single, starting node, and no nodes ever point back to it, right? Erm, no, _not_ matching reality. There are several servers as a destination... > Some more complex issues: > > It seems that your Hockeypuck status parsing is broken, as a significant > number of Hockeypuck nodes are listed consistently as Nil. It's probably due > to assumptions you're making about the DOM in Hpricot. Thankfully, > Hockeypuck emits a JSON stats page at 'pks/lookup?op=stats&options=mr' . If > you get Nil from Hpricot, maybe you could fall back to the JSON? Beware that > if you request the JSON page from SKS it will give you an HTML stats page > regardless. > > Also, the peers of some Hockeypuck nodes are throwing InvalidURIError - I > suspect this is because of an inconsistency between SKS and Hockeypuck in > how peers are reported - SKS says "host port", while Hockeypuck by default > uses "host:port", but beware that many Hockeypuck operators have changed > this back to SKS format so that the pool spider can parse the peer tree. > > Unfortunately, it's not as simple as splitting on /(\s+|:)/ as this will > bork on ipv6 addresses. I'd suggest something like the following: > > host, port = server.split(/\s+/) > if port == '' > host, port = host.split(/:/) > end Ooohhhh! Important and interesting. I have to look into how to deal with it, specially if the number of Hockeypuck servers grows! I cannot put more time into this today, but will look back into it.
signature.asc
Description: PGP signature
