In April, we started investigating a strange traceroute result reported by one of the RIPE Atlas anchors. The conclusion of our investigation is that the probe upgrade process for the v3 anchor firmware has been leaving old measurement daemons running as well as starting new ones.
This means that a gradually increasing number of measurements from the affected anchors have been performed multiple times more or less simultaneously. Because those processes were writing to the same file, this resulted in a lot of garbled results. Purely by chance, the weird result that triggered the investigation had the first part of one result and the last part of another result in a way that generated syntactically correct JSON. The net result is that for the last few months, measurements done by anchors may have resulted in multiple results at roughly the same time or no results because the results were rejected as invalid. For those who wish to check whether results for specific measurements were affected, the following links will direct you to lists of probe IDs for the affected anchors: https://atlas.ripe.net/api/v2/probes/?is_anchor=true&tags=system-virtual&fields=id (VM anchors) https://atlas.ripe.net/api/v2/probes/?is_anchor=true&tags=system-v3-pc-engines&fields=id (v3 hardware anchors) We are working to resolve the issue and will keep you updated as and when we have more to report. We apologise for any inconvenience this might cause. Philip
