> On Jul 6, 2017, at 2:58 PM, Andreas Koenig > <andreas.koenig.7os6v...@franz.ak.mind.de> wrote: > >>>>>> On Thu, 6 Jul 2017 00:47:19 -0500, Doug Bell <d...@preaction.me> said: > >> Slaven would have to answer definitively, but fast-matrix isn't >> necessarily guaranteed to be accurate. It's not supposed to be. It's >> supposed to be fast. Matrix uses the fully-processed CPAN Testers >> data, which has been built to ensure that no data is lost between the >> Metabase and the website. Fast-matrix polls an API that displays the >> last 1000 reports (which opens it up to potentially missing data if a >> lot of data comes in too quickly). There's also the possibility that >> fast-matrix validates SSL to the Metabase and couldn't connect while >> the SSL cert for metabase.cpantesters.org was expired. > >> In short, it sounds normal to me, but Slaven would have to answer >> definitively. > > Don't underestimate the amount of information you have about the > involved systems, Doug. I'm saying this as a comaintainer of the fast > matrix. But I may have missed some relevant information. Please bring us > all up to date. > > For the record: in the months March-May the fast matrix was the only > source of uptodate information because cpantesters was not able to keep > up with the incoming reports and thus the matrix was very much behind > all the time. > > Then came the Lyon PTS (Perl Toolchain Summit) and things got repaired > and you opened a second address where reports could get delivered. > People were testing this second address even with feedback going through > this mailing list but since then I'm not sure what the status is. Is > cpantesters collecting reports at one or two or even more addresses now? > And if so, how can they be tracked from outside to provide the amount of > data the fast matrix needs to stay operational? > > Fast matrix only tracks http://metabase.cpantesters.org/tail/log.txt at > the moment. Does this need to change?
Right, I should update people on that, but short version is: Everything seems to work but I haven't migrated all the backend processing to the new data format, which is the last piece of the puzzle. The API I had everyone test was a backwards-compatibility API. We will gather data at multiple addresses, yes, but they all will feed to the same database. It's important that the fast-matrix stays operational, so the new API has a log.txt view same as the old API. The new log.txt reads from the shared database so that no matter how the data comes in, the fast-matrix will be able to pick it up. It's still the raw, unprocessed data, so any problems with backend processing will not hold up the fast-matrix data feed. (this is it, and it's slow, but it will be behind a Fastly cache and I'll do other things if necessary: http://metabase-beta.cpantesters.org/tail/log.txt <http://metabase-beta.cpantesters.org/tail/log.txt>) (but, bonus: The new API means that the full text of the report is available as soon as it's submitted, meaning users of the fast-matrix can read the full text of the reports even if the CPAN Testers backend hasn't fully processed it yet)
