Alright, so my e-mails to the list get delayed, so right after I received my
own reply to the list, I have the answer: Since we moved to the new database
(approximately October 30), the backend process that generates the report pages
has been timing out on the MySQL server:
2016/10/30 01:05:25 Creating cpanstats matrix files
DBD::mysql::st execute failed: Lost connection to MySQL server during query at
/usr/local/share/perl/5.20.2/CPAN/Testers/Common/DBUtils.pm line 176.
err=Lost connection to MySQL server during query, sql=[ SELECT
osname,count(*) AS count
FROM cpanstats
WHERE dist=? AND version=?
GROUP BY osname], args[DynGig-Multiplex,1.03] at
lib/CPAN/Testers/WWW/Statistics/Pages.pm line 1194.
Cannot connect to DB [dbi:mysql:database=cpanstats;host=216.246.80.45]: DBI
connect('database=cpanstats;host=216.246.80.45','cpantesters',...) failed:
Can't connect to MySQL server on '216.246.80.45' (111) at
/usr/local/share/perl/5.20.2/CPAN/Testers/Common/DBUtils.pm line 538.
at lib/CPAN/Testers/WWW/Statistics.pm line 107.
I've upped the timeout from 30s to 60s, but this is a 10+ hour process, so I
likely won't know if that worked until tomorrow (at which point I'll up all the
timeouts to 3600s because I'm not doing this more than twice and Zach (the
other DB admin) can hate me all he wants). I'm going to attempt to build in a
bit more robustness so that the process can lose the DB and still continue, but
I'm not sure how far I'll get. Also, I need to figure out if I need to go back
in time and re-build the data from an earlier date (which might make this whole
repair process take even longer).
That's the bad news. The good news is that all the data is there in the
database, so it's just a matter of running the appropriate build process(es)
and shepherding them to completion.
mysql> select count(*) from cpanstats where dist="Test-Simple" &&
version='1.302075';
+----------+
| count(*) |
+----------+
| 904 |
+----------+
1 row in set (0.08 sec)
If y'all want to make a decision based on data while I'm working on fixing
things, I can get you a CSV or something. Also, the API
(http://api.cpantesters.org <http://api.cpantesters.org/>) avoids the website
build process entirely and talks directly to the database, so if someone wants
to use that they can (and, if they need, enhance it to give the data you need).
Doug Bell
d...@preaction.me
> On Feb 5, 2017, at 5:06 PM, Doug Bell <d...@preaction.me> wrote:
>
> I'll take a look at this. Usually it's just a few hours' delay, but you all
> are saying it's much, much more than that.
>
> Also, if anybody would like the credentials to be able to diagnose, fix, and
> document issues like this for the future, I can set you up with access to the
> account and the Github repo where I've been keeping operations documentation.
>
> Doug Bell
> d...@preaction.me
>
>
>
>> On Feb 5, 2017, at 7:45 AM, James E Keenan <jk...@verizon.net> wrote:
>>
>> On 02/05/2017 04:41 AM, Slaven Rezic wrote:
>> [snip]
>>
>>>
>>> To see the up-to-date test coverage you can switch to the "fast" matrix
>>> view --- this is the "Matrix via log.txt" link, the one with the tiny
>>> font. The Test-Simple coverage looks quite good, see
>>> http://fast-matrix.cpantesters.org/?dist=Test-Simple%201.302075
>>>
>>> Unfortunately, in the fast matrix view we see only the test results, but
>>> there are no links to the detailed test reports available.
>>>
>>> It seems that updates into the cpantesters database are delayed. On
>>> http://www.cpantesters.org/distro/T/Test-Simple.html I see the note
>>> "Please note that this page is in an update queue, and as such the
>>> latest data may not be currently available."
>>
>>
>> Yes, I'm now quite familiar with the distinction between the matrix- and
>> fast-matrix- views and the timelag between them. But ...
>>
>> However, as the stable
>>> Test-Simple release happened almost one month ago, I suspect that there
>>> could be another problem than just delayed processing.
>>>
>>
>> Yes, that's exactly what I wondered about. There's a much greater
>> discrepancy than normal between:
>>
>> http://matrix.cpantesters.org/?dist=Test-Simple%201.302075
>>
>> and
>>
>> http://fast-matrix.cpantesters.org/?dist=Test-Simple%201.302075
>>
>> And for the purpose of Blead-Breaks-CPAN, we really need the actual test
>> report in matrix-.
>>
>> Thank you very much.
>> Jim Keenan
