Good morning Dan, Thanks very much for the explanation. Is there a Phabricator task we can upvote (award a token?) to make this issue more visible?
As always, we really appreciate your help with this. Best, Amanda On Tue, Feb 17, 2015 at 7:20 AM, Dan Andreescu <[email protected]> wrote: > Sorry for the trouble, Amanda. The problem is solely with the underlying > database, which we don't maintain. It's a sanitized replica of all the > changes being made to all the wikis so it's a fairly complicated piece of > infrastructure that sometimes has problems. The folks who maintain it are > aware of the issues, but we'll continue representing them until they're > solved. > > On Mon, Feb 16, 2015 at 3:49 PM, Amanda Bittaker <[email protected]> > wrote: > >> Oop, thanks for the ping, Nuria. Wikimetrics seems to be working better >> now. I still get failures, especially when running three or four reports >> in one batch, but the reports work if you rerun them (sometimes a couple >> times.) >> >> I'm still getting "PENDING"s that turn into "FAILURE"s sometimes, which I >> just noticed for the first time last Thursday. Also, sometimes the >> "FAILURE"s change position in the Current Report Inbox list, moving up or >> down a spot. Not sure if that helps diagnose what might be happening... >> >> In any case, Wikimetrics is mostly functioning but seems to be having >> recurring troubles that sometimes blow up to freeze the whole tool. It >> would be great to resolve the troubles before the next explosion--is there >> anything I can do to help? Dan H and I still have plenty of reports to >> run, we can keep you updated on the reports ran and failure rate while you >> are fixing, if that would be useful. >> >> Many thanks, >> Amanda >> >> >> On Mon, Feb 16, 2015 at 10:15 AM, Nuria Ruiz <[email protected]> wrote: >> >>> Ping .... >>> >>> On Fri, Feb 13, 2015 at 2:19 PM, Nuria Ruiz <[email protected]> wrote: >>> >>>> Amanda, >>>> >>>> Looks like wikimetrics was able to run automatic reports last night w/o >>>> big issues, are your reports still failing? >>>> >>>> Thanks, >>>> >>>> Nuria >>>> >>>> On Thu, Feb 12, 2015 at 1:42 PM, Amanda Bittaker < >>>> [email protected]> wrote: >>>> >>>>> Alright, thanks so much for your help once again, Nuria. >>>>> >>>>> If there's anything I can do or any information I can contribute, >>>>> please don't hesitate to ping me. >>>>> >>>>> Best, >>>>> Amanda >>>>> >>>>> >>>>> >>>>> On Thu, Feb 12, 2015 at 1:36 PM, Nuria Ruiz <[email protected]> >>>>> wrote: >>>>> >>>>>> DB connections in labs look to be failing, unfortunately I think >>>>>> besides asking for help on the labs list there is not much we can do >>>>>> there. >>>>>> I will start a thread on this regard. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Nuria >>>>>> >>>>>> On Thu, Feb 12, 2015 at 1:32 PM, Amanda Bittaker < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Thanks so much for the quick response, Nuria. >>>>>>> >>>>>>> I ran the exact same reports on the same cohort as one of the last >>>>>>> batches that were failing. Last time 2/4 of the reports failed, when I >>>>>>> reran the individually they succeeded. (But they don't always, I reran >>>>>>> one >>>>>>> report 3 times this morning before it worked.) This time, my failure >>>>>>> rate >>>>>>> got worse: 4/4 failed, although they said "PENDING" for a few seconds >>>>>>> first, which is new. >>>>>>> >>>>>>> Is that useful information? Please do let me know what else I can >>>>>>> do to help solve this. >>>>>>> >>>>>>> Thanks again, >>>>>>> Amanda >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Thu, Feb 12, 2015 at 1:09 PM, Jonathan Morgan < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Thanks Nuria! >>>>>>>> >>>>>>>> On Thu, Feb 12, 2015 at 12:57 PM, Nuria Ruiz <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> If so a cohort + report to repro will be most useful. >>>>>>>>> >>>>>>>> >>>>>>>> Translation:* try to run the exact same reports on the same cohort >>>>>>>> again, to see if the same metrics fail. Let us know what you find. ;) >>>>>>>> >>>>>>>> Same goes for anyone else who experiences these issues: the more >>>>>>>> details we (users) can provide the engineers, the more effective they >>>>>>>> can >>>>>>>> be at diagnosing and addressing the problems. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> - J >>>>>>>> >>>>>>>> *for anyone who is not 100% familiar with that hip, new software >>>>>>>> engineering lingo >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Nuria >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Feb 12, 2015 at 12:35 PM, Dan Andreescu < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Recently there was a restart of the labsdb cluster. I'm sorry >>>>>>>>>> but I don't have time to check on it, but I bet that's the problem. >>>>>>>>>> I'm >>>>>>>>>> off tomorrow unfortunately but I'll try to check tomorrow night :( >>>>>>>>>> I hope >>>>>>>>>> someone else beats me to it. >>>>>>>>>> >>>>>>>>>> On Thu, Feb 12, 2015 at 3:20 PM, Jonathan Morgan < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> (ping Kevin and Dan A.) >>>>>>>>>>> >>>>>>>>>>> Hi Amanda, I've had some problems with report failures recently >>>>>>>>>>> when I ran a few test cohorts. On the same cohort, when I ran >>>>>>>>>>> multiple >>>>>>>>>>> concurrent reports (say, bytes added, edits, and pages created), >>>>>>>>>>> some would >>>>>>>>>>> fail and others succeed. It wasn't clear what the issue was. >>>>>>>>>>> >>>>>>>>>>> - J >>>>>>>>>>> >>>>>>>>>>> On Thu, Feb 12, 2015 at 12:16 PM, Amanda Bittaker < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hello all, >>>>>>>>>>>> >>>>>>>>>>>> I am getting failures again, both when uploading cohorts and >>>>>>>>>>>> running reports. Strangely, it seems the more reports you try to >>>>>>>>>>>> run in >>>>>>>>>>>> one batch the less likely it is any report will succeed. >>>>>>>>>>>> >>>>>>>>>>>> Is anyone else having these problems again? Wonderful >>>>>>>>>>>> Analytics people, could you please work your magic again? >>>>>>>>>>>> >>>>>>>>>>>> Many thanks, >>>>>>>>>>>> Amanda >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> Wikimetrics mailing list >>>>>>>>>>>> [email protected] >>>>>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/wikimetrics >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Jonathan T. Morgan >>>>>>>>>>> Community Research Lead >>>>>>>>>>> Wikimedia Foundation >>>>>>>>>>> User:Jmorgan (WMF) >>>>>>>>>>> <https://meta.wikimedia.org/wiki/User:Jmorgan_(WMF)> >>>>>>>>>>> [email protected] >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Wikimetrics mailing list >>>>>>>>>> [email protected] >>>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/wikimetrics >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Jonathan T. Morgan >>>>>>>> Community Research Lead >>>>>>>> Wikimedia Foundation >>>>>>>> User:Jmorgan (WMF) >>>>>>>> <https://meta.wikimedia.org/wiki/User:Jmorgan_(WMF)> >>>>>>>> [email protected] >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Wikimetrics mailing list >>>>>>>> [email protected] >>>>>>>> https://lists.wikimedia.org/mailman/listinfo/wikimetrics >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >> _______________________________________________ >> Wikimetrics mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/wikimetrics >> >> > > _______________________________________________ > Wikimetrics mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wikimetrics > >
_______________________________________________ Wikimetrics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikimetrics
