I added a comment to the ticket requesting a simple error log for validation errors. I think that would solve about 50% of the problem and should be easy to implement.
Kaldari On Mon, Dec 15, 2014 at 5:58 PM, Leila Zia <[email protected]> wrote: > > > > On Monday, December 15, 2014, Kevin Leduc <[email protected]> wrote: > >> >> I'd like to move this to a video conference call between analytics >> developers and analytics engineering to come to a mutual understanding of >> what the current pain points are and what's the biggest priority. >> > > It probably makes sense to have someone from R&D with experience in QA in > that meeting (Dario if you want a more experienced person, myself > otherwise). Not sure if you meant the same when you said analytics > engineering. > > Leila > > >> >> >> >> >> On Mon, Dec 15, 2014 at 4:37 PM, Nuria Ruiz <[email protected]> wrote: >>> >>> >QA in beta labs is good but not enough. We still need to do QA when a >>> feature goes to production and currently >>> This is true but at the same time, I do not see anything in the >>> description of your FF events that could not be tested on beta-labs. If we >>> are talking add-block that can be tested even earlier, vagrant will be a >>> fine venue. All the issues related to the client (browser) not emitting >>> events can be tested on the development environment with ease. >>> >>> >>> >>> On Mon, Dec 15, 2014 at 4:18 PM, Leila Zia <[email protected]> wrote: >>>> >>>> >>>> On Mon, Dec 15, 2014 at 10:06 AM, Toby Negrin <[email protected]> >>>> wrote: >>>>> >>>>> I share Christian's concerns - >>>>> >>>>> Dario/Leila - can you comment based on your recent experiences with >>>>> WikiGrok? >>>>> >>>> >>>> I agree with Christian. >>>> >>>> QA in beta labs is good but not enough. We still need to do QA when a >>>> feature goes to production and currently, it's very hard to figure out if >>>> there's a problem with logging. An example: >>>> >>>> While testing WikiGrok in production, we learned that after some point >>>> tests from Firefox browser from my machine were not logged. We did not get >>>> any errors for this. I found out about this because I was trying to >>>> manually make a trace of activities and see if I can stitch them together >>>> and make sense of them. We eventually figured out what was going on in that >>>> case [1], but it concerns me that there may be other important events that >>>> we don't log in the DB and we never know that we're not logging. >>>> >>>> Leila >>>> [1] >>>> https://lists.wikimedia.org/pipermail/analytics/2014-December/002864.html >>>> >>>> >>>>> >>>>> Thanks >>>>> >>>>> -Toby >>>>> >>>>> >>>>> > On Dec 15, 2014, at 9:42 AM, Christian Aistleitner < >>>>> [email protected]> wrote: >>>>> > >>>>> > Hi, >>>>> > >>>>> >> On Mon, Dec 15, 2014 at 08:34:39AM -0800, Kevin Leduc wrote: >>>>> >> I closed the Phabricator task with a links to this thread and the >>>>> wikitech >>>>> >> doc for testing on beta cluster. >>>>> > >>>>> > I am fine with keeping the task closed. >>>>> > >>>>> > But I am somewhat surprised to see beta mentioned in the >>>>> > resolution. Note that Dario's request set scope as [1] >>>>> > >>>>> > However, there are types of data quality issues that we only >>>>> > discover when collecting data at scale and in the wild (on >>>>> > browsers/platforms that we don’t necessarily test for internally). >>>>> > >>>>> > . That's a valid scope, but from my point of view, beta does not >>>>> match >>>>> > that scope. >>>>> > >>>>> > Neither is beta large scale, nor is it hammered on with crazy >>>>> devices. >>>>> > >>>>> > Beta is just a halfing the distance between EventLogging's devserver >>>>> > (Vagrant!) and production. >>>>> > >>>>> > Have fun, >>>>> > Christian >>>>> > >>>>> > >>>>> > >>>>> > [1] >>>>> https://lists.wikimedia.org/pipermail/analytics/2014-December/002884.html >>>>> > >>>>> > >>>>> > >>>>> > -- >>>>> > ---- quelltextlich e.U. ---- \\ ---- Christian Aistleitner ---- >>>>> > Companies' registry: 360296y in Linz >>>>> > Christian Aistleitner >>>>> > Kefermarkterstrasze 6a/3 Email: [email protected] >>>>> > 4293 Gutau, Austria Phone: +43 7946 / 20 5 81 >>>>> > Fax: +43 7946 / 20 5 81 >>>>> > Homepage: http://quelltextlich.at/ >>>>> > --------------------------------------------------------------- >>>>> > _______________________________________________ >>>>> > Analytics mailing list >>>>> > [email protected] >>>>> > https://lists.wikimedia.org/mailman/listinfo/analytics >>>>> >>>>> _______________________________________________ >>>>> Analytics mailing list >>>>> [email protected] >>>>> https://lists.wikimedia.org/mailman/listinfo/analytics >>>>> >>>> >>>> _______________________________________________ >>>> Analytics mailing list >>>> [email protected] >>>> https://lists.wikimedia.org/mailman/listinfo/analytics >>>> >>>> >>> _______________________________________________ >>> Analytics mailing list >>> [email protected] >>> https://lists.wikimedia.org/mailman/listinfo/analytics >>> >>> > _______________________________________________ > Analytics mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/analytics > >
_______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
