On Monday, December 15, 2014, Kevin Leduc <[email protected]> wrote:
> > I'd like to move this to a video conference call between analytics > developers and analytics engineering to come to a mutual understanding of > what the current pain points are and what's the biggest priority. > It probably makes sense to have someone from R&D with experience in QA in that meeting (Dario if you want a more experienced person, myself otherwise). Not sure if you meant the same when you said analytics engineering. Leila > > > > > On Mon, Dec 15, 2014 at 4:37 PM, Nuria Ruiz <[email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: >> >> >QA in beta labs is good but not enough. We still need to do QA when a >> feature goes to production and currently >> This is true but at the same time, I do not see anything in the >> description of your FF events that could not be tested on beta-labs. If we >> are talking add-block that can be tested even earlier, vagrant will be a >> fine venue. All the issues related to the client (browser) not emitting >> events can be tested on the development environment with ease. >> >> >> >> On Mon, Dec 15, 2014 at 4:18 PM, Leila Zia <[email protected] >> <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: >>> >>> >>> On Mon, Dec 15, 2014 at 10:06 AM, Toby Negrin <[email protected] >>> <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: >>>> >>>> I share Christian's concerns - >>>> >>>> Dario/Leila - can you comment based on your recent experiences with >>>> WikiGrok? >>>> >>> >>> I agree with Christian. >>> >>> QA in beta labs is good but not enough. We still need to do QA when a >>> feature goes to production and currently, it's very hard to figure out if >>> there's a problem with logging. An example: >>> >>> While testing WikiGrok in production, we learned that after some point >>> tests from Firefox browser from my machine were not logged. We did not get >>> any errors for this. I found out about this because I was trying to >>> manually make a trace of activities and see if I can stitch them together >>> and make sense of them. We eventually figured out what was going on in that >>> case [1], but it concerns me that there may be other important events that >>> we don't log in the DB and we never know that we're not logging. >>> >>> Leila >>> [1] >>> https://lists.wikimedia.org/pipermail/analytics/2014-December/002864.html >>> >>> >>>> >>>> Thanks >>>> >>>> -Toby >>>> >>>> >>>> > On Dec 15, 2014, at 9:42 AM, Christian Aistleitner < >>>> [email protected] >>>> <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: >>>> > >>>> > Hi, >>>> > >>>> >> On Mon, Dec 15, 2014 at 08:34:39AM -0800, Kevin Leduc wrote: >>>> >> I closed the Phabricator task with a links to this thread and the >>>> wikitech >>>> >> doc for testing on beta cluster. >>>> > >>>> > I am fine with keeping the task closed. >>>> > >>>> > But I am somewhat surprised to see beta mentioned in the >>>> > resolution. Note that Dario's request set scope as [1] >>>> > >>>> > However, there are types of data quality issues that we only >>>> > discover when collecting data at scale and in the wild (on >>>> > browsers/platforms that we don’t necessarily test for internally). >>>> > >>>> > . That's a valid scope, but from my point of view, beta does not match >>>> > that scope. >>>> > >>>> > Neither is beta large scale, nor is it hammered on with crazy devices. >>>> > >>>> > Beta is just a halfing the distance between EventLogging's devserver >>>> > (Vagrant!) and production. >>>> > >>>> > Have fun, >>>> > Christian >>>> > >>>> > >>>> > >>>> > [1] >>>> https://lists.wikimedia.org/pipermail/analytics/2014-December/002884.html >>>> > >>>> > >>>> > >>>> > -- >>>> > ---- quelltextlich e.U. ---- \\ ---- Christian Aistleitner ---- >>>> > Companies' registry: 360296y in Linz >>>> > Christian Aistleitner >>>> > Kefermarkterstrasze 6a/3 Email: [email protected] >>>> <javascript:_e(%7B%7D,'cvml','[email protected]');> >>>> > 4293 Gutau, Austria Phone: +43 7946 / 20 5 81 >>>> > Fax: +43 7946 / 20 5 81 >>>> > Homepage: http://quelltextlich.at/ >>>> > --------------------------------------------------------------- >>>> > _______________________________________________ >>>> > Analytics mailing list >>>> > [email protected] >>>> <javascript:_e(%7B%7D,'cvml','[email protected]');> >>>> > https://lists.wikimedia.org/mailman/listinfo/analytics >>>> >>>> _______________________________________________ >>>> Analytics mailing list >>>> [email protected] >>>> <javascript:_e(%7B%7D,'cvml','[email protected]');> >>>> https://lists.wikimedia.org/mailman/listinfo/analytics >>>> >>> >>> _______________________________________________ >>> Analytics mailing list >>> [email protected] >>> <javascript:_e(%7B%7D,'cvml','[email protected]');> >>> https://lists.wikimedia.org/mailman/listinfo/analytics >>> >>> >> _______________________________________________ >> Analytics mailing list >> [email protected] >> <javascript:_e(%7B%7D,'cvml','[email protected]');> >> https://lists.wikimedia.org/mailman/listinfo/analytics >> >>
_______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
