>QA in beta labs is good but not enough. We still need to do QA when a feature goes to production and currently This is true but at the same time, I do not see anything in the description of your FF events that could not be tested on beta-labs. If we are talking add-block that can be tested even earlier, vagrant will be a fine venue. All the issues related to the client (browser) not emitting events can be tested on the development environment with ease.
On Mon, Dec 15, 2014 at 4:18 PM, Leila Zia <[email protected]> wrote: > > > On Mon, Dec 15, 2014 at 10:06 AM, Toby Negrin <[email protected]> > wrote: >> >> I share Christian's concerns - >> >> Dario/Leila - can you comment based on your recent experiences with >> WikiGrok? >> > > I agree with Christian. > > QA in beta labs is good but not enough. We still need to do QA when a > feature goes to production and currently, it's very hard to figure out if > there's a problem with logging. An example: > > While testing WikiGrok in production, we learned that after some point > tests from Firefox browser from my machine were not logged. We did not get > any errors for this. I found out about this because I was trying to > manually make a trace of activities and see if I can stitch them together > and make sense of them. We eventually figured out what was going on in that > case [1], but it concerns me that there may be other important events that > we don't log in the DB and we never know that we're not logging. > > Leila > [1] > https://lists.wikimedia.org/pipermail/analytics/2014-December/002864.html > > >> >> Thanks >> >> -Toby >> >> >> > On Dec 15, 2014, at 9:42 AM, Christian Aistleitner < >> [email protected]> wrote: >> > >> > Hi, >> > >> >> On Mon, Dec 15, 2014 at 08:34:39AM -0800, Kevin Leduc wrote: >> >> I closed the Phabricator task with a links to this thread and the >> wikitech >> >> doc for testing on beta cluster. >> > >> > I am fine with keeping the task closed. >> > >> > But I am somewhat surprised to see beta mentioned in the >> > resolution. Note that Dario's request set scope as [1] >> > >> > However, there are types of data quality issues that we only >> > discover when collecting data at scale and in the wild (on >> > browsers/platforms that we don’t necessarily test for internally). >> > >> > . That's a valid scope, but from my point of view, beta does not match >> > that scope. >> > >> > Neither is beta large scale, nor is it hammered on with crazy devices. >> > >> > Beta is just a halfing the distance between EventLogging's devserver >> > (Vagrant!) and production. >> > >> > Have fun, >> > Christian >> > >> > >> > >> > [1] >> https://lists.wikimedia.org/pipermail/analytics/2014-December/002884.html >> > >> > >> > >> > -- >> > ---- quelltextlich e.U. ---- \\ ---- Christian Aistleitner ---- >> > Companies' registry: 360296y in Linz >> > Christian Aistleitner >> > Kefermarkterstrasze 6a/3 Email: [email protected] >> > 4293 Gutau, Austria Phone: +43 7946 / 20 5 81 >> > Fax: +43 7946 / 20 5 81 >> > Homepage: http://quelltextlich.at/ >> > --------------------------------------------------------------- >> > _______________________________________________ >> > Analytics mailing list >> > [email protected] >> > https://lists.wikimedia.org/mailman/listinfo/analytics >> >> _______________________________________________ >> Analytics mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/analytics >> > > _______________________________________________ > Analytics mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/analytics > >
_______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
