I reopened the task because discussions on this are still ongoing and the issue isn't entirely resolved.
I'd like to move this to a video conference call between analytics developers and analytics engineering to come to a mutual understanding of what the current pain points are and what's the biggest priority. We'll then communicate a plan back to the list and update the tasks involved. On Mon, Dec 15, 2014 at 4:37 PM, Nuria Ruiz <[email protected]> wrote: > > >QA in beta labs is good but not enough. We still need to do QA when a > feature goes to production and currently > This is true but at the same time, I do not see anything in the > description of your FF events that could not be tested on beta-labs. If we > are talking add-block that can be tested even earlier, vagrant will be a > fine venue. All the issues related to the client (browser) not emitting > events can be tested on the development environment with ease. > > > > On Mon, Dec 15, 2014 at 4:18 PM, Leila Zia <[email protected]> wrote: >> >> >> On Mon, Dec 15, 2014 at 10:06 AM, Toby Negrin <[email protected]> >> wrote: >>> >>> I share Christian's concerns - >>> >>> Dario/Leila - can you comment based on your recent experiences with >>> WikiGrok? >>> >> >> I agree with Christian. >> >> QA in beta labs is good but not enough. We still need to do QA when a >> feature goes to production and currently, it's very hard to figure out if >> there's a problem with logging. An example: >> >> While testing WikiGrok in production, we learned that after some point >> tests from Firefox browser from my machine were not logged. We did not get >> any errors for this. I found out about this because I was trying to >> manually make a trace of activities and see if I can stitch them together >> and make sense of them. We eventually figured out what was going on in that >> case [1], but it concerns me that there may be other important events that >> we don't log in the DB and we never know that we're not logging. >> >> Leila >> [1] >> https://lists.wikimedia.org/pipermail/analytics/2014-December/002864.html >> >> >>> >>> Thanks >>> >>> -Toby >>> >>> >>> > On Dec 15, 2014, at 9:42 AM, Christian Aistleitner < >>> [email protected]> wrote: >>> > >>> > Hi, >>> > >>> >> On Mon, Dec 15, 2014 at 08:34:39AM -0800, Kevin Leduc wrote: >>> >> I closed the Phabricator task with a links to this thread and the >>> wikitech >>> >> doc for testing on beta cluster. >>> > >>> > I am fine with keeping the task closed. >>> > >>> > But I am somewhat surprised to see beta mentioned in the >>> > resolution. Note that Dario's request set scope as [1] >>> > >>> > However, there are types of data quality issues that we only >>> > discover when collecting data at scale and in the wild (on >>> > browsers/platforms that we don’t necessarily test for internally). >>> > >>> > . That's a valid scope, but from my point of view, beta does not match >>> > that scope. >>> > >>> > Neither is beta large scale, nor is it hammered on with crazy devices. >>> > >>> > Beta is just a halfing the distance between EventLogging's devserver >>> > (Vagrant!) and production. >>> > >>> > Have fun, >>> > Christian >>> > >>> > >>> > >>> > [1] >>> https://lists.wikimedia.org/pipermail/analytics/2014-December/002884.html >>> > >>> > >>> > >>> > -- >>> > ---- quelltextlich e.U. ---- \\ ---- Christian Aistleitner ---- >>> > Companies' registry: 360296y in Linz >>> > Christian Aistleitner >>> > Kefermarkterstrasze 6a/3 Email: [email protected] >>> > 4293 Gutau, Austria Phone: +43 7946 / 20 5 81 >>> > Fax: +43 7946 / 20 5 81 >>> > Homepage: http://quelltextlich.at/ >>> > --------------------------------------------------------------- >>> > _______________________________________________ >>> > Analytics mailing list >>> > [email protected] >>> > https://lists.wikimedia.org/mailman/listinfo/analytics >>> >>> _______________________________________________ >>> Analytics mailing list >>> [email protected] >>> https://lists.wikimedia.org/mailman/listinfo/analytics >>> >> >> _______________________________________________ >> Analytics mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/analytics >> >> > _______________________________________________ > Analytics mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/analytics > >
_______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
