Sorry, that was me. >I would urge you to setup a per-user/per-service query resource limits, otherwise poorly performant queries will affect all users (and in cases like this, create downtime). I have >set up query limits for all research/analytics users temporarily until 3rd January.
Could we make this limits permanent? I do not see a reason not to do that. Let me know if I am missing something. Thanks, Nuria On Thu, Dec 29, 2016 at 12:56 PM, Dan Andreescu <[email protected]> wrote: > Thanks Jaime, we will use the Summit to show people how to use Hadoop / > Druid for some of this analysis‎, and that should get these types of > problems off your plate. > > *From: *Jaime Crespo > *Sent: *Sunday, December 25, 2016 04:57 > *To: *A mailing list for the Analytics Team at WMF and everybody who has > an interest in Wikipedia and analytics. > *Reply To: *A mailing list for the Analytics Team at WMF and everybody > who has an interest in Wikipedia and analytics. > *Subject: *[Analytics] analytics-store was down due to excessive load > > analytics-store was brought down at 6am, and then again at 9am UTC 25 Dec > due to multiple executions of long running queries (some of them 2 days > long) such as: > > SELECT LEFT(timestamp, 8) AS yearmonthday, timestamp, userAgent, clientIp, > webHost, COUNT(*) AS copies FROM log.PageContentSaveComplete ... > > SELECT COUNT(*) AS count, term_entity_type, term_type, term_language FROM > wikidatawiki.wb_terms ... > > select date('20161218000000') as day, actions, count(*) as repeated from > (select group_concat(event_action order by timestamp, action_order.ord > separator '-') as actions from (select ... > > I would urge you to setup a per-user/per-service query resource limits, > otherwise poorly performant queries will affect all users (and in cases > like this, create downtime). I have set up query limits for all > research/analytics users temporarily until 3rd January. > > -- > Jaime Crespo > <http://wikimedia.org> > > > _______________________________________________ > Analytics mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/analytics > >
_______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
