High number of 4xx requests on etcd (3.6 upgrade)

Post upgrade to 3.6 I'm noticing the API server seems to be responding a lot slower and my etcd metrics etcd_http_failed_total is returning a large number of failed GET requests. Has anyone seen this? ___ users mailing list

Re: High number of 4xx requests on etcd (3.6 upgrade)

Check how much IO is being used by etcd and how much you have provisioned. > On Aug 12, 2017, at 5:32 AM, Andrew Lau wrote: > > Post upgrade to 3.6 I'm noticing the API server seems to be responding a lot > slower and my etcd metrics etcd_http_failed_total is returning a

Re: High number of 4xx requests on etcd (3.6 upgrade)

etcd data is on dedicated drives and aws reports idle and burst capacity around 90% On Sun, 13 Aug 2017 at 00:28 Clayton Coleman wrote: > Check how much IO is being used by etcd and how much you have provisioned. > > > > On Aug 12, 2017, at 5:32 AM, Andrew Lau

Re: High number of 4xx requests on etcd (3.6 upgrade)

How big is your etcd working set in terms of number of keys? How many namespaces? If keys <50k then i would suspect software, hardware, or network issue in between masters and etcd. Http etcd failures should only happen when the master is losing elections and being turned over, or the

Re: Scheduler policy in 3.6 (default and custom at install time)

On Sat, Aug 12, 2017 at 11:59 AM, Avesh Agarwal wrote: > > > On Fri, Aug 11, 2017 at 2:28 AM, Per Carlson wrote: > >> Hi. >> >> We are in the process of rebuilding a cluster with a new topology, and >> I'm trying to fit the node labels to a scheduler

Re: Scheduler policy in 3.6 (default and custom at install time)

On Fri, Aug 11, 2017 at 2:28 AM, Per Carlson wrote: > Hi. > > We are in the process of rebuilding a cluster with a new topology, and I'm > trying to fit the node labels to a scheduler policy, and would like to base > the policy on the default one. > > I've searched both the