submission 503 errors

2017-09-28 Thread Nigel Horne
I've been seeing a number of messages such as this: http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd";>       503 backend read error         Error 503 backend read error     backend read error     Guru Mediation:     Details: cache-dca17747-DCA 1506600623 2675462800         Varnish cache s

Re: submission 503 errors

2017-09-28 Thread Doug Bell
Is this when you're submitting a report or when you're doing something else? Was that message at approximately Thu Sep 28 12:10:23 2017 GMT? > On Sep 28, 2017, at 7:14 AM, Nigel Horne wrote: > > I've been seeing a number of messages such as this: > > > "http://www.w3.org/TR/xhtml1/DTD/xhtm

Re: submission 503 errors

2017-09-28 Thread Nigel Horne
When I was submitting a report. Metabase submission doesn't timestamp, it just says things like "Submit '0OZncYKk5xGxvTcp8awfHg==' (4.2263879776001s) success" -Nigel Is this when you're submitting a report or when you're doing something else? Was that message at approximately Thu Sep 28 12:1

Re: submission 503 errors

2017-09-28 Thread Doug Bell
Luckily, the response contains the epoch timestamp (1506600623), is my guess. I've made the health checks and timeouts more aggressive, so Fastly should remove the bad node from the load balance rotation hopefully before an actual report is sent in. I can't make it perfect, I don't have enough n

Re: submission 503 errors

2017-09-28 Thread Nigel Horne
Doug, Luckily, the response contains the epoch timestamp (1506600623), is my guess. I've made the health checks and timeouts more aggressive, so Fastly should remove the bad node from the load balance rotation hopefully before an actual report is sent in. I can't make it perfect, I don't have

Re: submission 503 errors

2017-09-28 Thread Doug Bell
It doesn't help when immediately after I make the change (the dark thin bar to the left of the chart) both of the api nodes go down (the big red bit in the chart). :p So, right now we're running degraded. It should be back with some time, and then the errors should be reduced... > On Sep 28,