The issue appears to be back. For the past few nights I've been seeing sporadic 4-5 second response times on calls. Again, it's calls like OPTIONS that really stand out.
On Thursday, May 17, 2018 at 11:37:28 AM UTC-7, Dave Jensen wrote: > Thank you, I think this solved the issue. We set the IdleTimeout (in > Golang) to 620s and, in our staging environment, I have not seen a request > take longer than 200ms. > > On Wednesday, May 16, 2018 at 2:41:32 PM UTC-7, Nicks wrote: >> >> I created an HTTP LB setup on GCP using a golang HTTP server without >> kubernetes and was able to see rare long-tail latencies in >1 second. After >> I set `IdleTimeout` to larger than ten minutes, I stopped seeing those slow >> responses. The echoheaders image uses nginx and doesn't set >> `keepalive_timeout` (sent PR to update this). >> >> >> https://cloud.google.com/compute/docs/load-balancing/http/#timeouts_and_retries >> >> >> On Wednesday, May 16, 2018 at 8:08:26 AM UTC-7, ama...@upstream.tech >> wrote: >>> >>> On Monday, May 14, 2018 at 5:10:55 PM UTC-4, francois...@polynom.io >>> wrote: >>> > Le jeudi 19 avril 2018 02:14:28 UTC+2, Dave Jensen a écrit : >>> > > We have what I believe to be a very straightforward ingress setup on >>> GKE. However, we started noticing random slowdowns almost immediately. On >>> further investigation it looked like the time to first byte (TTFB) was very >>> slow (1 - 3 seconds) but sporadically. Sometimes it would be a pre-flight >>> OPTIONS request, sometimes an application request, and other times on >>> static files. Even the echoserver would sporadically have a long TTFB. >>> > > >>> > > >>> > > I set up a portforward to one of the pods serving up our REST API >>> server. Sure enough, the slowdown was eliminated. >>> > > >>> > > >>> > > Before I go down the rabbit hole of trying other ingress >>> controllers, I figured I'd ask the community if I was doing something >>> wrong. >>> > > >>> > > >>> > > ingress.yaml >>> > > >>> > > >>> > > >>> > > >>> > > >>> > > apiVersion: extensions/v1beta1 >>> > > kind: Ingress >>> > > metadata: >>> > > name: brewd-ingress >>> > > spec: >>> > > tls: >>> > > - hosts: >>> > > - stage-api2.example.com >>> > > - stage-app2.example.com >>> > > - stage-echo.example.com >>> > > secretName: redacted >>> > > rules: >>> > > - host: stage-api2.example.com >>> > > http: >>> > > paths: >>> > > - backend: >>> > > serviceName: gateway-service >>> > > servicePort: 7000 >>> > > - host: stage-app2.example.com >>> > > http: >>> > > paths: >>> > > - backend: >>> > > serviceName: web-service >>> > > servicePort: 8080 >>> > > - host: stage-echo.example.com >>> > > http: >>> > > paths: >>> > > - backend: >>> > > serviceName: echoserver >>> > > servicePort: 8080 >>> > > gateway-service.yaml >>> > > >>> > > >>> > > >>> > > apiVersion: v1 >>> > > kind: Service >>> > > metadata: >>> > > name: gateway-service >>> > > labels: >>> > > app: gateway >>> > > spec: >>> > > type: NodePort >>> > > ports: >>> > > - port: 7000 >>> > > selector: >>> > > app: gateway >>> > > --- >>> > > apiVersion: apps/v1beta2 >>> > > kind: Deployment >>> > > metadata: >>> > > name: gateway-deployment >>> > > spec: >>> > > selector: >>> > > matchLabels: >>> > > app: gateway >>> > > replicas: 1 >>> > > template: >>> > > metadata: >>> > > labels: >>> > > app: gateway >>> > > spec: >>> > > containers: >>> > > - name: gateway >>> > > image: us.gcr.io/redacted/gateway:1.3.0-rc.0 >>> > > imagePullPolicy: Always >>> > > ports: >>> > > - containerPort: 7000 >>> > > env: >>> > > - name: REDACTED_ENV >>> > > value: stage >>> > > The web-service yaml looks almost exactly the same as the above. >>> > >>> > Hi guyz, I have the exact same problem. If I deploy the service as LB, >>> it is blazing fast, but if I use the basic ingress on Google Ingress, I >>> have random latency whith response time ranging from 50ms to 5s. >>> >>> Hi Francois, >>> >>> We are having a similar issue. Could you give more information on how >>> you deployed as a LB? Want to get that setup while we continue looking for >>> a solution to this. >> >> -- You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group. To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscr...@googlegroups.com. To post to this group, send email to kubernetes-users@googlegroups.com. Visit this group at https://groups.google.com/group/kubernetes-users. For more options, visit https://groups.google.com/d/optout.
