I created an HTTP LB setup on GCP using a golang HTTP server without kubernetes and was able to see rare long-tail latencies in >1 second. After I set `IdleTimeout` to larger than ten minutes, I stopped seeing those slow responses. The echoheaders image uses nginx and doesn't set `keepalive_timeout` (sent PR to update this).
https://cloud.google.com/compute/docs/load-balancing/http/#timeouts_and_retries On Wednesday, May 16, 2018 at 8:08:26 AM UTC-7, ama...@upstream.tech wrote: > > On Monday, May 14, 2018 at 5:10:55 PM UTC-4, francois...@polynom.io > wrote: > > Le jeudi 19 avril 2018 02:14:28 UTC+2, Dave Jensen a écrit : > > > We have what I believe to be a very straightforward ingress setup on > GKE. However, we started noticing random slowdowns almost immediately. On > further investigation it looked like the time to first byte (TTFB) was very > slow (1 - 3 seconds) but sporadically. Sometimes it would be a pre-flight > OPTIONS request, sometimes an application request, and other times on > static files. Even the echoserver would sporadically have a long TTFB. > > > > > > > > > I set up a portforward to one of the pods serving up our REST API > server. Sure enough, the slowdown was eliminated. > > > > > > > > > Before I go down the rabbit hole of trying other ingress controllers, > I figured I'd ask the community if I was doing something wrong. > > > > > > > > > ingress.yaml > > > > > > > > > > > > > > > > > > apiVersion: extensions/v1beta1 > > > kind: Ingress > > > metadata: > > > name: brewd-ingress > > > spec: > > > tls: > > > - hosts: > > > - stage-api2.example.com > > > - stage-app2.example.com > > > - stage-echo.example.com > > > secretName: redacted > > > rules: > > > - host: stage-api2.example.com > > > http: > > > paths: > > > - backend: > > > serviceName: gateway-service > > > servicePort: 7000 > > > - host: stage-app2.example.com > > > http: > > > paths: > > > - backend: > > > serviceName: web-service > > > servicePort: 8080 > > > - host: stage-echo.example.com > > > http: > > > paths: > > > - backend: > > > serviceName: echoserver > > > servicePort: 8080 > > > gateway-service.yaml > > > > > > > > > > > > apiVersion: v1 > > > kind: Service > > > metadata: > > > name: gateway-service > > > labels: > > > app: gateway > > > spec: > > > type: NodePort > > > ports: > > > - port: 7000 > > > selector: > > > app: gateway > > > --- > > > apiVersion: apps/v1beta2 > > > kind: Deployment > > > metadata: > > > name: gateway-deployment > > > spec: > > > selector: > > > matchLabels: > > > app: gateway > > > replicas: 1 > > > template: > > > metadata: > > > labels: > > > app: gateway > > > spec: > > > containers: > > > - name: gateway > > > image: us.gcr.io/redacted/gateway:1.3.0-rc.0 > > > imagePullPolicy: Always > > > ports: > > > - containerPort: 7000 > > > env: > > > - name: REDACTED_ENV > > > value: stage > > > The web-service yaml looks almost exactly the same as the above. > > > > Hi guyz, I have the exact same problem. If I deploy the service as LB, > it is blazing fast, but if I use the basic ingress on Google Ingress, I > have random latency whith response time ranging from 50ms to 5s. > > Hi Francois, > > We are having a similar issue. Could you give more information on how you > deployed as a LB? Want to get that setup while we continue looking for a > solution to this. -- You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group. To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscr...@googlegroups.com. To post to this group, send email to kubernetes-users@googlegroups.com. Visit this group at https://groups.google.com/group/kubernetes-users. For more options, visit https://groups.google.com/d/optout.
