Yes. I setup a DNS entry. So where shall I add the “http/2” protocol?? Below is
my nginx ingress config.
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
creationTimestamp: "2020-07-01T16:39:41Z"
finalizers:
- service.kubernetes.io/load-balancer-cleanup
labels:
app: nginx-ingress
chart: nginx-ingress-1.40.2
component: controller
heritage: Helm
release: nginx-ingress
name: nginx-ingress-controller
namespace: espr-prometheus-nonprod
resourceVersion: "7098976"
selfLink:
/api/v1/namespaces/espr-prometheus-nonprod/services/nginx-ingress-controller
uid: e1238468-5d80-446b-b2d0-364f068208b6
spec:
clusterIP: xx.xx.xx.xx
externalTrafficPolicy: Cluster
ports:
- name: http
nodePort: 32615
port: 80
protocol: TCP
targetPort: http
- name: https
nodePort: 30561
port: 443
protocol: TCP
targetPort: https
selector:
app: nginx-ingress
app.kubernetes.io/component: controller
release: nginx-ingress
sessionAffinity: None
type: LoadBalancer
status:
loadBalancer:
ingress:
- ip: yy.yy.yy.yy
> On Jul 2, 2020, at 10:08 AM, Tristan Colgate <[email protected]> wrote:
>
> grpc uses http/2, some load balancers need to be explicitly configured, and
> most require http/2 over Https.
> You'll need to setup a DNS entry for the loadbalancer IP, or use the IP
> directly. What options are you providing to query ?
>
> On Thu, 2 Jul 2020, 17:56 Zhang Zhao, <[email protected]
> <mailto:[email protected]>> wrote:
> Hi Tristan,
> Thank you for your reply and sorry for the weird background color.
>
> For the backend, it doesn’t actually use http protocol. It uses GRPC. For
> your 2nd point, actually that’s what I tried. I added the annotation below
> when I was configuring nginx ingress controller and an internal ip to my
> network was assigned to the ingress service and pointed it to the sidecar
> service. The problem was that on Thanos Query side, the sidecar was not able
> to be discovered by the ingress load balancer ip.
>
> service.beta.kubernetes.io/azure-load-balancer-internal
> <http://service.beta.kubernetes.io/azure-load-balancer-internal>: "true"
>
>
>
>
>
>
> On Jul 2, 2020, at 1:21 AM, Tristan Colgate <[email protected]
> <mailto:[email protected]>> wrote:
>>
>> (before I answer, just a quick note that your mail client seems to use
>> HTML with a black background and white foreground for text, and it
>> looks super weird if the reader has a white background mail client as
>> it is all stripey).
>>
>> Two things:
>>
>> 1. it sounds like maybe your nginx ingress isn't doing the right
>> thing. I've not used the nginx ingress, but you may need to indicate
>> to it that the backend requires http2.
>>
>> 2. Whether you point the querier at the nginx ingress, or the sidecar
>> pod, you probably want to do that via a kubernetes service rather than
>> direct to the pod's IP. If you use a service with a clusterIP, that IP
>> is fixed until such time as your delete that service. If you are in a
>> different cluster you may need to use a service with type
>> "loadbalancer", you may need to look at how to make that an internal
>> only load balancer in azure (normally done by adding an annotation),
>> which would restrict it to be internal to the network of your cloud
>> provider, and not public.
>>
>> On Thu, 2 Jul 2020 at 07:04, Zhang Zhao <[email protected]
>> <mailto:[email protected]>> wrote:
>>>
>>> I setup 2 Kubernetes clusters on Azure, Thanos Querier is in one cluster,
>>> and a local prometheus+sidecar. I need to query the remote cluster Thanos
>>> sidecar. I created ingress-nginx with support for grpc on remote cluster.
>>> But the sidecar grpc failed in Thanos Querier so that Querier was not able
>>> to discover the sidecar on remote cluster. I still had to use the ip and
>>> port to discover the sidecar on remote clusters. Everytime the pod is
>>> restarted, the ip changes and the connection is broken. Any advice?
>>>
>>>
>>>
>>>
>>>
>>> apiVersion: extensions/v1beta1
>>>
>>> kind: Ingress
>>>
>>> metadata:
>>>
>>> name: thanos-sidecar-ingress
>>>
>>> namespace: espr-prometheus-nonprod
>>>
>>> annotations:
>>>
>>> kubernetes.io/ingress.class: <http://kubernetes.io/ingress.class:>
>>> “nginx”
>>>
>>> nginx.ingress.kubernetes.io/ssl-redirect:
>>> <http://nginx.ingress.kubernetes.io/ssl-redirect:> "false"
>>>
>>> nginx.ingress.kubernetes.io/backend-protocol:
>>> <http://nginx.ingress.kubernetes.io/backend-protocol:> “GRPC”
>>>
>>> spec:
>>>
>>> rules:
>>>
>>> - host: xxx.example.com <http://xxx.example.com/>
>>>
>>> http:
>>>
>>> paths:
>>>
>>> - backend:
>>>
>>> serviceName: prometheus-operated
>>>
>>> servicePort: 10901
>>>
>>> path: /(.*)
>>>
>>>
>>>
>>> Zhang
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "Prometheus Users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to [email protected]
>>> <mailto:[email protected]>.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/prometheus-users/97dd931f-4720-4572-9c89-cccb3d16ccf8o%40googlegroups.com
>>>
>>> <https://groups.google.com/d/msgid/prometheus-users/97dd931f-4720-4572-9c89-cccb3d16ccf8o%40googlegroups.com>.
>>
>>
>>
>> --
>> Tristan Colgate-McFarlane
>> ----
>> "You can get all your daily vitamins from 52 pints of guiness, and a
>> glass of milk"
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/A56F2136-3DFE-4CE6-BB7C-B031FA845E2B%40gmail.com.
