I can't speak for the project, but my guess would be it's out of scope.
There are many existing integrations which use the existing webhook
mechanism:
https://prometheus.io/docs/operating/integrations/#alertmanager-webhook-receiver
Doing it externally allows the request to be enriched with other data which
wouldn't be available in a template, and to use a variety of outbound
authentication mechanisms which Alertmanager itself doesn't implement, or
even non-HTTP protocols like STOMP.
In any case, text templating is a nasty and insecure way to create JSON or
XML payloads.
> it is possible to converse the receivable JSON on the hop and just send
notifications via curl method
You'd first have to *receive* the webhook payload from Alertmanager, which
involves plugging your curl script into a webserver somehow. You could
probably write one as a CGI script under Apache, for example.
But why would you do that, when you have an existing tested and working
solution?
On Monday, 12 June 2023 at 08:13:50 UTC+1 Near N wrote:
> Thanks for reply Brian. I do realize, that Google Chat API requires a
> particular JSON format, but was assuming Alertmanager is capable of doing
> it by using particular template, which is wrong. Now the questions is
> whether it is worth creating a PR to create such a feature or it is out of
> Alertmanager's standard scope?
>
> Another question is perhaps it is possible to converse the receivable JSON
> on the hop and just send notifications via curl method? I mean if the
> endpoint contains the following alert "
> [{"annotations":{"server":"main","number_of_nodes":"1"},"endsAt":"2023-06-12T07:02:00.233Z","fingerprint":"b21bf244f87cd47f","receivers":[{"name":"my_alerts"}],"startsAt":"2023-06-12T06:44:00.233Z","status":{"inhibitedBy":[],"silencedBy":[],"state":"active"},"updatedAt":"2023-06-12T06:58:00.424Z","generatorURL":"
> http://app1:8880/vmalert/alert?group_id=71689142540802366","labels":{"alertgroup":"INFRA","alertname":"The
>
> number of available Cassandra nodes has
> changed","server":"main","severity":"Critical"}}]", would it be possible to
> retrieve this json and post it via curl? This, I assume, wouldn't solve my
> problem, cause I believe Google API expects something peculiar in terms of
> the message it receives.
>
> понедельник, 12 июня 2023 г. в 13:04:17 UTC+6, Brian Candler:
>
>> > Invalid JSON payload received.
>>
>> Alertmanager's webhook sends a JSON message with a *fixed* payload
>> format. See
>> https://prometheus.io/docs/alerting/latest/configuration/#webhook_config
>> *"The Alertmanager will send HTTP POST requests in the following JSON
>> format to the configured endpoint"*
>>
>> You can't override it with a template. The Alertmanager JSON format is
>> different to what Google Chat requires, and therefore you can't send it
>> directly.
>>
>> That's why you need a separate piece of software to do the conversion
>> (Calert in your case) - it receives the Alertmanager webhook JSON payload,
>> builds a new payload for the target, and then sends it there.
>>
>> On Monday, 12 June 2023 at 06:59:14 UTC+1 Near N wrote:
>>
>>> Good day. I was wondering if there is a way to configure alertmanager to
>>> send notifications to google chat via a webhook.
>>>
>>> The current setup that I use consists of vmalert (loaded with set of
>>> promQL compatible rules) >> Alertmanager (to group them up) and Calert (to
>>> send them to google chat). Despite the flowchart is working fine, I found
>>> it difficult to amend proposed message template, so decided to simplify it
>>> by configuring alertmanager to send the alerts. It dawns on me that it can
>>> operate, but I seem to have failed to fine-tune it. Any help would be
>>> appreciated. The alertmanager.yml is below:
>>>
>>> global:
>>> http_config:
>>> proxy_url: '{{ alertmanager_proxy_url }}'
>>>
>>> route:
>>> receiver: 'my_alerts'
>>> group_wait: 30s
>>> group_interval: 60s
>>> repeat_interval: 15m
>>> group_by: ['room', 'alertName']
>>>
>>> receivers:
>>> - name: 'my_alerts'
>>> webhook_configs:
>>> - url: '{{ alertmanager_chat_webhook }}'
>>> send_resolved: true
>>>
>>> templates:
>>> - '{{ alertmanager_working_directory }}/google-chat.tmpl'
>>>
>>>
>>> And here is the error of alertmanager trying to send an alert:
>>>
>>> Jun 12 05:26:00 app1-cp alertmanager[140337]:
>>> ts=2023-06-12T05:26:00.599Z caller=dispatch.go:352 level=error
>>> component=dispatcher msg="Notify for alerts failed" num_alerts=3
>>> err="my_alerts/webhook[0]: notify retry canceled due to unrecoverable error
>>> after 1 attempts: unexpected status code 400:
>>> https://chat.googleapis.com/v1/spaces/webook_token: {\n \"error\": {\n
>>> \"code\": 400,\n \"message\": \"Invalid JSON payload received.
>>> Unknown name \\\"receiver\\\" at 'message': Cannot find field.\\nInvalid
>>> JSON payload received. Unknown name \\\"status\\\" at 'message': Cannot
>>> find field.\\nInvalid JSON payload received. Unknown name \\\"alerts\\\" at
>>> 'message': Cannot find field.\\nInvalid JSON payload received. Unknown name
>>> \\\"groupLabels\\\" at 'message': Cannot find field.\\nInvalid JSON payload
>>> received. Unknown name \\\"commonLabels\\\" at 'message': Cannot find
>>> field.\\nInvalid JSON payload received. Unknown name
>>> \\\"commonAnnotations\\\" at 'message': Cannot find field.\\nInvalid JSON
>>> payload received. Unknown name \\\"externalURL\\\" at 'message': Cannot
>>> find field.\\nInvalid JSON payload received. Unknown name \\\"version\\\"
>>> at 'message': Cannot find field.\\nInvalid JSON payload received. Unknown
>>> name \\\"groupKey\\\" at 'message': Cannot find field.\\nInvalid JSON
>>> payload received. Unknown name \\\"truncatedAlerts\\\" at 'message': Cannot
>>> find field.\",\n \"status\": \"INVALID_ARGUMENT\",\n \"details\": [\n
>>> {\n \"@type\": \"type.googleapis.com/google.rpc.BadRequest\
>>> <http://type.googleapis.com/google.rpc.BadRequest%5C>",\n
>>> \"fieldViolations\": [\n {\n \"field\": \"message\",\n
>>> \"description\": \"Invalid JSON payload received. Unknown name
>>> \\\"receiver\\\" at 'message': Cannot find field.\"\n },\n
>>> {\n \"field\": \"message\",\n \"description\":
>>> \"Invalid JSON payload received. Unknown name \\\"status\\\" at 'message':
>>> Cannot find field.\"\n },\n {\n \"field\":
>>> \"message\",\n \"description\": \"Invalid JSON payload received.
>>> Unknown name \\\"alerts\\\" at 'message': Cannot find field.\"\n
>>> },\n {\n \"field\": \"message\",\n
>>> \"description\": \"Invalid JSON payload received. Unknown name
>>> \\\"groupLabels\\\" at 'message': Cannot find field.\"\n },\n
>>> {\n \"field\": \"message\",\n \"description\":
>>> \"Invalid JSON payload received. Unknown name \\\"commonLabels\\\" at
>>> 'message': Cannot find field.\"\n },\n {\n
>>> \"field\": \"message\",\n \"description\": \"Invalid JSON
>>> payload received. Unknown name \\\"commonAnnotations\\\" at 'message':
>>> Cannot find field.\"\n },\n {\n \"field\":
>>> \"message\",\n \"description\": \"Invalid JSON payload received.
>>> Unknown name \\\"externalURL\\\" at 'message': Cannot find field.\"\n
>>> },\n {\n \"field\": \"message\",\n
>>> \"description\": \"Invalid JSON payload received. Unknown name
>>> \\\"version\\\" at 'message': Cannot find field.\"\n },\n
>>> {\n \"field\": \"message\",\n \"description\":
>>> \"Invalid JSON payload received. Unknown name \\\"groupKey\\\" at
>>> 'message': Cannot find field.\"\n },\n {\n
>>> \"field\": \"message\",\n \"description\": \"Invalid JSON
>>> payload received. Unknown name \\\"truncatedAlerts\\\" at 'message': Cannot
>>> find field.\"\n }\n ]\n }\n ]\n }\n}\n"
>>>
>>>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/d882d3d8-3cbb-4f26-a832-269f75d0c27an%40googlegroups.com.
