Brian, thanks for taking the time to run my config. The error you noted was
just in my email. I had to manually enter the config file because I'm
running alertmanager on an isolated network.
I finally discovered the problem. I'm running alertmanager in a docker
container, and I'm using nomad to run that container. To get the
alertmanager config file and the custom template file into the docker
container, I use a nomad template stanza to copy Here Document text to the
local docker volume, and I map that volume to the location where
alertmanager expects the files to be. After discovering that I don't get
the error when I start the docker container manually, only when I run it in
nomad, I realized that the "template not defined" error was referring to
the nomad template, not the alertmanager go template. They both use {{ and
}} for template delimiters, but nomad allows you to specify an alternate
character.
So my nomad template stanza now looks like this:
template: {
change_mode = "noop"
destination = "local/email-tmepolate.tmpl"
left_delimiter = "[["
right_delimiter = "]]"
data = <<EOH
---
<alertmanager config content here, including the text text: '{{ template
"custom.email" . }}'>
EOH
}
And that works. Without the left_delimiter and right_delimeter overrides,
nomad fails the job while parsing the template stanza, thinking that I'm
referring to a nomad template named "custom.email", which of course does
not exist. So the alertmanager was never even started, as nomad failed the
job when parsing its template stanza.
I was completely on the wrong track by not realizing it was the nomad
template that was having the problem. Thanks, everyone, for your help,
especially as the problem had nothing to do with alertmanager.
-Mark
On Sat, Jun 6, 2020 at 1:22 PM Brian Candler <[email protected]> wrote:
> The configuration file that you posted was mangled - partly by your E-mail
> client I guess, and partly by you (there are two single quotes before
> ''my.host')
>
> However, if I use *exactly* the following configuration file:
>
> global:
> smtp_smarthost: 'my.host:25'
> smtp_hello: 'my.system'
> smtp_from: 'my.adderss'
>
> route:
> receiver: test-email-receiver
> group_by: [label1, label2]
>
> receivers:
> - name: 'test-email-receiver'
> email_configs:
> - to: '[email protected]'
> text: '{{ template "custom.email" . }}'
>
> templates:
> - '/etc/alertmanager/templates/custom-email-template.tmpl'
>
> then alertmanager starts just fine:
>
> root@prometheus:~# /opt/alertmanager/alertmanager
> --config.file=/tmp/alert.yml
> level=info ts=2020-06-06T17:16:00.765Z caller=main.go:231 msg="Starting
> Alertmanager" version="(version=0.20.0, branch=HEAD,
> revision=f74be0400a6243d10bb53812d6fa408ad71ff32d)"
> level=info ts=2020-06-06T17:16:00.766Z caller=main.go:232
> build_context="(go=go1.13.5, user=root@00c3106655f8,
> date=20191211-14:13:14)"
> level=info ts=2020-06-06T17:16:00.770Z caller=cluster.go:161
> component=cluster msg="setting advertise address explicitly"
> addr=10.12.255.33 port=9094
> level=info ts=2020-06-06T17:16:00.777Z caller=cluster.go:623
> component=cluster msg="Waiting for gossip to settle..." interval=2s
> level=info ts=2020-06-06T17:16:00.871Z caller=coordinator.go:119
> component=configuration msg="Loading configuration file" file=/tmp/alert.yml
> level=info ts=2020-06-06T17:16:00.875Z caller=coordinator.go:131
> component=configuration msg="Completed loading of configuration file"
> file=/tmp/alert.yml
> level=info ts=2020-06-06T17:16:00.888Z caller=main.go:497 msg=Listening
> address=:9093
> level=info ts=2020-06-06T17:16:02.777Z caller=cluster.go:648
> component=cluster msg="gossip not settled" polls=0 before=0 now=1
> elapsed=2.00028059s
> level=info ts=2020-06-06T17:16:10.778Z caller=cluster.go:640
> component=cluster msg="gossip settled; proceeding" elapsed=10.001491515s
> q
> ^C
> level=info ts=2020-06-06T17:16:40.881Z caller=main.go:536 msg="Received
> SIGTERM, exiting gracefully..."
>
> root@prometheus:~# /opt/alertmanager/alertmanager --version
> alertmanager, version 0.20.0 (branch: HEAD, revision:
> f74be0400a6243d10bb53812d6fa408ad71ff32d)
> build user: root@00c3106655f8
> build date: 20191211-14:13:14
> go version: go1.13.5
>
> This is despite the fact that I haven't even created
> /etc/alertmanager/templates/custom-email-template.tmpl
>
> Therefore, I can only surmise that the configuration you're testing with
> is not the one you posted. Maybe you haven't given the correct command-line
> argument to alertmanager to tell it which config file to read, and so it's
> reading a different one than the one you think it is.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-users/9de8ab3d-7f4e-43e1-be5c-665908e776cdo%40googlegroups.com
> <https://groups.google.com/d/msgid/prometheus-users/9de8ab3d-7f4e-43e1-be5c-665908e776cdo%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/CAFHj5Be9ASTR8L88OnOZS%2B_YezJVJD8-aGY-jv4njtEs130g8Q%40mail.gmail.com.
