The problem is, valid shell expressions like "${#ARCHIVE_LOGS[@]}" contain
"{#" which is Jinja for "start-of-comment" and you have no "#}"
end-of-comment.
To get the string "${#ARCHIVE_LOGS[@]}" to be generated in a template, you
have to split the "{" from the "#". Here's one way:
while mapfile -t -n 20 ARCHIVE_LOGS && ((${{ '{' ~ '#'
}}ARCHIVE_LOGS[@]}))
There are lots of other ways to slice it up though. Good luck.
On Wednesday, September 14, 2022 at 10:03:34 AM UTC-4 [email protected]
wrote:
> I am trying to use template module to send 2 shell scripts to a remote.
>
> ### TASK
> ##################################################################################################
>
> - name: configure postgres - template backup scripts
> ansible.builtin.template:
> src: "{{ item }}.j2"
> dest: "{{ postgres_base }}/tools/backup/{{ item }}"
> backup: yes
> owner: postgres
> group: postgres
> mode: 0600
> become: yes
> loop:
> - backup_postgres10.sh
> - backup_archivelogs.sh
> ### TASK
> ##################################################################################################
>
> while the first item is being templated out fine the second fails with a
> "Missing end of comment tag. String: #!/bin/bash" error. With 0 experience
> with jinja2 I can not make any sense of what I found in the documentation
> in that respect.
> Content of both files it 100% original bash script content.
>
>
> ### backup_archivelogs.sh.j2
> #############################################################################
>
> #!/bin/bash
> #
> # $Header$
> #
> # backup postgres archivelogs to TSM
>
> # source postgres env
> source /opt/db/postgres/bin/.pg.env
>
> #set -x
>
> # write logs to backup
> # do an incremental backup of the postgres archive/wal logs
> # take a bunch of 20 logs per dsmc incr call
> cd $PGARCH
> while mapfile -t -n 20 ARCHIVE_LOGS && ((${#ARCHIVE_LOGS[@]}))
> do
> echo ">>> backing up: ${ARCHIVE_LOGS[@]}"
> sudo /opt/tivoli/tsm/client/ba/bin/dsmc incr "${ARCHIVE_LOGS[@]}"
> # check return code, if 0/true then purge logfile
> RET=$?
> if [ $RET -eq 0 ]; then
> echo
> echo ">>> TSM incremental backup successful, purging archive logs up to
> ${ARCHIVE_LOGS[-1]} <<<"
> rm "${ARCHIVE_LOGS[@]}"
> fi
> sleep 2
> done < <(ls -1 0*)
>
>
>
> ### OUTPUT
> ################################################################################################
>
> TASK [configure postgres - template backup scripts]
> ********************************************************************
> task path:
> /home/username/repos/ansible/open_source/postgres_create_service/postgres_create_service.yml:83
> <server.intern> ESTABLISH SSH CONNECTION FOR USER: None
> <server.intern> SSH: EXEC ssh -C -o ControlMaster=auto -o
> ControlPersist=60s -o KbdInteractiveAuthentication=no -o
> PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey
> -o PasswordAuthentication=no -o ConnectTimeout=10 -o
> 'ControlPath="/home/username/.ansible/cp/8ca9f05e1f"' server.intern
> '/bin/sh -c '"'"'echo ~ && sleep 0'"'"''
> <server.intern> (0, b'/home/username\n', b'')
> <server.intern> ESTABLISH SSH CONNECTION FOR USER: None
> <server.intern> SSH: EXEC ssh -C -o ControlMaster=auto -o
> ControlPersist=60s -o KbdInteractiveAuthentication=no -o
> PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey
> -o PasswordAuthentication=no -o ConnectTimeout=10 -o
> 'ControlPath="/home/username/.ansible/cp/8ca9f05e1f"' server.intern
> '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /var/tmp `"&& mkdir "` echo
> /var/tmp/ansible-tmp-1663162681.3683012-2675848-43258002595610 `" && echo
> ansible-tmp-1663162681.3683012-2675848-43258002595610="` echo
> /var/tmp/ansible-tmp-1663162681.3683012-2675848-43258002595610 `" ) &&
> sleep 0'"'"''
> <server.intern> (0,
> b'ansible-tmp-1663162681.3683012-2675848-43258002595610=/var/tmp/ansible-tmp-1663162681.3683012-2675848-43258002595610\n',
>
> b'')
> Using module file /usr/lib/python3.8/site-packages/ansible/modules/stat.py
> <server.intern> PUT
> /home/username/.ansible/tmp/ansible-local-2675679n2q7xy3g/tmpwcgv_g_c TO
> /var/tmp/ansible-tmp-1663162681.3683012-2675848-43258002595610/AnsiballZ_stat.py
> <server.intern> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o
> ControlPersist=60s -o KbdInteractiveAuthentication=no -o
> PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey
> -o PasswordAuthentication=no -o ConnectTimeout=10 -o
> 'ControlPath="/home/username/.ansible/cp/8ca9f05e1f"' '[server.intern]'
> <server.intern> (0, b'sftp> put
> /home/username/.ansible/tmp/ansible-local-2675679n2q7xy3g/tmpwcgv_g_c
> /var/tmp/ansible-tmp-1663162681.3683012-2675848-43258002595610/AnsiballZ_stat.py\n',
>
> b'')
> <server.intern> ESTABLISH SSH CONNECTION FOR USER: None
> <server.intern> SSH: EXEC ssh -C -o ControlMaster=auto -o
> ControlPersist=60s -o KbdInteractiveAuthentication=no -o
> PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey
> -o PasswordAuthentication=no -o ConnectTimeout=10 -o
> 'ControlPath="/home/username/.ansible/cp/8ca9f05e1f"' server.intern
> '/bin/sh -c '"'"'setfacl -m u:postgres:r-x
> /var/tmp/ansible-tmp-1663162681.3683012-2675848-43258002595610/
> /var/tmp/ansible-tmp-1663162681.3683012-2675848-43258002595610/AnsiballZ_stat.py
>
> && sleep 0'"'"''
> <server.intern> (0, b'', b'')
> <server.intern> ESTABLISH SSH CONNECTION FOR USER: None
> <server.intern> SSH: EXEC ssh -C -o ControlMaster=auto -o
> ControlPersist=60s -o KbdInteractiveAuthentication=no -o
> PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey
> -o PasswordAuthentication=no -o ConnectTimeout=10 -o
> 'ControlPath="/home/username/.ansible/cp/8ca9f05e1f"' -tt server.intern
> '/bin/sh -c '"'"'sudo -H -S -n -u postgres /bin/sh -c '"'"'"'"'"'"'"'"'echo
> BECOME-SUCCESS-ztbiuccpcrwstltpesmvsnlefldjhueu ;
> /usr/libexec/platform-python
> /var/tmp/ansible-tmp-1663162681.3683012-2675848-43258002595610/AnsiballZ_stat.py'"'"'"'"'"'"'"'"'
>
> && sleep 0'"'"''
> Escalation succeeded
> <server.intern> (0, b'\r\n\r\n{"changed": false, "stat": {"exists": true,
> "path": "/opt/db/postgres/tools/backup/backup_postgres10.sh", "mode":
> "0600", "isdir": false, "ischr": false, "isblk": false, "isreg": true,
> "isfifo": false, "islnk": false, "issock": false, "uid": 1001, "gid": 1001,
> "size": 9567, "inode": 6553734, "dev": 64772, "nlink": 1, "atime":
> 1663162307.594887, "mtime": 1662108268.8151002, "ctime":
> 1662108269.4020917, "wusr": true, "rusr": true, "xusr": false, "wgrp":
> false, "rgrp": false, "xgrp": false, "woth": false, "roth": false, "xoth":
> false, "isuid": false, "isgid": false, "blocks": 24, "block_size": 4096,
> "device_type": 0, "readable": true, "writeable": true, "executable": false,
> "pw_name": "postgres", "gr_name": "postgres", "checksum":
> "c25840eb364bd104b796679eef8082ad6887bae3", "mimetype":
> "text/x-shellscript", "charset": "us-ascii", "version": "3998925608",
> "attributes": [], "attr_flags": ""}, "invocation": {"module_args": {"path":
> "/opt/db/postgres/tools/back
> up/backup_postgres10.sh", "follow": false, "get_checksum": true,
> "checksum_algorithm": "sha1", "get_md5": false, "get_mime": true,
> "get_attributes": true}}}\r\n', b'Shared connection to server.intern
> closed.\r\n')
> Using module file /usr/lib/python3.8/site-packages/ansible/modules/file.py
> <server.intern> PUT
> /home/username/.ansible/tmp/ansible-local-2675679n2q7xy3g/tmpafre2fjl TO
> /var/tmp/ansible-tmp-1663162681.3683012-2675848-43258002595610/AnsiballZ_file.py
> <server.intern> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o
> ControlPersist=60s -o KbdInteractiveAuthentication=no -o
> PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey
> -o PasswordAuthentication=no -o ConnectTimeout=10 -o
> 'ControlPath="/home/username/.ansible/cp/8ca9f05e1f"' '[server.intern]'
> <server.intern> (0, b'sftp> put
> /home/username/.ansible/tmp/ansible-local-2675679n2q7xy3g/tmpafre2fjl
> /var/tmp/ansible-tmp-1663162681.3683012-2675848-43258002595610/AnsiballZ_file.py\n',
>
> b'')
> <server.intern> ESTABLISH SSH CONNECTION FOR USER: None
> <server.intern> SSH: EXEC ssh -C -o ControlMaster=auto -o
> ControlPersist=60s -o KbdInteractiveAuthentication=no -o
> PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey
> -o PasswordAuthentication=no -o ConnectTimeout=10 -o
> 'ControlPath="/home/username/.ansible/cp/8ca9f05e1f"' server.intern
> '/bin/sh -c '"'"'setfacl -m u:postgres:r-x
> /var/tmp/ansible-tmp-1663162681.3683012-2675848-43258002595610/
> /var/tmp/ansible-tmp-1663162681.3683012-2675848-43258002595610/AnsiballZ_file.py
>
> && sleep 0'"'"''
> <server.intern> (0, b'', b'')
> <server.intern> ESTABLISH SSH CONNECTION FOR USER: None
> <server.intern> SSH: EXEC ssh -C -o ControlMaster=auto -o
> ControlPersist=60s -o KbdInteractiveAuthentication=no -o
> PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey
> -o PasswordAuthentication=no -o ConnectTimeout=10 -o
> 'ControlPath="/home/username/.ansible/cp/8ca9f05e1f"' -tt server.intern
> '/bin/sh -c '"'"'sudo -H -S -n -u postgres /bin/sh -c '"'"'"'"'"'"'"'"'echo
> BECOME-SUCCESS-qttckqlufxsixkcjmiiesbbvfedjlfuy ;
> /usr/libexec/platform-python
> /var/tmp/ansible-tmp-1663162681.3683012-2675848-43258002595610/AnsiballZ_file.py'"'"'"'"'"'"'"'"'
>
> && sleep 0'"'"''
> Escalation succeeded
> <server.intern> (0, b'\r\n{"path":
> "/opt/db/postgres/tools/backup/backup_postgres10.sh", "changed": false,
> "diff": {"before": {"path":
> "/opt/db/postgres/tools/backup/backup_postgres10.sh"}, "after": {"path":
> "/opt/db/postgres/tools/backup/backup_postgres10.sh"}}, "uid": 1001, "gid":
> 1001, "owner": "postgres", "group": "postgres", "mode": "0600", "state":
> "file", "secontext": "system_u:object_r:usr_t:s0", "size": 9567,
> "invocation": {"module_args": {"owner": "postgres", "group": "postgres",
> "mode": 384, "dest": "/opt/db/postgres/tools/backup/backup_postgres10.sh",
> "_original_basename": "backup_postgres10.sh.j2", "recurse": false, "state":
> "file", "path": "/opt/db/postgres/tools/backup/backup_postgres10.sh",
> "force": false, "follow": true, "modification_time_format":
> "%Y%m%d%H%M.%S", "access_time_format": "%Y%m%d%H%M.%S", "unsafe_writes":
> false, "_diff_peek": null, "src": null, "modification_time": null,
> "access_time": null, "seuser": null, "serole": null, "selevel": null,
> "setype": nu
> ll, "attributes": null}}}\r\n', b'Shared connection to server.intern
> closed.\r\n')
> <server.intern> ESTABLISH SSH CONNECTION FOR USER: None
> <server.intern> SSH: EXEC ssh -C -o ControlMaster=auto -o
> ControlPersist=60s -o KbdInteractiveAuthentication=no -o
> PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey
> -o PasswordAuthentication=no -o ConnectTimeout=10 -o
> 'ControlPath="/home/username/.ansible/cp/8ca9f05e1f"' server.intern
> '/bin/sh -c '"'"'rm -f -r
> /var/tmp/ansible-tmp-1663162681.3683012-2675848-43258002595610/ > /dev/null
> 2>&1 && sleep 0'"'"''
> <server.intern> (0, b'', b'')
> ok: [server.intern] => (item=backup_postgres10.sh) => changed=false
> ansible_loop_var: item
> checksum: c25840eb364bd104b796679eef8082ad6887bae3
> dest: /opt/db/postgres/tools/backup/backup_postgres10.sh
> diff:
> after:
> path: /opt/db/postgres/tools/backup/backup_postgres10.sh
> before:
> path: /opt/db/postgres/tools/backup/backup_postgres10.sh
> gid: 1001
> group: postgres
> invocation:
> module_args:
> _diff_peek: null
> _original_basename: backup_postgres10.sh.j2
> access_time: null
> access_time_format: '%Y%m%d%H%M.%S'
> attributes: null
> dest: /opt/db/postgres/tools/backup/backup_postgres10.sh
> follow: true
> force: false
> group: postgres
> mode: 384
> modification_time: null
> modification_time_format: '%Y%m%d%H%M.%S'
> owner: postgres
> path: /opt/db/postgres/tools/backup/backup_postgres10.sh
> recurse: false
> selevel: null
> serole: null
> setype: null
> seuser: null
> src: null
> state: file
> unsafe_writes: false
> item: backup_postgres10.sh
> mode: '0600'
> owner: postgres
> path: /opt/db/postgres/tools/backup/backup_postgres10.sh
> secontext: system_u:object_r:usr_t:s0
> size: 9567
> state: file
> uid: 1001
> <server.intern> ESTABLISH SSH CONNECTION FOR USER: None
> <server.intern> SSH: EXEC ssh -C -o ControlMaster=auto -o
> ControlPersist=60s -o KbdInteractiveAuthentication=no -o
> PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey
> -o PasswordAuthentication=no -o ConnectTimeout=10 -o
> 'ControlPath="/home/username/.ansible/cp/8ca9f05e1f"' server.intern
> '/bin/sh -c '"'"'echo ~ && sleep 0'"'"''
> <server.intern> (0, b'/home/username\n', b'')
> <server.intern> ESTABLISH SSH CONNECTION FOR USER: None
> <server.intern> SSH: EXEC ssh -C -o ControlMaster=auto -o
> ControlPersist=60s -o KbdInteractiveAuthentication=no -o
> PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey
> -o PasswordAuthentication=no -o ConnectTimeout=10 -o
> 'ControlPath="/home/username/.ansible/cp/8ca9f05e1f"' server.intern
> '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /var/tmp `"&& mkdir "` echo
> /var/tmp/ansible-tmp-1663162682.168117-2675848-28324059801519 `" && echo
> ansible-tmp-1663162682.168117-2675848-28324059801519="` echo
> /var/tmp/ansible-tmp-1663162682.168117-2675848-28324059801519 `" ) && sleep
> 0'"'"''
> <server.intern> (0,
> b'ansible-tmp-1663162682.168117-2675848-28324059801519=/var/tmp/ansible-tmp-1663162682.168117-2675848-28324059801519\n',
>
> b'')
> <server.intern> ESTABLISH SSH CONNECTION FOR USER: None
> <server.intern> SSH: EXEC ssh -C -o ControlMaster=auto -o
> ControlPersist=60s -o KbdInteractiveAuthentication=no -o
> PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey
> -o PasswordAuthentication=no -o ConnectTimeout=10 -o
> 'ControlPath="/home/username/.ansible/cp/8ca9f05e1f"' server.intern
> '/bin/sh -c '"'"'rm -f -r
> /var/tmp/ansible-tmp-1663162682.168117-2675848-28324059801519/ > /dev/null
> 2>&1 && sleep 0'"'"''
> <server.intern> (0, b'', b'')
> The full traceback is:
> Traceback (most recent call last):
> File "/usr/lib/python3.8/site-packages/ansible/template/__init__.py", line
> 1118, in do_template
> t = myenv.from_string(data)
> File "/usr/lib/python3.8/site-packages/jinja2/environment.py", line 880,
> in from_string
> return cls.from_code(self, self.compile(source), globals, None)
> File "/usr/lib/python3.8/site-packages/jinja2/environment.py", line 591,
> in compile
> self.handle_exception(exc_info, source_hint=source_hint)
> File "/usr/lib/python3.8/site-packages/jinja2/environment.py", line 780,
> in handle_exception
> reraise(exc_type, exc_value, tb)
> File "/usr/lib/python3.8/site-packages/jinja2/_compat.py", line 37, in
> reraise
> raise value.with_traceback(tb)
> File "<unknown>", line 17, in <module>
> File "/usr/lib/python3.8/site-packages/jinja2/environment.py", line 497,
> in _parse
> return Parser(self, source, name, encode_filename(filename)).parse()
> File "/usr/lib/python3.8/site-packages/jinja2/parser.py", line 901, in
> parse
> result = nodes.Template(self.subparse(), lineno=1)
> File "/usr/lib/python3.8/site-packages/jinja2/parser.py", line 872, in
> subparse
> next(self.stream)
> File "/usr/lib/python3.8/site-packages/jinja2/lexer.py", line 359, in
> __next__
> self.current = next(self._iter)
> File "/usr/lib/python3.8/site-packages/jinja2/lexer.py", line 562, in wrap
> for lineno, token, value in stream:
> File "/usr/lib/python3.8/site-packages/jinja2/lexer.py", line 646, in
> tokeniter
> raise token(lineno, filename)
> File "/usr/lib/python3.8/site-packages/jinja2/lexer.py", line 229, in
> __call__
> raise self.error_class(self.message, lineno, filename)
> jinja2.exceptions.TemplateSyntaxError: Missing end of comment tag
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File
> "/usr/lib/python3.8/site-packages/ansible/plugins/action/template.py", line
> 150, in run
> resultant = templar.do_template(template_data,
> preserve_trailing_newlines=True, escape_backslashes=False)
> File "/usr/lib/python3.8/site-packages/ansible/template/__init__.py", line
> 1120, in do_template
> raise AnsibleError("template error while templating string: %s. String:
> %s" % (to_native(e), to_native(data)))
> ansible.errors.AnsibleError: template error while templating string:
> Missing end of comment tag. String: #!/bin/bash
> #
> # $Header$
> #
> # backup postgres archivelogs to TSM
>
> # source postgres env
> source /opt/db/postgres/bin/.pg.env
>
> #set -x
>
> # write logs to backup
> # do an incremental backup of the postgres archive/wal logs
> # take a bunch of 20 logs per dsmc incr call
> cd $PGARCH
> while mapfile -t -n 20 ARCHIVE_LOGS && ((${#ARCHIVE_LOGS[@]}))
> do
> echo ">>> backing up: ${ARCHIVE_LOGS[@]}"
> sudo /opt/tivoli/tsm/client/ba/bin/dsmc incr "${ARCHIVE_LOGS[@]}"
> # check return code, if 0/true then purge logfile
> RET=$?
> if [ $RET -eq 0 ]; then
> echo
> echo ">>> TSM incremental backup successful, purging archive logs up to
> ${ARCHIVE_LOGS[-1]} <<<"
> rm "${ARCHIVE_LOGS[@]}"
> fi
> sleep 2
> done < <(ls -1 0*)
>
>
> failed: [server.intern] (item=backup_archivelogs.sh) => changed=false
> ansible_loop_var: item
> item: backup_archivelogs.sh
> msg: |-
> AnsibleError: template error while templating string: Missing end of
> comment tag. String: #!/bin/bash
> #
> # $Header$
> #
> # backup postgres archivelogs to TSM
>
> # source postgres env
> source /opt/db/postgres/bin/.pg.env
>
> #set -x
>
> # write logs to backup
> # do an incremental backup of the postgres archive/wal logs
> # take a bunch of 20 logs per dsmc incr call
> cd $PGARCH
> while mapfile -t -n 20 ARCHIVE_LOGS && ((${#ARCHIVE_LOGS[@]}))
> do
> echo ">>> backing up: ${ARCHIVE_LOGS[@]}"
> sudo /opt/tivoli/tsm/client/ba/bin/dsmc incr "${ARCHIVE_LOGS[@]}"
> # check return code, if 0/true then purge logfile
> RET=$?
> if [ $RET -eq 0 ]; then
> echo
> echo ">>> TSM incremental backup successful, purging archive logs up to
> ${ARCHIVE_LOGS[-1]} <<<"
> rm "${ARCHIVE_LOGS[@]}"
> fi
> sleep 2
> done < <(ls -1 0*)
>
>
> ############################################################################################
>
--
You received this message because you are subscribed to the Google Groups
"Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/ansible-project/b7483a07-0c40-4dfd-83f2-efa73e5b6740n%40googlegroups.com.
