Re: [ansible-project] Re: how to set different backup directory than current while using backup=yes

[Todd Lewis]

That looks fine as far as it goes. I think we all agree that none of this 
is particularly hard to implement.

However, the way I read the original poster's original problem, there is an 
implied constraint: One cannot go creating additional files in the .conf 
directory. Full stop. So the "backup: true" option - which lacks the long 
sought "backup_dir:" modifier - is off the table.

The options I see are: (a) create and maintain a staging directory and copy 
the files from there to the target .conf directory; (b) handle copies in 
advance - which is essentially implementing what you wish "backup_dir:" 
would do if existed; (c) give up on "roll your own 'backup: true'" 
altogether.

On Tuesday, October 31, 2023 at 10:30:20 AM UTC-4 Vladimir Botka wrote:

> On Tue, 31 Oct 2023 05:17:35 -0700 (PDT)
> Todd Lewis  wrote:
>
> > There is no backup system. ... I wouldn't have been able 
> > to create such a thing ... I wouldn't dare use it in 
> > production ... the rest of us have to build our 
> > wheels with the stick-n-stones within our reach.
> > 
> > On Monday, October 30, 2023 at 11:34:33 AM UTC-4 Brian Coca wrote:
> > 
> > > The backup system returns the 'backup_file' information so you can
> > > then operate on the built in backup, like moving it to a central
> > > location ...
>
> This can be achieved with ~20 lines of code. For example, let's
> declare the module defaults
>
> - hosts: all
> module_defaults:
> lineinfile:
> create: true
> backup: true
>
> and register the results
>
> tasks:
>
> - lineinfile:
> path: /tmp/test1.txt
> line: "{{ item }}"
> loop: [line1, line2, line3]
> register: ns_result_001
>
> - lineinfile:
> path: /tmp/test2.txt
> line: line1
> register: ns_result_0021
>
> - lineinfile:
> path: /tmp/test2.txt
> line: line2
> register: ns_result_0022
>
> Declare the list of all variables *ns_result_* and declare the list
> of all backup files
>
> vars:
>
> my_ns_results: "{{ q('vars', *q('varnames', '^ns_result_*')) }}"
>
> ns_bfiles: "{{ (my_ns_results|json_query('[].backup') +
> my_ns_results|json_query('[].results[].backup'))|
> flatten|select }}"
>
> Fetch the backup files to the controller and optionally delete them on
> the remote hosts
>
> post_tasks:
>
> - fetch:
> src: "{{ item }}"
> dest: /tmp/ansible/backup
> loop: "{{ ns_bfiles }}"
>
> - file:
> src: "{{ item }}"
> state: absent
> loop: "{{ ns_bfiles }}"
> when: ns_bfiles_delete|d(false)|bool
>
> For example,
>
> shell> tree /tmp/ansible/backup/
> /tmp/ansible/backup/
> └── test_11
> └── tmp
> ├── test1.txt.7143.2023-10-31@14:18:07~
> ├── test1.txt.7156.2023-10-31@14:18:08~
> └── test2.txt.7182.2023-10-31@14:18:11~
>
> -- 
> Vladimir Botka
>

-- 
You received this message because you are subscribed to the Google Groups 
"Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ansible-project+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/ansible-project/5044711c-c426-4f81-a484-a00921d5b99an%40googlegroups.com.

Re: [ansible-project] Re: how to set different backup directory than current while using backup=yes

[Vladimir Botka]

On Tue, 31 Oct 2023 05:17:35 -0700 (PDT)
Todd Lewis  wrote:

> There is no backup system. ... I wouldn't have been able 
> to create such a thing ... I wouldn't dare use it in 
> production ... the rest of us have to build our 
> wheels with the stick-n-stones within our reach.
> 
> On Monday, October 30, 2023 at 11:34:33 AM UTC-4 Brian Coca wrote:
> 
> > The backup system returns the 'backup_file' information so you can
> > then operate on the built in backup, like moving it to a central
> > location ...

This can be achieved with ~20 lines of code. For example, let's
declare the module defaults

- hosts: all
  module_defaults:
lineinfile:
  create: true
  backup: true

and register the results

  tasks:

- lineinfile:
path: /tmp/test1.txt
line: "{{ item }}"
  loop: [line1, line2, line3]
  register: ns_result_001

- lineinfile:
path: /tmp/test2.txt
line: line1
  register: ns_result_0021

- lineinfile:
path: /tmp/test2.txt
line: line2
  register: ns_result_0022

Declare the list of all variables *ns_result_* and declare the list
of all backup files

  vars:

my_ns_results: "{{ q('vars', *q('varnames', '^ns_result_*')) }}"

ns_bfiles: "{{ (my_ns_results|json_query('[].backup') +
my_ns_results|json_query('[].results[].backup'))|
flatten|select }}"

Fetch the backup files to the controller and optionally delete them on
the remote hosts

  post_tasks:

- fetch:
src: "{{ item }}"
dest: /tmp/ansible/backup
  loop: "{{ ns_bfiles }}"

- file:
src: "{{ item }}"
state: absent
  loop: "{{ ns_bfiles }}"
  when: ns_bfiles_delete|d(false)|bool

For example,

shell> tree /tmp/ansible/backup/
/tmp/ansible/backup/
└── test_11
└── tmp
├── test1.txt.7143.2023-10-31@14:18:07~
├── test1.txt.7156.2023-10-31@14:18:08~
└── test2.txt.7182.2023-10-31@14:18:11~

-- 
Vladimir Botka

-- 
You received this message because you are subscribed to the Google Groups 
"Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ansible-project+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/ansible-project/20231031152959.3f0004d7%40gmail.com.


pgp6GljosgDtl.pgp
Description: OpenPGP digital signature

Re: [ansible-project] Re: how to set different backup directory than current while using backup=yes

[Sameer Modak]

Thanks todd for explanation 

On Tuesday, October 31, 2023 at 5:47:35 PM UTC+5:30 Todd Lewis wrote:

> (For background, a great discussion surrounding the missing backup_dir: 
> task option can be read here: here 
> . Highly recommended.)
>
> You are absolutely correct, and yet I must respectfully disagree with 
> practically every point.
> > The backup system …
> There is no backup system. I understand you may think there is, because 
> you've written some incredibly impressive code that uses it in the role you 
> cited elsewhere in this thread. But if it isn't documented it doesn't 
> exist. I've been crawling all over Ansible documentation for the last five 
> years, and I would never have guessed such a thing was possible.
> And even if I knew in my bones it could be done, I wouldn't have been able 
> to create such a thing.
> And even if I had written such a thing, I wouldn't dare use it in 
> production, because it deals with Ansible internals in a way that screams, 
> "Subject to change without notice."
>
> The docs have improved tremendously in that time, by the way. It's not 
> surprising that internal interfaces aren't as well documented as the user 
> side of the bread-n-butter modules, especially as so much has changed in 
> that time. That rapid change isn't slowing, though, which makes tying 
> business process to those internals even less attractive.
>
> > I would not 'manage it beforehand'…
> No, you wouldn't, because you are as familiar with the magic behind the 
> mirror - the internal workings of Ansible - as the rest of us who are 
> limited to the user-facing side. But the rest of us have to build our 
> wheels with the stick-n-stones within our reach.
>
> > …as you do not know if a backup is needed…
> He wants to ensure he has a copy of the file. That's not a big ask.
> It isn't a backup, because "backup" implies a lot of things that aren't 
> true. They also aren't true when you say "backup: true" on a task. But 
> that's beside the point. If the user creates and maintains a copy with the 
> copy module, with the dest name that includes the src's mtime, and does it 
> before potentially clobbering the live conf file, then he's effectively 
> done what "backup: true" would do, except he hasn't trashed his conf 
> directory. Also, the copy task wouldn't make another copy if the file 
> hadn't changed. It would only do anything if either the contents or the 
> mtime changed, which is really all he's asking for.
>
>
> On Monday, October 30, 2023 at 11:34:33 AM UTC-4 Brian Coca wrote:
>
>> The backup system returns the 'backup_file' information so you can 
>> then operate on the built in backup, like moving it to a central 
>> location on the target, copying it to a backup server and/or managing 
>> the number/recency of backups. 
>>
>> I would not 'manage it beforehand' as you do not know if a backup is 
>> needed until after the task that creates the backup runs, mtime is not 
>> the best indicator. 
>>
>> -- 
>> -- 
>> Brian Coca 
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ansible-project+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/ansible-project/094b23e8-058c-4e1d-85fd-6d4292477c75n%40googlegroups.com.

Re: [ansible-project] Re: how to set different backup directory than current while using backup=yes

[Todd Lewis]

(For background, a great discussion surrounding the missing backup_dir: 
task option can be read here: here 
. Highly recommended.)

You are absolutely correct, and yet I must respectfully disagree with 
practically every point.
> The backup system …
There is no backup system. I understand you may think there is, because 
you've written some incredibly impressive code that uses it in the role you 
cited elsewhere in this thread. But if it isn't documented it doesn't 
exist. I've been crawling all over Ansible documentation for the last five 
years, and I would never have guessed such a thing was possible.
And even if I knew in my bones it could be done, I wouldn't have been able 
to create such a thing.
And even if I had written such a thing, I wouldn't dare use it in 
production, because it deals with Ansible internals in a way that screams, 
"Subject to change without notice."

The docs have improved tremendously in that time, by the way. It's not 
surprising that internal interfaces aren't as well documented as the user 
side of the bread-n-butter modules, especially as so much has changed in 
that time. That rapid change isn't slowing, though, which makes tying 
business process to those internals even less attractive.

> I would not 'manage it beforehand'…
No, you wouldn't, because you are as familiar with the magic behind the 
mirror - the internal workings of Ansible - as the rest of us who are 
limited to the user-facing side. But the rest of us have to build our 
wheels with the stick-n-stones within our reach.

> …as you do not know if a backup is needed…
He wants to ensure he has a copy of the file. That's not a big ask.
It isn't a backup, because "backup" implies a lot of things that aren't 
true. They also aren't true when you say "backup: true" on a task. But 
that's beside the point. If the user creates and maintains a copy with the 
copy module, with the dest name that includes the src's mtime, and does it 
before potentially clobbering the live conf file, then he's effectively 
done what "backup: true" would do, except he hasn't trashed his conf 
directory. Also, the copy task wouldn't make another copy if the file 
hadn't changed. It would only do anything if either the contents or the 
mtime changed, which is really all he's asking for.


On Monday, October 30, 2023 at 11:34:33 AM UTC-4 Brian Coca wrote:

> The backup system returns the 'backup_file' information so you can
> then operate on the built in backup, like moving it to a central
> location on the target, copying it to a backup server and/or managing
> the number/recency of backups.
>
> I would not 'manage it beforehand' as you do not know if a backup is
> needed until after the task that creates the backup runs, mtime is not
> the best indicator.
>
> -- 
> --
> Brian Coca
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ansible-project+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/ansible-project/9feb79ed-cd9e-42a5-8093-a923a6ecd89en%40googlegroups.com.

Re: [ansible-project] Re: how to set different backup directory than current while using backup=yes

[Brian Coca]

This is an example role I made https://github.com/bcoca/local_backup.
this is just ONE way to do it. If using awx/Tower/Controller I would
recommend creating a workflow.




-- 
--
Brian Coca

-- 
You received this message because you are subscribed to the Google Groups 
"Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ansible-project+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/ansible-project/CACVha7crLUfm%2BH42OvM%3D723t%2Bvxh2HncbbVcG9wvgiY2tfR_NA%40mail.gmail.com.

Re: [ansible-project] Re: how to set different backup directory than current while using backup=yes

[solarflow99]

Is there any code example of this?



On Mon, Oct 30, 2023 at 8:34 AM Brian Coca  wrote:

> The backup system returns the 'backup_file'  information so you can
> then operate on the built in backup, like moving it to a central
> location on the target, copying it to a backup server and/or managing
> the number/recency of backups.
>
> I would not 'manage it beforehand' as you do not know if a backup is
> needed until after the task that creates the backup runs, mtime is not
> the best indicator.
>
> --
> --
> Brian Coca
>
> --
> You received this message because you are subscribed to the Google Groups
> "Ansible Project" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ansible-project+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/ansible-project/CACVha7fwrfeeatAO%3DwvtKG3eVPuH0x-LPtRnfZ_o0sem2B%3DGOQ%40mail.gmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ansible-project+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/ansible-project/CAO8i5OL3bHQW3FQre-6Z0YbcMYoo5NDqrr9wyG3SzpeCfqe2SQ%40mail.gmail.com.

Re: [ansible-project] Re: how to set different backup directory than current while using backup=yes

[Brian Coca]

The backup system returns the 'backup_file'  information so you can
then operate on the built in backup, like moving it to a central
location on the target, copying it to a backup server and/or managing
the number/recency of backups.

I would not 'manage it beforehand' as you do not know if a backup is
needed until after the task that creates the backup runs, mtime is not
the best indicator.

-- 
--
Brian Coca

-- 
You received this message because you are subscribed to the Google Groups 
"Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ansible-project+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/ansible-project/CACVha7fwrfeeatAO%3DwvtKG3eVPuH0x-LPtRnfZ_o0sem2B%3DGOQ%40mail.gmail.com.

[ansible-project] Re: how to set different backup directory than current while using backup=yes

[Todd Lewis]

That's not an option.
Your best bet is to handle the backup yourself before invoking 
ansible.builtin.copy or ansible.builtin.template etc. and don't specify 
backup (or "backup: false").

Use ansible.builtin.stat to get the mtime.
Then use ansible.builtin.copy to copy the .conf file where you want it. 
Specify "remote_src: true" and use the registered mtime to provide an 
extension to "dest:" that looks like what "backup: true" does. Or make it 
look different if you don't like it that way.
You can wrap all that up into a task file that you can pass variables to so 
it's easy to use for multiple files.

On Monday, October 30, 2023 at 9:29:50 AM UTC-4 Sameer Modak wrote:

> how to set different backup directory than current while using backup=yes 
> in ansible modules.
>
>
> It takes backup in current directory with .conf.date which actually valid 
> for application .conf anything
>
> do we have any way where we can give specific backup directory path 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ansible-project+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/ansible-project/9c663ed6-9132-480b-8cd3-025b00e48695n%40googlegroups.com.