What happens if you use shell instead of command? I'm not 100% certain but I think what's happening is that since you're using the ansible command module instead of the shell module, it is running the equivalent of: cat '/var/lib/pgsql/temp/truncate_sql' '/var/lib/pgsql/temp/postcodesio-*' '|' 'psql' 'postcodesiodb'
That's sending the entire output of the first couple files to stdout which the command module is then buffering to send back to the controller. If you use the shell module then the pipe will actually be interpreted by the shell and so it will do what you want. (If I'm wrong about why the MemoryError is happening, there might be another problem in addition but I am hopeful that it's all tied together). -Toshio On Tue, Nov 26, 2019 at 9:12 AM jonin <sebastian.coll...@gmail.com> wrote: > > Hi, > > I have created the following task to be run on a remote host: > > - hosts: nodes > remote_user: root > tasks: > - name: Step Seven - remove all current postcode data and then > re-populate the database with most recent data > become_user: postgres > become: yes > command: "cat /var/lib/pgsql/temp/truncate_sql > /var/lib/pgsql/temp/postcodesio-* | psql postcodesiodb" > > but it is failing with the below memory issue. A google makes me suspect it > is because I am trying to make ansible handle too large a file (a postcode > database) which it is loading into RAM. : > https://stackoverflow.com/questions/41367278/how-to-fix-memory-error-in-ansible/51320108. > The suggestion is to use the synchronise module or rsync directly as a > command. For either I am struggling to visualise the syntax / structure to do > this. Any help much appreciated. > > TASK [Step Seven - remove all current postcode data and then re-populate the > database with most recent data] > ***************************************************************** > fatal: [vm015018.bskyb.com]: FAILED! => {"changed": false, "module_stderr": > "Shared connection to vm015018.bskyb.com closed.\r\n", "module_stdout": > "Traceback (most recent call last):\r\n File > \"/tmp/ansible_ar0FKd/ansible_module_command.py\", line 213, in <module>\r\n > main()\r\n File \"/tmp/ansible_ar0FKd/ansible_module_command.py\", line > 207, in main\r\n module.fail_json(msg='non-zero return code', > **result)\r\n File > \"/tmp/ansible_ar0FKd/ansible_modlib.zip/ansible/module_utils/basic.py\", > line 2294, in fail_json\r\n File > \"/tmp/ansible_ar0FKd/ansible_modlib.zip/ansible/module_utils/basic.py\", > line 2273, in _return_formatted\r\n File > \"/tmp/ansible_ar0FKd/ansible_modlib.zip/ansible/module_utils/basic.py\", > line 2227, in jsonify\r\n File > \"/tmp/ansible_ar0FKd/ansible_modlib.zip/ansible/module_utils/basic.py\", > line 782, in jsonify\r\n File \"/usr/lib64/python2.7/json/__init__.py\", > line 250, in dumps\r\n sort_keys=sort_keys, **kw).encode(obj)\r\n File > \"/usr/lib64/python2.7/json/encoder.py\", line 207, in encode\r\n chunks = > self.iterencode(o, _one_shot=True)\r\n File > \"/usr/lib64/python2.7/json/encoder.py\", line 270, in iterencode\r\n > return _iterencode(o, 0)\r\nMemoryError\r\n", "msg": "MODULE FAILURE", "rc": > 1} > to retry, use: --limit > @/home/sco94/mwe-playbooks/playbooks/patching/postcodes.retry > > > -- > You received this message because you are subscribed to the Google Groups > "Ansible Development" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to ansible-devel+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/ansible-devel/6ff52565-cbdd-4dca-96ee-85382530d7b5%40googlegroups.com. -- You received this message because you are subscribed to the Google Groups "Ansible Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to ansible-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-devel/CAPrnkaTAj53zpokFtfJTOt3RfQBL8cyf6_SgeTwzfKOyVPHBhg%40mail.gmail.com.
