@Matt,
Got another question in concurrency support in Ansible.
Is there any way to limit the number of processes that could be spawned on
a given host?
My requirement is not to execute the commands/scripts remotely. In my case,
the whole play needs to be executed on locahost only.
I have tried a simple test program and noticed that there are as many as 6
processes are spawned to execute 'sleep 20' asynchronously.
Please kindly revert. Thank you inadvance.
*Command:* ansible-playbook test_playbook.yml --forks=1
*Processes:*
root 69484 34309 9 04:50 pts/10 00:00:00 /usr/bin/python2 /usr/bin/
*ansible*-playbook test_playbook.yml --forks=1
root 69509 1 0 04:50 ? 00:00:00 /usr/bin/python2 /root/.
*ansible*/tmp/*ansible*-tmp-1591876209.82-38354017880191/async_wrapper.py
198806654079 50
/root/.*ansible*/tmp/*ansible*-tmp-1591876209.82-38354017880191/command.py
_
root 69510 69509 0 04:50 ? 00:00:00 /usr/bin/python2 /root/.
*ansible*/tmp/*ansible*-tmp-1591876209.82-38354017880191/async_wrapper.py
198806654079 50
/root/.*ansible*/tmp/*ansible*-tmp-1591876209.82-38354017880191/command.py
_
root 69511 69510 0 04:50 ? 00:00:00 /usr/bin/python2 /root/.
*ansible*/tmp/*ansible*-tmp-1591876209.82-38354017880191/command.py
root 69512 69511 1 04:50 ? 00:00:00 /usr/bin/python2 /tmp/
*ansible*_f9ckPD/*ansible*_module_command.py
root 69520 69484 3 04:50 pts/10 00:00:00 /usr/bin/python2 /usr/bin/
*ansible*-playbook test_playbook.yml --forks=1
*Code:*
[root@oracle-siha file_copy_test]# cat test_playbook.yml
- name: Testing processes
gather_facts: no
hosts: localhost
tasks:
- name: run sleep command
async: 50
poll: 0
command: sleep 20
register: res
- name: wait for the completion
async_status:
jid: "{{ res.ansible_job_id }}"
register: output
until: output.finished
delay: 5
retries: 10
On Friday, June 5, 2020 at 9:50:19 AM UTC+5:30, Jagadeeshkumar Dittakavi
wrote:
>
> Thank you Matt for the detailed and quick reply.. Much appreciated the
> support from the community.
>
> On Friday, June 5, 2020 at 12:29:30 AM UTC+5:30, Matt Martz wrote:
>>
>> Yes, it would utilize the threading library in Python. The GIL is a
>> primary cause to the CPU restrictions. Our main process that orchestrates
>> all of the task executions is already heavily CPU bound, so adding
>> additional threads to the same core can cause a decrease in performance.
>> Assuming we create a process model plugin type, other process models are
>> possible, such as using asyncio, concurrent.futures, gevent, etc. But I
>> don't expect this work to be complete any time soon.
>>
>> So for now, consider forking the only process model for the near future.
>>
>> On Thu, Jun 4, 2020 at 1:51 PM Jagadeeshkumar Dittakavi <
>> [email protected]> wrote:
>>
>>> Thank you for the prompt reply.. Just a curious question: Is the
>>> threading work that is underway based on python threads or pthreads or any
>>> other threading mechanism? As you mentioned that the threading model is not
>>> going to be performant, was the reason being the python's GIL?
>>>
>>>
>>> On Friday, June 5, 2020 at 12:01:14 AM UTC+5:30, Matt Martz wrote:
>>>>
>>>> The only current process model is forking. There has been some work
>>>> done to add a threaded process model, but there are some large hurdles to
>>>> overcome.
>>>>
>>>> In practice, it is not necessarily more performant, and in many cases
>>>> it was less performant, as it causes more CPU contention on a single core
>>>> that is already resource constrained.
>>>>
>>>> On Thu, Jun 4, 2020 at 1:18 PM Jagadeeshkumar Dittakavi <
>>>> [email protected]> wrote:
>>>>
>>>>> I am a newbie to ansible but I got to explore how to run a tasks in
>>>>> parallel by spawing a thread for each task instead of a process. My
>>>>> requirement is to run the playbook on my localhost and there is no remote
>>>>> task execution needed.
>>>>> I also would like to wait for all threads to complete before I move on
>>>>> to a task that has to be serialised.
>>>>>
>>>>> Can I chose thread vs process when it comes to parallel task execution?
>>>>> If it is possible to spawn threads from ansible, are they equivalent
>>>>> to python greenthreads or pthreads or something else?
>>>>>
>>>>> Thank you in advance!
>>>>>
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Ansible Project" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/ansible-project/0d59cf96-b053-4390-8dbb-663c70403104o%40googlegroups.com
>>>>>
>>>>> <https://groups.google.com/d/msgid/ansible-project/0d59cf96-b053-4390-8dbb-663c70403104o%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>> .
>>>>>
>>>>
>>>>
>>>> --
>>>> Matt Martz
>>>> @sivel
>>>> sivel.net
>>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Ansible Project" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/ansible-project/48f0ed5f-83c7-4bf4-a991-94ec78ce832eo%40googlegroups.com
>>>
>>> <https://groups.google.com/d/msgid/ansible-project/48f0ed5f-83c7-4bf4-a991-94ec78ce832eo%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>
>>
>> --
>> Matt Martz
>> @sivel
>> sivel.net
>>
>
--
You received this message because you are subscribed to the Google Groups
"Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/ansible-project/f0093a03-d20b-4187-8bca-913e10770f5fo%40googlegroups.com.
