Yeah this is most definitely not a Tower specific thing since it's just running Ansible underneath -- but it's not something we have been seeing.
I'd say run things periodically and avoid use of the Atlantis or Pompeii availability zones? :) On Fri, Sep 12, 2014 at 12:05 PM, Gregory Taylor <[email protected]> wrote: > > > On Fri, Sep 12, 2014 at 11:45 AM, Michael DeHaan <[email protected]> > wrote: > >> >> You can definitely consider running the Ansible control machine *inside* >> EC2, where connections will be more reliable (and also faster), which is >> something I usually recommend to folks. >> > > We run an Ansible Tower instance in EC2 that runs these tasks. This is > where we are seeing the issues. We've tried running the playbooks from a > few different host machines on there, but we always eventually run into the > periodic SSH network failure where subsequent retries eventually work. > > >> Another thing is when spinning up new instances, using the "wait_for" >> trick, be sure to put a sleep in after the wait_for. SSH ports can come >> up but not be quite ready, which gives the appearance of SSH failure. I'm >> wondering if that might be part of it, or if you're seeing connection >> issues at effectively random points or just those. >> > > While we do use Ansible for provisioning new instances, that's not where > we're seeing the issue. It's our playbooks for rolling out code updates. > We're just SSH'ing into each (existing) app server, transferring the > updated code, and running a process restart. So by the time we run these > playbooks, the instances could be hours or days or months old at that > point, making the port readiness issue a non-factor. > > Most of the time the EC2 network is fast and reliable, but we deploy > frequently and do run into these issues from time to time. This is > consistent with the errors we've seen with our app servers temporarily > being unable to reach ElastiCache instances. Failure is just one of those > things we have to live with and build for in EC2. > > -- > You received this message because you are subscribed to the Google Groups > "Ansible Project" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/ansible-project/CAA0B%3D%3DStjXCd7F7SHM4xXRdcA00NJVUmFLCTA8MEkmjWhP%2BOSA%40mail.gmail.com > <https://groups.google.com/d/msgid/ansible-project/CAA0B%3D%3DStjXCd7F7SHM4xXRdcA00NJVUmFLCTA8MEkmjWhP%2BOSA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Ansible Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CA%2BnsWgyrVDncWBXS87cn4PB3ohGKX_u%3DSYioXXPYGjnvvwMRTQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
