Thanks. My ansible playbook does a system update, so that could be doing it.
Is there something like "expect_disconnect" that will work with ansible-local? On Friday, May 11, 2018 at 10:35:07 AM UTC-5, Rickard von Essen wrote: > > This happens because the ssh session is closed. (see > https://github.com/hashicorp/packer/blob/68287566b873c317144235a202c5e9da9f88f4f7/packer/communicator.go#L15). > > Most likely because sshd gets updated/restarted. > > On Fri, May 11, 2018, 15:34 Chris Stevens <[email protected] > <javascript:>> wrote: > >> I have an existing packer build that has been reliable for years, but >> just this week has become flaky with intermittent failures as I'll describe >> below. >> >> I don't have enough clarity into the issue right now, but wanted to >> describe it here in case others have a similar sounding issue. I'll add to >> this post as I have time to debug the issue further. >> >> The changes made since we last recall a stable build is updating to the >> latest versions for the build environment: >> Packer (1.2.3) >> VirtualBox (5.2.12) >> Ansible (2.5.2) >> >> We also moved up to the latest build target ( CentOS 7.5 (1804) from >> CentOS 7.4 (1708) ). >> >> I see the same flakiness if I revert back to the 1708 CentOS release. >> >> I see the same flakiness if I revert Packer back to 1.2.1. >> >> I doubled both the memory and CPU allocation that I normally use and >> still see the same issues. >> >> It is worth noting that we had 1 good build out of maybe 10 or 15 >> attempts. >> >> The most common failure mode is that the build will run properly for 10 >> or 20 minutes and then stall on random Ansible tasks within the provisioner >> for 2 to 5 or more minutes and then eventually fail as shown in the log >> dump below. >> >> 2018/05/10 11:21:51 ui: virtualbox-iso: >> virtualbox-iso: >> 2018/05/10 11:21:51 ui: virtualbox-iso: TASK [mysql : Install tools >> for Mysql] ***************************************** >> virtualbox-iso: TASK [mysql : Install tools for Mysql] >> ***************************************** >> 2018/05/10 11:31:17 packer: 2018/05/10 11:31:17 [ERROR] Remote command >> exited without exit status or exit signal. >> 2018/05/10 11:31:17 packer: 2018/05/10 11:31:17 [INFO] RPC endpoint: >> Communicator ended with: 2300218 >> 2018/05/10 11:31:17 [INFO] 32044 bytes written for 'stdout' >> 2018/05/10 11:31:17 [INFO] 885 bytes written for 'stderr' >> 2018/05/10 11:31:17 [INFO] RPC client: Communicator ended with: 2300218 >> 2018/05/10 11:31:17 [INFO] RPC endpoint: Communicator ended with: 2300218 >> 2018/05/10 11:31:17 packer: 2018/05/10 11:31:17 [INFO] 32044 bytes >> written for 'stdout' >> 2018/05/10 11:31:17 packer: 2018/05/10 11:31:17 [INFO] 885 bytes written >> for 'stderr' >> 2018/05/10 11:31:17 packer: 2018/05/10 11:31:17 [INFO] RPC client: >> Communicator ended with: 2300218 >> 2018/05/10 11:31:17 [INFO] (telemetry) ending ansible-local >> 2018/05/10 11:31:18 packer: 2018/05/10 11:31:18 Executing VBoxManage: >> []string{"controlvm", "packer-centos-7-x86_64", "poweroff"} >> 2018/05/10 11:31:18 packer: 2018/05/10 11:31:18 stdout: >> 2018/05/10 11:31:18 packer: 2018/05/10 11:31:18 stderr: >> 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% >> 2018/05/10 11:31:18 packer: 2018/05/10 11:31:18 Executing VBoxManage: >> []string{"storageattach", "packer-centos-7-x86_64", "--storagectl", "IDE >> Controller", "--port", "0", "--device", "1", "--medium", "none"} >> 2018/05/10 11:31:18 packer: 2018/05/10 11:31:18 stdout: >> 2018/05/10 11:31:18 packer: 2018/05/10 11:31:18 stderr: >> 2018/05/10 11:31:18 ui: ==> virtualbox-iso: Deregistering and deleting >> VM... >> ==> virtualbox-iso: Deregistering and deleting VM... >> 2018/05/10 11:31:18 packer: 2018/05/10 11:31:18 Executing VBoxManage: >> []string{"unregistervm", "packer-centos-7-x86_64", "--delete"} >> 2018/05/10 11:31:18 packer: 2018/05/10 11:31:18 stdout: >> 2018/05/10 11:31:18 packer: 2018/05/10 11:31:18 stderr: >> 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% >> 2018/05/10 11:31:18 ui: ==> virtualbox-iso: Deleting output directory... >> ==> virtualbox-iso: Deleting output directory... >> 2018/05/10 11:31:18 [INFO] (telemetry) ending virtualbox-iso >> 2018/05/10 11:31:18 ui error: Build 'virtualbox-iso' errored: Error >> executing Ansible: Non-zero exit status: 2300218 >> 2018/05/10 11:31:18 Builds completed. Waiting on interrupt barrier... >> 2018/05/10 11:31:18 machine readable: error-count []string{"1"} >> 2018/05/10 11:31:18 ui error: >> ==> Some builds didn't complete successfully and had errors: >> 2018/05/10 11:31:18 machine readable: virtualbox-iso,error >> []string{"Error executing Ansible: Non-zero exit status: 2300218"} >> 2018/05/10 11:31:18 ui error: --> virtualbox-iso: Error executing >> Ansible: Non-zero exit status: 2300218 >> 2018/05/10 11:31:18 ui: >> ==> Builds finished but no artifacts were created. >> 2018/05/10 11:31:18 [INFO] (telemetry) Finalizing. >> Build 'virtualbox-iso' errored: Error executing Ansible: Non-zero exit >> status: 2300218 >> >> ==> Some builds didn't complete successfully and had errors: >> --> virtualbox-iso: Error executing Ansible: Non-zero exit status: 2300218 >> >> ==> Builds finished but no artifacts were created. >> 2018/05/10 11:31:19 waiting for all plugin processes to complete... >> 2018/05/10 11:31:19 /usr/local/bin/packer: plugin process exited >> 2018/05/10 11:31:19 /usr/local/bin/packer: plugin process exited >> 2018/05/10 11:31:19 /usr/local/bin/packer: plugin process exited >> 2018/05/10 11:31:19 /usr/local/bin/packer: plugin process exited >> 2018/05/10 11:31:19 /usr/local/bin/packer: plugin process exited >> 2018/05/10 11:31:19 /usr/local/bin/packer: plugin process exited >> 2018/05/10 11:31:19 /usr/local/bin/packer: plugin process exited >> 2018/05/10 11:31:19 /usr/local/bin/packer: plugin process exited >> 2018/05/10 11:31:19 /usr/local/bin/packer: plugin process exited >> >> >> >> A less common failure mode occurs much earlier in the build run and looks >> like an Ansible module failure. >> >> virtualbox-iso: TASK [common : Update OS packages] >> ********************************************* >> virtualbox-iso: changed: [127.0.0.1] >> virtualbox-iso: >> virtualbox-iso: TASK [common : Install common OS tools / services] >> ***************************** >> virtualbox-iso: An exception occurred during task execution. To see >> the full traceback, use -vvv. The error was: AttributeError: 'Task' object >> has no attribute 'async_val' >> virtualbox-iso: fatal: [127.0.0.1]: FAILED! => {"msg": "Unexpected >> failure during module execution.", "stdout": ""} >> virtualbox-iso: to retry, use: --limit >> @/tmp/packer-provisioner-ansible-local/development.retry >> virtualbox-iso: >> virtualbox-iso: PLAY RECAP >> ********************************************************************* >> virtualbox-iso: 127.0.0.1 : ok=6 changed=4 >> unreachable=0 failed=1 >> virtualbox-iso: >> ==> virtualbox-iso: Deregistering and deleting VM... >> ==> virtualbox-iso: Deleting output directory... >> Build 'virtualbox-iso' errored: Error executing Ansible: Non-zero exit >> status: 2 >> >> ==> Some builds didn't complete successfully and had errors: >> --> virtualbox-iso: Error executing Ansible: Non-zero exit status: 2 >> >> >> Another less common failure mode. >> >> ==> virtualbox-iso: Provisioning with shell script: scripts/finalize.sh >> ==> virtualbox-iso: Deregistering and deleting VM... >> ==> virtualbox-iso: Deleting output directory... >> Build 'virtualbox-iso' errored: Retryable error: Error removing temporary >> script at /tmp/script_6974.sh: Timeout during SSH handshake >> >> ==> Some builds didn't complete successfully and had errors: >> --> virtualbox-iso: Retryable error: Error removing temporary script at >> /tmp/script_6974.sh: Timeout during SSH handshake >> >> >> -- >> This mailing list is governed under the HashiCorp Community Guidelines - >> https://www.hashicorp.com/community-guidelines.html. Behavior in >> violation of those guidelines may result in your removal from this mailing >> list. >> >> GitHub Issues: https://github.com/mitchellh/packer/issues >> IRC: #packer-tool on Freenode >> --- >> You received this message because you are subscribed to the Google Groups >> "Packer" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/packer-tool/f679de0a-06df-4667-b9b1-4919f70cc73a%40googlegroups.com >> >> <https://groups.google.com/d/msgid/packer-tool/f679de0a-06df-4667-b9b1-4919f70cc73a%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list. GitHub Issues: https://github.com/mitchellh/packer/issues IRC: #packer-tool on Freenode --- You received this message because you are subscribed to the Google Groups "Packer" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/f39d069e-ac12-49e3-b4c6-2aa14ab49ad7%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
