Damon -- I've been getting occasional stack overflow and out of memory exceptions when testing against Server 2008 R2 without updates. See https://github.com/ansible/ansible/pull/8345#issuecomment-52074837 for a hotfix that might work.
Running with -vvvvv when using winrm provides even more output, including all of stderr if something goes wrong, whereas -vvvv only displays a truncated version (e.g. WINRM RESULT <Response code 0, out "....", err "......">). Make sure you're running the latest pywinrm (pip install -U http://github.com/diyan/pywinrm/archive/master.zip#egg=pywinrm). I'd submitted some fixes earlier this week related to timeouts; previously you would get 500 errors for any command taking longer than 60 seconds. On Thu, Aug 28, 2014 at 6:06 PM, Damon Overboe <[email protected]> wrote: > Well I'm consistently getting StackOverflows now too, so I think the two > might be related; some kind of infinite loop that sometimes runs out of > memory before it can overflow the stack. > > The VMs should have plenty of memory, but, then again some of the VMs we > have are a bit laggy, so I'm trying some other servers too. > > I'm a bit fried though, so I'm probably going to put this on hold for > tonight, I'll hit it clean tomorrow. > > > I have the latest from your win_package branch, plus the patch to > powershell. I also have a clean checkout from the ansible devel branch, and > I'm seeing it in both. > > To be safe, I think tomorrow I'll do clean checkouts of both again, just > to make sure I haven't hosed something on my end. > > > > > On Thursday, August 28, 2014 4:05:36 PM UTC-5, Trond Hindenes wrote: > >> Out of memory? For real? I havent seen that since the 90's. Put some more >> ram in your box? >> >> On Thursday, August 28, 2014 7:38:24 PM UTC+2, Damon Overboe wrote: >> >> "seen it without that as well" = seen the StackOverflow on the mainline >> dev branch, using win_msi and the powershell.ps1 from the mainline as well. >> >> On Thursday, August 28, 2014 12:26:44 PM UTC-5, Damon Overboe wrote: >> >> Cool I just hit the StackOverflow again. >> >> I used win_get_url to download, and then win_package to install. I am >> using the updated powershell.ps1 script from 8759, but I have seen it >> without that as well. >> >> failed: [clone] => {"failed": true, "parsed": false} >> invalid output was: >> Process is terminated due to StackOverflowException. >> >> >> FATAL: all hosts have already failed -- aborting >> ____________ >> < PLAY RECAP > >> ------------ >> \ ^__^ >> \ (oo)\_______ >> (__)\ )\/\ >> ||----w | >> || || >> >> >> to retry, use: --limit @/home/damon/slaves-svn-package.retry >> >> clone : ok=2 changed=1 unreachable=0 failed=1 >> >> >> The only two files in the temp directory on the remote are the >> `arguments` file and `powershell.ps1`. Here are the contents of the args >> file: >> >> {"arguments": "ADDLOCAL=CLI", "creates": "C:\\Program >> Files\\TortoiseSVN", "ensure": "present", "name": "TortoiseSVN-1.7.15", >> "path": "E:\\.installs\\tortoise-1.7.msi", "product_id": >> "{087D2CC7-C1FA-45EC-9FC1-B06CA4A073A0}"} >> >> I'm going to try changing up the values a bit and see if it's bad >> arguments going into it. Here is the playbook (currently), and I'm also >> going to try rerunning with -vvvv >> >> - name: get svn client - version 1.7 >> win_get_url: >> url: 'http://sourceforge.net/projects/tortoisesvn/files/1. >> 7.15/Application/Tor$ >> dest: 'E:\.installs\tortoise-1.7.msi' >> >> - name: ensure svn is version 1.7 >> win_package: >> path='E:\.installs\tortoise-1.7.msi' >> name='TortoiseSVN-1.7.15' >> product_id='{087D2CC7-C1FA-45EC-9FC1-B06CA4A073A0}' >> creates='C:\Program Files\TortoiseSVN' >> ensure=present >> arguments="ADDLOCAL=CLI" >> >> >> >> >> On Thursday, August 28, 2014 10:59:29 AM UTC-5, Damon Overboe wrote: >> >> I changed it to run win_get_url to download the package so I could rename >> the file (using 'dest'). >> >> And then, I hit a nice System.OutOfMemoryException. This server isn't >> that beefy and it's wanting to run updates, so it could be that... but I'm >> going to try to grab the remote files anyway. >> >> I wasn't running -vvvv, here's what I saw: >> >> failed: [clone] => {"failed": true, "parsed": false} >> invalid output was: Exception of type 'System.OutOfMemoryException' was >> thrown. >> At C:\Users\dummy\AppData\Local\Temp\ansible-tmp-1409241114. >> 17-71859971213315 >> \win_package.ps1:1395 char:5 >> + if ($testdscresult -eq $true) >> + ~~~~~~~~~~~~~~~~~~~~~~~~ >> + CategoryInfo : OperationStopped: (:) [], >> OutOfMemoryException >> + FullyQualifiedErrorId : System.OutOfMemoryException >> >> >> >> FATAL: all hosts have already failed -- aborting >> ____________ >> < PLAY RECAP > >> ------------ >> \ ^__^ >> \ (oo)\_______ >> (__)\ )\/\ >> ||----w | >> || || >> >> >> to retry, use: --limit @/home/damon/slaves-svn-package.retry >> >> clone : ok=2 changed=1 unreachable=0 failed=1 >> >> >> >> >> On Thursday, August 28, 2014 10:44:52 AM UTC-5, Damon Overboe wrote: >> >> OK I'm testing it out. >> >> First, I love that you can pass arguments. I have been automating the >> Jenkins clients; I'm adding a specific version of SVN, and with the win_msi >> installer, I couldn't figure out how to pass arguments to it so it would >> install the Command Line Interface. >> >> With the win_package, that is super easy, so I was happy to see that! >> >> >> But I ran into my first situation; if you give it a URL and the download >> file doesn't include the extension, it will fail to install. Test out this >> url: >> >> http://sourceforge.net/projects/tortoisesvn/files/1. >> 7.15/Application/TortoiseSVN-1.7.15.25753-x64-svn-1.7.18.msi/download >> >> Running a plain wget, `wget http://sourceforge.net/ >> projects/tortoisesvn/files/1.7.15/Application/TortoiseSVN- >> 1.7.15.25753-x64-svn-1.7.18.msi/download` >> <http://sourceforge.net/projects/tortoisesvn/files/1.7.15/Application/TortoiseSVN-1.7.15.25753-x64-svn-1.7.18.msi/download> >> will download the installer as "download" so I'm assuming the scripts are >> doing the same. >> >> So, it probably needs an option to specify the output file, much like you >> would with `wget -o` or `wget --output-file`. >> >> >> >> >> >> >> >> On Tuesday, August 26, 2014 6:25:56 PM UTC-5, Damon Overboe wrote: >> >> Awesome I'll give that as shot. >> >> The biggest thing I wanted to know was how to catch the output so I can >> at least try to debug it or ask for help, so that'll be huge. >> >> I'm off for the night tonight but I'll be testing out win_package >> tomorrow for sure. I'll let you know how it goes. >> >> On Tuesday, August 26, 2014 5:53:24 PM UTC-5, Trond Hindenes wrote: >> >> Nice, >> So, Chris just tipped me about this ansible.cfg setting: >> [defaults] >> keep_remote_files = 1 >> >> Stick that in your /ansible/etc/ansible.cfg and re-run your playbook with >> -vvv >> >> All those temp paths Ansible uses for powershell files will now be left >> on your windows system. Each folder basically has a script file (the >> module) and an arguments file. That should do wonders for debugging windows >> modules (it would still require some PowerShell proficiency tho) >> >> >> >> >> On Wednesday, August 27, 2014 12:48:36 AM UTC+2, Damon Overboe wrote: >> >> There it is, finally!!! >> >> This is still using the win_msi, I'll switch to the win_package now. >> >> I'm not sure why but I just wasn't going to be happy until I broke it. >> >> >> >> changed: [node1.domain.com] >> Traceback (most recent call last): >> File "/home/damon/src/ansible/lib/ansible/runner/connection_ >> plugins/winrm.py", line 149, in exec_command >> result = self._winrm_exec(cmd_parts[0], cmd_parts[1:], from_exec=True >> ) >> File "/home/damon/src/ansible/lib/ansible/runner/connection_ >> plugins/winrm.py", line 118, in _winrm_exec >> response = Response(self.protocol.get_command_output(self.shell_id, >> command_id)) >> File "/usr/local/lib/python2.7/dist-packages/winrm/protocol.py", line >> 277, in get_command_output >> self._raw_get_command_output(shell_id, command_id) >> File "/usr/local/lib/python2.7/dist-packages/winrm/protocol.py", line >> 293, in _raw_get_command_output >> rs = self.send_message(xmltodict.unparse(rq)) >> File "/usr/local/lib/python2.7/dist-packages/winrm/protocol.py", line >> 173, in send_message >> return self.transport.send_message(message) >> File "/usr/local/lib/python2.7/dist-packages/winrm/transport.py", line >> 90, in send_message >> raise WinRMTransportError(error_message) >> WinRMTransportError: Bad HTTP response returned from server. Code 500 >> fatal: [node3.domain.com] => failed to exec cmd PowerShell -NoProfile - >> NonInteractive -EncodedCommand KABOAGUAdwAtAEkAdABlAG0AIAAtAF >> QAeQBwAGUAIABEAGkAcgBlAGMAdABvAHIAeQAgAC0AUABhAHQAaAAgACQAZQ >> BuAHYAOgB0AGUAbQBwACAALQBOAGEAbQBlACAAIgBhAG4AcwBpAGIAbABlAC >> 0AdABtAHAALQAxADQAMAA5ADAAOQAyADkAOAA2AC4ANgAtADIANwA2ADMAMA >> A3ADAAMAA2ADUAMgA0ADEAOAAiACkALgBGAHUAbABsAE4AYQBtAGUAIAB8AC >> AAVwByAGkAdABlAC0ASABvAHMAdAAgAC0AUwBlAHAAYQByAGEAdABvAHIAIAAnACcAOwA= >> _________________________________ >> < TASK: ensure svn is version 1.7 > >> --------------------------------- >> \ ^__^ >> \ (oo)\_______ >> (__)\ )\/\ >> ||----w | >> || || >> >> >> failed: [node1.domain.com] => {"failed": true, "parsed": false} >> invalid output was: >> Process is terminated due to StackOverflowException. >> >> >> FATAL: all hosts have already failed -- aborting >> ____________ >> < PLAY RECAP > >> ------------ >> \ ^__^ >> \ (oo)\_______ >> (__)\ )\/\ >> ||----w | >> || || >> >> >> to retry, use: --limit @/home/damon/slaves-svn-cleanup.retry >> >> node1.domain.com : ok=1 changed=1 unreachable=0 failed=1 >> node3.domain.com : ok=0 changed=0 unreachable=1 failed=0 >> >> >> >> On Tuesday, August 26, 2014 5:40:07 PM UTC-5, Trond Hindenes wrote: >> >> This thread is fine. I haven't submitted a PR for win_package just yet. >> >> On Wednesday, August 27, 2014 12:36:19 AM UTC+2, Damon Overboe wrote: >> >> That's exactly what I did to get your changes into the mainline, so I'm >> doing that again. >> >> You want me to post feedback (if I have any) in this thread or on the >> pull request? >> >> On Tuesday, August 26, 2014 5:25:46 PM UTC-5, Trond Hindenes wrote: >> >> Me too. I would just grab the file manually and stick it in your on fork >> for now: >> https://raw.githubusercontent.com/trondhindenes/ansible/win_ >> setup_json_depth/lib/ansible/module_utils/powershell.ps1 >> >> My repo has a branch for each change/PR i've submitted, so you're >> probably better off forking the original ansible code and just come up with >> your own way of managing the various changes you put in it. I guess I >> should try and rebase everything back into my devel branch, but it's that's >> not the case right now. >> >> On Wednesday, August 27, 2014 12:20:37 AM UTC+2, Damon Overboe wrote: >> >> I'm still somewhat of a git noob; >> >> I've cloned your repo, but I'm not sure how to get the pull request from >> 8759 into that clone... >> >> I'm guessing there's a command to merge, any pointers? >> >> On Tuesday, August 26, 2014 5:15:45 PM UTC-5, Trond Hindenes wrote: >> >> Here you go sir: >> https://github.com/trondhindenes/ansible/tree/win_package >> >> Note that I'm using the updated helper functions from the main >> powershell.ps1 script, so you'll need to grab that as well. You can grab if >> from this PR: >> https://github.com/ansible/ansible/pull/8759 (the powershell.ps1 file >> goes in /lib/ansible/module_utils in your ansible directory structure) >> >> On Wednesday, August 27, 2014 12:07:09 AM UTC+2, Trond Hindenes wrote: >> >> Cool, I'm writing up a short doc and putting it on a branch in my ansible >> fork right now. Will send you links and stuffs in a couple of minutes. >> >> On Wednesday, August 27, 2014 12:00:08 AM UTC+2, Damon Overboe wrote: >> >> Yah that'd be great. I'll happily test it out if you like. >> >> I'm only targeting Svr 12s, and just using it right now to set up new >> build agent nodes. I want to do a lot more with it but figured that would >> be a good place to start. >> >> On Tuesday, August 26, 2014 4:55:53 PM UTC-5, Trond Hindenes wrote: >> >> BTW, i have a brand spankin new win_package module coming up which will >> handle not only msis but all kinds of installs, and also supports >> auto-downloading source files from a url so that you don't need to >> win_get_url first. I'm waiting for some improvements to the powershell >> helper scripts to be merged before i PR that modu >> >> ... > > -- > You received this message because you are subscribed to the Google Groups > "Ansible Project" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/ansible-project/4603448b-a24a-49e8-8d8a-b1bb9770ad8e%40googlegroups.com > <https://groups.google.com/d/msgid/ansible-project/4603448b-a24a-49e8-8d8a-b1bb9770ad8e%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Ansible Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAH%2BKTJ7tV69tbEnZefFoAPPFn3SYhPT96YXtqOrLA0jtJH0Agw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
