Re: [Fab-user] fabtest
Hey Mike, On Sat, Feb 19, 2011 at 5:37 AM, Mike Korobov kmik...@googlemail.com wrote: I've recently released https://bitbucket.org/kmike/fabtest/ package. This package is for writing tests for Fabric scripts against VirtualBox VMs: before each test VM is rolled back to selected snapshot. That's neat! I haven't the time to try it out right now but I'm curious as to how long your snapshot rollbacks take (1s, 10s, 60s etc). I use Virtualbox right now (via Vagrant) but the snapshot functionality was somewhat buggy for me so I've not used it much. Instead I use Vagrant's package functionality (which is semi slow but pretty stable) and/or focus on idempotent Fabric recipes. Can please somebody review the code? Are there better ways to do this kind of things? Am I missing something in the implementation? For running Fabric's own tests, I wrote a somewhat hacky Paramiko-driven fake SSH/SFTP server -- it's in the master branch in tests/server.py (with to-be-merged improvements to the SFTP stuff in put-get-overhaul). It works reasonably well for the existing test suite, but needs more work to be generally useful for your use case, I think. I have vague plans to clean it up and make it its own project sometime. The greater discussion of fake SSH server versus real SSH server is one I had a while ago and I'm reasonably happy with the route I took for Fabric's test suite, but for manual testing of fabfile tasks (as opposed to frequent runs of a large test suite) I think your approach may have a lot of merit too. Again, I'll try to take a closer look in a while. -Jeff -- Jeff Forcier Unix sysadmin; Python/Ruby engineer http://bitprophet.org ___ Fab-user mailing list Fab-user@nongnu.org http://lists.nongnu.org/mailman/listinfo/fab-user
Re: [Fab-user] fabtest
Hi Jeff, I also found VirtualBox 4.0.0 and 4.0.2 buggy but with recent 4.0.4 release stability seems to improve a lot. Snapshots taking doesn't work for me if VM is running in headless mode but in standard mode (when window pops up and closes) all was fine. This doesn't matter much because snapshots are taken before tests, during tests snapshots are only gets restored and restoring works fine for me even in headless mode. Snapshots are quite fast: rolling back a snapshot and restoring the another snapshot takes a few seconds (immediately failing test take about 2-5 second to run on macbook 2008), this time is small compared to the time some tests itself take (I'm testing package installation and pre-configuration on several different linux distributions so it involve network operations). The nice thing about snapshots is that OS doesn't have to boot because RAM content is restored too. I definitely agree this approach is not for everyone and fake ssh server make sense because it is times faster. Tests against real boxes should be run only when necessary. Looking forward for your future ssh testing project! Fabtest is a small new project (sloccount reports less than 100 lines of code: VirtualBox api wrapper + fabric utils + a couple of base TestCases). Not so many tests are written using it now (it's the test suite for django-fab-deploy), fabtest needs more battle-testing and some rough edges should be expected, but for existing test suite it just works. Didn't know about Vargant, thanks for mentioning it, looks interesting and polished. 2011/2/22 Jeff Forcier j...@bitprophet.org: Hey Mike, On Sat, Feb 19, 2011 at 5:37 AM, Mike Korobov kmik...@googlemail.com wrote: I've recently released https://bitbucket.org/kmike/fabtest/ package. This package is for writing tests for Fabric scripts against VirtualBox VMs: before each test VM is rolled back to selected snapshot. That's neat! I haven't the time to try it out right now but I'm curious as to how long your snapshot rollbacks take (1s, 10s, 60s etc). I use Virtualbox right now (via Vagrant) but the snapshot functionality was somewhat buggy for me so I've not used it much. Instead I use Vagrant's package functionality (which is semi slow but pretty stable) and/or focus on idempotent Fabric recipes. Can please somebody review the code? Are there better ways to do this kind of things? Am I missing something in the implementation? For running Fabric's own tests, I wrote a somewhat hacky Paramiko-driven fake SSH/SFTP server -- it's in the master branch in tests/server.py (with to-be-merged improvements to the SFTP stuff in put-get-overhaul). It works reasonably well for the existing test suite, but needs more work to be generally useful for your use case, I think. I have vague plans to clean it up and make it its own project sometime. The greater discussion of fake SSH server versus real SSH server is one I had a while ago and I'm reasonably happy with the route I took for Fabric's test suite, but for manual testing of fabfile tasks (as opposed to frequent runs of a large test suite) I think your approach may have a lot of merit too. Again, I'll try to take a closer look in a while. -Jeff -- Jeff Forcier Unix sysadmin; Python/Ruby engineer http://bitprophet.org ___ Fab-user mailing list Fab-user@nongnu.org http://lists.nongnu.org/mailman/listinfo/fab-user
Re: [Fab-user] fabtest
On Tue, Feb 22, 2011 at 2:02 PM, Mike Korobov kmik...@googlemail.com wrote: Hi Jeff, I also found VirtualBox 4.0.0 and 4.0.2 buggy but with recent 4.0.4 release stability seems to improve a lot. Good to know, I appear to be using 4.0.2 atm. Snapshots taking doesn't work for me if VM is running in headless mode but in standard mode (when window pops up and closes) all was fine. This doesn't matter much because [...] restoring works fine for me even in headless mode. Also good to know; Vagrant uses headless by default (not even sure if there is an option for non-headless) but if restoration can be done in that mode I might still check that out. Thanks for the info! -Jeff 2011/2/22 Jeff Forcier j...@bitprophet.org: Hey Mike, On Sat, Feb 19, 2011 at 5:37 AM, Mike Korobov kmik...@googlemail.com wrote: I've recently released https://bitbucket.org/kmike/fabtest/ package. This package is for writing tests for Fabric scripts against VirtualBox VMs: before each test VM is rolled back to selected snapshot. That's neat! I haven't the time to try it out right now but I'm curious as to how long your snapshot rollbacks take (1s, 10s, 60s etc). I use Virtualbox right now (via Vagrant) but the snapshot functionality was somewhat buggy for me so I've not used it much. Instead I use Vagrant's package functionality (which is semi slow but pretty stable) and/or focus on idempotent Fabric recipes. Can please somebody review the code? Are there better ways to do this kind of things? Am I missing something in the implementation? For running Fabric's own tests, I wrote a somewhat hacky Paramiko-driven fake SSH/SFTP server -- it's in the master branch in tests/server.py (with to-be-merged improvements to the SFTP stuff in put-get-overhaul). It works reasonably well for the existing test suite, but needs more work to be generally useful for your use case, I think. I have vague plans to clean it up and make it its own project sometime. The greater discussion of fake SSH server versus real SSH server is one I had a while ago and I'm reasonably happy with the route I took for Fabric's test suite, but for manual testing of fabfile tasks (as opposed to frequent runs of a large test suite) I think your approach may have a lot of merit too. Again, I'll try to take a closer look in a while. -Jeff -- Jeff Forcier Unix sysadmin; Python/Ruby engineer http://bitprophet.org -- Jeff Forcier Unix sysadmin; Python/Ruby engineer http://bitprophet.org ___ Fab-user mailing list Fab-user@nongnu.org http://lists.nongnu.org/mailman/listinfo/fab-user
