The problem needs to be fixed - "have tested a solution that was discussed with koushik" - we will have to create PR with that change. Once the PR is pushed - we will have one less blocker/test failure.
I agree this is not a cloudstack core code problem - but we should NOT make that a criteria for priority of a bug - just my take. We should include BVT failures in the Blocker criteria (whether the fix is in the core functionality/test script). Otherwise, we will not hit 100% passrate on BVTs. You are right - only a subset of these tests are being run on "travis" otherwise we will catch more or some of these failures. -----Original Message----- From: Sebastien Goasguen [mailto:run...@gmail.com] Sent: Wednesday, September 30, 2015 2:09 PM To: dev@cloudstack.apache.org Subject: Re: [Blocker] test duplicated in test_scale_vm.py > On Sep 30, 2015, at 10:28 AM, Raja Pullela <raja.pull...@citrix.com> wrote: > > Yup... btw, I am on top of this and goal is to get to 100% passrate on BVTs! > Yes I got it, and we all have the same goal. It does appear that not all smoke tests are run by Travis, otherwise we would catch any errors on simulator for every PR. My point is that if a failure in your BVT is due to the test itself it is not a cloudstack failure per se and hence not a blocker. Which all said and done is not a problem because it should be fixed now if I am understanding your assessment and Koushik report of this failure correctly. > -----Original Message----- > From: Sebastien Goasguen [mailto:run...@gmail.com] > Sent: Wednesday, September 30, 2015 1:48 PM > To: dev@cloudstack.apache.org > Subject: Re: [Blocker] test duplicated in test_scale_vm.py > > >> On Sep 30, 2015, at 10:15 AM, Raja Pullela <raja.pull...@citrix.com> wrote: >> >> Sebastien, any BVT tests failing is a blocker and needs to be fixed - which >> is the reason for Blocker. We already have a fix. >> > > Right, but that’s a fix in the test itself, no an issue with the cloudstack > “core”. > > In any case, great, one less blocker. > >> Koushik, >> >> modified the test "test_01_scale_vm" with required_hardware=false on >> simulator setup and it works. So, I think we can let the second method go. >> I will also test this on XS. >> Raja >> >> >> root@localhost:~/cloudstack# nosetests --with-marvin >> --marvin-config=/root/cloudstack/setup/dev/local1.cfg >> --zone=Sandbox-simulator --hypervisor=simulator -a >> tags=basic,required_hardware=false >> /root/cloudstack/test/integration/smoke/test_scale_vm.py >> >> ==== Marvin Init Started ==== >> >> === Marvin Parse Config Successful === >> >> === Marvin Setting TestData Successful=== >> >> ==== Log Folder Path: /tmp//MarvinLogs//Sep_30_2015_08_06_32_AWNF1O. >> All logs will be available here ==== >> >> === Marvin Init Logging Successful=== >> >> ==== Marvin Init Successful ==== >> ===final results are now copied to: >> /tmp//MarvinLogs/test_scale_vm_OPS7AD=== >> root@localhost:~/cloudstack# cd /tmp//MarvinLogs/test_scale_vm_OPS7AD >> root@localhost:/tmp/MarvinLogs/test_scale_vm_OPS7AD# ls >> failed_plus_exceptions.txt results.txt runinfo.txt >> root@localhost:/tmp/MarvinLogs/test_scale_vm_OPS7AD# vi results.txt >> root@localhost:/tmp/MarvinLogs/test_scale_vm_OPS7AD# ls -al total 48 >> drwxr-xr-x 2 root root 4096 Sep 30 08:07 . >> drwxr-xr-x 8 root root 4096 Sep 30 08:06 .. >> -rw-r--r-- 1 root root 0 Sep 30 08:06 failed_plus_exceptions.txt >> -rw-r--r-- 1 root root 186 Sep 30 08:06 results.txt >> -rw-r--r-- 1 root root 36164 Sep 30 08:06 runinfo.txt >> root@localhost:/tmp/MarvinLogs/test_scale_vm_OPS7AD# >> root@localhost:/tmp/MarvinLogs/test_scale_vm_OPS7AD# cat results.txt >> Test scale virtual machine ... === TestName: test_01_scale_vm | >> Status >> : SUCCESS === ok >> >> --------------------------------------------------------------------- >> - >> Ran 1 test in 23.455s >> >> OK >> >> -----Original Message----- >> From: Sebastien Goasguen [mailto:run...@gmail.com] >> Sent: Wednesday, September 30, 2015 1:04 PM >> To: dev@cloudstack.apache.org >> Subject: Re: [Blocker] test duplicated in test_scale_vm.py >> >> >>> On Sep 30, 2015, at 9:10 AM, Koushik Das <koushik....@citrix.com> wrote: >>> >>> Raja, >>> The earlier scale vm test couldn't run on simulator because of the >>> validation steps (IIRC it tries to connect to the real VM created during >>> test). Since it wasn't possible to run with simulator, the new test was >>> added to run just on the simulator. Check the required_hardware tag. >>> >> >> Ok so that’s not really a blocker. >> >> Can we reduce the severity to major ? >> Or send a PR that will add this to the Travis run and run the correct test >> for simulator ? >> >> Or make you modifications in your BVT settings, re-run and close the issue ? >> >> >> >>> -----Original Message----- >>> From: Raja Pullela [mailto:raja.pull...@citrix.com] >>> Sent: Wednesday, 30 September 2015 12:12 >>> To: CloudStack Dev (dev@cloudstack.apache.org) >>> Subject: [Blocker] test duplicated in test_scale_vm.py >>> >>> Hi, >>> >>> Filed this blocker on test case "test_scale_vm.py" - >>> https://issues.apache.org/jira/browse/CLOUDSTACK-8924 >>> Fixing this will bring Simulator Basic and XS Basic to 100% passrate. >>> >>> Best, >>> Raja >>> >> >
