[
https://issues.apache.org/jira/browse/CLOUDSTACK-8975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14968950#comment-14968950
]
ASF GitHub Bot commented on CLOUDSTACK-8975:
--------------------------------------------
Github user runseb commented on the pull request:
https://github.com/apache/cloudstack/pull/960#issuecomment-150178317
I think you guys agree on the problem but disagree on the solution.
To me 100% success on BVT, if it includes ~20% (or whatever percentage) of
skipped tests is not a real 100% success rate on BVT.
So @miguelaferreira prefers to see 80% success on BVT and vote +1 on a
release, if he actually knows what's happening with the 20% failure.
While @nitt10prashant and @pvr9711 you seem to want 100% success on BVT
even if it includes some tests that are actually not run.
The nicira test is actually the only one in there that's testing a third
party plugin. I don't see any tests for solidfire, netscaler, open contrail,
midokura etc….All theses tests will fail without the proper hardware/software
setup and hence will fail with simulator run.
If we really want 100% on integration tests (so called BVT) we need to
define a test suite that everyone can run without special hardware or software.
And those tests *cannot* be skipped.
I personally fail to see the benefit of skipped tests versus a failed test.
For this particular tests I propose we move it to a folder
/test/integration/plugins and have our base tests defining "BVT" as the test in
/test/integration/smoke
So -1 on this PR
+1 on a PR that would move this test to a separate folder.
> [Automation]skip test case if niciraNvp settings are not available in config
> file
> ---------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-8975
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-8975
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Automation
> Affects Versions: 4.7.0
> Reporter: prashant kumar mishra
> Assignee: prashant kumar mishra
> Priority: Minor
> Fix For: 4.7.0
>
>
> /root/cloudstack/test/integration/smoke/test_nicira_controller.py is
> failing if niciraNvp settings are not available in config file. Modifying
> script to skip test case if settings are not available
> log
> =======
> File "/root/cloudstack/test/integration/smoke/test_nicira_controller.py",
> line 46, in setUpClass
> cls.nicira_hosts = cls.config.niciraNvp.hosts
> AttributeError: 'NoneType' object has no attribute 'hosts'
> -------------------- >> begin captured stdout << ---------------------
> === TestName: None | Status : EXCEPTION ===
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
