Jira (PUP-5625) Problems with the "allow_duplicate_csr" acceptance test

[Jeremy Barlow (JIRA)]

Title: Message Title

Jeremy Barlow created an issue   Puppet / PUP-5625 Problems with the "allow_duplicate_csr" acceptance test Issue Type: Bug Assignee: Jeremy Barlow Components: QA Created: 2015/12/17 12:42 PM Labels: qa-task Priority: Normal Reporter: Jeremy Barlow In running Puppet acceptance tests locally with Beaker, I encountered a couple of problems with the ticket_3360_allow_duplicate_csr_with_option_set.rb test: 1) assert_not_equal not found. Here's a snippet of the failed output from one run: Test Case ./ruby/puppet/acceptance/tests/ticket_3360_allow_duplicate_csr_with_option_set.rb reported: #<RuntimeError: PuppetAcceptance::DSL::Helpers.with_puppet_running_on failed (check backtrace for location) because: undefined method `assert_not_equal' for #<Beaker::TestCase:0x007f97945fd218> ./ruby/puppet/acceptance/tests/ticket_3360_allow_duplicate_csr_with_option_set.rb:48:in `block (3 levels) in run_test' I don't see a method called assert_not_equal in the version of minitest that we're pulling in from the Gemfile, 5.8.3. I saw that some other gems alias refute_equal, which does exist in minitest to assert_not_equal - e.g., nokogiri. This makes me think that this failure may not occur in full Jenkins CI runs because code that implicitly sources the alias is being run first. This seems dangerous, though, and that it might be better to just use refute_equals directly from this test. AFAICT, no other Puppet acceptance test uses assert_not_equal. 2) The test appears to have been written with the expectation that it will iterate through all of the agents setup for the test run, submitting duplicate csrs for each. It doesn't appear to take into account the pre-existence of other csrs in the ssldir, however, and could fail when those other csrs don't change. 3) The test always skips over the master node. So if a node is configured with both the master and agent roles, it will skip that node entirely - meaning that the test essentially does nothing for a pipeline configured to do just the master and agent on a single node. 4) The regex that the test uses to locate certs in the cert list doesn't seem precise enough, leaving some csrs to be skipped over completely. Add Comment   This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.