Re: [Freeipa-devel] [PATCHES] 0230-0244 Integration testing framework
On 15.7.2013 16:43, Petr Viktorin wrote: On 07/11/2013 01:28 PM, Jan Cholasta wrote: On 10.7.2013 17:50, Petr Viktorin wrote: Yes, I believe that for the integration tests it's better to see what is going on. Both for manual runs, and also this way the information is more easily picked up by CI tools. It's harder to see what's going on with so much noise IMHO, I would prefer less verbose output for manual runs by default. On the other hand, the integration tests take too long and without output they may seem stuck. I'd rather not make that the default. I'll try to find some balance and make it more easily configurable in future patches. Thanks. Anyway, it's nothing critical, so don't jump through hoops. Both the tests in test_simple_replication.py fail for me. I suspect it is because the data isn't replicated fast enough, a little delay between user-show and user-add might fix this. Hm, they worked for me. I've added a delay. It seems fragile, I wonder what the proper way to do this would be. Hmm, the replica - master test still fails for me, even with delay increased to 20 s. Is there anything in the DS logs? Let me check. Something less fragile than delay would certainly be nice. I'll read up on the DS replication docs, and ask for some guidance. Great, thank you. Honza -- Jan Cholasta ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCHES] 0230-0244 Integration testing framework
On 07/11/2013 01:28 PM, Jan Cholasta wrote: On 10.7.2013 17:50, Petr Viktorin wrote: On 07/10/2013 02:03 PM, Jan Cholasta wrote: make test seems to run fine with patches 230-242 applied, however ipa-run-tests produces the following output: [...Skipping nose output...] I guess the location of the test certificate should be made configurable in order to fix the host and service plugin test failures. Making the existing test suite pass out of tree it's not a goal for this patchset. There is a pending patch for the service cert. Better report it now than forget it later. Also, there is a lot of debugging messages in ipa-run-tests output which wasn't there before IIRC, is that intentional? Yes, I believe that for the integration tests it's better to see what is going on. Both for manual runs, and also this way the information is more easily picked up by CI tools. It's harder to see what's going on with so much noise IMHO, I would prefer less verbose output for manual runs by default. On the other hand, the integration tests take too long and without output they may seem stuck. I'd rather not make that the default. I'll try to find some balance and make it more easily configurable in future patches. If $MASTER (and possible other host names) is not resolvable, ipa-test-config --global crashes: $ ipa-test-config --global Traceback (most recent call last): File /usr/bin/ipa-test-config, line 104, in module print main(sys.argv[1:]), File /usr/bin/ipa-test-config, line 58, in main return config.env_to_script(get_object(conf, args).to_env(**kwargs)) File /usr/lib/python2.7/site-packages/ipatests/test_integration/config.py, line 168, in to_env env['MASTER'] = default_domain.master.hostname File /usr/lib/python2.7/site-packages/ipatests/test_integration/config.py, line 282, in master return self.masters[0] IndexError: list index out of range Fixed. Now, if the name is not resolvable, reading the config will fail. Alternatively, the IP address can be given in variables such as $BEAKERREPLICA1_IP_env1 (no, I didn't invent the name). This fixed ipa-test-config as advertised, but ipa-run-tests test_integration/test_simple_replication.py still fails. Both the tests in test_simple_replication.py fail for me. I suspect it is because the data isn't replicated fast enough, a little delay between user-show and user-add might fix this. Hm, they worked for me. I've added a delay. It seems fragile, I wonder what the proper way to do this would be. Hmm, the replica - master test still fails for me, even with delay increased to 20 s. Is there anything in the DS logs? Something less fragile than delay would certainly be nice. I'll read up on the DS replication docs, and ask for some guidance. I've made some other changes, mainly BeakerLib plugin output. Patch 240: Avoid infinite recursion that happened with some cases of bad SSH credentials Patch 241-243: Rework the BeakerLib plugin output to better match traditional Beaker tests Patch 244: Make it possible to explicitly specify IP addresses of hosts To sum things up, there are still some little bugs, but these can be fixed after the beta release, in general everything seems to work, so ACK. Thanks, pushed to master: 23d3fde05960d6e47485e246e11bec701b30cb59 Not closing the ticket yet until the issues are cleared (or moved to separate tickets). -- Petr³ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCHES] 0230-0244 Integration testing framework
On 10.7.2013 17:50, Petr Viktorin wrote: On 07/10/2013 02:03 PM, Jan Cholasta wrote: make test seems to run fine with patches 230-242 applied, however ipa-run-tests produces the following output: [...Skipping nose output...] I guess the location of the test certificate should be made configurable in order to fix the host and service plugin test failures. Making the existing test suite pass out of tree it's not a goal for this patchset. There is a pending patch for the service cert. Better report it now than forget it later. Also, there is a lot of debugging messages in ipa-run-tests output which wasn't there before IIRC, is that intentional? Yes, I believe that for the integration tests it's better to see what is going on. Both for manual runs, and also this way the information is more easily picked up by CI tools. It's harder to see what's going on with so much noise IMHO, I would prefer less verbose output for manual runs by default. If $MASTER (and possible other host names) is not resolvable, ipa-test-config --global crashes: $ ipa-test-config --global Traceback (most recent call last): File /usr/bin/ipa-test-config, line 104, in module print main(sys.argv[1:]), File /usr/bin/ipa-test-config, line 58, in main return config.env_to_script(get_object(conf, args).to_env(**kwargs)) File /usr/lib/python2.7/site-packages/ipatests/test_integration/config.py, line 168, in to_env env['MASTER'] = default_domain.master.hostname File /usr/lib/python2.7/site-packages/ipatests/test_integration/config.py, line 282, in master return self.masters[0] IndexError: list index out of range Fixed. Now, if the name is not resolvable, reading the config will fail. Alternatively, the IP address can be given in variables such as $BEAKERREPLICA1_IP_env1 (no, I didn't invent the name). This fixed ipa-test-config as advertised, but ipa-run-tests test_integration/test_simple_replication.py still fails. Both the tests in test_simple_replication.py fail for me. I suspect it is because the data isn't replicated fast enough, a little delay between user-show and user-add might fix this. Hm, they worked for me. I've added a delay. It seems fragile, I wonder what the proper way to do this would be. Hmm, the replica - master test still fails for me, even with delay increased to 20 s. Something less fragile than delay would certainly be nice. I've made some other changes, mainly BeakerLib plugin output. Patch 240: Avoid infinite recursion that happened with some cases of bad SSH credentials Patch 241-243: Rework the BeakerLib plugin output to better match traditional Beaker tests Patch 244: Make it possible to explicitly specify IP addresses of hosts To sum things up, there are still some little bugs, but these can be fixed after the beta release, in general everything seems to work, so ACK. Honza -- Jan Cholasta ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
