Re: [Freeipa-devel] [PATCHES] 0227-0229 freeipa-tests package Beaker integration plugin
On 06/24/2013 10:33 AM, Petr Viktorin wrote: On 06/17/2013 09:10 PM, Dmitri Pal wrote: On 06/17/2013 11:08 AM, Petr Viktorin wrote: ipa-run-tests --with-beakerlib is horribly slow for me, is that expected? Yes. For every logged line, BeakerLib's default logging backend starts up Python, parses a XML file, appends the line, and writes the XML out again. So especially with longer runs it's really slow. Is there any way to solve this problem? For example send the output over the DBUS to a special service that would have the python already loaded and would do the appending to the files and writing the output. Also there can be an optimization that it would not save the file up until the change affects a different file. The logic would be: loop: If do not have an open output file open one and keep it in memory Read a request for update until receive a special message for termination or a signal, then break out of the loop If the request for update for the same file update the file Else save currently open file and start a new one, add data to the newly started file end close currently open file I hope Beaker does something like you described. The slow part is only the XML backend, which gets selected if you run BeakerLib without Beaker. We could write (or find?) a faster logging backend but it's not really necessary. Without Beaker the BeakerLib logging is not of much use anyway. Right. In our upstream continuous integration testing, this option should be off - no logging performance issues for us. Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCHES] 0227-0229 freeipa-tests package Beaker integration plugin
On 06/17/2013 09:10 PM, Dmitri Pal wrote: On 06/17/2013 11:08 AM, Petr Viktorin wrote: ipa-run-tests --with-beakerlib is horribly slow for me, is that expected? Yes. For every logged line, BeakerLib's default logging backend starts up Python, parses a XML file, appends the line, and writes the XML out again. So especially with longer runs it's really slow. Is there any way to solve this problem? For example send the output over the DBUS to a special service that would have the python already loaded and would do the appending to the files and writing the output. Also there can be an optimization that it would not save the file up until the change affects a different file. The logic would be: loop: If do not have an open output file open one and keep it in memory Read a request for update until receive a special message for termination or a signal, then break out of the loop If the request for update for the same file update the file Else save currently open file and start a new one, add data to the newly started file end close currently open file I hope Beaker does something like you described. The slow part is only the XML backend, which gets selected if you run BeakerLib without Beaker. We could write (or find?) a faster logging backend but it's not really necessary. Without Beaker the BeakerLib logging is not of much use anyway. -- Petr³ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCHES] 0227-0229 freeipa-tests package Beaker integration plugin
On 06/17/2013 07:27 PM, Martin Kosek wrote: On 06/17/2013 06:59 PM, Jan Cholasta wrote: On 17.6.2013 17:08, Petr Viktorin wrote: We can fix individual out-of-tree failures later, the priority now is that in-tree tests are not broken, and that the beakerlib plugin works. Well, works just fine for me, so ACK. Honza Thanks for review! I just had to merge freeipa.spec.in and update the changelog date to avoid making the strict RPM date checker angry. Pushed all 3 to master. Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel There is one line in .gitignore which refers to the old location of the service.crt file. The attached patch fixes that. -- Regards, Ana Krivokapic Associate Software Engineer FreeIPA team Red Hat Inc. From e55fd3497062e5aa6b3b78ed5ac843896987c727 Mon Sep 17 00:00:00 2001 From: Ana Krivokapic akriv...@redhat.com Date: Tue, 18 Jun 2013 16:02:05 +0200 Subject: [PATCH] Fix location of service.crt in .gitignore --- .gitignore | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index 8a2829a376d875917d9acf086036503df8ee8822..738b00b13b7014ea915da9bcd06ab1ac4eee3764 100644 --- a/.gitignore +++ b/.gitignore @@ -80,7 +80,7 @@ ipa-client/missing ipa-client/py-compile ipa-client/stamp-h1 ipa-client/version.m4 -tests/test_xmlrpc/service.crt +ipatests/test_xmlrpc/service.crt freeipa.spec ipapython/setup.py ipapython/version.py -- 1.8.1.4 ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCHES] 0227-0229 freeipa-tests package Beaker integration plugin
On 06/18/2013 04:07 PM, Ana Krivokapic wrote: On 06/17/2013 07:27 PM, Martin Kosek wrote: On 06/17/2013 06:59 PM, Jan Cholasta wrote: On 17.6.2013 17:08, Petr Viktorin wrote: We can fix individual out-of-tree failures later, the priority now is that in-tree tests are not broken, and that the beakerlib plugin works. Well, works just fine for me, so ACK. Honza Thanks for review! I just had to merge freeipa.spec.in and update the changelog date to avoid making the strict RPM date checker angry. Pushed all 3 to master. Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel There is one line in .gitignore which refers to the old location of the service.crt file. The attached patch fixes that. Good point. ACK, pushed to master. Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCHES] 0227-0229 freeipa-tests package Beaker integration plugin
On 14.6.2013 16:01, Petr Viktorin wrote:
On 06/14/2013 03:20 PM, Jan Cholasta wrote:
Hi,
On 28.5.2013 17:55, Petr Viktorin wrote:
Hello,
Patch 0227 creates the freeipa-tests package.
As a system package, it needs a more unique name than tests, so I
renamed it to ipatests. I also changed imports and references to it.
Sorry to everyone developing tests right now – there will be conflicts,
but hopefully they'll be straightforward.
Note that the test suite does not yet pass when run outside the Git
tree. Work on that is ongoing but not a priority right now (it's more
important to get some integration tests running). Help would be
appreciated :)
Typo in commit message: Tename the 'tests' directory ...
Thanks for the catch!
The patch needs rebasing.
Attaching rebased versions.
Patch 0228 adds a wrapper based on make-test which runs the
system-installed test suite. freeipa-tests installs it as
/usr/bin/ipa-run-tests.
As I said above the tests currently fail when run this way.
Patch 0229 adds a Nose plugin for integration with BeakerLib[1]. When
the plugin is loaded (ipa-run-tests does that) and enabled (using the
--with-beakerlib option), it hooks into Nose and runs rlPhase*, rlPass,
rlFail and rlLog* Bash functions at appropriate events.
I still need to actually run the code, I will do that with your patches
230-240 included.
If I may suggest, please test this set by itself. The tests→ipatests
rename blocks or conflicts with other patches so it would be good to get
it in soon, without waiting on the integration testing review.
When I run make test I get the following error:
Traceback (most recent call last):
File /usr/lib/python2.7/site-packages/nose/loader.py, line 413, in
loadTestsFromName
addr.filename, addr.module)
File /usr/lib/python2.7/site-packages/nose/importer.py, line 47, in
importFromPath
return self.importFromDir(dir_path, fqname)
File /usr/lib/python2.7/site-packages/nose/importer.py, line 94, in
importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File
/home/jcholast/freeipa/ipatests/test_xmlrpc/test_host_plugin.py, line
59, in module
fd = open('tests/test_xmlrpc/service.crt', 'r')
IOError: [Errno 2] No such file or directory:
'tests/test_xmlrpc/service.crt'
I believe the certificate path should begin with ipatests instead of
tests.
When I run ipa-run-tests, I get an additional similar error in
test_service_plugin.py:
Traceback (most recent call last):
File /usr/lib/python2.7/site-packages/nose/loader.py, line 413, in
loadTestsFromName
addr.filename, addr.module)
File /usr/lib/python2.7/site-packages/nose/importer.py, line 47, in
importFromPath
return self.importFromDir(dir_path, fqname)
File /usr/lib/python2.7/site-packages/nose/importer.py, line 94, in
importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File
/usr/lib/python2.7/site-packages/ipatests/test_xmlrpc/test_service_plugin.py,
line 42, in module
fd = open('ipatests/test_xmlrpc/service.crt', 'r')
IOError: [Errno 2] No such file or directory:
'ipatests/test_xmlrpc/service.crt'
Also with ipa-run-tests, many of the cmdline tests are failing with:
Traceback (most recent call last):
File /usr/lib/python2.7/site-packages/nose/case.py, line 381, in setUp
try_run(self.inst, ('setup', 'setUp'))
File /usr/lib/python2.7/site-packages/nose/util.py, line 469, in
try_run
return func()
File
/usr/lib/python2.7/site-packages/ipatests/test_cmdline/cmdline.py,
line 58, in setUp
'Command %r not available' % self.command
AssertionError: Command 'ipa-client/ipa-getkeytab' not available
Honza
--
Jan Cholasta
___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCHES] 0227-0229 freeipa-tests package Beaker integration plugin
On 17.6.2013 14:39, Jan Cholasta wrote:
On 14.6.2013 16:01, Petr Viktorin wrote:
On 06/14/2013 03:20 PM, Jan Cholasta wrote:
Hi,
On 28.5.2013 17:55, Petr Viktorin wrote:
Hello,
Patch 0227 creates the freeipa-tests package.
As a system package, it needs a more unique name than tests, so I
renamed it to ipatests. I also changed imports and references to it.
Sorry to everyone developing tests right now – there will be
conflicts,
but hopefully they'll be straightforward.
Note that the test suite does not yet pass when run outside the Git
tree. Work on that is ongoing but not a priority right now (it's more
important to get some integration tests running). Help would be
appreciated :)
Typo in commit message: Tename the 'tests' directory ...
Thanks for the catch!
The patch needs rebasing.
Attaching rebased versions.
Patch 0228 adds a wrapper based on make-test which runs the
system-installed test suite. freeipa-tests installs it as
/usr/bin/ipa-run-tests.
As I said above the tests currently fail when run this way.
Patch 0229 adds a Nose plugin for integration with BeakerLib[1]. When
the plugin is loaded (ipa-run-tests does that) and enabled (using the
--with-beakerlib option), it hooks into Nose and runs rlPhase*, rlPass,
rlFail and rlLog* Bash functions at appropriate events.
I still need to actually run the code, I will do that with your patches
230-240 included.
If I may suggest, please test this set by itself. The tests→ipatests
rename blocks or conflicts with other patches so it would be good to get
it in soon, without waiting on the integration testing review.
When I run make test I get the following error:
Traceback (most recent call last):
File /usr/lib/python2.7/site-packages/nose/loader.py, line 413, in
loadTestsFromName
addr.filename, addr.module)
File /usr/lib/python2.7/site-packages/nose/importer.py, line 47, in
importFromPath
return self.importFromDir(dir_path, fqname)
File /usr/lib/python2.7/site-packages/nose/importer.py, line 94, in
importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File
/home/jcholast/freeipa/ipatests/test_xmlrpc/test_host_plugin.py, line
59, in module
fd = open('tests/test_xmlrpc/service.crt', 'r')
IOError: [Errno 2] No such file or directory:
'tests/test_xmlrpc/service.crt'
I believe the certificate path should begin with ipatests instead of
tests.
When I run ipa-run-tests, I get an additional similar error in
test_service_plugin.py:
Traceback (most recent call last):
File /usr/lib/python2.7/site-packages/nose/loader.py, line 413, in
loadTestsFromName
addr.filename, addr.module)
File /usr/lib/python2.7/site-packages/nose/importer.py, line 47, in
importFromPath
return self.importFromDir(dir_path, fqname)
File /usr/lib/python2.7/site-packages/nose/importer.py, line 94, in
importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File
/usr/lib/python2.7/site-packages/ipatests/test_xmlrpc/test_service_plugin.py,
line 42, in module
fd = open('ipatests/test_xmlrpc/service.crt', 'r')
IOError: [Errno 2] No such file or directory:
'ipatests/test_xmlrpc/service.crt'
Also with ipa-run-tests, many of the cmdline tests are failing with:
Traceback (most recent call last):
File /usr/lib/python2.7/site-packages/nose/case.py, line 381, in setUp
try_run(self.inst, ('setup', 'setUp'))
File /usr/lib/python2.7/site-packages/nose/util.py, line 469, in
try_run
return func()
File
/usr/lib/python2.7/site-packages/ipatests/test_cmdline/cmdline.py,
line 58, in setUp
'Command %r not available' % self.command
AssertionError: Command 'ipa-client/ipa-getkeytab' not available
Honza
ipa-run-tests --with-beakerlib is horribly slow for me, is that expected?
Honza
--
Jan Cholasta
___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCHES] 0227-0229 freeipa-tests package Beaker integration plugin
On 17.6.2013 15:09, Jan Cholasta wrote:
On 17.6.2013 14:39, Jan Cholasta wrote:
On 14.6.2013 16:01, Petr Viktorin wrote:
On 06/14/2013 03:20 PM, Jan Cholasta wrote:
Hi,
On 28.5.2013 17:55, Petr Viktorin wrote:
Hello,
Patch 0227 creates the freeipa-tests package.
As a system package, it needs a more unique name than tests, so I
renamed it to ipatests. I also changed imports and references to it.
Sorry to everyone developing tests right now – there will be
conflicts,
but hopefully they'll be straightforward.
Note that the test suite does not yet pass when run outside the Git
tree. Work on that is ongoing but not a priority right now (it's more
important to get some integration tests running). Help would be
appreciated :)
Typo in commit message: Tename the 'tests' directory ...
Thanks for the catch!
The patch needs rebasing.
Attaching rebased versions.
Patch 0228 adds a wrapper based on make-test which runs the
system-installed test suite. freeipa-tests installs it as
/usr/bin/ipa-run-tests.
As I said above the tests currently fail when run this way.
Patch 0229 adds a Nose plugin for integration with BeakerLib[1]. When
the plugin is loaded (ipa-run-tests does that) and enabled (using the
--with-beakerlib option), it hooks into Nose and runs rlPhase*,
rlPass,
rlFail and rlLog* Bash functions at appropriate events.
I still need to actually run the code, I will do that with your patches
230-240 included.
If I may suggest, please test this set by itself. The tests→ipatests
rename blocks or conflicts with other patches so it would be good to get
it in soon, without waiting on the integration testing review.
When I run make test I get the following error:
Traceback (most recent call last):
File /usr/lib/python2.7/site-packages/nose/loader.py, line 413, in
loadTestsFromName
addr.filename, addr.module)
File /usr/lib/python2.7/site-packages/nose/importer.py, line 47, in
importFromPath
return self.importFromDir(dir_path, fqname)
File /usr/lib/python2.7/site-packages/nose/importer.py, line 94, in
importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File
/home/jcholast/freeipa/ipatests/test_xmlrpc/test_host_plugin.py, line
59, in module
fd = open('tests/test_xmlrpc/service.crt', 'r')
IOError: [Errno 2] No such file or directory:
'tests/test_xmlrpc/service.crt'
I believe the certificate path should begin with ipatests instead of
tests.
When I run ipa-run-tests, I get an additional similar error in
test_service_plugin.py:
Traceback (most recent call last):
File /usr/lib/python2.7/site-packages/nose/loader.py, line 413, in
loadTestsFromName
addr.filename, addr.module)
File /usr/lib/python2.7/site-packages/nose/importer.py, line 47, in
importFromPath
return self.importFromDir(dir_path, fqname)
File /usr/lib/python2.7/site-packages/nose/importer.py, line 94, in
importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File
/usr/lib/python2.7/site-packages/ipatests/test_xmlrpc/test_service_plugin.py,
line 42, in module
fd = open('ipatests/test_xmlrpc/service.crt', 'r')
IOError: [Errno 2] No such file or directory:
'ipatests/test_xmlrpc/service.crt'
Also with ipa-run-tests, many of the cmdline tests are failing with:
Traceback (most recent call last):
File /usr/lib/python2.7/site-packages/nose/case.py, line 381, in
setUp
try_run(self.inst, ('setup', 'setUp'))
File /usr/lib/python2.7/site-packages/nose/util.py, line 469, in
try_run
return func()
File
/usr/lib/python2.7/site-packages/ipatests/test_cmdline/cmdline.py,
line 58, in setUp
'Command %r not available' % self.command
AssertionError: Command 'ipa-client/ipa-getkeytab' not available
Honza
ipa-run-tests --with-beakerlib is horribly slow for me, is that expected?
Honza
Also I get the following failures with --with-beaker (in addition to the
previously reported ones):
:: [ LOG] :: Nose test: test_help.test_ipa_help
:: [ ERROR ] :: [ipa.ipalib.cli.cli] non-public: AttributeError:
'API' object has no attribute 'parser'
:: [ ERROR ] :: Traceback (most recent call last):
:: [ ERROR ] :: File
/usr/lib/python2.7/site-packages/ipalib/backend.py, line 129, in execute
:: [ ERROR ] :: result = self.Command[_name](*args, **options)
:: [ ERROR ] :: File
/usr/lib/python2.7/site-packages/ipalib/frontend.py, line 435, in __call__
:: [ ERROR ] :: ret = self.run(*args, **options)
:: [ ERROR ] :: File
/usr/lib/python2.7/site-packages/ipalib/cli.py, line 765, in run
:: [ ERROR ] :: self.api.parser.print_help(outfile)
:: [ ERROR ] :: AttributeError: 'API' object has no attribute 'parser'
:: [ ERROR ] :: Traceback (most recent call last):
:: [ ERROR ] :: File /usr/lib64/python2.7/unittest/case.py, line
369, in run
Re: [Freeipa-devel] [PATCHES] 0227-0229 freeipa-tests package Beaker integration plugin
On 17.6.2013 17:08, Petr Viktorin wrote: We can fix individual out-of-tree failures later, the priority now is that in-tree tests are not broken, and that the beakerlib plugin works. Well, works just fine for me, so ACK. Honza -- Jan Cholasta ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCHES] 0227-0229 freeipa-tests package Beaker integration plugin
On 06/17/2013 06:59 PM, Jan Cholasta wrote: On 17.6.2013 17:08, Petr Viktorin wrote: We can fix individual out-of-tree failures later, the priority now is that in-tree tests are not broken, and that the beakerlib plugin works. Well, works just fine for me, so ACK. Honza Thanks for review! I just had to merge freeipa.spec.in and update the changelog date to avoid making the strict RPM date checker angry. Pushed all 3 to master. Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCHES] 0227-0229 freeipa-tests package Beaker integration plugin
On 06/17/2013 11:08 AM, Petr Viktorin wrote: ipa-run-tests --with-beakerlib is horribly slow for me, is that expected? Yes. For every logged line, BeakerLib's default logging backend starts up Python, parses a XML file, appends the line, and writes the XML out again. So especially with longer runs it's really slow. Is there any way to solve this problem? For example send the output over the DBUS to a special service that would have the python already loaded and would do the appending to the files and writing the output. Also there can be an optimization that it would not save the file up until the change affects a different file. The logic would be: loop: If do not have an open output file open one and keep it in memory Read a request for update until receive a special message for termination or a signal, then break out of the loop If the request for update for the same file update the file Else save currently open file and start a new one, add data to the newly started file end close currently open file If we need this cross systems we can do AMQP. I bet this would be powers of magnitude faster. -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. --- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
