2012/4/16 Patrick Ohly <[email protected]>: > On Fri, 2012-04-13 at 15:28 +0200, Patrick Ohly wrote: >> This is just a quick-and-dirty solution, not committed anywhere. Shall I >> do the base work for a few tests and then let you continue with adding >> the rest? > > Below are the commits, already merged into master after a quick test > run: > http://syncev.meego.com/2012-04-16-12-37_lucid-amd64_prebuilt-amd64_testing_dbus/ > > I started with your initial commit and then included the changes that we > were discussing, plus a fix for a failure still visible in that test > run. > > The reason is that your scanFiles() implementation is slightly different > from the C++ version. In contrast to the C++ version it includes lines > like: > > sources/addressbook/config.ini:# ActiveSync Address Book = eas-contacts > sources/addressbook/config.ini:# ActiveSync Events = eas-events > sources/addressbook/config.ini:# ActiveSync Todos = eas-todos > sources/addressbook/config.ini:# ActiveSync Memos = eas-memos
Yeah, I noticed that yesterday - it was mostly fault of isPropAssignment - I was stripping all whitespace at the beginning of the passed line. That was bad idea. :) > These need to be excluded by the comparison because these comments can > change, depending on how the binaries were compiled or run. Your > reference text in the ScheduleWorld config includes one particular > variation and therefore it worked for me when testing manually, but the > nightly testing differs. > > A good test for the assertEqualDiff() :-) See: > > http://syncev.meego.com/2012-04-16-12-37_lucid-amd64_prebuilt-amd64_testing_dbus/testing-amd64/7-dbus/Cmdline_SetupScheduleWorld.log.html > > With everything in place, do you still want to convert some more tests? It should be a breeze, so I will try to be quick with it. > I think one commit per converted test would be good, with C++ removed > and Python added in the same test. I expect that most tests can and > should be moved. The only reason for keeping a test in the C++ code > would be if it explicitly tests some aspect that cannot be tested when > going through D-Bus - not sure whether there is any such case. I guess there are possibly some: testPeerConfigMigration - has some calls to SyncContext::setStableRelease testFutureConfig - it calls SyncConfig directly in expectTooOld testWebDAV - how can we detect if we have webdav enabled? I guess that some of this code can be mimicked by calling syncevolution with some options. Will look at it later. > > commit 2925b97bac45ae4dc4dac92b04b91683cad29ddc > Author: Krzesimir Nowak <[email protected]> > Date: Thu Apr 12 15:45:02 2012 +0200 > > D-Bus Testing: add Cmdline tests framework > > This is a translation of the corresponding C++ TestCmdline from > Cmdline.cpp into Python. > > commit aa562e5ef97eba78b7c45ab095df5071134840dc > Author: Patrick Ohly <[email protected]> > Date: Mon Apr 16 09:56:50 2012 +0200 > > D-Bus Testing: added assertEqualDiff() > > Replaces the diffStrings() from the previous commit. Renamed it > because it really is specifying an assertion and moved it to > DBusUtil because it may be useful outside of the command line > testing. > > Also made sure that the diff is part of the assertion because then it > will be repeated when listing the failure. > > commit 5389b8eeeddc328e29a86c91aec8606ac3fb9741 > Author: Patrick Ohly <[email protected]> > Date: Mon Apr 16 10:54:52 2012 +0200 > > D-Bus Testing: ensure that we have assertRegexpMatches() > > unittest.assertRegexpMatches() is very useful for command line > output testing. But it is not yet available in Ubuntu Lucid, > the current base line for SyncEvolution releases. Therefore > provide a fallback implementation. > > commit 0e63bc8e59b53a8821beff9f29d061cae48a66fc > Author: Patrick Ohly <[email protected]> > Date: Mon Apr 16 17:26:12 2012 +0200 > > D-Bus Testing: fixed scanFiles() > > The function must return all property assignments, including those > which are commented out. The Python version accidentally included some > comment lines which looked like assignments, partly because it ignored > leading spaces, partly because it looked anywhere in the line for an > equal sign. > > Fixed by using a regex for the matching. > > commit 67ce74b57fcb51f57b8cf29e353e13da16d60b8b > Author: Patrick Ohly <[email protected]> > Date: Mon Apr 16 12:13:14 2012 +0200 > > D-Bus Testing: eliminate TestCmdline.testdir > > All D-Bus tests already have their private xdg_root = 'temp-test-dbus' > which is cleared before running each test. Use it also for TestCmdline > instead of creating our own temporary directory, for the sake of > consistency. > > Another advantage is that the syncevo-dbus-server is running with that > same XDG root and reading/writing files in it automatically. The > command line client currently *does* send its XDG env variables and > the server uses them, but perhaps that feature will have to be > removed, because it has a design mistake (*), so don't depend on it. > > (*) If different clients and the auto-syncing code in the server are > allowed to use different config roots, the 'config name' alone no > longer is a unique identifier for configs. > > commit 7c2de4ed5199f9c971d1474595f5a04988c1eac3 > Author: Patrick Ohly <[email protected]> > Date: Mon Apr 16 12:58:52 2012 +0200 > > D-Bus Testing: added running of command line tool > > The 'syncevolution' command line tool is started inside the same D-Bus > session as the 'syncevo-dbus-server' that is started before running > each test. They use the same XDG env variables. > > The test-dbus.py script is able to watch and report all D-Bus traffic > between tool and server. > > Because tests are meant to be realistic, SYNCEVOLUTION_DEBUG has to be > optional and must be turned off for all tests which test for real > output as seen by users. > > Every single invocation of runCmdline() should check stdout and > stderr, to detect unexpected output. The return code is already tested > by runCmdline() itself, unless explicitly turned off (may be useful for > tests which expect certain kind of failures). > > Ideally the entire string gets tested, either via an exact string > comparison or a full regex match. Note that assertRegexpMatches() only > does a search, so use ^ and $ to ensure full match. > > If (and only if) the command is expected to produce output on both > stdout and stderr where the order matters (for example, syncing), then > stdout and stderr should be combined into one stream. Otherwise the > order cannot be tested. > > commit 470f0e7d7a56b96adff21e5e918c58ee0a9e3420 > Author: Patrick Ohly <[email protected]> > Date: Mon Apr 16 13:02:23 2012 +0200 > > D-Bus Testing: added testSetupScheduleWorld > > The test is a straight copy from Cmdline.cpp. Because the hard-coded > default SSL certificate path is not known in test-dbus.py, it has to > extract that string by printing a template once with the command line > tool (only the command line exposes the comments, normal D-Bus API > doesn't) and finding it in the output. > > This allows copying the C++ tests without making them less strict by > filtering out the SSL certificate path before the result comparisons. > > commit a27f2af580a00d81107f69bc5802c079cef23766 > Author: Patrick Ohly <[email protected]> > Date: Mon Apr 16 14:38:34 2012 +0200 > > D-Bus Testing: work around Python 2.6 subprocess bug > > Explicitly pass an environment when running the command line > tool. Otherwise subprocess.Popen() from Python 2.6 uses not > os.environ (which would be okay) but rather the environment passed to > a previous call to subprocess.Popen() (which will fail if the previous > test ran with an environment which had SYNCEVOLUTION_DEBUG set). > > -- > Best Regards, Patrick Ohly > > The content of this message is my personal opinion only and although > I am an employee of Intel, the statements I make here in no way > represent Intel's position on the issue, nor am I authorized to speak > on behalf of Intel on this matter. > > _______________________________________________ SyncEvolution mailing list [email protected] http://lists.syncevolution.org/listinfo/syncevolution
