sync_integration_tests should be only in proc browser tests. If it includes other types of tests, that's a mistake and they should be moved to the unit tests target. Could you share some details of what you've found, and what it would take to make these tests as robust as browser_tests?
As background, the sync integration tests run against a live sync server, possibly a test server running the same code as the Google production servers, using real Google accounts. Currently the tests expect an external force (probably the python script we have) to provide it wish an sync server URL, user account, and password combo that is in a clean state. As a result, we run the tests one at a time currently. So carried state bugs in memory between individual tests are not presently risky. Eventually, I'd like to see a more normal setup with gtest driving execution of multiple tests, calling out to some external provider to get the sync server/account/password info, instead of having it passed in. Probably, the provider would reset/restart the server instance through some backdoor. The dependence on an external server is why we put these tests in their own executable; they should all be in-proc browser tests. - nick On Wed, Sep 2, 2009 at 9:44 AM, Paweł Hajdan Jr. <[email protected]>wrote: > So, here are the choices I currently see: > - let each binary only use one kind of tests (probably both UI) > - split each binary to only one kind of tests binaries (probably too much > test binaries anyway) > > What do you think? I can help with the switch, just need a decision what > kind of tests/split would be better. > > > On Mon, Aug 31, 2009 at 11:44, Paweł Hajdan Jr. > <[email protected]>wrote: > >> I don't quite know how to solve the problem. When IN_PROC_BROWSER_TEST is >> used, a launcher that will isolate the test cases should be used (like >> running them in forked processes or reloading a dll). >> However, currently only browser_tests binary does that. Other users, >> sync_integration_tests and interactive_ui_tests, don't. This is a fantastic >> recipe for flakiness. In-process browser test carries a lot of state, all >> the singletons, etc. It does cause problems if not cleaned up properly, and >> I removed that kind of problem from unit_tests (and introduced >> ALLOW_IN_PROC_BROWSER_TEST to prevent the problem from growing). >> >> The problem is, that the mentioned binaries mix UI tests with browser >> tests. So it's not trivially possible to use browser_tests launcher for them >> (or maybe it is?). What do you think? >> > > > > > --~--~---------~--~----~------------~-------~--~----~ Chromium Developers mailing list: [email protected] View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~----------~----~----~----~------~----~------~--~---
