Le 05/03/2014 08:10, Didier Roche a écrit :
Le 04/03/2014 20:14, Thomi Richards a écrit :
Hi,
On Wed, Mar 5, 2014 at 6:30 AM, Didier Roche <[email protected]
<mailto:[email protected]>> wrote:
However, some unity8 tests reliably failed and Michal reported
it[1]. We reverted autopilot-qt and autopilot (part of the
transaction). We avoided reverting xpathselect which was part of
the transaction as well but only contained packaging changes.
Corey, can you ensure that unity8 AP tests are part of the
autopilot test plan? I think you will need to have a wiki page
like most projects with the autopilot test plan now and including
a big suit of tests to run (or add them to the autopilot tests
themselves).
Feel free to ask for another landing or reuse the same silo to
reland your changes once you have the fix and the tests.
This revert will be part of image #220.
Unity8 is on the test schedule. We found failing tests in the unity8
package with the new autopilot, but after Leo and I looked into it
further, we were able to get the same tests to fail with the existing
autopilot as well. The consistently failing test was:
unity8.application_lifecycle.tests.test_url_dispatcher.URLDispatcherTestCase.test_swipe_out_application_started_by_url_dispatcher(Native
Device)
The test tries to start an app via the url dispatcher (as far as I
can make out anyway - please correct me if that's wrong), but often
the application will not start, which causes the tests to fail.
We also found tests that fail intermittenly with both the new, and
old autopilot versions. These include:
unity8.shell.tests.test_notifications.EphemeralNotificationsTests.test_icon_summary_body(Native
Device)
unity8.shell.tests.test_notifications.EphemeralNotificationsTests.test_update_notification_same_layout(Native
Device)
unity8.shell.tests.test_notifications.EphemeralNotificationsTests.test_icon_summary(Native
Device)
unity8.shell.tests.test_emulators.DashAppsEmulatorTestCase.test_get_applications_with_unexisting_category(Native
Device)
Given that we found these regressions in both versions of autopilot,
we made the assumption that the failure was not related to the new
autopilot packages. You've obviously come to a different conclusion.
Please let me know what I need to do to convince you to let us land
autopilot.
As per the bug and reproduced by 2 people + the CI dashboard, we got
this failure (see the bug report) reliably on the new autopilot on
unity8 and didn't get it with the previous version. I would suggest
that you work with the Unity8 guys to ensure that's the case and work
with them to get this test passing with the new autopilot.
The other failure is what is explained in extends in all landing
threads, the Qt 5.0 crash which doesn't reproduce everytime. So we
rerun multiple times to ensure that the crash doesn't hide any other
regressions and you have to do that as well until we are out of this
degraded landing mode.
Reintroducing the bug reference which was dropped by Thomi's post:
https://bugs.launchpad.net/autopilot-qt/+bug/1287727. You wil note the
new failure was:
unity8.shell.tests.test_notifications.InteractiveNotificationBase.test_sd_incoming_call
(just to be clear that this isn't
unity8.application_lifecycle.tests.test_url_dispatcher.URLDispatcherTestCase.test_swipe_out_application_started_by_url_dispatcher(Native
Device) which indeed reliably fails even without a Qt 5.0 crash)
Didier
--
Mailing list: https://launchpad.net/~ubuntu-phone
Post to : [email protected]
Unsubscribe : https://launchpad.net/~ubuntu-phone
More help : https://help.launchpad.net/ListHelp
