#27105: Fix Tor Browser testsuite
-------------------------------------------------+-------------------------
Reporter: boklm | Owner: acat
Type: task | Status:
| assigned
Priority: Medium | Milestone:
Component: Applications/Quality Assurance and | Version:
Testing |
Severity: Normal | Resolution:
Keywords: tbb-testsuite, | Actual Points:
TorBrowserTeam202004, ReleaseTrainMigration |
Parent ID: #33654 | Points:
Reviewer: | Sponsor:
| Sponsor58-must
-------------------------------------------------+-------------------------
Comment (by acat):
I think it might be better to do all reviewing/followup here instead of
all the N different children, but I'm not sure.
For now, here is a branch https://github.com/acatarineu/tor-browser-
bundle-testsuite/commits/27105 fixing several tests, mostly targeting a
`33533+5` build (based on mozilla-beta 77). I only tested locally
(linux64) via `./tbb-testsuite --no-xdummy` command. Note: the fix for
#34250 is needed for several tests to pass. This will need to be ongoing
work, in parallel to #33533, as we still have to do more rebases until
esr78.
There are a couple of tests (`noscript`, `https-everywhere`) which I
enabled and partially fixed but do not work completely yet. For
`noscript`, I suspect this might be a regression of `33533`, since the
test passes with a `tor-browser-linux64-9.0.10_en-US.tar.xz` build (with
the #34250 fix). I'll create a ticket to investigate that. For `https-
everywhere` test, it seems to be flaky, as it usually works when tests are
run with `./tbb-testsuite --no-xdummy --enable-tests tor_bootstrap,https-
everywhere`, but usually doesn't when the full testsuite is run. I could
not find out yet why `https-everywhere` is not redirecting to `https` for
the test page in some cases, but I guess further investigation can be done
later in #30431.
Apart from this I think this should also be tested in the other platforms
we support, as I only tried in linux64. I could try locally for Windows,
but not sure about macosx. Do we still have a test server that we could
use for this?
I also think it would make sense to start moving some of the tests to
`tor-browser.git` repo, at least the `marionette` ones. We currently can't
test the full browser there, but we could try to make changes so that
something close enough to "full" tor-browser is built from `tor-
browser.git` to allow tests to pass (and be helpful). For me, I can see
at least two reasons why this would be helpful.
A couple of questions wrt the changes I made (or not made):
* Added `TorBrowser/Tor/PluggableTransports/snowflake-client` to
exceptions in `Fix readelf_RELRO test` and `Fix readelf_stack_canary
test`, is this expected to be needed?
* Added `updater` to exceptions in `Fix readelf_no_rpath test`, is this
fine, or should we do something about it?
* I did not try to enable `fpcentral`, as I'm not sure if we are going to
maintain that?
* I did not try to enable `acid3`, not sure if it's helpful as it is now,
but maybe we could "pin" it's state via a screenshot to track changes
between versions?
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/27105#comment:21>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs
