Re: [tor-bugs] #27105 [Applications/Quality Assurance and Testing]: Fix Tor Browser testsuite

[Tor Bug Tracker & Wiki]

#27105: Fix Tor Browser testsuite
-+-
 Reporter:  boklm|  Owner:  acat
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Applications/Quality Assurance and   |Version:
  Testing|
 Severity:  Normal   | Resolution:
 Keywords:  tbb-testsuite,   |  Actual Points:
  ReleaseTrainMigration, TorBrowserTeam202006R   |
Parent ID:  #33654   | Points:
 Reviewer:  gk,sysrqb|Sponsor:
 |  Sponsor58-must
-+-
Changes (by acat):

 * reviewer:  gk => gk,sysrqb


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27105 [Applications/Quality Assurance and Testing]: Fix Tor Browser testsuite

[Tor Bug Tracker & Wiki]

#27105: Fix Tor Browser testsuite
-+-
 Reporter:  boklm|  Owner:  acat
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Applications/Quality Assurance and   |Version:
  Testing|
 Severity:  Normal   | Resolution:
 Keywords:  tbb-testsuite,   |  Actual Points:
  ReleaseTrainMigration, TorBrowserTeam202006R   |
Parent ID:  #33654   | Points:
 Reviewer:  gk   |Sponsor:
 |  Sponsor58-must
-+-

Comment (by acat):

 For the settings tests, here is the list of changes (moved from #34189):
 {{{
 Removed in 27268:
 -

 browser.download.manager.retention
 browser.download.manager.scanWhenDone
 browser.newtabpage.preload
 browser.pocket.api
 browser.pocket.site
 browser.safebrowsing.enabled -> browser.safebrowsing.phishing.enabled
 browser.search.countryCode
 browser.zoom.siteSpecific
 datareporting.healthreport.about.reportUrl
 datareporting.healthreport.about.reportUrlUnified
 datareporting.healthreport.service.enabled
 device.sensors.enabled
 devtools.appmanager.enabled: None != False
 devtools.webide.autoinstallADBHelper ->
 devtools.webide.autoinstallADBExtension
 devtools.webide.autoinstallFxdtAdapters
 dom.enable_performance
 dom.enable_user_timing
 dom.event.highrestimestamp.enabled
 dom.gamepad.enabled
 dom.maxHardwareConcurrency
 general.appname.override
 general.appversion.override
 general.buildID.override
 general.oscpu.override
 general.platform.override
 general.productSub.override
 general.useragent.vendor
 general.useragent.vendorSub
 intl.charset.default
 media.audio_data.enabled
 media.eme.apiVisible
 media.gmp-eme-adobe.enabled
 media.gmp-eme-adobe.visible (not removed there, but mentioned)
 media.video_stats.enabled
 media.webspeech.synth.enabled
 network.jar.block-remote-files
 plugin.expose_full_path
 privacy.suppressModifierKeyEvents
 privacy.use_utc_timezone
 security.pki.sha1_enforcement_level -> 4
 security.tls.unrestricted_rc4_fallback
 ui.use_standins_for_native_colors

 Other
 -

 dom.indexedDB.enabled (removed in #23745)
 browser.pocket.enabled -> extensions.pocket.enabled (#31602)
 extensions.langpacks.signatures.required (reenabled in #31942)
 extensions.legacy.exceptions (doesn't do anything)
 extensions.enabledItems (doesn't do anything)
 extensions.enabledAddons (doesn't do anything)
 app.update.auto (set to False by marionette, we can maybe cover this with
 separate update tests in the future)
 }}}

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27105 [Applications/Quality Assurance and Testing]: Fix Tor Browser testsuite

[Tor Bug Tracker & Wiki]

#27105: Fix Tor Browser testsuite
-+-
 Reporter:  boklm|  Owner:  acat
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Applications/Quality Assurance and   |Version:
  Testing|
 Severity:  Normal   | Resolution:
 Keywords:  tbb-testsuite,   |  Actual Points:
  ReleaseTrainMigration, TorBrowserTeam202006R   |
Parent ID:  #33654   | Points:
 Reviewer:  gk   |Sponsor:
 |  Sponsor58-must
-+-

Comment (by acat):

 The noscript test was fixed in noscript-11.0.27 (#34305). Now it only
 remains the `https-everywhere` test, which is still flaky.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27105 [Applications/Quality Assurance and Testing]: Fix Tor Browser testsuite

[Tor Bug Tracker & Wiki]

#27105: Fix Tor Browser testsuite
-+-
 Reporter:  boklm|  Owner:  acat
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Applications/Quality Assurance and   |Version:
  Testing|
 Severity:  Normal   | Resolution:
 Keywords:  tbb-testsuite,   |  Actual Points:
  TorBrowserTeam202005R, ReleaseTrainMigration   |
Parent ID:  #33654   | Points:
 Reviewer:  gk   |Sponsor:
 |  Sponsor58-must
-+-

Comment (by acat):

 I had to fix (again) the bridge test, due to the used bridge not being
 available anymore. This new bridge should keep working for some time.
 Revised in ​https://github.com/acatarineu/tor-browser-bundle-
 testsuite/commits/27105+1.

 Besides, this tor-browser-build branch should be used for testing:
 https://github.com/acatarineu/tor-browser-build/commits/33533+7. I removed
 the `clang9` bump (as it was not working and I had wrongly added it) and
 cherry-picked the patch for #34242.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27105 [Applications/Quality Assurance and Testing]: Fix Tor Browser testsuite

[Tor Bug Tracker & Wiki]

#27105: Fix Tor Browser testsuite
-+-
 Reporter:  boklm|  Owner:  acat
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Applications/Quality Assurance and   |Version:
  Testing|
 Severity:  Normal   | Resolution:
 Keywords:  tbb-testsuite,   |  Actual Points:
  TorBrowserTeam202005R, ReleaseTrainMigration   |
Parent ID:  #33654   | Points:
 Reviewer:  gk   |Sponsor:
 |  Sponsor58-must
-+-
Changes (by acat):

 * reviewer:   => gk


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27105 [Applications/Quality Assurance and Testing]: Fix Tor Browser testsuite

[Tor Bug Tracker & Wiki]

#27105: Fix Tor Browser testsuite
-+-
 Reporter:  boklm|  Owner:  acat
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Applications/Quality Assurance and   |Version:
  Testing|
 Severity:  Normal   | Resolution:
 Keywords:  tbb-testsuite,   |  Actual Points:
  TorBrowserTeam202005R, ReleaseTrainMigration   |
Parent ID:  #33654   | Points:
 Reviewer:   |Sponsor:
 |  Sponsor58-must
-+-

Comment (by acat):

 This can be tested with a `testbuild-linux-x86_64` build from the
 https://github.com/acatarineu/tor-browser-build/commits/33533+6 branch
 (`d88a471e4d2f5e6e91361e09e00a9480a4089b23`), and command `./tbb-testsuite
 --no-xdummy /path/to/tor-browser.xz`.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27105 [Applications/Quality Assurance and Testing]: Fix Tor Browser testsuite

[Tor Bug Tracker & Wiki]

#27105: Fix Tor Browser testsuite
-+-
 Reporter:  boklm|  Owner:  acat
 Type:  task | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:
Component:  Applications/Quality Assurance and   |Version:
  Testing|
 Severity:  Normal   | Resolution:
 Keywords:  tbb-testsuite,   |  Actual Points:
  TorBrowserTeam202005R, ReleaseTrainMigration   |
Parent ID:  #33654   | Points:
 Reviewer:   |Sponsor:
 |  Sponsor58-must
-+-
Changes (by acat):

 * keywords:  tbb-testsuite, TorBrowserTeam202004, ReleaseTrainMigration =>
 tbb-testsuite, TorBrowserTeam202005R, ReleaseTrainMigration
 * status:  assigned => needs_review


Comment:

 Marking this for review, but note that followup work will be needed, for
 77b->78esr, and to address the remaining issues.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27105 [Applications/Quality Assurance and Testing]: Fix Tor Browser testsuite

[Tor Bug Tracker & Wiki]

#27105: Fix Tor Browser testsuite
-+-
 Reporter:  boklm|  Owner:  acat
 Type:  task | Status:
 |  assigned
 Priority:  Medium   |  Milestone:
Component:  Applications/Quality Assurance and   |Version:
  Testing|
 Severity:  Normal   | Resolution:
 Keywords:  tbb-testsuite,   |  Actual Points:
  TorBrowserTeam202004, ReleaseTrainMigration|
Parent ID:  #33654   | Points:
 Reviewer:   |Sponsor:
 |  Sponsor58-must
-+-

Comment (by acat):

 I think it might be better to do all reviewing/followup here instead of
 all the N different children, but I'm not sure.

 For now, here is a branch https://github.com/acatarineu/tor-browser-
 bundle-testsuite/commits/27105 fixing several tests, mostly targeting a
 `33533+5` build (based on mozilla-beta 77). I only tested locally
 (linux64) via `./tbb-testsuite --no-xdummy` command. Note: the fix for
 #34250 is needed for several tests to pass. This will need to be ongoing
 work, in parallel to #33533, as we still have to do more rebases until
 esr78.

 There are a couple of tests (`noscript`, `https-everywhere`) which I
 enabled and partially fixed but do not work completely yet. For
 `noscript`, I suspect this might be a regression of `33533`, since the
 test passes with a `tor-browser-linux64-9.0.10_en-US.tar.xz` build (with
 the #34250 fix). I'll create a ticket to investigate that. For `https-
 everywhere` test, it seems to be flaky, as it usually works when tests are
 run with `./tbb-testsuite --no-xdummy --enable-tests tor_bootstrap,https-
 everywhere`, but usually doesn't when the full testsuite is run. I could
 not find out yet why `https-everywhere` is not redirecting to `https` for
 the test page in some cases, but I guess further investigation can be done
 later in #30431.

 Apart from this I think this should also be tested in the other platforms
 we support, as I only tried in linux64. I could try locally for Windows,
 but not sure about macosx. Do we still have a test server that we could
 use for this?

 I also think it would make sense to start moving some of the tests to
 `tor-browser.git` repo, at least the `marionette` ones. We currently can't
 test the full browser there, but we could try to make changes so that
 something close enough to "full" tor-browser is built from `tor-
 browser.git` to allow tests to pass (and be helpful). For me, I can see
 at least two reasons why this would be helpful.

 A couple of questions wrt the changes I made (or not made):

 * Added `TorBrowser/Tor/PluggableTransports/snowflake-client` to
 exceptions in `Fix readelf_RELRO test` and `Fix readelf_stack_canary
 test`, is this expected to be needed?

 * Added `updater` to exceptions in `Fix readelf_no_rpath test`, is this
 fine, or should we do something about it?

 * I did not try to enable `fpcentral`, as I'm not sure if we are going to
 maintain that?

 * I did not try to enable `acid3`, not sure if it's helpful as it is now,
 but maybe we could "pin" it's state via a screenshot to track changes
 between versions?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27105 [Applications/Quality Assurance and Testing]: Fix Tor Browser testsuite

[Tor Bug Tracker & Wiki]

#27105: Fix Tor Browser testsuite
-+-
 Reporter:  boklm|  Owner:  acat
 Type:  task | Status:
 |  assigned
 Priority:  Medium   |  Milestone:
Component:  Applications/Quality Assurance and   |Version:
  Testing|
 Severity:  Normal   | Resolution:
 Keywords:  tbb-testsuite,   |  Actual Points:
  TorBrowserTeam202004, ReleaseTrainMigration|
Parent ID:  #33654   | Points:
 Reviewer:   |Sponsor:
 |  Sponsor58-must
-+-
Changes (by acat):

 * owner:  tbb-team => acat


Old description:

> We need to fix the Tor Browser testsuite to work with esr60 based Tor
> Browser.

New description:

 We need to fix the Tor Browser testsuite to work with esr78 based Tor
 Browser.

--

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27105 [Applications/Quality Assurance and Testing]: Fix Tor Browser testsuite

[Tor Bug Tracker & Wiki]

#27105: Fix Tor Browser testsuite
-+-
 Reporter:  boklm|  Owner:  boklm
 Type:  task | Status:
 |  assigned
 Priority:  Medium   |  Milestone:
Component:  Applications/Quality Assurance and   |Version:
  Testing|
 Severity:  Normal   | Resolution:
 Keywords:  tbb-testsuite,   |  Actual Points:
  TorBrowserTeam202004, ReleaseTrainMigration|
Parent ID:  #33654   | Points:
 Reviewer:   |Sponsor:
 |  Sponsor58-must
-+-
Changes (by gaba):

 * sponsor:  Sponsor58 => Sponsor58-must


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27105 [Applications/Quality Assurance and Testing]: Fix Tor Browser testsuite

[Tor Bug Tracker & Wiki]

#27105: Fix Tor Browser testsuite
-+-
 Reporter:  boklm|  Owner:  boklm
 Type:  task | Status:
 |  assigned
 Priority:  Medium   |  Milestone:
Component:  Applications/Quality Assurance and   |Version:
  Testing|
 Severity:  Normal   | Resolution:
 Keywords:  tbb-testsuite,   |  Actual Points:
  TorBrowserTeam202004, ReleaseTrainMigration|
Parent ID:  #33654   | Points:
 Reviewer:   |Sponsor:
 |  Sponsor58
-+-
Changes (by pili):

 * parent:   => #33654


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27105 [Applications/Quality Assurance and Testing]: Fix Tor Browser testsuite (was: Fix Tor Browser testsuite for esr60)

[Tor Bug Tracker & Wiki]

#27105: Fix Tor Browser testsuite
-+-
 Reporter:  boklm|  Owner:  boklm
 Type:  task | Status:
 |  assigned
 Priority:  Medium   |  Milestone:
Component:  Applications/Quality Assurance and   |Version:
  Testing|
 Severity:  Normal   | Resolution:
 Keywords:  tbb-testsuite,   |  Actual Points:
  TorBrowserTeam202004, ReleaseTrainMigration|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
 |  Sponsor58
-+-

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27105 [Applications/Quality Assurance and Testing]: Fix Tor Browser testsuite for esr60

[Tor Bug Tracker & Wiki]

#27105: Fix Tor Browser testsuite for esr60
-+-
 Reporter:  boklm|  Owner:  boklm
 Type:  task | Status:
 |  assigned
 Priority:  Medium   |  Milestone:
Component:  Applications/Quality Assurance and   |Version:
  Testing|
 Severity:  Normal   | Resolution:
 Keywords:  tbb-testsuite,   |  Actual Points:
  TorBrowserTeam202004, ReleaseTrainMigration|
Parent ID:   | Points:
 Reviewer:   |Sponsor:
 |  Sponsor58
-+-
Changes (by sysrqb):

 * priority:  Very High => Medium
 * cc: acat, pospeselr (added)
 * keywords:
 tbb-testsuite, boklm201904, TorBrowserTeam201905,
 ReleaseTrainMigration
 => tbb-testsuite, TorBrowserTeam202004, ReleaseTrainMigration


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27105 [Applications/Quality Assurance and Testing]: Fix Tor Browser testsuite for esr60

[Tor Bug Tracker & Wiki]

#27105: Fix Tor Browser testsuite for esr60
-+-
 Reporter:  boklm|  Owner:  boklm
 Type:  task | Status:
 |  assigned
 Priority:  Very High|  Milestone:
Component:  Applications/Quality Assurance and   |Version:
  Testing|
 Severity:  Normal   | Resolution:
 Keywords:  tbb-testsuite, boklm201904,  |  Actual Points:
  TorBrowserTeam201904   |
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by boklm):

 * keywords:  tbb-testsuite, boklm201811, TorBrowserTeam201904 => tbb-
 testsuite, boklm201904, TorBrowserTeam201904


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs