Re: Intent to prototype: Delegate and restrict permission in third party context

[Nils Ohlmeier]

Hi Thomas,

Thank you for pushing feature policy over the finish line and making the web a 
safer place!

Best
  Nils Ohlmeier

> On 25Nov, 2019, at 04:41, Thomas Nguyen  wrote:
> 
> Summary: People don’t have a good understanding of iframes, because
> generally, no UI indicates that iframes are visible on a page, or what
> their origin is. Permission requests from iframes cause significant
> confusion for users because it is hard to determine where the requests come
> from, as the address bar does not match the site in the permission prompt.
> 
> Currently, Firefox allows iframes on a site to make permission requests and
> show up a permission prompt using the origin of the iframes. A user making
> a decision based on the third party context presented in the notification
> prompt is complicated and confusing. This confusion is exacerbated when
> managing previously stored permission decisions.
> 
> To address this problem, we would like to impose a restriction on
> permissions coming from third party context. There would be two main
> changes proposed:
> 
>   -
> 
>   Give an ability to delegate permissions from first party to third party
>   embedded iframes, and impose a restriction to embedded iframes to request
>   permission only when the iframe’s embedder has explicitly delegated it. The
>   permission request will use the top level origin to show in the prompt,
>   then users are only required to make permission decisions about the first
>   party context.
>   -
> 
>  This change is dependent on the ability of Feature Policy to disable
>  permissions by default in cross-origin iframes. It will require a site to
>  explicitly allow permissions for cross-origin iframes (setting allow
>  attribute, e.g allow=”geolocation”) otherwise, the permission
> requests will
>  be denied on that iframes.
>  -
> 
>  The change will be applied to geolocation, camera, microphone and
>  screen-sharing permission, and fullscreen request.
> 
> 
>   -
> 
>   Completely deny permissions from third party context for vibration,
>   notification, and persistent-storage permission.
> 
> 
> The plan is:
> 
>   -
> 
>   Enable Feature Policy allow attribute.
>   -
> 
>   Make permission camera/microphone/geolocation/display-capture/fullscreen
>   disabled by default in third-party iframe.
>   -
> 
>   Delegate Permissions: only cross-origin iframes that have explicit
>   delegated permission from their parent through the allow attribute will
>   have the right to make permission requests.
>   -
> 
>   Reduce the number of supported features to geolocation, camera,
>   microphone screen-sharing, and fullscreen (the above features are supported
>   for permissions UI with notification prompts, except fullscreen). And we
>   will move all other features to experimental phrase under a user preference
>   which is disabled by default.
>   -
> 
>   Simplify prompts/dialogs to only contain the top-level origin.
>   -
> 
>   Deny vibration, persistent-storage permission from third party iframe
>   (notification permission was disabled in third party context,  just do some
>   minor refactors).
> 
> 
> 
> 
> Bug: The tracking bug https://bugzilla.mozilla.org/show_bug.cgi?id=1572461
> 
> Standard: Feature Policy
> https://w3c.github.io/webappsec-feature-policy/#iframe-allow-attribute
> 
> Platform coverage: All.
> 
> Preference:
> 
> dom.security.featurePolicy.experimental.enabled: disabled by default, we
> will limit supported features in Feature Policy to geolocation, camera,
> microphone, fullscreen, display-capture and move others to experimental
> phase.
> 
> permissions.delegate.enabled: enabled by default
> 
> dom.security.featurePolicy.enabled: this pref is implemented in Firefox 65
> but enabled by default in Nightly only
> 
> Other browsers: Chrome supports permission delegation from Chrome 71.
> 
> web-platform-tests: We only have web platform tests for feature policy but
> not permission delegation
> 
> Some of Feature Policy web-platform-tests that the permissions are disabled
> by default in cross origin iframe:
> 
> https://searchfox.org/mozilla-central/source/testing/web-platform/meta/feature-policy
> 
> testing <https://searchfox.org/mozilla-central/source/testing>/web-platform
> <https://searchfox.org/mozilla-central/source/testing/web-platform>/tests
> <https://searchfox.org/mozilla-central/source/testing/web-platform/tests>/
> permissions
> <https://searchfox.org/mozilla-central/source/testing/web-platform/tests/permissions>
> /feature-policy-permissions-query.html
> <https://searchfox.org/mozilla-central/source/tes

Intent to unship: DTLS 1.0 for WebRTC

[Nils Ohlmeier]

With the intent to unship TLS 1.0 and 1.1 
https://groups.google.com/forum/#!topic/mozilla.dev.platform/8EFRYDR3N1c 
<https://groups.google.com/forum/#!topic/mozilla.dev.platform/8EFRYDR3N1c> we 
don’t want to leave Firefox users left with the old DTLS 1.0 when using WebRTC.

The latest draft on WebRTC security architecture (which soon going to be 
published as an RFC) requires all implementations to support DTLS 1.2
https://tools.ietf.org/html/draft-ietf-rtcweb-security-arch-20#section-6.5 
<https://tools.ietf.org/html/draft-ietf-rtcweb-security-arch-20#section-6.5>

In Firefox 71 we landed user prefs which enables developers to test their 
WebRTC services with DTLS 1.2 only.

Chrome has announced to also turn off DTLS 1.0 for WebRTC in M81 
https://groups.google.com/forum/?utm_medium=email_source=footer#!topicsearchin/discuss-webrtc/dtls;context-place=searchin/discuss-webrtc/PSA$3A/discuss-webrtc/Dsq_14_WoUk
 
<https://groups.google.com/forum/?utm_medium=email_source=footer#!topicsearchin/discuss-webrtc/dtls;context-place=searchin/discuss-webrtc/PSA$3A/discuss-webrtc/Dsq_14_WoUk>

Last time when we measured DTLS 1.0 usage was 1.88% in Firefox 68 Beta 
https://telemetry.mozilla.org/new-pipeline/dist.html#!cumulative=0_date=2019-06-18_spill=0=__none__!__none__!__none___channel_version=beta%252F67=WEBRTC_DTLS_PROTOCOL_VERSION_channel_version=null=*=Firefox=0_by_value=0_keys=submissions_date=2019-03-10=0=0_submission_date=0
 
<https://telemetry.mozilla.org/new-pipeline/dist.html#!cumulative=0_date=2019-06-18_spill=0=__none__!__none__!__none___channel_version=beta%2F67=WEBRTC_DTLS_PROTOCOL_VERSION_channel_version=null=*=Firefox=0_by_value=0_keys=submissions_date=2019-03-10=0=0_submission_date=0>

We want to disable DTLS 1.0 in WebRTC together with TLS 1.0 and 1.1 in March 
2020.

Disabling DTLS 1.0 is tracked at 
https://bugzilla.mozilla.org/show_bug.cgi?id=1506392 
<https://bugzilla.mozilla.org/show_bug.cgi?id=1506392>

Best
  Nils Ohlmeier

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to Ship New Certificate Viewer

[Nils Ohlmeier]

Wow. This is awesome. So much better then the old certificate viewer!

Thanks
  Nils

> On 16Aug, 2019, at 12:52, Danielle Leblanc-Cyr  
> wrote:
> 
> Hi everyone,
> 
> We’re Carolina and Danielle, two Outreachy interns who have been working
> with the Security Engineering team. We’ve spent the past few months working
> on porting over pieces of the Certainly Something
>  web
> extension (shout out to April for all of the work she put into it!), to
> create a new certificate viewer. We’re now at the point where we’ve landed
> most of our patches and the project is really coming together – we’re
> hoping to soon replace the existing Firefox certificate viewer.
> 
> We’ve worked really hard the past few months – getting accustomed to the
> Firefox codebase and workflow, learning about new tools, and creating as
> many tests as we could think of. We really appreciate all of the people and
> channels who’ve answered our (many) questions this summer, and who’ve
> helped to steer us in the right direction - especially our mentors, Johann,
> Dana and April.
> 
> The new implementation currently lives behind pref
> "security.aboutcertificate.enabled", which bug 1572368 will default to
> true. We’re hoping for this new certificate viewer to ride the trains in
> Firefox 71.
> 
> The summer has flown by, but we’re excited to continue contributing to
> Mozilla and to remain active members of this awesome community.
> 
> Thanks for all of the support!
> 
> This is the link to the web-extension which we used as a blueprint:
> https://github.com/april/certainly-something
> 
> And here is all the development of the project:
> https://bugzilla.mozilla.org/show_bug.cgi?id=1553524
> 
> Carolina and Danielle
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Detecting and mitigating DNS attacks (DNSCatcher project)

[Nils Ohlmeier]

Hi Michael,

> On 10Jul, 2019, at 13:53, Michael Casadevall  wrote:
> 
> Hello all,
> 
> I am working on a research project called DNSCatcher which is designed
> to provide a level of validation and security to standard DNS.
> DNSCatcher is designed as a framework to prevent clients from being
> redirected to malicious records and detect potential MITN attacks.  A
> technical writeup of the project, problem statement, and its modus
> operandi are available here:
> https://github.com/NCommander/dnscatcher/blob/master/doc/technical_overview.md.

I have to admit that I have not read your full description.
But I have not found any reference to the RIPE Atlas project 
https://atlas.ripe.net/ <https://atlas.ripe.net/>
I think it would be worth checking what can be achieved with RIPE Atlas and 
what is missing from that project in comparison to yours.

Best regards
  Nils Ohlmeier

> It is my intent to design a system that can be widely adopted to help
> understand the health and security of the DNS ecosystem. To that end, I
> would like to get feedback from the Mozilla community on this proposal
> and help craft it into a component that can easily be deployed.
> 
> As of the time of this email, the current proof-of-concept code is
> written in Ada. I intend to standardize the protocol and submit it to
> the IETF for publication. In the interests of full disclosure, I am
> currently seeking funding from the OTF to complete this project,
> although I do intend to work on it regardless of whether funding is
> secured or not.
> 
> For implementation as a browser extension, it appears that Mozilla only
> offers the browser.dns API to make lookups, and is extremely limited.
> Given the constraints of the BrowserExtension API, it appears  that if I
> wish to have full functionality for this project, I will need to deploy
> a client daemon on the end user system to provide an HTTP interface on
> 127.0.0.1. I am open to advice on better mechanisms to achieve this goal.
> 
> It is my hope that as this project develops and matures that support for
> this extension could eventually make its way into Mozilla’s core
> libraries as a native implementation. While I realize we are far from
> that point, I welcome any feedback or criticisms of the design of this
> project.
> 
> Michael
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Next Year in web-platform-tests - 2018/19 Edition

[Nils Ohlmeier]

> On 7Dec, 2018, at 07:45, James Graham  wrote:
> 
> I would also very much like to hear about remaining pain points and reasons 
> that developers choose to write browser-specific tests for web-platform 
> features. Those are issues we need to prioritize fixing.

Without going too much into the details, but in my opinion WebRTC is far, far 
away from being able to be tested sufficiently via web-platform tests. The list 
is long: browser interoperability, different network topologies affecting call 
establishment, network conditions affecting call quality, dependencies on 
external servers (depending on the network), not a common A/V test input 
format,… We do have improved in some of these areas for example in mochitests 
only.

I don’t think this is something the WebRTC ecosystem can solve short term. In 
the long run some of these could probably be tackled. But I’m raising this to 
clarify that WPT as of right now can not solved all existing problems.

Best
  Nils Ohlmeier
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Intent to remove DHE ciphers in Fx 64 (Was: Intent to remove DHE ciphers from WebRTC DTLS handshake)

[Nils Ohlmeier]

As the Telemetry data [1] shows no significant usage of the DHE ciphers in Beta 
63 and Nightly 64 we are planing to remove the DHE ciphers in Firefox 65.

Please voice your concerns now, if you have any.

Best
  Nils Ohlmeier

[1] 
https://telemetry.mozilla.org/new-pipeline/dist.html#!cumulative=0_date=2018-10-01_spill=0=__none__!__none__!__none___channel_version=beta%252F63=WEBRTC_DTLS_CIPHER_channel_version=nightly%252F60=*=Firefox=0_keys=submissions_date=2018-09-04=1=0_submission_date=0

> On Aug 29, 2018, at 3:56 PM, Nils Ohlmeier  wrote:
> 
> Summary:
> 
> We are looking at removing the DHE cipher suites from the DTLS handshake in 
> Firefox soon.
> 
> Ciphers:
> - TLS_DHE_RSA_WITH_AES_128_CBC_SHA 
> - TLS_DHE_RSA_WITH_AES_256_CBC_SHA
> are the  two suites which we want to remove, because they are considered too 
> weak.
> 
> A Telemetry probe landed in Firefox 63 Nightly to monitor the usage of the 
> different cipher suites:
> https://telemetry.mozilla.org/new-pipeline/dist.html#measure=WEBRTC_DTLS_CIPHER
>  
> <https://telemetry.mozilla.org/new-pipeline/dist.html#measure=WEBRTC_DTLS_CIPHER>
> 
> Bug tracking the deactivation:
> https://bugzilla.mozilla.org/show_bug.cgi?id=1227519 
> <https://bugzilla.mozilla.org/show_bug.cgi?id=1227519>
> 
> Targeted release: Firefox 66
> 
> Best
>   Nils Ohlmeier

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Intent to remove DHE ciphers from WebRTC DTLS handshake

[Nils Ohlmeier]

Summary:

We are looking at removing the DHE cipher suites from the DTLS handshake in 
Firefox soon.

Ciphers:
- TLS_DHE_RSA_WITH_AES_128_CBC_SHA
- TLS_DHE_RSA_WITH_AES_256_CBC_SHA
are the  two suites which we want to remove, because they are considered too 
weak.

A Telemetry probe landed in Firefox 63 Nightly to monitor the usage of the 
different cipher suites:
https://telemetry.mozilla.org/new-pipeline/dist.html#measure=WEBRTC_DTLS_CIPHER 
<https://telemetry.mozilla.org/new-pipeline/dist.html#measure=WEBRTC_DTLS_CIPHER>

Bug tracking the deactivation:
https://bugzilla.mozilla.org/show_bug.cgi?id=1227519 
<https://bugzilla.mozilla.org/show_bug.cgi?id=1227519>

Targeted release: Firefox 66

Best
  Nils Ohlmeier


signature.asc
Description: Message signed with OpenPGP
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to ship MediaSource SourceBuffer.changeType

[Nils Ohlmeier]

Which version of Firefox are you planing to ship this?

Thanks
  Nils

> On Aug 6, 2018, at 02:37, Jean-Yves Avenard  wrote:
> 
> Summary:
> 
> enable by default changeType method on MediaSource’s Source Buffer, allowing 
> to change content type (codecs and/or container) on the fly…
> The method has been availably since 61 behind the preference 
> media.mediasource.experimental.enabled
> 
> Bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1481166 
> 
> 
> Detail:
> 
> Up to now, when using Media Source Extension, you had to create a source 
> buffer of a specific type using the MediaSource.addSourceBuffer method, 
> providing the mime type describing the container and optionally the codec. 
> You could then no longer change the container nor the codec.
> 
> Comes changeType , it allows to mix different codec within the same video 
> element.
> One particular use case would be to use different codecs according to the 
> selected resolution.
> 
> Like using AV1 for the very low bitrate due to the exceptional performance of 
> AV1 there, and switch later to VP9 or H264 as they are a bit more friendly 
> resource-wise.
> 
> JY___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform



signature.asc
Description: Message signed with OpenPGP
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Compile webrtc from source in firefox

[Nils Ohlmeier]

> On Jul 6, 2018, at 01:15, amantell...@gmail.com wrote:
> 
> Il giorno giovedì 5 luglio 2018 10:17:32 UTC+2, amant...@gmail.com ha scritto:
>> Hi,
>> I don't understand which webrtc code is used by firefox builder.
>> I need to modify the webrtc part and recompile it to be used by firefox.
>> any help?
>> 
>> Angelo
> 
> Hi,
> I tried to break webrtc to understand if it is compiled or not (in particular 
> media/webrtc/trunk/webrtc/pc/channel.cc), but i did not receive errors.
> so I think that firefox used an external library about webrtc?

No Firefox does not use an external library for webrtc.
You can take a look here 
https://searchfox.org/mozilla-central/source/media/webrtc/trunk/moz.build 
<https://searchfox.org/mozilla-central/source/media/webrtc/trunk/moz.build> to 
get an idea which WebRTC files get compiled into Firefox.

Since this discussion has reached pretty low levels of WebRTC it might be time 
to move this conversation to dev-media, which has more expert subscribers then 
this generic mailing list.

Best
  Nils Ohlmeier



signature.asc
Description: Message signed with OpenPGP
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: open socket and read file inside Webrtc

[Nils Ohlmeier]

> On Jul 6, 2018, at 08:26, Randell Jesup  wrote:
> 
>>> On 05/07/2018 10:16, amantell...@gmail.com wrote:
>>>> I want to open a file inside webrtc core part.
>>> 
>>> If you could extend the reasons on the why it would allow us to help you.
>>> 
>>> What is your ultimate objective? why do you think you need to open a
>>> file inside webrtc core?
> 
>> We are modifying the webrtc library to use another transport protocol (based 
>> on ICN).
>> The problem is that in this preliminary phase we have configure the socket 
>> using static parameters inside a file.
>> The new socket is implemented in a library that communicates via TCP with an 
>> external daemon.
>> I want to use IPC, but I don’t have examples about this approach.
> 
> If this is just for personal experimentation, you can locally disable
> the sandbox via about:config, and directly (locally) modify the
> media/mtransport code to use your transport protocol instead of UDP via
> IPC to the SocketTransportThread in the main process.

Besides the sandbox you might want to start your locally compiled Firefox with 
‘./mach run —disable-e10s’, because then it will use this networking code 
https://searchfox.org/mozilla-central/source/media/mtransport/nr_socket_prsock.h#151
 
<https://searchfox.org/mozilla-central/source/media/mtransport/nr_socket_prsock.h#151>
rather then the IPC based code here 
https://searchfox.org/mozilla-central/source/media/mtransport/nr_socket_prsock.h#214
 
<https://searchfox.org/mozilla-central/source/media/mtransport/nr_socket_prsock.h#214>

> Note that there's a lot of code that implements ICE to determine what
> ports are open, etc.  That may complicate what you're doing.
> 
> If you want to do more than personal/local experimentation, much more
> extensive discussions and work would be required.

If you are part of the IRTF research group and you are going to be at the 
Montreal meeting it might be the easiest to find me (or someone else from 
Mozilla) to discuss this in person.

Best
  Nils Ohlmeier


signature.asc
Description: Message signed with OpenPGP
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Compile webrtc from source in firefox

[Nils Ohlmeier]

> On Jul 5, 2018, at 10:34, Jean-Yves Avenard  wrote:
> On 05/07/2018 10:17, amantell...@gmail.com wrote:
>> Hi,
>> I don't understand which webrtc code is used by firefox builder.
>> I need to modify the webrtc part and recompile it to be used by firefox.
>> any help?
>> 
>> 
> 
> Firefox use its own copy, the code is found in media/webrtc

Firefox uses a stripped down version of the webrtc.org <http://webrtc.org/> 
code. You can find the code in media/webrtc/trunk, e.g. 
https://searchfox.org/mozilla-central/source/media/webrtc/trunk 
<https://searchfox.org/mozilla-central/source/media/webrtc/trunk>
Everything in media/webrtc, but outside of the trunk subdirectory is Mozilla’s 
own code, which is not part of the webrtc.org <http://webrtc.org/> code base.
Note: the code used in Firefox is also several versions behind current 
webrtc.org <http://webrtc.org/> (right now Firefox is at version 57).

Best
  Nils Ohlmeier




signature.asc
Description: Message signed with OpenPGP
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to implement: AudioWorklet

[Nils Ohlmeier]

Looks like the existing WebAudio issue 
https://github.com/WebAudio/web-audio-api/issues/1471 
<https://github.com/WebAudio/web-audio-api/issues/1471>
is where this should get resolved (for people who haven’t switched over there 
yet).

Best regards
  Nils Ohlmeier

> On May 2, 2018, at 09:05, hongc...@chromium.org wrote:
> 
> It's great to see the intent for the AudioWorklet, but also I can see the GC 
> observability is still being discussed here.
> 
> Karl, can you open a new spec issue if you think this needs another look from 
> AudioWG and TAG?
> 
> -Hongchan
> 
> 
> On Wednesday, May 2, 2018 at 8:17:30 AM UTC-7, Boris Zbarsky wrote:
>> On 5/2/18 5:21 AM, Karl Tomlinson wrote:
>>> [[AudioNode Lifetime]] https://github.com/WebAudio/web-audio-api/issues/1471
>> 
>> I've read through that thread, but I'm still a little unclear on where
>> thing stand.  With the latest proposal, can there be observable
>> situations where the output sound depends on GC timing?
>> 
>> If so, that doesn't sound acceptable.  It also sounds to me like Alex
>> Russell in
>> https://github.com/WebAudio/web-audio-api/issues/1471#issuecomment-362604396
>> didn't believe there was an observable sound difference possible.  If
>> one is possible, we should communicate this to him very clearly
>> 
>> -Boris
> 
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform



signature.asc
Description: Message signed with OpenPGP
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: disabled non-e10s tests on trunk

[Nils Ohlmeier]

> On Aug 16, 2017, at 07:23, James Graham <ja...@hoppipolla.co.uk> wrote:
> 
> On 16/08/17 01:26, Nils Ohlmeier wrote:
>> I guess not a lot of people are aware of it, but for WebRTC we still have 
>> two distinct implementations for the networking code.
>> So if I understand the impact here right we just lost test coverage for 
>> probably a couple of thousand lines of code.
> […]
> 
>> I’m not sure how others do it, but our low level C++ unit tests don’t have 
>> an e10s mode at all.
>> Therefore we can’t simply delete the non-e10s WebRTC networking code either 
>> (without loosing a ton of test coverage).
> 
> If the networking code is only covered by C++ unit tests, there is separate 
> code for non-e10s vs e10s,  and the unit tests don't work in e10s mode 
> doesn't that mean we currently don't have any test coverage for our shipping 
> configuration on desktop? What am I missing?

So we have mochitest-media which works as kind of integration test on a higher 
level. They execute high level JS API tests, but also try to ensure that the 
lower level networking pieces (the once which are exposed through JS) match the 
expectations.
The mochitest-media got executed for e10s and non-e10s and therefore covered 
both implementations.

And then we have C++ unit tests, which cover a lot more corner cases of 
different scenarios for networking. And yes these only work with non-e10s right 
now. It would be a lot of work to create the same amount of tests with a higher 
level test suite like mochitest to get the e10s coverage. Plus these tests 
would probably take a lot execution time.

Technically that leaves us with a somewhat blind spot for the coverage of 
networking corner cases under e10s. I guess if there is a high demand for 
turning off all non-e10s tests we need to look at how to get our C++ unit tests 
working with something like e10s.
But until we can get to that I think we really should keep running 
mochitest-media with e10s and without it.

Best
  Nils Ohlmeier




signature.asc
Description: Message signed with OpenPGP
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: disabled non-e10s tests on trunk

[Nils Ohlmeier]

I guess not a lot of people are aware of it, but for WebRTC we still have two 
distinct implementations for the networking code.
So if I understand the impact here right we just lost test coverage for 
probably a couple of thousand lines of code.

And yes you can potentially blame it on WebRTC networking for not having 
unified it’s networking code for e10s and non-e10s.
But when e10s got turned on I asked around if we could delete the non-e10s code 
soon, I was told no.
So I assumed both technologies would stick around for some more time.

I’m not sure how others do it, but our low level C++ unit tests don’t have an 
e10s mode at all.
Therefore we can’t simply delete the non-e10s WebRTC networking code either 
(without loosing a ton of test coverage).

With this being said I think the right thing here would be to turn the non-e10s 
mochitests back on for WebRTC until we have a better solution.

  Nils

> On Aug 15, 2017, at 13:54, J. Ryan Stinnett  wrote:
> 
> I think Ben's argument has merit:
> 
> 1. Even after Firefox 57, we will still be shipping a product in non-e10s
> mode: Firefox for Android
> 2. If WPT (and potentially other suites) aren't being run in non-e10s mode,
> it increases risk because we are shipping untested code paths to our users
> 
> Someone might add code that assumes e10s is the only mode and land it
> successfully, only later to hear that it fails or crashes on Android, since
> e10s doesn't exist there today.
> 
> - Ryan
> 
> On Tue, Aug 15, 2017 at 3:43 PM, Joel Maher  wrote:
> 
>> This is a discussion about tests in e10s mode, not WPT on Android.
>> 
>> What specific coverage are we missing by not running WPT in non-e10s mode
>> on desktop?  When can we ensure we have that coverage in e10s mode?  I
>> assume this is just making sure the tests that we have disabled on e10s
>> mode need to get fixed.
>> 
>> On Tue, Aug 15, 2017 at 4:39 PM, Ben Kelly  wrote:
>> 
>>> On Tue, Aug 15, 2017 at 4:37 PM, Joel Maher  wrote:
>>> 
 All of the above mentioned tests are not run on Android (well
 mochitest-media is to some degree).  Is 4 months unreasonable to fix the
 related tests that do not run in e10s?  Is there another time frame that
 seems more reasonable?
 
>>> 
>>> Last I checked it was your team that told me WPT on android was not an
>>> immediate priority.  The WPT harness itself does not run there.
>>> 
>> ___
>> dev-platform mailing list
>> dev-platform@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/dev-platform
>> 
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform



signature.asc
Description: Message signed with OpenPGP
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Please do NOT hand-edit web platform test MANIFEST.json files

[Nils Ohlmeier]

> On Mar 17, 2017, at 11:01, Boris Zbarsky  wrote:
> 
> We have tools for this: "mach wpt-manifest-update" will do the right thing.
> 
> The typical result of hand-edits is that later changesets that do use the 
> tools end up conflicting with each other, as they all fix up the incorrect 
> hand-edits.  Please don't cause this pain for other developers and the 
> sheriffs.

Wouldn’t it make more sense to let the build system detect and reject/warn 
about (?) such a manual modification?
My assumption here is that mailing list archives are not a good place to 
document processes or systems.

Best
  Nils


signature.asc
Description: Message signed with OpenPGP
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Checkin-needed requests - Please include complete information in the commit message :)

[Nils Ohlmeier]

> On Jul 10, 2016, at 21:18, Xidorn Quan  wrote:
> 
> On Mon, Jul 11, 2016, at 12:29 PM, Martin Thomson wrote:
>> Is now the right time to start talking about retiring checkin-needed,
>> or is it still heavily used?
> 
> Isn't it still necessary for people who don't yet have permission to
> push?

Another use case for checkin-needed are probably sec bugs, as you can’t use 
mozreview for them AFAIK.

  Nils


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to deprecate: MacOS 10.6-10.8 support

[Nils Ohlmeier]

> On May 10, 2016, at 19:58, Lawrence Mandel <lman...@mozilla.com> wrote:
> 
> The post states "Mozilla will end support for Firefox on OS X 10.6, 10.7,
> and 10.8 in August, 2016." This means that we will end support with the
> Firefox 48 release. i.e. Firefox 48 will not support OS X 10.6-10.8.

Why do Firefox DevEdition users on Mac OS X 10.6 then still get updates to 
DevEd 48?

I just tested it on a 10.6 machine and updates have stopped for Nightly. Which 
is good, because the Nightly 49 build which I manually downloaded crashes and 
brings up the Mac OS X crash reporter.
But the DevEdition installed on the same system updated itself to 48. And 48 
started and appeared to be still working.

Best
  Nils Ohlmeier


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to deprecate: MacOS 10.6-10.8 support

[Nils Ohlmeier]

> On May 3, 2016, at 15:18, Adam Roach <a...@mozilla.com> wrote:
> 
> On 5/3/16 4:59 PM, Justin Dolske wrote:
>> On 5/3/16 12:21 PM, Gregory Szorc wrote:
>> 
>>> * The update server has been reconfigured to not serve Nightly updates to
>>> 10.6-10.8 (bug 1269811)
>> 
>> Are we going to be showing some kind of notice to affected users upon 
>> Release? That is, if I'm a 10.6 user and I update to Firefox 48, at some 
>> point should I see a message saying I'll no longer receive future updates?
> 
> Even better, is there any way to get the update system to automatically move 
> such users over to 45ESR?

I agree. At least users should get updated to the latest version in their 
channel. If not automatically or getting pointed to manually switch to the 
latest secure release for their version/channel.

Right now I started a Nightly 40 on 10.6.8 and it simply refused to do any 
update at all. That is less then ideal I would say.

Best regards
  Nils Ohlmeier


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Moving XP to ESR?

[Nils Ohlmeier]

> On Apr 18, 2016, at 09:56, Milan Sreckovic  wrote:
> 
> What’s the “XP tax”?  Graphics usually tries to simplify the playing field as 
> much as possible, but I can’t say that XP has been causing any trouble, or 
> that we have been getting too many XP specific problems (certainly fewer than 
> Windows 10 :)

Well as we still have to compile for XP there are certain API’s which we can’t 
use like here:
https://dxr.mozilla.org/mozilla-central/source/media/mtransport/third_party/nICEr/src/stun/addrs.c#228

The good news is that dxr does not find anything using IsXPSP3OrLater(). But 
this looks like we have a bit of version specific code in our tree:
https://dxr.mozilla.org/mozilla-central/search?q=XP_WIN=false=true

And on top of that come the costs for maintaining XP machines as part of the 
treeherder setup.

So it is not easy to quantify, but there is a “XP tax” we pay.

Regards
  Nils


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to deprecate: MacOS 10.6-10.8 support

[Nils Ohlmeier]

> On Mar 10, 2016, at 10:03, Benjamin Smedberg <benja...@smedbergs.us> wrote:
> 
> This is notice of an intent to deprecate support within Firefox for the 
> following old versions of MacOS: 10.6, 10.7, and 10.8

Excuse my ignorance, but what means “deprecate support” exactly?

I’m only asking because of the opposing reply’s so far. I’m assuming it means 
we stop testing and building/releasing for these. Would it be a possible 
alternative to turn of the tests, but continue to build and release unsupported 
builds?
I know that this only prolongs it and doesn’t help with regards to the C++ 
problems, but would be interested in the counter arguments for such a “middle 
ground solution”.

Best regards
  Nils Ohlmeier



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: PSA: Flaky timeouts in mochitest-plain now fail newly added tests

[Nils Ohlmeier]

 On Dec 19, 2014, at 6:56 AM, Ehsan Akhgari ehsan.akhg...@gmail.com wrote:
 
 Let me try to rephrase the problem in different terms, to hopefully make it 
 clearer why using timers like this is a bad idea.
 
 setTimeout(foo, 1000) may seem to suggest run foo after 1 second, but that is 
 *not* what that function does, at all.  What it does is, run foo after 1 
 second, or perhaps at any other point in time later than that, and it could 
 even be *minutes* later.
 
 A lot of people are under the assumption that just because their local 
 timings pan out, and they've added a, let's say, 10x error margin to their 
 timers, their use of timers is fine.  It's not, as has been demonstrated as 
 intermittent oranges over the years.  We have no way of knowing whether those 
 calculations hold under various types of machine loads, on new platforms, on 
 much lower powered machines (think of running the tests where the such 
 measurements have been done on a beefy desktop on a low-end phone.)  And 
 increasing the error margin won't save us, it will just move the problem 
 further into the future.
 
Ok. That’s fine by my.
First of all I assume that the slowness of the machine also effects mochitest’s 
over all timer, right? If not, then we are in big trouble.
Secondly remember that I’m NOT doing anything useful when my 
“missing-event-timer” pops. I’m just logging a message that the event is 
missing and then I’m trying to exit the test early (earlier then the mochitests 
general test timeout).

The only problem I can see with that approach is: if the “missing-event-timer” 
actually pops more or less precisely after the requested time, BUT the actual 
work is running super slow motion for some reason. And so the event would have 
arrived, but after the even-timeout.
So if my “missing-event-timers” are popping on time, because they are for some 
bizarre reason not affected by the over all slowness of the machine then we 
have a problem with this. But in that case we should have a problem with the 
over test timeout of mochitest as well, or?

 Also, please note that letting the test time out if it doesn't succeed is a 
 perfectly valid failure scenario, we have numerous tests that do things like 
 fire an async event, setup a listener and wait for the listener to fire and 
 SimpleTest.finish() there, without any timers to catch the case where the 
 event does not fire.

My test are not about testing this one timer. Doing that would be relatively 
easy.
In WebRTC I need to orchestrate two participants to establish a real time audio 
video call. Lots of the tests need to establish a full call, before assessing 
if the call got properly established under the given criteria. The most common 
scenario for intermittent failures is that the call does not even gets 
established. From start to a full call I need to verify several state machines 
and lots of events firing and all of that twice for the two participants of 
each call.
So yes I could simply try to establish a call (and not worrying about all the 
intermediary steps and events) and then do the verification/assessment. And let 
call failures be caught by the generic mochitest timeout. But then I would 
spend hours and hours staring at pretty useless log files (see below) every 
time a call got stuck.

  Logging sufficiently is almost always enough to not have to use these 
 timers, as those tests have demonstrated in practice.
 

I disagree. Especially as the logging of mochitest is really poor.
- the timestamps of log messages are lost as described in bug 1020516
- SimpleTest only has one log level: info(). Which I think is the main reason 
the log files get big, because test writers can’t fine tune the logging.
- Log messages from mochitest are not printed together with the stdout/stderr 
of the browser (so correlating these two is almost impossible)
- Logging behaves differently on your local machine then on the build servers

BTW I understand that WebRTC has pretty special and probably unique 
requirements, compared to what the rest of the mochitest users do with it. 
That’s why I was trying to make the case for adding a special connivence 
function for setting up these special “missing-event-timer”, which BTW would 
also allow us to increase or decrease the timeout values based on the 
platform/machine the test currently gets executed on.

But as I don’t seem to be able to make my case here I’ll simply live with 
requesting flaky timeouts for all WebRTC tests.

Best
  Nils

 Cheers,
 Ehsan
 
 On 2014-12-19 1:07 AM, Boris Zbarsky wrote:
 On 12/18/14, 2:28 PM, Nils Ohlmeier wrote:
 Well there is an event listener waiting for the event to fire.
 But how else then through a timeout, obviously with a high timeout
 value like
 30 or 60 seconds
 
 We've had quite a number of random oranges from precisely this scenario.
  It seems that it's not uncommon for the test VMs to completely lose
 the timeslice for 60 seconds or more.
 
 If the test is in fact expecting

Re: PSA: Flaky timeouts in mochitest-plain now fail newly added tests

[Nils Ohlmeier]

 On Dec 12, 2014, at 10:34 AM, Ehsan Akhgari ehsan.akhg...@gmail.com wrote:
 
 We had a session on intermittent test failures in Portland 
 https://etherpad.mozilla.org/ateam-pdx-intermittent-oranges, and one of
 the things that we discussed was adding analyses to our test suites that
 detect known bad test writing practices 
 https://developer.mozilla.org/en-US/docs/Mozilla/QA/Avoiding_intermittent_oranges
 and fails the offending tests.  As part of that effort, I just landed the
 patch to bug 649012 which enables this analysis on mochitest-plain.

Great idea.

 What this means for you as a test author:
 * Using setTimeout(foo, N); where N is not 0 will cause your tests to
 fail.  You should double check why you're using such timeouts.  They are
 almost never necessary.
 * If you have a legitimate reason for using timeouts like this (such as if
 your test needs to wait a bit to make sure an event doesn't happen in the
 future

What about the other way around?
My test is waiting for an event to proceed with the test. All of our WebRTC
mochitests highly depend on events from the WebRTC API’s. Now the
obvious answer is: mochitest has a generic timeout of 300 seconds (I think)
which will terminate the test in case the expected event never fired. The
problem I have with that is that as said before the WebRTC tests depend
on whole sequence of events to happen. And if the generic mochitest timeout
kills a test it is very hard to tell which events exactly was missing, 
especially
as we are orchestrating two actors in the browser and both wait for the same
events.
For that reason I got used to the habit of creating a timeout while the test is
waiting for an event to happen, because if that timer fires I can precisely
print an error message what is missing and why test is exiting early (besides
the fact that the test is not blocking for another 2xx seconds).

So while I totally agree with the description here 
https://developer.mozilla.org/en-US/docs/Mozilla/QA/Avoiding_intermittent_oranges
that it is bad to magically wait for time and hope that it is safe to proceed
with your test, I do not agree with the assessment that setTimeout in general
is evil.

 , and there is no other way of checking that), you should call
 SimpleTest.requestFlakyTimeout(reason);.  The argument to the function is
 a string which is meant to document why you need to use these kinds of
 timeouts, and why that doesn't lead into intermittent test failures.

If we really want to have people not use setTimeout at all in test, but have a
legitimate use case for it:
- your test needs to wait for X time and check that either an event happened
or is missing and exit the test execution if the check failed
Can we then maybe add a connivence function for exactly that purpose?
I would prefer a connivence function, because that would allow us to still
catch the evil wait magically for a little bit as bad behavior. Where right now
I have no other choice then requesting to lift the whole protection via
requestFlakyTimeout().

What I really do not like is the name chosen for this, as it seems to suggest
that timeouts in general are flaky, which they are not (see above).

Can we please rename this to something more appropriate like
  SimpleTest.requestUnreliableTimeout(“reason”);
Because I think with “flaky” you mean unreliable.

And while we are on improving this: please, please, please improve the error
message. Right now someone who writes a new test he will see something
like this in his try logs:
 Test attempted to use a flaky timeout value 5000
And he will be baffled like me and some colleagues this morning about the
error message and what it means. Would it have been acceptable if I 
would have used 4999 instead of 5000? In case of the WebRTC tests you can
very easily write a legit test case without ever calling setTimeout yourself,
but some of the generic helper code will do it for the test writer. And then
the current error message essentially means nothing to him.

Best
  Nils Ohlmeier
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: B2G emulator issues

[Nils Ohlmeier]

Thanks Jonathan for the update.

I would like to point out that at least for the WebRTC tests which do 
test the connection between two WebRTC clients we theoretically also 
have the option to split the tests into two half's (which we do for 
steeplechase tests already anyhow), and then start two emulators on the 
same host machine. This could potentially speed up some of the test 
execution as the emulator seems to utilize only one host CPU.
This approach would have the advantage of scaling well as we could keep 
using EC2. But this doe not help with the media issues Randel was 
describing below.
But if we get to this point it might make sense to evaluate which test 
platform works best for which tests.


Best
  Nils

On 8/28/14 5:28 PM, Jonathan Griffin wrote:
Some more details on how we're approaching this problem from the 
infrastructure side:


Releng recently gave us the ability to run select jobs on faster VM's 
than the default, see 
https://bugzilla.mozilla.org/show_bug.cgi?id=1031083.  We have B2G 
emulator media mochitests scheduled on cedar using these faster VM's.  
After fixing a minor problem with these, we'll be able to see if these 
faster VM's solve the problem.  Local experiments suggest they do, but 
it will take a number of runs in buildbot to be sure.


If that doesn't fix the problem, we have the option of trying still 
faster VM's (at greater cost), or trying to run the tests on real 
hardware.  The disadvantage of running the tests on real hardware is 
that such hardware doesn't scale very readily and is already stretched 
pretty thin, and the emulator doesn't currently run on our linux 
hardware slaves, and will require some amount of work to fix.


This work is being tracked in 
https://bugzilla.mozilla.org/show_bug.cgi?id=994920.


Jonathan


On 8/28/2014 3:06 PM, Randell Jesup wrote:

I wrote in April:
The B2G emulator design is causing all sorts of problems.  We just 
fixed

the #2 orange which was caused by the Audio channel StartPlaying()
taking up to 20 seconds to run (and we fixed it by effectively
removing some timeouts).  However, we just wasted half a week trying to
land AEC  MediaStreamGraph improvements.  We still haven't landed due
to yet another B2G emulator orange, but the solution we used for the 
M10

problem doesn't fix the fundamental problems with B2G emulator.

You can read the earlier thread (starting 7-apr) about this issue.  We
wallpapered over the issues (including turning down 'fake' audio
generation to 1/10th realtime and letting it underflow).

The problems with the b2g emulator have just gotten worse as we add more
tests and make changes to improve the system that give the emulators
fits.

Right now, we're looking at being blocked from landing important
improvements (that make things *not* fail due to perf timeouts in
real-user-scenarios) because b2g-emulator chokes on anything even
smelling of realtime data.  It can stall for 10's of seconds (see
above), or even minutes.  Even running a single test can cause other,
unrelated tests to perma-orange.

The stuff we've had to do (like turning down audio generation) to block
oranges in the current setup makes the tests very non-real-world, and so
greatly diminishes their utility anyways.

There was work being done to move media and other semi-realtime tests to
faster hardware; that is happening but it's not ready yet. (For 
reference,

in April tests showed that a b2g emulator mochitest that took 10
seconds on my Xeon took 350-450 seconds on tbpl.)


The fundamental problem is that b2g-emulator can't deal safely with any
sort of realtime or semi-realtime data unless run on a fast machine.
The architecture for the emulator setup means the effective CPU 
power is

dependent on the machine running the test, and that varies a lot (and
tbpl machines are WAY slower than my 2.5 year old desktop). Combine
that with Debug being much slower, and it's recipe for disaster for any
sort of time-dependent tests.

...

So, what do we do?  Because if we do nothing, it will only get worse.

So we've done nothing (that's landed at least), and it has gotten worse,
and we're at the breaking point where b2g emulator (especially debug)
for media tests (especially webrtc) is providing negative value, and
blocking critically important improvements.

We've just landed bug 1059867 to disable most webrtc tests on the 
emulator

until we can get them running on hardware that has the power to run them
(or other fixes make them viable again (bug 1059878)). We may need to
consider similar measures for other media tests (webaudio, etc). In the
meantime, we're going to try to run local emulator pull/build/mochitest
cronjobs on faster desktop machines (perhaps mine) on a daily or perhaps
continuous basis.  (Poor man's tbpl - maybe I'll un-mothball tinderbox
for some nostalgic flames...)

Also note that webrtc tests do run on the b2g desktop tbpl runs, so we
have some coverage.

I hope we can find a better solution than run it on my dev