Re: Intent to implement and ship navigator.webdriver

[Andreas Tolfsen]

Also sprach Andreas Tolfsen:

> Summary:
> I intend to implement and ship a new enumerable, configurable,
> read-only "webdriver" attribute on the Navigator object.  The
> attribute is true when the -marionette flag is passed to Firefox
> or the marionette.enabled preference is true.  Otherwise it is
> false.

We decided to also throw in a preference dom.webdriver.enabled that
controls the exposure of the navigator.webdriver attribute.  When
flipped to false it will remove the attribute.

I intend to land the patches on https://bugzil.la/1169290 soon.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to implement and ship navigator.webdriver

[Andreas Tolfsen]

Also sprach Martin Thomson:

> On Tue, Feb 6, 2018 at 3:37 AM, Andreas Tolfsen  wrote:
>> Motivation:
>> To give web authors a way to infer if user agent is controlled by
>> automation, so the document can take alternate code paths when under
>> test.
> 
> Can you speak more about why this is a good idea?  I've poor
> experience with "this code is now under test" in the past.  You
> create the risk that code becomes split into two: code that works
> as tested, and code that doesn't work because it isn't tested.
> 
> (Not an objection, there's definitely value in consistency between
> implementations.)

As far as I’m aware the interface was added to the spec at the
explicit request of web authors.

I share your concern of code that gets written for “test only”
environments and that later fails to work in reality.  Another
aspect is that it is also important for test environments to induce
reproducibility.

One part to mitigating instability in tests is to eliminate sources
of indeterministic variance.  For example it might be reasonable
to want to replace Math.random with a PRNG with a fixed seed during
testing.

The most important characteristic of navigator.webdriver is that
it provides consistency.  With navigator.webdriver there’s _one
way_ to indicate to a document that it is under test, and you no
longer need an in-house standard on how to do that.

Sufficiently advanced test environments typically have the ability
to serve entirely custom versions of web apps, but as more application
code lives in the browser (in JS), it becomes vital to have a good
standard for testing also in the web platform layer.

Without exposing this attribute on the navigator object, it becomes
hard to make decisions about code that executes before the document
finishes loading.  By the point WebDriver returns control to the
user after loading a document, it may be too late to invoke
executeScript("window.isTest = true").

Whatever the web author uses navigator.webdriver for, if their test
strategy to begin with is not sound, then there is little we can
do to protect them from using the attribute as a foot-gun.  They
might want to test that their site still works in production before
publishing.

In the Blink intent-to-ship thread [1] they cite anti-abuse use
cases for navigator.webdriver, where a foreign website you don’t
own could use it to block unsolicited bots/crawlers.  As I also
mention in the commit message [2], this is not what the attribute
is designed for.

For better or worse (though probably the better!) the source code
of all the major browser engines are available for anyone to
scrutinise, and given the ability to compile custom browsers it is
not really a technically sound argument to make that navigator.webdriver
can realistically be used as an anti-abuse measure.

  [1] 
https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/6GysDZCWwB8/rXbGoRohBgAJ
  [2] https://reviewboard.mozilla.org/r/216250/diff/4#index_header

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to implement and ship navigator.webdriver

[Andreas Tolfsen]

Also sprach Boris Zbarsky:

> And in particular, the implementation patches don't match the intent
> description (e.g. we implement a configurable property)

Making it unforgeable is obviously a rather bad idea.  I did mean
that the attribute is _configurable_.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to implement and ship navigator.webdriver

[Andreas Tolfsen]

Also sprach Boris Zbarsky:

> On 2/5/18 11:37 AM, Andreas Tolfsen wrote:
>> Standard:
>> https://w3c.github.io/webdriver/webdriver-spec.html#interface
> 
> All the IDL snippets in this standard are empty, which makes it
> hard to evaluate the standard or how our implementation aligns with
> it.

Through a series of unfortunate events, it appears a new version
of ReSpec was released in close proximity to my intent-to-ship
email.  That broke parsing of the WebIDL interface, but I didn’t
notice because I had it cached in my local browsing session.

In any case the interface is now back in the spec, but I will quote
it here for posterity:

> Navigator includes NavigatorAutomationInformation;
> 
> interface mixin NavigatorAutomationInformation {
>   readonly attribute boolean webdriver;
> };

Sorry about that.

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to implement and ship navigator.webdriver

[Martin Thomson]

On Tue, Feb 6, 2018 at 3:37 AM, Andreas Tolfsen  wrote:
> Motivation:
> To give web authors a way to infer if user agent is controlled by
> automation, so the document can take alternate code paths when under
> test.

Can you speak more about why this is a good idea?  I've poor
experience with "this code is now under test" in the past.  You create
the risk that code becomes split into two: code that works as tested,
and code that doesn't work because it isn't tested.

(Not an objection, there's definitely value in consistency between
implementations.)
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to implement and ship navigator.webdriver

[Boris Zbarsky]

On 2/5/18 11:37 AM, Andreas Tolfsen wrote:

Standard:
https://w3c.github.io/webdriver/webdriver-spec.html#interface


All the IDL snippets in this standard are empty, which makes it hard to 
evaluate the standard or how our implementation aligns with it.


-Boris
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to implement and ship navigator.webdriver

[Boris Zbarsky]

On 2/5/18 11:47 AM, Boris Zbarsky wrote:
All the IDL snippets in this standard are empty, which makes it hard to 
evaluate the standard or how our implementation aligns with it.


And in particular, the implementation patches don't match the intent 
description (e.g. we implement a configurable property)


-Boris
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Intent to implement and ship navigator.webdriver

[Andreas Tolfsen]

Summary:
I intend to implement and ship a new enumerable, non-configurable,
read-only "webdriver" attribute on the Navigator object.  The
attribute is true when the -marionette flag is passed to Firefox
or the marionette.enabled preference is true.  Otherwise it is
false.

Motivation:
To give web authors a way to infer if user agent is controlled by
automation, so the document can take alternate code paths when under
test.

Bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=1169290

Standard:
https://w3c.github.io/webdriver/webdriver-spec.html#interface

Estimated target release:
60

Testing:
Covered by WPT.

Do other engines have this?
Blink [1], Edge, and WebKit [2] have this.  Blink only exposes the
attribute during automation, but we expect them to unconditionally
expose it given a recent specification change [3].

  [1] 
https://groups.google.com/a/chromium.org/d/msg/blink-dev/6GysDZCWwB8/rXbGoRohBgAJ
  [2] 
https://git.webkit.org/?p=WebKit.git;a=commit;h=ce5ef8d5e72fb49cf2a0b72e2a5a8fb551c57f5a
  [3] 
https://github.com/w3c/webdriver/commit/a224627dcea6448367523bf0420de3d547946300

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform