Re: [webkit-dev] New iOS versions sending bogus User-Agent build data

[Michael Catanzaro]

On Thu, Apr 26, 2018 at 12:48 PM, Colin Bendell | +1.613.914.3387 
 wrote:

Can you give me an example where UA parsing is punishing users of
alternative user agents? Is this a theoretical problem, or a
widespread problem? I'm not asking to be divisive, but because I know
for a fact that UA parsing is improving the user experience. I can
give you dozens of examples where we must resort to UA parsing for the
betterment of the user (for all flavors of UAs).


I would say it's the most serious web compatibility problem that exists 
today. Our users complain and blame us when important websites are 
broken in WebKit because of it. I have personally wasted days [1] of 
[2] development [3] effort [4] playing with WebKit's user agent quirks 
to get important websites to work properly. You can look at the history 
of our quirks list for non-Safari ports [5] to get an idea of the 
trouble we've had to deal with in recent years. That doesn't count 
cases like [6] where I suspect user agent trouble, but just don't care 
to investigate.


As a web engine developer, it's hard to understate how frustrating this 
is, especially in cases like [7] where we had tons of users complaining 
and the quirks were particularly difficult to write. It sometimes feels 
like website developers are our enemy, just out to break our web 
engine. Sometimes that's even true, e.g. when websites intentionally 
decide to block access to unrecognized browsers, or scare our users 
with claims that a browser is "unsupported" even though it works fine 
with a UA quirk.


This is not a healthy situation for the web.

If everyone was careful and responsible with how they use the user 
agent, it wouldn't be a problem, but at this point it's long been 
spoiled for everyone. I'm sorry that this fallout affects developers 
like you who are just trying to work around a bug. :(


On Thu, Apr 26, 2018 at 12:48 PM, Colin Bendell | +1.613.914.3387 
 wrote:

Again I ask, is there room for compromise where we can expose the
version details in the UA (or some alternative) so that we ensure a
consistent and optimized user experience?


I don't know. I wish there was, but I don't think so. If you have any 
suggestions, that'd be great, but I think it's going to be extremely 
difficult or impossible to solve this problem in a way that makes 
everyone happy.


I don't think I'll be happy until major browsers send the same, 
standardized user agent as other major browsers. (Or send 
fully-randomized user agents, but that's probably an impossible dream.) 
Freezing just the version numbers is not good enough, but it's a step 
in the right direction, and I really appreciate it.


[1] https://trac.webkit.org/changeset/206519/webkit
[2] https://trac.webkit.org/changeset/216139/webkit
[3] https://trac.webkit.org/changeset/216343/webkit
[4] https://trac.webkit.org/changeset/217203/webkit
[5] 
https://trac.webkit.org/log/webkit/trunk/Source/WebCore/platform/UserAgentQuirks.cpp

[6] https://bugs.webkit.org/show_bug.cgi?id=180995
[7] https://bugs.webkit.org/show_bug.cgi?id=171770#c4

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] New iOS versions sending bogus User-Agent build data

[Ali Juma]

It's worth noting that https://bugs.webkit.org/show_bug.cgi?id=182629 added
back the OS version to the UA string, at least on trunk (the reasons given
there, in https://bugs.webkit.org/show_bug.cgi?id=182629#c6, sound exactly
like those mentioned by Colin in this thread).

On Thu, Apr 26, 2018 at 1:49 PM Colin Bendell | +1.613.914.3387 <
co...@bendell.ca> wrote:

> On 26 April 2018 at 13:18, Konstantin Tokarev  wrote:
> > 26.04.2018, 20:09, "Colin Bendell | +1.613.914.3387 <(613)%20914-3387>"
> :
> >> On 26 April 2018 at 12:23, Konstantin Tokarev 
> wrote:
> >>>  Not to mention those evil people who reject page loading for user
> agents
> >>>  they don't (want to) support
> >>
> >> How does locking the UA solve the misbehaving parsers in the wild?
> >
> > There will be less information in UA and therefore less reasons for
> developers
> > to do this, instead of using direct feature testing
> >
> >> They will still misbehave and break the user experience. However,
> >> those of us that are trying to optimize the user experience by working
> >> around bugs for specific versions are now handicapped and punished.
> >
> > While parsing UA string handicaps and punishes users of alternative user
> agents.
>
> Can you give me an example where UA parsing is punishing users of
> alternative user agents? Is this a theoretical problem, or a
> widespread problem? I'm not asking to be divisive, but because I know
> for a fact that UA parsing is improving the user experience. I can
> give you dozens of examples where we must resort to UA parsing for the
> betterment of the user (for all flavors of UAs).
>
> Do you have any alternatives other than "more javascript"?
>
> Again I ask, is there room for compromise where we can expose the
> version details in the UA (or some alternative) so that we ensure a
> consistent and optimized user experience?
>
> /colin
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] New iOS versions sending bogus User-Agent build data

[Colin Bendell | +1.613.914.3387]

On 26 April 2018 at 13:18, Konstantin Tokarev  wrote:
> 26.04.2018, 20:09, "Colin Bendell | +1.613.914.3387" :
>> On 26 April 2018 at 12:23, Konstantin Tokarev  wrote:
>>>  Not to mention those evil people who reject page loading for user agents
>>>  they don't (want to) support
>>
>> How does locking the UA solve the misbehaving parsers in the wild?
>
> There will be less information in UA and therefore less reasons for developers
> to do this, instead of using direct feature testing
>
>> They will still misbehave and break the user experience. However,
>> those of us that are trying to optimize the user experience by working
>> around bugs for specific versions are now handicapped and punished.
>
> While parsing UA string handicaps and punishes users of alternative user 
> agents.

Can you give me an example where UA parsing is punishing users of
alternative user agents? Is this a theoretical problem, or a
widespread problem? I'm not asking to be divisive, but because I know
for a fact that UA parsing is improving the user experience. I can
give you dozens of examples where we must resort to UA parsing for the
betterment of the user (for all flavors of UAs).

Do you have any alternatives other than "more javascript"?

Again I ask, is there room for compromise where we can expose the
version details in the UA (or some alternative) so that we ensure a
consistent and optimized user experience?

/colin
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Style guidelines for braced initialization

[Filip Pizlo]

> On Apr 26, 2018, at 10:35 AM, ross.kirsl...@sony.com wrote:
> 
> Hi everybody,
>  
> A simple question regarding https://webkit.org/code-style-guidelines/ 
> :
>  
> I've currently got a clang-format patch up (https://reviews.llvm.org/D46024) 
>  to support our space-before-brace style 
> for object initialization (i.e. `Foo foo { bar };` and not `Foo foo{ bar };`).
> Although we’re enforcing this with check-webkit-style, the guidelines page 
> currently makes no mention of braced initialization in particular. Should we 
> add a clause for this?

Yeah, sounds like we should!

-Filip


>  
> Thanks!
> Ross
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org 
> https://lists.webkit.org/mailman/listinfo/webkit-dev 
> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

[webkit-dev] Style guidelines for braced initialization

[Ross.Kirsling]

Hi everybody,

A simple question regarding https://webkit.org/code-style-guidelines/:

I've currently got a clang-format patch up (https://reviews.llvm.org/D46024) to 
support our space-before-brace style for object initialization (i.e. `Foo foo { 
bar };` and not `Foo foo{ bar };`).
Although we’re enforcing this with check-webkit-style, the guidelines page 
currently makes no mention of braced initialization in particular. Should we 
add a clause for this?

Thanks!
Ross
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] New iOS versions sending bogus User-Agent build data

[Konstantin Tokarev]

26.04.2018, 20:09, "Colin Bendell | +1.613.914.3387" :
> On 26 April 2018 at 12:23, Konstantin Tokarev  wrote:
>>  Not to mention those evil people who reject page loading for user agents
>>  they don't (want to) support
>
> How does locking the UA solve the misbehaving parsers in the wild?

There will be less information in UA and therefore less reasons for developers
to do this, instead of using direct feature testing

> They will still misbehave and break the user experience. However,
> those of us that are trying to optimize the user experience by working
> around bugs for specific versions are now handicapped and punished.

While parsing UA string handicaps and punishes users of alternative user agents.

-- 
Regards,
Konstantin

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] New iOS versions sending bogus User-Agent build data

[Colin Bendell | +1.613.914.3387]

On 26 April 2018 at 12:23, Konstantin Tokarev  wrote:
> 26.04.2018, 19:16, "Michael Catanzaro" :
>> On Thu, Apr 26, 2018 at 11:13 AM, Michael Catanzaro  
>> wrote:
>>>  By fixing the WebKit bug, of course. And in the meantime you can work
>>>  around it on the server side by not using img src=mp4, right?

Today, we can work around it by ignoring the Accept field. However,
tomorrow once the bug is fixed we will be in the paradox of not
knowing which versions of Safari are telling the truth.


>> Consider the other perspective on this problem. If other servers look
>> at the WebKit version in the UA to determine if WebKit supports img
>> src=mp4, other WebKit ports that don't support this are going to be out
>> of luck and get broken pages. I know that's not what you're doing --
>> you're looking at iOS version instead, and only doing it to work around
>> a specific bug, which is much better -- but the problem of websites
>> sending bad content based on bad user agent parsing is so severe that
>> we don't have many good options, here. :/
>
> Not to mention those evil people who reject page loading for user agents
> they don't (want to) support

How does locking the UA solve the misbehaving parsers in the wild?
They will still misbehave and break the user experience. However,
those of us that are trying to optimize the user experience by working
around bugs for specific versions are now handicapped and punished.

The reality is that bugs exist. There is a give and take here where
the content negotiation is sometimes smarter on the client, and
sometimes smarter on the server. In the case of media, the server is
usually smarter and must be because of the long tail of adoption.

Ideally everyone on the planet will adopt to ios V.Latest overnight,
but sadly this will not be the case. Servers must account for the long
tail of users on many different versions. Right now we are forced to
decide to apply a solution that works only for v.latest, or apply a
solution that works for v.lowest-common-denominator. In both scenarios
we compromise the user experience. Ideally we wouldn't have to
sacrifice some users and would be able to account for bugs in the
wild.

As I said before, if this detail is available in javascript, then why
hide it on the wire? Perhaps it's time for a User-Agent2 with precise
feature declarations? Even still a version number is needed because,
well, bugs.

While I understand the fingerprinting concern, I'm not particularly
convinced that UA obfuscation is the best solution for users and the
user experience. Without opening up the religious debate of
fingerprinting with the UA, how can provide enough detail (version
numbers) so that service providers can work toward the greater good of
improving the user experience?

Is there room for compromise?

/colin
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] New iOS versions sending bogus User-Agent build data

[Konstantin Tokarev]

26.04.2018, 19:16, "Michael Catanzaro" :
> On Thu, Apr 26, 2018 at 11:13 AM, Michael Catanzaro
>  wrote:
>>  By fixing the WebKit bug, of course. And in the meantime you can work
>>  around it on the server side by not using img src=mp4, right?
>
> Consider the other perspective on this problem. If other servers look
> at the WebKit version in the UA to determine if WebKit supports img
> src=mp4, other WebKit ports that don't support this are going to be out
> of luck and get broken pages. I know that's not what you're doing --
> you're looking at iOS version instead, and only doing it to work around
> a specific bug, which is much better -- but the problem of websites
> sending bad content based on bad user agent parsing is so severe that
> we don't have many good options, here. :/

Not to mention those evil people who reject page loading for user agents
they don't (want to) support

-- 
Regards,
Konstantin

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] New iOS versions sending bogus User-Agent build data

[Michael Catanzaro]

On Thu, Apr 26, 2018 at 11:13 AM, Michael Catanzaro 
 wrote:
By fixing the WebKit bug, of course. And in the meantime you can work 
around it on the server side by not using img src=mp4, right?


Consider the other perspective on this problem. If other servers look 
at the WebKit version in the UA to determine if WebKit supports img 
src=mp4, other WebKit ports that don't support this are going to be out 
of luck and get broken pages. I know that's not what you're doing -- 
you're looking at iOS version instead, and only doing it to work around 
a specific bug, which is much better -- but the problem of websites 
sending bad content based on bad user agent parsing is so severe that 
we don't have many good options, here. :/


Michael

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] New iOS versions sending bogus User-Agent build data

[Michael Catanzaro]

On Thu, Apr 26, 2018 at 10:57 AM, Colin Bendell | +1.613.914.3387 
 wrote:

How do we navigate this?


By fixing the WebKit bug, of course. And in the meantime you can work 
around it on the server side by not using , right?


Michael

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] New iOS versions sending bogus User-Agent build data

[Colin Bendell | +1.613.914.3387]

I'm running into this issue right now with the locked UA. Here are the
challenges:
1. ios 11.3 safari bug broke  despite advertising
`accept: video/*` in the request. (see bug 185029). I now need to use
UA detection to patch versions of a response based on existing bugs.
In short, we as an industry, depend on UA parsing to work around bugs.
This happened last year when the jp2k support increased to 10bit depth
and the decoder was inflating the 8bit colour pallet to 10bit. We
needed to use UA parsing to determine if it was safe to send jp2k
images to a device. I have a laundry list of other examples where the
server might actually know more than the client to prevent a) a broken
user experience and b) prevent cellular data waste.

2. Javascript based feature detection is inefficient and slows down
safari' preloader. The preloader is one of the best performance
enhancements to come to the modern browser. Using javscript for
feature detection is a step backward for the preloader. Webkit does
not provide enough declarations of its feature set in HTTP headers so
we resort to UA device characteristics. If the features are available
with javascript inquiry, then how is that any different than exposing
these features at request time?

In short, we are stuck. blocking the UA like this will degrading the
user experience because it will force us to either not take advantage
of cell-data/cpu/battery improvements (like sending mp4s instead of
animated gifs) or it will force us to use javascript and likewise
degrade the user experience with slower web experiences.

How do we navigate this?

/colin

On 26 April 2018 at 11:22, Michael Catanzaro  wrote:
> On Thu, Apr 26, 2018 at 7:15 AM, Ricky Young  wrote:
>>
>> I guess that soon it won't be possible to determine iOS version
>> distribution using User-Agent sniffing. Right? Is there an alternative for
>> the browser to tell the iOS version?
>
>
> Hi,
>
> The very point of no longer sending the verison in the user agent is to make
> it difficult for websites to determine the operating system version.
>
> We already have to deal with enough serious web compatibility problems
> caused by the rest of the information in the user agent. I'm pleased that
> the version number is now stable -- that's a huge improvement -- but I won't
> be happy until Safari and other major browsers start sending
> fully-randomized UAs, or else a stub or blank UA ("Mozilla/5.0").
>
> Michael
>
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] New iOS versions sending bogus User-Agent build data

[Ricky Young]

Actually it does, a lot!

I guess that soon it won't be possible to determine iOS version distribution 
using User-Agent sniffing. Right? Is there an alternative for the browser to 
tell the iOS version? 
 

Sent: Thursday, April 26, 2018 at 1:54 PM
From: "Anne van Kesteren" 
To: "Ricky Young" 
Cc: "Konstantin Tokarev" , "webkit-dev@lists.webkit.org" 

Subject: Re: [webkit-dev] New iOS versions sending bogus User-Agent build data
On Thu, Apr 26, 2018 at 12:30 PM, Ricky Young  wrote:
> I read the report and still find it hard to understand, if "User Agent 
> sniffing is a terrible way to determine whether a browser supports certain 
> features", what is the correct way of doing it?

https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Feature_detection

Hope that helps,


--
https://annevankesteren.nl/[https://annevankesteren.nl/]
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] New iOS versions sending bogus User-Agent build data

[Anne van Kesteren]

On Thu, Apr 26, 2018 at 12:30 PM, Ricky Young  wrote:
> I read the report and still find it hard to understand, if "User Agent 
> sniffing is a terrible way to determine whether a browser supports certain 
> features", what is the correct way of doing it?

https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Feature_detection

Hope that helps,


-- 
https://annevankesteren.nl/
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] New iOS versions sending bogus User-Agent build data

[Ricky Young]

Hi Konstantin,

Thanks for your prompt response :-)

I didn't notice this bug and the respective patches.

I read the report and still find it hard to understand, if "User Agent sniffing 
is a terrible way to determine whether a browser supports certain features", 
what is the correct way of doing it?
 

Sent: Thursday, April 26, 2018 at 1:16 PM
From: "Konstantin Tokarev" 
To: "Ricky Young" , "webkit-dev@lists.webkit.org" 

Subject: Re: [webkit-dev] New iOS versions sending bogus User-Agent build data

26.04.2018, 13:08, "Ricky Young" :
> Until iOS 11.3, User-Agent build data contained the build that corresponds to 
> the device (e.g. Mobile/15D100 if the requesting device was running iOS 
> 11.2.6).
>
> Since iOS 11.3 was released I noticed that the User-Agent that is reported to 
> the brower is reporting a bogus build, i.e Mobile/15E148, regardless to the 
> iOS version that the requesting device is running. I would expect that a 
> device that runs 11.3 will reprot a different User-Agent than a device that 
> runs 11.3.1 and certianly than a device that runs 11.4 (Beta 1 & 2).
>
> I just wonder if my observation is an expected behavior, and if so, what is 
> the intention behind it. If this behavior is not intended and expected, I 
> would love to research it and propose a patch (would be my first Open Source 
> contribution).

Hi,

User-Agent string is frozen since https://bugs.webkit.org/show_bug.cgi?id=180365

-- 
Regards,
Konstantin
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] New iOS versions sending bogus User-Agent build data

[Konstantin Tokarev]

26.04.2018, 13:08, "Ricky Young" :
> Until iOS 11.3, User-Agent build data contained the build that corresponds to 
> the device (e.g. Mobile/15D100 if the requesting device was running iOS 
> 11.2.6).
>
> Since iOS 11.3 was released I noticed that the User-Agent that is reported to 
> the brower is reporting a bogus build, i.e Mobile/15E148, regardless to the 
> iOS version that the requesting device is running. I would expect that a 
> device that runs 11.3 will reprot a different User-Agent than a device that 
> runs 11.3.1 and certianly than a device that runs 11.4 (Beta 1 & 2).
>
> I just wonder if my observation is an expected behavior, and if so, what is 
> the intention behind it. If this behavior is not intended and expected, I 
> would love to research it and propose a patch (would be my first Open Source 
> contribution).

Hi,

User-Agent string is frozen since https://bugs.webkit.org/show_bug.cgi?id=180365

-- 
Regards,
Konstantin
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

[webkit-dev] New iOS versions sending bogus User-Agent build data

[Ricky Young]

Until iOS 11.3, User-Agent build data contained the build that corresponds to the device (e.g. Mobile/15D100 if the requesting device was running iOS 11.2.6).
 
Since iOS 11.3 was released I noticed that the User-Agent that is reported to the brower is reporting a bogus build, i.e Mobile/15E148, regardless to the iOS version that the requesting device is running. I would expect that a device that runs 11.3 will reprot a different User-Agent than a device that runs 11.3.1 and certianly than a device that runs 11.4 (Beta 1 & 2).
 
I just wonder if my observation is an expected behavior, and if so, what is the intention behind it. If this behavior is not intended and expected, I would love to research it and propose a patch (would be my first Open Source contribution).

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev