On 8/13/2014 12:38 PM, Nils Ohlmeier wrote:
As development on OpenH264 seems to be pretty active I assume that we
will deploy/push updates at some point to our download server.
* Do we plan to deploy updates regularly, or only on a per need base?
As needed.
* How will we test such updates (as we don't have any automated tests
(I don't consider the fake plugin to give us any decent test
coverage here) I fear we are stuck with lots of manual testing here)?
Yes, we have no automated tests for end-to-end, so these updates should
be given careful manual QA from the desktop team, and should be approved
by release-drivers before they are deployed.
Even without deploying any updates of the plugin I'm concerned how do
we learn if something is broken for end users if anything is wrong on
the download server end. For example the server becomes unreachable,
or the XML can no longer be parsed, or...
My assumption here is no matter what goes wrong here the result will
be that a new user would end up with no H264 support on his browser.
I'll leave it to other people to judge how bad that would be.
I think it is pretty hard to predict how likely it is that we will run
into any kind of errors which results in plugin downloads not working.
I hope someone is monitoring the actual server.
We do have ADI counts from the update manifest server, but that is not
especially useful for the question you are trying to answer. We don't
have any data from the download server itself, which is a CDN.
* But do we have for example telemetry data for success or failure
rate of the plugin (or how many times a browser re-tried before it
successfully downloaded the plugin)?
* If we don't have any or not enough visibility into download success
rates, is it worth investing into more monitoring or do we consider
the risk so low or the result not important enough to invest into
more monitoring?
The best thing, and the thing that we *ought* to be collecting in both
telemetry and FHR, is the currently installed version of the OpenH264
addon and whether the addon is enabled. Then we will at least be able to
see whether OpenH264 is the version we expect to see.
Because we're not using a normal addon type, though, I'm not 100% sure
whether this actually works. Georg, do you know? If not, can you please
file a bug to add this measurement to FHR/telemetry and add it to the
Firefox backlog?
I'm sure that there is no reporting or alerting for this field, though.
So to the extent that we are using this to ensure product quality, Nils,
you should probably file a bug to get some automatic reporting/alerting.
That should be filed in the Firefox Health Report product/Reporting
component and then we'll have to ask the metrics team to implement it.
--BDS
_______________________________________________
dev-media mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-media
