On 8/20/2015 12:56 PM, Nathan Froyd wrote:
On Wed, Aug 19, 2015 at 7:11 PM, Benjamin Smedberg
<benja...@smedbergs.us <mailto:benja...@smedbergs.us>> wrote:
On 8/19/15 11:35 AM, Nathan Froyd wrote:
These statistics are reported through Telemetry.
Have the in-tree docs been updated to document this? I don't
recall being asked to review the final data collection proposal
for this mechanism.
They have not. I didn't realize there were documents to update or
that adding new histograms required separate review.
Yes. The Firefox data stewards (myself and peers Vladan and Ally) review
all changes to Firefox data collection. See the policy doc at
https://wiki.mozilla.org/Firefox/Data_Collection
This really does need some docs. I don't particularly care whether this
ends up in the telemetry document tree at
http://mxr.mozilla.org/mozilla-central/source/toolkit/components/telemetry/docs/
or add a new area for DOM documentation. But we absolutely need to
describe the system and the histograms it produces.
I do think we should end up documenting the use counters in the
auto-generated docs, but that can be a followup bug. Bug 1194287 is
currently filed to get Histograms.json itself included in the
auto-generated docs.
In particular:
* Will this only collect to the opt-in (prerelease) population, or
does this also affect the release population?
Since these are just histograms within Telemetry, I assume this
affects prerelease and release populations. (That was the intent,
anyway; it's possible I didn't flip the right switches for this to
happen.)
Will that solve all of the use-cases? I'm perfectly fine with being able
to specify that use counters are opt-out (FHR), but opt-in/telemetry is
the default in Histograms.json. In any case, this should be tested as
part of QA verification for the bug.
* This involves collecting a numerator and denominator as separate
histograms, correct?
I don't know what numerator and denominator refer to here, maybe # of
pages using feature and # of total pages, respectively?
Isn't that the use-case that this was trying to solve?
For each defined use counter, there are two separate boolean
histograms. One describes the use of that thing for individual
documents, and the other describes the use of that thing for
"top-level" pages: basically what we think of as a "web page".
Interesting, this is not how I imagined this would work. I figured we'd
have a couple of global counters:
* counter: number of documents
* counter: number of toplevel documents
And then for each feature we'd have a couple of counters:
* counter: per-document feature usage
* counter: per-toplevel-document feature usage
In terms of the total data collected, it appears to be equivalent. But
one reason for doing it this is that it can significantly reduce the
telemetry payload size for measuring infrequently-used features: we
don't include unused histograms at all in the telemetry payload.
* Is there reporting in place for this mechanism already? If not,
who is going to write that?
Since these are just histograms, reporting should not be a problem.
There have been problems on the server-side when the first part of
this landed back in May (bug 1168409), so it's possible that there are
some changes needed on the server side for histogram validation, etc.
Has somebody verified that the current histogram displays on
telemetry.mozilla.org can answer the questions you care about?
* How are we going to do data-collection reviews for additions,
especially if the collection is added through WebIDL/UseCounters?
I do not know. hg push hooks?
Let's start out by just stating the obvious, that you cannot add a new
use counter without data collection review. If that stops working, we
can think about the more complicated hook process.
* Is there a mechanism for auto-expiring data collection like
there is for histograms?
The histograms are currently set to never expire. It is probably
worth rethinking that.
Yes, I think this is a requirement. Very few of the use-cases for use
counters are candidates for permanent data collection, and so we should
explicitly be able to specify the expiration.
--BDS
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
