> On 17. Aug 2018, at 15:38, Jesse Glick <[email protected]> wrote:
>
> On Fri, Aug 17, 2018 at 7:57 AM Daniel Beck <[email protected]> wrote:
>>> Do you plan to implement this data collection in the core directly or in a
>>> plugin?
>>
>> Core (or core module).
>
> Fine for the initial example of collecting system properties, but in
> general this sort of collection will need to make use of plugin code.
> For example, Pipeline developers might like to know things such as how
> many `FlowNode`s are created per build on overage. This is currently
> sent to `support-core` by `workflow-cps` but we would like to gather
> this sort of thing anonymously. (Note that `support-core` already
> includes an anonymization system.) I see no straightforward way to
> gather such a metric without explicit support from one of the Pipeline
> plugins. That implies an _API_ defined in core for transmitting data
> but _implementations_ possibly spread across core and various plugins.
That makes sense. Yes, that should definitely be possible.
> And this just highlights a fundamental question: how exactly do you
> plan to collect this kind of information during a “specific time
> frame”? You can add the collector logic to a core weekly release, or a
> plugin release, and enable the server side for that statistic at the
> same time; and then later turn off the server and remove the collector
> logic from the next component release.
That's one option, coupled with collectors defining the start and end dates
during which they collect and submit data, otherwise being disabled
client-side. Combined with shutting down collection on the server side for
specific collectors should take care of this restriction.
> But users update software on
> whatever schedule they like, so lots of systems will not have the
> right collector logic in place during the window in which it is
> allowed to send data. Will that not bias the results toward
> installations that update frequently? Do you care?
Usage stats point to fairly good update behavior: More than a third of
installations are on a release no older than eight weeks. This is true for both
weeklies and LTS. That seems like a large enough user base to get telemetry
from, while still being in a fairly limited time window.
Still, without extending the collection window to many months, we'd only really
collect information on those 35-50% of users. It's also difficult to say right
now how we'd handle LTS releases -- would collectors be eligible for backports?
(I say yes, as optional extension points written such that they mostly silently
fail.)
While my original intention was to limit the collection time span as much as
possible while retaining useful data, it seems we might need to adapt over time
to the upgrade behavior of our users. That said, I'd prefer to start with
rather small windows by default, and extend them as needed based on our
experience.
(Realistically, how much do we care about the experience of users who haven't
upgraded in several months? We frequently reject their bug reports, they get no
security updates, we only have a single LTS line…)
>>> we need to review conflicts with JEP-308 (Telemetry API for evergreen)
>>
>> The client side of that is outside Jenkins itself, while mine would be
>> inside. This alone means it's practically independent.
>
> Well, in terms of current implementation, but we would like to align
> these things if at all possible so that developers of a given
> statistic can collect it both from Evergreen systems or from (more or
> less up to date) traditional installations as a single data stream.
> After all, what you are describing is very much one of the key goals
> of Evergreen (if not JEP-308 specifically):
To be useful, we need a large enough user base (one of the concerns you brought
up above), and therefore cannot tie ourselves to Evergreen here, at least not
in the short term.
When I first brought up this idea with Tyler a few months ago, IIRC the
response (from him and/or Baptiste) was there's nothing we can realistically
share. That's why I went ahead with an independent proposal.
--
You received this message because you are subscribed to the Google Groups
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-dev/D0033E93-A3C8-4BEA-9803-20F1B8BF85D8%40beckweb.net.
For more options, visit https://groups.google.com/d/optout.
