Re: Call for designers for our ports website

[Daniel J. Luke]

Presumably we actually have more complete stats already if we were to aggregate 
the mirror logs for the distfiles + the binary archives.

If we think having more data is valuable, we could add something to port (maybe 
display on selfupdate) asking people to opt-in.

> On Jun 13, 2020, at 11:56 AM, Ken Cunningham 
>  wrote:
> No doubt it caused some tempest.
> 
> I was wrong, homebrew’s published stats say they have 5 million openssl 
> installs this year 
> 
> and our analytics say we have 547 
> 
> 
> And if you think that doesn’t drive everyone’s decision-making extremely 
> powerfully, I would say we are missing the marketing train.
> 
> Here’s their blurb  about justifying it.
> 
> Again, I know MacPorts is not going to change that (no point now). But from a 
> ‘business’ point of view, it was masterful.

-- 
Daniel J. Luke

Re: Call for designers for our ports website

[Joshua Root]

On 2020-6-14 05:38 , Craig Treleaven wrote:
> Also, a quibble on the word “users”.  More than a few of our users
> administer several systems that are all reporting statistics*.  We know
> how many systems are submitting stats but we really don’t know the
> number of individual users that represents.

I guess something like "installations" would be more accurate.

> * In fact, if MacPorts is installed in more than one prefix on a system,
> couldn’t each prefix be submitting statistics independently?

The mpstats LaunchDaemon has a fixed label, so it can only be loaded
once per machine.

- Josh

Re: Call for designers for our ports website

[Craig Treleaven]

> On Jun 13, 2020, at 2:24 PM, Arjun Salyan  wrote:
> 
> Thank you for the mockup and the inputs.
> 
> http://macports.silentfox.tech/port/gnuplot/stats/?days=365 
>  my version of 
> it does not look as good as yours probably due to a larger amount of data.
> 
> Thank you
> 

Thinking about it a little more, I would suggest using an area chart something 
like this:

https://drive.google.com/file/d/17Ch4dwgNNu7Nv5wGrzFguDV7m5v3prwJ/view?usp=sharing
 


Using areas rather than stacked bars emphasizes the trends in the data 
(presenting it as changing uniformly throughout the month).  In this chart, it 
highlights how the installed base of a port migrates to a new version over time 
(with some holdouts).

Conversely, I would suggest the that “Port installations by month” chart should 
be a bar chart rather than an area chart.  

Note that I wasn’t trying to suggest that we need a table of numeric 
percentages alongside the graphical representation.  I think  the table will be 
redundant with a good chart.

Also, a quibble on the word “users”.  More than a few of our users administer 
several systems that are all reporting statistics*.  We know how many systems 
are submitting stats but we really don’t know the number of individual users 
that represents.

Thanks for ‘sweating the details’ on this project!

Craig

* In fact, if MacPorts is installed in more than one prefix on a system, 
couldn’t each prefix be submitting statistics independently?

Re: Call for designers for our ports website

[Arjun Salyan]

Thank you for the mockup and the inputs.

http://macports.silentfox.tech/port/gnuplot/stats/?days=365 my version of
it does not look as good as yours probably due to a larger amount of data.

Thank you

Re: Call for designers for our ports website

[Ken Cunningham]

sorry —we’re 835 openssl installs. wrong drop-down.

K

> On Jun 13, 2020, at 8:56 AM, Ken Cunningham  
> wrote:
> 
> No doubt it caused some tempest.
> 
> I was wrong, homebrew’s published stats say they have 5 million openssl 
> installs this year  >
> 
> and our analytics say we have 547 
>  >
> 
> And if you think that doesn’t drive everyone’s decision-making extremely 
> powerfully, I would say we are missing the marketing train.
> 
> Here’s their blurb  > about justifying it.
> 
> Again, I know MacPorts is not going to change that (no point now). But from a 
> ‘business’ point of view, it was masterful.
> 
> 
> K
> 
> 
> 
> 
> 
>> On Jun 13, 2020, at 8:25 AM, Andrew Janke > > wrote:
>> 
>> Hi y'all,
>> 
>> I was a core Homebrew maintainer at the time they added analytics. Just
>> want to say that Saagar is right; there were a *lot* of Homebrew users
>> who did in fact have a problem with it.
>> 
>> Cheers,
>> Andrwe
>> 
>> 
>> On 6/12/20 9:03 PM, Saagar Jha wrote:
>>> I believe the lack of change there is almost certainly a matter of the 
>>> project’s personal stance rather than “nobody having a problem with it”. In 
>>> fact, after the change was merged in there was a fairly long discussion 
>>> about first disclosing that there were analytics collected at all (which 
>>> did eventually get implemented) and then switching off of Google Analytics 
>>> or making it opt-in, which weren’t. Actually, there were multiple 
>>> discussions but they like the original were generally closed as “WONTFIX” 
>>> and this has been the policy to this day.
>>> 
>>> Personally, I would be fairly disappointed if MacPorts went opt-in as such 
>>> policies suffer from statistical issues in addition to the obvious 
>>> privacy-related ones.
>>> 
>>> Saagar Jha
>>> 
 On Jun 12, 2020, at 16:48, Ken Cunningham >>> > wrote:
 
 Just FYI Homebrew has always been opt-out for stats. Nobody seems to have 
 a problem with that sufficient to make them change that policy.
 
 We'll never know if that is why they seem to have 10 x the users on their 
 stats page.
 
 K
>> 
> 

Re: Call for designers for our ports website

[Ken Cunningham]

No doubt it caused some tempest.

I was wrong, homebrew’s published stats say they have 5 million openssl 
installs this year >

and our analytics say we have 547 
>

And if you think that doesn’t drive everyone’s decision-making extremely 
powerfully, I would say we are missing the marketing train.

Here’s their blurb > about justifying it.

Again, I know MacPorts is not going to change that (no point now). But from a 
‘business’ point of view, it was masterful.


K





> On Jun 13, 2020, at 8:25 AM, Andrew Janke  wrote:
> 
> Hi y'all,
> 
> I was a core Homebrew maintainer at the time they added analytics. Just
> want to say that Saagar is right; there were a *lot* of Homebrew users
> who did in fact have a problem with it.
> 
> Cheers,
> Andrwe
> 
> 
> On 6/12/20 9:03 PM, Saagar Jha wrote:
>> I believe the lack of change there is almost certainly a matter of the 
>> project’s personal stance rather than “nobody having a problem with it”. In 
>> fact, after the change was merged in there was a fairly long discussion 
>> about first disclosing that there were analytics collected at all (which did 
>> eventually get implemented) and then switching off of Google Analytics or 
>> making it opt-in, which weren’t. Actually, there were multiple discussions 
>> but they like the original were generally closed as “WONTFIX” and this has 
>> been the policy to this day.
>> 
>> Personally, I would be fairly disappointed if MacPorts went opt-in as such 
>> policies suffer from statistical issues in addition to the obvious 
>> privacy-related ones.
>> 
>> Saagar Jha
>> 
>>> On Jun 12, 2020, at 16:48, Ken Cunningham  
>>> wrote:
>>> 
>>> Just FYI Homebrew has always been opt-out for stats. Nobody seems to have a 
>>> problem with that sufficient to make them change that policy.
>>> 
>>> We'll never know if that is why they seem to have 10 x the users on their 
>>> stats page.
>>> 
>>> K
> 

Re: Call for designers for our ports website

[Andrew Janke]

Hi y'all,

I was a core Homebrew maintainer at the time they added analytics. Just
want to say that Saagar is right; there were a *lot* of Homebrew users
who did in fact have a problem with it.

Cheers,
Andrwe


On 6/12/20 9:03 PM, Saagar Jha wrote:
> I believe the lack of change there is almost certainly a matter of the 
> project’s personal stance rather than “nobody having a problem with it”. In 
> fact, after the change was merged in there was a fairly long discussion about 
> first disclosing that there were analytics collected at all (which did 
> eventually get implemented) and then switching off of Google Analytics or 
> making it opt-in, which weren’t. Actually, there were multiple discussions 
> but they like the original were generally closed as “WONTFIX” and this has 
> been the policy to this day.
>
> Personally, I would be fairly disappointed if MacPorts went opt-in as such 
> policies suffer from statistical issues in addition to the obvious 
> privacy-related ones.
>
> Saagar Jha
>
>> On Jun 12, 2020, at 16:48, Ken Cunningham  
>> wrote:
>>
>> Just FYI Homebrew has always been opt-out for stats. Nobody seems to have a 
>> problem with that sufficient to make them change that policy.
>>
>> We'll never know if that is why they seem to have 10 x the users on their 
>> stats page.
>>
>> K

Re: Call for designers for our ports website

[Craig Treleaven]

> On Jun 12, 2020, at 10:30 PM, Arjun Salyan  wrote:
> 
> Hi Craig,
> 
> Thank you. You make a valid point regarding the possible distortion due to 
> weekly submissions being bundled to calculate monthly charts. 
> 
> But what we are seeing here is a known issue with the query that calculates 
> this chart.
> https://github.com/macports/macports-webapp/issues/79 
> 
> 
> The current query has a limitation. Let’s say we receive two submissions from 
> a user within one month. One has port version X.1 and the other has upgraded 
> version X.2, then this query counts that user as using both the versions and 
> not just the latest one. This is the cause for the sudden jump in Mar 2020. 
> This problem is only with the "versions vs month" chart and should be fixed 
> soon. Rest all charts, including "installations by month" display accurate 
> information (https://ports.macports.org/port/gnuplot/stats?days=365 
> ).
> 
> Thank you for the percentage suggestion. I am just wondering the right way to 
> display that information graphically.
> 
> I was trying to combine "installations by months" and "versions by month", 
> but it turns out they would be better separate.
> 
> Thank you
> 
> 

The following is a quick mockup of how versions over time might be reported:

https://drive.google.com/file/d/1piEDpd_rq5xnSMAgEsvLO5eu9uu70OpV/view?usp=sharing
 


To display percentages, I don’t think we need the ‘count distinct’.  Suppose 
only a single system is reporting that it uses a particular port and that port 
is updated during the month.  Suppose further, that the first two reports in 
the month from that single system say it is using version 1.0 and the last two 
say it is has version 1.1 installed.  Given the way we collect stats, I think 
it would be accurate to report usage as 50% for each of the versions of the 
port for that month.  Over the course of the month, that was what was reported.

Either that or only use the last report for the month and have the date 
displayed as the last day of each month.

Craig

Re: Call for designers for our ports website

[Joshua Root]

On 2020-6-13 12:18 , Arjun Salyan wrote:
> Hi Clemens,
> 
> On Sat, Jun 13, 2020 at 4:46 AM Clemens Lang  > wrote:
> 
> Please keep in mind that we do value URL stability. Specifically, I've
> recently added redirects so that our old ports.php script will now
> redirect the various links that are out there on upstream websites to
> ports.macports.org , preserving the
> search terms where possible:
>   https://github.com/macports/macports-www/pull/20
> 
>  
> Oh, wow. Thank you for this information.
>  
> 
> It would be appreciated if we could keep the existing URL structure for
> ports.macports.org  and not yet change it
> again, but it seems your
> changes have modified it.
> 
> 
> While setting the URLs initially I wasn't sure that we would be using
> the subdomain (ports.macports.org ), so I had
> appended every URL with an extra term (port or ports). For example, the
> category page is located at '/ports/category/ [1]. I saw this
> as an opportunity to drop that extra "ports/" from the URL and use just
> '/category/' [2] (I just realised that I was using plural
> categories till now). If this is not a good idea and is discouraged, we
> can switch back to the old URLs.

You can change the URL scheme if there are good reasons, but make sure
that old URLs are redirected to their new equivalent.

- Josh

Re: Call for designers for our ports website

[Arjun Salyan]

Hi Craig,

Thank you. You make a valid point regarding the possible distortion due to
weekly submissions being bundled to calculate monthly charts.

But what we are seeing here is a known issue with the query that calculates
this chart.
https://github.com/macports/macports-webapp/issues/79

The current query has a limitation. Let’s say we receive two submissions
from a user within one month. One has port version X.1 and the other has
upgraded version X.2, then this query counts that user as using both the
versions and not just the latest one. This is the cause for the sudden jump
in Mar 2020. This problem is only with the "versions vs month" chart and
should be fixed soon. Rest all charts, including "installations by month"
display accurate information (
https://ports.macports.org/port/gnuplot/stats?days=365).

Thank you for the percentage suggestion. I am just wondering the right way
to display that information graphically.

I was trying to combine "installations by months" and "versions by month",
but it turns out they would be better separate.

Thank you

Re: Call for designers for our ports website

[Arjun Salyan]

Hi Clemens,

On Sat, Jun 13, 2020 at 4:46 AM Clemens Lang  wrote:

> Please keep in mind that we do value URL stability. Specifically, I've
> recently added redirects so that our old ports.php script will now
> redirect the various links that are out there on upstream websites to
> ports.macports.org, preserving the search terms where possible:
>   https://github.com/macports/macports-www/pull/20


Oh, wow. Thank you for this information.


> It would be appreciated if we could keep the existing URL structure for
> ports.macports.org and not yet change it again, but it seems your
> changes have modified it.
>

While setting the URLs initially I wasn't sure that we would be using the
subdomain (ports.macports.org), so I had appended every URL with an extra
term (port or ports). For example, the category page is located at
'/ports/category/ [1]. I saw this as an opportunity to drop that
extra "ports/" from the URL and use just '/category/' [2] (I just
realised that I was using plural categories till now). If this is not a
good idea and is discouraged, we can switch back to the old URLs.

'/search' [3] is a new page with the addition of multiple new filters
(compared to just 2 now). This asks for a change to the [case 'library]
redirect. If we do not want that too, I can try to accommodate it.

Thank you

[1] https://ports.macports.org/ports/category/perl/
[2] http://macports.silentfox.tech/category/perl
[3] http://macports.silentfox.tech/search/

Re: Call for designers for our ports website

[Ken Cunningham]

> On Jun 12, 2020, at 6:03 PM, Saagar Jha  wrote:
> 
> I believe the lack of change there is almost certainly a matter of the 
> project’s personal stance rather than “nobody having a problem with it”.

"sufficient to make them change that policy"

> Personally, I would be fairly disappointed if MacPorts went opt-in 

I presume you meant to say opt-out



Off topic anyway, just went the new website model where I was struck by initial 
page highlighting the tiny number of total opt-in stats contributors, and what 
that means, if anything, regarding stats.

I think homebrew had 1,000,000 installs of — what was it — openssl or something.

Anyway, we won’t change that policy, and I don’t want a flame war about this. 
Just that our stats might not best be highlighted :>

K

Re: Call for designers for our ports website

[Saagar Jha]

I believe the lack of change there is almost certainly a matter of the 
project’s personal stance rather than “nobody having a problem with it”. In 
fact, after the change was merged in there was a fairly long discussion about 
first disclosing that there were analytics collected at all (which did 
eventually get implemented) and then switching off of Google Analytics or 
making it opt-in, which weren’t. Actually, there were multiple discussions but 
they like the original were generally closed as “WONTFIX” and this has been the 
policy to this day.

Personally, I would be fairly disappointed if MacPorts went opt-in as such 
policies suffer from statistical issues in addition to the obvious 
privacy-related ones.

Saagar Jha

> On Jun 12, 2020, at 16:48, Ken Cunningham  
> wrote:
> 
> Just FYI Homebrew has always been opt-out for stats. Nobody seems to have a 
> problem with that sufficient to make them change that policy.
> 
> We'll never know if that is why they seem to have 10 x the users on their 
> stats page.
> 
> K

Re: Call for designers for our ports website

[Ken Cunningham]

Just FYI Homebrew has always been opt-out for stats. Nobody seems to have a 
problem with that sufficient to make them change that policy.

We'll never know if that is why they seem to have 10 x the users on their stats 
page.

K

Re: Call for designers for our ports website

[Clemens Lang]

Hi Mojca, Arjun,

On Fri, Jun 12, 2020 at 07:24:49PM +0200, Mojca Miklavec wrote:
> - http://macports.silentfox.tech/search/?installed_file==root=on

Please keep in mind that we do value URL stability. Specifically, I've
recently added redirects so that our old ports.php script will now
redirect the various links that are out there on upstream websites to
ports.macports.org, preserving the search terms where possible:
  https://github.com/macports/macports-www/pull/20

It would be appreciated if we could keep the existing URL structure for
ports.macports.org and not yet change it again, but it seems your
changes have modified it.

-- 
Clemens