Bug#632438: [Popcon-developers] Bug#681721: #632438:

2020-11-13 Thread Bill Allombert 
On Sun, Nov 08, 2020 at 10:10:10PM +0900, Kentaro Hayashi wrote:
> On Wed, 23 Sep 2020 16:33:54 +0200 Bill Allombert  wrote:
> snip
> > Not always. Sometimes that points to packages that are missing in Debian
> > and should be packaged.
> 
> I've missed this point of view.
> 
> > But as soon as a single system report a package name, it appears in the
> > statistics. So unless everyone set up popcon to discard it, there is the
> > same amount of noise with less accurate statistics.
> 
> As you mentioned, it may be meaningless unless everyone set up popcon
> to discard it.
> So, to make statistics accurately, it may be a bad idea to filter them.

Thanks for getting back to us!

So how about the proposal to use a dpkg field to identify packages than
need filtering ?

Cheers,
-- 
Bill. 

Imagine a large red swirl here.

Bug#632438: [Popcon-developers] Bug#681721: #632438:

2020-11-08 Thread Kentaro Hayashi 
On Wed, 23 Sep 2020 16:33:54 +0200 Bill Allombert  wrote:
snip
> Not always. Sometimes that points to packages that are missing in Debian
> and should be packaged.

I've missed this point of view.

> But as soon as a single system report a package name, it appears in the
> statistics. So unless everyone set up popcon to discard it, there is the
> same amount of noise with less accurate statistics.

As you mentioned, it may be meaningless unless everyone set up popcon
to discard it.
So, to make statistics accurately, it may be a bad idea to filter them.


Regards,

Bug#632438: [Popcon-developers] Bug#681721: #632438:

2020-09-23 Thread Bill Allombert 
On Wed, Sep 23, 2020 at 09:23:13PM +0900, Kentaro Hayashi wrote:
> On Sun, 20 Sep 2020 13:41:53 +0200 Bill Allombert  wrote:
> > On Sun, Sep 20, 2020 at 08:08:35PM +0900, Kentaro Hayashi wrote:
> > > I want to exclude installed packages from 3rd party vendor
> > > such as /etc/apt/sources.list.d/*.list. (some may be proprietary software)
> > 
> > OK, but what is your purpose in excluding them from popcon ?
> 
> Personally, I think that popcon data from 3rd party packages
> is just a noise because there is nothing to do with Debian.

Not always. Sometimes that points to packages that are missing in Debian
and should be packaged.

> Therefore, I think that it seems better to exclude.
> 
> But this is my personal opinion, so I don't mean to force others
> to do so. I'm happy if I have an option to exclude them.

But as soon as a single system report a package name, it appears in the
statistics. So unless everyone set up popcon to discard it, there is the
same amount of noise with less accurate statistics.

One other option would be for popularity-contest to detect third-party 
packages, but
this is difficult to do client-side. However this is done server-side,
see 

Cheers,
-- 
Bill. 

Imagine a large red swirl here.

Bug#632438: [Popcon-developers] Bug#681721: #632438:

2020-09-23 Thread Petter Reinholdtsen 
[Kentaro Hayashi]
> Personally, I think that popcon data from 3rd party packages
> is just a noise because there is nothing to do with Debian.
> Therefore, I think that it seems better to exclude.

I've used these lists in the past to find packages we should have a look
at for inclusion in Debian, so I do not believe it is noise.

--
Happy hacking
Petter Reinholdtsen

Bug#632438: [Popcon-developers] Bug#681721: #632438:

2020-09-23 Thread Kentaro Hayashi 
On Sun, 20 Sep 2020 13:41:53 +0200 Bill Allombert  wrote:
> On Sun, Sep 20, 2020 at 08:08:35PM +0900, Kentaro Hayashi wrote:
> > I want to exclude installed packages from 3rd party vendor
> > such as /etc/apt/sources.list.d/*.list. (some may be proprietary software)
> 
> OK, but what is your purpose in excluding them from popcon ?

Personally, I think that popcon data from 3rd party packages
is just a noise because there is nothing to do with Debian.
Therefore, I think that it seems better to exclude.

But this is my personal opinion, so I don't mean to force others
to do so. I'm happy if I have an option to exclude them.

Bug#632438: [Popcon-developers] Bug#681721: #632438: popularity-contest: a way to exclude certain packages

2020-09-20 Thread Bill Allombert 
On Sun, Sep 20, 2020 at 10:56:02PM +0800, Paul Wise wrote:
> On Sun, 2020-09-20 at 11:04 +0200, Bill Allombert wrote:
> 
> > Instead I would suggest to add a new dpkg control field 'X-Popcon:
> > private' and have popularity-contest skip packages having this field.
> 
> This isn't going to be useful for users who want to exclude packages
> from repos that they do not control

But again why would they want that ? The only thing popcon report
is the package names  (which would be public anyway) and some timing
data. If they do not trust popcon anonymization, then it is safer to
disable popcon entirely.

It is a given users can mess with popcon reports in any way then want.
However randomly hiding packages from popcon report is not something
that should be sanctionned by the popularity-contest package.

> or packages built by mk-build-deps
> or other tools that do not allow adding extra dpkg control fields.

I suppose mk-build-deps and other tools could then be updated to
support this feature. This is not really an objection. In fact it
would make this much easier.

Cheers,
-- 
Bill. 

Imagine a large red swirl here.

Bug#632438: [Popcon-developers] Bug#681721: #632438: popularity-contest: a way to exclude certain packages

2020-09-20 Thread Paul Wise 
On Sun, 2020-09-20 at 11:04 +0200, Bill Allombert wrote:

> Could you explain why you want to exclude some package ?

Personally I'm excluding packages created by mk-build-deps as well as
the metapackages I create for my own systems, using a simple `grep -v`
in the popcon submission cron job.

The patch posted by Kentaro is not sufficient for my use-case, which
relies on excluding packages via regex instead of full package name.

> Instead I would suggest to add a new dpkg control field 'X-Popcon:
> private' and have popularity-contest skip packages having this field.

This isn't going to be useful for users who want to exclude packages
from repos that they do not control or packages built by mk-build-deps
or other tools that do not allow adding extra dpkg control fields.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise


signature.asc
Description: This is a digitally signed message part

Bug#632438: [Popcon-developers] Bug#681721: #632438:

2020-09-20 Thread Bill Allombert 
On Sun, Sep 20, 2020 at 08:08:35PM +0900, Kentaro Hayashi wrote:
> I want to exclude installed packages from 3rd party vendor
> such as /etc/apt/sources.list.d/*.list. (some may be proprietary software)

OK, but what is your purpose in excluding them from popcon ?

Cheers,
-- 
Bill. 

Imagine a large red swirl here.

Bug#632438: [Popcon-developers] Bug#681721: #632438:

2020-09-20 Thread Kentaro Hayashi 
 popularity-contest: a  way to exclude certain packages
Message-Id: <20200920200835.682a9d43e058cc71a15a6...@xdump.org>
In-Reply-To: <20200920090418.GC23508@yellowpig>
X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; x86_64-pc-linux-gnu)
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Hi,

On Sun, 20 Sep 2020 11:04:18 +0200 Bill Allombert  wrote:
> On Sun, Sep 20, 2020 at 01:12:56PM +0900, Kentaro Hayashi wrote:
> > Control: tags -1 patch
> > 
> > Hi,
> > 
> > I've attached a simple PoC patch to exclude certain packages.
> > patch is generated against 1.70.
> > 
> > It may be better spec or implementation, but just works for me.
> 
> Dear Kentaro,
> Could you explain why you want to exclude some package ?
> I am concerned this will squew the statistic, if used indiscriminately.

I want to exclude installed packages from 3rd party vendor
such as /etc/apt/sources.list.d/*.list. (some may be proprietary software)

> Instead I would suggest to add a new dpkg control field 'X-Popcon: private' 
> and 
> have popularity-contest skip packages having this field.

I will appreciate this idea.
If the packages are under my control, I could add 'X-Popcon: private',
but not for 3rd party vendor's package every updates, I think.

> This way users will be able to create private packages that will never
> register even on misconfigured hosts.

So I think that BOTH of approach is appropriate.

1. Add (popularity-contenst) configuration option to exclude specific
   packages (such as 3rd party packages)
   I've atached a PoC patch.
2. Use 'X-Popcon: private' for private packages.


Regards,

Bug#632438: [Popcon-developers] Bug#681721: #632438: popularity-contest: a way to exclude certain packages

2020-09-20 Thread Bill Allombert 
On Sun, Sep 20, 2020 at 01:12:56PM +0900, Kentaro Hayashi wrote:
> Control: tags -1 patch
> 
> Hi,
> 
> I've attached a simple PoC patch to exclude certain packages.
> patch is generated against 1.70.
> 
> It may be better spec or implementation, but just works for me.

Dear Kentaro,
Could you explain why you want to exclude some package ?
I am concerned this will squew the statistic, if used indiscriminately.

Instead I would suggest to add a new dpkg control field 'X-Popcon: private' and 
have popularity-contest skip packages having this field.

This way users will be able to create private packages that will never
register even on misconfigured hosts.

Cheers,
Bill

Bug#632438: [Popcon-developers] Bug#681721: #632438: popularity-contest: a way to exclude certain packages

2020-09-19 Thread Kentaro Hayashi 
Control: tags -1 patch

Hi,

I've attached a simple PoC patch to exclude certain packages.
patch is generated against 1.70.

It may be better spec or implementation, but just works for me.

--- popularity-contest-1.70.orig/popularity-contest	2020-03-31 02:47:56.0 +0900
+++ popularity-contest-1.70/popularity-contest	2020-09-20 13:08:25.858919252 +0900
@@ -28,6 +28,7 @@
 my $dpkg_db="/var/lib/dpkg/info";
 my $dpkg_origin="/etc/dpkg/origins/default";
 my $popcon_conf="/etc/popularity-contest.conf";
+my $donotsend_conf="/etc/popularity-contest.donotsend.conf";
 
 # $popcon_conf is in shell-script format
 my $HOSTID = qx(unset MY_HOSTID; . $popcon_conf; echo \$MY_HOSTID );
@@ -204,6 +205,19 @@
 
 close PACKAGES;
 
+# We do not send package name which is listed on /etc/popularity-contest.donotsend.conf.
+if ( -r $donotsend_conf && -s $donotsend_conf ) {
+open DONOTSEND, $donotsend_conf;
+while () {
+	chomp $_;
+	my $name = $_;
+	if (exists $popcon{$name}) {
+	delete $popcon{$name};
+	}
+}
+close (DONOTSEND);
+}
+
 # We're not done yet.  Sort the output in reverse by atime, and
 # add a header/footer.

Bug#632438: [Popcon-developers] Bug#681721: #632438: popularity-contest: a way to exclude certain packages

2014-07-12 Thread Bill Allombert 
On Fri, Jul 11, 2014 at 04:17:54PM -0700, Vagrant Cascadian wrote:
> Control: merge 632438 681721 
> 
> These two bugs seem identical:
> 
>   #632438: popularity-contest: a way to exclude certain packages
>   #681721: popularity-contest: option to limit the list of packages sended to 
> popcon
> 
> Hopefully you'll agree.

It is not me who should agree but the submitters, which you did not CC.

I am sure you meant well, but as far as I am concerned this is a distraction.
There is too much diverging discussion to handle them as a single report.

What I am more interested is what are the proposed use cases for the feature
and whether this is something popularity-contest should support.

I offered 'X-Popcon-report: no' but the reporters do not seem interested.

Cheers,
-- 
Bill. 

Imagine a large red swirl here.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org