Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

On Sun, 18 Aug 2019 at 13:57:58 +0200, Marc Haber wrote: > On Tue, 13 Aug 2019 18:30:51 +0100, Simon McVittie > >bubblewrap and other container-runners often use this when setting > >up containers - for example if you have a Flatpak app installed, try > >something like > > > >flatpak run

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

On Tue, 13 Aug 2019 18:30:51 +0100, Simon McVittie wrote: >On Tue, 13 Aug 2019 at 14:22:31 +0200, Marc Haber wrote: >> On Tue, 13 Aug 2019 12:01:13 +0100, Simon McVittie >> wrote: >> >(systemd cannot create a mount point that doesn't exist yet on a read-only >> >file system, which is why a

Re: duplicate popularity-contest ID

> Change popularity-contest by transmissing the hostid after it has been > hashed with the content of /etc/machine-id. Heh, there is no /etc/machine-id on my Debian system. I have an …/etc/machine-id in buster and stretch chroots I created and xenial, bionic and disco pbuilder base.cow

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

On Thu, 15 Aug 2019 at 09:54:44 +0100, Ian Jackson wrote: > Do we have a list of all the things this is (or might be) used for ? As I said, I don't think a comprehensive list is feasible without resorting to something like codesearch, because it's of similar scope to a list of reasons to use the

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

Simon McVittie writes ("Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)"): > Somehow describing which containers and chroots should have a machine ID, > which ones should share the host's machine ID and which ones don't need > either is a gap in

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

On Tue, 13 Aug 2019 at 22:01:34 -0400, Theodore Y. Ts'o wrote: > That's just a matter of having sysvinit (and other non-systemd init > systems) have an init script which runs as soon as the root file > system is remounted read/write to initialize /etc/machine-id if it > doesn't exist or if it is a

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

On Tue, Aug 13, 2019 at 06:30:51PM +0100, Simon McVittie wrote: > > >> >Maybe /etc/machine-id should be part of the "API" of a Debian system in > > >> >general (systemd or not)? > > > > So /etc/machine-id should be in Policy? > > Probably yes, if that proposal has consensus, although a

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

On Tue, 13 Aug 2019 at 14:22:31 +0200, Marc Haber wrote: > On Tue, 13 Aug 2019 12:01:13 +0100, Simon McVittie > wrote: > >(systemd cannot create a mount point that doesn't exist yet on a read-only > >file system, which is why a zero-byte file is preferred. > > but you can bind-mount over a file?

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

On Tue, 13 Aug 2019 12:01:13 +0100, Simon McVittie wrote: >(systemd cannot create a mount point that doesn't exist yet on a read-only >file system, which is why a zero-byte file is preferred. but you can bind-mount over a file? I wasn't aware of that. >If you use systemd as init, install dbus,

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

> "Simon" == Simon McVittie writes: Simon> If you use systemd as init, install dbus, delete or empty Simon> /etc/machine-id, delete /var/lib/dbus/machine-id and reboot, It is my experience that deleting /etc/machine-id doesn't actually work (even if I delete the dbus machine id

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

On Tue, 13 Aug 2019 at 11:50:27 +0200, Marc Haber wrote: > On Thu, 8 Aug 2019 22:44:19 +0100, Simon McVittie > wrote: > >Making /etc/machine-id a 0-byte file is considered to be the canonical > >way to clear it, rather than actually deleting it, because if systemd is > >running on a completely

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

On Thu, 8 Aug 2019 22:44:19 +0100, Simon McVittie wrote: >Making /etc/machine-id a 0-byte file is considered to be the canonical >way to clear it, rather than actually deleting it, because if systemd is >running on a completely read-only root filesystem, it has code to create >a machine ID on a

Re: duplicate popularity-contest ID

On Tue, Aug 06, 2019 at 02:01:16PM -0400, Sam Hartman wrote: > > "Bill" == Bill Allombert writes: > > Bill> This is potentially an excellent idea! > > Bill> Does not /etc/machine-id suffer of exactly the same issue as > Bill> /etc/popularity-contest.conf ? > > A lot more

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

On Thu, 08 Aug 2019 at 13:39:54 +0200, Marc Haber wrote: > Generating a new machine-id doesn't seem as easy as generating a new > ssh key: Removing /etc/machine-id doesn't do it as > systemd-machine-id-setup seems to pull the machine-id from dbus. For historical reasons (dbus originated the

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

On Wed, 7 Aug 2019 11:15:22 -0400, Marvin Renich wrote: >I think this is a good idea, but will require work and coordination to >accomplish. A wiki.debian.org page with your ideas and (perhaps on a >separate page) a place to list things that need updating after the >physical copying is complete

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

Am 08.08.19 um 13:39 schrieb Marc Haber: > On Wed, 07 Aug 2019 09:28:12 -0400, The Wanderer > wrote: >> On 2019-08-07 at 04:26, Russell Stuart wrote: >> >>> On Wed, 2019-08-07 at 09:34 +0200, Marc Haber wrote: >>> I am using Debian for two decades now, and I realized that necessity two

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

On Wed, 07 Aug 2019 09:28:12 -0400, The Wanderer wrote: >On 2019-08-07 at 04:26, Russell Stuart wrote: > >> On Wed, 2019-08-07 at 09:34 +0200, Marc Haber wrote: >> >>> I am using Debian for two decades now, and I realized that >>> necessity two days ago. >> >> Ditto - except for me it was a few

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

On 2019-08-07 at 16:59, Andrei POPESCU wrote: > On Mi, 07 aug 19, 09:28:12, The Wanderer wrote: > >> I've begun to wonder whether it might be worth the overhead to set up >> some type of mechanism to let packages which define such >> machine-specific IDs A: declare the fact, in a central

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

On Thu, Aug 8, 2019 at 4:59 AM Andrei POPESCU wrote: > 1. Delete the contents of /etc (all of it) > 2. If a package doesn't find its "stuff" in /etc it regenerates it from > defaults. There is still way too much stuff that defaults to installing important files in /etc (default config settings,

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

On Mi, 07 aug 19, 09:28:12, The Wanderer wrote: > > I've begun to wonder whether it might be worth the overhead to set up > some type of mechanism to let packages which define such > machine-specific IDs A: declare the fact, in a central location which Do you mean /etc? :) > the sysadmin of a

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

The Wanderer writes: > This isn't the first time I've discovered that some aspect of a Debian > system would actually need to be cleared and re-generated when that > system is cloned, well after the point where it would have been easy for > me to address that need. (Fortunately, although I've

Re: duplicate popularity-contest ID

Michael Stone writes: > I don't think popcon is a good reason to pause if there are valid > concerns suggesting removal is a good thing, for the exact reason that > it's skewed to propagating existing practice. I'm not sure there's any > really good use for popcon, but I'll continue to believe

Re: duplicate popularity-contest ID

On Wed, Aug 07, 2019 at 04:02:14PM +0100, Ian Jackson wrote: Michael Stone writes ("Re: duplicate popularity-contest ID"): I guess the question is what is the point of the popcon statistics. Insofar as they're used to determine defaults, skewing them toward custom images (which

Re: Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

* The Wanderer [190807 09:28]: > Cloning isn't the only example of a case where some machine-specific > configuration detail may need to be updated, without that being obvious > in advance. > > I've begun to wonder whether it might be worth the overhead to set up > some type of mechanism to let

Re: duplicate popularity-contest ID

Michael Stone writes ("Re: duplicate popularity-contest ID"): > I guess the question is what is the point of the popcon statistics. > Insofar as they're used to determine defaults, skewing them toward > custom images (which likely do not care about defaults) is probably a

Re: duplicate popularity-contest ID

On Wed, Aug 07, 2019 at 09:31:34AM +0200, Marc Haber wrote: On Tue, 6 Aug 2019 11:33:42 +, Bill Allombert wrote: Yesterday I received the same popcon ID 2600 times, and 4700 differents ID were received two times and 22000 ID were received exactly once. I understand the need for totally

Generating new IDs for cloning (was Re: duplicate popularity-contest ID)

On 2019-08-07 at 04:26, Russell Stuart wrote: > On Wed, 2019-08-07 at 09:34 +0200, Marc Haber wrote: > >> I am using Debian for two decades now, and I realized that >> necessity two days ago. > > Ditto - except for me it was a few seconds ago. In my case, it was when I read this thread last

Re: duplicate popularity-contest ID

On Wed, 2019-08-07 at 09:34 +0200, Marc Haber wrote: > I am using Debian for two decades now, and I realized that necessity > two days ago. Ditto - except for me it was a few seconds ago. signature.asc Description: This is a digitally signed message part

Re: duplicate popularity-contest ID

On Tue, 06 Aug 2019 14:01:16 -0400, Sam Hartman wrote: >> "Bill" == Bill Allombert writes: > >Bill> This is potentially an excellent idea! > >Bill> Does not /etc/machine-id suffer of exactly the same issue as >Bill> /etc/popularity-contest.conf ? > >A lot more procedures for

Re: duplicate popularity-contest ID

On Tue, 6 Aug 2019 11:33:42 +, Bill Allombert wrote: >Yesterday I received the same popcon ID 2600 times, and 4700 differents ID >were received >two times and 22000 ID were received exactly once. > >I understand the need for totally identical systems, but then probably >it does not make

Re: duplicate popularity-contest ID

On Tue, Aug 6, 2019 at 7:34 PM Bill Allombert wrote: > A related issue is that the submission time is randomized, but if > 2600 systems have identical /etc/cron.d/popularity-contest files, they > will report at the same time, causing network spikes. BTW, a systemd service timer has native

Re: duplicate popularity-contest ID

> "Bill" == Bill Allombert writes: Bill> This is potentially an excellent idea! Bill> Does not /etc/machine-id suffer of exactly the same issue as Bill> /etc/popularity-contest.conf ? A lot more procedures for cloning images know that they need to generate new /etc/machine-ids.

Re: duplicate popularity-contest ID

On Tue, Aug 06, 2019 at 12:08:13AM +0200, Marco d'Itri wrote: > On Aug 05, Bill Allombert wrote: > > > Each Debian popularity-contest submitter is supposed to have > > a different random 128bit popcon ID. > > However, the popularity-constest server > > receives a lot

Re: duplicate popularity-contest ID

On 2019-08-06 08:33:36 -0700 (-0700), Russ Allbery wrote: [...] > Hm. I think that's still in the range of what could be explained > by VM cloning, although the 2600 with the same ID is surprising. [...] A CI system which is using cloned virtual machines could easily do that. I help operate a CI

Re: duplicate popularity-contest ID

Bill Allombert writes: > Both. > Yesterday I received the same popcon ID 2600 times, and 4700 differents > ID were received two times and 22000 ID were received exactly once. Hm. I think that's still in the range of what could be explained by VM cloning, although the 2600 with the same ID is

Re: duplicate popularity-contest ID

On Mon, Aug 05, 2019 at 08:46:15AM -0700, Russ Allbery wrote: > Bill Allombert writes: > > > Each Debian popularity-contest submitter is supposed to have a different > > random 128bit popcon ID. However, the popularity-constest server > > receives a lot of

Re: duplicate popularity-contest ID

On Aug 05, Bill Allombert wrote: > Each Debian popularity-contest submitter is supposed to have > a different random 128bit popcon ID. > However, the popularity-constest server > receives a lot of submissions with identical popcon ID, which cause them > to be treated

Re: duplicate popularity-contest ID

Bill Allombert writes: > Each Debian popularity-contest submitter is supposed to have a different > random 128bit popcon ID. However, the popularity-constest server > receives a lot of submissions with identical > popcon ID, which cause them to be treated as a single

Re: duplicate popularity-contest ID

On 2019-08-05 15:29, Bill Allombert wrote: > However, the popularity-constest server > receives a lot of submissions with identical popcon ID, which cause them > to be treated as a single submission. I would suspect cloned VMs to have identical popcon IDs. In this case

Re: duplicate popularity-contest ID

On Mon, Aug 05, 2019 at 02:29:33PM +0200, Bill Allombert wrote: > Dear Debian developers, > > Each Debian popularity-contest submitter is supposed to have > a different random 128bit popcon ID. > However, the popularity-constest server > receives a lot of submissions

Re: duplicate popularity-contest ID

Hey Yao and Bill On 2019/08/05 14:31, "Yao Wei (魏銘廷)" wrote: >> I am not quite sure what it is the reason for this problem. >> Maybe people use prebuild system images with a pregenerated >> /etc/popularity-contest.conf file (instead of being generated >> by popcon postinst). > > Could this be

Re: duplicate popularity-contest ID

> On Aug 5, 2019, at 20:29, Bill Allombert wrote: > > I am not quite sure what it is the reason for this problem. > Maybe people use prebuild system images with a pregenerated > /etc/popularity-contest.conf file (instead of being generated > by popcon postinst). Could this be caused by

duplicate popularity-contest ID

Dear Debian developers, Each Debian popularity-contest submitter is supposed to have a different random 128bit popcon ID. However, the popularity-constest server receives a lot of submissions with identical popcon ID, which cause them to be treated as a single