Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-10 Thread John Baldwin
On Tuesday, May 10, 2016 06:30:53 PM Glen Barber wrote:
> On Tue, May 10, 2016 at 10:22:04AM -0700, John Baldwin wrote:
> > On Tuesday, May 10, 2016 05:12:28 PM Glen Barber wrote:
> > > On Tue, May 10, 2016 at 10:04:47AM -0700, John Baldwin wrote:
> > > > On Tuesday, May 10, 2016 06:32:29 AM Glen Barber wrote:
> > > > > On Tue, May 10, 2016 at 08:25:22AM +0200, Thomas Zander wrote:
> > > > > > On 10 May 2016 at 08:18, O. Hartmann  
> > > > > > wrote:
> > > > > > 
> > > > > > > I haven't figured out so far how far this goes. Lucky for those 
> > > > > > > having
> > > > > > > recent /etc/ backups. A pity FreeBSD doens't backup this by 
> > > > > > > default.
> > > > > > 
> > > > > > After having shot myself in the foot some time ago, "zfs snapshot" 
> > > > > > has
> > > > > > become a part of my standard upgrade procedures :-)
> > > > > > 
> > > > > 
> > > > > No argument that this is valuable, but we cannot rely on filesystem
> > > > > specific solutions.  Similar topic came up a few days ago following
> > > > > lunch.  It got me thinking of a better way to ensure this kind of 
> > > > > thing
> > > > > does not require home-grown foot protection from cannons.
> > > > > 
> > > > > It should be fairly trivial to automatically backup /etc (and related)
> > > > > when 'distribution' is run, either intentionally or accidentally (or 
> > > > > by
> > > > > commit mistakes, such as this).
> > > > 
> > > > Saving the output of 'etcupdate diff' nightly might not be a bad first 
> > > > step.
> > > > 
> > > 
> > > This is also a good way to alleviate such things, however I am unsure
> > > how to handle cases where 'etcupdate' would inadvertently run into
> > > a conflict.  This was my concern with implementing an "automatic"
> > > etcupdate run in the runtime package.
> > 
> > I mean as part of the nightly jobs we could add one that stores
> > 'etcupdate diff' in /var the same as we do with backups of the 
> > master.passwd,
> > group, and aliases files in /var/backups.  You can then at least use that to
> > reconstruct altered /etc files by applying the diffs.  This isn't meant to 
> > be
> > an automated update run, but just saving a diff as part of the nightly jobs.
> > 
> 
> To be clear, I do not disagree with the idea as a whole.  However,
> I think we should considering making this part of the 'installworld' (or
> a dependent step of installworld) so we don't need to rely on periodic
> script execution.  (Consider cases where one may shut down their laptop
> before going to sleep, and the job never runs).

On many machines config changes to /etc happen more often than installworlds
(e.g. adding a new user, etc.).  What we could do is to add a suggestion of
saving an etcupdate snapshot after installworld to the list of world steps in
the handbook.  Warren is working on adding etcupdate to the world instructions
in the handbook anyway so this could perhaps be done after that.  However, I
don't think 'make installworld' itself should do this.

> > As far as what to do in runtime packages, presumably there isn't a single
> > package with all of etc, but etc files can be split up (ppp.conf in the ppp
> > package, etc.) and pkg needs to do its own 3-way merge of changes to conf
> > files when upgrading.
> 
> As the state of things are now, /etc is not included in any package
> (config files exempted), but pkg(8) does have merge ability now.  The
> problem with the initial implementation of "packaging /etc" was that it
> broke etcupdate(8) and presumably mergemaster(8), as the files were now
> part of an 'install'-style target, not 'distribute'-style target.
> 
> But, I think you gave me an idea on how to handle this.  (I'll test what
> I have in mind, but I'm not sure if it will continue to break the merge
> tools again as a side effect.)

My expectation is that if pkg managed /etc files you wouldn't need etcupdate
or mergemaster anymore with a packaged base.  People who are using source-based
upgrades could continue to do that, but people using packages would just use
'pkg upgrade' and have /etc files handled as part of that.

> > (This would be nice for conf files for ports in
> > /usr/local/etc as well.)  You still need to figure out how to handle
> > conflicts, but if pkg manages /etc files as config files and does a 3-way
> > merge of the old package and new package then that will serve to reimplement
> > etcupdate as part of 'pkg upgrade'.
> 
> Merge conflicts where human intervention is required is why I am
> hesitant to add an implicit 'etcupdate' invocation as a post-install
> script, as it breaks automated, non-interactive upgrades.

If pkg is going to really handle config files it has to handle this case
anyway.  Note that etcupdate already punts if this happens as well, but in my
experience such conflicts are quite rare.  A similar model as to what
happens with etcupdate (leave existing file in place until conflicts are
resolved but provide a command to resolve conflicts 

Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-10 Thread Glen Barber
On Tue, May 10, 2016 at 10:22:04AM -0700, John Baldwin wrote:
> On Tuesday, May 10, 2016 05:12:28 PM Glen Barber wrote:
> > On Tue, May 10, 2016 at 10:04:47AM -0700, John Baldwin wrote:
> > > On Tuesday, May 10, 2016 06:32:29 AM Glen Barber wrote:
> > > > On Tue, May 10, 2016 at 08:25:22AM +0200, Thomas Zander wrote:
> > > > > On 10 May 2016 at 08:18, O. Hartmann  
> > > > > wrote:
> > > > > 
> > > > > > I haven't figured out so far how far this goes. Lucky for those 
> > > > > > having
> > > > > > recent /etc/ backups. A pity FreeBSD doens't backup this by default.
> > > > > 
> > > > > After having shot myself in the foot some time ago, "zfs snapshot" has
> > > > > become a part of my standard upgrade procedures :-)
> > > > > 
> > > > 
> > > > No argument that this is valuable, but we cannot rely on filesystem
> > > > specific solutions.  Similar topic came up a few days ago following
> > > > lunch.  It got me thinking of a better way to ensure this kind of thing
> > > > does not require home-grown foot protection from cannons.
> > > > 
> > > > It should be fairly trivial to automatically backup /etc (and related)
> > > > when 'distribution' is run, either intentionally or accidentally (or by
> > > > commit mistakes, such as this).
> > > 
> > > Saving the output of 'etcupdate diff' nightly might not be a bad first 
> > > step.
> > > 
> > 
> > This is also a good way to alleviate such things, however I am unsure
> > how to handle cases where 'etcupdate' would inadvertently run into
> > a conflict.  This was my concern with implementing an "automatic"
> > etcupdate run in the runtime package.
> 
> I mean as part of the nightly jobs we could add one that stores
> 'etcupdate diff' in /var the same as we do with backups of the master.passwd,
> group, and aliases files in /var/backups.  You can then at least use that to
> reconstruct altered /etc files by applying the diffs.  This isn't meant to be
> an automated update run, but just saving a diff as part of the nightly jobs.
> 

To be clear, I do not disagree with the idea as a whole.  However,
I think we should considering making this part of the 'installworld' (or
a dependent step of installworld) so we don't need to rely on periodic
script execution.  (Consider cases where one may shut down their laptop
before going to sleep, and the job never runs).

> As far as what to do in runtime packages, presumably there isn't a single
> package with all of etc, but etc files can be split up (ppp.conf in the ppp
> package, etc.) and pkg needs to do its own 3-way merge of changes to conf
> files when upgrading.

As the state of things are now, /etc is not included in any package
(config files exempted), but pkg(8) does have merge ability now.  The
problem with the initial implementation of "packaging /etc" was that it
broke etcupdate(8) and presumably mergemaster(8), as the files were now
part of an 'install'-style target, not 'distribute'-style target.

But, I think you gave me an idea on how to handle this.  (I'll test what
I have in mind, but I'm not sure if it will continue to break the merge
tools again as a side effect.)

> (This would be nice for conf files for ports in
> /usr/local/etc as well.)  You still need to figure out how to handle
> conflicts, but if pkg manages /etc files as config files and does a 3-way
> merge of the old package and new package then that will serve to reimplement
> etcupdate as part of 'pkg upgrade'.

Merge conflicts where human intervention is required is why I am
hesitant to add an implicit 'etcupdate' invocation as a post-install
script, as it breaks automated, non-interactive upgrades.

> Having a 'pkg confdiff' or some such to
> output diffs made to conf files would be the equivalent of 'etcupdate diff'
> in that case (and would be nice as it would apply to conf files in ports as
> well).
> 

Agreed.

But, to be honest, I'd like to use etcupdate for this if it comes down
to it.  We use it in the cluster, and have never run into an issue
(until I introduced the change mentioned above, which removed things
like /etc/auto_master (part of the autofs package).

Glen



signature.asc
Description: PGP signature


Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-10 Thread Garance A Drosehn
On 10 May 2016, at 2:24, Glen Barber wrote:

> On Tue, May 10, 2016 at 08:18:44AM +0200, O. Hartmann wrote:
>>
>> It is not only master.passwd, it is also group and several other
>> config files, I suspect it is the whole bunch of files located
>> in /etc/ getting reset to their initial file values.
>>
>> My OpenLDAP environment isn't working anymore due to /etc/pam.d
>> reset. X11 doesn't start anymore due to reset of /etc/ttys. also,
>> sysctl.conf has been reset.
>
> The change (incorrectly) invoked the 'distribution' target, so
> anything that gets "touched" by that will likely be affected.
>
> You are correct that we should have an additional failsafe for
> this kind of thing, not just a subset of files arbitrarily placed
> in /var/backups via a periodic(8) script.

Hmm.  When working on some non-BSD open-source system, I found it
prudent to backup /etc.  And I'm lazy, so I went with a simple
tactic of:

   MLET=$(awk -v "MDIG=$(date +%m)" \
 'BEGIN { print substr("ABCDEFGHJKLMxyz", MDIG, 1); }')
   ETCTARNAME="/tmp/$(hostname -s)-etc-$(date +%Y${MLET}%d).tbz2"
   ETCLNKNAME="etc-$(hostname -s)-$(date +%Y${MLET}%d)"
   cd /
   ln -s etc "$ETCLNKNAME"
   nice tar cjf "$ETCTARNAME" "$ETCLNKNAME"/*
   scp -p  "$ETCTARNAME" $ETCSAV_DEST:Downloads/SAV-etcs
   rm -f   "$ETCTARNAME" "$ETCLNKNAME"

The idea is to create a symlink of etc which includes a timestamp
(eg: "etc-freefall-2016E10"), and create a compressed tar archive
which saves all the files as being under that directory-name instead
of /etc.  I then copy that to a different host, and remove the
archive file.  Maybe I should add something like that to my own
installworld script.  Probably should adjust it somewhat to pay
better attention to potential security issues.  (you wouldn't want
to copy that archive file to a public FTP server, for instance!)

Then when something goes haywire, I would create a new archive
and then compare the two complete sets of /etc files to see what
has changed.

-- 
Garance Alistair Drosehn= dro...@rpi.edu
Senior Systems Programmer   or   g...@freebsd.org
Rensselaer Polytechnic Institute; Troy, NY;  USA
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-10 Thread Nikolai Lifanov
On 05/10/2016 13:22, John Baldwin wrote:
> On Tuesday, May 10, 2016 05:12:28 PM Glen Barber wrote:
>> On Tue, May 10, 2016 at 10:04:47AM -0700, John Baldwin wrote:
>>> On Tuesday, May 10, 2016 06:32:29 AM Glen Barber wrote:
 On Tue, May 10, 2016 at 08:25:22AM +0200, Thomas Zander wrote:
> On 10 May 2016 at 08:18, O. Hartmann  wrote:
>
>> I haven't figured out so far how far this goes. Lucky for those having
>> recent /etc/ backups. A pity FreeBSD doens't backup this by default.
>
> After having shot myself in the foot some time ago, "zfs snapshot" has
> become a part of my standard upgrade procedures :-)
>

 No argument that this is valuable, but we cannot rely on filesystem
 specific solutions.  Similar topic came up a few days ago following
 lunch.  It got me thinking of a better way to ensure this kind of thing
 does not require home-grown foot protection from cannons.

 It should be fairly trivial to automatically backup /etc (and related)
 when 'distribution' is run, either intentionally or accidentally (or by
 commit mistakes, such as this).
>>>
>>> Saving the output of 'etcupdate diff' nightly might not be a bad first step.
>>>
>>
>> This is also a good way to alleviate such things, however I am unsure
>> how to handle cases where 'etcupdate' would inadvertently run into
>> a conflict.  This was my concern with implementing an "automatic"
>> etcupdate run in the runtime package.
> 
> I mean as part of the nightly jobs we could add one that stores
> 'etcupdate diff' in /var the same as we do with backups of the master.passwd,
> group, and aliases files in /var/backups.  You can then at least use that to
> reconstruct altered /etc files by applying the diffs.  This isn't meant to be
> an automated update run, but just saving a diff as part of the nightly jobs.
> 

That's what I do. The periodic "etcupdate diff" dumps, which I was
already taking despite boot environments helped me work through various
pkgbase issues.

> As far as what to do in runtime packages, presumably there isn't a single
> package with all of etc, but etc files can be split up (ppp.conf in the ppp
> package, etc.) and pkg needs to do its own 3-way merge of changes to conf
> files when upgrading.  (This would be nice for conf files for ports in
> /usr/local/etc as well.)  You still need to figure out how to handle
> conflicts, but if pkg manages /etc files as config files and does a 3-way
> merge of the old package and new package then that will serve to reimplement
> etcupdate as part of 'pkg upgrade'.  Having a 'pkg confdiff' or some such to
> output diffs made to conf files would be the equivalent of 'etcupdate diff'
> in that case (and would be nice as it would apply to conf files in ports as
> well).
> 

Having "pkg confdiff" would be wonderful, for both base and ports.

- Nikolai Lifanov

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-10 Thread Eric van Gyzen
On 05/10/2016 01:25, Thomas Zander wrote:
> On 10 May 2016 at 08:18, O. Hartmann  wrote:
>
>> I haven't figured out so far how far this goes. Lucky for those having
>> recent /etc/ backups. A pity FreeBSD doens't backup this by default.
> After having shot myself in the foot some time ago, "zfs snapshot" has
> become a part of my standard upgrade procedures :-)

Similarly, "beadm create" has become a part of my standard upgrade
procedures.  If the new world is hosed, I just boot an old Boot Environment.

I'm grateful to all the people who made this possible.  I would call you
by name, but I would probably miss some people.

Making this part of installworld (or similar) would be interesting.

Eric

$ beadm list
BE  Active Mountpoint  Space Created
r295605 -  -9.2G 2016-03-12 14:07
r296766 -  -7.1G 2016-03-24 15:19
r297219 -  -  788.0M 2016-04-11 07:04
r297684 -  -  781.0M 2016-04-28 09:00
default NR /   50.9G 2016-05-04 20:19
r298743 -  -5.4G 2016-05-07 09:37
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-10 Thread John Baldwin
On Tuesday, May 10, 2016 05:12:28 PM Glen Barber wrote:
> On Tue, May 10, 2016 at 10:04:47AM -0700, John Baldwin wrote:
> > On Tuesday, May 10, 2016 06:32:29 AM Glen Barber wrote:
> > > On Tue, May 10, 2016 at 08:25:22AM +0200, Thomas Zander wrote:
> > > > On 10 May 2016 at 08:18, O. Hartmann  
> > > > wrote:
> > > > 
> > > > > I haven't figured out so far how far this goes. Lucky for those having
> > > > > recent /etc/ backups. A pity FreeBSD doens't backup this by default.
> > > > 
> > > > After having shot myself in the foot some time ago, "zfs snapshot" has
> > > > become a part of my standard upgrade procedures :-)
> > > > 
> > > 
> > > No argument that this is valuable, but we cannot rely on filesystem
> > > specific solutions.  Similar topic came up a few days ago following
> > > lunch.  It got me thinking of a better way to ensure this kind of thing
> > > does not require home-grown foot protection from cannons.
> > > 
> > > It should be fairly trivial to automatically backup /etc (and related)
> > > when 'distribution' is run, either intentionally or accidentally (or by
> > > commit mistakes, such as this).
> > 
> > Saving the output of 'etcupdate diff' nightly might not be a bad first step.
> > 
> 
> This is also a good way to alleviate such things, however I am unsure
> how to handle cases where 'etcupdate' would inadvertently run into
> a conflict.  This was my concern with implementing an "automatic"
> etcupdate run in the runtime package.

I mean as part of the nightly jobs we could add one that stores
'etcupdate diff' in /var the same as we do with backups of the master.passwd,
group, and aliases files in /var/backups.  You can then at least use that to
reconstruct altered /etc files by applying the diffs.  This isn't meant to be
an automated update run, but just saving a diff as part of the nightly jobs.

As far as what to do in runtime packages, presumably there isn't a single
package with all of etc, but etc files can be split up (ppp.conf in the ppp
package, etc.) and pkg needs to do its own 3-way merge of changes to conf
files when upgrading.  (This would be nice for conf files for ports in
/usr/local/etc as well.)  You still need to figure out how to handle
conflicts, but if pkg manages /etc files as config files and does a 3-way
merge of the old package and new package then that will serve to reimplement
etcupdate as part of 'pkg upgrade'.  Having a 'pkg confdiff' or some such to
output diffs made to conf files would be the equivalent of 'etcupdate diff'
in that case (and would be nice as it would apply to conf files in ports as
well).

-- 
John Baldwin

signature.asc
Description: This is a digitally signed message part.


Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-10 Thread Glen Barber
On Tue, May 10, 2016 at 10:04:47AM -0700, John Baldwin wrote:
> On Tuesday, May 10, 2016 06:32:29 AM Glen Barber wrote:
> > On Tue, May 10, 2016 at 08:25:22AM +0200, Thomas Zander wrote:
> > > On 10 May 2016 at 08:18, O. Hartmann  wrote:
> > > 
> > > > I haven't figured out so far how far this goes. Lucky for those having
> > > > recent /etc/ backups. A pity FreeBSD doens't backup this by default.
> > > 
> > > After having shot myself in the foot some time ago, "zfs snapshot" has
> > > become a part of my standard upgrade procedures :-)
> > > 
> > 
> > No argument that this is valuable, but we cannot rely on filesystem
> > specific solutions.  Similar topic came up a few days ago following
> > lunch.  It got me thinking of a better way to ensure this kind of thing
> > does not require home-grown foot protection from cannons.
> > 
> > It should be fairly trivial to automatically backup /etc (and related)
> > when 'distribution' is run, either intentionally or accidentally (or by
> > commit mistakes, such as this).
> 
> Saving the output of 'etcupdate diff' nightly might not be a bad first step.
> 

This is also a good way to alleviate such things, however I am unsure
how to handle cases where 'etcupdate' would inadvertently run into
a conflict.  This was my concern with implementing an "automatic"
etcupdate run in the runtime package.

Glen



signature.asc
Description: PGP signature


Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-10 Thread John Baldwin
On Tuesday, May 10, 2016 06:32:29 AM Glen Barber wrote:
> On Tue, May 10, 2016 at 08:25:22AM +0200, Thomas Zander wrote:
> > On 10 May 2016 at 08:18, O. Hartmann  wrote:
> > 
> > > I haven't figured out so far how far this goes. Lucky for those having
> > > recent /etc/ backups. A pity FreeBSD doens't backup this by default.
> > 
> > After having shot myself in the foot some time ago, "zfs snapshot" has
> > become a part of my standard upgrade procedures :-)
> > 
> 
> No argument that this is valuable, but we cannot rely on filesystem
> specific solutions.  Similar topic came up a few days ago following
> lunch.  It got me thinking of a better way to ensure this kind of thing
> does not require home-grown foot protection from cannons.
> 
> It should be fairly trivial to automatically backup /etc (and related)
> when 'distribution' is run, either intentionally or accidentally (or by
> commit mistakes, such as this).

Saving the output of 'etcupdate diff' nightly might not be a bad first step.

-- 
John Baldwin

signature.asc
Description: This is a digitally signed message part.


Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-10 Thread Glen Barber
On Tue, May 10, 2016 at 10:31:03AM -0400, Allan Jude wrote:
> On 2016-05-10 02:32, Glen Barber wrote:
> > On Tue, May 10, 2016 at 08:25:22AM +0200, Thomas Zander wrote:
> >> On 10 May 2016 at 08:18, O. Hartmann  wrote:
> >>
> >>> I haven't figured out so far how far this goes. Lucky for those having
> >>> recent /etc/ backups. A pity FreeBSD doens't backup this by default.
> >>
> >> After having shot myself in the foot some time ago, "zfs snapshot" has
> >> become a part of my standard upgrade procedures :-)
> >>
> > 
> > No argument that this is valuable, but we cannot rely on filesystem
> > specific solutions.  Similar topic came up a few days ago following
> > lunch.  It got me thinking of a better way to ensure this kind of thing
> > does not require home-grown foot protection from cannons.
> > 
> > It should be fairly trivial to automatically backup /etc (and related)
> > when 'distribution' is run, either intentionally or accidentally (or by
> > commit mistakes, such as this).
> > 
> 
> I wonder if you couldn't actually package it. Before installing the new
> /etc, it would create a package of what is in your old /etc and put it
> somewhere safe, so an upgrade could be reverted by manually installing
> that package.
> 

In the context through which the above was intended, it wasn't intended
for a new 'package' (assuming you mean pkg(8)).  Just an archive of what
was there before the changes, so some notion of a "rollback" (filesystem
agnostic) is possible.

Glen



signature.asc
Description: PGP signature


Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-10 Thread Allan Jude
On 2016-05-10 02:32, Glen Barber wrote:
> On Tue, May 10, 2016 at 08:25:22AM +0200, Thomas Zander wrote:
>> On 10 May 2016 at 08:18, O. Hartmann  wrote:
>>
>>> I haven't figured out so far how far this goes. Lucky for those having
>>> recent /etc/ backups. A pity FreeBSD doens't backup this by default.
>>
>> After having shot myself in the foot some time ago, "zfs snapshot" has
>> become a part of my standard upgrade procedures :-)
>>
> 
> No argument that this is valuable, but we cannot rely on filesystem
> specific solutions.  Similar topic came up a few days ago following
> lunch.  It got me thinking of a better way to ensure this kind of thing
> does not require home-grown foot protection from cannons.
> 
> It should be fairly trivial to automatically backup /etc (and related)
> when 'distribution' is run, either intentionally or accidentally (or by
> commit mistakes, such as this).
> 
> Glen
> 

I wonder if you couldn't actually package it. Before installing the new
/etc, it would create a package of what is in your old /etc and put it
somewhere safe, so an upgrade could be reverted by manually installing
that package.

-- 
Allan Jude
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-10 Thread Thomas Zander
On 10 May 2016 at 08:32, Glen Barber  wrote:
> On Tue, May 10, 2016 at 08:25:22AM +0200, Thomas Zander wrote:
>> On 10 May 2016 at 08:18, O. Hartmann  wrote:
>>
>> > I haven't figured out so far how far this goes. Lucky for those having
>> > recent /etc/ backups. A pity FreeBSD doens't backup this by default.
>>
>> After having shot myself in the foot some time ago, "zfs snapshot" has
>> become a part of my standard upgrade procedures :-)
>>
>
> No argument that this is valuable, but we cannot rely on filesystem
> specific solutions.  Similar topic came up a few days ago following
> lunch.  It got me thinking of a better way to ensure this kind of thing
> does not require home-grown foot protection from cannons.

Absolutely. I didn't mean to propose zfs snapshots as the one true solution.
But since they saved me from a couple of late-in-the-night-mistakes in
the past, we could have "make installworld / distribution" create a
snapshot if the corresponding fs happens to be zfs. This could be
enabled or disabled via a variable in make.conf.

Best regards
Riggs
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-10 Thread Glen Barber
On Tue, May 10, 2016 at 02:38:36PM +0800, Marcelo Araujo wrote:
> 2016-05-10 14:32 GMT+08:00 Glen Barber :
> 
> > On Tue, May 10, 2016 at 08:25:22AM +0200, Thomas Zander wrote:
> > > On 10 May 2016 at 08:18, O. Hartmann 
> > wrote:
> > >
> > > > I haven't figured out so far how far this goes. Lucky for those having
> > > > recent /etc/ backups. A pity FreeBSD doens't backup this by default.
> > >
> > > After having shot myself in the foot some time ago, "zfs snapshot" has
> > > become a part of my standard upgrade procedures :-)
> > >
> >
> > No argument that this is valuable, but we cannot rely on filesystem
> > specific solutions.  Similar topic came up a few days ago following
> > lunch.  It got me thinking of a better way to ensure this kind of thing
> > does not require home-grown foot protection from cannons.
> >
> > It should be fairly trivial to automatically backup /etc (and related)
> > when 'distribution' is run, either intentionally or accidentally (or by
> > commit mistakes, such as this).
> >
> >
> As an idea, when we run make installworld before it replaces the files, it
> could make a backup of those files that will be replaced to
> /var/backups/etc${DATE}/ .
> 

Yes, this is similar to what I was considering implementing.

Glen



signature.asc
Description: PGP signature


Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-10 Thread Marcelo Araujo
2016-05-10 14:32 GMT+08:00 Glen Barber :

> On Tue, May 10, 2016 at 08:25:22AM +0200, Thomas Zander wrote:
> > On 10 May 2016 at 08:18, O. Hartmann 
> wrote:
> >
> > > I haven't figured out so far how far this goes. Lucky for those having
> > > recent /etc/ backups. A pity FreeBSD doens't backup this by default.
> >
> > After having shot myself in the foot some time ago, "zfs snapshot" has
> > become a part of my standard upgrade procedures :-)
> >
>
> No argument that this is valuable, but we cannot rely on filesystem
> specific solutions.  Similar topic came up a few days ago following
> lunch.  It got me thinking of a better way to ensure this kind of thing
> does not require home-grown foot protection from cannons.
>
> It should be fairly trivial to automatically backup /etc (and related)
> when 'distribution' is run, either intentionally or accidentally (or by
> commit mistakes, such as this).
>
> Glen
>
>
As an idea, when we run make installworld before it replaces the files, it
could make a backup of those files that will be replaced to
/var/backups/etc${DATE}/ .


Best,
-- 

-- 
Marcelo Araujo(__)ara...@freebsd.org
\\\'',)http://www.FreeBSD.org    \/  \ ^
Power To Server. .\. /_)
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-10 Thread Glen Barber
On Tue, May 10, 2016 at 08:31:46AM +0200, O. Hartmann wrote:
> On Tue, 10 May 2016 06:24:36 + Glen Barber  wrote:
> > The change (incorrectly) invoked the 'distribution' target, so anything
> > that gets "touched" by that will likely be affected.
> 
> In my case, it is *EVERY* file located in /usr/share/examples/etc which is now
> reset. That includes also profile and csh.cshrc and fellows. 
> 

Well, yes, these are part of the 'distribution' target.

Glen



signature.asc
Description: PGP signature


Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-10 Thread Glen Barber
On Tue, May 10, 2016 at 08:25:22AM +0200, Thomas Zander wrote:
> On 10 May 2016 at 08:18, O. Hartmann  wrote:
> 
> > I haven't figured out so far how far this goes. Lucky for those having
> > recent /etc/ backups. A pity FreeBSD doens't backup this by default.
> 
> After having shot myself in the foot some time ago, "zfs snapshot" has
> become a part of my standard upgrade procedures :-)
> 

No argument that this is valuable, but we cannot rely on filesystem
specific solutions.  Similar topic came up a few days ago following
lunch.  It got me thinking of a better way to ensure this kind of thing
does not require home-grown foot protection from cannons.

It should be fairly trivial to automatically backup /etc (and related)
when 'distribution' is run, either intentionally or accidentally (or by
commit mistakes, such as this).

Glen



signature.asc
Description: PGP signature


Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-10 Thread O. Hartmann
On Tue, 10 May 2016 06:24:36 +
Glen Barber  wrote:

> On Tue, May 10, 2016 at 08:18:44AM +0200, O. Hartmann wrote:
> > On Tue, 10 May 2016 05:53:41 +
> > Glen Barber  wrote:
> >   
> > > Thanks to O. Hartmann promptly reporting this, it was discovered that
> > > 'installworld' on revisions r299292-r299317 will silently replace
> > > /etc/passwd, /etc/master.passwd, and /etc/group with the defaults.  It
> > > is possible there are other files affected.  One file I can think of
> > > off-hand is /etc/mail/aliases, but in my development system, did not
> > > have local changes to this, so cannot 100% confirm.
> > > 
> > > Please avoid this range of revisions.
> > > 
> > > I am very sorry this went unnoticed before this change was committed.
> > >   
> > 
> > Great!
> > 
> > It is not only master.passwd, it is also group and several other config
> > files, I suspect it is the whole bunch of files located in /etc/ getting
> > reset to their initial file values.
> > 
> > My OpenLDAP environment isn't working anymore due to /etc/pam.d reset. X11
> > doesn't start anymore due to reset of /etc/ttys. also, sysctl.conf has been
> > reset.
> > 
> > I haven't figured out so far how far this goes. Lucky for those having
> > recent /etc/ backups. A pity FreeBSD doens't backup this by default.
> >   
> 
> The change (incorrectly) invoked the 'distribution' target, so anything
> that gets "touched" by that will likely be affected.

In my case, it is *EVERY* file located in /usr/share/examples/etc which is now
reset. That includes also profile and csh.cshrc and fellows. 

> 
> You are correct that we should have an additional failsafe for this kind
> of thing, not just a subset of files arbitrarily placed in /var/backups
> via a periodic(8) script.
> 
> Glen
> 

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-10 Thread Thomas Zander
On 10 May 2016 at 08:18, O. Hartmann  wrote:

> I haven't figured out so far how far this goes. Lucky for those having
> recent /etc/ backups. A pity FreeBSD doens't backup this by default.

After having shot myself in the foot some time ago, "zfs snapshot" has
become a part of my standard upgrade procedures :-)

Riggs
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-10 Thread Glen Barber
On Tue, May 10, 2016 at 08:18:44AM +0200, O. Hartmann wrote:
> On Tue, 10 May 2016 05:53:41 +
> Glen Barber  wrote:
> 
> > Thanks to O. Hartmann promptly reporting this, it was discovered that
> > 'installworld' on revisions r299292-r299317 will silently replace
> > /etc/passwd, /etc/master.passwd, and /etc/group with the defaults.  It
> > is possible there are other files affected.  One file I can think of
> > off-hand is /etc/mail/aliases, but in my development system, did not
> > have local changes to this, so cannot 100% confirm.
> > 
> > Please avoid this range of revisions.
> > 
> > I am very sorry this went unnoticed before this change was committed.
> > 
> 
> Great!
> 
> It is not only master.passwd, it is also group and several other config files,
> I suspect it is the whole bunch of files located in /etc/ getting reset to
> their initial file values.
> 
> My OpenLDAP environment isn't working anymore due to /etc/pam.d reset. X11
> doesn't start anymore due to reset of /etc/ttys. also, sysctl.conf has been
> reset.
> 
> I haven't figured out so far how far this goes. Lucky for those having
> recent /etc/ backups. A pity FreeBSD doens't backup this by default.
> 

The change (incorrectly) invoked the 'distribution' target, so anything
that gets "touched" by that will likely be affected.

You are correct that we should have an additional failsafe for this kind
of thing, not just a subset of files arbitrarily placed in /var/backups
via a periodic(8) script.

Glen



signature.asc
Description: PGP signature


Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-10 Thread O. Hartmann
On Tue, 10 May 2016 05:53:41 +
Glen Barber  wrote:

> Thanks to O. Hartmann promptly reporting this, it was discovered that
> 'installworld' on revisions r299292-r299317 will silently replace
> /etc/passwd, /etc/master.passwd, and /etc/group with the defaults.  It
> is possible there are other files affected.  One file I can think of
> off-hand is /etc/mail/aliases, but in my development system, did not
> have local changes to this, so cannot 100% confirm.
> 
> Please avoid this range of revisions.
> 
> I am very sorry this went unnoticed before this change was committed.
> 
> Glen
> 

Great!

It is not only master.passwd, it is also group and several other config files,
I suspect it is the whole bunch of files located in /etc/ getting reset to
their initial file values.

My OpenLDAP environment isn't working anymore due to /etc/pam.d reset. X11
doesn't start anymore due to reset of /etc/ttys. also, sysctl.conf has been
reset.

I haven't figured out so far how far this goes. Lucky for those having
recent /etc/ backups. A pity FreeBSD doens't backup this by default.

oh
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

2016-05-09 Thread Glen Barber
Thanks to O. Hartmann promptly reporting this, it was discovered that
'installworld' on revisions r299292-r299317 will silently replace
/etc/passwd, /etc/master.passwd, and /etc/group with the defaults.  It
is possible there are other files affected.  One file I can think of
off-hand is /etc/mail/aliases, but in my development system, did not
have local changes to this, so cannot 100% confirm.

Please avoid this range of revisions.

I am very sorry this went unnoticed before this change was committed.

Glen



signature.asc
Description: PGP signature