/var/db/etcupdate/ in hier(7) (was: user problems when upgrading to v15)
On 09/09/2023 17:02, John Baldwin wrote: On 9/2/23 7:11 AM, Dimitry Andric wrote: … … /var/db/etcupdate/conflicts, … Shall I make a simple (non-contentious) pull request for hier(7) to include /var/db/etcupdate/ and the conflicts/ subdirectory? /var/db/etcupdate/ is present by default with a clean installation of 15.0-CURRENT. I guess that conflicts/ appears (and remains after resolution) only after the first conflict is found. True?
Re: user problems when upgrading to v15
Over the last week or so, stable/13, stable/14 and current have improved. Finally, I can make it through a build and install with a password on the root account and a user in the wheel group without having it fail. Brian On 9/9/2023 9:02 AM, John Baldwin wrote: On 9/2/23 7:11 AM, Dimitry Andric wrote: On 1 Sep 2023, at 03:42, brian whalen wrote: Repeating the entire process: I created a 13.2 vm with 6 cores and 8GB of ram. Ran freebsd-update fetch and install. Ran pkg install git bash ccache open-vm-tools-nox11 Used git clone to get current and ports source files. Edited /etc/make.conf to use ccache Ran make -j6 buildworld && make -j6 kernel I then rebooted in single user mode and did the next steps saving output to a file with > filename. etcupdate -p was pretty uneventful. It did show the below and did not prompt to edit. root@f15:~ # less etcupdatep C /etc/group C /etc/master.passwd This is a problem: the "C" characters mean there were conflicts, and it's indeed very unfortunate that etcupdate does not immediately force you to resolve them. Because now you basically have mangled group and master.passwd files, with conflict markers in them! No, the conflicted files are in /var/db/etcupdate/conflicts, the files in /etc are still the old ones at this point and won't be updated until you run 'etcupdate resolve' to fix them. I suspect what happened here is that Brian chose the 'tf' (theirs-full) option for 'etcupdate resolve' when he really wanted to do 'e' to edit the conflicted version. Immediately after this, you should run "etcupdate resolve", and fix any conflicts that it has found. Note that recently there was a lot of churn due to the removal of $FreeBSD$ keywords, and this almost always creates conflicts in the group and passwd files. For lots of other files in /etc, the conflicts are resolved automatically, but unfortunately not for the files that are essential to log in! make installworld seemed mostly error free though I did see a nonzero status for a man page failed inn the man4 directory. etcupdate -B only showed the below. This was my first build after install. root@f15:~ # less etcupdateB Conflicts remain from previous update, aborting. Yes, that is indeed the problem. You must first resolve conflicts from any previous etcupdate run, before doing anything else. As to why it does not immediately forces you to do so, and delegates this to a separate step, which can easily be forgotten, I have no idea. So that if you are doing scripted upgrades, you don't hang forever in a script. The intention is that after doing a bunch of scripted installworld + etcupdate's on various hosts you can use 'etcupdate status' to see if there are any remaining steps requiring manual intervention. There could be an option to request batched vs interactivate updates perhaps. If I type exit in single user mode to go multi user mode, the local user still works. After a reboot the local user still works. This local user can also sudo as expected. This wasn't the case for the previous build when I first reported this. However, if I run etcupdate resolve it is still presenting /etc/group and /etc/master/passwd as problems. If this is is expected behavior for current then no big deal. I just wasn't sure. The conflicts themselves are expected, alas. But you _must_ resolve them, otherwise you can end up with a mostly-bricked system. No, the conflict markers are not placed in the versions in /etc. However, etucpdate does refuse to do a "new" upgrade until you resolve all the conflicts from your previous upgrade to ensure that conflicted upgrades aren't missed.
Re: user problems when upgrading to v15
On 9/2/23 7:11 AM, Dimitry Andric wrote: On 1 Sep 2023, at 03:42, brian whalen wrote: Repeating the entire process: I created a 13.2 vm with 6 cores and 8GB of ram. Ran freebsd-update fetch and install. Ran pkg install git bash ccache open-vm-tools-nox11 Used git clone to get current and ports source files. Edited /etc/make.conf to use ccache Ran make -j6 buildworld && make -j6 kernel I then rebooted in single user mode and did the next steps saving output to a file with > filename. etcupdate -p was pretty uneventful. It did show the below and did not prompt to edit. root@f15:~ # less etcupdatep C /etc/group C /etc/master.passwd This is a problem: the "C" characters mean there were conflicts, and it's indeed very unfortunate that etcupdate does not immediately force you to resolve them. Because now you basically have mangled group and master.passwd files, with conflict markers in them! No, the conflicted files are in /var/db/etcupdate/conflicts, the files in /etc are still the old ones at this point and won't be updated until you run 'etcupdate resolve' to fix them. I suspect what happened here is that Brian chose the 'tf' (theirs-full) option for 'etcupdate resolve' when he really wanted to do 'e' to edit the conflicted version. Immediately after this, you should run "etcupdate resolve", and fix any conflicts that it has found. Note that recently there was a lot of churn due to the removal of $FreeBSD$ keywords, and this almost always creates conflicts in the group and passwd files. For lots of other files in /etc, the conflicts are resolved automatically, but unfortunately not for the files that are essential to log in! make installworld seemed mostly error free though I did see a nonzero status for a man page failed inn the man4 directory. etcupdate -B only showed the below. This was my first build after install. root@f15:~ # less etcupdateB Conflicts remain from previous update, aborting. Yes, that is indeed the problem. You must first resolve conflicts from any previous etcupdate run, before doing anything else. As to why it does not immediately forces you to do so, and delegates this to a separate step, which can easily be forgotten, I have no idea. So that if you are doing scripted upgrades, you don't hang forever in a script. The intention is that after doing a bunch of scripted installworld + etcupdate's on various hosts you can use 'etcupdate status' to see if there are any remaining steps requiring manual intervention. There could be an option to request batched vs interactivate updates perhaps. If I type exit in single user mode to go multi user mode, the local user still works. After a reboot the local user still works. This local user can also sudo as expected. This wasn't the case for the previous build when I first reported this. However, if I run etcupdate resolve it is still presenting /etc/group and /etc/master/passwd as problems. If this is is expected behavior for current then no big deal. I just wasn't sure. The conflicts themselves are expected, alas. But you _must_ resolve them, otherwise you can end up with a mostly-bricked system. No, the conflict markers are not placed in the versions in /etc. However, etucpdate does refuse to do a "new" upgrade until you resolve all the conflicts from your previous upgrade to ensure that conflicted upgrades aren't missed. -- John Baldwin
Re: user problems when upgrading to v15
On 1 Sep 2023, at 03:42, brian whalen wrote: > > Repeating the entire process: > > I created a 13.2 vm with 6 cores and 8GB of ram. > > Ran freebsd-update fetch and install. > > Ran pkg install git bash ccache open-vm-tools-nox11 > > Used git clone to get current and ports source files. > > Edited /etc/make.conf to use ccache > > Ran make -j6 buildworld && make -j6 kernel > > I then rebooted in single user mode and did the next steps saving output to a > file with > filename. > > etcupdate -p was pretty uneventful. It did show the below and did not prompt > to edit. > > root@f15:~ # less etcupdatep > C /etc/group > C /etc/master.passwd This is a problem: the "C" characters mean there were conflicts, and it's indeed very unfortunate that etcupdate does not immediately force you to resolve them. Because now you basically have mangled group and master.passwd files, with conflict markers in them! Immediately after this, you should run "etcupdate resolve", and fix any conflicts that it has found. Note that recently there was a lot of churn due to the removal of $FreeBSD$ keywords, and this almost always creates conflicts in the group and passwd files. For lots of other files in /etc, the conflicts are resolved automatically, but unfortunately not for the files that are essential to log in! > make installworld seemed mostly error free though I did see a nonzero status > for a man page failed inn the man4 directory. > > etcupdate -B only showed the below. This was my first build after install. > > root@f15:~ # less etcupdateB > Conflicts remain from previous update, aborting. Yes, that is indeed the problem. You must first resolve conflicts from any previous etcupdate run, before doing anything else. As to why it does not immediately forces you to do so, and delegates this to a separate step, which can easily be forgotten, I have no idea. > > If I type exit in single user mode to go multi user mode, the local user > still works. After a reboot the local user still works. This local user can > also sudo as expected. This wasn't the case for the previous build when I > first reported this. However, if I run etcupdate resolve it is still > presenting /etc/group and /etc/master/passwd as problems. > > If this is is expected behavior for current then no big deal. I just wasn't > sure. The conflicts themselves are expected, alas. But you _must_ resolve them, otherwise you can end up with a mostly-bricked system. -Dimitry signature.asc Description: Message signed with OpenPGP
Re: user problems when upgrading to v15
Repeating the entire process: I created a 13.2 vm with 6 cores and 8GB of ram. Ran freebsd-update fetch and install. Ran pkg install git bash ccache open-vm-tools-nox11 Used git clone to get current and ports source files. Edited /etc/make.conf to use ccache Ran make -j6 buildworld && make -j6 kernel I then rebooted in single user mode and did the next steps saving output to a file with > filename. etcupdate -p was pretty uneventful. It did show the below and did not prompt to edit. root@f15:~ # less etcupdatep C /etc/group C /etc/master.passwd make installworld seemed mostly error free though I did see a nonzero status for a man page failed inn the man4 directory. etcupdate -B only showed the below. This was my first build after install. root@f15:~ # less etcupdateB Conflicts remain from previous update, aborting. If I type exit in single user mode to go multi user mode, the local user still works. After a reboot the local user still works. This local user can also sudo as expected. This wasn't the case for the previous build when I first reported this. However, if I run etcupdate resolve it is still presenting /etc/group and /etc/master/passwd as problems. If this is is expected behavior for current then no big deal. I just wasn't sure. Brian On 8/30/2023 7:35 PM, Graham Perrin wrote: On 31/08/2023 03:31, brian whalen wrote: Understood. I guess I was expecting the update process of etcupdate -p && make installworld && etcupdate -B to not whack existing users or delete an existing root user's password. I accepted the remote and then recreated users and reset passwords and am retrying this. BW Thanks. For clarity: did the routine /not/ prompt you to edit the file (in which, you would have seen conflict markers etc.)? On 8/30/2023 7:21 PM, Graham Perrin wrote: On 31/08/2023 03:00, brian whalen wrote: … I ran etcupdate resolve accepting the remote option and saw 2 issues. The root user's password was deleted. The non root user no longer existed. Logically, remote does not include things such as your root user's password.
Re: user problems when upgrading to v15
On 31/08/2023 03:31, brian whalen wrote: Understood. I guess I was expecting the update process of etcupdate -p && make installworld && etcupdate -B to not whack existing users or delete an existing root user's password. I accepted the remote and then recreated users and reset passwords and am retrying this. BW Thanks. For clarity: did the routine /not/ prompt you to edit the file (in which, you would have seen conflict markers etc.)? On 8/30/2023 7:21 PM, Graham Perrin wrote: On 31/08/2023 03:00, brian whalen wrote: … I ran etcupdate resolve accepting the remote option and saw 2 issues. The root user's password was deleted. The non root user no longer existed. Logically, remote does not include things such as your root user's password.
Re: user problems when upgrading to v15
Understood. I guess I was expecting the update process of etcupdate -p && make installworld && etcupdate -B to not whack existing users or delete an existing root user's password. I accepted the remote and then recreated users and reset passwords and am retrying this. BW On 8/30/2023 7:21 PM, Graham Perrin wrote: On 31/08/2023 03:00, brian whalen wrote: … I ran etcupdate resolve accepting the remote option and saw 2 issues. The root user's password was deleted. The non root user no longer existed. Logically, remote does not include things such as your root user's password.
Re: user problems when upgrading to v15
On 31/08/2023 03:00, brian whalen wrote: … I ran etcupdate resolve accepting the remote option and saw 2 issues. The root user's password was deleted. The non root user no longer existed. Logically, remote does not include things such as your root user's password.
Re: user problems when upgrading to v15
I have seen this twice. Once when going from 13.2 to current 14.0 alpha1 and then to 15, and a 2nd time when going from 13.2 to 15. I have a user that is a member of the wheel group. After I upgraded and ran the post reboot commands in single user mode I was alerted to merge conflicts. I ran etcupdate resolve accepting the remote option and saw 2 issues. The root user's password was deleted. The non root user no longer existed.
