Re: Remove booting from kernels in raw/qcow2 images in vmd(8)

[Grégoire Jadi]

Dave Voutila  writes:

Hi,

> Any takers?
>
> Here's an updated diff also removes some logic in config.c related to
> checking the value sent by vmctl(8)'s -b flag to see if it's the same as
> the root disk image (-d).

I don't know if this patch was committed already, but there is a mention
of ramdisk in vm.conf(5).

Here is a small patch to replace the example with a boot on disk.

Best,

diff --git usr.sbin/vmd/vm.conf.5 usr.sbin/vmd/vm.conf.5
index e32ab65b16d..d47d4020504 100644
--- usr.sbin/vmd/vm.conf.5
+++ usr.sbin/vmd/vm.conf.5
@@ -82,10 +82,10 @@ Macros are not expanded inside quotes.
 .Pp
 For example:
 .Bd -literal -offset indent
-ramdisk="/bsd.rd"
+disk1="/var/disks/disk1.qcow2"
 vm "vm1.example.com" {
memory 512M
-   boot $ramdisk
+   disk $disk1
 }
 .Ed
 .Sh GLOBAL CONFIGURATION


-- 
gjadi
PGP : AF26 E9C2 A1C8 8D32 A868  4386 1373 5477 2B65 1894

Re: Replace libfuse from base with libfuse from ports

[Grégoire Jadi]

Helg  writes:

Hi,

> I currently only have a port of the 2.x branch of libfuse (attached) but
> will also port the 3.x libfuse once I have user mounting working. I am
> not aware of any ports other than the latest version of sshfs that
> depends on 3.x. In any case, we will need to support libfuse 2.x for
> some time until all file systems are updated upstream to work with 3.x.   
>

I can't comment for the rest of your work, but borgbackup seems to
prefer libfuse 3.x

https://github.com/borgbackup/borg/blob/master/setup.py#L77
> # note for package maintainers: if you package borgbackup for distribution,
> # please (if available) add pyfuse3 (preferably) or llfuse (not maintained 
> any more)
> # as a *requirement*. "borg mount" needs one of them to work.
> # if neither is available, do not require it, most of borgbackup will
> work.

https://pypi.org/project/pyfuse3/
> pyfuse3 is a set of Python 3 bindings for libfuse 3. It provides an
> asynchronous API compatible with Trio and asyncio, and enables you to
> easily write a full-featured Linux filesystem in Python.

Thanks for your work.
Best,

-- 
gjadi
PGP : AF26 E9C2 A1C8 8D32 A868  4386 1373 5477 2B65 1894

Re: ssh_config(5) man page improvement for RemoteCommand

[Grégoire Jadi]

Jason McIntyre  writes:

> On Tue, Jan 05, 2021 at 08:11:51PM +0100, Gr??goire Jadi wrote:
>
>> Hi,
>> 
>> When both RemoteCommand and ssh command are provided, the connection
>> fails with:
>> 
>> Cannot execute command-line and remote command.
>> 
>> Though the message is clear, I was puzzled because I couldn't find any
>> information in the man pages.  This patch adds a note in ssh_config(5).
>> 
>
> hi.
>
> this feels a bit like overkill: as you've said, the error message is
> clear. isn;t that hint enough? it doesn;t seem logical to expect that
> both would work.
>
> on the other hand, i suppose in theory the command in the config
> file could be ignored and the command-line command run, so maybe
> it does have to be said.

Actually, that's what happen with the User option.

Given ssh_config:

Host domain.tld
  User foo

Then ssh b...@domain.tld will ignore the User foo option.


Re-reading the man page of ssh_config(5) it says the following at the
very beginning:

 ssh(1) obtains configuration data from the following sources in the
 following order:

   1.   command-line options
   2.   user's configuration file (~/.ssh/config)
   3.   system-wide configuration file (/etc/ssh/ssh_config)

 For each parameter, the first obtained value will be used.

So, I think that ignoring the RemoteCommand (like the User option) is
more coherent.

> still i think what's there is enough.

Yes, I agree.

Thanks for the feedbacks.

> jmc
>
>> diff --git usr.bin/ssh/ssh_config.5 usr.bin/ssh/ssh_config.5
>> index 1007847662c..fa12092645e 100644
>> --- usr.bin/ssh/ssh_config.5
>> +++ usr.bin/ssh/ssh_config.5
>> @@ -1465,6 +1465,12 @@ Arguments to
>>  accept the tokens described in the
>>  .Sx TOKENS
>>  section.
>> +.Pp
>> +.Cm RemoteCommand
>> +is incompatible with
>> +.Xr ssh 1
>> +.Ar command
>> +argument given on the command line.
>>  .It Cm RemoteForward
>>  Specifies that a TCP port on the remote machine be forwarded over
>>  the secure channel.
>> 
>> Best,
>> 
>> -- 
>> gjadi
>> PGP : AF26 E9C2 A1C8 8D32 A868  4386 1373 5477 2B65 1894
>> 

-- 
gjadi
PGP : AF26 E9C2 A1C8 8D32 A868  4386 1373 5477 2B65 1894

ssh_config(5) man page improvement for RemoteCommand

[Grégoire Jadi]

Hi,

When both RemoteCommand and ssh command are provided, the connection
fails with:

Cannot execute command-line and remote command.

Though the message is clear, I was puzzled because I couldn't find any
information in the man pages.  This patch adds a note in ssh_config(5).

diff --git usr.bin/ssh/ssh_config.5 usr.bin/ssh/ssh_config.5
index 1007847662c..fa12092645e 100644
--- usr.bin/ssh/ssh_config.5
+++ usr.bin/ssh/ssh_config.5
@@ -1465,6 +1465,12 @@ Arguments to
 accept the tokens described in the
 .Sx TOKENS
 section.
+.Pp
+.Cm RemoteCommand
+is incompatible with
+.Xr ssh 1
+.Ar command
+argument given on the command line.
 .It Cm RemoteForward
 Specifies that a TCP port on the remote machine be forwarded over
 the secure channel.

Best,

-- 
gjadi
PGP : AF26 E9C2 A1C8 8D32 A868  4386 1373 5477 2B65 1894

Re: Should rm(1) -Pf change file permission?

[Grégoire Jadi]

Ingo Schwarze  writes:

Hello Ingo,

> Hi Gregoire,
>
> Gregoire Jadi wrote on Tue, Apr 17, 2018 at 11:44:41AM +0200:
>> Ingo Schwarze  writes:
>
>>> Feedback is welcome both on the general idea and on the specific
>>> implementation.
>
> The result of that feeback was "we don't want it, the whole
> idea of changing this is pointless and dangerous", so i deleted
> my patch quite some time ago.

Ok.

> The conversation resulted in a clarification in the manual page,
> though.

Thanks for the clarification. It's clear now it's a "best effort".

>> Thank you for working on this.  I have tested your patch
>> and both `rm -P` and `rm -Pf` work as expected.
>
> Sorry, it won't be committed, it was definitely rejected.
> Besides, it wasn't correct, there were race conditions.
> Working on improving it would be a waste of time.

Ok. So, this issue is closed. :)

Best,

> Yours,
>   Ingo

Re: Should rm(1) -Pf change file permission?

[Grégoire Jadi]

Ingo Schwarze  writes:

Hi,

> Hi,
>
> Gregoire Jadi wrote on Fri, Mar 30, 2018 at 06:07:42PM +0200:
>
>> While working on a port of keyringer, I observed the following behavior
>> of rm(1) with the -P option: if the file does not have write permission,
>> the file is removed without being overwritten.
>> 
>> This is not the same behavior as shred(1) (from sysutils/coreutils) which
>> do not remove the file if it cannot be overwritten. If the -f option is
>> used with shred(1), the permission of the file are changed to allow
>> writing if necessary.
>> 
>> Is the current behavior desired? (need to update the manpage?)
>> 
>> Or should `rm -P` and `rm -Pf` have the same behavior as shred. That is:
>> `rm -P` should fail without removing the file if the file cannot be
>> overwritten and `rm -Pf` should change the permission and overwrite
>> file.
>
> That makes more sense to me than the current behaviour.
>
> In addition, when running with -P and without -f and the user
> interactive confirms that the missing write protection shall be
> overriden, changing the permissions seems right to me, too.
>
> Here is a patch implementing that.
>
> Lightly tested by hand by running various combinations of rm(1)
>  * with and without -P
>  * with and without -f
>  * on files with different permissions
>
> Feedback is welcome both on the general idea and on the specific
> implementation.

Thank you for working on this. I have tested your patch and both `rm -P`
and `rm -Pf` work as expected.

I have no opinion on your implementation but it does solve the problem
and make rm(1) more consistent with gshred(1).

Best,

> Note that the return value of rm_overwrite() is currently unused.
> So we are free to change it to reflect success or failure, such
> that deleting the file can be skipped in case of failure even
> after trying to relax the permissions.  Also note that checking
> fflag in rw_overwrite is not needed (and would even be wrong)
> because the program doesn't get there in the first place unless
> fflag was given or deletion was interactively confirmed.
>
> Yours,
>   Ingo
>
> Index: rm.c
> ===
> RCS file: /cvs/src/bin/rm/rm.c,v
> retrieving revision 1.42
> diff -u -p -r1.42 rm.c
> --- rm.c  27 Jun 2017 21:49:47 -  1.42
> +++ rm.c  31 Mar 2018 17:02:37 -
> @@ -103,7 +103,7 @@ main(int argc, char *argv[])
>   argv += optind;
>
>   if (Pflag) {
> - if (pledge("stdio rpath wpath cpath getpw", NULL) == -1)
> + if (pledge("stdio rpath wpath cpath fattr getpw", NULL) == -1)
>   err(1, "pledge");
>   } else {
>   if (pledge("stdio rpath cpath getpw", NULL) == -1)
> @@ -213,9 +213,9 @@ rm_tree(char **argv)
>
>   case FTS_F:
>   case FTS_NSOK:
> - if (Pflag)
> - rm_overwrite(p->fts_accpath, p->fts_info ==
> - FTS_NSOK ? NULL : p->fts_statp);
> + if (Pflag && !rm_overwrite(p->fts_accpath,
> + p->fts_info == FTS_NSOK ? NULL : p->fts_statp))
> + continue;
>   /* FALLTHROUGH */
>   default:
>   if (!unlink(p->fts_accpath) ||
> @@ -264,8 +264,8 @@ rm_file(char **argv)
>   else if (S_ISDIR(sb.st_mode))
>   rval = rmdir(f);
>   else {
> - if (Pflag)
> - rm_overwrite(f, );
> + if (Pflag && !rm_overwrite(f, ))
> + continue;
>   rval = unlink(f);
>   }
>   if (rval && (!fflag || errno != ENOENT)) {
> @@ -308,9 +308,12 @@ rm_overwrite(char *file, struct stat *sb
>   if (sbp->st_nlink > 1) {
>   warnx("%s (inode %llu): not overwritten due to multiple links",
>   file, (unsigned long long)sbp->st_ino);
> - return (0);
> + return (1);
>   }
> - if ((fd = open(file, O_WRONLY|O_NONBLOCK|O_NOFOLLOW, 0)) == -1)
> + if ((fd = open(file, O_WRONLY|O_NONBLOCK|O_NOFOLLOW, 0)) == -1 &&
> + (errno != EACCES ||
> +  chmod(file, sb.st_mode | S_IWUSR) == -1 ||
> +  (fd = open(file, O_WRONLY|O_NONBLOCK|O_NOFOLLOW, 0)) == -1))
>   goto err;
>   if (fstat(fd, ))
>   goto err;

Should rm(1) -Pf change file permission?

[Grégoire Jadi]

Hello,

While working on a port of keyringer, I observed the following behavior
of rm(1) with the -P option: if the file does not have write permission,
the file is removed without being overwritten.

This is not the same behavior as shred(1) (from sysutils/coreutils) which do
not remove the file if it cannot be overwritten. If the -f option is
used with shred(1), the permission of the file are changed to allow
writing if necessary.

Is the current behavior desired? (need to update the manpage?)

Or should `rm -P` and `rm -Pf` have the same behavior as shred. That is:
`rm -P` should fail without removing the file if the file cannot be
overwritten and `rm -Pf` should change the permission and overwrite
file.


Here are excerpts from rm(1) and shred(1) manpages:
rm(1)
 -f  Attempt to remove the files without prompting for confirmation,
 regardless of the file's permissions.  If the file does not
 exist, do not display a diagnostic message or modify the exit
 status to reflect an error.  The -f option overrides any previous
 -i options.

 -P  Overwrite regular files before deleting them.  Files are
 overwritten once with a random pattern.  Files with multiple
 links will be unlinked but not overwritten.

shred(1)
   -f, --force
  change permissions to allow writing if necessary

And here is a small test to demonstrate this behavior:

$ echo bar > foo
$ chmod -w foo
$ rm -P foo
override r--r--r--  daimrod/wheel for foo? y
rm: foo: Permission denied
$ ls -l foo
ls: foo: No such file or directory

$ echo bar > foo
$ chmod -w foo
$ rm -Pf foo
rm: foo: Permission denied
$ ls -l foo
ls: foo: No such file or directory

$ echo bar > foo
$ chmod -w foo
$ gshred -u foo
gshred: foo: failed to open for writing: Permission denied
$ ls -l foo
-r--r--r--  1 daimrod  wheel  4 Mar 30 17:45 foo
$ gshred -uf foo
$ ls -l foo
ls: foo: No such file or directory

Best,

Re: Properly sync filesystems during {suspend,hibernate}/resume

[Grégoire Jadi]

"Theo de Raadt"  writes:

>> > I'd appreciate reports about how well it delivers and if any new
>> > problems show up.
>> 
>> Tested on amd64 with softdep and softraid (CRYPTO).
>> Works as expected with suspend and hibernate.
>
> Thanks.
>
>> However, in both cases there is a warning in dmesg during the reboot :
>> > softraid0 sd1 was not shutdown properly
>
> But that is not new.  Take an older kernel, and perform the test of
> suspend followed by removing the power.
>
> Previous to this diff there was no code to gaurantee filesystem sync,
> and no code to syncronize softraid.  I've written the filesystem sync
> code, but I haven't written any softraid syncronization code.

Ok, sorry for the false positive.

Best.

Re: Properly sync filesystems during {suspend,hibernate}/resume

[Grégoire Jadi]

Theo de Raadt  writes:

> After recent reboot related improvements, I have some familiarity with
> tricking the vnode layer into a stable state.
>
> This should settle filesystems "clean" onto disk.  If one does a
> suspend and then runs out of battery.. or experiences a crash or out
> of battery during a hibernate, the filesystems will now be in a clean
> state and not require an fsck.
>
> After the screen blanks, there may be 1-2 seconds more IO than before.
> Those weren't hitting the platter, but remained in the vfs softlayer
> through to resume.  The new situation is much better.
>
> I have wanted to solve this problem for years..
>
> There may be problems with softraid again, we'll need to see.  I
> haven't tested softdep but the correct syncronization functions appear
> to be called.  USB sticks get detached during the suspend phase, and
> are as good as before.
>
> This isn't quite finished, and I have a few more tricks up my sleeve,
> and the "vfs_stalling" global will probably become a parameter for
> VFS_SYNC()...
>
> I'd appreciate reports about how well it delivers and if any new
> problems show up.

Tested on amd64 with softdep and softraid (CRYPTO).
Works as expected with suspend and hibernate.

However, in both cases there is a warning in dmesg during the reboot :
> softraid0 sd1 was not shutdown properly

It probably needs 'bioctl -d sd1' or something similar.

Best,

> Index: kern/vfs_subr.c
> ===
> RCS file: /cvs/src/sys/kern/vfs_subr.c,v
> retrieving revision 1.265
> diff -u -p -u -r1.265 vfs_subr.c
> --- kern/vfs_subr.c   14 Dec 2017 20:23:15 -  1.265
> +++ kern/vfs_subr.c   16 Dec 2017 23:50:21 -
> @@ -1583,6 +1583,42 @@ vaccess(enum vtype type, mode_t file_mod
>   return (file_mode & mask) == mask ? 0 : EACCES;
>  }
>
> +int vfs_stalling;
> +
> +int
> +vfs_stallmp(struct mount *mp, struct proc *p, int stall)
> +{
> + int error;
> +
> + if (stall) {
> + error = vfs_busy(mp, VB_WRITE|VB_WAIT);
> + if (error) {
> + printf("%s: busy\n", mp->mnt_stat.f_mntonname);
> + return (error);
> + }
> + uvm_vnp_sync(mp);
> + vfs_stalling = stall;
> + error = VFS_SYNC(mp, MNT_WAIT, p->p_ucred, p);
> + if (error) {
> + printf("%s: failed to sync\n", 
> mp->mnt_stat.f_mntonname);
> + vfs_unbusy(mp);
> + return (error);
> + }
> + } else {
> + vfs_unbusy(mp);
> + }
> + return (0);
> +}
> +
> +void
> +vfs_stall(struct proc *p, int stall)
> +{
> + struct mount *mp, *nmp;
> +
> + TAILQ_FOREACH_REVERSE_SAFE(mp, , mntlist, mnt_list, nmp)
> + (void) vfs_stallmp(mp, p, stall);
> +}
> +
>  int
>  vfs_readonly(struct mount *mp, struct proc *p)
>  {
> @@ -1627,10 +1663,8 @@ vfs_rofs(struct proc *p)
>  {
>   struct mount *mp, *nmp;
>
> - TAILQ_FOREACH_REVERSE_SAFE(mp, , mntlist, mnt_list, nmp) {
> - /* XXX Here is a race, the next pointer is not locked. */
> + TAILQ_FOREACH_REVERSE_SAFE(mp, , mntlist, mnt_list, nmp)
>   (void) vfs_readonly(mp, p);
> - }
>  }
>
>  /*
> Index: ufs/ffs/ffs_vfsops.c
> ===
> RCS file: /cvs/src/sys/ufs/ffs/ffs_vfsops.c,v
> retrieving revision 1.170
> diff -u -p -u -r1.170 ffs_vfsops.c
> --- ufs/ffs/ffs_vfsops.c  14 Dec 2017 20:20:38 -  1.170
> +++ ufs/ffs/ffs_vfsops.c  17 Dec 2017 03:03:06 -
> @@ -1145,7 +1145,8 @@ struct ffs_sync_args {
>  };
>
>  int
> -ffs_sync_vnode(struct vnode *vp, void *arg) {
> +ffs_sync_vnode(struct vnode *vp, void *arg)
> +{
>   struct ffs_sync_args *fsa = arg;
>   struct inode *ip;
>   int error;
> @@ -1193,8 +1194,9 @@ ffs_sync(struct mount *mp, int waitfor, 
>  {
>   struct ufsmount *ump = VFSTOUFS(mp);
>   struct fs *fs;
> - int error, allerror = 0, count;
> + int error, allerror = 0, count, clean, fmod;
>   struct ffs_sync_args fsa;
> + extern int vfs_stalling;
>
>   fs = ump->um_fs;
>   /*
> @@ -1240,12 +1242,23 @@ ffs_sync(struct mount *mp, int waitfor, 
>   VOP_UNLOCK(ump->um_devvp, p);
>   }
>   qsync(mp);
> +
>   /*
>* Write back modified superblock.
>*/
> -
> + clean = fs->fs_clean;
> + fmod = fs->fs_fmod;
> + if (vfs_stalling && (fs->fs_ronly == 0)) {
> + fs->fs_fmod = 1;
> + fs->fs_clean = (fs->fs_flags & FS_UNCLEAN) ? 0 : 1;
> +//   printf("%s clean %d fmod %d\n",
> +//   mp->mnt_stat.f_mntonname, fs->fs_clean,
> +//   fs->fs_fmod);
> + }
>   if (fs->fs_fmod != 0 && (error = ffs_sbupdate(ump, waitfor)) != 0)
>   allerror = error;
> + fs->fs_clean = clean;
> + fs->fs_fmod = fmod;
>
>   return