Re: I can get zfs snapshot/rollback in a jail to work 99% but it isn't quite 100% working. What am I missing?
> So as I mentioned I’ve able to mail the dataset. It gets mounted upon > starting the jail. It shows up in “zfs list”. If you can see your dataset with `zfs-list(8)` it does not mean that it is mounted. You should check it using `mount -t zfs` or `zfs mount`. > And when I do zfs snapshot on the dataset it appears to create the snapshot > as it shows up in a “zfs list -t snapshot” but the snapdir isn’t visible even > after setting snapdir to visible, and when I rollback using the snapshot it > doesn’t actually rollback. You can create snapshots and run `zfs-rollback(8)` even when your dataset is not mounted and it will not fail. --- I have seen your `rc.conf(5)` from your jail and it sets `zfs_enable` to `YES`. As I mentioned, it doesn't work at startup and maybe it's a bug. When your jail is started you can run `service zfs start` and it will work. That rc script knows it is inside a jail so it just runs `zfs mount -a` and `zfs umount -a` (when running `service zfs stop`). I don't recommend using that rc script to mount your datasets because when you run `service zfs stop` it will try to unmount `/` which is an error. As a workaround use the approach I share with you: mount and unmount your datasets using your `jail.conf(5)` (see the AppJail template example).
Re: I can get zfs snapshot/rollback in a jail to work 99% but it isn't quite 100% working. What am I missing?
So as I mentioned I’ve able to mail the dataset. It gets mounted upon
starting the jail. It shows up in “zfs list”. And when I do zfs snapshot on
the dataset it appears to create the snapshot as it shows up in a “zfs list
-t snapshot” but the snapdir isn’t visible even after setting snapdir to
visible, and when I rollback using the snapshot it doesn’t actually
rollback. I’m so close to this working, something just isn’t right and I
can’t figure out what. I really wish this was written up in the handbook.
“How to jail a dataset from the host” and “how to snapshot and rollback a
jailed dataset”. :) if I figure this I’ll definitely be writing this up.
Chris
On Mon, Nov 6, 2023 at 2:35 PM DtxdF wrote:
> Hi Chris,
>
> Maybe your dataset is not mounted inside the jail. I thought that simply
> enabling `/etc/rc.d/zfs` was fine, but no, it just doesn't work. I don't
> know if this behavior is a bug or something else, but at the moment I don't
> have time to investigate.
>
> I have a similar setup for a jail with a delegated dataset. I use AppJail,
> but the steps are similar to other tools:
>
> ```
> # zfs create -o jailed=on -o mountpoint=/jailed zroot/jailed
> # appjail quick jtest \
> mount_devfs \
> device='include $devfsrules_hide_all' \
> device='include $devfsrules_unhide_basic' \
> device='include $devfsrules_unhide_login' \
> device='path zfs unhide' \
> template=template.conf \
> overwrite=force \
> start
> ```
>
> In AppJail, a template configuration file is similar to `jail.conf(5)`:
>
> ```
> # cat template.conf
> exec.start: "/bin/sh /etc/rc"
> exec.stop: "/bin/sh /etc/rc.shutdown jail"
> allow.mount
> allow.mount.zfs
> enforce_statfs: 1
> exec.poststart: "zfs jail ${name} zroot/jailed"
> exec.poststart+: "appjail cmd jexec ${name} zfs mount zroot/jailed"
> exec.prestop: "appjail cmd jexec ${name} zfs umount zroot/jailed"
> exec.prestop+: "zfs unjail ${name} zroot/jailed"
> ```
>
> As you can see, the dataset is mounted after running `zfs-jail(8)`. The
> steps are similar when the jail is stopped, but the dataset is unmounted
> and `zfs-unjail(8)` is executed.
>
> Inside the jail I can see the mounted datasets:
>
> ```
> # appjail cmd jexec jtest zfs list -r
> NAME USED AVAIL REFER MOUNTPOINT
> zroot 34.1G 249G 96K /zroot
> zroot/jailed96K 249G 96K /jailed
> # appjail cmd jexec jtest mount -t zfs
> zroot/appjail/jails/jtest/jail on / (zfs, local, noatime, nfsv4acls)
> zroot/jailed on /jailed (zfs, local, noatime, nfsv4acls)
> # appjail cmd jexec jtest ls /jailed
> index.txt
> # appjail cmd jexec jtest cat /jailed/index.txt
> Hi!
> ```
>
> And I can use `zfs-rollback(8)` just fine:
>
> ```
> # appjail cmd jexec jtest zfs snapshot zroot/jailed@guard
> # appjail cmd jexec jtest zfs list -t snapshot zroot/jailed
> NAME USED AVAIL REFER MOUNTPOINT
> zroot/jailed@guard 0B - 96K -
> # appjail cmd jexec jtest dd if=/dev/random of=/jailed/index.txt bs=16
> count=1
> 1+0 records in
> 1+0 records out
> 16 bytes transferred in 0.000102 secs (157318 bytes/sec)
> # appjail cmd jexec jtest hd /jailed/index.txt
> a1 26 2a 9c f5 96 7b 81 90 8d ba 36 d6 f9 4d 93
> |.&*...{6..M.|
> 0010
> # appjail cmd jexec jtest zfs list -t snapshot zroot/jailed
> NAME USED AVAIL REFER MOUNTPOINT
> zroot/jailed@guard56K - 96K -
> # appjail cmd jexec jtest zfs rollback zroot/jailed@guard
> # appjail cmd jexec jtest hd /jailed/index.txt
> 48 69 21 0a |Hi!.|
> 0004
> ```
>
> I hope this can help you.
>
>
> ~ DtxdF
>
>
> El 6 de noviembre de 2023 6:07:06 p. m. UTC, Chris Watson <
> bsduni...@gmail.com> escribió:
>
>> I've been trying to get a zfs dataset delegated into a jail (to run PG
>> on), and allow snapshots and rollback to take place inside the jail. I can
>> get the dataset mounted into the jail, I can get zfs to take the snapshot,
>> list the snapshot, but when I rollback or try to ls -la the directory to
>> see the '.zfs' dir it isn't there and the zfs rollback completes but it
>> doesn't actually rollback. I'm so close to getting this to work! I'm just
>> missing *something* in the sauce. When I do the zfs rollback zfs looks like
>> it completes the rollback and goes back to a shell prompt but the files I
>> remove before the rollback are not in the /var/db/postgres/data16 directory
>> nor is ".zfs" shown in ls -la. So something is wonky on my end. I'm so
>> close, it's halfway there, it looks like it takes a snapshot, the snapshot
>> shows up in a zfs list -t snapshot, but it's also not really there. I'm
>> doing something just slightly wrong here. I just cant figure out what I
>> have wrong.
>>
>> Below are the configs:
>> # The jail's config
>> https://bsd.to/P176
>> # zfs list from inside the jail
>> https://bsd.to/mPde
>> # zfs list -t snapshot from inside the jail
>> https://bsd.to/R8dw
>> # ls -la /var/db/postgres/data16
Re: I can get zfs snapshot/rollback in a jail to work 99% but it isn't quite 100% working. What am I missing?
Hi Chris,
Maybe your dataset is not mounted inside the jail. I thought that simply
enabling `/etc/rc.d/zfs` was fine, but no, it just doesn't work. I don't know
if this behavior is a bug or something else, but at the moment I don't have
time to investigate.
I have a similar setup for a jail with a delegated dataset. I use AppJail, but
the steps are similar to other tools:
```
# zfs create -o jailed=on -o mountpoint=/jailed zroot/jailed
# appjail quick jtest \
mount_devfs \
device='include $devfsrules_hide_all' \
device='include $devfsrules_unhide_basic' \
device='include $devfsrules_unhide_login' \
device='path zfs unhide' \
template=template.conf \
overwrite=force \
start
```
In AppJail, a template configuration file is similar to `jail.conf(5)`:
```
# cat template.conf
exec.start: "/bin/sh /etc/rc"
exec.stop: "/bin/sh /etc/rc.shutdown jail"
allow.mount
allow.mount.zfs
enforce_statfs: 1
exec.poststart: "zfs jail ${name} zroot/jailed"
exec.poststart+: "appjail cmd jexec ${name} zfs mount zroot/jailed"
exec.prestop: "appjail cmd jexec ${name} zfs umount zroot/jailed"
exec.prestop+: "zfs unjail ${name} zroot/jailed"
```
As you can see, the dataset is mounted after running `zfs-jail(8)`. The steps
are similar when the jail is stopped, but the dataset is unmounted and
`zfs-unjail(8)` is executed.
Inside the jail I can see the mounted datasets:
```
# appjail cmd jexec jtest zfs list -r
NAME USED AVAIL REFER MOUNTPOINT
zroot 34.1G 249G 96K /zroot
zroot/jailed96K 249G 96K /jailed
# appjail cmd jexec jtest mount -t zfs
zroot/appjail/jails/jtest/jail on / (zfs, local, noatime, nfsv4acls)
zroot/jailed on /jailed (zfs, local, noatime, nfsv4acls)
# appjail cmd jexec jtest ls /jailed
index.txt
# appjail cmd jexec jtest cat /jailed/index.txt
Hi!
```
And I can use `zfs-rollback(8)` just fine:
```
# appjail cmd jexec jtest zfs snapshot zroot/jailed@guard
# appjail cmd jexec jtest zfs list -t snapshot zroot/jailed
NAME USED AVAIL REFER MOUNTPOINT
zroot/jailed@guard 0B - 96K -
# appjail cmd jexec jtest dd if=/dev/random of=/jailed/index.txt bs=16 count=1
1+0 records in
1+0 records out
16 bytes transferred in 0.000102 secs (157318 bytes/sec)
# appjail cmd jexec jtest hd /jailed/index.txt
a1 26 2a 9c f5 96 7b 81 90 8d ba 36 d6 f9 4d 93 |.&*...{6..M.|
0010
# appjail cmd jexec jtest zfs list -t snapshot zroot/jailed
NAME USED AVAIL REFER MOUNTPOINT
zroot/jailed@guard56K - 96K -
# appjail cmd jexec jtest zfs rollback zroot/jailed@guard
# appjail cmd jexec jtest hd /jailed/index.txt
48 69 21 0a |Hi!.|
0004
```
I hope this can help you.
~ DtxdF
El 6 de noviembre de 2023 6:07:06 p. m. UTC, Chris Watson
escribió:
>I've been trying to get a zfs dataset delegated into a jail (to run PG on),
>and allow snapshots and rollback to take place inside the jail. I can get
>the dataset mounted into the jail, I can get zfs to take the snapshot, list
>the snapshot, but when I rollback or try to ls -la the directory to see the
>'.zfs' dir it isn't there and the zfs rollback completes but it doesn't
>actually rollback. I'm so close to getting this to work! I'm just missing
>*something* in the sauce. When I do the zfs rollback zfs looks like it
>completes the rollback and goes back to a shell prompt but the files I
>remove before the rollback are not in the /var/db/postgres/data16 directory
>nor is ".zfs" shown in ls -la. So something is wonky on my end. I'm so
>close, it's halfway there, it looks like it takes a snapshot, the snapshot
>shows up in a zfs list -t snapshot, but it's also not really there. I'm
>doing something just slightly wrong here. I just cant figure out what I
>have wrong.
>
>Below are the configs:
># The jail's config
>https://bsd.to/P176
># zfs list from inside the jail
>https://bsd.to/mPde
># zfs list -t snapshot from inside the jail
>https://bsd.to/R8dw
># ls -la /var/db/postgres/data16 output from inside the jail
>https://bsd.to/1di2
># rc.conf of the jail
>https://bsd.to/JcnH
>
>The jail is running 13.2-P4.
>Using bastillebsd 0.10.20231013 for creation/management.
>
>Thanks!
>Chris
