Re: ZFS and Jail :: nullfs mount :: nothing visible from host :: solved [partial]

[Miroslav Lachman]

Alexander Leidinger wrote on 2016/12/19 20:54:

Quoting Miroslav Lachman <000.f...@quip.cz> (from Mon, 19 Dec 2016
18:57:39 +0100):


Alexander Leidinger wrote on 2016/12/19 17:56:


Quoting Miroslav Lachman <000.f...@quip.cz> (from Sun, 18 Dec 2016
13:20:31 +0100):



I don't expect it to be in the docs. I try to come up with something for
the man page for zfs (for the "attach to jail" part), but anyone shall
feel free to beat me with this.

Anyone with an idea where in the jail man page we should add something
too (I only had a look at the zfs man page when this issue came up)?


It would be nice to have this mentioned in zfs(8) man page (that user
in jail cannot manage jail's root dataset but can manage some
sub-dataset not required to boot the jail)


What about this? Better wording welcome.
---snip---
Index: zfs.8
===
--- zfs.8   (Revision 298108)
+++ zfs.8   (Arbeitskopie)
@@ -450,8 +450,11 @@
  dataset can be attached to a jail by using the
  .Qq Nm Cm jail
  subcommand. You cannot attach a dataset to one jail and the children
of the
-same dataset to another jails. To allow management of the dataset from
within
-a jail, the
+same dataset to another jails. You can also not attach the root file
system
+of the jail or any dataset which needs to be mounted before the zfs rc
script
+is run inside the jail, as it would be attached unmounted until it is
+mounted from the rc script inside the jail. To allow management of the
+dataset from within a jail, the
  .Sy jailed
  property has to be set and the jail needs access to the
  .Pa /dev/zfs
---snip---


And there can be some useful example in jail(8) man page in EXAMPLES.
There is section "Jails and File Systems" and there can be new section
"Manage ZFS from within jail" with basic notes about required jail
params, zfs set jailed property and example "hierarchy". (and warning
about gotchas with jailed=0 on jail's root directory)


Are you willing to come up with some text-only version/draft/outline for
this one?


I am not good at English but I will try something.

Thank you!

Miroslav Lachman


___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: ZFS and Jail :: nullfs mount :: nothing visible from host :: solved [partial]

[Alexander Leidinger]

Quoting Miroslav Lachman <000.f...@quip.cz> (from Mon, 19 Dec 2016  
18:57:39 +0100):



Alexander Leidinger wrote on 2016/12/19 17:56:


Quoting Miroslav Lachman <000.f...@quip.cz> (from Sun, 18 Dec 2016
13:20:31 +0100):

Alexander Leidinger wrote on 2016/12/17 19:59:

Quoting SK  (from Fri, 16 Dec 2016 14:02:20



Correct.

You need the data in the root of the jail to boot, if you then attribute
this dataset to the jail, it will vanish until "zfs mount -a" is run (rc
script inside the jail). As it will vanish during the boot of the jail
(if added automatically), the rc script to mount all datasets can not be
found.


[...]


I think what you are trying to tell here is, unless and until that
"vanished" dataset is put to use (mounted) from inside the jail, it
will remain vanished/unusable from the host itself; however, once that
dataset is put to use, the host system should be able to "see" and
maybe even work on that dataset. Could you please confirm if I
understood you correctly?


Correct.

A sub-dataset which is not needed to boot, or a dataset not within the
subtree of the jail (and not needed to boot) can be used.


Thank you for this information! If it is somewhere in the docs it is
well hidden to me :)


I don't expect it to be in the docs. I try to come up with something for
the man page for zfs (for the "attach to jail" part), but anyone shall
feel free to beat me with this.

Anyone with an idea where in the jail man page we should add something
too (I only had a look at the zfs man page when this issue came up)?


It would be nice to have this mentioned in zfs(8) man page (that  
user in jail cannot manage jail's root dataset but can manage some  
sub-dataset not required to boot the jail)


What about this? Better wording welcome.
---snip---
Index: zfs.8
===
--- zfs.8   (Revision 298108)
+++ zfs.8   (Arbeitskopie)
@@ -450,8 +450,11 @@
 dataset can be attached to a jail by using the
 .Qq Nm Cm jail
 subcommand. You cannot attach a dataset to one jail and the children of the
-same dataset to another jails. To allow management of the dataset from within
-a jail, the
+same dataset to another jails. You can also not attach the root file system
+of the jail or any dataset which needs to be mounted before the zfs rc script
+is run inside the jail, as it would be attached unmounted until it is
+mounted from the rc script inside the jail. To allow management of the
+dataset from within a jail, the
 .Sy jailed
 property has to be set and the jail needs access to the
 .Pa /dev/zfs
---snip---

And there can be some useful example in jail(8) man page in  
EXAMPLES. There is section "Jails and File Systems" and there can be  
new section "Manage ZFS from within jail" with basic notes about  
required jail params, zfs set jailed property and example  
"hierarchy". (and warning about gotchas with jailed=0 on jail's root  
directory)


Are you willing to come up with some text-only version/draft/outline  
for this one?


Bye,
Alexander.
--
http://www.Leidinger.net alexan...@leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.orgnetch...@freebsd.org  : PGP 0x8F31830F9F2772BF


pgpiPSC9kMRZ8.pgp
Description: Digitale PGP-Signatur

Re: ZFS and Jail :: nullfs mount :: nothing visible from host :: solved [partial]

[Miroslav Lachman]

Alexander Leidinger wrote on 2016/12/19 17:56:


Quoting Miroslav Lachman <000.f...@quip.cz> (from Sun, 18 Dec 2016
13:20:31 +0100):

Alexander Leidinger wrote on 2016/12/17 19:59:

Quoting SK  (from Fri, 16 Dec 2016 14:02:20



Correct.

You need the data in the root of the jail to boot, if you then attribute
this dataset to the jail, it will vanish until "zfs mount -a" is run (rc
script inside the jail). As it will vanish during the boot of the jail
(if added automatically), the rc script to mount all datasets can not be
found.


[...]


I think what you are trying to tell here is, unless and until that
"vanished" dataset is put to use (mounted) from inside the jail, it
will remain vanished/unusable from the host itself; however, once that
dataset is put to use, the host system should be able to "see" and
maybe even work on that dataset. Could you please confirm if I
understood you correctly?


Correct.

A sub-dataset which is not needed to boot, or a dataset not within the
subtree of the jail (and not needed to boot) can be used.


Thank you for this information! If it is somewhere in the docs it is
well hidden to me :)


I don't expect it to be in the docs. I try to come up with something for
the man page for zfs (for the "attach to jail" part), but anyone shall
feel free to beat me with this.

Anyone with an idea where in the jail man page we should add something
too (I only had a look at the zfs man page when this issue came up)?


It would be nice to have this mentioned in zfs(8) man page (that user in 
jail cannot manage jail's root dataset but can manage some sub-dataset 
not required to boot the jail)


And there can be some useful example in jail(8) man page in EXAMPLES. 
There is section "Jails and File Systems" and there can be new section 
"Manage ZFS from within jail" with basic notes about required jail 
params, zfs set jailed property and example "hierarchy". (and warning 
about gotchas with jailed=0 on jail's root directory)


Miroslav Lachman
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: ZFS and Jail :: nullfs mount :: nothing visible from host :: solved [partial]

[Alexander Leidinger]

Quoting Miroslav Lachman <000.f...@quip.cz> (from Sun, 18 Dec 2016  
13:20:31 +0100):

Alexander Leidinger wrote on 2016/12/17 19:59:

Quoting SK  (from Fri, 16 Dec 2016 14:02:20 +):



If I understand you correctly, what you are suggesting is, the dataset
used by the jail itself for its root/base cannot be "worked on" from
within the jail, but if I define a different dataset (under the same
branch below the jail dataset), and attribute it to the jail, then I
can manipulate that "other" dataset. Could you please confirm if I
understood it correctly?


Correct.

You need the data in the root of the jail to boot, if you then attribute
this dataset to the jail, it will vanish until "zfs mount -a" is run (rc
script inside the jail). As it will vanish during the boot of the jail
(if added automatically), the rc script to mount all datasets can not be
found.


[...]


I think what you are trying to tell here is, unless and until that
"vanished" dataset is put to use (mounted) from inside the jail, it
will remain vanished/unusable from the host itself; however, once that
dataset is put to use, the host system should be able to "see" and
maybe even work on that dataset. Could you please confirm if I
understood you correctly?


Correct.

A sub-dataset which is not needed to boot, or a dataset not within the
subtree of the jail (and not needed to boot) can be used.


Thank you for this information! If it is somewhere in the docs it is  
well hidden to me :)


I don't expect it to be in the docs. I try to come up with something  
for the man page for zfs (for the "attach to jail" part), but anyone  
shall feel free to beat me with this.


Anyone with an idea where in the jail man page we should add something  
too (I only had a look at the zfs man page when this issue came up)?


Bye,
Alexander.

--
http://www.Leidinger.net alexan...@leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.orgnetch...@freebsd.org  : PGP 0x8F31830F9F2772BF


pgpYU_3k8pU6p.pgp
Description: Digitale PGP-Signatur

Re: ZFS and Jail :: nullfs mount :: nothing visible from host :: solved [partial]

[Miroslav Lachman]

Alexander Leidinger wrote on 2016/12/17 19:59:

Quoting SK  (from Fri, 16 Dec 2016 14:02:20 +):



If I understand you correctly, what you are suggesting is, the dataset
used by the jail itself for its root/base cannot be "worked on" from
within the jail, but if I define a different dataset (under the same
branch below the jail dataset), and attribute it to the jail, then I
can manipulate that "other" dataset. Could you please confirm if I
understood it correctly?


Correct.

You need the data in the root of the jail to boot, if you then attribute
this dataset to the jail, it will vanish until "zfs mount -a" is run (rc
script inside the jail). As it will vanish during the boot of the jail
(if added automatically), the rc script to mount all datasets can not be
found.


[...]


I think what you are trying to tell here is, unless and until that
"vanished" dataset is put to use (mounted) from inside the jail, it
will remain vanished/unusable from the host itself; however, once that
dataset is put to use, the host system should be able to "see" and
maybe even work on that dataset. Could you please confirm if I
understood you correctly?


Correct.

A sub-dataset which is not needed to boot, or a dataset not within the
subtree of the jail (and not needed to boot) can be used.


Thank you for this information! If it is somewhere in the docs it is 
well hidden to me :)


Miroslav Lachman

___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: ZFS and Jail :: nullfs mount :: nothing visible from host :: solved [partial]

[Alexander Leidinger]

Quoting SK  (from Fri, 16 Dec 2016 14:02:20 +):


On 16/12/2016 13:15, Alexander Leidinger wrote:


For one of the filesystems I have set "zfs allow" permissions, but  
just that a specific user in the jail can do something on those FS  
without the need to switch to root. So as long as you try to do a  
zfs create/snapshot with an user with UID 0 inside the jail, the  
"zfs allow" part doesn't come into play.


So assume space/jails/xyz.leidinger.net/ to be the dataset which  
contains the root of the jail but is not attached/attributed to the  
jail itself. space/jails/xyz.leidinger.net/data with  
mountpoint=none to be attributed ("zfs jail xyz  
space/jails/xyz.leidinger.net/data") to the jail (similar to the  
"space/something" in the ezjail config above, I have some  
iocage-managed jails were this works). In this case you should be  
able to do from inside the jail "zfs create -o mpuntpoint=/mnt  
space/jails/xyz.leidinger.net/data/test".


hmmm, I'm slightly confused at this point. Let me see if I can  
clarify that in my brain


If I understand you correctly, what you are suggesting is, the  
dataset used by the jail itself for its root/base cannot be "worked  
on" from within the jail, but if I define a different dataset (under  
the same branch below the jail dataset), and attribute it to the  
jail, then I can manipulate that "other" dataset. Could you please  
confirm if I understood it correctly?


Correct.

You need the data in the root of the jail to boot, if you then  
attribute this dataset to the jail, it will vanish until "zfs mount  
-a" is run (rc script inside the jail). As it will vanish during the  
boot of the jail (if added automatically), the rc script to mount all  
datasets can not be found.


And now to everyone, I am still confused about zfs set jailed=on.  
As I mentioned on my previous emails, as soon as I do that, the  
dataset vanishes from the host system (as I understand, that is  
expected behaviour). Then the jail fails as it is unable to mount  
/dev, /proc


From the zfs man page:
---snip---
After a dataset is attached to a jail and the jailed property is set, a
jailed file system cannot be mounted outside the jail, since the jail
administrator might have set the mount point to an unacceptable value.
---snip---

So yes, it is expected that it "vanishes", but it should be visible  
from the parent host at the place inside the jail FS subtree were  
it is mounted there (after setting the mountpoint of the dataset).


I think what you are trying to tell here is, unless and until that  
"vanished" dataset is put to use (mounted) from inside the jail, it  
will remain vanished/unusable from the host itself; however, once  
that dataset is put to use, the host system should be able to "see"  
and maybe even work on that dataset. Could you please confirm if I  
understood you correctly?


Correct.

A sub-dataset which is not needed to boot, or a dataset not within the  
subtree of the jail (and not needed to boot) can be used.


Bye,
Alexander.

--
http://www.Leidinger.net alexan...@leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.orgnetch...@freebsd.org  : PGP 0x8F31830F9F2772BF


pgp_F6E8Xf12a.pgp
Description: Digitale PGP-Signatur

Re: ZFS and Jail :: nullfs mount :: nothing visible from host :: solved [partial]

[SK]

On 16/12/2016 13:15, Alexander Leidinger wrote:
Quoting SK  (from Mon, 12 Dec 2016 17:13:27 
+):


b) Alexander, I am still not able to do snapshot or any other action 
from within my jail. My understanding is that you are using ezjail, 
which might be doing something that my regular jail creation is 
ommitting. If you do not mind sharing your configuration steps, I can 
try to reproduce it at this end. If it is exactly as it is on the 
site you pointed to earlier, please let me know, I will follow that 
verbatim (even though I do not remember seeing anything there that I 
have not tried already, but I might be mistaken).


Do you use quotas on the datasets you want to add to the jail? If yes, 
try without. The man-page of zfs tells that this value can not be 
changed (but from the wording I would expect hat an already set value 
should work).



Hi Alexander, short answer, NO, I have not used quota yet

ezjail is just a shell script which simplifies some things with jails. 
For a particular jail where I can manage the datasets which are handed 
over to the jail I have those settings in ezjail which correspond to 
the settings you can specify in jail.conf:

---snip---
export jail_xyz_leidinger_net_devfs_ruleset="17"
export jail_xyz_leidinger_net_zfs_datasets="space/something"
export jail_xyz_leidinger_net_parameters="allow.mount allow.mount.zfs 
enforce_statfs=1"

---snip---
Check if you have allow.mount and allow.mount.zfs for the jails in 
question.



Yes, those are in place.

Note, "space/something" is not the root of the jail, it's a seperate 
dataset. Do not add the root of the jail as a dataset. Example bellow.


I think that might have been the cause for it to not work for me. I had 
space/something/jailRoot, and defined that as the dataset for the jail



devfs.rules part:
---snip---
[devfsrules_unhide_zfs=12]
add path zfs unhide

[devfsrules_jail_withzfs=17]
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add include $devfsrules_unhide_zfs
---snip---

The rc.conf inside this jail:
---snip---
zfs_enable="YES"
---snip---


These are all in place as expected.

For one of the filesystems I have set "zfs allow" permissions, but 
just that a specific user in the jail can do something on those FS 
without the need to switch to root. So as long as you try to do a zfs 
create/snapshot with an user with UID 0 inside the jail, the "zfs 
allow" part doesn't come into play.


So assume space/jails/xyz.leidinger.net/ to be the dataset which 
contains the root of the jail but is not attached/attributed to the 
jail itself. space/jails/xyz.leidinger.net/data with mountpoint=none 
to be attributed ("zfs jail xyz space/jails/xyz.leidinger.net/data") 
to the jail (similar to the "space/something" in the ezjail config 
above, I have some iocage-managed jails were this works). In this case 
you should be able to do from inside the jail "zfs create -o 
mpuntpoint=/mnt space/jails/xyz.leidinger.net/data/test".


hmmm, I'm slightly confused at this point. Let me see if I can clarify 
that in my brain


If I understand you correctly, what you are suggesting is, the dataset 
used by the jail itself for its root/base cannot be "worked on" from 
within the jail, but if I define a different dataset (under the same 
branch below the jail dataset), and attribute it to the jail, then I can 
manipulate that "other" dataset. Could you please confirm if I 
understood it correctly?


And now to everyone, I am still confused about zfs set jailed=on. As 
I mentioned on my previous emails, as soon as I do that, the dataset 
vanishes from the host system (as I understand, that is expected 
behaviour). Then the jail fails as it is unable to mount /dev, /proc


From the zfs man page:
---snip---
 After a dataset is attached to a jail and the jailed property is 
set, a
 jailed file system cannot be mounted outside the jail, since the 
jail
 administrator might have set the mount point to an unacceptable 
value.

---snip---

So yes, it is expected that it "vanishes", but it should be visible 
from the parent host at the place inside the jail FS subtree were it 
is mounted there (after setting the mountpoint of the dataset).


I think what you are trying to tell here is, unless and until that 
"vanished" dataset is put to use (mounted) from inside the jail, it will 
remain vanished/unusable from the host itself; however, once that 
dataset is put to use, the host system should be able to "see" and maybe 
even work on that dataset. Could you please confirm if I understood you 
correctly?


and so on. I have to change jail.conf and comment out mount.devfs and 
mount.procfs -- but than in turn makes /dev/zfs unavaulable and I 
cannot do anything from inside the jail.


Could it be that you try to attribute the root of the jail as a 
dataset into the jail?


Yes, I did. But now that I have a few more slightly different 
explanation from you, once 

Re: ZFS and Jail :: nullfs mount :: nothing visible from host :: solved [partial]

[SK]

On 09/12/2016 13:36, Miroslav Lachman wrote:


My last idea - put zfs_enable="YES" in jails /etc/rc.conf.

Maybe the dataset is not mounted if has property jailed=on (I don't 
know I didn't test it yet)

Good evening Miroslav, good evening Alexander

Thank you both for your support in this matter. I have completed (I 
think) my tests with the test box and have concluded as following


a) Miroslav, you were correct, I could only see from the root of the 
dataset to the dataset itself, all other dataset that are not part of 
this branch is invisible from within the jail. This serves my purpose, 
so I am content (to some extent). The explanation about enforce_statfs 
was really helpful -- I think that was one thing I was missing (cannot 
confirm, but I believe that is what the error was on my part)


b) Alexander, I am still not able to do snapshot or any other action 
from within my jail. My understanding is that you are using ezjail, 
which might be doing something that my regular jail creation is 
ommitting. If you do not mind sharing your configuration steps, I can 
try to reproduce it at this end. If it is exactly as it is on the site 
you pointed to earlier, please let me know, I will follow that verbatim 
(even though I do not remember seeing anything there that I have not 
tried already, but I might be mistaken).


And now to everyone, I am still confused about zfs set jailed=on. As I 
mentioned on my previous emails, as soon as I do that, the dataset 
vanishes from the host system (as I understand, that is expected 
behaviour). Then the jail fails as it is unable to mount /dev, /proc and 
so on. I have to change jail.conf and comment out mount.devfs and 
mount.procfs -- but than in turn makes /dev/zfs unavaulable and I cannot 
do anything from inside the jail.


I do not need it now, given that I am happy with the current situaion, 
but am curious to know how that zfs parameter works and how I can make 
it work, hence "solved" is "partial" in the subject line.


Thanks to you both for your continuous support and suggestions, it is 
very much apprecaited.


Best regards
SK


___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"