Re: What's going on with vnets and epairs w/ addresses?

[Bjoern A. Zeeb]

t before returning.
- If there are no jailed PCBs in a hash chain in_pcblookup_hash_locked()
 always scans the whole chain.
- If we match only one PCB in a lookup, we can probably(?) return that
 PCB without dereferencing the cred pointer at all.  If not, then the
 scan only has to keep track of a fixed number of PCBs before picking
 which one to return.  So it looks like we can perform a lockless scan
 and keep track of matches on the stack, then lock the matched PCBs and
 perform prison checks if necessary, without making the common case
 more expensive.

In fact there is a parallel thread on freebsd-jail which reports that
this inp_cred access is a source of frequent cache misses.  I was
surprised to see that the scan calls prison_flag() before even checking
the PCB's local address.  So if the hash chain is large then we're
potentially performing a lot of unnecessary memory accesses (though
presumably it's common for most of the PCBs to be sharing a single
cred?).  In particular we can perhaps solve two problems at once.


I haven't heard back after I sent the test program there;  I hope that
can be solved independently first and any optimisations can then come.



Any thoughts?  Are there some fundamental reasons this can't work?



--
Bjoern A. Zeeb r15:7

Re: What's going on with vnets and epairs w/ addresses?

[Bjoern A. Zeeb]

On Sun, 18 Dec 2022, Zhenlei Huang wrote:




On Dec 18, 2022, at 3:23 AM, Bjoern A. Zeeb  wrote:

On Sat, 17 Dec 2022, Gleb Smirnoff wrote:


Zhenlei,

On Fri, Dec 16, 2022 at 06:30:57PM +0800, Zhenlei Huang wrote:
Z> I managed to repeat this issue on CURRENT/14 with this small snip:
Z>
Z> ---
Z> #!/bin/sh
Z>
Z> # test jail name
Z> n="test_ref_leak"
Z>
Z> jail -c name=$n path=/ vnet persist
Z> # The following line trigger jail pr_ref leak
Z> jexec $n ifconfig lo0 inet 127.0.0.1/8
Z>
Z> jail -R $n
Z>
Z> # wait a moment
Z> sleep 1
Z>
Z> jls -j $n
Z>
Z> After DDB debugging and tracing , it seems that is triggered by a combine of 
[1] and [2]
Z>
Z> [1] https://reviews.freebsd.org/rGfec8a8c7cbe4384c7e61d376f3aa5be5ac895915 
<https://reviews.freebsd.org/rGfec8a8c7cbe4384c7e61d376f3aa5be5ac895915>
Z> [2] https://reviews.freebsd.org/rGeb93b99d698674e3b1cc7139fda98e2b175b8c5b 
<https://reviews.freebsd.org/rGeb93b99d698674e3b1cc7139fda98e2b175b8c5b>


I can confirm [2] also affects Non-VNET jails.
Prison pr_ref leak cause jail stuck in dying state.


Usually a TCP connection in TW would do this in the old days and things
would solve themselves after a while.  This was always the case even
long before vnet or multi-IP jails.



Z>
Z>
Z> In [1] the per-VNET uma zone is shared with the global one.
Z> `pcbinfo->ipi_zone = pcbstor->ips_zone;`
Z>
Z> In [2] unref `inp->inp_cred` is deferred called in inpcb_dtor() by 
uma_zfree_smr() .
Z>
Z> Unfortunately inps freed by uma_zfree_smr() are cached and inpcb_dtor() is 
not called immediately ,
Z> thus leaking `inp->inp_cred` ref and hence `prison->pr_ref`.
Z>
Z> And it is also not possible to free up the cache by per-VNET SYSUNINIT 
tcp_destroy / udp_destroy / rip_destroy.

This is known issue and I'd prefer not to call it a problem. The "leak" of a 
jail
happens only if machine is idle wrt the networking activity.

Getting back to the problem that started this thread - the epair(4)s not 
immediately
popping back to prison0. IMHO, the problem again lies in the design of if_vmove 
and
epair(4) in particular. The if_vmove shall not exist, instead we should do a 
full
if_attach() and if_detach(). The state of an ifnet when it undergoes if_vmove 
doesn't
carry any useful information. With Alexander melifaro@ we discussed better 
options
for creating or attaching interfaces to jails that if_vmove. Until they are 
ready
the most easy workaround to deal with annoying epair(4) come back problem is to
remove it manually before destroying a jail, like I did in 80fc25025ff.


Ok, move an em0 or cxl0 into the jail;  the problem will be the same I
bet and you need the physical interface to not disappear as then you
cannot re-create a new jail with it.


Re-read sys/kern/kern_jail.c, if pr_ref leaks, vnet_destroy() has no chance to 
be called, thus
if_vmove is not called and epair(4)s or em0, exl0 are not returned to home vnet.

That can be confirmed by setting debug point on vnet_destroy by DDB, and then 
create and destroy vnet jails.

So before the problem prison pr_ref count leaks is resolved, it will cover 
other potential problems such as @glebius
pointed out.

I think the problem that prison ref count leaks should be resolved first.

I'm also reviewing the life cycles of prison / vnet and it seems they could 
still be improved.


But that's the not the problem here as your own test case pointed out.

The point is that if you start a plain vnet jail put an interface in and
destroy the jail that works instantly.
The moment you put an address on any interface (incl. loopback as your
test showed, which will not do ARP/NDP things compared to an ethernet
interface) the jail will no longer die immediately.

Simply putting an address on an interface should not defer things.
So indeed something holds onto things there and is not cleaned up
anymore.  Finding that "something" is the important bit and being able
to clean it up.

I always say, if you have a machine in shutdown -r you don't want it
hanging for hours either (now if you toggle the power switch you can do
a lot more without panicing the rest of the system but with jails we
cannot do that).  And we did have vnet jails shutting down preoperly and
clearing up for years.  People had spent a lot of time on that.  So it is
possible and we need to get back to that state.

/bz

--
Bjoern A. Zeeb r15:7

Re: What's going on with vnets and epairs w/ addresses?

[Bjoern A. Zeeb]

On Sat, 17 Dec 2022, Gleb Smirnoff wrote:


 Zhenlei,

On Fri, Dec 16, 2022 at 06:30:57PM +0800, Zhenlei Huang wrote:
Z> I managed to repeat this issue on CURRENT/14 with this small snip:
Z>
Z> ---
Z> #!/bin/sh
Z>
Z> # test jail name
Z> n="test_ref_leak"
Z>
Z> jail -c name=$n path=/ vnet persist
Z> # The following line trigger jail pr_ref leak
Z> jexec $n ifconfig lo0 inet 127.0.0.1/8
Z>
Z> jail -R $n
Z>
Z> # wait a moment
Z> sleep 1
Z>
Z> jls -j $n
Z>
Z> After DDB debugging and tracing , it seems that is triggered by a combine of 
[1] and [2]
Z>
Z> [1] https://reviews.freebsd.org/rGfec8a8c7cbe4384c7e61d376f3aa5be5ac895915 
<https://reviews.freebsd.org/rGfec8a8c7cbe4384c7e61d376f3aa5be5ac895915>
Z> [2] https://reviews.freebsd.org/rGeb93b99d698674e3b1cc7139fda98e2b175b8c5b 
<https://reviews.freebsd.org/rGeb93b99d698674e3b1cc7139fda98e2b175b8c5b>
Z>
Z>
Z> In [1] the per-VNET uma zone is shared with the global one.
Z> `pcbinfo->ipi_zone = pcbstor->ips_zone;`
Z>
Z> In [2] unref `inp->inp_cred` is deferred called in inpcb_dtor() by 
uma_zfree_smr() .
Z>
Z> Unfortunately inps freed by uma_zfree_smr() are cached and inpcb_dtor() is 
not called immediately ,
Z> thus leaking `inp->inp_cred` ref and hence `prison->pr_ref`.
Z>
Z> And it is also not possible to free up the cache by per-VNET SYSUNINIT 
tcp_destroy / udp_destroy / rip_destroy.

This is known issue and I'd prefer not to call it a problem. The "leak" of a 
jail
happens only if machine is idle wrt the networking activity.

Getting back to the problem that started this thread - the epair(4)s not 
immediately
popping back to prison0. IMHO, the problem again lies in the design of if_vmove 
and
epair(4) in particular. The if_vmove shall not exist, instead we should do a 
full
if_attach() and if_detach(). The state of an ifnet when it undergoes if_vmove 
doesn't
carry any useful information. With Alexander melifaro@ we discussed better 
options
for creating or attaching interfaces to jails that if_vmove. Until they are 
ready
the most easy workaround to deal with annoying epair(4) come back problem is to
remove it manually before destroying a jail, like I did in 80fc25025ff.


Ok, move an em0 or cxl0 into the jail;  the problem will be the same I
bet and you need the physical interface to not disappear as then you
cannot re-create a new jail with it.

/bz

--
Bjoern A. Zeeb r15:7

Re: What's going on with vnets and epairs w/ addresses?

[Bjoern A. Zeeb]

On Fri, 16 Dec 2022, Zhenlei Huang wrote:

Hi,


I managed to repeat this issue on CURRENT/14 with this small snip:

---
#!/bin/sh

# test jail name
n="test_ref_leak"

jail -c name=$n path=/ vnet persist
# The following line trigger jail pr_ref leak
jexec $n ifconfig lo0 inet 127.0.0.1/8

jail -R $n

# wait a moment
sleep 1

jls -j $n


---


After DDB debugging and tracing , it seems that is triggered by a combine of 
[1] and [2]

[1] https://reviews.freebsd.org/rGfec8a8c7cbe4384c7e61d376f3aa5be5ac895915 
<https://reviews.freebsd.org/rGfec8a8c7cbe4384c7e61d376f3aa5be5ac895915>
[2] https://reviews.freebsd.org/rGeb93b99d698674e3b1cc7139fda98e2b175b8c5b 
<https://reviews.freebsd.org/rGeb93b99d698674e3b1cc7139fda98e2b175b8c5b>


In [1] the per-VNET uma zone is shared with the global one.
`pcbinfo->ipi_zone = pcbstor->ips_zone;`

In [2] unref `inp->inp_cred` is deferred called in inpcb_dtor() by 
uma_zfree_smr() .

Unfortunately inps freed by uma_zfree_smr() are cached and inpcb_dtor() is not 
called immediately ,
thus leaking `inp->inp_cred` ref and hence `prison->pr_ref`.

And it is also not possible to free up the cache by per-VNET SYSUNINIT 
tcp_destroy / udp_destroy / rip_destroy.


Thanks a lot for tracking it down.

That seems to be a regression then that needs to be fixed before
14.0-RELEASE will happen as it'll break management utilities of people.

Could you open a bug report and flag it as such?

/bz





Best regards,
Zhenlei


On Dec 14, 2022, at 9:56 AM, Zhenlei Huang  wrote:


Hi,

I also encounter this problem while testing gif tunnel between jails.

My script is similar but with additional gif tunnels.


There are reports in mailing list [1], [2], and another one in forum [3] .

Seem to be a long standing issue.

[1] https://lists.freebsd.org/pipermail/freebsd-stable/2016-October/086126.html 
<https://lists.freebsd.org/pipermail/freebsd-stable/2016-October/086126.html>
[2] https://lists.freebsd.org/pipermail/freebsd-jail/2017-March/003357.html 
<https://lists.freebsd.org/pipermail/freebsd-jail/2017-March/003357.html>
[3] 
https://forums.freebsd.org/threads/jails-stopping-prolonged-deaths-starting-networking-et-cetera.84200/
 
<https://forums.freebsd.org/threads/jails-stopping-prolonged-deaths-starting-networking-et-cetera.84200/>


Best regards,
Zhenlei


On Dec 14, 2022, at 7:03 AM, Bjoern A. Zeeb mailto:b...@freebsd.org>> wrote:

Hi,

I have used scripts like the below for almost a decade and a half
(obviously doing more than that in the middle).  I haven't used them
much lately but given other questions I just wanted to fire up a test.

I have an end-November kernel doing the below my eapirs do not come back
to be destroyed (immediately).
I have to start polling for the jid to be no longer alive and not in
dying state (hence added the jls/ifconfig -l lines and removed the
error checking from ifconfig destroy).  That seems sometimes rather
unreasonably long (to the point I give up).

If I don't configure the addresses below this isn't a problem.

Sorry I am confused by too many incarnations of the code; I know I once
had a version with an async shutdown path but I believe that never made
it into mainline, so why are we holding onto the epairs now and not
nuking the addresses and returning them and are clean?

It's a bit more funny; I added a twiddle loop at the end and nothing
happened.  So I stop the script and start it again and suddenly another
jail or two have cleaned up and their epairs are back.  Something feels
very very wonky.  Play around with this and see ... and let me know if
you can reproduce this...  I quite wonder why some test cases haven't
gone crazy ...

/bz


#!/bin/sh

set -e
set -x

js=`jail -i -c -n jl host.hostname=left.example.net <http://left.example.net/> 
vnet persist`
jb=`jail -i -c -n jr host.hostname=right.example.net 
<http://right.example.net/> vnet persist`

# Create an epair connecting the two machines (vnet jails).
ep=`ifconfig epair create | sed -e 's/a$//'`

# Add one end to each vnet jail.
ifconfig ${ep}a vnet ${js}
ifconfig ${ep}b vnet ${jb}

# Add an IP address on the epairs in each vnet jail.
# XXX Leave these out and the cleanup seems to work fine.
jexec ${js}  ifconfig ${ep}a inet  192.0.2.1/24
jexec ${jb}  ifconfig ${ep}b inet  192.0.2.2/24

# Clean up.
jail -r ${jb}
jail -r ${js}

# You want to be able to remove this line ...
set +e

# No epairs to destroy with addresses configured; fine otherwise.
ifconfig ${ep}a destroy
# echo $?

# Add this is here only as things are funny ...
# jls -av jid dying
# ifconfig -l

# end
----

--
Bjoern A. Zeeb     r15:7








--
Bjoern A. Zeeb r15:7

Re: prison_flag() check in hot path of in_pcblookup()

[Bjoern A. Zeeb]

On Tue, 13 Dec 2022, Andrew Gallatin wrote:


[ I added pjd, since the original patch came from him ]

Just to make sure I understand, I have a simple yes/no question:

Can  jails and the host ever share the same (local) port and the same IP?


Can they currently (I tested only for TCP)?

- local binds can overlap like they can with just the base system.
  so bind(... {AF_INET, laddr, lport} ... ) works fine (REUSEPORT).

- tcp connect of a 2nd socket to the same {faddr, fport} from the above
  bind will fail with 'Address already in use'  [currently]
  [I believe that would mean your patch could go in? Where does the error come 
from [%]?] [*]

- tcp listen will work on {laddr, lport} if run in paralllel (REUSEPORT)
  or in base and jail at the same time.

[%] likely in_pcbconnect_setup() ?  Also one should check the other
order (jail first then base);  also we assume no other race
conditions in this rather simple testing...

[*] Now someone should run this on a FreeBSD 7.3 / 8.x or later and see how it 
behaves as the stack might have behaved differently.
Also if you have two physical machines or two VMs connected remove the VNET 
layer and just (manually) test the two parts firing up one extra jail on each 
base system.
I just used vnets for simplicity of my testing.

(sorry vnet cleanup currently screwed as it seems 15 years of working also have 
changed due to other changes; you cal always run jail -r jl jr manually).

I haven't done user space socket programming in a while so this was fun (and 
hopefully does what it should for this test case).
I put the simple sources (shell script and C file) up at:
 https://people.freebsd.org/~bz/tmp/jail-in_pcblookup/

HTH /bz

+ pwd
+ STESTBIN=/home/test/socket
+ PORT=7
+ jail -i -c -n jl 'host.hostname=server.example.net' vnet persist 
'children.max=1'
+ js=211
+ jail -i -c -n jr 'host.hostname=base.example.net' vnet persist 
'children.max=1'
+ jb=212
+ jexec 211 ifconfig lo0 inet 127.0.0.1/8 alias up
+ jexec 211 ifconfig lo0 inet6 ::1/128 alias
+ jexec 212 ifconfig lo0 inet 127.0.0.1/8 alias up
+ jexec 212 ifconfig lo0 inet6 ::1/128 alias
+ ifconfig epair create
+ sed -e 's/a$//'
+ ep=epair102
+ ifconfig epair102a vnet 211
+ ifconfig epair102b vnet 212
+ jexec 211 ifconfig epair102a inet 192.0.2.1/24
+ jexec 212 ifconfig epair102b inet 192.0.2.2/24
+ jexec 211 jail -i -c -n jsj 'host.hostname=jails.example.net' 
'ip4.addr=192.0.2.1' persist
+ jexec 211 /home/test/socket 192.0.2.1 7
/home/test/socket pid 23254 listening on [192.0.2.1 7]
+ jsj=213
+ echo 'Listing listening connections from the server (base) system'
+ jexec 213 /home/test/socket 192.0.2.1 7
Listing listening connections from the server (base) system
+ jexec 211 netstat -an
/home/test/socket pid 23257 listening on [192.0.2.1 7]
Active Internet connections (including servers)
Proto Recv-Q Send-Q Local Address  Foreign Address(state) 
tcp4   0  0 192.0.2.1.7*.*LISTEN 
tcp4   0  0 192.0.2.1.7*.*LISTEN 
+ jexec 212 jail -i -c -n jbj 'host.hostname=jailb.example.net' 'ip4.addr=192.0.2.2' persist

+ jbj=214
+ sleep 1
+ echo 'Starting connection from base jail'
Starting connection from base jail
+ sleep 1
+ jexec 212 /home/test/socket 192.0.2.2 12345 192.0.2.1 7
/home/test/socket pid 23257 accepted [192.0.2.2 12345]
/home/test/socket pid 23261 [192.0.2.2 12345] sleeping 60.
+ echo 'Starting connection from plain-old IP jail'
Starting connection from plain-old IP jail
+ sleep 1
+ jexec 214 /home/test/socket 192.0.2.2 12345 192.0.2.1 7
socket: connect
: Address already in use
+ echo 'Listing server connections from the server (base) system'
Listing server connections from the server (base) system
+ jexec 211 netstat -an
Active Internet connections (including servers)
Proto Recv-Q Send-Q Local Address  Foreign Address(state) 
tcp4   0  0 192.0.2.1.7192.0.2.2.12345ESTABLISHED
tcp4   0  0 192.0.2.1.7*.*LISTEN 
tcp4   0  0 192.0.2.1.7*.*LISTEN 
+ echo 'Listing client connections from the base system'

Listing client connections from the base system
+ jexec 212 netstat -an
Active Internet connections (including servers)
Proto Recv-Q Send-Q Local Address  Foreign Address(state) 
tcp4   0  0 192.0.2.2.12345192.0.2.1.7ESTABLISHED

+ sleep 60
^C

--
Bjoern A. Zeeb r15:7

What's going on with vnets and epairs w/ addresses?

[Bjoern A. Zeeb]

Hi,

I have used scripts like the below for almost a decade and a half
(obviously doing more than that in the middle).  I haven't used them
much lately but given other questions I just wanted to fire up a test.

I have an end-November kernel doing the below my eapirs do not come back
to be destroyed (immediately).
I have to start polling for the jid to be no longer alive and not in
dying state (hence added the jls/ifconfig -l lines and removed the
error checking from ifconfig destroy).  That seems sometimes rather
unreasonably long (to the point I give up).

If I don't configure the addresses below this isn't a problem.

Sorry I am confused by too many incarnations of the code; I know I once
had a version with an async shutdown path but I believe that never made
it into mainline, so why are we holding onto the epairs now and not
nuking the addresses and returning them and are clean?

It's a bit more funny; I added a twiddle loop at the end and nothing
happened.  So I stop the script and start it again and suddenly another
jail or two have cleaned up and their epairs are back.  Something feels
very very wonky.  Play around with this and see ... and let me know if
you can reproduce this...  I quite wonder why some test cases haven't
gone crazy ...

/bz


#!/bin/sh

set -e
set -x

js=`jail -i -c -n jl host.hostname=left.example.net vnet persist`
jb=`jail -i -c -n jr host.hostname=right.example.net vnet persist`

# Create an epair connecting the two machines (vnet jails).
ep=`ifconfig epair create | sed -e 's/a$//'`

# Add one end to each vnet jail.
ifconfig ${ep}a vnet ${js}
ifconfig ${ep}b vnet ${jb}

# Add an IP address on the epairs in each vnet jail.
# XXX Leave these out and the cleanup seems to work fine.
jexec ${js}  ifconfig ${ep}a inet  192.0.2.1/24
jexec ${jb}  ifconfig ${ep}b inet  192.0.2.2/24

# Clean up.
jail -r ${jb}
jail -r ${js}

# You want to be able to remove this line ...
set +e

# No epairs to destroy with addresses configured; fine otherwise.
ifconfig ${ep}a destroy
# echo $?

# Add this is here only as things are funny ...
# jls -av jid dying
# ifconfig -l

# end


--
Bjoern A. Zeeb r15:7

Re: prison_flag() check in hot path of in_pcblookup()

[Bjoern A. Zeeb]

On Tue, 13 Dec 2022, Andrew Gallatin wrote:


Are there regression tests for jails where this patch could be run to
ensure it is safe?


Not that I now of but it would certainly good to have one for that case.

But it's likely not going to be deterministic so the question will be
more of the case "theoretically possible or not"?

--
Bjoern A. Zeeb r15:7

Re: prison_flag() check in hot path of in_pcblookup()

[Bjoern A. Zeeb]

On Tue, 13 Dec 2022, James Gritton wrote:

Hi,

Argh, sorry Drew, I looked at the wrong of the two checks in the
function earlier.  Sorry, that's what happens if trying to be helpful
when firefighting elsewhere.


On 2022-12-13 09:18, Andrew Gallatin wrote:

I was trying to improve the performance of in_pcblookup(), as it is a very 
hot path for us (Netflix). One thing I noticed was the prison_flag() check 
in in_pcblookup_hash_locked() can cause a cache miss just by deref'ing the 
cred pointer, and it can also cause multiple misses in tables with 
collisions by causing us to walk the entire chain even after finding a 
perfect match.


I'm curious why this check is needed.  Can you explain it to me?  It 
originated in this commit:


commit 413628a7e3d23a897cd959638d325395e4c9691b
Author: Bjoern A. Zeeb 
Date:   Sat Nov 29 14:32:14 2008 +

MFp4:
Bring in updated jail support from bz_jail branch.

This enhances the current jail implementation to permit multiple
addresses per jail. In addtion to IPv4, IPv6 is supported as well.

My thinking is that a jail will either use the host IP, and share its port 
space, or it will have its own IP entirely (but I know nothing about 
jails).


Well having its own IP address entirely doesn't work with classic jails
as there is only one network stack where they can be configured on
an interface, and that is the base system.

So de-facto all jail address/port space is always shared with the host
(ignoring vnet jails with their own entire virtual network stack).
Whether the host would bind/listen is a different story.  I know 15
years ago people would bind the sshd of the base to a single-IP address
which was not assigned to jails and then the sshd inside a jail would
bind to a different (single) IP address.   If one doesn't do that and the
sshd inside the jail isn't runnig one would end up trying to connect to
the sshd in the base system (sshd being a popular example).  Not sure if
people still do but that's still the case.

There are special cases with classic multi-IP jails in that they then
cannot have overlapping IP-ranges as otherwise it would not be
deterministic which jail would get an inbound connection on
inaddr_any:port_n if two jails were to listen on the same port_n.

Hope that helps for basic understanding.


In either case, a perfect 4-tuple match should be enough to 
uniquely identify the connection.


Even if this somehow is not the case and we have multiple connections 
somehow sharing the same 4-tuple, how does checking the prison flag help 
us?


That logic predates me and came from [1].  The jail_jailed_sockets_first
sysctl got removed in the review process with rwatson.  I am still trying
to see where the SO_REUSEPORT comment (back then) came from.  I know I
only had the first lines initially, so must have been sometime during
review with rwatson as well.   Sadly p4 emails where truncated to 1000
lines so I cannot simply grep for the change (if it is in my mail
archives) or had a useful commit message (but at least would give a
date to check further private email).

My current guess is that if we have the 4-tuple in both the base
and a jail (hence the SO_REUSEPORT comment) we want the jail not getting
a socket of the base system returned as that would mean one could "break
out of prison".  But if the inp belongs to a jail we know we can simply
return.  So if you find the one of the base system first you'll have to
go and look through the others.

XXX-jamie: is that all still true in hierarchical jails?

Now whether fabricating the case of having both inps on the hash is
still theorectically possible I cannot say.  I haven't followed all the
changes to that code lately close enough.


It would prefer the jailed connection over the non jailed, but that 
would shadow a host connection.  And if we had 2 jails sharing the same 
4-tuple, the first jail would win.


I can't see how this check is doing anything useful, so I'd very much like 
to remove this check if possible.   Untested patch attached.


For a complete 4-tuple, it should indeed be the case that a match would only 
ever identify a single prison.  The later part of the function that examines 
wildcards definitely needs the check.  I don't get the XXX comment about both 
being bound with SO_REUSEPORT, because I would only expect that to apply to 
listening, not to full connections. But I also expect Bjoern to know more 
than I do here...


/bz

[1] https://people.freebsd.org/~pjd/patches/jail_2004120901.patch

--
Bjoern A. Zeeb r15:7

Re: how to determine primary (source) IP address in jail

[Bjoern A. Zeeb]

On 28 Feb 2019, at 10:58, Miroslav Lachman wrote:

Is there some easy way to determine the primary (source) address which 
is used in jail with multiple IP addresses?


I came to this problem with running local_unbound in jail. Unbound 
refuses queries originating in this jail because the do not come from 
real 127.0.0.1 (which is the only one allowed by default). Unbound in 
jail see requests come from jails IP. It is easy to determine (in 
shell script) if jail has only one IP.
But what in case where jail has multiple IPs? Is there some sysctl or 
some call to ifconfig or any other util to get the IP which will be 
used as source address for queries on local services in jail?


Bind the listen socket of the local unbound to any IP of your jail and 
other services (unless the source port got bound) will select the same 
IP address as the destination if both are in the same jail.




I know I can allow all IPs of jail in
access-control: a.b.c.d/32 allow
access-control: e.f.g.h/32 allow

I am just curios if there is some way to get "primary" IP in jail 
without calling anything from the host environment.


Open a UDP socket; bind to 127.1; call getsockname;
https://reviews.freebsd.org/D19218   is currently having a similar issue 
solving it exactly that way.



There were people who in the past added a 127.{2,3,4,5,..}  for each 
jail and then used that one instead of 127.1 but I’ve never been a 
huge fan of that, especially given one  may run the resolver for other 
services outside that jail (maybe in others) as well and they need to be 
able to reach that in a reliable way.



/bz
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: init in a jail

[Bjoern A. Zeeb]

On 11 Feb 2019, at 17:23, James Gritton wrote:


On 2019-02-11 08:48, Michael W. Lucas wrote:

Sadly, my google-fu has turned up thousands of man pages but no real
discussion on this.

According to init(8), you can run init inside a jail.

 If init is run in a jail, the security level of the "host 
system" will
 not be affected.  Part of the information set up in the kernel 
to support
 a jail is a per-jail security level.  This allows running a 
higher
 security level inside of a jail than that of the host system.  
See

 jail(8) for more information about jails.


If you actually try, though, the jail dies:

storm~;jail -vc loghost
loghost: run command: /sbin/ifconfig jailether inet 198.51.100.225 
netmask

255.255.255.255 alias
loghost: run command: /sbin/mount -t devfs -oruleset=4 . 
/jail/loghost/dev

loghost: run command: logger trying to start jail loghost...
loghost: jail_set(JAIL_CREATE) persist name=loghost 
path=/jail/loghost

host.hostname=loghost.mwl.io ip4.addr=19 8.51.100.225
loghost: created
loghost: run command in jail: /sbin/init
jail: loghost: /sbin/init: failed
loghost: removed
loghost: run command: /sbin/umount /jail/loghost/dev
loghost: run command: /sbin/ifconfig jailether inet 198.51.100.225 
netmask

255.255.255.255 -alias

Is that init(8) text left over from an earlier jail incarnation? Or 
is

there some other way to run init in a jail?

And WHY would you run init in a jail?


Interesting - I wonder how long it's been since init worked inside 
jails.  From the look of your error messages, probably not since devfs 
started being used.  I wasn't even aware the init(8) had anything to 
say on the matter, but it's clearly erroneous.


Ken Smith added that message to init(8) 15 years ago and from the sounds 
of it, I think it was more related to securelevels.




AS to why it would be good to have a per-jail init, there would be a 
few advantages.  Orphaned processes could then reparent to the jail's 
init instead of the real init, and the jail root could easily reboot 
jails. Doing it right would require presenting jailed init as pid 1, 
but that's not really very hard.


It’s not just PID 1 but yeah;  I have open reviews (which I should 
update) from the vps branch to do a virtualised pid space, real init to 
jails along with it, console, and then init would also manage ttys, ..  
I need to work on the management bits from the host side to make it a 
real thing (ps, kill, etc. to work with a (jid, pid) combination as 
jexec won’t work anymore (possible collisions etc).  But that’s 
unrelated to this thread.



/bz
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: icmp (IPv4) issues with VIMAGE JAILs and IPv6

[Bjoern A. Zeeb]

On 28 Jan 2019, at 12:44, O. Hartmann wrote:


I ran into severe problems on CURRENT ( FreeBSD 13.0-CURRENT #193
r343521: Mon Jan 28 10:26:36 CET 2019 amd64), VIMAGE enabled host with 
jails

utilizing IPv6.


and you forget to mention in the subject that it seems to be an ipfw 
problem and thus missing your target audience most likely.



Stopping all jails, destroying all epairs and bridge0 doesn't change 
anything.


The problems occured when IPv6 came into play on the specific host in 
question.


Does that mean you could reproduce the problem just with bce0 and no 
jail+vnets at all just with IPv4, IPv6, and ipfw on the main host?


If you cannot, would starting a vnet-jail (without the bridge and 
connecting the epair), just starting a jail in persist mode, make a 
difference;  or would strating a jail and applying the ipfw rules 
therein start the problem?




Does anyone have any ideas? I'm out of ideas.


y best guess is to move the thread to freebsd-ipfw (Cc: and Reply-To: 
set) and see if people pick it up there some more and also finding the 
minimalistic case to reproduce.


/bz
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: enforce_statfs showing leading path

[Bjoern A. Zeeb]

On 9 Jan 2019, at 9:42, Alexander Leidinger via freebsd-jail wrote:

Hi,

I’ll be a bit verbose also for mwlucas.

You see the dataset name of zfs without stripping. The mount point is 
correctly stripped. I don't remember how this looks on ufs.


/dev/ada0p19 on / (ufs, local, journaled soft-updates)
/local/data/www1/users/johndoe on /usr/home/johndoe/www1 (nullfs, local)

The “device” is also visible there, as well as file system type and 
specific options.


I also added a nullfs example.  For that the mount point is properly 
treated and lost the /local/jails/whatever/ prefix with enforce_statfs=1 
but the “device” side is just as visible in full as it is for a any 
other real device.



With jailed datasets we would need more than just some code to remove 
parts of the name.


So it's a doc bug (clarity about mount points and dataset names) and a 
zfs issue.


Well, no it’s not a zfs specific issue.  And the docs talk about mount 
points not about the “device” (or dataset in zfs parlance).


If anything for clarity one could add a sentence to the jail(8) page 
saying that the “device” part of the mount output is not being 
restricted or altered.



One of the reasons for enforce_statfs certainly was to limit the amount 
of information;  that also has the side-effect of scripts parsing the 
mount (mount points) output actually finding the paths they might be 
looking for.



The df command output might make some of this all a bit more clear.



Am 8. Januar 2019 8:34:17 nachm. schrieb "Michael W. Lucas" 
:



Hi,

I'm experimenting with enforce_statfs for the jails book, and have 
hit
an inconsistency. Not sure if the bug should go to src or doc. 
Running

last week's -current.

According to jail(8):

When set to 1, only mount points below the jail's chroot
directory are visible.  In addition to that, the path to 
the
jail's chroot directory is removed from the front of 
their path‐

names.

Seems pretty clear that I shouldn't see anything other than

# jls -h name enforce_statfs
...
ioc-www1 1

So, as I read it, the jail's chroot directory should be stripped down
to /. But inside the jail:

root@www1:~ # mount
iocage/iocage/jails/www1/root on / (zfs, local, nfsv4acls)


   it is stripped down to /


devfs on /dev (devfs, local, multilabel)
fdescfs on /dev/fd (fdescfs)

I see the jail's chroot directory.

This seems to contradict the man page, unless I'm misunderstanding.

Is this a software bug? A ZFS thing? A doc bug? Or am I just an 
idiot?


Also, should this path be stripped when enforce_statfs is set to 1 
*or

above*? Or is this strictly when set to 1? If I'm filing a bug, it
might as well be complete...

Thanks,
==ml

--
Michael W. Lucas https://mwl.io/
author of: Absolute OpenBSD, SSH Mastery, git commit murder,
Immortal Clay, PGP & GPG, Absolute FreeBSD, etc, etc, etc...



--
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to 
"freebsd-jail-unsubscr...@freebsd.org"




___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to 
"freebsd-jail-unsubscr...@freebsd.org"

___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Issue with 127.0.0.1 when reconfiguring running Jail

[Bjoern A. Zeeb]

On 6 Aug 2018, at 16:32, Support SimpleRezo wrote:


Hi !

I'm fancing an issue when i'm using "jail -m ip4.addr=..." for
reconfiguring ip4.addr of a running jail: accessing or binding 
127.0.0.1 is

not redirect anymore by kernel to the jail IP.

Is it expected? Do I missing something there?


The change of address should not make a difference;  the primary jail 
address should be used.


Can you show us a very short (scripted) example of the problem?

/bz
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: Time for those old global jail sysctls to go

[Bjoern A. Zeeb]

On 22 Mar 2018, at 4:13, James Gritton wrote:

I've got a revision in the works  
to

remove the security.jail.foo_allowed sysctls:


The old jail system had sysctls to set jail permissions for all jails
(e.g. security.jail.mount_allowed), which were superseded by per-jail
permissions (e.g. allow.mount). These old sysctls remain a constant
source of confusion to users, who expect that setting the sysctl will
change the behavior of existing jails. That the sysctl value at the 
time
a jail is created may matter is a backward-compatibility hack that 
does
little or nothing to relieve the confusion. So it's time for them to 
go.



Also, jail(2) has been replaced by jail_set(2) for a number of years
now, and it really ought to retire - at least into the COMPAT world.


This may be of interest to anyone who works with jails.  My hope is 
that
no current software relies on these old sysctls, and they can be 
removed
with little trouble.  But removing old things never seems to go that 
easy.


I think #1 action item is to put them under BURN_BRIDGES or however it 
was spelt if you really want to remove them.
Then for the next major version they could go away ( I’d be all up for 
removing them immediately (incl. from the man pages ) but I remember 
there used to be 2-3 ports which used the jail v1 stuff;  might be worth 
checking that they were updated or are gone?



/bz
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: testing 11.0-RC1 vnet jails with ipfilter

[Bjoern A. Zeeb]

On 16 Aug 2016, at 12:47, krad wrote:

is ipfilter supported in vnet jails? Last time I looked and tried pf 
didnt

work (kernel panics), and only ipfw was supported.


In 11-RC* it is present for all 3 firewalls;  like VIMAGE due to memory 
footprint you might have to compile the firewall into the kernel rather 
than kldload it (especially ipfilter).


/bz
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: testing 11.0-RC1 vnet jails with ipfilter

[Bjoern A. Zeeb]

On 15 Aug 2016, at 15:37, Ernie Luzar wrote:


Hello list;

Running 11.0-RC1 with only option vimage compiled into the generic 
kernel.


I can run ipfilter on the host and start vnet jails containing no 
firewalls just fine. But when I try to also have ipfilter run in the 
vnet jail nothing happens. I added this to the vnet jails rc.conf

ipfilter_enable="YES"
ipfilter_rules="/etc/ipf.boot.rules"
ipmon_enable="YES"
ipmon_flags="-Ds"

Then start the vnet jail and its like those ipfilter statements in the 
vnet jails rc.conf are not there. The vnet jails /var/log/messages 
file is not even there. Issuing "ipfstat" inside the running vnet jail 
to display the jails ipfilter rules gives this error message 
"open(IPSTATE_NAME): No such file or directory"
To me this means ipfilter is not running in the vnet jail even though 
I requested it in the vnet jails rc.conf file.


So my question to this list is, has anyone managed to get ipfilter to 
run inside a vnet jail using any of the 11.0 alpha, beta, or rc 
versions? If so would you please share your setup with me?


Maybe I am to close to the bleeding edge for there to be other users 
in the same test loop?



The startup script contains “nojail”.   I think someone opened a bug 
report the other day but I can’t find it anymore;  so the startup 
script won’t automatically run inside a jail.   Can you remove that 
line and try again?



/bz
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Request for VIMAGE testing in 11.0-ALPHA6 and later

[Bjoern A. Zeeb]

Hi,

during the last weeks and months a lot of changes went into the tree to
allow a top-to-bottom network stack teardown to stabilize VNET
shutdown and plug some memory leaks.

In addition some missing parts were virtualised or the virtualisation
was fixed, e.g., pf and ipfilter, ipfw log interface.

I have done some testing and stress testing but it's impossible to catch 
all combinations and setups or even options.  So once 11.0-ALPHA6 is

out please do test (or if you want to do so now r302302 or later).

These changes are only and will only be in FreeBSD 11 for the time being.

You will still need to compile your own kernel;  GENERIC will not have
VIMAGE enabled for 11.0 as that requires at least a performance
analysis (due to extra layer of indirection).   It will also still
print the "experimental" feature line, as we do not want to commit to
KPI/KBI or other things yet and we feel more testing would be good.
I would advise to start testing on dedicated test-systems and not
necessarily production servers but obviously that is your choice.

Also if you are using ports that bring their own ifnet interfaces and
you are experiencing problems please let us know.

If you find problems please file a bug report and make sure to set
"vimage" in the Keywords field but feel also free to post to
freebsd-virtualisation@ which I'll be monitoring.

Thanks a lot to everyone!

Bjoern

--
Bjoern A. Zeeb r15:7
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: jails in different private subnets on the same host

[Bjoern A. Zeeb]

> On 18 May 2016, at 14:00 , Grzegorz Junka  wrote:
> 
> Is it possible to have two jails on the same host each one in a different 
> private subnet, e.g. 192.168.1.0 and 10.33.1.0, and have routing between them 
> working without issues?
> 
> I know it's possible to run jails with IPs in those two subnets but it seems 
> there is no routing and I am not sure if it's because I can't configure my 
> router properly or there is a more fundamental problem. One issue I see is 
> that the jail can't have a different default gateway than the host, and that 
> for now is 192.168.1.1, but I don't see a reason why 10.33.1.0 wouldn't be 
> able to use 192.168.1.1 as it's default gateway provided there is routing 
> between those two subnets.

Given they are both on the same base system host,  both addresses are connected 
locally and thus the kernel knows where to deliver these packets.  If that 
doesn’t work, there is a bug somewhere.

If you want different default gateways then you may want to look into using 
different FIBs for different jails.  See route(8) and jail(8) for parameters to 
set and tune.

/bz

___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: (VNET) jails not going away

[Bjoern A. Zeeb]

Hi,

sorry for the cross-post, Reply-To: set.


> On 22 Feb 2016, at 13:41 , Bjoern A. Zeeb <bzeeb-li...@lists.zabbadoz.net> 
> wrote:
> 
> Hi,
> 
> has anyone else experienced VNET jails to not fully go away anymore on a 
> recent HEAD kernel (or possibly an older kernel)?
> 
> I have test cases with which I can have them in DYING state (see jls -av) for 
> ever or at least more than half a day.   I am in the process of trying to 
> find the cause but would be good to know if anyone else is experiencing this?

Ok, I found more funny behaviour that I can get rid of the previous jail by 
cleaning up the next one.

root@rabbit4:/home/test # jail -i -c -n test19 host.hostname=foo vnet persist
19
root@rabbit4:/home/test # jexec 19 /bin/csh
root@foo:/ # ifconfig lo0 inet 127.19/8
root@foo:/ # exit
root@rabbit4:/home/test # jail -r 19

Jail 19 is in DYING and hangs there forever;  If I repeat this upon exit from 
jail 20, jail 19 will go away.

If I’ll just do this

root@rabbit4:/home/test # jail -i -c -n test20 host.hostname=foo vnet persist
21
root@rabbit4:/home/test # jail -r 21

20 and 21 are going.


I’ll keep tracing this but if it ring a bell for anyone please let me know ;-)



> Thanks,
> Bjoern
> 
> Example (after more than 12 hours of jail -r ..):
> 
> # jls -av
>   JID  Hostname  Path
>Name  State
>CPUSetID
>IP Address(es)
> 1  left.example.net  /
>lef827DYING
>18
> 2  center.example.net/
>mid827DYING
>19
> 3  right.example.net /
>right827  DYING
>20
> 6  right.example.net /
>right923  DYING
>23



— 
Bjoern A. Zeeb  Charles Haddon Spurgeon:
"Friendship is one of the sweetest joys of life.  Many might have failed
 beneath the bitterness of their trial  had they not found a friend."

___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

VNET jails not going away

[Bjoern A. Zeeb]

Hi,

has anyone else experienced VNET jails to not fully go away anymore on a recent 
HEAD kernel (or possibly an older kernel)?

I have test cases with which I can have them in DYING state (see jls -av) for 
ever or at least more than half a day.   I am in the process of trying to find 
the cause but would be good to know if anyone else is experiencing this?


Thanks,
Bjoern

Example (after more than 12 hours of jail -r ..):

# jls -av
   JID  Hostname  Path
Name  State
CPUSetID
IP Address(es)
 1  left.example.net  /
lef827DYING
18
 2  center.example.net/
mid827DYING
19
 3  right.example.net /
right827  DYING
20
 6  right.example.net /
right923  DYING
23
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

VNET teardown changes (part I)

[Bjoern A. Zeeb]

Hi,

sorry for the cross-post;  Reply-To set.

I extracted a patch from projects VNET which tries to get the VNET teardown
more robust (and in a next step plug the remaining [TCP] memory leaks).

If anyone has an interest in testing some parts on a non-production setup
(you have been warned) please do so and report back to me (privately) in case
of success or panics.

There is more to come.

https://people.freebsd.org/~bz/20160222-01-projects-vnets.diff

Thanks,
Bjoern
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"

Re: How to implement jail-aware SysV IPC (with my nasty patch)

[Bjoern A. Zeeb]

Hi,

removed hackers, added virtualization.


 On 12 Jun 2015, at 01:17 , kikuc...@uranus.dti.ne.jp wrote:
 
 Hello,
 
 I’m (still) trying to figure out how jail-aware SysV IPC mechanism should be.

The best way probably is to finally get the “common” VIMAGE framework into HEAD 
to allow easy virtualisation of other services.  That work has been sitting in 
perforce for a few years and simply needs updating for sysctls I think.

Then use that to virtualise things and have a vipc like we have vnets.  The 
good news is that you have identified most places and have the cleanup 
functions already so it’d be a matter of transforming your changes (assuming 
they are correct and working fine; haven’t actually read the patch in detail;-) 
 to the different infrastructure.  And that’s the easiest part.


Bjoern

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: How to implement jail-aware SysV IPC (with my nasty patch)

[Bjoern A. Zeeb]

 On 15 Jun 2015, at 17:10 , kikuc...@uranus.dti.ne.jp wrote:
 
 On Mon, 15 Jun 2015 09:53:53 +, Bjoern A. Zeeb 
 bzeeb-li...@lists.zabbadoz.net wrote:
 Hi,
 
 removed hackers, added virtualization.
 
 
 On 12 Jun 2015, at 01:17 , kikuc...@uranus.dti.ne.jp wrote:
 
 Hello,
 
 I’m (still) trying to figure out how jail-aware SysV IPC mechanism should 
 be.
 
 The best way probably is to finally get the “common” VIMAGE framework into 
 HEAD to allow easy virtualisation of other services.  That work has been 
 sitting in perforce for a few years and simply needs updating for sysctls I 
 think.
 
 Then use that to virtualise things and have a vipc like we have vnets.  The 
 good news is that you have identified most places and have the cleanup 
 functions already so it’d be a matter of transforming your changes (assuming 
 they are correct and working fine; haven’t actually read the patch in 
 detail;-)  to the different infrastructure.  And that’s the easiest part.
 
 
 Bjoern
 
 Hi Bjoern,
 Thank you for your reply.
 
 The common VIMAGE framework sounds good, I really want it.
 
 I want to know what the IPC system looks like for user-land after virtualized,
 and what happen if vnet like vipc is implemented.
 
 For example, jail 1, 2, 3 join vipc group A, and jail 4, 5, 6 join vipc group 
 B ??
 Hmm, it looks good.


That’s not exactly how it works currently and I think the mixing of options 
will be harder and something we’l have to figure out more carefully.
You would be able to say jail 1 has a vipc and jail 2 and 3 and “child jails” 
and inherit it.  (similar for 4 + 5,6) so it’s nested but not side-by-side.

If we want more of the “mixing” and independentness we’ll have to re-think the 
way we “manage” jails.

Bjoern
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Current state of VIMAGE on 10-STABLE?

[Bjoern A. Zeeb]

 On 26 Mar 2015, at 15:54 , Ernie Luzar luzar...@gmail.com wrote:
 
 Kai Gallasch wrote:
 Hi.
 What is the current state of VIMAGE jails on 10-STABLE?
 I'm asking, because I saw that Craig Rodrigues and others are working on
 some long known problems with VIMAGE and there were some related patches
 committed to the tree.
 When I experimented with VIMAGE jails on 10.1-RELENG a few weeks ago, I
 got this when I stopped a running VIMAGE jail.
 Freed UMA keg (udp_inpcb) was not empty (180 items).  Lost 18 pages of
 memory.
 Freed UMA keg (udpcb) was not empty (1757 items). Lost 7 pages
 of memory.
 Freed UMA keg (ripcb) was not empty (150 items).  Lost 15
 pages of memory.
 hhook_vnet_uninit: hhook_head type=1, id=1 cleanup
 required
 hhook_vnet_uninit: hhook_head type=1, id=0 cleanup required
 Has this already been adressed by recent development?
 Regards,
 K.
 
 This lost memory problem is a long time known bug. 2 - 3 years at least.
 Guess no one has addressed it yet. There are many problems with having a 
 firewall running on host and in the vimage jail at same time.
 
 Since you say Craig Rodrigues has committed some patches, maybe you should 
 contact him directly and post you finding here for others to read.

All the rest but TCP is my old p4 branch but needs to be carefully reviewed and 
updated due to a couple of years of stack changes.

The one for TCP just emerged as well, and is in reviews (I guess without the 
VIMAGE parts but that will be the easy thing to add if the harder problem is 
indeed solved :-)



— 
Bjoern A. Zeeb  Charles Haddon Spurgeon:
Friendship is one of the sweetest joys of life.  Many might have failed
 beneath the bitterness of their trial  had they not found a friend.

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: VNET performance

[Bjoern A. Zeeb]

On 10 Jul 2014, at 20:45 , Clint Armstrong cl...@clintarmstrong.net wrote:

 What is the expected network performance of a VNET jail for network
 communication between the jail and the host, or between multiple jails? I
 expected it to approach the 10Gbps of the epair device, but I'm not seeing
 that.
 
 I see between 800 - 1200 Mbps in standard iperf tests both between the host
 bridge interface and the vnet jail inteface. I see the same poor speeds if
 I make 2 vnet jails and put one side of the epair in each and test between
 them.
 
 Is the overhead of vnet causing this? Is there anything I can do to improve
 this performance.
 
 I’ve tested and seen similar performance on 10.0-RELEASE and 11.0-CURRENT.

epair has a netisr queuing in between as you cannot call the input routines 
directly from the output routines.  I was able to get a bit more traffic 
through by doing pinning games.

I wonder what a vale switch for vnets could achieve.

— 
Bjoern A. Zeeb Come on. Learn, goddamn it., WarGames, 1983

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: kern/68189 and kern/169751: what jails are allowed to see in a routing socket

[Bjoern A. Zeeb]

On Wed, 2 Jan 2013, Jamie Gritton wrote:

I've been looking at PR kern/169751, which was noting that routing sockets 
don't work inside a jail.  It made the point that setting 
security.jail.socket_unixiproute_only or security.jail.allow_raw_sockets 
didn't help things.  It would seem kind of a given from the unixiproute 
name that a route socket ought to work.  And indeed, such a socket is 
permitted to be created in such a jail.  It's just using it that doesn't 
work.


I narrowed this failure down to line 816 of sys/net/rtsock.c, which 
explicitly denies jails from reading routes, regardless of the setting of the 
above two sysctls (or the jail allow.* bits they work with).  And that bit of 
code came about in response to PR kern/68189, which noted that jails could 
see interfaces that aren't theirs (i.e. their address doesn't live on it).


So we have two PRs that are kind of at cross purposes.  It would be nice to 
keep hiding non-jail interfaces from a jail, but it would also be nice to let


jails have no notion of interfaces, only addresses, so by defintiion
there cannot be non-jail interfaces.


a jailed process know the route to somewhere - at least sometimes.  One 
solution would be to add a much finer layer of control to the jail test in 
rtsock.c, looking at interfaces and seeing if they're somehow connected with 
one of the jail's IP addresses.  But that just seems like a lot of messy 
corner-case code.


Another way around this, and what I'd like to go with if there are no 
objections, is to allow the route sockets to be used by jails that have 
raw_sockets permission.  I know that's kind of a semantic leap, but it seems 
that a jail that has the power of using raw sockets would be able to do 
pretty much as it pleases with routes anyway if it tried hard enough.  Also, 
it would be consistent to allow such operations on jails that aren't 
IP-restricted, or in VIMAGE jails.


I have not further looked at the code but the answer is that we should
not further complicate jails after 14 years when we have a perfect
good solution for the problem;  vnets are there for exactly this.
Someone with enough interest and time should just finish things (tm) ;-)

Meanwhile your suggestion might be ok given simple enough, but I wonder
if a different flag would be helpful still.  I would not be able to
trust (the little that is possible anyway) raw_sockets anymore if they
suddently could fiddle with the routing table - even read-only, should
that really be enough.
I would explicitly advertise it as 'do not use - will go away again'
feature and it should the moment vnets are declared non-experimental.

Just my 2cts.

/bz

--
Bjoern A. Zeeb You have to have visions!
 Stop bit received. Insert coin for new address family.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: IPv6 multicast sent to jail

[Bjoern A. Zeeb]

On Wed, 5 Sep 2012, Curtis Villamizar wrote:



In message alpine.bsf.2.00.1209031219120.76...@ai.fobar.qr
Bjoern A. Zeeb writes:


On Sat, 25 Aug 2012, Jamie Gritton wrote:

...

Curtis


Offhand, it does sound like a bug. I imagine the solution would be to
reject the join - at least the easy solution to be done first until
something more complicated can be done to make jails play nice with
multicast.

- Jamie



Jamie,

Certainly not the preferred solution.  Best would be a
jail.allow-ipv6multicast sysctl variable with rejecting the join if 0
and accepting the join and passing in multicast if 1.  Same for v4,
though not of immediate concern since DHCPv4 doesn't need it.

If you (or someone) would like to point me in the right direction, I
would be willing to put some time into learning the relevant code and
proposing a fix.  No promises, but I can put some time into it.  Off
list if you prefer.

Curtis


It'll have to be someone besides me - I don't know enough about
multicast myself to be able to do more than keep it out of jails.


sysctl souns bad to me;  I think it should actually be grouped by
ip4.* and ip6.*.  What dod we currently do for raw sockets?  Can we
have a third level easily, as in ip4.raw.*, ip6.mc.*, ...  which of
course would kill the classic allow thing for raw sockets myabe?

/bz


For raw sockets the sysctl variable is:

security.jail.allow_raw_sockets

One sysctl variable for both inet and inet6 AF.  Perhaps a reasonable
name would be:

 security.jail.ip4.allow_multicast
 security.jail.ip6.allow_multicast

Just to be clear, I was hoping to get some help if I were to make an
attempt to allow ipv6 multicast through, though I suspect that the
code would be very similar for ipv4.


The sysctls are mostly not relevant anymore but yes, if we can get
these options we can look at the code.  Defaults to off.
I might be able to help on the v6 trailing end.  Jamie could you
prepare the jail options changes for us?

--
Bjoern A. Zeeb You have to have visions!
 Stop bit received. Insert coin for new address family.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Fixed Jail ID for ZFS - need proper mgmt?

[Bjoern A. Zeeb]

On Tue, 4 Sep 2012, Jamie Gritton wrote:


It's true that a jail left in the DYING state can't be re-created
normally. But it can with the -d flag or the allow.dying parameter.
In that case, an existing but dying jail will be re-attached to and this
resurrected. So it can be gotten around, and would be a matter of
education. Or perhaps we could change the default behavior to silently
all re-creation of dying jails. Is there any harm in this? I.e. would
there be any difference noticeable to the user if a jail was created
with some old TCP connections attached to it?


Yes, really bad and TCP is not the only thing in theory.  Assume
your management does not make sure the same users gets the same jail;
you elak a lot of (possibly security related) information.  Would also
make it quite hard in terms of auditing etc. to get this right unless
done knowingly and on purpose.

--
Bjoern A. Zeeb You have to have visions!
 Stop bit received. Insert coin for new address family.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: [patch] etc/rc.d/jail: allow extra parameters for each jails

[Bjoern A. Zeeb]

On Thu, 16 Aug 2012, Jun Kuriyama wrote:



Hi,

Here is a patch which I'm using for years in my production
environment.

I usually changes parameters documented in jail(8) for each jails, but
current rc.d/jail has no feature to pass extra parameters at starting
jails.

I hope this patch will not prevent jamie's recent/next work for
rc.d/jail.  I'll commit this if there is no objection.


Why not just use his work?

--
Bjoern A. Zeeb You have to have visions!
 Stop bit received. Insert coin for new address family.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: [jail] Allowing root privledged users to renice

[Bjoern A. Zeeb]

On 25. May 2012, at 16:48 , Sean Bruno wrote:

 I've been toying with the idea of letting jails renice processes ... how
 dangerous and/or stupid is this idea?
 
  //depot/yahoo/ybsd_9/src/sys/kern/kern_jail.c#5 -
 /home/seanbru/ybsd_9/src/sys/kern/kern_jail.c 
 270a271,275
 + int   jail_allow_renice = 0;
 + SYSCTL_INT(_security_jail, OID_AUTO, allow_renice, CTLFLAG_RW,
 +jail_allow_renice, 0,
 +Prison root can renice processes);
 
 3857a3863,3865
 +  case PRIV_SCHED_SETPRIORITY:
 +  if (!jail_allow_renice)
 +   return (EPERM);


I think sysctls are a bad idea given jails have per-jail flags these days.

Maybe also only allow re-nicing to be nicer but not less nice?

/bz

-- 
Bjoern A. Zeeb You have to have visions!
   It does not matter how good you are. It matters what good you do!

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Jail source address selection broken, patch for ping

[Bjoern A. Zeeb]

On 9. Apr 2012, at 16:20 , Mark Felder wrote:

Hi Mark,

thanks a lot for posting the summary.

 By pure chance I was able to contact bz@ and he provided me with a patch for 
 ping based on his recent work on a similar issue with traceroute. This solved 
 my problem with the system ping utility, but my tests with fping and the ping 
 utility included with our monitoring software still exhibited the same issue.
 
 bz informed me that he believes he knows where the bug is in the kernel -- I 
 believe he pointed me to the area of sys/netinet/ip_raw.c around line 461. 
 Jails are getting the first IP as a source no matter what.

And maybe to confirm - yes I have told a lot of people in the past to try 
telnet or similar thing as ping was special, as it's raw sockets etc.  In 
case you have a PR open about this issue please email me the PR number directly 
(not Cc:ing the list) or ask some FreeBSD committer to assign it to me.

As I had originally left the comment there when committed the multi-IP jail 
source code (or follow-up) and the grief this seems to regularly cause, I will 
try to get it fixed soon:  
http://svnweb.freebsd.org/base/head/sys/netinet/raw_ip.c?annotate=229265#l461

 Anyway, attached is the patch he asked me to post to the mailing list for 
 those that need a workaround for ping. I'm sure fixing this in the kernel 
 will probably require further discussion among those with actual programming 
 skills :-)

It's also available here but it's considered a work-around and prove of concept 
that this really was the issue:
http://people.freebsd.org/~bz/20120407-01-ping-source-addr.diff

/bz

-- 
Bjoern A. Zeeb You have to have visions!
   It does not matter how good you are. It matters what good you do!

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Practical limit to number of jails on a given host?

[Bjoern A. Zeeb]

On 6. Feb 2012, at 20:29 , Doug Barton wrote:

 Howdy,
 
 Thinking about implementing a poor-man's virtualization solution with
 lots'o'jails, and wondering what people think about the practical limits
 of such a system. I realize that part of the answer is going to depend
 on CPU and RAM, so let's assume for the sake of argument that the answer
 to that bit is, Lots of both.
 
 So first question is, is there some sort of hard-coded limit somewhere?
 If not, what is the largest number of jails that you've created
 successfully/reliably on a system, and what are the specs for that system?


Yes, jails provide you 6 9s ... though that's not 99.% but 99 is
the maximum number of jails.  And yes, I have started this many before --
without processes or anything.
It took a couple of days, due to some list handling, which could be
optimized.  You will find that once you get there, you'll have a syscall
which never returns...
You notice once the start loop slows down if you print a . every 100 or 1000.

The machine was a 4 or 8 core amd64 with 8G of memory.

I think I had a slide in
there:  
http://www.bsdcan.org/2010/schedule/attachments/130_2010-bz-the-new-vvorld.pdf

I know if using vnets; you can get the 4k (or more) but I also have reports
that it may not scale.

The other limit you'll run into is the number of PIDs.

And eventually scheduling depending on what you want to do.


 And finally, has anyone run into trouble with a large number of IP
 addresses for the jails? ISTR that way back when, the IP addresses
 associated with a particular interface were stored in a linked list, so
 as you added more you would start seeing O(N) slowdown on a lot of
 network stuff in the kernel.

Yeah, we still do list walks here and there.

/bz

-- 
Bjoern A. Zeeb You have to have visions!
   It does not matter how good you are. It matters what good you do!

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: * Re: * Re: Getting Jail v2 working with 9-stable

[Bjoern A. Zeeb]

On 23. Jan 2012, at 16:26 , Devin Teske wrote:

 On Jan 23, 2012, at 3:36 AM, Denny Schierz linuxm...@4lin.net wrote:
 
 hi,
 
 Am 20.01.2012 um 15:05 schrieb Devin Teske:
 
 Try my vimage rc.d script for this.
 
 http://druidbsd.sf.net/vimage.html
 
 http://druidbsd.sourceforge.net/download/vimage-1.4.tbz
 
 I tried it, but doesn't work. I think, it must be a bug in the SPARC Kernel, 
 because other with I386 and same options  doesn't have the same problems.
 
 
 I don't know if VIMAGE is supported yet on SPARC platform. Maybe someone 
 wants to chime in that's more familiar with which-platforms VIMAGE is 
 supported.

VIMAGE should be arch independent.

-- 
Bjoern A. Zeeb You have to have visions!
   It does not matter how good you are. It matters what good you do!
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: mtr doesn't work in a jail even with security.jail.allow_raw_sockets: 1

[Bjoern A. Zeeb]

On 17. Jan 2012, at 20:41 , Michal Vančo wrote:

 Try with --address option. Address selection doesn't work when mtr is run 
 within jail.

I should or should be fixed.

 Wanted to use mtr to diagnose an issue in a jail
 but it seems it totally fails even with
 security.jail.allow_raw_sockets: 1

which version of freebsd?  Anything newer than incl. 8.0 the systls are not 
what you want anymore; it's per jail flags.

/bz

-- 
Bjoern A. Zeeb You have to have visions!
   It does not matter how good you are. It matters what good you do!

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Passing additional options to jail(8) via rc.conf

[Bjoern A. Zeeb]

On Sep 20, 2011, at 5:25 PM, Moritz Wilhelmy wrote:

 On Tue, Sep 20, 2011 at 16:54:33 +, Bjoern A. Zeeb wrote:
 On Sep 20, 2011, at 3:21 PM, Moritz Wilhelmy wrote:
 
 Please keep me in CC, I am not subscribed to freebsd-jail.
 
 Which is your problem as the real solution is being discussed there and
 is really looking for more eyes.  Search for Jamie's posts in the list
 archive.
 
 I wasn't subscribed to freebsd-questions either, back in July. I had
 some FreeBSDs running before, but I really wasn't reading any mailing
 lists. I thought I should ask -questions first, because it is the most
 busy one, with all the chatter and noise. Guess I will either patch
 jail(8) or do it via rc.local on 8.2 and wait for the release of 9,
 which seems to be quite an improvement in general. I'm also not that
 actively involved in FreeBSD, and I try not to subscribe too many
 mailing lists in general. I thought the PR was the only thing there was,
 and google tricked me into believing this. Maybe someone could close the
 PR in case this is resolved.
 
 My apologies. Thanks to Jamie for adding support for the config file.

It's not in 9 for 9.0.  But we'd appreciate if you'd give it a try:)
If you have questions, posting them to freebsd-jail should give you help.

Bjoern

-- 
Bjoern A. Zeeb You have to have visions!
 Stop bit received. Insert coin for new address family.

___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: ipv6 loopback behaviour inside jail

[Bjoern A. Zeeb]

On Wed, 30 Mar 2011, Rob Evers wrote:


P.S. I can supply any further information needed.


Which verison of FreeBSD are you running?

--
Bjoern A. Zeeb You have to have visions!
 Stop bit received. Insert coin for new address family.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: FTPD not working properly on jail

[Bjoern A. Zeeb]

On Wed, 8 Dec 2010, Redd Vinylene wrote:


On Wed, Dec 8, 2010 at 4:52 PM, Bjoern A. Zeeb 
bzeeb-li...@lists.zabbadoz.net wrote:



a) have you tried without SSL?
b) have you tried ftpd from base?

It pretty much smells like a bug in vsftpd.

Out of curiosity - which version of freebsd is that?

/bz



Greetings!

a) Good question. Just tested - without a doubt, yes it works perfectly
without SSL. But we need SSL though.


application or configuration issue unrelated to jails.

--
Bjoern A. Zeeb  Welcome a new stage of life.
ks Going to jail sucks -- bz All my daemons like it!
  http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/jails.html
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: loopback in jail

[Bjoern A. Zeeb]

On Thu, 11 Nov 2010, Andrei Kolu wrote:


OK, I tried this way:

ezjail config:
export jail_crashtest_ip=194.xxx.yyy.22,127.0.0.1

I have loopback ip address now:
lo0: flags=8049UP,LOOPBACK,RUNNING,MULTICAST metric 0 mtu 16384
   options=3RXCSUM,TXCSUM
   inet 127.0.0.1 netmask 0xff00

But, can't bind anything to 127.0.0.1 anyway. Is this a bug or something?


No, it's intentional. You would bind to your public 194.x.x.x IP.

Quoting from jail(2):

 All connec-
 tions to/from the loopback address (127.0.0.1 for IPv4, ::1 for IPv6)
 will be changed to be to/from the primary address of the jail for the
 given address family.

/bz

--
Bjoern A. Zeeb  Welcome a new stage of life.
ks Going to jail sucks -- bz All my daemons like it!
  http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/jails.html
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Mutiple ipv4 and ipv6.

[Bjoern A. Zeeb]

On Mon, 1 Nov 2010, Peter Ankerstål wrote:

Hi,


Im trying to configure a jail with mutiple ipv6 and ipv4-addresses. One 
ipv4-address and multiple ipv6
works fine but not multiple of both.

...

and others:
Starting jails:/etc/rc.d/jail: WARNING: jail_extract_address: type not 
identified
/etc/rc.d/jail: WARNING: jail_extract_address: type not identified
cannot start jail jailid:
.


can you give us the complete global and configuration for this jail?
It's hard to guess at the moment.

Subsitute the domain name for example.com, the IPv4 prefix with
192.0.2. and the IPv6 prefix with 2001:db8: (which are the
example/docmentation prefixes) or, if you want, send them to me
privately.

/bz

--
Bjoern A. Zeeb  Welcome a new stage of life.
ks Going to jail sucks -- bz All my daemons like it!
  http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/jails.html___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: sysvipc in jails + CURRENT

[Bjoern A. Zeeb]

On Thu, 22 Jul 2010, Isaac Levy wrote:

Hi ike,

long time no see.


I could be doing something stupid, or I've dug up an old bug, =
(http://www.mail-archive.com/freebsd-jail@freebsd.org/msg00859.html).

I cannot get good ol' trusty enforce_statfs to work, allowing me to see =
different mounts from within a jail.

--
The example jail command I'm using, (new-style),
 jail -c path=3D$JDIR host.hostname=3D$JHOSTNAME ip4.addr=3D$INET =
enforce_statfs=3D1 command=3D/bin/sh /etc/rc

I've tried everything- including attempting to change my sysctls over =
and over, (including /etc/sysctl.conf with rebooting).
Interestingly:
The old standard 'security.jail.enforce_statfs' was not something I =
could modify, *until* I put a sysctl value in /etc/sysctl.conf which was =
not 0 (1 or 2 both will let me set the sysctl value once the system is =
booted).
If I have security.jail.enforce_statfs=3D0, to my surprise, I cannot =
change that sysctl on the host system as I would usually expect.
(This is what makes me think this smells like a bug)

My extra mounts are UFS volumes, mounted right into the jail directory, =
(on another ufs volume).

What follows, are just machine stats if anyone wants them?

I'd love any thoughts, urls, no matter how brief...


I am confused but maybe I can help you with some explanation:

1) do not change the sysctl anywhere; that is neither in sysctl.conf
   nor by other magic or by hand.   The default on 8 and 9 should be
   2.  You can check that with sysctl security.jail.enforce_statfs
   still I think.

2) Creating a new jail
 jail -c path=/jail/j1 persist
   I can see:
 jexec 1 mount
192.168.5.1:/zoo/bz/HEAD on / (nfs)
   And
 jls -s -j 1 enforce_statfs
enforce_statfs=2
   confirms the default.

3) modifying the jail:
 jail -m jid=1 enforce_statfs=1
   I can now see:
 jexec 1 mount
192.168.5.1:/zoo/bz/HEAD on / (nfs)
devfs on /dev (devfs, local, multilabel)
192.168.5.1:/zoo/bz on /zoo/bz (nfs)
   And jls confirms that the modfication was successful:
 jls -s -j 1 enforce_statfs
enforce_statfs=1

4) If you lower the default by changing the sysctl, all your jails
   that have a higher level will be lowered as well.

5) But if you up the default again, they won't change back up.


I think that you are right, that there is a bug here, as 4) and 5)
should be working the other way round I think.


Anyway, the summary is: if you don't change the default a
jail -c enforce_statfs=1 ...
should just work fine.


Hope this helps.

/bz

--
Bjoern A. Zeeb   This signature is about you not me.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: configuration of multiple IPs for a jail

[Bjoern A. Zeeb]

On Thu, 28 Jan 2010, t...@diogunix.com wrote:

Hi,


Jase,


This behaviour has been addressed in RELENG_7 recently with r202924 [1].


thank you very much. That's what I was watching out for :-).
I somehow could not find that hint in all the resources I used.


This commit allows you to set : sysctl security.jail.ip4_saddrsel 0 ,
which makes the kernel use the first IP passed to jail (8) as the
default source address instead of the default behaviour which picks the
first matching ip for that jail on the interface.


That's not exactly true.  Source address uses the first matching
address for the destination on the outgoing interface if possible.
There is a route lookup involved as well.  So if you are serving more
than one subnet it won't necessarily be the first IP of the interface
seen within the jail.

For the case given, it most likely will, though.



Just great. I run 7.2 stable on most machines and thanks to your information
it will be much easier than what I meanwhile did to fix things.


A workaround (if you're not able to update to a RELENG_7 following that
commit) is to reorder your interface aliases in /etc/rc.conf ,so that
your primary jail ip has a lower alias # than any secondary ips for that
jail.


Yes. I've meanwhile found exactly that out the hard way and by trial and
error. Works nice (or however, it works), even when the kernel setting method
of course is much more elegant.


Hope this helps,


I did already.


Though it might help, if you only need it for postfix, using the
smtp_bind_address (and smtp_bind_address6) options might be more
elegant rather than using the hammer of forcing things in the kernel.
See man 5 postconf.

If more services across all jails should be using the intended
behavior using the sysctl and kernel is probably the right thing.

/bz

--
Bjoern A. Zeeb It will not break if you know what you are doing.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: kern/142341: [jail] Jail escape when cwd is moved from the host system

[Bjoern A. Zeeb]

The following reply was made to PR kern/142341; it has been noted by GNATS.

From: Bjoern A. Zeeb b...@freebsd.org
To: bug-follo...@freebsd.org, ve...@kajtaz.net
Cc:  
Subject: Re: kern/142341: [jail] Jail escape when cwd is moved from the host
 system
Date: Tue, 5 Jan 2010 19:36:36 + (UTC)

 Hi,
 
 this is the expected behaviour but is probably not explicitly
 documented.  Patches to update the man page are welcome.
 
 -- 
 Bjoern A. Zeeb It will not break if you know what you are doing.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Jail on 2 interfaces?

[Bjoern A. Zeeb]

On Tue, 22 Dec 2009, Mel Flynn wrote:

Hi,

first of all this would find more people to help on freebsd-jail as it
has nothing to do with hackers ;-)


I don't see this documented in jail(8) nor rc(8) nor defaults/rc.conf, so is
it possible to have 2 IP's on 2 ethernet interfaces? And if so, is it settable
for rc(8)?

The usage case is to have the same jailed proxy server on two seperate
internal networks. Ideally, the proxy will use one address for outgoing, so I
guess I'll need a default route or dive into the squid config.

At present I have:
ifconfig_bge0=inet 192.168.177.60  netmask 255.255.255.0
ifconfig_em0=inet 192.168.176.60 netmask 255.255.255.0
ifconfig_em0_alias0=inet 192.168.176.62 netmask 255.255.255.255
jail_squid_rootdir=/usr/squid
jail_squid_ip=192.168.177.62
jail_squid_ip_multi0=192.168.176.62
jail_squid_interface=bge0

But this created the IP on bge0 even though one exists on em0. Is it as simple
as not specifying the interface and add the 177.62 alias on bge0?
Ideally I'd have a jail_$jail_ip_multi$aliasno_interface=foo0, but my main
worry is that the jail infrastructure understands the routing involved.



From what you are writing I assume that you are on FreeBSD 7.2-Release

or later; no official FreeBSD version before had supported
multiple-IPs with a jail.

What it did was what you were asking for.  That's the problem.

1) either use ifconfig
2) or use jail + interfaces
3) but do not mix them (especially not overlapping)

So I would suggest to do it like this:

# Base system IPs.
ifconfig_bge0=inet 192.168.177.60/24
ifconfig_em0=inet 192.168.176.60/24

jail_squid_rootdir=/usr/squid
# Either use:
jail_squid_ip=bge0|192.168.177.62/32,em0|192.168.176.62/32
# or:
jail_squid_ip=bge0|192.168.177.62/32
jail_squid_ip_multi0=em0|192.168.176.62/32

but do not use jail_squid_interface=.. as that will be a global
default for that jail.

As you can see, I removed the ifconfig_em0_alias0 line.  If you want
to keep that and mix things then you could do:

jail_squid_ip=bge0|192.168.177.62/32
jail_squid_ip_multi0=192.168.176.62/32

again without the jail_squid_interface=.. line.


HTH
/bz

--
Bjoern A. Zeeb It will not break if you know what you are doing.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: ioctl call freebsd 7.2 in jail

[Bjoern A. Zeeb]

On Fri, 18 Dec 2009, Axel Scheepers wrote:

Hi,


We run Direct Admin in a jailed setup on FreeBSD 7.2 (last supported
release for Direct Admin). We have a problem retrieving the correct ip
address of the jail which makes Direct Admin misbehave.

..

Would it be trivial to backport just the changes which changed this
behaviour and if so how can I get a patch for the changes to
7.2-RELEASE?

I've read that under FreeBSD it's better to use getifaddrs(3) but we
can't change the ioctl call.


I think I remember the patch; I guess it was the samba patch.  I can
extract it for you; not sure if it'll work easily w/o the other
infrastructure but I'll see what I can do.

I can see no chance that it'll ever make it into 7.1 as an Errata
Notice though, so you would have to keep patching your system
yourself.

/bz

--
Bjoern A. Zeeb It will not break if you know what you are doing.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: ioctl call freebsd 7.2 in jail

[Bjoern A. Zeeb]

On Fri, 18 Dec 2009, Axel Scheepers wrote:

hi,


Bjoern A. Zeeb bzeeb-li...@lists.zabbadoz.net writes:


I think I remember the patch; I guess it was the samba patch.  I can
extract it for you; not sure if it'll work easily w/o the other
infrastructure but I'll see what I can do.

I can see no chance that it'll ever make it into 7.1 as an Errata
Notice though, so you would have to keep patching your system
yourself.



Hi Bjoern,

I know it won't make it as an errata, that's not a problem. I'm looking
for a way to fix the problem without having to upgrade all our virtual
hosts to 7.2. The machines in question run a mix of freebsd versions, if
it's not to much trouble it's highly appreciated if you can send me some
more information about the patch you mention. Is this the multi-ip jail
patch you talked about in
http://lists.freebsd.org/pipermail/freebsd-jail/2008-October/000488.html


well, I think that was the earlier workaround; the multi-IP jail
includes one of the patches that actually fix the problem; but the fix
for this case really just is a fraction of the change.

What I was thinking was this commit to head and the PR mentioned
there:
http://svn.freebsd.org/viewvc/base?view=revisionrevision=186948

Note that 7.1 or earlier won;t have the prison_check_ip4() function,
different byte order requirements etc (and not IPv6 support).
It's s pretty short thing but has to be converted correctly.

I might be able to do that later today.

/bz

--
Bjoern A. Zeeb It will not break if you know what you are doing.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: ezjail with vimage

[Bjoern A. Zeeb]

On Mon, 7 Dec 2009, Miroslav Lachman wrote:

Hi Miroslav,

The last time I wrote with Bjoern A. Zeeb about jailname, cpuset etc. support 
in rc.conf (back in March 2009) he stated that there is no need to add 
anything because it can be done by jail_NAME_flags.
AFAIK current system still doesn't allow me to set cpuset to jail from 
rc.conf


Check /etc/defaults/rc.conf for jail_example_exec_afterstart.

/bz

--
Bjoern A. Zeeb It will not break if you know what you are doing.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: ezjail with vimage

[Bjoern A. Zeeb]

On Mon, 7 Dec 2009, Miroslav Lachman wrote:


Bjoern A. Zeeb wrote:

On Mon, 7 Dec 2009, Miroslav Lachman wrote:

Hi Miroslav,


The last time I wrote with Bjoern A. Zeeb about jailname, cpuset etc.
support in rc.conf (back in March 2009) he stated that there is no
need to add anything because it can be done by jail_NAME_flags.
AFAIK current system still doesn't allow me to set cpuset to jail from
rc.conf


Check /etc/defaults/rc.conf for jail_example_exec_afterstart.


You already said that in the past and it was the reason why I found bug in 
cpuset.

http://lists.freebsd.org/pipermail/freebsd-jail/2009-April/000830.html

As I said, exec_afterstart is executed inside the jail and it means that I 
can not use it to bind the jail to specific CPU cores.

...but maybe I am blind.

Can you correct me if I am wrong?


*mumble*  *tired*  *again* ..

Let me cite man rc.conf to not mess it up again:

 jail_jname_exec_afterstartN
 (str) Unset by default.  This is the command run as Nth com-
 mand in a jail after jail startup, where N is 1, 2, and so
 on.

 jail_jname_exec_poststartN
 (str) Unset by default.  This is the command run as Nth com-
 mand after jail startup, where N is 0, 1, and so on.  It is
 run outside the jail.

HTH

/bz

--
Bjoern A. Zeeb It will not break if you know what you are doing.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: [patch] Improved jail fstab functionality inside rc.d (needs testers and review)

[Bjoern A. Zeeb]

On Sun, 29 Nov 2009, Merijn Verstraaten wrote:

My apologies if these are the wrong lists for this sort of thing but it was 
unclear to me where else to go with additions like this.


You may try freebsd-jail@

Make sure to get a review from simon@ for this.

/bz

--
Bjoern A. Zeeb It will not break if you know what you are doing.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Networking from jail - errata

[Bjoern A. Zeeb]

On Tue, 17 Nov 2009, Scheithauer, Lars (FH) wrote:

Hi,


Quick note:
Forgot to replace two values.
Jail - x.y.z.61
Host - x.y.z.60
Router - x.y.z.62


-Ursprüngliche Nachricht-
Von: owner-freebsd-j...@freebsd.org [mailto:owner-freebsd-j...@freebsd.org] Im 
Auftrag von Scheithauer, Lars (FH)
Gesendet: Dienstag, 17. November 2009 10:19
An: freebsd-jail@freebsd.org
Betreff: Networking from jail

Hi everyone!

I'm having a little trouble with my jail's networking and I'm not sure
what to make of it.

My jailhost has an IP of x.y.z.48, my test jail is x.y.z.49. The
jailhost has both IP-adresses, the jail has just it's own:

Jail# ifconfig
bce0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu
1500

options=1bbRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,
TSO4
   ether xx:xx:xx:xx:xx:10
   inet x.y.z.60 netmask 0xffc0 broadcast x.y.z.63
   media: Ethernet autoselect (1000baseSX full-duplex)
   status: active
[...]
Host# ifconfig
bce0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu
1500

options=1bbRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,
TSO4
   ether xx:xx:xx:xx:xx:10
   inet x.y.z.61 netmask 0xffc0 broadcast x.y.z.63
   inet x.y.z.60 netmask 0xffc0 broadcast x.y.z.63
   media: Ethernet autoselect (1000baseSX full-duplex)
   status: active
[...]

I am able to access the ssh-server running on the jail, and I am able to
access the proxyserver of our network via telnet and get some pages of
the internet. However, if I want to install something from the ports,
the jail is unable to fetch it:

Jail# cd /usr/ports/ftp/wget
Jail# make
===  Vulnerability check disabled, database not found
===  Found saved configuration for wget-1.11.4_1
= wget-1.11.4.tar.bz2 doesn't seem to exist in /usr/ports/distfiles/.
= Attempting to fetch from http://ftp.gnu.org/gnu/wget/.
fetch: http://ftp.gnu.org/gnu/wget/wget-1.11.4.tar.bz2: Operation timed
out
= Attempting to fetch from ftp://ftp.gnu.org/gnu/wget/.
[...]

I've set the appropriate environment variables HTTP_PROXY, HTTPS_PROXY
and FTP_PROXY. If I test the connection with netcat, I get the following
error message:
# nc -zvw 1 -x 'proxy.example.com:8080' www.freebsd.org 80
nc: read failed (0/3): Broken pipe


The usual thing I am interested at that point is - does name
resolution work properly from within the jail?  /etc/resolv.conf setup
correctly etc?




The funny thing is, that I have no problem installing ports from the
Host-system. From what I can tell, all the config files are correct:

Jail# cat /etc/rc.conf
sshd_enable=YES
ifconfig_bce0=inet x.y.z.60 netmask 255.255.255.192
defaultrouter=x.y.z.62
hostname=jail.example.com


That's not going to work, really (the ifconfig, defaultrouter, and
unless you changed the defaults on the host system not even the
hostname).  You should actually remove those.



Host# cat /etc/rc.conf
sshd_enable=NO
ifconfig_bce0=inet x.y.z.61 netmask 255.255.255.192
defaultrouter=x.y.z.62
hostname=host.example.com
ipv6_enable=NO
jail_enable=YES
jail_set_hostname_allow=NO
jail_list=jail
jail_jail_hostname=jail
jail_jail_ip=x.y.z.60
jail_jail_rootdir=my/jail/root
jail_jail_devfs_enable=YES


That doesn't really match your ifconfig output from above; something
on the host system would have to set the IP address of the host. I
would expect something like (you may have mixed jail and host
addresses so properly sort this):

# host system IP address
ifconfig_bce0=inet x.y.z.61 netmask 255.255.255.192
# jail IP address
ifconfig_bce0_alias0=inet x.y.z.60 netmask 255.255.255.255

Note that the alias has a /32 netmask.


/bz

--
Bjoern A. Zeeb It will not break if you know what you are doing.___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: AW: Networking from jail - errata

[Bjoern A. Zeeb]

On Tue, 17 Nov 2009, Scheithauer, Lars (FH) wrote:

Hi,


thanks for the clarification, I changed the values according to your 
suggestions. However, it did not resolve the problem.


Did you aslo check resolv.conf inside the jail?
Does host www.freebsd.org work?



I've checked the proxy logfiles and it seems, that the Makefile(s) don't try to 
access the proxy at all while fetching files. Is there any reason, why the 
Makefile(s) should not use the *_PROXY-variables on the jails?


I assume the proxy is squid and that the proxy itself works?
What if you set the http_proxy variables to an IP address rather than
the name (don't use 127.0.0.1 as address, just to rule that out as
well).

/bz

--
Bjoern A. Zeeb It will not break if you know what you are doing.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Broadcast under Jail problems

[Bjoern A. Zeeb]

On Mon, 16 Nov 2009, Vagif Zeynalov wrote:

Hi,


...I can provide more details if it will be necessary...


error ogs from the application would be interesting to see which
(sys)call return which error so that we can narrow it down.

/bz

--
Bjoern A. Zeeb It will not break if you know what you are doing.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Setting the jail identifier from /etc/rc.conf

[Bjoern A. Zeeb]

On Thu, 22 Oct 2009, Ed Schouten wrote:


Hi,

I haven't played with Jails for a long time, but I wanted to figure out
how hard it is to make init spawn getties for certain jails. It
shouldn't be too hard (jexec foo /usr/libexec/getty), but I can't seem
to find a way to set the jid to a certain value from within rc.conf.

It also seems jids cannot contain dots, which means I cannot set the jid
equal to the hostname of the jail.

Maybe a Jail hacker can give me some advice here? Wouldn't it be more
sane if the kernel just used the hostname as an identifier if there is
no jail with the same hostname yet? Or maybe we should at least provide
a config tunable for this?


Redirect to freebsd-jail@  ; you may even find the answers to those
int he mail archive (unless those had been private threads I was on
Cc: on;-)


--
Bjoern A. Zeeb It will not break if you know what you are doing.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Per Jail Memory Limits

[Bjoern A. Zeeb]

On Mon, 5 Oct 2009, Julian Elischer wrote:


Tom Judge wrote:

Adam Vande More wrote:
On Mon, Oct 5, 2009 at 12:47 PM, Tom Judge t...@tomjudge.com 
mailto:t...@tomjudge.com wrote:


Julian Elischer wrote:

Tom Judge wrote:

Hi,

Does anyone know of a patch that will add per jail memory
limits so that a jail can't swallow the resources of the
entire box?


Thanks

Tom

not yet..


I started to port this to 7.1 today:

http://wiki.freebsd.org/JailResourceLimits


What are the peoples opinions on this patch?


Tom


If you're soliciting opinions if this will be used and is needed, I would 
love to see this functionality.  This is the main reason I've had to chose 
XEN over jails.  If you need some help testing, let me know.


--
Adam Vande More

Hi Adam,

I have a patch against 7.1 here: 
http://svn.tomjudge.com/freebsd/patches/jail-resource-limits/jail-limits.patch 




probably the person who should work with this in -current is james (CC'd)


Probably the person who should be contacted is trasz who worked on
hierachical resource limit per .., jail in p4.  Though this is
slightly different.

I think it's ok if people need those things to update the pathes but I
doubt any will probably ever make it into FreeBSD as those things are kind
of contrary to the V_ plans.

BTW, I think the patch referenced is not the latest I had seen and I
thought that we also had one for 7.x or even for 8 already floating around.
Maybe some investigation on list archives etc. might be helpful before
starting to hack things.  Maybe also check the links on
http://wiki.freebsd.org/Jails





I will try to bring the patch up to current when I get a chance but I have 
no real need to do this as we use 7.1 in production.


Notes:

   * CPU limiting is not support is not supported unless you use 
shecd_4bsd.
   * I have not tested this on any system yet, just compile tested, I am 
putting it though its paces right now.


Tom


--
Bjoern A. Zeeb It will not break if you know what you are doing.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Not getting an IPv6 in a jail

[Bjoern A. Zeeb]

On Tue, 1 Sep 2009, Major Domo wrote:

Hi,


Apologies if this has been discussed already but I searched the web
and the mailing lists and haven't found hints on my problem.

I've got a jail, I assign it a set of IP addresses, and it just won't
take the IP6 I give it.


Uname:
FreeBSD 7.2-STABLE

jail_ns_ip=192.168.0.252,fe80::c0a8:fc

jls -v:
  JID  Hostname  Path
   Name  State
   CPUSetID
   IP Address(es)
   23  [snip]  /var/jail/ns
 ALIVE
   2
   192.168.0.252
   fe80::c0a8:fc


ifconfig lo252 from the host:
lo252: flags=8049UP,LOOPBACK,RUNNING,MULTICAST metric 0 mtu 16384
   inet 192.168.0.252 netmask 0x
   inet6 fe80::c0a8:fc%lo252 prefixlen 128 scopeid 0x5


ifconfig from the jail:
re0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500
   
options=389bRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC
   ether 00:e0:f4:19:e9:d2
   media: Ethernet autoselect (100baseTX full-duplex)
   status: active
lo0: flags=8049UP,LOOPBACK,RUNNING,MULTICAST metric 0 mtu 16384
pflog0: flags=141UP,RUNNING,PROMISC metric 0 mtu 33204
lo252: flags=8049UP,LOOPBACK,RUNNING,MULTICAST metric 0 mtu 16384
   inet 192.168.0.252 netmask 0x



This is a rather special case.  For link-local addresses you have to
give the scope as well but it won't take the scope with the %lo252
notation but only in the KAME in-kernel syntax I would assume.
Can you try:

jail_ns_ip=192.168.0.252,fe80:5::c0a8:fc

Note the added 5 in the second group of hex digits.  That five is the
interface index.  I took it from the scopeid 0x5. In case your
interface index changes you will need to adjust the address.

I cannot say if it'll work but it would be worth a try.

/bz

--
Bjoern A. Zeeb   What was I talking about and who are you again?
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: 8.0 still allow creating ipv6 udp socket in jail without ipv6 ip

[Bjoern A. Zeeb]

On Sat, 25 Jul 2009, Mykola Dzham wrote:

Hi,


After r188146 creating tcp ipv6 socket in jail without ipv6 ip is not
allowed, but udp socket is allowed.


I cannot really follow what you are trying to say as wrt IPv4 and IPv6
sockets and what about UDP.

Your sample further down is trying to use an IPv4 address on an IPv6
Datagram socket which is an error either way.

Prior to FreeBSD 7.2 IPv6 hadn't been supported at all for jails.

With 7.2 it was possible to create IPv6 sockets (but only shortly and
then fail on bind/connect/...).  With the commit you reference the
Protocol not supported came back in case there was no address of
that address family for a given jail.

With 8 the primary syntax for jails has changed and the backward
compat mode again allows you to create a socket on a jail even if
no address of the same family was configured for the jail.

This should be addressed by the following patch:
http://people.freebsd.org/~bz/20090727-01-jail8-legacy.diff

Can you give it a try and report if that fixes your problem?

Regards,
Bjoern

--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Multicast in jail?

[Bjoern A. Zeeb]

On Tue, 7 Jul 2009, Alexander Leidinger wrote:

Quoting Bill Marquette bill.marque...@ucsecurity.com (from Mon, 6 Jul 2009 
20:14:02 -0500 (CDT)):


I'm trying to run Avahi in a jail, much the same as Alexander Leidinger in 
this email from late last year 
http://www.mail-archive.com/freebsd-jail@freebsd.org/msg00587.html.  I 
couldn't find any replies to that thread and it seems that I'm running into 
the same issues - the service announcements make it on the wire and the 
other devices in the network see them.


So far I have nothing working.

I assume that the mcast traffic is not arriving at all IPs. wild 
guessEither because on overly restrictive jail check, and/or just because 
there's no code which is distributing the traffic to all IPs./wild guess


It seems kern_jail.c is a place to check if there's some code which handles


No, in_pcb.c in6_pcb.c in_m*.[ch] in6_m*.[ch] are the files you need
as a starting point; there's more and more and more.

this. Maybe prison_check_ip[46] if mcast is on top of this, or something new 
to write if mcast is a different AF. Again, this is a wild guess, I don't 
have enough understanding of the network code in the kernel to even make 
educated guesses about the real reason.


But you first will have to understand all implications, that need a
proper design plan and after that thoughtout implementation.

Alternatively I wouldn't wonder if enabling raw sockets would give
what you want or you'll wait for virtualization to be ready.

/bz

--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Can't login Jailed system

[Bjoern A. Zeeb]

On Mon, 29 Jun 2009, Sam Wun wrote:

Hi,

we've got a freebsd-jail list that I am Cc:ing.



With FreeBSD 7.2Stable,
I have done this many times before.
After about a month left the jail behind, now when I done a
/etc/rc.d/jail start and ssh into it, I ended up login to the host
system.
Here is the network configuraiton of the host system and the jail system:

# ifconfig
rl0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500
   options=8VLAN_MTU
   ether 00:00:21:ef:27:f7
   media: Ethernet autoselect (100baseTX full-duplex)
   status: active
rl1: flags=8802BROADCAST,SIMPLEX,MULTICAST metric 0 mtu 1500
   options=8VLAN_MTU
   ether 00:50:fc:65:78:c0
   media: Ethernet autoselect
   status: no carrier
fxp0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500
   options=8VLAN_MTU
   ether 00:13:20:65:a9:be
   inet 192.168.1.246 netmask 0xff00 broadcast 192.168.1.255
   inet 192.168.1.245 netmask 0xff00 broadcast 192.168.1.255
   inet 192.168.1.235 netmask 0xff00 broadcast 192.168.1.255
   inet 192.168.1.242 netmask 0x broadcast 192.168.1.242
   media: Ethernet autoselect (100baseTX full-duplex)
   status: active
plip0: flags=108810POINTOPOINT,SIMPLEX,MULTICAST,NEEDSGIANT metric 0 mtu 1500
enc0: flags=0 metric 0 mtu 1536
pflog0: flags=141UP,RUNNING,PROMISC metric 0 mtu 33204
pfsync0: flags=0 metric 0 mtu 1460
   syncpeer: 224.0.0.240 maxupd: 128
lo0: flags=8049UP,LOOPBACK,RUNNING,MULTICAST metric 0 mtu 16384
   inet6 fe80::1%lo0 prefixlen 64 scopeid 0x8
   inet6 ::1 prefixlen 128
   inet 127.0.0.1 netmask 0xff00
twp1:# jls
  JID  IP Address  Hostname  Path
5  192.168.1.242   twp5.ip6.com.au   /usr/jail2/twp5

192.168.1.242 is the jailed system,
twp1 is the host system.

After I login 192.168.1.242, I ended up logged in twp1 which is my host system.
Now I am stuck. I don't know how I logged in the jailed system a month ago.

Can anyone shred some lights on me?


Try to jexec 5 /bin/sh (5 is the jailID from the jls output)  and check
with ps if sshd is running inside the jail, and check the usual things
are up and there.


/bz

--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Switching /etc/rc.d/jail to new syntax (+ new features)

[Bjoern A. Zeeb]

On Sat, 27 Jun 2009, Alexander Leidinger wrote:


at http://www.leidinger.net/FreeBSD/current-patches/jail.diff I have a
patch to switch the jail rc script to the new jail (8-current) syntax.
This includes new config options for a jail (see etc/defaults/rc.conf
after patching). The patch also contains my X-in-a-jail stuff (feel
free to ignore this part, it's disabled by default).

If you do not make any config change, you will be able to see all
mounted filesystems of the entire machine. To get back to the previous
behavior, you have to add a config option:
 jail_XXX_startparams=enforce_statfs=2

This config option can also take other jail parameters like
allow.sysvipc and other ones described in the jail man-page (additional
parameters need to be space separated).

Feedback welcome.


1) it break various things that will no longer work
2) it's not a poper solution

/bz

--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: bind()/sendto() behavior in RELENG_7

[Bjoern A. Zeeb]

On Fri, 8 May 2009, Brian A. Seklecki wrote:

Hi,


All:

Did the behavior of bind()/sendto() functions WRT jails change in
proximity to the RELENG_7_2 branch?

I just spent 1.5 days chasing, what I thought was a bug in Courier-MTA's
IPv6 socket selection code within Jails, to realize a paradox of a
configuration scenario:

My ESTMP client libraries in Courier were programed to explicitly bind()
to a specific source address.  The system in question was RELENG_7 from
last month; but was upgraded to 7.2-R last week, when this problem was
observed.  After which, I began to receive:
  Can't assign requested address, as expected.

Unfortunately, we also enabled IPv6 on the system at the same time,
complicating troubleshooting.

The configuration for Courier in the jail is being rsync(1)'d every hour
from a production environment (where explicit binding for System-Service
abstraction is a security policy requirement) to a DRP system within a
Jail.

So as far as I know, the explicit bind was always present in the DRP
jail and in theory, should never have worked.

I hypothesize that after 7.2-R was installed, the correct behavior of
bind() began to occur, and that prior to that, it was gracefully
allowing Courier to bind() to an IP that wasn't present in the jail.

Unfortunately, I don't have any records of what the RELENG_7 build date
was of the original jail environment to test this hypothesis.


So I am having trouble understanding the actual problem with what on
which system what fails and enough things are coming together So let
me ask a few questions/explain:

1) Had you been running the multi-IP jail work on the 7-STABLE before
already?

2) In the past you did bind to an IPv4 address and the same address
worked on machines even if the IP wasn't there. Right?

3) Now you switched on IPv6 as well 2) no longer works?

4) can you give me the output of sysctl net.inet6.ip6.v6only ?

/bz

--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: changing cpuset of jail from inside of jail - is it feature?

[Bjoern A. Zeeb]

On Wed, 22 Apr 2009, Miroslav Lachman wrote:

Hi,


Bjoern A. Zeeb wrote:


On Wed, 22 Apr 2009, Miroslav Lachman wrote:

Hi,

I am running system FreeBSD 7.1-STABLE amd64 GENERIC (Wed Feb 11 09:56:08 
CET 2009) hosting few jails.
The machine has dual core CPU and some jails are set to run only on one 
core (core 0 in this example):


   host# cpuset -l 0 -j 25

As I tested today, root user inside the jail can change this by the same 
command as I am doing it from the host system:


  injail# cpuset -l 0,1 -j 25

And from now, jail with JID 25 is running on both cores.

Is it expected behavior of cpuset to allow user inside the jail change 
cpuset of the jail itself or is it a bug?


It seems to me as undesirable.



it is (undesirable) and it seems to be a bug as even if you do

host# cpuset -l 0 -r -j 25

you can get back to 0,1 from within the jail.

I'll check how/why this is possible.

/bz

PS: moving this to freebsd-jail@


Ok, I am not sure what is going wrong here; well I know but I don't
know if it's intended in cpuset.  Trying to talk to the right people
but they seen to be AWOL atm.


If you are brave, you could try:

http://people.freebsd.org/~bz/20090423-01-cpuset-jails.diff

I haven't even compiled it yet. It may work, it may not work, it may
make your machine panicing, ... just to warn you.

it should still allow you to create further sets within a jail but you
should not be able to change the root set of the jail from inside
the jail anymore (in case it works;)

/bz

--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Regarding multi-ip Bjoern head patch

[Bjoern A. Zeeb]

On Wed, 22 Apr 2009, Vivek Gite wrote:


Hi,

I'm running FreeBSD 7.1_AMD_P4 release with 4 jails. Recently, our ISP 
provided us IPv6 and I'd like to use both multiple IPv4 and IPv6 for my 
jails. According to FreeBSD wiki (http://wiki.freebsd.org/Jails) - there is a 
patch  http://svn.freebsd.org/viewvc/base?view=revisionrevision=188281 ; 
which  is committed to FreeBSD. But I'm not able to use it under said 
version. So I'm looking to grab this one and manually patch it up. Is there 
any tar-ball to grab a patch? Is it included in FreeBSD 7.2RC1? How do I grab 
HEADS UP r185435?


Yes, all of FreeBSD 7.2 (BETA, RC1, upcomig RC2 and RELEASE) have and
will have it.  So if you are going to update your system to any of
those versions you'll have it.

/bz

PS: in case of reply please remove the -virtualization Cc:

--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: lo0's IPv6 address overwritten

[Bjoern A. Zeeb]

On Fri, 10 Apr 2009, JAKO Andras wrote:

Hi,


This works fine here too.


Good.


I wonder what's going onfor you.  Can you check with
netstat -rn -f inet6
that what you are seeing is indeed true?


It's always the same:


I cannot see the /128 on lo0 so that's fine too.



Can you try starting the jail to get an interactive shell and not
running any scripts like I did and check what happens then?


That works, and ifconfig doesn't show any change on lo0.


Good, as I said above.



I found that when I start a telnet in the jail to an arbitrary global IPv6
address, lo0's ::1 changes to the jail's IPv6 address. The routing table
doesn't change.


telnet to where? To the jail IP? To an IP of the base system? To world?


Which version of RELENG_7 are you on (as what does a few days mean)?

/bz

--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: lo0's IPv6 address overwritten

[Bjoern A. Zeeb]

On Fri, 10 Apr 2009, JAKO Andras wrote:


I found that when I start a telnet in the jail to an arbitrary global IPv6
address, lo0's ::1 changes to the jail's IPv6 address. The routing table
doesn't change.


telnet to where? To the jail IP? To an IP of the base system? To world?


I started telnet inside the jail, to the world:

[r...@splash /usr/home/goya]# jail -l -U root -i /usr/jail/ro.noc
r-noc.net.bme.hu 2001:738:2001:1000::2 /bin/sh
1
# ifconfig lo0 ; ifconfig lo1
lo0: flags=8049UP,LOOPBACK,RUNNING,MULTICAST metric 0 mtu 16384
lo1: flags=8049UP,LOOPBACK,RUNNING,MULTICAST metric 0 mtu 16384
   inet6 2001:738:2001:1000::2 prefixlen 128
# telnet 2001:738::abcd
Trying 2001:738::abcd...
^C#
# ifconfig lo0 ; ifconfig lo1
lo0: flags=8049UP,LOOPBACK,RUNNING,MULTICAST metric 0 mtu 16384
   inet6 2001:738:2001:1000::2 prefixlen 128
lo1: flags=8049UP,LOOPBACK,RUNNING,MULTICAST metric 0 mtu 16384
   inet6 2001:738:2001:1000::2 prefixlen 128


*wow*, that's indeed ... confusing. I'll try to (get someone to) look
into this.

/bz

--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Adding ips to running jail.

[Bjoern A. Zeeb]

On Thu, 2 Apr 2009, Peter Ankerstål wrote:


Is it possible to add ip-addresses to an already running jail?


Not yet but possibly soon (in FreeBSD 8).

--
Bjoern A. Zeeb  The greatest risk is not taking one.___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Patching for multi-ip.

[Bjoern A. Zeeb]

On Tue, 24 Mar 2009, Peter Ankerstål wrote:

Hi,

Im running FreeBSD 7.1-RELEASE-p2 and want to upgrade my jail for 
multi-ip-support.
But I cant find an easy way to to this? Is the simplest way just to build a 
new world with
RELENG_7? I would really appreciate a guide or simple directions to get this 
without

building world.


there is no way w/o building a world and a kernel or waiting another
few days for 7.2-{BETA,RC*,RELEASE} which will have all this.

--
Bjoern A. Zeeb  The greatest risk is not taking one.___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Problems with Jails and Samba3

[Bjoern A. Zeeb]

On Wed, 18 Mar 2009, Felipe Carlo wrote:


Hello,

I am new in this mailing-list, and I have one problem with Samba in a Jail.

When I try to start the samba I have this message:

# /usr/local/etc/rc.d/samba.sh start
%%WINBIND%%#: not found
%%WINBIND%%#winbindd_enable=YES: not found
.: Can't open %%RC_SUBR%%: No such file or directory

This is the rc.conf of the jail:

# cat /etc/rc.conf
hostname=teste3
network_interfaces=
sshd_enable=YES
rpcbind_enable=NO
sendmail_enable=NONE
samba_enable=YES
nmbd_enable=YES
smbd_enable=YES
winbindd_enable=YES

What I can do to fix this??

Thank you !!!


That sounds like the port has a problem. I'd try to mail ports@ and
the port maintainer. You'll find it listed here:
http://www.freebsd.org/cgi/ports.cgi?query=samba-3.0stype=namesektion=all

/bz

--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Problem using bz's multi-IP/IPv6/No-IP Jail Patch (7-STABLE)

[Bjoern A. Zeeb]

On Mon, 9 Mar 2009, Jamie Gritton wrote:


Kage wrote:


Encountering more issues now.  Binding just an IPv6 address to a jail
shows up in jls -v, but when I run ifconfig -a in the jail, I get an
error I've never encountered, and doesn't show up on any Google
search:

[r...@nub:/etc] jls -v
   JID  Hostname  Path
Name  State
CPUSetID
IP Address(es)
 9  jail.template.tld /usr/jails/TEMPLATE
  ALIVE
10
2610:150:c248:dead:beef:c0ff:eec0:deaa

[r...@jail:/] ifconfig -a
ifconfig: socket(family 2,SOCK_DGRAM): Protocol not supported


Recent patches reject sockets in jails that have no addresses in the
socket's family.  So if you jail has no IPv6 addresses, you won't be
able to create any IPv6 sockets.  Likewise your case: if that jail has
no IPv4 addresses, then it's an IPv4-less jail, and IPv4 sockets won't
work (Protocol not supported).  For actual network connections, this
makes sense: you won't be able to bind or connect with this socket, as
there are no IPv4 addresses in the system.

But ifconfig is a different situation.  It just needs a socket of some
sort, and AF_INET has always worked, because any networked system always
has IPv4 support.  But in an IPv4-less system (which an IPv4-less jail
not acts like), this default isn't useful.  Something will need to be
fixed.  I'm not sure if that something is ifconfig or the kernel.


I'd suggest fixing ifconfig if (easily) possible; that would avoid us
running into it again in a few months/year(s) when it might be
possible to compile an INET6 but no INET kernel.

--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: BIND in jail problem

[Bjoern A. Zeeb]

On Sat, 14 Feb 2009, Anders Hagman wrote:

Hi,

I am inclined to say that something is not right with your setup and I
am not able to reproduce any of the symptoms on 7-STABLE pre-jail-MFC
but that's not going to help.

Those named inside jail things come up regularly and either end
without any results as people stop to reply or a pilot error quickly
identified. It might be hard to resolve the problem in mail or might
need lots of mails so I'd suggest to take your reply off-list, and
we'll post a summary with the results once things are solved.



I'm trying to use BIND inside a jail and have passed the chroot
problem and have a running named without chroot.


what does netstat -an | grep '\.53' say inside your jail?



The problem is that the jail does not have the address 127.0.0.1 or does not


that's becoming a FAQ and later jail2 man pages say:

 :: All connections to/from the loopback address (127.0.0.1 for IPv4, ::1
 :: for IPv6) will be changed to be to/from the primary address of the jail
 :: for the given address family.

so for your jail (I assume a stock 7.1-RELEASE) ignore the IPv6 part
and the primary part as there is only one IP (which is the primary
IP in that case).



use the info in resolv.conf.

When I use the host command I get:

[r...@ippbx1 ~]# host ippbx1
;; reply from unexpected source: 172.16.101.3#53, expected 127.0.0.1#53

/etc/resolv.conf
domain kalmar.se
search kalmar.se


man resolv.conf says:

 :: The domain and search keywords are mutually exclusive.  If more than one
 :: instance of these keywords is present, the last instance will override.

so you can remove the domain line.



nameserver 127.0.0.1

tcpdump:
21:33:49.569332 IP (tos 0x0, ttl 64, id 31390, offset 0, flags [none], proto 
UDP (17), length 52) 172.16.101.3.62278  172.16.101.3.53: 28477+ A? ippbx1. 
(24)


21:33:49.569890 IP (tos 0x0, ttl 64, id 31393, offset 0, flags [none], proto 
UDP (17), length 52) 172.16.101.3.53  172.16.101.3.62278: 28477 ServFail 
0/0/0 (24


This looks fine from the IP point of view as if 172.16.101.3 is our
jail IP is correct.


As you can see the destination address is 172.16.101.3 despite the name 
server address in resolv.conf. The host command does not add the domain as it 
should and sends the query as A? ippbx1 instead of A? ippbx1.kalmar.se. 
The host command expects to get an answer from 127.0.0.1.


I am not yet sure where this comes from but if that's really a problem
change it to
nameserver 172.16.101.3
as this is what it is effectively anyway.


Changing the nameserver address in resolv.conf to 172.16.101.3 does not 
change anything. Using the FQDN does not help because it's still the wrong 
expected address.


Now that does not make any sense. You changed the IP but it still
reporting the reply from unexpected source: ... expected ..?


 The only thing that works is: host ippbx1.kalmar.se 
172.16.101.3.


Using ping give a different picture:


You enabled raw sockets for jails?



[r...@ippbx1 ~]# ping ippbx1
ping: cannot resolve ippbx1: Host name lookup failure

/etc/resolv.conf
domain kalmar.se
search kalmar.se
nameserver 172.16.101.3


tcpdump:
21:47:39.143152 IP (tos 0x0, ttl 64, id 31817, offset 0, flags [none], proto 
UDP (17), length 62) 172.16.101.3.60878  127.0.0.1.53: 35805+ A? 
ippbx1.kalmar.se. (34)
21:47:39.143165 IP (tos 0x0, ttl 64, id 31818, offset 0, flags [none], proto 
ICMP (1), length 56) 127.0.0.1  172.16.101.3: ICMP 127.0.0.1 udp port 53 
unreachable, length 36



ping does add the domain to the query but does not read the address from 
resolv.conf and sends the query to 127.0.0.1. And 127.0.0.1 is the host 0 
machine and does not run BIND.


I start wondering if you are editing the correct resolve.conf inside
the correct jail and run your commands inside the same jail?

/bz

--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: HEADS UP: multi-IPv4/v6/no-IP jails merge to 7-STABLE ahead

[Bjoern A. Zeeb]

On Wed, 28 Jan 2009, Bjoern A. Zeeb wrote:


Hi,

I have a possible MFC candidate patch at:
http://people.freebsd.org/~bz/20090128-02-jail7-mfc.diff

to merge the multi-IPv4/v6/no-IP jails to 7-STABLE.  My plan would be
to do so during the weekend of 6-8th February 2009.

In addition to what the patch says at the beginning (__FreeBSD_version
bump), the patch also has the regenerated compat/freebsd32 sysctl
stuff in it so that people can apply, compile and run it directly.
For the merge this would be a second commit.

For committers who want to review that I have done the merge right, it
is an svn diff with mergeinfo included.

For details about the patch, features, .. see the original commit
message and follow-up a few days later (both in one post):
http://lists.freebsd.org/pipermail/freebsd-jail/2008-December/000631.html

Since then a few bug fixes went in, some older PRs were handled, ...

Now is the time for you to try and review it for 7-STABLE, etc.



One more thing that I had forgotten and was pointed at:
sys/kern/kern_jail.c includes the __FBSDID() line.
I just manually edited the patch to contain the proper CVS (not SVN) value.

You may a) want to check that things apply cleanly and/or b) to sure
to manually apply the hunk from the .rej.

/bz

--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Problem with ezjail: Manually restarted jails don't come up again

[Bjoern A. Zeeb]

On Wed, 28 Jan 2009, Frank Steinborn wrote:

...

jails are hanging somewhere in the boot-process, and i guess it's
/etc/rc.

I even doubt that this is an ezjail-only problem, but this is just a
guess.

Any hints?


if it's network services hanging on startup, check firewall and
resolve.conf inside the jail or wait a few minutes to let possible dns
queries timeout.
Also tpcdumping on the nase system for the jail IP might give a clue
in that case.

If it's something else that's hanging you can find out easily looking
at jail startup logs and/or the last process started inside the
jail...

/bz

--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: bsnmp module for monitoring jails: bsnmp-jails

[Bjoern A. Zeeb]

On Wed, 21 Jan 2009, Miroslav Lachman wrote:

Hi,


Stef wrote:

Miroslav Lachman wrote:


Stef wrote:


I've released a bsnmp module for monitoring jails via SNMP. Stuff like
network traffic, disk space, CPU utilization etc...

FreeBSD port attached, available here:

http://memberwebs.com/stef/software/bsnmp-jails/


Thank you for your announcement and your work! I will test it as soon as
possible.

Is there some limitation of FreeBSD version (6.x / 7.x / 8.x; i386 /
amd64) or is it compatible with all?



I hope it's compatible with all of the above. If you find problems with
later OS's or other architectures, I'd be happy to help find the
problems, or include patches.

When it was initially developed, 6.3 was the latest stable release of
FreeBSD. It's been deployed on a  dozen production 6.3-RELEASE i386
servers (each with lots of jails).


I added link to your website on http://wiki.freebsd.org/Jails
Do you plan to submit PR with port? Let me know if you submit it, so I can 
update the wiki page.



I see a few problems with the module (and I haven't investigated a lot
yet):

- the entire pcap stuff in there
- the inode and cpu usage stuff in there

This is all going to break on the assumption that jails do use things
exlusively. For example there can be 10 jails all sharing the same IP.
There can be jails all sharing the same fs, nullfs mounts, ...
And to my understanding the cpu usage reported is at best a snapshot
guess but no clean statics value.

I admit that those things (apart from traffic which really belongs
elsewhere) can become interesting with resource limit patches where we
get get proper values from elsewhere w/o having to do guess-math.

- no support for jails in HEAD (and soon in 7)
- does the MIB list the IP address(es)?
- private copies of xprison structures
- ...

I have the feeling that this will need a bit of polishing and
separation of things...

I hope Shteryana may join in here ...

--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: kern/89528: [jail] [patch] impossible to kill a jail

[Bjoern A. Zeeb]

The following reply was made to PR kern/89528; it has been noted by GNATS.

From: Bjoern A. Zeeb b...@freebsd.org
To: bug-follo...@freebsd.org
Cc:  
Subject: Re: kern/89528: [jail] [patch] impossible to kill a jail
Date: Sat, 10 Jan 2009 21:11:01 + (UTC)

 Before I am going to look it up another few times, this is the commit
 referenced by Andrew Thompson at Fri, 7 Dec 2007 20:21:09 +1300.
 
 Can also be looked up as SVN r174280 these days.
 
 -- 
 Bjoern A. Zeeb  The greatest risk is not taking one.
 
 -- Forwarded message --
 Date: Wed, 5 Dec 2007 01:22:03 + (UTC)
 From: Andrew Thompson thom...@freebsd.org
 To: src-committ...@freebsd.org, cvs-...@freebsd.org, cvs-...@freebsd.org
 Subject: cvs commit: src/sys/kern kern_conf.c
 
 thompsa 2007-12-05 01:22:03 UTC
 
FreeBSD src repository
 
Modified files:
  sys/kern kern_conf.c
Log:
Apply a workaround for the unkillable jail problem where some devices 
created
within the jail are never freed. si_cred is only used by the MAC framework 
so
make the cred reference conditional on it being compiled in, this is not a 
fix
and will need to be reviewed for any new consumers of si_cred.
 
This will quell some user complaint when using jails with a default kernel.
 
Reviewed by:rwatson
MFC after:  3 days
 
Revision  ChangesPath
1.209 +2 -0  src/sys/kern/kern_conf.c
 
 Index: sys/kern/kern_conf.c
 ===
 --- sys/kern/kern_conf.c(revision 174279)
 +++ sys/kern/kern_conf.c(revision 174280)
 @@ -608,9 +608,11 @@ make_dev_credv(int flags, struct cdevsw *devsw,
 in
  }
 
  dev-si_flags |= SI_NAMED;
 +#ifdef MAC
  if (cr != NULL)
  dev-si_cred = crhold(cr);
  else
 +#endif
  dev-si_cred = NULL;
  dev-si_uid = uid;
  dev-si_gid = gid;
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Problems with Samba -- svn commit: r186948 - in head/sys: netinet netinet6 (fwd)

[Bjoern A. Zeeb]

Hi,

in case anyone had trouble for example with Samba inside a jail
(and had to set interfaces = ...) you may want to update to this on
HEAD or grab the patch form the PR if you are running the multi-IP
jail patch. I'll include this in the next (upcoming) patchset.

/bz

--
Bjoern A. Zeeb  The greatest risk is not taking one.

-- Forwarded message --
Date: Fri, 9 Jan 2009 13:06:57 + (UTC)
From: Bjoern A. Zeeb b...@freebsd.org
To: src-committ...@freebsd.org, svn-src-...@freebsd.org,
svn-src-h...@freebsd.org
Subject: svn commit: r186948 - in head/sys: netinet netinet6

Author: bz
Date: Fri Jan  9 13:06:56 2009
New Revision: 186948
URL: http://svn.freebsd.org/changeset/base/186948

Log:
  Make SIOCGIFADDR and related, as well as SIOCGIFADDR_IN6 and related
  jail-aware. Up to now we returned the first address of the interface
  for SIOCGIFADDR w/o an ifr_addr in the query. This caused problems for
  programs querying for an address but running inside a jail, as the
  address returned usually did not belong to the jail.
  Like for v6, if there was an ifr_addr given on v4, you could probe
  for more addresses on the interfaces that you were not allowed to see
  from inside a jail. Return an error (EADDRNOTAVAIL) in that case
  now unless the address is on the given interface and valid for the
  jail.

  PR:   kern/114325
  Reviewed by:  rwatson
  MFC after:4 weeks

Modified:
  head/sys/netinet/in.c
  head/sys/netinet6/in6.c

Modified: head/sys/netinet/in.c
==
--- head/sys/netinet/in.c   Fri Jan  9 12:38:41 2009(r186947)
+++ head/sys/netinet/in.c   Fri Jan  9 13:06:56 2009(r186948)
@@ -41,7 +41,9 @@ __FBSDID($FreeBSD$);
 #include sys/malloc.h
 #include sys/priv.h
 #include sys/socket.h
+#include sys/jail.h
 #include sys/kernel.h
+#include sys/proc.h
 #include sys/sysctl.h
 #include sys/vimage.h

@@ -261,13 +263,19 @@ in_control(struct socket *so, u_long cmd
LIST_FOREACH(iap, INADDR_HASH(dst.s_addr), ia_hash)
if (iap-ia_ifp == ifp 
iap-ia_addr.sin_addr.s_addr == dst.s_addr) {
-   ia = iap;
+   if (td == NULL || prison_check_ip4(
+   td-td_ucred, dst))
+   ia = iap;
break;
}
if (ia == NULL)
TAILQ_FOREACH(ifa, ifp-if_addrhead, ifa_link) {
iap = ifatoia(ifa);
if (iap-ia_addr.sin_family == AF_INET) {
+   if (td != NULL 
+   !prison_check_ip4(td-td_ucred,
+   iap-ia_addr.sin_addr))
+   continue;
ia = iap;
break;
}

Modified: head/sys/netinet6/in6.c
==
--- head/sys/netinet6/in6.c Fri Jan  9 12:38:41 2009(r186947)
+++ head/sys/netinet6/in6.c Fri Jan  9 13:06:56 2009(r186948)
@@ -68,6 +68,7 @@ __FBSDID($FreeBSD$);

 #include sys/param.h
 #include sys/errno.h
+#include sys/jail.h
 #include sys/malloc.h
 #include sys/socket.h
 #include sys/socketvar.h
@@ -329,6 +330,9 @@ in6_control(struct socket *so, u_long cm
error = in6_setscope(sa6-sin6_addr, ifp, NULL);
if (error != 0)
return (error);
+   if (td != NULL  !prison_check_ip6(td-td_ucred,
+   sa6-sin6_addr))
+   return (EADDRNOTAVAIL);
ia = in6ifa_ifpwithaddr(ifp, sa6-sin6_addr);
} else
ia = NULL;
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

jail startup script for multi-IPs + ifconfig *sigh* stuff

[Bjoern A. Zeeb]

Hi,

I have a barely tested patch to rc.d/jail for HEAD (or the multi-IP
jail patch on 7) with the defaults/rc.conf and a for sure enhanceable
rc.conf.5 patch here:
http://people.freebsd.org/~bz/20090108-02-rc-jail.diff

For everyone who wants to grab the entire rc.d/jail file, fetch it from
http://people.freebsd.org/~bz/jail

This entire patch is only needed for thos people who like to get their
IPs configured/deconfigured upon jail start/stop and you see what a
mess of extra code this gives -- I am sure someone could improve the
sh(1) code but ...

I do NOT like this and neither do other people who will need to
approve this to go in.


I have been trying to support (most, all but the _netmask) from the
old version so you can still only give a single IP, or an IP list (of
mixed address families) but you can now also leave the IP part
entirely empty and start a no-IP jail or add a _multin with n
starting at 0 (like with _aliasn) and give the IPs on an extra line
each.

If you want to give an interface you can still use the jaiL_interface
or jail_jname_interface but you can also give an interface per
address now in that you prefix the address with ifName| (yes a pipe
and no blanks!).

If you want to give a netmask you can suffix an address with one of
those:
- /n  -- prefix notation, no spaces allowed
-  netmask a.b.c.d  -- netmask with a space between the
  adress and the work netmask and a full dot-quad mask.
  You are not allowed to be clever and wirte netmask a.b.c
-  prefixlen n -- similar to netmask but for v6

Obviously netmask will not work for a v6 address and prefixlen not for
v4 as what you give is directly passed to ifconfig.

If you give interface but no netmask '/32' is assumed for v4 and
'/128' for v6.

Anything I missed?


What I want to know from you:

1) does you current rc.conf setup work if you just replace
   /etc/rc.d/jail? (keep a backup of the old - outside of that directory!)

2) does this work for all the features *sigh* you need?

3) does it work with whatever management tool you use for jails?

4) any other comments?


In case there are bugs or problems, let me know - I'll update and
repost links.

/bz


PS: special thanks to Ruben van Staveren who had maintained a
(slightly) different version supporting v4/v6 ifconfig all the
time!

--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: Nagios Jail

[Bjoern A. Zeeb]

On Tue, 6 Jan 2009, Albert Shih wrote:


Le 06/01/2009 à 15:06:37+, Bjoern A. Zeeb a écrit

On Tue, 6 Jan 2009, Albert Shih wrote:


In fact I found the problem :

When I compile nagios-plugin ports in a jail the «configure» don't find
syntax of ping :

checking for ping... /sbin/ping
checking for ping6... /sbin/ping6
checking for ICMP ping syntax... configure: WARNING: unable to find usable ping 
syntax

But if I compile the same ports in a «normal» server (both are amd64).

checking for ping... /sbin/ping
checking for ping6... /sbin/ping6
checking for ICMP ping syntax... /sbin/ping -n -c %d %s
checking for ICMPv6 ping syntax... /sbin/ping6 -n -c %d %s

So if I use the check_ping produce by compiling in a no-jail server on a
jail-server it's working.

I think it's a bug about the nagios-plugins ports. What you think ?


I think most of all configure stuff out there is ... ok, if you
compile the port inside a jail and permit raw sockets, does it work
then --
either by using the rc.conf option and restarting the jail with
rc.d/jail or using sysctl security.jail.allow_raw_sockets=1  ?


You mean I MUST restart the jail after I change the sysctl value ? Because
after I change it, I can make a ping from inside the jail without
restarting the jail.

Well I'm going to make a new jail to check that (all other jail is in
production).


No, if you manually change the sysctl it's all fine and production
immediately.

If you change the option .. wait; my fault, raw sockets is not
supported by the rc framework in contrast to other things, so there is
no option there. I confused this with jail_socket_unixiproute_only in
which case just changing it in rc.conf would not be sufficient.

/bz

--
Bjoern A. Zeeb  The greatest risk is not taking one.___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: HEADS UP: r185435 multi-IPv4/v6/no-IP jails in HEAD

[Bjoern A. Zeeb]

On Thu, 11 Dec 2008, Bjoern A. Zeeb wrote:

Hi,


ok, after another round of private mails I got it; I had been living
with jail patches for too long; the jls output (without -v) should be
on one line and not on two. That wasn't intended. Unfortunately noone
had complained the months before.. I'll look at this.


can you try this patch?
http://people.freebsd.org/~bz/20081214-01-jls-v1.diff

/bz

--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: HEADS UP: r185435 multi-IPv4/v6/no-IP jails in HEAD

[Bjoern A. Zeeb]

On Sun, 14 Dec 2008, Philipp Wuensche wrote:

Hi,


ok, after another round of private mails I got it; I had been living
with jail patches for too long; the jls output (without -v) should be
on one line and not on two. That wasn't intended. Unfortunately noone
had complained the months before.. I'll look at this.


can you try this patch?
http://people.freebsd.org/~bz/20081214-01-jls-v1.diff


Works for me, jls without arguments now gives the old output, -v shows
all the new features!


Thanks for testing. Comitted it to HEAD.

/bz

--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: HEADS UP: r185435 multi-IPv4/v6/no-IP jails in HEAD

[Bjoern A. Zeeb]

On Thu, 11 Dec 2008, Philipp Wuensche wrote:

Hi,


Brian A. Seklecki wrote:

On Fri, 2008-12-05 at 20:47 +0100, Dag-Erling Smørgrav wrote:

The question is, does it change existing behavior, or just add new
functionality?


The syntax semantics should be backward compatible, so likely the
latter.


Not entirely true, the jls output is totaly different than before and
breaks third-party applications like jailaudit and ezjail.


This is only true if you use any of the new features. In case you use
single-IPv4 jails as before there should be absoultely no change in the
output format.

/bz

PS: I trimmed the CC: list as noone was able to adhere to Reply-To.

--
Bjoern A. Zeeb  The greatest risk is not taking one.___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: HEADS UP: r185435 multi-IPv4/v6/no-IP jails in HEAD

[Bjoern A. Zeeb]

On Thu, 11 Dec 2008, Philipp Wuensche wrote:

Hi,


Not entirely true, the jls output is totaly different than before and
breaks third-party applications like jailaudit and ezjail.


This is only true if you use any of the new features. In case you use
single-IPv4 jails as before there should be absoultely no change in the
output format.


Why do I get the new jls output then when I only use one ipaddr. for a
jail and none of the new features at all?


What are you using? The version from HEAD or are you running a patch
on either HEAD or 7 and if so from when?

/bz

--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to freebsd-jail-unsubscr...@freebsd.org

Re: HEADS UP: r185435 multi-IPv4/v6/no-IP jails in HEAD

[Bjoern A. Zeeb]

On Fri, 5 Dec 2008, Peter Jeremy wrote:

Hi,


On 2008-Dec-02 21:00:23 -0500, alexus [EMAIL PROTECTED] wrote:

as far as I understood HEAD is 8.0-CURRENT


Yes.


is there a way for us to start using it before 8.0 hits -RELEASE


There are two ways.  The first is:
1) Checkout a copy of the HEAD src tree via your chosen source tracker
  (cvs/cvsup/ctm/...)
2) Follow the instructions in /usr/src/UPDATING to build and install
3) Test well on a non-production box in as close to your production
  environment as possible.  Be prepared to feed back problems and
  test fixes.
4) Once you are satisfied that it works for you, place it in production.

This is basically the same as any other FreeBSD release except that you
should test more rigourously.


That's for running HEAD. I would be careful doing this on a production
system if one does not know what one is really doing when doing this;)



Your second option is to take the patches from r185435 and apply them
to your 7.x source tree.  This may take some massaging (I'm not sure
how much 7 and 8 differ in the affected areas).  bz@ may be interested
in your experiences.  Then test and roll-out as above.


There is difference, though not much. Thus just taking the patch won't
work but the solution was posted like 2 weeks ago:
http://lists.freebsd.org/pipermail/freebsd-jail/2008-November/000615.html
Look for where it says RELENG_7.



lucky), I somehow was under impression (and i guess i was wrong) that
it will come out in 7.1,


It's far too late for any new features in 7.1 but the commit log says
it should be in 7.2.


Yupp that's the plan.
And the reason it will not be in 7.1-RELEASE is that noone provided
the needed bribing money. See
http://lists.freebsd.org/pipermail/freebsd-jail/2008-November/000619.html
(not serious here). It's been just too late.


Regards,
Bjoern


--
Bjoern A. Zeeb  The greatest risk is not taking one.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]

HEADS UP: r185435 multi-IPv4/v6/no-IP jails in HEAD

[Bjoern A. Zeeb]

Hi,

as you may have already noticed multi-IPv4/v6/no-IP jails have hit
HEAD. See commit message attached.

The bad news first: expect an update on the rc script to make the
more obscure rc features like configuring IPs on interfaces when
starting jails and giving a possible netmask work with multiple IPs
and IPv6.

The good news:
In case you do not use those features or still only use one IP per
jail everything should just work fine and there are no changes needed.

More news:
In case you want to use multiple IPs or a mix of v4 and v6 addresses
you just give them as a comma separated list on both the command line
or in rc.conf like:
  jail / example 
192.0.2.250,2001:db8::75,2001:db8::99,2001:db8::55,2001:db8::14,192.0.2.254 
/bin/sh
or:
  
jail_example_ip=192.0.2.2,2001:db8::2,2001:db8::1,2001:db8::4,2001:db8::13,192.0.2.3

In case you do want to start a jail without any IP, give an empty
argument on command line:
 jail / noip.example.net  /bin/sh


Additionally you can give a jail a name now using the -n option:
  jail -n bz's private noip jail / noip.example.net  /bin/sh
You may not want to use special characters or whitespace but it is
just a string, so you can. There are no restrictions and even 10 jails
could have the same name. The jail (inside) cannot change the name.
It's set upon jail creation and unchangeable from then on.

What else is new: the -h option to jail makes it resolve the hostname
to IP addresses and will merge those to the jail IPs. Note: that this
can give you unexpected results on the primary jail IP. See jail(8)
for more information.


jls tries to be as backward compatible as possible. That means it will
only show one IPv4 if called as `jls`; obviously this won't work well
for no-IP or IPv6-only jails. This was done to try to not confuse scripts
people have in their classic setups.

jls -v will give you the full information, including:
- state: usually ACTIVE.
- in case you also give '-a' you will also see jails in other states,
  for example jails hanging around waiting for a socket to timeout
  but with no processes left after it was stopped; it will say DYING.

- Every jail gets its own cpuset inherited from the process that
  started the jail. You can list, etc the mask
  by jail id: cpuset -g -j 8
  or by set id: cpuset -g -s 5
  Or even change it if you want.
  Threads within jails should be able to further restrict themselves
  even within the jail but nothing outside their scope. See the cpuset
  manpages for further information.

The IPs will be listed in the following order:
the primary IP per AF which is the first IP of that AF given to the
jail command and then they should be sorted in ascending order.


jexec now takes the optional jail name to attach to a jail but will
refuse to do anything if the jail cannot be uniquely identifed.
In case you use the jail name you have to give an empty argument for
the jail id like:
  jexec -n bz's private noip jail  /bin/sh
You can also give both jail name and jail ID and both will have to
match, else it will complain. Obviously only giving the jail id still
works.  The -h hostname option is gone again. You should use the jail
name for management purposes now.


A sample full jls output (admittedly a bit ugly this way):

sun$ jls -av
   JID  Hostname  Path
Name  State
CPUSetID
IP Address(es)
21  sun   /
hangtest  DYING
6
192.0.2.99
 8  noip.example.net  /
bz's private noip jailALIVE
5
 3  j3.sunny.example.net  /local/jails/j1
  ALIVE
4
2001:db8::5
 2  j2.sunny.example.net  /local/jails/j1
  ALIVE
3
192.0.2.1
 1  j1.sunny.example.net  /local/jails/j1
  ALIVE
2
192.0.2.2
192.0.2.3
2001:db8::2
2001:db8::1
2001:db8::4
2001:db8::13


In case you have more questions the man pages do not address, or
problem, etc. please follow-up to freebsd-jail@ .

Regards,
Bjoern

PS: the MFC question was answered in the commit message so do not ask.

--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.

-- Forwarded message --
Date: Sat, 29 Nov 2008 14:32:14 + (UTC)
Subject: svn commit: r185435 - in head: lib/libc/sys lib/libkvm share/man/man4
sys/compat/freebsd32 sys/kern sys/net sys/netinet sys/netinet6
sys/security/mac_bsdextended sys/sys usr.bin/cpuset usr.sbin/jai...

Author: bz
Date: Sat Nov 29 14:32:14 2008
New Revision: 185435
URL: http://svn.freebsd.org/changeset/base/185435

Log:
  MFp4:
Bring in updated jail support from bz_jail branch.

  This enhances the current jail implementation to permit multiple
  addresses per jail. In addtion to IPv4, IPv6 is supported as well.
  Due

Re: Anyone interested in jail patches?

[Bjoern A. Zeeb]

On Sun, 30 Nov 2008, Frank Behrens wrote:

Hi,


Bjoern A. Zeeb wrote:

On Thu, 27 Nov 2008, Frank Behrens wrote:

On the other side I still read in the patched jail(2) man page:
Similarly, it might be a good idea to add an address alias flag such
that daemons listening on all IPs (INADDR_ANY) will not bind on that
address Can you explain the current behaviour?


I think this question is related to your PR kern/84215.

Yes.


The current situation is: jails take precendence. So if sshd is
listening on inaddr_any on the host and on inaddr_any inside a jail
the connection to an IP belonging to a jail will end up inside the
jail; any connections to IPs not beloning to jails will end up on the
base.

So we have now the desired behaviour. Your explanation should replace
the (now incorrect) sentence in the man page. Please excuse my error, it is 
in jail(8),

not jail(2).


Obviously if you stop the jail and ssh to a former jail IP you'll end
up on the bsae system and ssh would complain about different keys
possibly while telnet or similar things won't notice.

This is expected and not easily to circumvent.


Yes it is. You don't bind your sshd (or whatever) to inaddr_any on the
base system but an IP exclusive to the base system. If the jail is
stopped, you'll get connection refused instead of an unexpected
behaviour. So what is in the man page is not entirely wrong.

/bz

--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: Anyone interested in jail patches?

[Bjoern A. Zeeb]

On Thu, 27 Nov 2008, Mars G Miro wrote:

Hi,


Tried both on recent 7.X and 8.X. Used about 4,5 different IPs ( IPv4
and v6 ) for the jails.

So far so good ;-)


That was good news the next morning:)

Thanks.


Regards,
Bjoern

--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]

HEADS UP: multi/no-IPv4/v6 jails going to hit HEAD

[Bjoern A. Zeeb]

Hi,

I haven't heard back anything bad after (almost) 24 hours since I
had released the latest patchset. So this is the HEADS UP for you
that unless major regessions or other important stops show up
I plan to commit the latest multi/no-IPv4/v6 jail patch to HEAD
saturday (2008-11-29) morning UTC.

You'll find a few things like man page dates, etc. updated to the
current diff so it will slightly change - but there should be no
functional changes anymore (unless a regression is found or make
universe won't like me later today;-).

For patches see my original mail to freebsd-jail from last night
below and the entire thread here:
http://lists.freebsd.org/pipermail/freebsd-jail/2008-November/000615.html

This is mostly intended for two things:
- to get out of the way for other vimage/mgmt work for 8.x
- possible MFC to 7 in a few weeks or rather months (don't even think
  about asking for 7.1-RELEASE; the answer would be: in case you are
  going to donate 100.000 USD I could start talking to re@ about that
  but we might need more money for bribing during the negotiations;)
and:
- add a lot of good FreeBSD marketing after the commit here - send
  patches! :-)

Regards,
Bjoern


Date: Wed, 26 Nov 2008 23:56:55 + (UTC)
From: Bjoern A. Zeeb [EMAIL PROTECTED]
To: freebsd-jail@freebsd.org
Subject: Anyone interested in jail patches?

Hi,

it's 1am and I am out of caffeine so excuse all those typos and in case
there will be bugs blame them on whatever you want...; I just want to
get this out, finally, to you.

If you are interested in a new set of jail patches... anyone?;-)

1) read the changelog from http://perforce.freebsd.org/chv.cgi?CH=153529
that's a good summary for the diff to the last set of patches.

2) I freshly integrated both branches; there had been a few changes
since yesterday after my testing but those should be ok. It also
means that the patches should apply to the sources of `now`.

2a) for HEAD:
http://people.freebsd.org/~bz/bz_jail-20081126-02-at153644.diff

2b) for RELENG_7:
http://people.freebsd.org/~bz/bz_jail7-20081126-02-at153644.diff

2c) there is no 7.0-RELEASE support anymore; sorry.


As always please report problems or success stories to the list rather
than to me directly. Same usually applies for questions. In case you
are happy consider http://www.freebsdfoundation.org/donate/ .


Regards,
Bjoern



--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: correct syntax to bind ipv6 to jails?

[Bjoern A. Zeeb]

On Mon, 20 Oct 2008, Brian wrote:

Hi,


im running releng_7_0 with bz's muli ipv4/6/none patch, i cant get ipv6 to 
bind, ive tried every possible way, i keep getting this error:
 
??(/home/brian)-- # jail /usr/jails/test.jinxshells.com test.jinxshells.com 
2a02:780:a002::3 /bin/sh
jail: Address family 28 not supported. Ignoring.
: No such file or directory


huh, are your sure you are running with my patch and that the kernel
is compiled with INET6 support?

/bz

--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: multi-ip v4/v6

[Bjoern A. Zeeb]

On Thu, 16 Oct 2008, Brian wrote:


Is there a patch for multi-ip/ipv4/ipv6 for freebsd 6.3-release-p5?


no; there is an multi-IPv4 patch for 6.x somehere around ... let
mevfind it ... here it is:
http://people.freebsd.org/~bz/multi-ip-jail-6.4-pre-20080926-01.diff

That's the best I can give you. I have no plan to support
multi-ipv4/v6/no-IP patches for before 7.1-PRE (anymore) atm.

/bz

--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: Compilation question 64bit, 32 bit

[Bjoern A. Zeeb]

On Fri, 17 Oct 2008, Bjoern A. Zeeb wrote:

Hi,


On Fri, 17 Oct 2008, Andrew Snow wrote:

Hi,


Alexander Leidinger wrote:
Sort of. You can install a 32bit world into the jail and make sure 32bit 
support is activated in the kernel. The 32bit programs will then run just 
fine in the jail (but 64bit ones should run fine too). 


How is this done?

I've never been able to find out how, it doesn't appear to be documented 
anywhere.


I have been trying to get the people who know best to document it (at
least roughly) and have failed so far.

It' is more than simply installing a 32bit world as jails and starting
the jail.


ok, it turns out that if you just want to run things it should just
work.

If you want to build ports or things in there, there are a bunch of
enviroment variables to set.

/bz

--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: FreeBSD 7.0-RELEASE

[Bjoern A. Zeeb]

On Sun, 12 Oct 2008, Patrick Vanguardia wrote:


Hi,

Is there anyone of you have a working multi-ip patch for RELENG_7_0_0? The
reason im using RELENG_7_0_0 is because i cant just upgrade my kernel to my
dedicated hosting since it has a WHM and a CPANEL that might not work after
the update.

I'm seeing some patches in the mailing list like the ff.. (see below) and i
just want to confirmed if thats the patch for RELENG_7_0_0

HEAD: bz_jail-20080727-10-at146056.diff
STABLE: bz_jail7-20080727-11-at146062.diff


I have put the latest 7.0-R patch I had done and found here tmeporary:
http://people.freebsd.org/~bz/20080617-01-jail-7.0R.diff

Do not expect any update for it or anything.

/bz

--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: bz_jail7-20080920-01-at150161.diff

[Bjoern A. Zeeb]

On Fri, 10 Oct 2008, alexus wrote:


1. latest patch for 7.0-RELEASE isn't working


so which patch was that? Have an URL or filename?


/bz

--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: samba inside jails [was: jail/broadcast IP [was: ...]]

[Bjoern A. Zeeb]

On Wed, 1 Oct 2008, Nejc S(koberne wrote:

Hi,

Cc:ing freebsd-jail again.


I would like to make Samba, running in jail, to listen at a broadcast address.
Normally Samba would listen on *.138 and *.137 (UDP), but when in jail, it can
just listens at IP.138 and IP.137, which makes it unable to see the requests.


So it listens on INADDR_ANY which is not the broadcast address.
However the windows world is (was) high on broadcasts.
If you have multiple IPs it does listen on *:{port} again but that's
only partly the same as what you are probably thinking about.

You can still run samba inside a (multi-IP) jail. Back in 2006, about
this multi-IP patch, and samba from then I found the following:

1) samba does not respond from the same IP the packet was directed to
   but from your Primary IP.
   This is interesting if you have multiple IPs from the same subnet
   on the same link and jail.

2) with the multi-IP jail patch I preserve the primary IP (the first
   IP given for each address family) as such. So you can actually
   tell a jail what the primary/fallback IP would be in case the
   introduced source address selection does not find any better.

3) In samba it used to be the
interfaces =
   config option that you would set to the (primary) IP of your jail.

With the above you should be able to address the samba server inside
the jail and exchange files and all that. At least I was able to back
then. Things may have changed.

Depending on your setup browsing via good old braodcast stuff might
not work but in any modern setup that should no longer be needed imho.


Good luck.
/bz

--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: jail/broadcast IP [was: Multiple IPS - Freebsd 7.1]

[Bjoern A. Zeeb]

On Wed, 1 Oct 2008, Nejc S(koberne wrote:

Hi,


does this patch maybe also makes it possible for a jail to listen at a 
broadcast address?


So before you are going to post this to another thread -- what are you
trying to achive?

--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: Multiple IPS - Freebsd 7.1

[Bjoern A. Zeeb]

On Wed, 1 Oct 2008, Sami Halabi wrote:

Hi,


i applied the following patch (which i changed to get to work with the
current src):
http://www.royalshells.com/download/freebsd/bz_jail7-20080727-11-at146062-Fixed_By_Sody_1.10.08.diff


I do not know what the Fixed_By_Sody_1.10.08 part is but
it's at least based on a backlevel patch of mine so I hope
someone fixed the locking for 7.1-PRE.

I had posted a patch for 7-STABLE (7.1-PRE) last month
http://people.freebsd.org/~bz/bz_jail7-20080920-01-at150161.diff
so taking that one you could have save yourself a lot of time I guess.

In case you have changes it would be good to know what you did
or what I do not have so the do not have forks as my version will hit
HEAD soon and has changed (also the user space) since July.

/bz

--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: multi-/no-ipv4/6 patch for releng_7

[Bjoern A. Zeeb]

On Sat, 20 Sep 2008, Bjoern A. Zeeb wrote:

Hi,


here's a new patch for RELENG_7. In contrast to before I have NOT
TESTED this patch THOROUGHLY.


FYI: I know production machines with ipv4/ipv6 jails that have been
up for two days running this patch.



In case you find any problem let me know and I might be able to fix
it (quickly) and post a new patch (if your bugreport is good:).

Changes since last release (same as for HEAD):
- SCTP enabled (again) for IPv4 and on for v6 as well. Might need
 another pari of eyes or someone to write regression tests.
- jls -a/-v implemented/documented -- output format has changed.
- updated ipv4 source address selection (changes semantics, please
 test/review carefully)
- minor cleanup

Please report anything you want/that need sto be/... changed/fixed/...

Ah the patch is here:
http://people.freebsd.org/~bz/bz_jail7-20080920-01-at150161.diff

/bz




--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: request for (security) comments on this setup

[Bjoern A. Zeeb]

On Mon, 22 Sep 2008, Randy Schultz wrote:

Hi,


I'm mounting some iSCSI storage in a jail.  It's mounting in the jail via
fstab.jailname.  When the jail is up and I'm logged into the jail I can cd
to the mount point, r/w etc., everything seems to work.  What's weird tho' 
is,

while a df on the parent shows the partion mounted as expected, a df inside
the jail shows the local disk but not the iSCSI mount.
...
So, my first question is what am I missing, the second is does mounting 
things

this way into a jail pose any sort of risk for escaping the jail?


Does anything change if you do a
sysctl security.jail.enforce_statfs=1

If that's what you want you can add the following lines to
/etc/sysctl.conf in the base system so it is automatically set upon
boot:

# jails
security.jail.enforce_statfs=1


/bz


--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]

[CFR/T] multi-/no-IP jail patch for HEAD

[Bjoern A. Zeeb]

Hi,

I have put a close to be commit candidate (if ipv4 src addr selection
is in upfront) online here:

http://people.freebsd.org/~bz/bz_jail-20080915-02-svn.diff

This is for HEAD, for review and testing.

Changes since last release:
- SCTP enabled (again) for IPv4 and on for v6 as well. Might need
  another pari of eyes or someone to write regression tests.
- jls -a/-v implemented/documented
- updated ipv4 source address selection (changes semantics, please
  test/review carefully)
- minor cleanup

Please report anything you want/that need sto be/... changed/fixed/...

Thanks.
/bz


PS: for anyone crying for RELENG_7. I am trying to put a patch
together the next days.

--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: Multiple IPs

[Bjoern A. Zeeb]

On Tue, 2 Sep 2008, Matkhamtkha Brekher wrote:


this patch is broken

today ive tried to compile the patched world and it stops compiling
with due to error:


I bet you got errors over errors when you tried to apply it.

check the output or the return code from patch in your scripts...
patch  
case $? in
0)  ;;  # all fine
*   echo PATCH DID NOT APPLY CLEANLY 2
exit 1
;;
esac

You could try with patch -C first btw to not hose your src tre...



I am wroking towards getting it into HEAD and once 7 will be in freeze
I'll generate a new patch but 7 has been hosed for a while and people
are doing last minute MFCs now so I would have had to regen it every
few hours.


--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: kern/126368: Running ktrace/kdump in jail leads to stale jails

[Bjoern A. Zeeb]

On Fri, 8 Aug 2008, Mateusz Guzik wrote:


The following reply was made to PR kern/126368; it has been noted by GNATS.

From: Mateusz Guzik [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Cc:
Subject: Re: kern/126368: Running ktrace/kdump in jail leads to stale jails
Date: Fri, 8 Aug 2008 19:30:22 +0200

Err, I made a mistake. crfree() will be called in case of failure
(loop starting at line 959), so the following patch should be ok:

--- sys/kern/kern_ktrace.c.orig 2008-08-08 16:37:45.0 +0200
+++ sys/kern/kern_ktrace.c  2008-08-08 19:25:16.0 +0200
@@ -933,12 +933,14 @@
error = VOP_WRITE(vp, auio, IO_UNIT | IO_APPEND, cred);
VOP_UNLOCK(vp, 0, td);
vn_finished_write(mp);
vrele(vp);
VFS_UNLOCK_GIANT(vfslocked);
-   if (!error)
+   if (!error) {
+   crfree(cred);
return;
+   }


that sounds more plausible w/o seeing the surrounding code. I had
wondered already earlier today when I was pointed at.

I'll look into this.



/*
 * If error encountered, give up tracing on this vnode.  We defer
 * all the vrele()'s on the vnode until after we are finished walking
 * the various lists to avoid needlessly holding locks.
 */
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]



--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: identd on jail with multiple IPs

[Bjoern A. Zeeb]

On Wed, 6 Aug 2008, Redd Vinylene wrote:


I cannot seem to make identd work on a jail with multiple IPs (Bjoern
Zeeb's patch):


So do you have any kind of error message? packet traces or anything to
further isolate the problem rather than does not work?

--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]

Patch no longer applying cleanly

[Bjoern A. Zeeb]

Hi,

FYI:

I am aware of that the jail patches are no longer applying cleanly.
There are upcoming changes during this week which will add further
conflicts.

I'll update the patches once those changes are in and the tree
should be stable again with regard to the jail work.


/bz

--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: Can't SSH into my jails after a makeworld

[Bjoern A. Zeeb]

On Sat, 2 Aug 2008, Redd Vinylene wrote:

Hi,


I have a little problem here. I can't seem to SSH into my jails any
longer, I get taken straight back to the mothership. jexec works
though.


what does
sysctl security.jail.jailed_sockets_first
give?

If you jexec into  jail, does sshd actually run? Did it give an
error/warning? What does netstat -an show? (in case this is long do
not psate it into mail and/or make sure there are no extra line wraps).

--
Bjoern A. Zeeb  Stop bit received. Insert coin for new game.
___
freebsd-jail@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to [EMAIL PROTECTED]