Re: clonable bpf

2012-11-28 Thread Janne Johansson
2012/11/27 Mike Belopuhov m...@belopuhov.com:

 apply the diff and see how all your apps are happily using single
 device node: /dev/bpf0.
 i'm not 100% sure we want this but why wouldn't we?
 generally speaking we also need to move to a single /dev/bpf
 node, but that can be done, once this diff is tested and we all
 agree that this is a way to go.

I for one get bitten at times from not having enough bpf's in /dev to
run all the dhcrelays I want and tcpdumps.
I just need a larger-than-normal amount of /dev/bpf* entries, but if
this also removes that limit so I won't have to MAKEDEV a lot, I'd
be happy.

-- 
 To our sweethearts and wives.  May they never meet. -- 19th century toast



Re: clonable bpf

2012-11-28 Thread Reyk Floeter
On Tue, Nov 27, 2012 at 10:17 PM, Mike Belopuhov m...@belopuhov.com wrote:
 apparently it works just fine.  the number of clones is limited
 by the v_specbitmap which currently allows for 64 clones total
 (per system, not per process).


Please clarify: Does it mean 64 cloned bpf devices per system or 64
cloned devices of any type in total? Or would it even allow to use
bpf0 for the first 64 clones, bpf1 for the next 64 and so on?

If running a custom kernel (according to the diff in your next email)
is the only way to increase this limit, I would propose to bump the
default to at least 256. I know of at least a few people who run many
interfaces/VLANs ( 64) and need one bpf per interface.

 apply the diff and see how all your apps are happily using single
 device node: /dev/bpf0.

 thanks to pedro and thib for the spec_open_clone and guenther
 for helping out big time.

 i'm not 100% sure we want this but why wouldn't we?
 generally speaking we also need to move to a single /dev/bpf
 node, but that can be done, once this diff is tested and we all
 agree that this is a way to go.

 so please test and share your thoughts.


I want this. Actually I want this for a long time (about 6 years?).
But I cannot judge the fs layer of the implementation.

At least NetBSD, FreeBSD, DragonFlyBSD and Solaris seem to have a
clonable /dev/bpf for a long time now. Going to a single /dev/bpf
shouldn't be a big problem as we can update our in-tree bpf users
easily, most ports including snort/daq seem to use our libpcap, and
other upstream code already checks for a clonable /dev/bpf.

Reyk


 diff --git sys/sys/conf.h sys/sys/conf.h
 index b2ee90c..46937fd 100644
 --- sys/sys/conf.h
 +++ sys/sys/conf.h
 @@ -313,7 +313,7 @@ extern struct cdevsw cdevsw[];
 dev_init(c,n,open), dev_init(c,n,close), dev_init(c,n,read), \
 dev_init(c,n,write), dev_init(c,n,ioctl), (dev_type_stop((*))) 
 enodev, \
 0, dev_init(c,n,poll), (dev_type_mmap((*))) enodev, \
 -   0, 0, dev_init(c,n,kqfilter) }
 +   0, D_CLONE, dev_init(c,n,kqfilter) }

  /* open, close, ioctl */
  #definecdev_lkm_init(c,n) { \



Re: clonable bpf

2012-11-28 Thread Mike Belopuhov
On Wed, Nov 28, 2012 at 12:43 PM, Reyk Floeter r...@openbsd.org wrote:
 On Tue, Nov 27, 2012 at 10:17 PM, Mike Belopuhov m...@belopuhov.com wrote:
 apparently it works just fine.  the number of clones is limited
 by the v_specbitmap which currently allows for 64 clones total
 (per system, not per process).


 Please clarify: Does it mean 64 cloned bpf devices per system or 64
 cloned devices of any type in total?


64 (or whatever we set the bitmask to be) cloned bpf devices per system.

 Or would it even allow to use bpf0 for the first 64 clones, bpf1 for the
 next 64 and so on?

there's not supposed to be bpf0,1,2,3... it's supposed to be just /dev/bpf
with a minor of 0. but pcap and other programs will need to be adjusted
for that (upstream pcap already has code for that, and naturally it's not
complicated at all).

 If running a custom kernel (according to the diff in your next email)
 is the only way to increase this limit, I would propose to bump the
 default to at least 256. I know of at least a few people who run many
 interfaces/VLANs ( 64) and need one bpf per interface.


it's safe to bump it to 1024. that bitmask (and the structure itself) is
allocated once per character device (i.e. one bitmask for drm, one
for bpf -- two chunks of memory in total).

 apply the diff and see how all your apps are happily using single
 device node: /dev/bpf0.

 thanks to pedro and thib for the spec_open_clone and guenther
 for helping out big time.

 i'm not 100% sure we want this but why wouldn't we?
 generally speaking we also need to move to a single /dev/bpf
 node, but that can be done, once this diff is tested and we all
 agree that this is a way to go.

 so please test and share your thoughts.


 I want this. Actually I want this for a long time (about 6 years?).
 But I cannot judge the fs layer of the implementation.

 At least NetBSD, FreeBSD, DragonFlyBSD and Solaris seem to have a
 clonable /dev/bpf for a long time now. Going to a single /dev/bpf
 shouldn't be a big problem as we can update our in-tree bpf users
 easily, most ports including snort/daq seem to use our libpcap, and
 other upstream code already checks for a clonable /dev/bpf.

 Reyk



Re: clonable bpf

2012-11-28 Thread Theo de Raadt
 there's not supposed to be bpf0,1,2,3...

Unfortunately history argues otherwise.



Re: clonable bpf

2012-11-28 Thread Mike Belopuhov
On Wed, Nov 28, 2012 at 11:38 PM, Theo de Raadt dera...@cvs.openbsd.org wrote:
 there's not supposed to be bpf0,1,2,3...

 Unfortunately history argues otherwise.

no, i meant if we move to the clonable bpf, we will need only one node.
sorry for not making it clear.



clonable bpf

2012-11-27 Thread Mike Belopuhov
apparently it works just fine.  the number of clones is limited
by the v_specbitmap which currently allows for 64 clones total
(per system, not per process).

apply the diff and see how all your apps are happily using single
device node: /dev/bpf0.

thanks to pedro and thib for the spec_open_clone and guenther
for helping out big time.

i'm not 100% sure we want this but why wouldn't we?
generally speaking we also need to move to a single /dev/bpf
node, but that can be done, once this diff is tested and we all
agree that this is a way to go.

so please test and share your thoughts.


diff --git sys/sys/conf.h sys/sys/conf.h
index b2ee90c..46937fd 100644
--- sys/sys/conf.h
+++ sys/sys/conf.h
@@ -313,7 +313,7 @@ extern struct cdevsw cdevsw[];
dev_init(c,n,open), dev_init(c,n,close), dev_init(c,n,read), \
dev_init(c,n,write), dev_init(c,n,ioctl), (dev_type_stop((*))) enodev, \
0, dev_init(c,n,poll), (dev_type_mmap((*))) enodev, \
-   0, 0, dev_init(c,n,kqfilter) }
+   0, D_CLONE, dev_init(c,n,kqfilter) }
 
 /* open, close, ioctl */
 #definecdev_lkm_init(c,n) { \



Re: clonable bpf

2012-11-27 Thread Mike Belopuhov
On Tue, Nov 27, 2012 at 22:17 +0100, Mike Belopuhov wrote:
 apparently it works just fine.  the number of clones is limited
 by the v_specbitmap which currently allows for 64 clones total
 (per system, not per process).
 
 apply the diff and see how all your apps are happily using single
 device node: /dev/bpf0.
 
 thanks to pedro and thib for the spec_open_clone and guenther
 for helping out big time.
 
 i'm not 100% sure we want this but why wouldn't we?
 generally speaking we also need to move to a single /dev/bpf
 node, but that can be done, once this diff is tested and we all
 agree that this is a way to go.
 
 so please test and share your thoughts.
 

for those who wants to have lots of (1024) nodes available:

diff --git sys/sys/conf.h sys/sys/conf.h
index b2ee90c..46937fd 100644
--- sys/sys/conf.h
+++ sys/sys/conf.h
@@ -313,7 +313,7 @@ extern struct cdevsw cdevsw[];
dev_init(c,n,open), dev_init(c,n,close), dev_init(c,n,read), \
dev_init(c,n,write), dev_init(c,n,ioctl), (dev_type_stop((*))) enodev, \
0, dev_init(c,n,poll), (dev_type_mmap((*))) enodev, \
-   0, 0, dev_init(c,n,kqfilter) }
+   0, D_CLONE, dev_init(c,n,kqfilter) }
 
 /* open, close, ioctl */
 #definecdev_lkm_init(c,n) { \
diff --git sys/sys/specdev.h sys/sys/specdev.h
index 4341a35..1801d9a 100644
--- sys/sys/specdev.h
+++ sys/sys/specdev.h
@@ -46,7 +46,7 @@ struct specinfo {
daddr64_t si_lastr;
union {
struct vnode *ci_parent; /* pointer back to parent device */
-   u_int8_t ci_bitmap[8]; /* bitmap of devices cloned off us */
+   u_int8_t ci_bitmap[128]; /* bitmap of devices cloned off us */
} si_ci;
 };