Re: simulating wireless device (if_alloc panic, VirtualBox, VIMAGE)
On 2/2/11 7:06 AM, Monthadar Al Jaberi wrote: Thanx makes more sense, but I have noticed something weired if you can shade some light on. I added printfs one when the module is first loaded (static int event_handler(module_t module, int event, void *arg)): curthread=0xc3f95870 curthread-td_vnet=0xc3170e00 curthread-td_ucred=0xc3185d00 TD_TO_VNET=0 CRED_TO_VNET=0 And same printf in wtap_ioctl which is called from a user space program (I am root): curthread=0xc3f952d0 curthread-td_vnet=0 curthread-td_ucred=0xc3185d00 TD_TO_VNET=0 CRED_TO_VNET=0 In both cases TD_TO_VNET/CRED_TO_VNET return NULL... shouldn't they return a pointer same as curthread-td_vnet? Another thing is that in wtap_ioctl curthread-td_vnet is NULL that depends on where in your code you did the print.. have you read this file? http://p4db.freebsd.org/fileViewer.cgi?FSPC=//depot/projects/vimage/porting_to_vimage.txtREV=18 use the 'download' button to get a more readable version. it goes into some of the details of this. especially the initialization or vimage modules. I still get a panic... br, On Tue, Feb 1, 2011 at 8:37 PM, Julian Elischerjul...@freebsd.org wrote: ok here's how it works.. any place you access a V_xxx variable you need to have the current vnet set. so somewhere in your code path to get to that point you have to have done CURVNET_SET() and after you have finised on the way out you should do CURVNET_RESTORE(). you can get the vnet from several places: 1/ as shown below, you can get it from any thread's cred if teh running thread is part of a process in the jail in question. 2/ if you have an ifp pointer, you can use ifp-vnet . I think that's right, it's been a while... 3/ I believe though I may be wrong (I may be thinking of multi FIBS) that it maybe in the socket structure too but don't trust me on that one.. check it. if, like in a timer thread you have access to NONE of those, you have several choices.. 1/ the caller of the timer may have given you indirect access to it in the arg. 2/ maybe you juaast have to interate through all the vimages.. to do whatever it is that you do (that happens in some protocols) On 2/1/11 11:04 AM, Monthadar Al Jaberi wrote: On Tue, Feb 1, 2011 at 6:25 PM, Julian Elischerjul...@freebsd.org wrote: On 2/1/11 8:40 AM, Monthadar Al Jaberi wrote: Hi, I hope I am on the write place, second try... I have written a module that loads fake wifi devices (wtap?) and distributes packets between them. For now I use route command to route packets between them from upper layers (TCP,...). I want to take it to next step and create jails with VNET. I started reading Julian Elischer's Vimage: what is it? and he says that if I am writing hardware drivers I dont need to make any changes when I enable VIMAGE option on the kernel, because each one will have their own stack. I can give a more detailed explanation on how my module works. But for now I get a panic when calling if_alloc() when using option VIMAGE. Thank you, while this was true to some extent it i snot 100% true now. during allocation we now try to have separate interface indexes per vimage which means that the setup routines do need to know the current vnet. so I cant call if_alloc directly? it looks like wtap_ioctl or wtap_attach should have the following: (copied from the tun driver) this should not be needed from real hardware based drivers as far as I can tell. CURVNET_SET(CRED_TO_VNET(cred)); /* find any existing device, or allocate new unit number */ i = clone_create(tunclones,tun_cdevsw,u, dev, 0); [blah] if_clone_create(name, namelen, NULL); CURVNET_RESTORE(); My wtap is based on ath driver code (if_ath.c) which should look like a real device right? if_ath.c is not using VNET, as far as I can tell Currently my module creates a couple of wtaps, which I then create a corresponding wlan. These wtaps are interconnected together, so no out world yet... so I dont have struct cdev *dev Basic idea is my module have a main queue (simulating air) and each wtap have a rx_task which sends packets up to higher layers, plus callout timer for generation beacons... I will try to use CURVET_SET(...) tomo My setup: FreeBSD Current 201010 guest on VirtualBox on Ubuntu 10.04. Kernel page fault with the following non-sleepable locks held: exclusive rw ifnet_rw (ifnet_rw) r = 0 (0xc0fc8284) locked @ /usr/src/sys/net/if.c:414 KDB: stack backtrace: db_trace_self_wrapper(c0cf3cdb,1,0,0,0,...) at db_trace_self_wrapper+0x26 kdb_backtrace(19e,1,,c0f9b194,c2fc9a1c,...) at kdb_backtrace+0x2a _witness_debugger(c0cf6408,c2fc9a30,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0d2c479,3,c4070d48,...) at witness_warn+0x1fe trap(c2fc9abc) at trap+0x195 calltrap() at calltrap+0x6 --- trap 0xc, eip = 0xc0970999, esp = 0xc2fc9afc, ebp = 0xc2fc9b1c --- ifindex_alloc_locked(c0d003cf,c2fc9b36,19e,19e,c15ab714,...) at ifindex_alloc_locked+0x19
Re: simulating wireless device (if_alloc panic, VirtualBox, VIMAGE)
On 2/2/11 8:42 AM, Julian Elischer wrote: On 2/2/11 7:06 AM, Monthadar Al Jaberi wrote: Thanx makes more sense, but I have noticed something weired if you can shade some light on. I added printfs one when the module is first loaded (static int event_handler(module_t module, int event, void *arg)): curthread=0xc3f95870 curthread-td_vnet=0xc3170e00 curthread-td_ucred=0xc3185d00 TD_TO_VNET=0 CRED_TO_VNET=0 And same printf in wtap_ioctl which is called from a user space program (I am root): curthread=0xc3f952d0 curthread-td_vnet=0 curthread-td_ucred=0xc3185d00 TD_TO_VNET=0 CRED_TO_VNET=0 In both cases TD_TO_VNET/CRED_TO_VNET return NULL... shouldn't they return a pointer same as curthread-td_vnet? Another thing is that in wtap_ioctl curthread-td_vnet is NUL that depends on where in your code you did the print.. have you read this file? http://p4db.freebsd.org/fileViewer.cgi?FSPC=//depot/projects/vimage/porting_to_vimage.txtREV=18 use the 'download' button to get a more readable version. it goes into some of the details of this. especially the initialization or vimage modules. oops sent too early. I think you said you had seen it but read it again now that I have explained some of it to you and maybe it will give new insights. The VNET_SYSINIT() code will get called once for every vnet, and the current vnet is set up for you before it calls you. it is also called once when you start a NEW vnet.. this is the place where Marko jumps in because my head is currently full of other stuff from work and I have paged the vimage stuff out to swap.. :-) I still get a panic... br, On Tue, Feb 1, 2011 at 8:37 PM, Julian Elischerjul...@freebsd.org wrote: ok here's how it works.. any place you access a V_xxx variable you need to have the current vnet set. so somewhere in your code path to get to that point you have to have done CURVNET_SET() and after you have finised on the way out you should do CURVNET_RESTORE(). you can get the vnet from several places: 1/ as shown below, you can get it from any thread's cred if teh running thread is part of a process in the jail in question. 2/ if you have an ifp pointer, you can use ifp-vnet . I think that's right, it's been a while... 3/ I believe though I may be wrong (I may be thinking of multi FIBS) that it maybe in the socket structure too but don't trust me on that one.. check it. if, like in a timer thread you have access to NONE of those, you have several choices.. 1/ the caller of the timer may have given you indirect access to it in the arg. 2/ maybe you juaast have to interate through all the vimages.. to do whatever it is that you do (that happens in some protocols) On 2/1/11 11:04 AM, Monthadar Al Jaberi wrote: On Tue, Feb 1, 2011 at 6:25 PM, Julian Elischerjul...@freebsd.org wrote: On 2/1/11 8:40 AM, Monthadar Al Jaberi wrote: Hi, I hope I am on the write place, second try... I have written a module that loads fake wifi devices (wtap?) and distributes packets between them. For now I use route command to route packets between them from upper layers (TCP,...). I want to take it to next step and create jails with VNET. I started reading Julian Elischer's Vimage: what is it? and he says that if I am writing hardware drivers I dont need to make any changes when I enable VIMAGE option on the kernel, because each one will have their own stack. I can give a more detailed explanation on how my module works. But for now I get a panic when calling if_alloc() when using option VIMAGE. Thank you, while this was true to some extent it i snot 100% true now. during allocation we now try to have separate interface indexes per vimage which means that the setup routines do need to know the current vnet. so I cant call if_alloc directly? it looks like wtap_ioctl or wtap_attach should have the following: (copied from the tun driver) this should not be needed from real hardware based drivers as far as I can tell. CURVNET_SET(CRED_TO_VNET(cred)); /* find any existing device, or allocate new unit number */ i = clone_create(tunclones,tun_cdevsw,u, dev, 0); [blah] if_clone_create(name, namelen, NULL); CURVNET_RESTORE(); My wtap is based on ath driver code (if_ath.c) which should look like a real device right? if_ath.c is not using VNET, as far as I can tell Currently my module creates a couple of wtaps, which I then create a corresponding wlan. These wtaps are interconnected together, so no out world yet... so I dont have struct cdev *dev Basic idea is my module have a main queue (simulating air) and each wtap have a rx_task which sends packets up to higher layers, plus callout timer for generation beacons... I will try to use CURVET_SET(...) tomo My setup: FreeBSD Current 201010 guest on VirtualBox on Ubuntu 10.04. Kernel page fault with the following non-sleepable locks held: exclusive rw ifnet_rw (ifnet_rw) r = 0 (0xc0fc8284) locked @
Re: simulating wireless device (if_alloc panic, VirtualBox, VIMAGE)
I just tried something that seems to work, but please dont hit me ^^;;; in wtap_ioctl I assigned curthread-td_vnet myself to point to a VNET (saved it when the module first loaded) (I have not created any jails yet)... and it works... I didnt put any CURVNET macros... my assumption is that if ath drivers dont use VNET I shouldnt :P What is wrong with this hack? br, P.S. I have printed porting to vnet text to have it always at hand, but its a bit hard for me... doing my best. On Wed, Feb 2, 2011 at 6:30 PM, Julian Elischer jul...@freebsd.org wrote: On 2/2/11 9:12 AM, Bjoern A. Zeeb wrote: On Wed, 2 Feb 2011, Monthadar Al Jaberi wrote: Hi, Thanx makes more sense, but I have noticed something weired if you can shade some light on. I added printfs one when the module is first loaded (static int event_handler(module_t module, int event, void *arg)): curthread=0xc3f95870 curthread-td_vnet=0xc3170e00 curthread-td_ucred=0xc3185d00 TD_TO_VNET=0 CRED_TO_VNET=0 Try to load it from laoder on boot; I think that should work as we are setting the curvent for the kernel startup. The problem you are seeing is a bug in the current implementation that you cannot add any physical network interface after the kernel started. This applies to cardbus/usb/... as well as any kind of ethernet interface, so a kldload igb should yield it as well. The fix for that is easy and hard at the same time: A) either touch all drivers B) or touch all cloned interfaces and change 3 common lines. or try to make cloners aware of vimages. Solution B) is sitting in perforce with the entire stuff that it depends on and was started with CH=179022,179255 but not limited to that if you want to have a peek. What you certainly can do locally to your driver for now is to make a change like this: +#ifdef VIMAGE + CURVNET_SET(vnet0); +#endif ifp = if_alloc(IFT_ETHER); +#ifdef VIMAGE + CURVNET_RESTORE(); +#endif you don't really need the #ifdef except for readability as CURVNET_XXX ar enot defined for !vnet It's the type A) kind of change from above that will break eventually in the future. /bz -- //Monthadar Al Jaberi ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: simulating wireless device (if_alloc panic, VirtualBox, VIMAGE)
On 2/2/11 10:05 AM, Monthadar Al Jaberi wrote: I just tried something that seems to work, but please dont hit me ^^;;; in wtap_ioctl I assigned curthread-td_vnet myself to point to a VNET (saved it when the module first loaded) (I have not created any jails yet)... and it works... I didnt put any CURVNET macros... td-td_vnet is exactly what the CURVNET_SET macro sets. You should use the Macros because we may change the place where we store it. The vnet for the current thread is picked up from several places depending on the context, and it is cleared again when it is not needed. the V_xxx usages in the code end up being in effect expanded to curthread-td_vnet.xxx, where each 'xxx' is sort of like an element in a structure but not quite. Now, theoretically we could just leave it set all the time but then it would be nearly impossible to find places where we should have changed it, but forgot and just got the existing one. if you want to find the correct place to go, then look at the vnet of the calling process which should be in the process cred. or just use vnet0. I don't understand why you saw a CRED_TO_VNET of 0 I was under the impression that every process/thread in the system would be on vnet0 in a vimage kernel. your stored vnet idea is ok as well, but may go strange if you load the driver from a vnet jail and then remove the jail. my assumption is that if ath drivers dont use VNET I shouldnt :P What is wrong with this hack? br, P.S. I have printed porting to vnet text to have it always at hand, but its a bit hard for me... doing my best. On Wed, Feb 2, 2011 at 6:30 PM, Julian Elischerjul...@freebsd.org wrote: On 2/2/11 9:12 AM, Bjoern A. Zeeb wrote: On Wed, 2 Feb 2011, Monthadar Al Jaberi wrote: Hi, Thanx makes more sense, but I have noticed something weired if you can shade some light on. I added printfs one when the module is first loaded (static int event_handler(module_t module, int event, void *arg)): curthread=0xc3f95870 curthread-td_vnet=0xc3170e00 curthread-td_ucred=0xc3185d00 TD_TO_VNET=0 CRED_TO_VNET=0 Try to load it from laoder on boot; I think that should work as we are setting the curvent for the kernel startup. The problem you are seeing is a bug in the current implementation that you cannot add any physical network interface after the kernel started. This applies to cardbus/usb/... as well as any kind of ethernet interface, so a kldload igb should yield it as well. The fix for that is easy and hard at the same time: A) either touch all drivers B) or touch all cloned interfaces and change 3 common lines. or try to make cloners aware of vimages. Solution B) is sitting in perforce with the entire stuff that it depends on and was started with CH=179022,179255 but not limited to that if you want to have a peek. What you certainly can do locally to your driver for now is to make a change like this: +#ifdef VIMAGE + CURVNET_SET(vnet0); +#endif ifp = if_alloc(IFT_ETHER); +#ifdef VIMAGE + CURVNET_RESTORE(); +#endif you don't really need the #ifdef except for readability as CURVNET_XXX ar enot defined for !vnet It's the type A) kind of change from above that will break eventually in the future. /bz ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
