Re: User mode drivers: part 2: PCI device handling (patch 1/2 for 2.6.11)
On Llu, 2005-03-14 at 00:13, Peter Chubb wrote: > Greg> see mmap(2) > > mmap maps a file's contents into your own virtual memory. > usr_pci_map maps part of your own virtual memory into pci bus space > for a particular device (using the IOMMU if your machine has one), and > returns a scatterlist of bus addresses to hand to the device. You can't really do it that way around because you don't know what the memory constraints of the device are compared to your user pages. Suppose your user pages are in high memory over 4GB and the device is 32bit DMA constrained ? You don't want bounce buffers clearly. In addition you have to be very careful about shared pages when doing DMA because you don't want to DMA into a COW page but that is handleable (as is done by O_DIRECT) Alan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: User mode drivers: part 2: PCI device handling (patch 1/2 for 2.6.11)
On Gwe, 2005-03-11 at 21:04, Albert Cahalan wrote: > > Still insufficient because the device might be hotplugged on you. You > > need a file handle that has the expected revocation effects on unplug > > and refcounts > > I was under the impression that a file handle would be returned. Then lets use that popular "open" system call - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: User mode drivers: part 2: PCI device handling (patch 1/2 for 2.6.11)
> "Greg" == Greg KH <[EMAIL PROTECTED]> writes: Greg> On Fri, Mar 11, 2005 at 07:34:46PM +1100, Peter Chubb wrote: >> > "Greg" == Greg KH <[EMAIL PROTECTED]> writes: >> Greg> On Fri, Mar 11, 2005 at 02:37:17PM +1100, Peter Chubb wrote: >> >> +/* + * The PCI subsystem is implemented as yet-another pseudo >> >> filesystem, + * albeit one that is never mounted. + * This is >> its >> magic number. + */ +#define USR_PCI_MAGIC (0x12345678) >> Greg> If you make it a real, mountable filesystem, then you don't need Greg> to have any of your new syscalls, right? Why not just do that Greg> instead? >> >> >> The only call that would go is usr_pci_open() -- you'd still need >> usr_pci_map() Greg> see mmap(2) mmap maps a file's contents into your own virtual memory. usr_pci_map maps part of your own virtual memory into pci bus space for a particular device (using the IOMMU if your machine has one), and returns a scatterlist of bus addresses to hand to the device. Different semantics entirely. Greg> In fact, both of the above can be done today from /proc/bus/pci/ Greg> right? Nope. -- Dr Peter Chubb http://www.gelato.unsw.edu.au peterc AT gelato.unsw.edu.au The technical we do immediately, the political takes *forever* - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: User mode drivers: part 2: PCI device handling (patch 1/2 for 2.6.11)
Here's a two year old patch for doing the same thing inside of UML but it was never merged. The advantage to this scheme is that UML implements the kernel. The same driver source code can be used inside UML and the main kernel. I modified UML so that it implemented all of the common calls drivers need, like ioremap, request_irq, etc. The UML implementation of these calls is a proxy for a device driver in the main kernel that actually gets the DMA consistent memory, irq, etc. I was able to get DRM, fbdev and USB all running unchanged inside UML with this patch. The entire device driver is debugable with GDB. I never solved the problem of UML crashing without acking a pending interrupt. But now it seems that that problem does not exist on PCI Express. BTW, I found this to be a great way to debug drivers that don't need real-time response to interrupts. -- Jon Smirl [EMAIL PROTECTED] vpci.tgz Description: application/compressed-tar
Re: User mode drivers: part 2: PCI device handling (patch 1/2 for 2.6.11)
On Fri, 11 Mar 2005 14:37:17 +1100, Peter Chubb
<[EMAIL PROTECTED]> wrote:
> + npages = get_user_pages(current,
> + current->mm,
> + (unsigned long)m.virtaddr,
> + maxpages,
> + write,
> + 0,
> + imp->pages,
> + NULL);
Can't comment on usermode drivers overall, so just some code comments.
do you need a down_read(current->mm->mmap_sem) here? I'm not sure but
that seems to be what most other users of this function are doing.
> + /*
> +* Build scatterlist, one entry per page.
> +* Allow for partial pages at start and end.
> +*/
> + i = 1;
> + imp->sg[0].page = imp->pages[0];
> + imp->sg[0].offset = ((unsigned long)m.virtaddr) & (PAGE_SIZE - 1);
> + imp->sg[0].length = PAGE_SIZE - imp->sg[0].offset;
> + if (imp->sg[0].length >= m.size) {
> + imp->sg[0].length = m.size;
> + } else {
> + unsigned long len = m.size - imp->sg[0].length;
> + for (;len >= PAGE_SIZE && i < npages ; i++) {
> + imp->sg[i].page = imp->pages[i];
> + imp->sg[i].offset = 0;
> + imp->sg[i].length = PAGE_SIZE;
> + len -= PAGE_SIZE;
> + }
> + if (len) {
> + BUG_ON(i >= npages);
> + BUG_ON(len >= PAGE_SIZE);
> + imp->sg[i].page = imp->pages[i];
> + imp->sg[i].offset = 0;
> + imp->sg[i].length = len;
> + i++;
> + }
> + }
size_t len = m.size;
int offset = (unsigned long) m.virtaddr & ~PAGE_MASK;
for (i = 0; i < npages; i++)
{
imp->sg[i].page = imp->pages[i];
imp->sg[i].offset = offset;
imp->sg[i].length = min(len, PAGE_SIZE - offset);
offset = 0;
len -= imp->sg[i].length;
}
instead possibly?
Regards -- Andy
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: User mode drivers: part 2: PCI device handling (patch 1/2 for 2.6.11)
On Fri, 2005-03-11 at 19:15 +, Alan Cox wrote: > > You forgot the PCI domain (a.k.a. hose, phb...) number. > > Also, you might encode bus,slot,function according to > > the PCI spec. So that gives: > > > > long usr_pci_open(unsigned pcidomain, unsigned devspec, __u64 dmamask); > > Still insufficient because the device might be hotplugged on you. You > need a file handle that has the expected revocation effects on unplug > and refcounts I was under the impression that a file handle would be returned. I'm not so sure that is a sane way to handle hot-plug though. First of all, in general, it's going to be like this: Fan, meet shit. Shit, meet fan. Those who care might best be served by SIGBUS with si_code and si_info set appropriately. Perhaps a revoke() syscall that handled mmap() would work the same way. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: User mode drivers: part 2: PCI device handling (patch 1/2 for 2.6.11)
On Gwe, 2005-03-11 at 08:34, Peter Chubb wrote: > Greg> If you make it a real, mountable filesystem, then you don't need > Greg> to have any of your new syscalls, right? Why not just do that > Greg> instead? > The only call that would go is usr_pci_open() -- you'd still need > usr_pci_map(), usr_pci_unmap() and usr_pci_get_consistent(). mmap, munmap, mmap with M_SYNC - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: User mode drivers: part 2: PCI device handling (patch 1/2 for 2.6.11)
> You forgot the PCI domain (a.k.a. hose, phb...) number. > Also, you might encode bus,slot,function according to > the PCI spec. So that gives: > > long usr_pci_open(unsigned pcidomain, unsigned devspec, __u64 dmamask); Still insufficient because the device might be hotplugged on you. You need a file handle that has the expected revocation effects on unplug and refcounts Alan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: User mode drivers: part 2: PCI device handling (patch 1/2 for 2.6.11)
On Friday, March 11, 2005 7:21 am, Greg KH wrote: > > The only call that would go is usr_pci_open() -- you'd still need > > usr_pci_map() > > see mmap(2) > > > , usr_pci_unmap() > > see munmap(2) Aren't those different cases though? E.g. you might have a buffer in userland that you want to DMA to a card, sure you can mmap the registers to program the DMA, but you need a way to pin the memory and get the bus address to send down, right? > In fact, both of the above can be done today from /proc/bus/pci/ right? > > > and usr_pci_get_consistent(). > > Hm, this one might be different. How about just opening and mmap a new > file for the pci device for this? New syscalls seem like the cleanest solution. For basic, non-dma programming though, I'd suggest just using sysfs, since all the PCI resources are exported there; you can mmap them and do simple programming that way. As for the idea of userland drivers in general, I'm not sure I like it. I'm afraid it will just encourage more undebuggable, x86 binary blobs (or big source code blobs with their own bridge drivers like X) that people are forced to run to get their hardware to work... Jesse - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: User mode drivers: part 2: PCI device handling (patch 1/2 for 2.6.11)
On Fri, Mar 11, 2005 at 07:34:46PM +1100, Peter Chubb wrote: > > "Greg" == Greg KH <[EMAIL PROTECTED]> writes: > > Greg> On Fri, Mar 11, 2005 at 02:37:17PM +1100, Peter Chubb wrote: > >> +/* + * The PCI subsystem is implemented as yet-another pseudo > >> filesystem, + * albeit one that is never mounted. + * This is its > >> magic number. + */ +#define USR_PCI_MAGIC (0x12345678) > > Greg> If you make it a real, mountable filesystem, then you don't need > Greg> to have any of your new syscalls, right? Why not just do that > Greg> instead? > > > The only call that would go is usr_pci_open() -- you'd still need > usr_pci_map() see mmap(2) > , usr_pci_unmap() see munmap(2) In fact, both of the above can be done today from /proc/bus/pci/ right? > and usr_pci_get_consistent(). Hm, this one might be different. How about just opening and mmap a new file for the pci device for this? thanks, greg k-h - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: User mode drivers: part 2: PCI device handling (patch 1/2 for 2.6.11)
Hi! > >> +/* + * The PCI subsystem is implemented as yet-another pseudo > >> filesystem, + * albeit one that is never mounted. + * This is its > >> magic number. + */ +#define USR_PCI_MAGIC (0x12345678) > > Greg> If you make it a real, mountable filesystem, then you don't need > Greg> to have any of your new syscalls, right? Why not just do that > Greg> instead? > > > The only call that would go is usr_pci_open() -- you'd still need > usr_pci_map(), usr_pci_unmap() and usr_pci_get_consistent(). There are quite a lot of buses. Having to add #buses * 3 seems quite wrong to me... Pavel -- People were complaining that M$ turns users into beta-testers... ...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl! - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: User mode drivers: part 2: PCI device handling (patch 1/2 for 2.6.11)
> "Greg" == Greg KH <[EMAIL PROTECTED]> writes: Greg> On Fri, Mar 11, 2005 at 02:37:17PM +1100, Peter Chubb wrote: >> +/* + * The PCI subsystem is implemented as yet-another pseudo >> filesystem, + * albeit one that is never mounted. + * This is its >> magic number. + */ +#define USR_PCI_MAGIC (0x12345678) Greg> If you make it a real, mountable filesystem, then you don't need Greg> to have any of your new syscalls, right? Why not just do that Greg> instead? The only call that would go is usr_pci_open() -- you'd still need usr_pci_map(), usr_pci_unmap() and usr_pci_get_consistent(). -- Dr Peter Chubb http://www.gelato.unsw.edu.au peterc AT gelato.unsw.edu.au The technical we do immediately, the political takes *forever* - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: User mode drivers: part 2: PCI device handling (patch 1/2 for 2.6.11)
On Fri, Mar 11, 2005 at 02:37:17PM +1100, Peter Chubb wrote: > +/* > + * The PCI subsystem is implemented as yet-another pseudo filesystem, > + * albeit one that is never mounted. > + * This is its magic number. > + */ > +#define USR_PCI_MAGIC (0x12345678) If you make it a real, mountable filesystem, then you don't need to have any of your new syscalls, right? Why not just do that instead? thanks, greg k-h - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: User mode drivers: part 2: PCI device handling (patch 1/2 for 2.6.11)
Peter Chubb writes: > There are three new system calls: > > long usr_pci_open(int bus, int slot, int function, __u64 dma_mask); > Returns a filedescriptor for the PCI device described > by bus,slot,function. It also enables the device, and sets it > up as a bus-mastering DMA device, with the specified dma mask. You forgot the PCI domain (a.k.a. hose, phb...) number. Also, you might encode bus,slot,function according to the PCI spec. So that gives: long usr_pci_open(unsigned pcidomain, unsigned devspec, __u64 dmamask); (with the user library returning an int instead of long) - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
User mode drivers: part 2: PCI device handling (patch 1/2 for 2.6.11)
USER LEVEL DRIVERS: enable PCI device drivers at user space. This patch adds the capability for suitably privileged user-level processes to enable a PCI device, and set up DMA for it. A subsequent patch hooks up the actual system calls. There are three new system calls: long usr_pci_open(int bus, int slot, int function, __u64 dma_mask); Returns a filedescriptor for the PCI device described by bus,slot,function. It also enables the device, and sets it up as a bus-mastering DMA device, with the specified dma mask. Error codes are: ENOMEM: insufficient kernel memory to fulfil your request ENOENT: the specified device doesn't exist, or is otherwise invisible to Linux. EBUSY: Another driver has claimed the device EIO: The specified dma mask is invalid for this device. ENFILE: too many open files long usr_pci_get_consistent(int fd, size_t size, void **vaddrp, unsigned long *dmaaddrp) Call pci_alloc_consistent() to get size worth of pci consistent memory (currently an error if size != PAGESIZE); map the allocated memory into the user's address space; return the virtual user address in *vaddrp, and the bus address in *dmaaddrp ERRORS: EINVAL: the filedescriptor was not one obtained from usr_pci_open(), or size != PAGESIZE ENOMEM: insufficient appropriate memory or insufficient free virtual address space in the user program. EFAULT: vaddrp or dmaaddrp didn't point to writeable memory. The mapping obtained can be cleaned up with munmap(). long usr_pci_mmap(int fd, struct mapping_info *mp) -- map some memory for DMA to/from the device represented by fd, which was obtained from usr_pci_open(). struct mapping_info contains: void *virtaddr -- the virtual address to dma to int size -- how many bytes to set up struct usr_pci_sglist *sglist -- a pointer to a scatterlist int nents -- how many entries in the scatterlist enum dma_data_direction direction --- which way the dma is going to happen. The scatterlist should be sized at least size/PAGESIZE + 2. usr_pci_mmap() will call pci_map_sg() on the virtual region, then copy the resulting scatterlist into *sglist. The nents field will be updated with the actual number of scatterlist entries filled in. Failure codes are: EINVAL: the fd wasn't obtained from usr_pci_open, or direction wasn't one of DMA_TO_DEVICE, DMA_FROM_DEVICE or DMA_BIDIRECTIONAL, or the size of the scatterlist is insufficient to map the region. EFAULT: mp was a bad pointer, or the region of memory spanned by (virtaddr, virtaddr + size) was not all mapped. ENOMEM: insufficient appropriate memory long usr_pci_munmap(int fd, struct mapping_info *mp) Unmap a dma region mapped by usr_pci_map(). Struct mapping info is the same one used in usr_pci_mmap(). Error codes are: EINVAL: : the fd wasn't obtained from usr_pci_open, or the struct mapping_info was never mapped for this device Signed-off-by: Peter Chubb <[EMAIL PROTECTED]> # # drivers/Makefile |3 # drivers/pci/Kconfig|6 # drivers/usr/Makefile |2 # drivers/usr/sys.c | 952 + # include/linux/usrdrv.h | 63 +++ # 5 files changed, 1026 insertions(+) # Index: linux-2.6.11-usrdrivers/drivers/Makefile === --- linux-2.6.11-usrdrivers.orig/drivers/Makefile 2005-03-11 12:25:29.169139978 +1100 +++ linux-2.6.11-usrdrivers/drivers/Makefile2005-03-11 12:25:41.159270471 +1100 @@ -13,6 +13,9 @@ # was used and do nothing if so obj-$(CONFIG_PNP) += pnp/ +# User level device drivers +obj-$(CONFIG_USRDEV) += usr/ + # char/ comes before serial/ etc so that the VT console is the boot-time # default. obj-y += char/ Index: linux-2.6.11-usrdrivers/drivers/usr/Makefile === --- /dev/null 1970-01-01 00:00:00.0 + +++ linux-2.6.11-usrdrivers/drivers/usr/Makefile2005-03-11 12:25:41.160247026 +1100 @@ -0,0 +1,2 @@ +obj-y += sys.o +obj-$(CONFIG_USRBLKDEV) += blkdev.o Index: linux-2.6.11-usrdrivers/drivers/usr/sys.c === --- /dev/null 1970-01-01 00:00:00.0 + +++ linux-2.6.11-usrdrivers/drivers/usr/sys.c 2005-03-11 14:15:59.897394833 +1100 @@ -0,0 +1,952 @@ +/* + * Expose PCI-DMA interface to user mode. + * + * Copyright 2005 Peter Chubb + * National ICT Au
