On Tue, 8 Dec 2015, Steinar H. Gunderson wrote:
> Do you know what the status is for the LPM blacklisting patch we discussed
> earlier?
Last I heard, Baolu Lu was working on a more general solution. In its
absence, maybe I should go ahead and submit the patch.
Alan Stern
--
To unsubscribe fro
On Tue, Dec 08, 2015 at 11:07:55PM +0100, Steinar H. Gunderson wrote:
> On Tue, Dec 08, 2015 at 05:01:46PM -0500, Alan Stern wrote:
> > I don't see anything else to change. You can submit this to Greg KH
> > and add:
> >
> > Acked-by: Alan Stern
>
> Great!
>
> How do I submit? Just do git sen
On Tue, Dec 08, 2015 at 05:01:46PM -0500, Alan Stern wrote:
> I don't see anything else to change. You can submit this to Greg KH
> and add:
>
> Acked-by: Alan Stern
Great!
How do I submit? Just do git send-email to some address?
Do you know what the status is for the LPM blacklisting patch
On Tue, 8 Dec 2015, Steinar H. Gunderson wrote:
> On Tue, Dec 08, 2015 at 03:26:28PM -0500, Alan Stern wrote:
> > I just noticed this -- usbm->memlist needs to be initialized before
> > dec_usb_memory_use_count() can be called. INIT_LIST_HEAD before the
> > "if" statement will fix the problem.
>
On Tue, Dec 08, 2015 at 03:26:28PM -0500, Alan Stern wrote:
> I just noticed this -- usbm->memlist needs to be initialized before
> dec_usb_memory_use_count() can be called. INIT_LIST_HEAD before the
> "if" statement will fix the problem.
Done.
> The patch could use one more ingredient. In
> in
On Mon, 7 Dec 2015, Steinar H. Gunderson wrote:
> On Mon, Dec 07, 2015 at 10:45:55AM -0500, Alan Stern wrote:
> >> Here we are. Lightly tested, I believe all comments should be addressed.
> > This looks quite good. I have only a couple of comments.
>
> Excellent. I'm sorry we've needed so many r
On Mon, Dec 07, 2015 at 10:45:55AM -0500, Alan Stern wrote:
>> Here we are. Lightly tested, I believe all comments should be addressed.
> This looks quite good. I have only a couple of comments.
Excellent. I'm sorry we've needed so many rounds; this is what happens when
you pick up someone else's
On Sun, 6 Dec 2015, Steinar H. Gunderson wrote:
> On Sun, Dec 06, 2015 at 01:06:08AM +0100, Steinar H. Gunderson wrote:
> > I'll try to update your patch with the other suggestions tomorrow. Thanks!
>
> Here we are. Lightly tested, I believe all comments should be addressed.
This looks quite goo
On Sun, Dec 06, 2015 at 01:06:08AM +0100, Steinar H. Gunderson wrote:
> I'll try to update your patch with the other suggestions tomorrow. Thanks!
Here we are. Lightly tested, I believe all comments should be addressed.
/* Steinar */
>From 73833276a4f359c35edffc2a741dba57f2e82a12 Mon Sep 17 00:0
On Sun, 6 Dec 2015, Steinar H. Gunderson wrote:
> >>> You don't really need to do it earlier. An unnecessary calculation of
> >>> num_sgs won't hurt.
> >> Is it unnecessary? I thought the test was really to force num_sgs==0 for
> >> the
> >> DMA case, not to save a little arithmetic.
> > Well, i
On Sat, Dec 05, 2015 at 05:12:03PM -0500, Alan Stern wrote:
> To tell the truth, I'm not sure whether it would be a problem or not.
> That's why I said it "may" not be a good idea.
Fair enough.
>>> You don't really need to do it earlier. An unnecessary calculation of
>>> num_sgs won't hurt.
>>
On Sat, 5 Dec 2015, Steinar H. Gunderson wrote:
> On Sat, Dec 05, 2015 at 12:36:51PM -0500, Alan Stern wrote:
> > I meant that this "if" statement should test only uurb_start. If the
> > test succeeds, a nested "if" statement should test buffer_length and
> > return an error if it is too big.
>
On Sat, Dec 05, 2015 at 12:36:51PM -0500, Alan Stern wrote:
> I meant that this "if" statement should test only uurb_start. If the
> test succeeds, a nested "if" statement should test buffer_length and
> return an error if it is too big.
>
> That's as opposed to the way you have it now, where i
On Sat, 5 Dec 2015, Steinar H. Gunderson wrote:
> > (Yes, I did.) On further thought, testing uurb_start is sufficient.
> > If uurb->buffer_length then turns out to be too big, the function
> > should return an error (or rather, an ERR_PTR) and the URB submission
> > should fail.
>
> I don't und
On Wed, Dec 02, 2015 at 10:54:52AM -0500, Alan Stern wrote:
> Well, it's partly a matter of taste. But there's also a race: This
> code adds usbm to ps->memory_list (making it available to URB
> submissions running on other CPUs) before usbm has been completely
> initialized (vm_start isn't set
On Fri, 4 Dec 2015, Steinar H. Gunderson wrote:
> On Fri, Dec 04, 2015 at 01:29:05AM +0100, Steinar H. Gunderson wrote:
> > I must be doing something wrong, because I don't seem to get any frames
> > from my video capture, and when I close the application, I get a slowpath
> > warning:
>
> OK, th
On Thu, Dec 03, 2015 at 04:31:35PM -0500, Alan Stern wrote:
>> Seemingly controller is already a pointer, so the & is redundant. No idea why
>> it didn't crash plain out. Fixing that, I can allocate, although I have some
>> other bug causing an oops somewhere else.
> Ah, an easy mistake to miss. B
On Fri, Dec 04, 2015 at 01:29:05AM +0100, Steinar H. Gunderson wrote:
> I must be doing something wrong, because I don't seem to get any frames
> from my video capture, and when I close the application, I get a slowpath
> warning:
OK, the slowpath warning is a red herring -- it's just because I do
On Thu, Dec 03, 2015 at 10:18:20PM +0100, Steinar H. Gunderson wrote:
> But you still think usb_alloc_coherent() is the better way to go? Should be
> easy enough to change, just let me know.
I must be doing something wrong, because I don't seem to get any frames
from my video capture, and when I c
On Thu, 3 Dec 2015, Steinar H. Gunderson wrote:
> On Thu, Dec 03, 2015 at 10:37:27AM -0500, Alan Stern wrote:
> >> FWIW, I tried against &ps->dev->bus->controller, and it didn't give me any
> >> error messages, but still returns NULL.
> > I'm at a loss.
>
> Seemingly controller is already a poi
On Thu, Dec 03, 2015 at 10:37:27AM -0500, Alan Stern wrote:
>> FWIW, I tried against &ps->dev->bus->controller, and it didn't give me any
>> error messages, but still returns NULL.
> I'm at a loss.
Seemingly controller is already a pointer, so the & is redundant. No idea why
it didn't crash plai
On Thu, 3 Dec 2015, Steinar H. Gunderson wrote:
> On Wed, Dec 02, 2015 at 10:54:52AM -0500, Alan Stern wrote:
> >> [ 28.796244] DMAR: Allocating domain for 2-2 failed
> > I don't know what the reason is for that. It may be that your kernel
> > isn't configured to allocate as much coherent me
On Thu, Dec 03, 2015 at 12:51:14AM +0100, Steinar H. Gunderson wrote:
> I'm thinking; maybe should the memory not be allocated against the USB
> device, but against the controller it hangs on? (Note that it complains about
> allocating the _domain_, not the memory itself.) Do you know if that's
> p
On Wed, Dec 02, 2015 at 10:54:52AM -0500, Alan Stern wrote:
>> [ 28.796244] DMAR: Allocating domain for 2-2 failed
> I don't know what the reason is for that. It may be that your kernel
> isn't configured to allocate as much coherent memory as you are asking
> for. We'll have to investigate
On Tue, 1 Dec 2015, Steinar H. Gunderson wrote:
> >> + usbm->mem = mem;
> >> + usbm->offset = virt_to_phys(mem);
> >> + usbm->size = size;
> >> + usbm->ps = ps;
> >> + spin_lock_irqsave(&ps->lock, flags);
> >> + list_add_tail(&usbm->memlist, &ps->memory_list);
> >> + spin_unlock_irqrestore(
On Mon, Nov 30, 2015 at 12:14:59PM -0500, Alan Stern wrote:
> As I recall, Markus's original patch took care of this by checking to
> see whether the transfer buffer was in one of the mmap'ed areas. If it
> was then the transfer would be zerocopy; otherwise it would be normal.
> That seems like
On Mon, 30 Nov 2015, Oliver Neukum wrote:
> On Mon, 2015-11-30 at 12:14 -0500, Alan Stern wrote:
> > > +static struct usb_memory *
> > > +find_memory_area(struct usb_dev_state *ps, const struct
> > usbdevfs_urb *uurb)
> > > +{
> > > + struct usb_memory *usbm = NULL, *iter;
> > > + unsigned
On Mon, 2015-11-30 at 12:14 -0500, Alan Stern wrote:
> > +static struct usb_memory *
> > +find_memory_area(struct usb_dev_state *ps, const struct
> usbdevfs_urb *uurb)
> > +{
> > + struct usb_memory *usbm = NULL, *iter;
> > + unsigned long flags;
> > + unsigned long uurb_start = (unsign
On Thu, 26 Nov 2015, Steinar H. Gunderson wrote:
> On Wed, Nov 25, 2015 at 10:29:53AM -0500, Alan Stern wrote:
> > I want to see a modified version of your patch. Several things need to
> > be changed or fixed, but the major change needs to be the way memory is
> > allocated. It should be done
On Fri, 2015-11-27 at 00:04 +0100, Steinar H. Gunderson wrote:
> On Thu, Nov 26, 2015 at 01:26:32AM +0100, Steinar H. Gunderson wrote:
> >> There are numerous smaller issues: The allocated memory needs to be
> >> charged against usbfs_memory_usage
> > I'll fix this.
>
> Fixed in updated patch (at
On Thu, Nov 26, 2015 at 01:26:32AM +0100, Steinar H. Gunderson wrote:
>> There are numerous smaller issues: The allocated memory needs to be
>> charged against usbfs_memory_usage
> I'll fix this.
Fixed in updated patch (attached).
> I'll fix this, too.
Also fixed.
Now also checkpatch clean.
/
On Wed, Nov 25, 2015 at 10:29:53AM -0500, Alan Stern wrote:
> I want to see a modified version of your patch. Several things need to
> be changed or fixed, but the major change needs to be the way memory is
> allocated. It should be done as part of the mmap system call, not as a
> separate ioc
On Wed, Nov 25, 2015 at 10:29:53AM -0500, Alan Stern wrote:
> I want to see a modified version of your patch. Several things need to
> be changed or fixed, but the major change needs to be the way memory is
> allocated. It should be done as part of the mmap system call, not as a
> separate ioc
On Wed, 25 Nov 2015, Markus Rechberger wrote:
> Seems like it's getting stuck again, since we're entering devices with
> a higher bandwidth need now it seems to be necessary to really fix it
> in the kernel.
>
> I propose add it as it is, it's proven that it works and there are 2
> testcases out
Seems like it's getting stuck again, since we're entering devices with
a higher bandwidth need now it seems to be necessary to really fix it
in the kernel.
I propose add it as it is, it's proven that it works and there are 2
testcases out there right now and it decreases the CPU load on intel
Atom
By the way QNAP NAS systems are shipped with a 64bit kernel but a
32bit system environment.
Those systems support USB 2.0 and USB 3.0.
You can expect any kind of combination out there.
On Wed, Nov 18, 2015 at 7:23 PM, Bjørn Mork wrote:
> "Steinar H. Gunderson" writes:
>> On Tue, Nov 17, 2015 a
"Steinar H. Gunderson" writes:
> On Tue, Nov 17, 2015 at 03:16:55PM -0500, Alan Stern wrote:
>> xHCI always uses 64-bit addresses. But many EHCI controllers don't,
>> and only a few of the EHCI platform drivers support 64-bit DMA.
>
> OK, sure. But are systems with USB2 only and more than 4GB of
On Wed, 18 Nov 2015, Peter Stuge wrote:
> Alan Stern wrote:
> > if we can directly map the user-provided buffer for DMA.
>
> Given a memory buffer either kernel or userspace has to apply the
> hardware constraints to find out what part if any is usable for DMA.
>
> I think it's fine to push this
From: Christoph Hellwig
> Sent: 18 November 2015 14:02
> On Tue, Nov 17, 2015 at 02:07:58PM -0500, Alan Stern wrote:
> > If we really want to do zerocopy I/O then we should not use a bounce
> > buffer. But the only way to avoid bounce buffers may be to give user
> > programs a way of allocating me
On Wed, Nov 18, 2015 at 3:02 PM, Christoph Hellwig wrote:
> On Tue, Nov 17, 2015 at 02:07:58PM -0500, Alan Stern wrote:
>> If we really want to do zerocopy I/O then we should not use a bounce
>> buffer. But the only way to avoid bounce buffers may be to give user
>> programs a way of allocating m
On Tue, Nov 17, 2015 at 02:07:58PM -0500, Alan Stern wrote:
> If we really want to do zerocopy I/O then we should not use a bounce
> buffer. But the only way to avoid bounce buffers may be to give user
> programs a way of allocating memory pages below 4 GB, because lots of
> USB hardware can on
On Tue, 2015-11-17 at 14:13 -0500, Alan Stern wrote:
> On Tue, 17 Nov 2015, Steinar H. Gunderson wrote:
>
> > Alan, could we perhaps let the zerocopy flag make the request fail (instead
> > of going through a bounce buffer) if direct DMA is not possible? That way,
> > it would be quite obvious th
Alan Stern wrote:
> if we can directly map the user-provided buffer for DMA.
Given a memory buffer either kernel or userspace has to apply the
hardware constraints to find out what part if any is usable for DMA.
I think it's fine to push this task to userspace - as long as
userspace has a way to
On Tue, 17 Nov 2015, Steinar H. Gunderson wrote:
> > xHCI always uses 64-bit addresses. But many EHCI controllers don't,
> > and only a few of the EHCI platform drivers support 64-bit DMA.
>
> OK, sure. But are systems with USB2 only and more than 4GB of RAM common?
> (And will they not increas
On Tue, Nov 17, 2015 at 03:16:55PM -0500, Alan Stern wrote:
>>> But what other way of allocating memory is there?
>> For my part, GPU memory versus malloc(). (You can ask OpenGL to permanently
>> map up a chunk of GPU memory for you into userspace, but you have no
>> guarantees as of if it's DMA-ab
On Tue, 17 Nov 2015, Steinar H. Gunderson wrote:
> On Tue, Nov 17, 2015 at 02:13:49PM -0500, Alan Stern wrote:
> > But what other way of allocating memory is there?
>
> For my part, GPU memory versus malloc(). (You can ask OpenGL to permanently
> map up a chunk of GPU memory for you into userspac
On Tue, Nov 17, 2015 at 03:01:39PM -0500, Alan Stern wrote:
>> On Tue, Nov 17, 2015 at 02:07:58PM -0500, Alan Stern wrote:
>>> Is there an API for allocating user memory below 4 GB? Would a new
>>> MMAP flag be acceptable?
>> MAP_32BIT (to mmap) can seemingly do this, but from the man page, it so
On Tue, 17 Nov 2015, Steinar H. Gunderson wrote:
> On Tue, Nov 17, 2015 at 02:07:58PM -0500, Alan Stern wrote:
> > Is there an API for allocating user memory below 4 GB? Would a new
> > MMAP flag be acceptable?
>
> MAP_32BIT (to mmap) can seemingly do this, but from the man page, it sounds
> mo
On Tue, Nov 17, 2015 at 02:07:58PM -0500, Alan Stern wrote:
> Is there an API for allocating user memory below 4 GB? Would a new
> MMAP flag be acceptable?
MAP_32BIT (to mmap) can seemingly do this, but from the man page, it sounds
more like a kludge than anything else.
/* Steinar */
--
Homepa
Hi,
Markus Rechberger writes:
> Such kind of security is a little bit odd for a stack that doesn't
> work nearly as well as Mac or Windows.
And you say this based on what, exactly ?
Linux supports many more devices, it's usually faster with most devices
(actually Mac's is on par with Linux, Wi
By the way on MacOSX we also pre-allocate some buffer in userspace
first for everything.
Well it's up to you anyway I haven't seen a move into a good direction
in that area for years anyway on Linux so I'm used to deal with what
is currently available.
On Tue, Nov 17, 2015 at 8:13 PM, Alan Stern
On Tue, Nov 17, 2015 at 02:13:49PM -0500, Alan Stern wrote:
> But what other way of allocating memory is there?
For my part, GPU memory versus malloc(). (You can ask OpenGL to permanently
map up a chunk of GPU memory for you into userspace, but you have no
guarantees as of if it's DMA-able. But ty
On Tue, 17 Nov 2015, Steinar H. Gunderson wrote:
> On Tue, Nov 17, 2015 at 07:17:31PM +0100, Markus Rechberger wrote:
> > 1. the memset on isochronous transfers to empty the buffers in order
> > to avoid leaking raw memory to userspace (this costs a lot on intel
> > Atoms and is also noticeable on
On Mon, 16 Nov 2015, Christoph Hellwig wrote:
> On Mon, Nov 16, 2015 at 08:03:12PM +0100, Steinar H. Gunderson wrote:
> > The original use case: DVB capture on embedded devices.
> >
> > My use case: USB3 uncompressed HD video capture (from multiple cards).
> >
> > Both of these hit (beyond the s
Just a side note, for older videodevices it was also common to
allocate the DMA memory during bootup because it might have failed
during runtime because of memory fragmentation.
The reason for the memset on the isochronous transfer is that USB
might not fill up the entire buffer leaving half of th
On Tue, 17 Nov 2015, Markus Rechberger wrote:
> There are 2 problems with the current implementation:
>
> 1. the memset on isochronous transfers to empty the buffers in order
> to avoid leaking raw memory to userspace (this costs a lot on intel
> Atoms and is also noticeable on other systems).
I
On Tue, Nov 17, 2015 at 07:17:31PM +0100, Markus Rechberger wrote:
> 1. the memset on isochronous transfers to empty the buffers in order
> to avoid leaking raw memory to userspace (this costs a lot on intel
> Atoms and is also noticeable on other systems).
>
> 2. the memory fragmentation. Seems l
There are 2 problems with the current implementation:
1. the memset on isochronous transfers to empty the buffers in order
to avoid leaking raw memory to userspace (this costs a lot on intel
Atoms and is also noticeable on other systems).
2. the memory fragmentation. Seems like recent systems hav
On Tue, 17 Nov 2015, Christoph Hellwig wrote:
> On Mon, Nov 16, 2015 at 03:22:06PM -0500, Alan Stern wrote:
> > In other words, you're suggesting we do this:
> >
> > Check that userspace requested zerocopy (otherwise the user
> > program might try to access other data stored in the same
On Tue, 2015-11-17 at 13:07 +0100, Steinar H. Gunderson wrote:
> On Mon, Nov 16, 2015 at 03:22:06PM -0500, Alan Stern wrote:
> > Check that userspace requested zerocopy (otherwise the user
> > program might try to access other data stored in the same cache
> > lines as the buffer whil
On Mon, Nov 16, 2015 at 03:22:06PM -0500, Alan Stern wrote:
> Check that userspace requested zerocopy (otherwise the user
> program might try to access other data stored in the same cache
> lines as the buffer while the I/O is in progres);
Needed for send only, right? For recei
On Mon, Nov 16, 2015 at 03:22:06PM -0500, Alan Stern wrote:
> In other words, you're suggesting we do this:
>
> Check that userspace requested zerocopy (otherwise the user
> program might try to access other data stored in the same cache
> lines as the buffer while the I/O is i
On Mon, 16 Nov 2015, Christoph Hellwig wrote:
> On Mon, Nov 16, 2015 at 08:03:12PM +0100, Steinar H. Gunderson wrote:
> > The original use case: DVB capture on embedded devices.
> >
> > My use case: USB3 uncompressed HD video capture (from multiple cards).
> >
> > Both of these hit (beyond the s
On Mon, Nov 16, 2015 at 08:03:12PM +0100, Steinar H. Gunderson wrote:
> The original use case: DVB capture on embedded devices.
>
> My use case: USB3 uncompressed HD video capture (from multiple cards).
>
> Both of these hit (beyond the speed boost from zerocopy) the problem that
> as time goes b
On Mon, Nov 16, 2015 at 07:55:45PM +0100, Christoph Hellwig wrote:
>> A proposed patch has been posted
>> (http://marc.info/?l=linux-usb&m=144763502829452&w=2), but I'm not
>> convinced that it is the best approach. For instance, it always tries
>> to get contiguous pages and so is vulnerable to
On Mon, Nov 16, 2015 at 01:29:58PM -0500, Alan Stern wrote:
> Allocating memory pages that match the device's DMA mask, for
> use as I/O buffers, and locking them so their physical
> addresses don't change (and they don't get paged out);
>
> Mapping those pages into a use
On Mon, Nov 16, 2015 at 01:29:58PM -0500, Alan Stern wrote:
> A proposed patch has been posted
> (http://marc.info/?l=linux-usb&m=144763502829452&w=2), but I'm not
> convinced that it is the best approach. For instance, it always tries
> to get contiguous pages and so is vulnerable to memory frag
Christoph:
Can you help answer some questions about implementing zerocopy I/O?
We would like to add zerocopy support for USB devices. Basically this
means:
Allocating memory pages that match the device's DMA mask, for
use as I/O buffers, and locking them so their physical
68 matches
Mail list logo