> Implement your USB i/o handler in your preferred language. Make the handler a server listening on a socket
Yeah, that's my current plan. I really like J for DSP prototyping, but I think I am just going to have to bite the bullet and build out the USB level in a systems language. Thanks! -- Ian On Mon, Jul 15, 2013 at 7:28 PM, Jack Andrews <[email protected]> wrote: > I don't have time to fully work through an example... But a few points: > > - Implement your USB i/o handler in your preferred language. Make the > handler a server listening on a socket > - write your UI in J and send messages to your handler and select() on the > socket, async'ly looking for data. > > Sockets are implemented using shared memory when you are sending messages > on the same host, so it should be fast enough - easily 4MB/s. Even on > different hosts, you can get [more than] 100Mb across Ethernet. > > On Tuesday, July 16, 2013, I.T. Daniher wrote: > > > Raul, > > > > Thanks for your response. I do, in fact, have access to a variety of test > > equipment, but I already have a strong theory as to where the variable > > latency originates, informed by my startup's first draft to handle > > streaming USB data processing, https://github.com/nonolith/connect. > > > > I believe the aforementioned USB latency to originate between the > userspace > > and the kernel, where the libusb userspace drivers hand off transactions > > (IN or OUT) to the kernel, and, in turn, the USB controller. The way we > > solved this problem with mkI was to have a buffer of pending > transactions, > > submitted to the kernel well in advance of their expected completion, and > > to receive callbacks when data is ready. LibUSB makes the realization of > > this design pattern very easy, via their async API. This allows for > robust > > high-throughput streaming regardless of the CPU utilization, as the > kernel > > does not need to wait for userspace communication to happen in order to > > pass off a request to the hardware. > > > > My issue here originates with attempting to use the API which decouples > > submission and completion of transactions with J, which seems to be > unable > > to share memory between tasks as needed to call the libusb event handler > as > > per the "simple" route ( > > > > > http://libusb.sourceforge.net/api-1.0/group__poll.html#ga0bc99f39e4cf5ad393cd5936c36037d1 > , > > ) > > and equally unable to add file descriptors produced via > > > > > http://libusb.sourceforge.net/api-1.0/group__poll.html#gab1a72869a926552b27a6c667695df3a2 > > to > > the internal event loop used for the async socket operation. > > > > A few plots: > > > > * http://itdaniher.com/static/bulkReadUnderCPULoad.html > > * http://itdaniher.com/static/bulkReadNOCPULoad.html > > > > The plots show the time difference between series attempts to read 8192B > > chunks from a USB device producing information at a constant 4.096 > > megabytes per second, for an expected tick period of 2ms. > > > > Note that with CPU load, the average read duration is not significantly > > longer (same datarate / average 2ms tick) but the peak delay between > > subsequent completed transactions is considerably higher. > > > > With transaction / request buffering via the async API, the average > > device-host latency is not changed, but the peak delay seldom hits that > > 1ms+ delay demonstrated above. > > > > It's worth noting that the problem is substantially worse with smaller > > chunks - 8192B requests sixteen full 512B packets, essentially emulating > > the request buffer paradigm described above, in miniature, with higher > > latency. If you look at > > http://itdaniher.com/static/bulkReadUnderCPULoad-512B.html, you can see > > that similar duration delays occur, even with smaller packets, but the > net > > effect is considerably more extreme. > > > > Please let me know if I can clarify any of the above - this particular > > world where hardware and software collides is a rather strange place. > > > > Thanks for your assistance! > > -- > > Ian > > > > > > On Mon, Jul 15, 2013 at 10:22 AM, Raul Miller <[email protected] > <javascript:;>> > > wrote: > > > > > On Mon, Jul 15, 2013 at 9:01 AM, I.T. Daniher <[email protected] > <javascript:;> > > > > > > wrote: > > > > Any thoughts here? I've been able to prototype my application well > > enough > > > > with the synchronous API, but I'd like to bring jitter* down and > > > throughput > > > > up, and it looks like this is going to be sufficiently difficult to > > pull > > > > off in J. > > > ... > > > > * I'm concerned about the 1-10ms stalls frequently demonstrated by > USB > > > bulk > > > > transfers under moderate system load, not the hundred microseconds in > > > > either direction that I've observed to be typical of an interpreted J > > > > program. > > > > > > Do you have access to any monitoring gear of any sort? > > > > > > There's a huge variety of monitoring gear available to you, > oscilloscopes > > > are > > > maybe the crudest electronic monitoring mechanism with any plausible > > > relevance, and usb monitoring software would probably be the highest > > level > > > relevant tool for this kind of thing. > > > > > > Anyways, it seems to me that you need to isolate your problem before > > > you can solve it. > > > > > > But it sounds to me as if you can reproduce the problem easily, and > > that's > > > the hardest part... > > > > > > If you get stuck, we may need to bring in other people, so once you've > > > made some progress, please do not hesitate to ask for help. > > > > > > Thanks, > > > > > > -- > > > Raul > > > ---------------------------------------------------------------------- > > > For information about J forums see http://www.jsoftware.com/forums.htm > > > > > ---------------------------------------------------------------------- > > For information about J forums see http://www.jsoftware.com/forums.htm > > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
