Multiprocessing is a decent solution for concurrency, but alas there are some situations where message passing is insufficient.
LibUSB has an asynchronous API ( http://libusb.sourceforge.net/api-1.0/group__asyncio.html), and you're pretty much stuck using it if you wish to achieve anywhere close to the throughput USB is capable of reaching on a modern computer, or if you wish to provide your application with a modicum of jitter-resistance by having an outstanding transfer queue. Using this API requires an event handler to be called either frequently or in a separate thread. I don't think multiprocess could be made to work here (one process for running data handling callbacks, one process for polling the handler function) due to the requirements for the polling task sharing memory with the application task. Alternatively, on Linux and OSX, LibUSB can spit out file descriptors that can be monitored via poll/select in the main event loop, something along the lines of how sdasync works. Unfortunately, I don't think the libJ event loop / jconsole event loop is exposed, at least not to the extent of registering additional file descriptors for monitoring. Any thoughts here? I've been able to prototype my application well enough with the synchronous API, but I'd like to bring jitter* down and throughput up, and it looks like this is going to be sufficiently difficult to pull off in J. Thanks! -- Ian * I'm concerned about the 1-10ms stalls frequently demonstrated by USB bulk transfers under moderate system load, not the hundred microseconds in either direction that I've observed to be typical of an interpreted J program. On Wed, Jul 10, 2013 at 7:05 AM, Jack Andrews <[email protected]> wrote: > On 10 July 2013 02:42, I.T. Daniher <[email protected]> > wrote in thread "ymacs": > > .... This is perhaps more a petition for better > > threading / async support in J in general > > As Raul said in the ymacs thread, concurrency (async) can be acheived > using multiple processes [multiprocessing]. Trendy things like .NET > and Java do multithreading. > > A language implementation that doesn't support multithreading is not > necessarily deficient. For example, through the language 'k' (an > APL inspired language), kdb+ has blown away competitors in a STAC M3 > benchmark [focused on finance and automated trading] partially with a > new multiprocessing feature, which performs better than multithreading > in this context. > > Multiple processes can communicate using sockets (as Raul illustrated) > Using sockets involves "message passing" which is a lot easier than > using threads and locks. Erlang, a language built around concurrency, > uses message passing. > > On 10 July 2013 02:42, I.T. Daniher <[email protected]> wrote: > > Unfortunately, the collection of data isn't always a low-overhead task, > so > >... workflow consists of writing an experiment, running the script, and > viewing > > / processing the resulting data. > > ... J IDEs to be ill suited to a more realtime/interactive paradigm due > to UI > > requests, and in some cases, interaction with the text editor window > > itself, being blocked by "long-running" data collection. > > Here's a recipe for multiprocessing your example: > - start a J process that listens for 'experiments' on socket with port 4321 > - start the J IDE and send an experiment message (eg: send 'sin 0.2 * > i.60') to port 4321. > - continue to refine your experiment in the IDE > - when the the result message arrives, the J IDE can pop up a plot. . > > In addition to Raul's pointers, this might be a good place to start: > http://www.jsoftware.com/jwiki/Studio/Socket%20Driver > > Ta, Jack. > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
