Certainly, you can implement concurrent processes on top of sockets. And using select (or poll) can be a big help there. You can also use threads or processes but that introduces more overhead (but sometimes that overhead is insignificant for the project you're working on, when compared to dealing with the other details).
Another thing to keep in mind, though - if we are going to be using select/poll - is that sometimes select (or poll) is not enough. For example, when using tcp on linux, select can return when a tcp packet begins to arrive. But if the checksum on the packet turns out to not match the received content you can't actually read anything on that socket (and attempting to do so would block). So it's a good idea to set your socket to non-blocking IO (so that if you attempt to read and there's nothing there, it won't wait for something to show up - it'll just give you an empty result). Nevertheless, I stand by my assertion that the abstraction *provided by sockets* is a stop and wait abstraction. I agree that you can build support for other abstractions on top of this abstraction. Also, sometimes, "stop and wait" is the right abstraction. But of course, the file abstraction should also not be so lightly dismissed. Thanks, -- Raul On Thu, Feb 26, 2015 at 12:18 PM, Michael Dykman <[email protected]> wrote: > Sockets are not necessarily a stop-and-wait proposition. The management of > multiple sockets using select() is well established and is used in a quite > a lot of real-world systems for coordinating clusters of cooperative > machines. Introducing the file abstraction into the process might make the > programming somewhat less hairy but can not help but introduce latency into > your process. For a great many systems, that doesn't matter. If your > ultimate goal is to implement large-scale, actively coordinating clusters, > sockets should not be so lightly dismissed. > > On Thu, Feb 26, 2015 at 12:02 PM, Devon McCormick <[email protected]> > wrote: > >> That's an interesting point. Unfortunately, it would make my job easier >> (as I've already implemented a file-based mutex) and wouldn't give me the >> chance to learn something new. >> >> On Thu, Feb 26, 2015 at 11:49 AM, Raul Miller <[email protected]> >> wrote: >> >> > Unless you are using multiple machines, files are probably a better >> > coordination abstraction than sockets. >> > >> > (And if you are using multiple machines, you might still want to be >> > using a file-like abstraction.) >> > >> > The reason is that sockets give you a "stop and wait" abstraction >> > while files give you a "named data" abstraction. >> > >> > Also, modern OS's are pretty good about caching files, not so good >> > about caching stuff going across sockets. >> > >> > Thanks, >> > >> > -- >> > Raul >> > >> > On Thu, Feb 26, 2015 at 11:29 AM, Devon McCormick <[email protected]> >> > wrote: >> > > I run parallelized J on multi-core machines and have done so for years. >> > I >> > > looked at the "CPU affinity" thing and concluded it was a waste of time >> > and >> > > an unnecessary constraint for what I was doing. What I've found is >> that >> > > the OS - Windows in my case - does a fine job on its own of >> distributing >> > > multiple J sessions across the available cores. >> > > >> > > A detailed write-up of what I've done is here: >> > > >> > >> http://www.jsoftware.com/jwiki/Community/Conference2012/Talks/ParallelSimulationInJ >> > > . An earlier version is here: >> > > >> http://www.jsoftware.com/jwiki/DevonMcCormick/ParallelizedJCodeExamples >> > . >> > > >> > > The next advance I'd like to come up with would be to spin off >> multiple J >> > > processes, then co-ordinate work between them with something like >> > sockets. >> > > This would allow me to do some simple load-balancing. >> > > >> > > On Wed, Feb 25, 2015 at 9:04 PM, Joe Bogner <[email protected]> >> wrote: >> > > >> > >> On Wed, Feb 25, 2015 at 5:41 PM, N.N. <[email protected]> wrote: >> > >> >> > >> > 1) does Windows use the 8 processors to run the j-console ? >> > >> > >> > >> >> > >> No, J is single threaded >> > >> >> > >> 2) is it possible to run 8 j-consoles and assign each console on one >> > >> > processor ? >> > >> > >> > >> >> > >> You could, but I'm not sure what the benefit would be. You could >> assign >> > CPU >> > >> affinity to each process. Each would run independently. >> > >> >> > >> 3) is it possible to use 1 console and manually create threads >> > >> > assigning a process on the 7 remaining processors ? >> > >> > >> > >> > >> > >> J is single threaded and doesn't provide a capability to create >> threads >> > >> natively. You can spawn processes so you would need to write your own >> > code >> > >> to spawn the processes and submit the work to each process. This is >> > >> conceptually how other single threaded console apps cluster their work >> > >> (node.js and R) >> > >> >> > >> I have done some prototyping on this topic: >> > >> >> > >> 1. Integrating tinyc to just-in-time compile multi-threaded C code - >> > >> http://www.jsoftware.com/pipermail/chat/2015-February/006488.html >> > >> >> > >> 2. Integrating go-lang to function as a multi-threaded web server - >> > >> http://www.jsoftware.com/pipermail/source/2015-February/000663.html >> > >> >> > >> 3. Spawning another J process and duplicating the socket to >> effectively >> > >> function as a multi-threaded server - >> > >> http://www.jsoftware.com/pipermail/chat/2013-August/005328.html >> > >> >> > >> >> > >> Still, it would be worthwhile to understand your use-case for >> > threading. My >> > >> experience has been that threading is most useful when different tasks >> > need >> > >> to run in parallel. I haven't seen as much benefit in breaking up a >> > single >> > >> task into chunks and executing it on multiple threads / processes. J's >> > >> performance has been largely sufficient for single task execution. >> > >> ---------------------------------------------------------------------- >> > >> For information about J forums see >> http://www.jsoftware.com/forums.htm >> > >> >> > > >> > > >> > > >> > > -- >> > > Devon McCormick, CFA >> > > ---------------------------------------------------------------------- >> > > For information about J forums see http://www.jsoftware.com/forums.htm >> > ---------------------------------------------------------------------- >> > For information about J forums see http://www.jsoftware.com/forums.htm >> > >> >> >> >> -- >> Devon McCormick, CFA >> ---------------------------------------------------------------------- >> For information about J forums see http://www.jsoftware.com/forums.htm >> > > > > -- > - michael dykman > - [email protected] > > May the Source be with you. > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
