Hi Don, I would just like to relate my experience when I implemented this. Although I had generic communication (socket) code, I did not have a generic algorithm. All process splitting was handled in the master verb/function. Then I tweaked it after the first run works. One of the implementation was in computing raw material requirements for apparel/garments manufacturing.
Instead of breaking the process up in the number of data, I logically broke it up by categorizing the data through its use. For example, if the material is needed by color, it is consumed by the verb mrpColor, its if by size then mrpSize and if its color and size then the verb is mrpColorBySize and I call the master verb as mrpProcess. I know this is not what your really looking for but I do hope that the following inputs would help. 1. No need to clone the J instance. In my implementation, the smaller the new J instance the better. The only big one is the first server call to the Master J Server. Subsequent calls are very specific in verbs and nouns/data. 2. In sending and receiving data, compression is our friend. 3. I exclusively used 3!:1 and 3!:2. I tried memory mapping but I got into data contention problems and does not support boxed arrays. 4. I initially tried building the server in J ... I kept having problems with detecting lost connections between the client and the server. In the end I gave up and made a windows service in C++. The windows service then creates a new instance of J every time a valid connection is received. The windows service also kills the J session when it completes its operation. A side effect of building a windows service was that I did not use J sockets and instead used the library that came with windows which transfers/receives data faster than using the native J sockets library. 5. With my design, I did not send the verb/function/script to the J instances. What I did was all J instances preloads a definition file which defines the public verb name. When this public name is called, it is the only time it loads the actual verb from the file and passes the parameters to it. So I would have something like this: MRPColorBySize=: verb define NB. Code to load the actual file here 3 : '0!:0 ' <BINPATH,'mrpcolorbysize.ijs' NB. Call the actual function after it is loaded mrpColorBySize y ) 6. Returning data is not straight forward. I initially returned the data directly when needed but I was having problems handling race conditions. So I changed the return value to follow this format: <numeric error code>;<data> If the error code is Zero, then there's no problem and the 2nd item should have the actual data. If the error code is non-Zero, the 2nd item should be error information. 7. The communication modules was in a class but the actual verbs were loaded in the 'base' locale. I'll add some more when I think up of some but I have a meeting in a few minutes so I have to stop here. I hope this help. Thanks. r/Alex -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Don Guinn Sent: Tuesday, February 16, 2010 7:17 AM To: Chat forum Subject: Re: [Jchat] Multiple cores Thank you for the comments. So far Raul and Dan have shown interest. After this message I would like to take this out of chat and go to private messages so as not to burden those not interested. Before getting into how to implement, I would like to get a feel as to what we need to build. So here is a list of things I would want to see available and easy to use when using J to develop an application. Please feel free to add and/or modify. Functions: Clone another J instance - In same machine - In another machine Send and receive nouns - via sockets - via mapped files Send scripts to clones Send and receive semaphores Send verbs to be executed and possibly get results back Stop cloned J sessions - By a command to the cloned session - Forcing clone to stop Detect lost clone Detect lost parent Comments: Mapped Files - Advantages: - Reduces memory requirements - Avoids copying - Disadvantages: - Does not support boxed arrays - Mapped files can only be used with shared memory systems - Question: - If a file on a file server shared by several computers is mapped, does this make sense? What would be needed to make it work? Cloning in the same machine should require no additional tools outside those already in J. Some facilities listed may be easier to implement or coming with J7. We should be able to use J7 for development as it is J6 console with a few added foreigns. Should it be implemented as a class, or just add names to the z locale? Objects would be easier to keep track of if multiple clones are created, although arrays of locatives are unwieldy. Whether nouns are passed via mapped files or through sockets should not be a concern of the application. If a command sent to a clone was to return a result, it should not cause the sending system to hang. Receiving the results should be a separate operation. Right now distributing work should be a defined by the application developer. No automatic distribution of work at this time. ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
