In a system I'm working with, we are considering using JMS for grid processing, mainly for asynchronous jobs. I'm wondering how far we can push the envelope and let JMS be the main bus behind everything, even the more synchronous jobs where time to respond is critical. So, before I go much further I would like a sanity check.
The overall app is very basic. Clients talk to Managers, which submit jobs to request queues, handing a requestID back to the clients. Workers consume these requests, do relevant DB queries and create binary results. The result blobs are dumped to a database to be persisted for a certain period of time. A short message dumped on a result queue notifies the manager process of availability. Clients poll managers periodically until the process is complete. Now... for the synchronous implementation, we'd want to skip the database entirely. Managers would hold the client port open while submitting items to the request queue. In their response messages, workers would include the binary results, skipping the database entirely. So - can we use JMS Streams to ensure clients begin to receive data as soon as it becomes available? eg, - Worker queries database, receives results and transforms them to XML - Worker creates a JMS message containing the raw content, streaming 100MB message to the response queue - Message appears on response queue even before the worker has finished streaming the message - In turn, Manager begins to consume response and pass the stream along to the client, well before the worker has finished streaming to the broker or the broker to the manager Just in case I'm not being clear, I don't want any step to have to "finish" before the client can start receiving the results. As the manager will be "co-located" with the response queue via an embedded broker, there are no more network hops than I would anticipate in a direct of Client->Manager->Worker->Database. Thanks for any responses. Feel free to be critical. -- View this message in context: http://www.nabble.com/ActiveMQ-Stream-Performance--tp25196653p25196653.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
