I personally like options 2 and 3. The message just contains a URL to where the whole stream can be downloaded. The nice part about this is that then you have many option for where your store the message. You make the URL a reference to the producer. And a consumer would just connect back to the producer to get the stream. The producer could first upload the message to the broker, and then pass a URL to the broker, and the consumer would connect back to the broker for the stream. You could upload the file to and HTTP server or even something like S3 and have the consumer connect to that for the stream. In all these cases the only thing that varies is where the producer uploads the data to. The message and the consumer behave the same way in all the cases.
I think that the simplest case to implement which will provide the most robust operation is to have the producer upload the stream to the broker and pass a reference to the broker's blob and have clients connect back to it. On 2/22/07, Aleksi Kallio <[EMAIL PROTECTED]> wrote:
I'll move this discussion over to dev@activemq.apache.org as it belongs there... Below I go on sketching how FileMessage might work. As you can see it is still quite sketchy in some parts so all comments and good ideas are welcome. Btw. is FileMessage a good name? In some cases we are not transferring files, but data in memory. Basically we are transferring a finite byte sequence. Is ByteSequenceMessage too awkward? I think it is maybe... > There are a few different possible implementations... > > (i) FileMessage ends up being a wrapper on top of the existing JMS streams This would be the preferred method in our case. Current streaming API uses a designated destination for the stream. If that destination is used for other purposes (other streams or messages) there has to be a way of separating current stream from the rest of the traffic. Streams support message selectors, which looks like the best solution. The problem is that if there are non-FileMessages sent to that destination also and listeners for them, they have to also use selectors to weed out FileMessage streams. It would be great if FileMessage would behave just like other types of messages. In the streaming case I guess it is not possible to achieve? > (ii) FileMessage uses some out of band transfer mechanism. > (iii) direct connection. This option is similar to (ii) but rather > than putting the file on some remote file server, the file stays on > the producer until the consumer has received it; > > The nice thing is I think all of these approaches can be handled > nicely by the single FileMessage API; then it can be a > configuration/policy issue as to exactly which implementation is used. If we look at the three implementations and what they would require from the two endpoints: 1. producer must inform what selector is to be used to receive data through JMS stream, consumers must acknowledge when they are ready to receive data 2. producer must place file available to external server and place URL to the message 3. producer must open a port and place URL to the message .. we see that streaming is maybe the trickiest one. If producer just starts streaming there is no guarantee (in the general case) that consumer will receive the whole stream from the beginning. Is that correct? In case 2, there is the question about pruning the files. Are they left to external server for ever? Or should consumer be capable of confirming the transfer, and after confirmation producer would remove the file? I don't think we should assume the consumer has a write access to file server. Case 3 is actually pretty straightforward. Do we want allow also a push option where consumer opens the port and producer delivers the file? > In terms of getting started, the simplest route is probably to add the > API in first (to start with assuming just a URL to the file which is a > no brainer) then we should be able to start adding different providers > to suit. Yes, I think that's the best way to go forward. I'll write something based on that JIRA issue and send it to dev@activemq.apache.org for comments. Does that sound good?
-- Regards, Hiram Blog: http://hiramchirino.com
