On 7/28/06, nlif <[EMAIL PROTECTED]> wrote:
Thanks, James. Sending just the URI is not an option in my case, since the participating applications do not share a file server (or a database, for that matter). As for the second suggestion, just to make sure I understand: do you mean that I should implement the equivalent of the ActiveMQ streams feature, by splliting the files into JMS ByteMessages and then include them in the MessageGroup and in the transaction? (btw I've looked at the Streams code, and I know this is pretty much what it does, only that the input stream auto-acknowledges each chunk, right?).
I mean split your transaction from sending many Mbs of data in 1 transaction to splitting the file into more manageable chunks and having lots of littler transactions. The other option is just to buy a big box with heaps of RAM :)
Now, could there be a different approach to all of this? My whole problem stems from the fact I don't want to acknowledge ANY of the items (messages and/or files) until I have ALL of them. The reason is simple: if my cosumer crashes half-way, some data is lost. However, trying to sync the acks for all different items, that may be passed via different mechanisms, makes things very complicated for me. Is there another way? Is it possible to go back to simple MessageListeners and InputStream consumers, which acknowledge every item separately, but have a different mechanism for ensuring their state is recoverable? Maybe I should go for a consumer that can persist its data? Or is this even harder to achieve?
Why can't you acknowledge in batches; say a few 100K at a time?
This seems to me like something many applications might require. Am I really the only one faced with a scenario in which a transmission is composed of several items, that cannot be sent as a single JMS message? Is there no generic solution for this?
For most of the customers I work with, their transaction boundaries are reasonably small with 10-100K of data per transaction. If you need to process something larger you can often split it up into smaller chunks. -- James ------- http://radio.weblogs.com/0112098/
