Thomas Tempe wrote: >On Mon, 24 Jun 2002 17:11:55 +0200 >[EMAIL PROTECTED] wrote: > > > >>Harddisk queueing seems necessary, because it is very important that every >>message could be restored from disk if the server crashes. >> >> > >If I may ask a question, I don't really see the point of hard disk >storage for crash resilience. > Thomas, thanks for this discussion. After fixing some bugs this will be the main issue i want to investigate and code into xmlBlaster - a much higher guarantee of delivery. I'm not talking about transaction with two phase commit but the step before this. Transactions with two phase commit will be addressed in later future (version 0.9->1.0).
> >I do agree that it sounds reassuring, maybe more that memory. However, >computer hardware and OSs include a lot of layers of cache. If a >hardware crash occurs when the message has been acknowledged by the >XmlBlaster server, but still resides eg in an OS buffer or the hard >disk's write cache, it is lost anyway. If you cannot garantee that >the message has been written to disk, then your mechanism is not >reliable. And as far as I know, the Linux kernel at least does not >provides any mechanism to offer such garanties. > The current implementation of the InvocationRecorder has a configurable SYNC mode which does a harddisk sync after every message written or read from HD. It slows down the message write and access roundtrip from ~1000msg/sec to ~100msg/sec on my Linux 600 MHz with an average performing SCSI harddisk. (Note: Doing this in RAM is ~10000 msg/sec) Interesting is that with or without SYNC i never could force any data loss by killing (kill -9)/crashing xmlBlaster. So for this case (xmlBlaster crash), the SYNC mode seems not to be necessary, but HD storage of queues helped in 100% of my tests to recover from application crash or shudown (note that the InvocationRecorder is currently only implemented to be used on client side). But - what i believe (but not have tested yet) is that the SYNC mode would allow save recovery on any OS crash imaginable. So even if Linux crashes because you pull out the CPU i think the SYNC'd data is not lost. Having a RAID level 5 or similar would allow harddisk crash as well. > >Moreover, XmlBlaster messages look exactly like the kind of data that >will never go beyond a kernel buffer : relatively short messages which >are written once, read a few times shortly thereafter, and then >discarded. > >If you really want garanteed delivery, your best bet would probably be to >have a transaction mechanism between the sending and the receiving >application. This mechanism would make sure that an overall operation >is completely done. Having hard disk storage for crash resilience >on OTC hardware/OS would be much like having two locks on a door : it >does not provide garantees, it only feels safer than one lock. > Yes i think you are right here. Currently the sending/receiving application could implement an ACK/NAK mode to have guaranteed delivery. And in future the xmlBlaster 2 phase commit transaction will support the ease of developing such scenarios. regards Marcel > >Is this correct, or am I missing the point? > >Have fun, >-- >Thomas Tempe >jabber ID: [EMAIL PROTECTED] > >No matter how hard the other apes try, >the real winner of the soccer world cup >is always Ronald McDonald. > >
