A transaction has not committed until the journal entry is on disk. This is the "D" in ACID: "durable". So when timing an update transaction, the measurement includes any journal writes.
If a host should fail before the commit phase, there is nothing in the journal. The response is sent after the commit phase, so there is no response either. The transaction did not happen. If a host should fail during the commit phase, there may be a partial entry in the journal. When the forest comes back up, it will examine its journal as part of its recovery process. Any transactions that committed to an in-memory stand, but not to an on-disk stand, will replay into a new in-memory stand. Any transactions that began, but did not commit, will rewind (aka roll back): they never happened. All of this is standard behavior for any system that implements ACID transactions, modulo the specialized implementation detail of in-memory stands. -- Mike On 20 Apr 2012, at 09:58 , Danny Sinang wrote: > Hi Mike, > > From an administrator's standpoint, I just want to know what possible failure > scenarios there would be (for future trouble-shooting). > > I'm told that ML survives outages despit using In-memory-stands, thanks to > the journals. > > But my question was, what happens if an outage occurs while stuff is being > written to the journal. > > Of course, some would say, that doesn't matter. Journal writing is fast > enough. This, to me, would be correct if the implementation was action-based. > > Hope this makes sense. > > In any case, I appreciate your taking time to explain things. > > Regards, > Danny > > > On Fri, Apr 20, 2012 at 12:52 PM, Michael Blakeley <[email protected]> wrote: > I am not sure that the distinction you are making means anything in the > context of MarkLogic's implementation. As Ron said, the journal has no > user-servicable parts. The journal could be delta-based or action-based, or > some hybrid. As a developer or administrator, you would never know. > MarkLogic's implementation could change at any time, without any warning, and > you would never know. It is a black box. > > So if you want to evaluate the journal implementation, ask questions that are > appropriate to a black box. The question I would ask is: "are transactions > fast enough for my application?" If they are, then you have ACID transactions > with the performance you need. In that case, what does it matter how they are > implemented? > > -- Mike > > On 20 Apr 2012, at 09:44 , Danny Sinang wrote: > > > Hi Michael, > > > > Thanks. I was hoping that the ML journals would contain only the > > transaction (i.e. action) logs. That way it, i feel it would it fast. > > > > However, Ron mentioned that the journal contains the delta data already. > > So, to me, deriving the delta data would take a bit longer than just > > recording the actions. > > > > Just my thoughts. > > > > Regards, > > Danny > > > > > > On Fri, Apr 20, 2012 at 12:40 PM, Michael Blakeley <[email protected]> > > wrote: > > Danny, http://en.wikipedia.org/wiki/Transaction_log might be useful > > background reading. A similar technique is used by every ACID-compliant > > database that I am familiar with. > > > > -- Mike > > > > On 20 Apr 2012, at 06:37 , Ron Hitchens wrote: > > > > > > > > The journal certainly does contain the data. It > > > records the deltas to the database. That means that > > > when an update request runs, all the changes it made > > > to the state of the database are recorded in the journal. > > > Those changes can be applied again from the journal, > > > without running the request again. > > > > > > Updates can be additions, replacements or deletions > > > of documents (fragments actually, which are not always > > > 1-1 with documents). > > > > > > It's the journal that protects you from outages. > > > Writing journal frames is as efficient as it possibly > > > can be. There is some overhead to write to the journal, > > > but that's the cost of insuring that when a transaction > > > commit completes, it is guaranteed to survive a system > > > crash thereafter. > > > > > > Journal creation efficiency is not something you need > > > to be concerned about. It's rarely significant as overhead > > > in most cases. > > > > > > For those cases where you might be making large updates > > > on a regular basis, it's possible to tell MarkLogic to > > > pre-allocate journal space so that it's even more efficient. > > > > > > On Apr 20, 2012, at 2:23 PM, Danny Sinang wrote: > > > > > >> Hi Ron, > > >> > > >> Thanks for the info. > > >> > > >> So in effect, the journal entries somehow store the actions to be taken, > > >> not the new data to be applied ? > > >> > > >> I'm asking this to somehow assure myself that journal entry creation > > >> would be fast, even for large transactions. The faster the journal > > >> creation, the less chances of it failing due to an outage. > > >> > > >> Regards, > > >> Danny > > >> > > >> On Fri, Apr 20, 2012 at 8:58 AM, Ron Hitchens <[email protected]> wrote: > > >> > > >> No user serviceable parts inside. > > >> > > >> The journal contains information that describes > > >> sets of updates done by transactions. In case of a > > >> crash, the journal can be replayed to re-apply any > > >> updates that may not yet have been written to the > > >> on-disk forests. > > >> > > >> A transaction does not complete until the journal > > >> entry is written to disk, but the corresponding updates > > >> to the forest(s) are written later. The database view > > >> your queries see are the sum of the on-disk in in-memory > > >> stands that comprise forests which make up a database. > > >> > > >> Journal entries are in a compact binary format that > > >> is only used internally by the server to assure data > > >> consistency. > > >> > > >> On Apr 20, 2012, at 1:43 PM, Danny Sinang wrote: > > >> > > >>> Hi, > > >>> > > >>> I'm told that for document inserts and updates, an ML journal entry > > >>> gets written. > > >>> > > >>> Does anyone know what gets written to the journal ? Is it just the > > >>> xquery getting executed ? Or is it the new data to be appended to the > > >>> database ? > > >>> > > >>> Regards, > > >>> Danny > > >>> > > >>> _______________________________________________ > > >>> General mailing list > > >>> [email protected] > > >>> http://developer.marklogic.com/mailman/listinfo/general > > >> > > >> --- > > >> Ron Hitchens {mailto:[email protected]} Ronsoft Technologies > > >> +44 7879 358 212 (voice) http://www.ronsoft.com > > >> +1 707 924 3878 (fax) Bit Twiddling At Its Finest > > >> "No amount of belief establishes any fact." -Unknown > > >> > > >> > > >> > > >> > > >> _______________________________________________ > > >> General mailing list > > >> [email protected] > > >> http://developer.marklogic.com/mailman/listinfo/general > > >> > > >> _______________________________________________ > > >> General mailing list > > >> [email protected] > > >> http://developer.marklogic.com/mailman/listinfo/general > > > > > > --- > > > Ron Hitchens {mailto:[email protected]} Ronsoft Technologies > > > +44 7879 358 212 (voice) http://www.ronsoft.com > > > +1 707 924 3878 (fax) Bit Twiddling At Its Finest > > > "No amount of belief establishes any fact." -Unknown > > > > > > > > > > > > > > > _______________________________________________ > > > General mailing list > > > [email protected] > > > http://developer.marklogic.com/mailman/listinfo/general > > > > > > > _______________________________________________ > > General mailing list > > [email protected] > > http://developer.marklogic.com/mailman/listinfo/general > > > > _______________________________________________ > > General mailing list > > [email protected] > > http://developer.marklogic.com/mailman/listinfo/general > > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general > > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
