Thanks Karl. Thanks, Jitu
On Tue, Sep 30, 2014 at 7:36 PM, Karl Wright <[email protected]> wrote: > Hi Jitu, > > Using transactions to prevent others from changing jobs or connections is > not how we do things in ManifoldCF. Instead, the standard flow is as > follows: > > - the object is loaded, and presented to the user > - the object is edited by the user > - the object is saved back to the database > > Because the editing process may take an extended period of time, database > transactions are ONLY appropriate to atomicize the individual operations of > loading and saving. Therefore there's no provision for wrapping this > entire cycle in a single database transaction. When ManifoldCF needs a > model that prevents objects from being changed by someone else during one > of these sessions, I intended to introduce a system of editing locks, which > expire if not released for a long enough time. > > In your case, if what you are looking for is protection against concurrent > editing as well as rollback, I suggest the following flow: > > -- Obtain a specific write lock or write locks that represents your > combined object. These would be locks that you define and obtain using > ILockManager. > -- Make the modifications you desire WITHOUT a transaction -- always being > sure to load the original object before you modify it. > -- If you get errors, undo what you did by saving the original object(s) > (which you loaded before anything was changed) > -- Release the write lock(s) > > Doing it this way instead of using database transactions will allow you to > use methods like IJobManager.load and IJobManager.save without concern. > You will also need to be careful, of course, as to what order you do things > in. Your only alternative is to go directly to the database tables > themselves and write ALL your own queries, with appropriate cache > management, and this is much harder and seems unnecessary to me. > > Karl > > > > > On Tue, Sep 30, 2014 at 9:43 AM, Jitu <[email protected]> wrote: > >> Hi Karl, >> Thanks for your continued support. That answers my problem >> perfectly. but while doing so i have encountered new problem with caching >> and locking. please help me out. >> >> try >> { >> doStuff(); >> } >> catch (MyException e) >> { >> database.signalRollback(); >> throw e; >> } >> ... >> finally >> { >> database.endTransaction(); >> } >> >> doStuff(){ >> connection = outputManager.load(connectionName); >> if (connection == null) { >> connection = outputManager.create(); >> if (connectionName != null && connectionName.length() > 0) >> connection.setName(connectionName); >> } >> connection.setDescription(connectionName); >> outputManager.save(connection); >> } >> >> >> consider the above case where i have a top level transaction. Inside >> which i am calling doStuff() method. which basically tries to load existing >> connection by name (which acquires read lock by cache manager) if >> connection exist i am updating else i am creating a new connection object >> and saving it (which needs a write lock). While doing this i am getting >> "Illegal lock sequence: Write lock can't be within read lock or non-ex >> write lock" error. Am i doing it wrongly. please advice. Thanks in advance. >> >> Thanks, >> Jitu >> >> >> On Tue, Sep 23, 2014 at 9:47 PM, Karl Wright <[email protected]> wrote: >> >>> Hi Jitu, >>> >>> Editing multiple connections and jobs within a single transaction is >>> possible, with care. The care comes in because of caching and locking. >>> >>> First, the general form of a transaction is as follows: >>> >>> database.beginTransaction(); >>> try >>> { >>> ... do stuff ... >>> } >>> catch (MyException e) >>> { >>> database.signalRollback(); >>> throw e; >>> } >>> ... >>> finally >>> { >>> database.endTransaction(); >>> } >>> >>> It doesn't seem like you are doing it this way from your comments, but >>> that is the only supported way to do it. You may also see cases in some >>> classes where exceptions where transaction deadlock is detected and retries >>> are done. That is also a permitted way to structure a transaction. >>> >>> Second, you can't always wrap arbitrary inner transactions in outer >>> transactions. There are rules. These rules have to do with locking and >>> caching. Specifically, when saving would invalidate a cached object, and >>> in other parts of the transaction you read that very same cached object, >>> you may wind up with a situation where locks are thrown in an unsupportable >>> order. That's when you get exceptions of the kind you mention. >>> >>> If you include a sample transaction that fails for you, I can comment >>> further. >>> >>> Karl >>> >>> >>> On Tue, Sep 23, 2014 at 11:54 AM, Jitu <[email protected]> wrote: >>> >>>> Thanks Karl fir your prompt response as usual. Appreciated! Let me >>>> explain what I am trying to achieve here. >>>> >>>> >>>> >>>> I am trying to build a web client for the manifold framework based on >>>> the theme and design in-line with a product support. I looked into the >>>> sample example Web-UI provided with the Manifold framework and what I >>>> understand from that design is, the commits are reserved per “tab/menu >>>> item” for a given entity. Example: To create a job, I need to first set up >>>> input and output connections and refer that info in create Job action. By >>>> this time, the DB is already populated with the connections details and >>>> every action from the different menu items makes an individual DB commits >>>> in separate calls. And there is no way to roll back once it is half >>>> committed. >>>> >>>> >>>> >>>> In my design, I tried to achieve the save on all dependent tables >>>> including the job tables and (Yes, I have few custom tables to updates as >>>> well) related tables in single transaction. By this my flow looks something >>>> like this: >>>> >>>> >>>> >>>> Through Create Job I like to populate Jobs, Repoconnections, schedules, >>>> etc tables and also two of my custom tables in single transaction. And I >>>> should able to rollback when the transaction fails. >>>> >>>> I tried to achieve this using the IDBInterfaces public void >>>> beginTransaction(int transactionType) and then followed rollback signals, >>>> closeTransaction() etc.. I hit this issue >>>> >>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Illegal lock >>>> sequence: Write lock can't be within read lock or non-ex write lock >>>> >>>> >>>> >>>> Not sure whats going wrong here with my logic and hence I thought let >>>> me get some help understanding the multi-level transaction ability support >>>> OR if you can help on how to wire up all the individual DB actions that is >>>> happening from multiple menu items under “Create New Job” feature from >>>> sample UI in single transaction and ability to rollback from all tables >>>> within single transaction, that will be highly appreciated! >>>> >>>> >>>> I hope I am able to articulate what I am trying to achieve here,,, >>>> Thanks for your help and tim >>>> >>>> On Tue, Sep 23, 2014 at 4:55 PM, Karl Wright <[email protected]> >>>> wrote: >>>> >>>>> Hi Jitu, >>>>> >>>>> To clarify further: you can create nested transactions ONLY where the >>>>> outer-level transaction is as weak or weaker than the inner-level >>>>> transaction. Furthermore, you will almost certainly cause MCF performance >>>>> to suffer badly, as well as needing to handle back-off and retry >>>>> situations. >>>>> >>>>> In general, long transactions are a very bad idea anyhow, because they >>>>> force single-threadedness on the entire application. Connectors, to the >>>>> extent that they use database operations, are required to exit any >>>>> transactions they open before returning to their callers, AND they must >>>>> not >>>>> leave any transactions open when they use any of the I*Activity methods. >>>>> >>>>> There are almost always data-centric ways of structuring things so >>>>> that long-lived transactions are unneeded. For example, you can introduce >>>>> a "context identifier" in your tables that will allow you to keep track of >>>>> what data is associated with what event. Careful data design should be >>>>> sufficient to resolve your issue. I can't say anything further because >>>>> you >>>>> haven't provided any actual details as to what you are trying to do. >>>>> >>>>> Thanks, >>>>> Karl >>>>> >>>>> >>>>> On Tue, Sep 23, 2014 at 6:28 AM, Karl Wright <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Jitu, >>>>>> >>>>>> I'm afraid what you are trying to do will not work with the MCF >>>>>> architecture. You will need to find another way. >>>>>> >>>>>> Karl >>>>>> >>>>>> >>>>>> On Tue, Sep 23, 2014 at 4:52 AM, Jitu <[email protected]> wrote: >>>>>> >>>>>>> Hi, >>>>>>> I have a requirement where on page submit i need to persist both >>>>>>> job and repository connection. To maintain atomicity i added another >>>>>>> level >>>>>>> of transaction at the top level. But i am getting >>>>>>> "org.apache.manifoldcf.core.interfaces.ManifoldCFException: Illegal lock >>>>>>> sequence: Write lock can't be within read lock or non-ex write lock" >>>>>>> error. >>>>>>> when i debugged i noticed different transaction maintained in >>>>>>> IDBInterface. >>>>>>> Please let me know when to use which transaction. i googled but did not >>>>>>> find sufficient information on below transaction levels. >>>>>>> >>>>>>> public static int TRANSACTION_ENCLOSING = 0; >>>>>>> public static int TRANSACTION_READCOMMITTED = 1; >>>>>>> public static int TRANSACTION_SERIALIZED = 2; >>>>>>> public static int TRANSACTION_REPEATABLEREAD = 3; >>>>>>> >>>>>>> Basically first i am trying to persist repository connection and if >>>>>>> i get error while persisting job then it should roll back repository >>>>>>> connection which it persisted already. >>>>>>> >>>>>>> Thanks, >>>>>>> Jitu >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> >
