Luca, have the issues raised in this thread been resolved in the v2 drop? If partial, in what way?
Thanks! Alan On Monday, July 23, 2012 7:58:45 AM UTC-4, Lvc@ wrote: > > Hi, > not yet. This is something we will think for 2.0. > > Lvc@ > > On 23 July 2012 13:27, Danny Thomas <[email protected] <javascript:>> > wrote: > > Luca, are distributed transactions also on the road map in that kind of > > timescale? > > > > Danny > > > > > > On Monday, 23 July 2012 11:01:02 UTC+1, Lvc@ wrote: > >> > >> Hi Mic, > >> you're right: transactions against the "remote" protocol have some > >> limitations we'd like to fix in 1.2. Probably not al of them but most. > >> > >> Lvc@ > >> > >> On 22 July 2012 02:41, Mich Geni <[email protected] <javascript:>> > wrote: > >>> > >>> Hi Luca, > >>> > >>> What are the plans in the roadmap for implementing support for real > ACID > >>> local, remote and distributed transactions, coherently? > >>> > >>> Reference Links: > >>> > >>> http://code.google.com/p/orient/issues/detail?id=442 > >>> http://code.google.com/p/orient/issues/detail?id=931 > >>> http://code.google.com/p/orient/issues/detail?id=784 > >>> http://code.google.com/p/orient/issues/detail?id=90 > >>> http://code.google.com/p/orient/issues/detail?id=12 > >>> http://code.google.com/p/orient/issues/detail?id=890 > >>> http://code.google.com/p/orient/issues/detail?id=484 > >>> http://code.google.com/p/orient/issues/detail?id=713 > >>> http://code.google.com/p/orient/issues/detail?id=181 > >>> > https://groups.google.com/d/msg/orient-database/7zvllV_ItYo/WM1rg9cyozIJ > >>> https://groups.google.com/forum/#!topic/orient-database/sqo84YE6UV8 > >>> ...and other posts by users on OrientDB google group > >>> > >>> > >>> To have a real ACID Transactions compliant database, we need serious > >>> consideration of the issues involved. > >>> > >>> I think, we need a complete new mindset, and a world-view to create a > >>> base for supporting transactions. > >>> > >>> SideNote: Transactions are a necessity. We cannot go to classical era > of > >>> RDBMSes for transactions support. All new applications surround the > NoSQL > >>> solutions. There are so many use-cases in modern applications, where > >>> consistency is a must. So who is the first to fully support them? will > be a > >>> clear winner. > >>> > >>> > >>> Regards, > >>> > >>> Mic > >>> > >>> > >>> On Monday, 2 July 2012 22:53:22 UTC+5, Lvc@ wrote: > >>>> > >>>> On 1 July 2012 11:43, Mich Geni <[email protected] <javascript:>> > wrote: > >>>>> > >>>>> Hi Luca, > >>>>> > >>>>> I am glad to see caring and thoughtful people here. > >>>>> Yes, a combination of client and server involvement in the > transaction > >>>>> will be much beneficial. > >>>>> > >>>>> e.g. to elaborate it more: > >>>>> > >>>>> When a client begins a transaction, it notifies the server. The > server > >>>>> creates a transaction slot/session for the client. Which is also an > >>>>> indication (to server) from client, that write the changes only when > I say > >>>>> commit. And behave accordingly. > >>>> > >>>> +1 > >>>>> > >>>>> The transaction can have a timeout (on server), in case the client > >>>>> begins a transaction and never commits it, or disconnects in the > middle. In > >>>>> that case the slot/session will be destroyed, and all changes > discarded. > >>>> > >>>> +1 > >>>>> > >>>>> During the transaction: the changes in data client make, should > remain > >>>>> at client side, to prevent unnecessary data transfer. But when > client needs > >>>>> something from server, the server cooperates in a manner (and send > it to the > >>>>> client such) that the client thinks it as actual information. We can > call it > >>>>> transaction contextual information. In this way, the client will > have > >>>>> complete (but fictional and also sometimes real) information > necessary for > >>>>> the transaction. So server's transaction awareness is crucial. > >>>> > >>>> +1 > >>>>> > >>>>> We can have a some special format/notation/range for RIDs during > >>>>> transactions. This will be a flag for client, server and > developer/admin, > >>>>> that this information is contextual. But these RIDs should not be > isolated > >>>>> only at client side, such that the server has no information about > them. But > >>>>> I think such contextual RIDs should be provided by server on > client's > >>>>> request, and booked for it until free'd by client. So that client > can use it > >>>>> for its transaction, and server can map/translate them later after > commit to > >>>>> actual/real RIDs. > >>>> > >>>> This is already implemented with transient RIDs having negative > numbers, > >>>> example: #3:-4 > >>>>> > >>>>> The client will create a changeset (a small graph, a set of > documents > >>>>> etc), containing some information to be made real/actual or decided > on > >>>>> server only (or in other words leave some patches in the changeset > for the > >>>>> server to cover up). And send this information to server for commit. > The > >>>>> server will validate those changes (check if valid and possible) and > write > >>>>> the changes as permanent. > >>>> > >>>> This is already in this way. > >>>>> > >>>>> People using the transactions API, should be educated to do only > atomic > >>>>> operations in transactions. (an atom is very very small, by the > way). So > >>>>> bulk (insertions/updation/deletion) operations should not be done in > >>>>> transactions. As transactions are meant to make small changes and to > ensure > >>>>> data integrity. In this way, the data exchanged between client and > server > >>>>> will be minimum for a transaction. > >>>> > >>>> This is partially true. Now transactions are used to speed up > "remote" > >>>> operations by grouping them in transactions of X operations. This is > used > >>>> setting the tx.useTxLog(false). > >>>>> > >>>>> In case: a record X (set of records, or portion of graph) was read > by a > >>>>> client A at time t1. Then at time t2 (i.e. t2 > t1), client B reads > and make > >>>>> changes in it/them, in a transaction. Then at time t3 (i.e. t3 > > t2) , when > >>>>> client A wants to make changes in it/them, the server should give an > error, > >>>>> that your version of information needs to read again before making > any > >>>>> changes, because it was recently changed. As the server knows that, > client B > >>>>> has initiated a transaction and not client A, so it will give > priority to > >>>>> client B, and notify A. And yes, this does include some conflicting > >>>>> situations, when both client A and client B begin transaction one > after > >>>>> another, and try to make changes to same set of data. In that case > the > >>>>> change-request by the later client can be denied, to read the new > data, and > >>>>> try again. > >>>> > >>>> This is already supported by the MVCC system but happens at commit() > >>>> time. We planned to support pessimistic approach as well but it's not > >>>> urgent, at least today, as priority. > >>>>> > >>>>> I was thinking on this solution, whether there is a possibility of > >>>>> deadlocks or dog-trying-to-catch-its-tail kind of problems. But am > unclear > >>>>> on this yet. > >>>>> Managing transactions in distributed version of OrientDB, will also > >>>>> bring some new questions. But again, we are already sitting in a > world, > >>>>> where many of such problems have been thoroughly studied and solved. > >>>> > >>>> Now distributed TX are not supported, let see in the future. > >>>> > >>>> Lvc@ > >>>> > >>>>> > >>>>> > >>>>> Regards > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> On Sunday, July 1, 2012 4:33:31 AM UTC+5, Lvc@ wrote: > >>>>>> > >>>>>> Hi Mich, > >>>>>> the choice to leave everything on the client side was to maximize > >>>>>> concurrency on the server side. Unfortunately this approach has the > >>>>>> limitations you exposed. > >>>>>> > >>>>>> So today transactions works fine using the embedded/local mode but > >>>>>> against a remote servers lack of these problems. > >>>>>> > >>>>>> To solve them this is the solution we thought: leave all as is, but > >>>>>> filter the result set on the client side reading the TX changes. > This would > >>>>>> leave all the benefits of current approach in scalability, but this > means > >>>>>> also that commands like SQL UPDATE and DELETE should be divided in > these > >>>>>> steps: > >>>>>> > >>>>>> client sends the command to the server > >>>>>> the server get the target records to change and send to the client > the > >>>>>> RIDs > >>>>>> the client filter them and sends to the servers the changes (mostly > >>>>>> none or few) > >>>>>> the server do the really SQL statement against filtered records > >>>>>> > >>>>>> In case you update 1,000 records this means that 1,000 RIDs will be > >>>>>> sent to the client side. One serialized RID is 10bytes, so 10k will > be > >>>>>> transferred to the client. > >>>>>> > >>>>>> For the SQL Select the overhead is usually minimum because the > target > >>>>>> is already transferred to the client and just filtered on it. > >>>>>> > >>>>>> Lvc@ > >>>>>> > >>>>>> > >>>>>> On 30 June 2012 01:55, Mich Geni <[email protected]> wrote: > >>>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> I want to illuminate a problem people have faced and discussed in > >>>>>>> this forum before. Which is transactions. Most people when first > start to > >>>>>>> use OrientDB think that they can do pretty stuff in a transaction > (means by > >>>>>>> which they think to be done in their transaction). But soon they > find that > >>>>>>> since everything remains in memory on client side until the > transaction is > >>>>>>> committed (especially temporary RIDs stuff) and not on > server-side, so > >>>>>>> immediately they start to break their 'transaction' into more > pieces (in > >>>>>>> OrientDB sense of implementing transactions) in order to make it > work. > >>>>>>> > >>>>>>> But one thinks that what is the need of transactions then? Orient > is > >>>>>>> a Document-Graph DB with ACID Tx. So one expects that it will be > >>>>>>> transactional for graphs. But when you push the transaction thing > on the > >>>>>>> client side, it really becomes a nightmare. Consider an example > below: > >>>>>>> > >>>>>>> (NOTE: I have tried to give a more complete code example. OrientDB > >>>>>>> version: 1.0.1 (but to date svn trunk 5965 still is relevant to > consider). > >>>>>>> In order to be simple, I have skipped many checks and efficiency > jugglery) > >>>>>>> > >>>>>>> We are taking an example of a financial application which works in > >>>>>>> today's modern days of social networks and graphs. A user sends an > amount of > >>>>>>> $300 from his account to his friend account. Here is how we will > do it > >>>>>>> OrientDB using Graph API: > >>>>>>> > >>>>>>> Steps: > >>>>>>> > >>>>>>> Create domain classes (vertices and edges) > >>>>>>> Insert initial records > >>>>>>> Write and Run our code to perform the transaction (money transfer) > >>>>>>> See results > >>>>>>> > >>>>>>> Step 1: > >>>>>>> > >>>>>>> orientdb> Create Class Account Extends V > >>>>>>> > >>>>>>> Class created successfully with id=10 > >>>>>>> > >>>>>>> orientdb> Create Class BalanceUpdated Extends E > >>>>>>> > >>>>>>> Class created successfully with id=11 > >>>>>>> > >>>>>>> orientdb> Create Class SendAmount Extends E > >>>>>>> > >>>>>>> Class created successfully with id=12 > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> Step 2: > >>>>>>> > >>>>>>> orientdb> insert into Account (accountNumber, currentBalance) > values > >>>>>>> ("516544646", "1000") > >>>>>>> > >>>>>>> Inserted record > >>>>>>> 'Account#8:0{accountNumber:516544646,currentBalance:1000} v0' in > 0.005000 > >>>>>>> sec(s). > >>>>>>> > >>>>>>> orientdb> insert into Account (accountNumber, currentBalance) > values > >>>>>>> ("5165485491", "100") > >>>>>>> > >>>>>>> Inserted record > >>>>>>> 'Account#8:1{accountNumber:5165485491,currentBalance:100} v0' in > 0.011000 > >>>>>>> sec(s). > >>>>>>> > >>>>>>> > >>>>>>> Step 3: > >>>>>>> > >>>>>>> // Graph Transaction test > >>>>>>> OGraphDatabase graphDB = > >>>>>>> > OGraphDatabasePool.global().acquire("remote:localhost:2424/graphtest1", > >>>>>>> "admin", "admin"); > >>>>>>> Date currDate = new Date(); > >>>>>>> int amountToTransfer = 300; // in some currency > >>>>>>> String senderAccountNum = "516544646"; > >>>>>>> String receiverAccountNum = "5165485491"; > >>>>>>> > >>>>>>> try { > >>>>>>> // graphDB.begin(TXTYPE.OPTIMISTIC); // > >>>>>>> <-------------------- > >>>>>>> > >>>>>>> List<ODocument> listAcc = graphDB.query(new > >>>>>>> OSQLSynchQuery<Knows>("select from Account where accountNumber = > '" + > >>>>>>> senderAccountNum + "'")); > >>>>>>> > >>>>>>> ODocument senderAccount = listAcc.get(0); > >>>>>>> > >>>>>>> int senderCurrentBalance = > >>>>>>> > Integer.parseInt(senderAccount.field("currentBalance").toString()); > >>>>>>> > >>>>>>> if (senderCurrentBalance - amountToTransfer <= 0) > { > >>>>>>> //throw BalanceBelowTransferAmount; > >>>>>>> } > >>>>>>> > >>>>>>> // useful in generating statements > >>>>>>> ODocument senderAccountHistory = > >>>>>>> graphDB.createVertex("Account").fields( > >>>>>>> "accountNumber", senderAccountNum, > >>>>>>> "currentBalance", senderCurrentBalance, > >>>>>>> "date", currDate > >>>>>>> // and other necessary info > >>>>>>> ).save(); > >>>>>>> > >>>>>>> // Deduct the amount to be transferred from sender > >>>>>>> account > >>>>>>> ODocument senderAccountUpdated = > >>>>>>> senderAccount.fields( > >>>>>>> "currentBalance", senderCurrentBalance - > >>>>>>> amountToTransfer, > >>>>>>> "date", currDate > >>>>>>> // and other necessary info > >>>>>>> ).save(); > >>>>>>> > >>>>>>> // link sender account latest to previous balance > >>>>>>> ODocument senderBalanceHistory = > >>>>>>> graphDB.createEdge(senderAccountUpdated, senderAccountHistory, > >>>>>>> "BalanceUpdated").field("date", currDate).save(); > >>>>>>> > >>>>>>> > >>>>>>> List<ODocument> listAcc1 = graphDB.query(new > >>>>>>> OSQLSynchQuery<Knows>("select from Account where accountNumber = > '" + > >>>>>>> receiverAccountNum + "'")); > >>>>>>> > >>>>>>> ODocument receiverAccount = listAcc1.get(0); > >>>>>>> int receiverCurrentBalance = > >>>>>>> > Integer.parseInt(receiverAccount.field("currentBalance").toString()); > >>>>>>> > >>>>>>> // useful in generating statements > >>>>>>> ODocument receiverAccountHistory = > >>>>>>> graphDB.createVertex("Account").fields( > >>>>>>> "accountNumber", receiverAccountNum, > >>>>>>> "currentBalance", receiverCurrentBalance, > >>>>>>> "date", currDate > >>>>>>> // and other necessary info > >>>>>>> ).save(); > >>>>>>> > >>>>>>> // Deduct the amount to be transferred from sender > >>>>>>> account > >>>>>>> ODocument receiverAccountUpdated = > >>>>>>> receiverAccount.fields( > >>>>>>> "currentBalance", receiverCurrentBalance + > >>>>>>> amountToTransfer, > >>>>>>> "date", currDate > >>>>>>> // and other necessary info > >>>>>>> ).save(); > >>>>>>> > >>>>>>> // link receiver account latest to previous > balance > >>>>>>> ODocument receiverBalanceHistory = > >>>>>>> graphDB.createEdge(receiverAccountUpdated, receiverAccountHistory, > >>>>>>> "BalanceUpdated").field( > >>>>>>> "date", currDate).save(); > >>>>>>> > >>>>>>> // link sender and receiver accounts > >>>>>>> ODocument sendAmount = > >>>>>>> graphDB.createEdge(senderAccountHistory, receiverAccountHistory, > >>>>>>> "SendAmount").fields( > >>>>>>> "amount", amountToTransfer, > >>>>>>> "date", currDate > >>>>>>> ).save(); > >>>>>>> > >>>>>>> // graphDB.commit(); // <-------------------- > >>>>>>> > >>>>>>> } finally { > >>>>>>> graphDB.close(); > >>>>>>> } > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> Step 4: > >>>>>>> > >>>>>>> orientdb> select from Account > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > ---+---------+--------------------+--------------------+--------------------+-------------------- > > > >>>>>>> #| RID |accountNumber |currentBalance |date > >>>>>>> |out > >>>>>>> > >>>>>>> > ---+---------+--------------------+--------------------+--------------------+-------------------- > > > >>>>>>> 0| #8:0|516544646 |700 |2012-06-30 > >>>>>>> 03:00:52 |[1] > >>>>>>> 1| #8:1|5165485491 |400 |2012-06-30 > >>>>>>> 03:00:52 |[1] > >>>>>>> 2| #8:2|516544646 |1000 |2012-06-30 > >>>>>>> 03:00:52 |[1] |[1] > >>>>>>> 3| #8:3|5165485491 |100 |2012-06-30 > >>>>>>> 03:00:52 |null |[2] > >>>>>>> > >>>>>>> > ---+---------+--------------------+--------------------+--------------------+--------------------+-------------------- > > > >>>>>>> > >>>>>>> > >>>>>>> 4 item(s) found. Query executed in 0.206 sec(s). > >>>>>>> > >>>>>>> > >>>>>>> orientdb> select from BalanceUpdated > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > ---+---------+--------------------+--------------------+-------------------- > > >>>>>>> #| RID |out |in |date > >>>>>>> > >>>>>>> > ---+---------+--------------------+--------------------+-------------------- > > >>>>>>> 0| #13:0|#8:0 |#8:2 |2012-06-30 > >>>>>>> 03:00:52 > >>>>>>> 1| #13:1|#8:1 |#8:3 |2012-06-30 > >>>>>>> 03:00:52 > >>>>>>> > >>>>>>> > ---+---------+--------------------+--------------------+-------------------- > > >>>>>>> > >>>>>>> > >>>>>>> 2 item(s) found. Query executed in 0.095 sec(s). > >>>>>>> > >>>>>>> > >>>>>>> orientdb> select from SendAmount > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > ---+---------+--------------------+--------------------+--------------------+-------------------- > > > >>>>>>> #| RID |out |in |amount > >>>>>>> |date > >>>>>>> > >>>>>>> > ---+---------+--------------------+--------------------+--------------------+-------------------- > > > >>>>>>> 0| #14:0|#8:2 |#8:3 |300 > >>>>>>> |2012-06-30 03:00:52 > >>>>>>> > >>>>>>> > ---+---------+--------------------+--------------------+--------------------+-------------------- > > > >>>>>>> > >>>>>>> > >>>>>>> 1 item(s) found. Query executed in 0.078 sec(s). > >>>>>>> > >>>>>>> > >>>>>>> orientdb> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> As depicted in Step 3, we are doing a single financial transaction > on > >>>>>>> graph, inside the try/finally block. If we face any runtime error > in the > >>>>>>> middle, our transaction is not done (and needs to be rolled back). > But as > >>>>>>> you can see the transaction begin() and commit() statements are > commented > >>>>>>> out. Because OrientDB's current implementation doesn't allow such > >>>>>>> transaction. Every thing is client side until committed, and sent > to server. > >>>>>>> And as you can see, we are doing many things in this transaction, > which are > >>>>>>> decided/found on server side. So we are left with doing no > transaction and > >>>>>>> doing each step as non atomic. Not to mention the helper code > required in > >>>>>>> case of errors if happen in the middle, to undo the previous > changes to the > >>>>>>> DB before error. And the most concerning thing is, this will be > done > >>>>>>> thousands of times by thousands of developers to cover these few > loop holes > >>>>>>> in OrientDB. Isn't it more easy to cover the loop holes by > themselves. > >>>>>>> > >>>>>>> Therefore, it will be great to see real transactions (be it server > >>>>>>> side or in any other way, but correct and non-surprising) > implementation in > >>>>>>> OrientDB. And as OrientDB is undergoing heavy > evolution/development, we can > >>>>>>> expect such problems to go away in near future. > >>>>>>> > >>>>>>> > >>>>>>> Regards > >>>>>> > >>>>>> > >>>> > >>> -- > >>> > >>> > >>> > >> > >> > > -- > > > > > > > -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
