Dima, My proposal would require two changes: 1) Public - for sure we will continue support old API 2) Internal TX engine - these changes are inevitable for transactional SQL support.
So we have no freedom to choose whether to spend time on internals or not. We will have to do that anyway. On Tue, Sep 26, 2017 at 9:02 AM, Dmitriy Setrakyan <dsetrak...@apache.org> wrote: > Vladimir, > > I do not think we have a luxury of changing Ignite transaction APIs. It is > almost the same as changing Ignite put and get APIs. Too much code has > already been written to these APIs. The upgrade path will be so onerous > that no one will ever take it. > > As far as the current transaction flags, we will have to continue to > support them. Ideally we should extend them to Ignite SQL as well. If there > is a new transaction flag that you wish to add to Ignite SQL, please > suggest it here. > > D. > > On Mon, Sep 25, 2017 at 10:48 PM, Vladimir Ozerov <voze...@gridgain.com> > wrote: > > > Dima, > > > > Becuase if you take any pair of our concurrency/isolation modes, you will > > not find anything similar in SQL. The main problem is that there is no > > "OPTIMISTIC" or "PESSIMISTIC" in SQL world. All reads are "optimistic", > but > > can be converted to "perssimistic" optionally, and all writes are > > "pessimistic". Another problem is that some of our pairs cannot be used > in > > practice (OPTIMISTIC/READ_COMMITTED, OPTIMISTIC/REPEATABLE_READ), and > some > > duplicates each other (PESSIMISTIC/REPEATABLE_READ - > > PESSIMISTIC/SERIALIZABLE). This make our API very hard to explain and > > reason about. If you cannot explain API in 5 mins, it is broken :-) > > > > If we were many years in the past allowed to choose between Ignite and > SQL > > approaches I would prefer SQL because it is much simpler. > > > > On Tue, Sep 26, 2017 at 1:29 AM, Dmitriy Setrakyan < > dsetrak...@apache.org> > > wrote: > > > > > Vladimir, > > > > > > This looks like a complete overhaul of our transactional behavior and I > > do > > > not think we are at a liberty to make such drastic changes. Can you > > please > > > explain why the current behavior would not map to the SQL transactions? > > > > > > D. > > > > > > On Mon, Sep 25, 2017 at 12:09 PM, Vladimir Ozerov < > voze...@gridgain.com> > > > wrote: > > > > > > > Folks, > > > > > > > > Sorry for late reply. I had a chat with several Ignite veterans > today. > > We > > > > tried to design transactional SQL for Ignite. One of our questions > was > > > how > > > > to align SQL transactions with current Ignite transactions. And we > > > failed. > > > > And then we came to conclusion that current transaction API is > > unusable. > > > We > > > > have 6 pairs of modes from API standpoint and 4 real modes. This is > > very > > > > counterintuitive and cannot be mapped to any transactional framework > > our > > > > users are familiar with. > > > > > > > > So we thought how new tx API might looks like, and here is the draft. > > > > > > > > 1) Define new enum *TransactionIsolationLevel *(to avoid clashes with > > > > current enum) with three standard modes - READ_COMMITTED, > > > REPEATABLE_READ, > > > > SERIALIZABLE. > > > > 2) Define new enum *TransactionHint* - READ_ONLY, OPTIMISTIC_LOCKING > > > > 3) No more OPTIMISTIC and PESSIMISTIC. Seriously. > > > > 4) Reads never acuire locks > > > > 5) Writes always acquire locks > > > > 6) *IgniteCache.withReadForUpdate()* will return special facade > which > > > will > > > > obtain locks on reads. This is analogue of SELECT FOR UPDATE in SQL. > > > > 7) *TransactionHint.READ_ONLY* - forces transaction to throw an > > exception > > > > on any update > > > > 8) *TransactionHint.OPTIMISTIC_LOCKING* - turns transaction into our > > > > current deadlock-free OPTIMISTIC/SERIALIZABLE mode. Applicable only > to > > > > SERIALIZABLE isolation level. > > > > 9) Define new API methods: > > > > - IgniteTransactions.txStart(TransactionIsolationLevel isolation) > > > > - IgniteTransactions.txStart(TransactionIsolationLevel isolation, > > > > TransactionHint... hints) > > > > 10) Deprecate old TX start methods > > > > > > > > As a result we will have simple, clean and extensible API. Which can > be > > > > explained to users in 5 minutes, instead of current half an hour. And > > > which > > > > is perfectly aligned with upcoming transactional SQL. > > > > > > > > Thoughts? > > > > > > > > > > > > On Thu, Sep 7, 2017 at 6:48 AM, Dmitriy Setrakyan < > > dsetrak...@apache.org > > > > > > > > wrote: > > > > > > > > > Vova, > > > > > > > > > > Thanks for doing the research. The changes you are suggesting are a > > bit > > > > too > > > > > bold, so let's discuss them in some more detail... > > > > > > > > > > On Wed, Sep 6, 2017 at 4:51 AM, Vladimir Ozerov < > > voze...@gridgain.com> > > > > > wrote: > > > > > > > > > > > Igniters, > > > > > > > > > > > > We are moving towards DBMS system. None of them has a notion of > > > > > > OPTIMISTIC/PESSIMISTIC transactions. Instead, they work as > follows: > > > > > > 1) Reads (SELECT) do not acquire exclusive row locks > > > > > > 2) Exclusive lock on read could be forced explicitly (SELECT ... > > FOR > > > > > > UPDATE) > > > > > > 3) Writes do acuire explicit row locks > > > > > > 4) Locks are always acquired immediately once statement is > executed > > > > > > 5) The strictest concurrency level - typically SERIALIZABLE - > rely > > on > > > > > > so-called *range locks* (or *predicate locks*) to track > > dependencies > > > > > > between transactions. Some vendors throw an exception in case of > > > > > conflict - > > > > > > these are ones where snapshot-based MVCC is used - PostgreSQL, > > > Oracle. > > > > > > Others do aggressive locking - ones where two-phase locking > > algorithm > > > > is > > > > > > used - SQL Server, MySQL. > > > > > > > > > > > > As you see, there is no concept of PESSIMISTIC/OPTIMISTIC modes. > > > > Instead, > > > > > > all updates are "PESSIMISTIC", reads are "OPTIMISTIC" but could > > > become > > > > > > "PESSIMISTIC" if requested explicitly, and for snapshot-based > > vendors > > > > (we > > > > > > are going in this direction) read-write conflicts are resolved in > > > > manner > > > > > > somewhat similar to our OPTIMISTIC/SERIALIZABLE. > > > > > > > > > > > > That said, I would propose to think on how transactions could > look > > > like > > > > > in > > > > > > future Ignite versions (say, 3.0). My rough vision: > > > > > > > > > > > > 1) No OPTIMISTIC mode at all - too counterintuitive and complex. > > It's > > > > > only > > > > > > advantage is deadlock-freedom when combined with SERIALIZABLE. If > > we > > > > have > > > > > > good deadlock detector and nice administrative capabilities, this > > > would > > > > > not > > > > > > be a problem for us. > > > > > > > > > > > > > > > Hm... The advantage of Optimistic Serialiazable mode is actually > > > > lock-free > > > > > transactions. The deadlock is impossible in this case. I doubt any > > > > deadlock > > > > > detector would match the performance advantage we get from > lock-free > > > > > transactions. > > > > > > > > > > > > > > > > > > > > > > 2) Behavior of reads could be controlled through "with" facade: > > > > > > V val1 = cache.get(key1); // Shared lock or no > lock > > > > > > V val2 = cache.withForUpdate().get(key2); // Exclusive lock > > > > > > > > > > > > > > > > Don't like the API. We are not trying to abandon the data grid > > use-case > > > > or > > > > > API, we are trying to add the database use case. > > > > > > > > > > > > > > > > 3) REPEATABLE_READ - throw exception in case of write-write > > conflict > > > > > > > > > > > > > > > > Well, I would like to preserve the PESSIMISTIC mode. I find it more > > > > > convenient than the "withForUpdate" API. It almost seems like you > are > > > > > trying to force the pendulum too far in the opposite direction. > > > > > > > > > > > > > > > > 4) SERIALIZABLE - throw exception in case of write-write and > > > write-read > > > > > > confilct (this is how our OPTIMISTIC/SERIALZABLE works now, but > it > > > > > doesn't > > > > > > support predicates) > > > > > > > > > > > > > > > > So, no change here? Good :) > > > > > > > > > > > > > > > > 5) Add READ_ONLY isolation mode where updates will not be allowed > > at > > > > all. > > > > > > Such transacrtons would be able to bypass some Ignite internals > to > > > > > achieve > > > > > > greater performance, what could be valuable for mostly-read use > > cases > > > > > (e.g. > > > > > > OLAP). > > > > > > > > > > > > > > > > Love the idea. We have already seen many use cases that could > benefit > > > > from > > > > > it. > > > > > How hard is it to implement? > > > > > > > > > > > > > > > > > > > > > > Thoughts? > > > > > > > > > > > > Vladimir. > > > > > > > > > > > > > > > > > > > > >
