Hi, I had a brief discussion with Andrii in the discord chat regarding the transaction api.
My idea is that the transaction api should be unopinionated. That means it should follow closely to `javax.transaction.UserTransaction` which in turn means we have a dedicated class modelling the transaction with `begin()`, `commit()` and `rollback()`. My reasoning is that gremlin is a low level graph language. There are many ways different developers / architects can go about integrating gremlin into their stack. Some like lambdas, some like IOT containers, some have domain entity frameworks, some use aspect orienated programming and on an on. It is common for IOT containers to manage transaction boundaries and lifecycles. Similarly it is common to use the web server's interceptors to manage transaction boundaries. And it is very common to do it all manually... Further, I'd say that transactions should be optional. Many persistent db/graph systems are eventually consistent and do not support transactions, so they should not be burdened with concepts foreign to them. Regarding binding the transaction to the thread: This is the only way I have ever worked so its entirely natural to me. However I can image that people want to defince their own transaction context, so perhaps we should not hardcode that part. Long ago Gavin King from Hibernate fame had the notion of `conversation` context. Perhaps such things are still around? Regards Pieter On Thu, 2025-12-11 at 09:02 +0100, Andrii Lomakin via dev wrote: > Unfortunately, all images have been removed, so I am sending a link > to the diagrams instead > https://miro.com/app/board/uXjVGdd8m44=/?share_link_id=686668490384 > . > > > > On Thu, Dec 11, 2025 at 8:53 AM Andrii Lomakin > <[email protected]> wrote: > > Good day. > > > > If I followed Ken's line of thought correctly, I would like to > > expand it to be part of the new proposal for the transaction API. > > First of all, about the model, I think we came to a consensus that > > it should be: > > > > > > image.png > > The point is that we do not have a notion of connection in > > TinkerPop, but we have a notion of GraphTraversal. > > So, from my perspective, this diagram can be changed to > > image.png > > > > There is a corner case for Non-TX databases that we need to > > discuss. > > I believe Non-TX mode is not appropriate for handling graph data > > structures. GQL, for example, requires serializable isolation to > > satisfy the standard requirements. > > Instead, we should require to have presence of a transaction, which > > is started implicitly when GraphTraversal is created and committed > > once it is closed. > > > > I think we DO need methods for explicit transaction management, but > > 90% of the cases will be covered by implicit transaction management > > throughout the GraphTraversal life cycle. > > 1. TX is started when GraphTraversal is created. > > 2. TX is committed when GraphTraversal is closed (which includes > > terminal operations like toList or iterate). > > 3. TX is rolled back in case of exceptional execution of > > GraphTraversal. Users can always roll back a transaction by using > > the fail() step. > > > > We also provide in the YTDB the following methods that I hope will > > become part of the framework: gSource.computeInTx(g ->{}), > > gSource.executeInTx(g - {}), gSource.autoExecuteInTx(g - {}: g) > > (this one executes g.iterate() on the returned GraphTraversal, > > which we find really handy). > > > > Now, regarding explicit TX management. > > I think that explicit TX management is still needed if you want to > > execute several queries and mix them with complex logic, but I > > think that methods that explicitly cause TX begin/start/commit are > > confusing and provide bad DevX. > > What I propose is to use lambda-controlled TXs (slightly different > > from the above). I think that an explicit rollback/commit of TX by > > the user is similar to the goto operator in old programming > > languages; it gives finer control over TXs, but overall only adds > > confusion and can be completely avoided in modern languages. > > > > Explicit control on TXs is proposed to be implemented using methods > > similar to above, with a different signature, namely: > > gSource.executeInTx(graphTraversalSupplier -> { > > var g = graphTraversalSupplier.get(); > > var v = g.V().next(); > > g.close() > > > > v.property("p", "val"); > > int i = v.value("int"); > > > > if(i > 0) { > > g = graphTraversalSupplier.get(); > > g.V().has("i2", i).property("i2", i + 1).iterate(); > > } > > }) > > > > In this case, the transaction is detached from GraphTraversal, and > > its lifecycle is not controlled by GraphTraversal. > > Instead, TX is committed if an exception is not thrown from the > > lambda method. > > > > To inform the user if GraphTraversal controls TX or not, a simple > > isTxAttached method is proposed to be introduced > > gSource.<V>computeInTX(graphTraversalSupplier -> {}) works in > > similar way but returns value. > > > > > > So the final diagram could look like the following: > > > > > > image.png > > > > Please note that in this model, there is no need for an explicit > > transaction object. > > > > On Fri, Dec 5, 2025 at 10:18 AM Andrii Lomakin > > <[email protected]> wrote: > > > Good day, Ken. > > > > > > >This could map to multiple threads on the provider side. It's > > > probably OK if something like "g.tx()..." returns the same > > > transaction. We could require users to open multiple > > > DriverRemoteConnections/Clients to achieve this and that would be > > > similar to other data connectivity APIs. > > > > > > Unfortunately I do not follow, could you elaborate more or give > > > me example how do you see it ? > > > > > > On Thu, Dec 4, 2025 at 7:06 PM Ken Hu <[email protected]> wrote: > > > > I might be thinking about this too much for just the remote > > > > case and not considering the embedded case enough. In my mind, > > > > there just needs to be an easy way for users of the language > > > > variants (in particular JavaScript because of its single > > > > threaded nature) to start multiple transactions from a single > > > > thread. This could map to multiple threads on the provider > > > > side. It's probably OK if something like "g.tx()..." returns > > > > the same transaction. We could require users to open multiple > > > > DriverRemoteConnections/Clients to achieve this and that would > > > > be similar to other data connectivity APIs. > > > > > > > > On Thu, Dec 4, 2025 at 2:37 AM Andrii Lomakin via dev > > > > <[email protected]> wrote: > > > > > The point about having multiple concurrent transactions > > > > > within the same > > > > > thread warrants a more detailed discussion. > > > > > > > > > > To compare this behavior, consider how major SQL databases > > > > > handle attempts > > > > > to start a second transaction when one is already active: > > > > > > > > > > 1. Error out: The database prevents the operation, signaling > > > > > that a > > > > > transaction is already active (e.g., "Transaction already > > > > > active"). > > > > > 2. Create a nested transaction (savepoint): The database > > > > > creates a > > > > > sub-scope of the first transaction, not an independent > > > > > transaction. If the > > > > > parent transaction fails, the child scope also fails. > > > > > 3. Implicitly commit the first: The database automatically > > > > > commits the > > > > > first transaction to open the second one (a common behavior > > > > > in MySQL DDL). > > > > > > > > > > As far as I know, only Firebird has historically allowed > > > > > several truly > > > > > independent transactions concurrently on the same thread. I > > > > > suspect support > > > > > for simultaneous transactions may be even more limited among > > > > > graph database > > > > > vendors. > > > > > > > > > > On Thu, Dec 4, 2025 at 10:47 AM Andrii Lomakin > > > > > <[email protected]> > > > > > wrote: > > > > > > > > > > > Good day. > > > > > > As for questions from Ken > > > > > > >What OGMs are you referring to here? Regular transactions > > > > > are complicated > > > > > > enough by itself, I would only want to introduce nested > > > > > transactions if > > > > > > there were well maintained OGMs that would have a > > > > > demonstrable benefit from > > > > > > this. > > > > > > > > > > > > We have such an OGM using TinkerPop that is well > > > > > maintained, if you mean > > > > > > by this years of support and thousands of instances running > > > > > on OS version. > > > > > > However, I have given it thought and believe that it is a > > > > > relatively rare > > > > > > case, and OGM is so complex that it can manage nested > > > > > transactions itself > > > > > > if needed. > > > > > > I think we can support nested transactions on pause for > > > > > now, till we > > > > > > receive more feedback from users. > > > > > > > > > > > > There are many inconsistencies to discuss in the TX > > > > > lifecycle, even > > > > > > without nested transactions. > > > > > > > > > > > > >Sessions won't exist moving forward (4.x+) as they are > > > > > tied to remote > > > > > > Groovy execution. So let's continue this conversation as > > > > > simply > > > > > > transactions and not refer to sessions at all to prevent > > > > > confusion, unless > > > > > > you intend to have these changes in the 3.8.x line? > > > > > > Yeah, we already removed Groovy support in our fork, so it > > > > > is even now in > > > > > > our distribution TX, and the session is the same. > > > > > > > > > > > > > > > > > > >Could you expand a bit more on "Transaction scope is > > > > > limited to the > > > > > > current thread only". > > > > > > I mean that TX visibility and manipulation are limited to a > > > > > single thread > > > > > > where it is created. > > > > > > If some of the providers support TX visibility between > > > > > several threads, > > > > > > that will only add a bonus for them if such a need arises, > > > > > of course, > > > > > > without breaking the API. > > > > > > > > > > > > > I find it limiting that an attempt to open multiple > > > > > transactions from > > > > > > the same thread ends up returning the same Transaction > > > > > instance > > > > > > Do you propose to support several transactions in the same > > > > > thread? > > > > > > I am afraid not all current vendors can support this model. > > > > > > > > > > > > On Wed, Dec 3, 2025 at 1:23 PM Stephen Mallette > > > > > <[email protected]> > > > > > > wrote: > > > > > > > > > > > >> Andrii, I'd like to just quickly clarify Ken's point > > > > > stating " Sessions > > > > > >> won't exist moving forward (4.x+) ". He's saying what you > > > > > said: " > > > > > >> 'session' > > > > > >> and 'tx' are essentially the same concepts". We > > > > > acknowledged that and > > > > > >> decided to stop using "session" terminology and instead > > > > > use "transaction" > > > > > >> terminology only to avoid confusion. > > > > > >> > > > > > >> Much of what you wrote is in the spirit of what has been > > > > > considered for > > > > > >> 4.x, so that is nice alignment. I do think the goal though > > > > > is to have a > > > > > >> consistent transaction API/system that works for both > > > > > embedded and remote > > > > > >> cases. It will be interesting to see how that is achieved. > > > > > >> > > > > > >> On Tue, Dec 2, 2025 at 3:53 PM Ken Hu <[email protected]> > > > > > wrote: > > > > > >> > > > > > >> > Hi Andrii, > > > > > >> > > > > > > >> > Good day to you as well. I have a couple questions about > > > > > this that I > > > > > >> would > > > > > >> > like some more details on. > > > > > >> > > > > > > >> > 1. What OGMs are you referring to here? Regular > > > > > transactions are > > > > > >> > complicated enough by itself, I would only want to > > > > > introduce nested > > > > > >> > transactions if there were well maintained OGMs that > > > > > would have a > > > > > >> > demonstrable benefit from this. > > > > > >> > 2. I like some of your ideas about removing Transaction > > > > > as its own > > > > > >> class, > > > > > >> > but I'm not sure GraphTraversalSource is where it should > > > > > be instead. Do > > > > > >> you > > > > > >> > have some more details about what you think the > > > > > Transaction API should > > > > > >> look > > > > > >> > like? > > > > > >> > 3. Could you expand a bit more on "Transaction scope is > > > > > limited to the > > > > > >> > current thread only". I always get confused by > > > > > TinkerPop's threaded vs > > > > > >> > multithreaded transaction. I find it limiting that an > > > > > attempt to open > > > > > >> > multiple transactions from the same thread ends up > > > > > returning the same > > > > > >> > Transaction instance. Maybe I didn't quite understand > > > > > what you meant by > > > > > >> > this. > > > > > >> > 4. Sessions won't exist moving forward (4.x+) as they > > > > > are tied to remote > > > > > >> > Groovy execution. So let's continue this conversation as > > > > > simply > > > > > >> > transactions and not refer to sessions at all to prevent > > > > > confusion, > > > > > >> unless > > > > > >> > you intend to have these changes in the 3.8.x line? > > > > > >> > > > > > > >> > Regards, > > > > > >> > Ken > > > > > >> > > > > > > >> > > > > > > >> > On Tue, Dec 2, 2025 at 3:33 AM Andrii Lomakin via dev < > > > > > >> > [email protected]> wrote: > > > > > >> > > > > > > >> > > Small clarification. > > > > > >> > > Implementation notes are related to changes in the > > > > > remote protocol. > > > > > >> > > > > > > > >> > > On Tue, Dec 2, 2025 at 8:55 AM Andrii Lomakin < > > > > > >> > > [email protected]> > > > > > >> > > wrote: > > > > > >> > > > > > > > >> > > > To clarify: in this model, 'session' and 'tx' are > > > > > essentially the > > > > > >> same > > > > > >> > > > concepts. > > > > > >> > > > > > > > > >> > > > I also think it is essential to have a common > > > > > denominator that > > > > > >> makes TP > > > > > >> > > > applications portable across implementations. > > > > > >> > > > That is why I also propose to restrict transaction > > > > > scope to the > > > > > >> thread > > > > > >> > > > scope. It will be a common denominator that will not > > > > > harm user > > > > > >> > experience > > > > > >> > > > but will ensure uniform and expected API behavior. > > > > > >> > > > There should not be both `begin` and `open` methods > > > > > because the > > > > > >> > contract > > > > > >> > > > of the beginning method is vague and unpredictable. > > > > > >> > > > > > > > > >> > > > On Tue, Dec 2, 2025 at 8:18 AM Andrii Lomakin < > > > > > >> > > > [email protected]> wrote: > > > > > >> > > > > > > > > >> > > >> Good day, > > > > > >> > > >> > > > > > >> > > >> I have a proposal to enhance the transaction > > > > > handling logic in TP, > > > > > >> > > >> specifically regarding nested transaction calls, > > > > > which are common > > > > > >> in > > > > > >> > EE > > > > > >> > > >> development, particularly when utilizing OGM. > > > > > >> > > >> > > > > > >> > > >> Currently, calling open multiple times causes an > > > > > exception. This > > > > > >> > > behavior > > > > > >> > > >> is restrictive, and the term "open" can be > > > > > confusing. > > > > > >> > > >> > > > > > >> > > >> I propose a design that improves the developer > > > > > experience by using > > > > > >> an > > > > > >> > > >> internal counter for transaction calls: > > > > > >> > > >> > > > > > >> > > >> Proposed transaction counter logic > > > > > >> > > >> 1. begin/commit counter: > > > > > >> > > >> - begin would increment an internal counter. > > > > > >> > > >> - commit would decrement the internal counter, > > > > > causing the actual > > > > > >> > > >> transaction commit only when the counter reaches > > > > > zero. > > > > > >> > > >> 2. rollback behavior: > > > > > >> > > >> - rollback would immediately force a transaction > > > > > rollback and > > > > > >> > > decrements > > > > > >> > > >> the internal counter > > > > > >> > > >> - the subsequent calls to rollback on the same > > > > > transaction object > > > > > >> > > should > > > > > >> > > >> be allowed until the stack trace reaches the > > > > > initial begin method > > > > > >> > call, > > > > > >> > > >> after which further rollback calls would throw an > > > > > exception. > > > > > >> > > >> 3. Error handling: > > > > > >> > > >> - A commit without a matching begin (e.g., counter > > > > > is already zero) > > > > > >> > > >> should throw an exception. > > > > > >> > > >> - A rollback without a matching begin should throw > > > > > an exception > > > > > >> > > >> > > > > > >> > > >> I would also consider hiding the Transaction object > > > > > altogether and > > > > > >> > > >> delegating the logic of transaction management to > > > > > >> GraphTraversalSource > > > > > >> > > it > > > > > >> > > >> will simplify the API a lot, both for users and > > > > > vendors. > > > > > >> > > >> > > > > > >> > > >> I also propose to restrict transaction scope to the > > > > > thread scope, > > > > > >> it > > > > > >> > > will > > > > > >> > > >> be a common denominator that will not harm user > > > > > experience but will > > > > > >> > > ensure > > > > > >> > > >> uniform and expected API behavior across all > > > > > implementations. > > > > > >> > > >> > > > > > >> > > >> Proposed API changes > > > > > >> > > >> - I propose deprecating open and using begin with > > > > > the semantics > > > > > >> > > >> described above to avoid confusion. > > > > > >> > > >> - Transaction object is deprecated, and TX control > > > > > functionality is > > > > > >> > > >> delegated to the GraphTraversal/Graph instances. > > > > > >> > > >> - tx() method is deprecated. > > > > > >> > > >> - Transaction scope is limited to the current > > > > > thread only. > > > > > >> > > >> > > > > > >> > > >> Behavior for non-direct calls > > > > > >> > > >> When begin/commit/rollback methods are not called > > > > > directly, the > > > > > >> > > >> transaction should be automatically committed by a > > > > > terminal > > > > > >> operation > > > > > >> > > >> (e.g., when hasNext returns false) in both remote > > > > > and embedded > > > > > >> modes. > > > > > >> > > >> > > > > > >> > > >> Implementation notes: > > > > > >> > > >> 1. The begin command should be added. Relying on > > > > > the implicit > > > > > >> start of > > > > > >> > > TX > > > > > >> > > >> by traversal can lead to non-controlled side > > > > > effects. > > > > > >> > > >> 2. For the sake of optimization of the remote > > > > > protocol, while > > > > > >> multiple > > > > > >> > > >> calls of begin/commit/rollback are allowed in the > > > > > remote protocol, > > > > > >> > > >> practically, it will mean that we will track the > > > > > counter locally on > > > > > >> > the > > > > > >> > > >> client and will send begin/commit/rollback only > > > > > once for the TX > > > > > >> > > lifecycle. > > > > > >> > > >> > > > > > >> > > >> This model should be uniform across all deployment > > > > > types. > > > > > >> > > >> What are your thoughts on this approach? > > > > > >> > > >> > > > > > >> > > >> ----- > > > > > >> > > >> Andrii Lomakin > > > > > >> > > >> YouTrackDB development lead > > > > > >> > > >> > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > -- > > > > > >> > > > Andrii Lomakin > > > > > >> > > > YouTrackDB development lead > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > >> > > -- > > > > > >> > > Andrii Lomakin > > > > > >> > > YouTrackDB development lead > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > -- > > > > > > Andrii Lomakin > > > > > > YouTrackDB development lead > > > > > > > > > > > > > > > >
