Unfortunately, all images have been removed, so I am sending a link to the diagrams instead https://miro.com/app/board/uXjVGdd8m44=/?share_link_id=686668490384 .
On Thu, Dec 11, 2025 at 8:53 AM Andrii Lomakin <[email protected]> wrote: > Good day. > > If I followed Ken's line of thought correctly, I would like to expand it > to be part of the new proposal for the transaction API. > First of all, about the model, I think we came to a consensus that it > should be: > > [image: image.png] > The point is that we do not have a notion of connection in TinkerPop, but > we have a notion of GraphTraversal. > So, from my perspective, this diagram can be changed to > [image: image.png] > There is a corner case for Non-TX databases that we need to discuss. > I believe Non-TX mode is not appropriate for handling graph data > structures. GQL, for example, requires serializable isolation to satisfy > the standard requirements. > Instead, we should require to have presence of a transaction, which is > started *implicitly* when GraphTraversal is created and committed once it > is closed. > > I think we DO need methods for explicit transaction management, but 90% of > the cases will be covered by implicit transaction management throughout the > GraphTraversal life cycle. > 1. TX is started when GraphTraversal is created. > 2. TX is committed when GraphTraversal is closed (which includes terminal > operations like toList or iterate). > 3. TX is rolled back in case of exceptional execution of > GraphTraversal. Users can always roll back a transaction by using the > fail() step. > > We also provide in the YTDB the following methods that I hope will become > part of the framework: gSource.computeInTx(g ->{}), gSource.executeInTx(g - > {}), gSource.autoExecuteInTx(g - {}: g) (this one executes g.iterate() on > the returned GraphTraversal, which we find really handy). > > Now, regarding explicit TX management. > I think that explicit TX management is still needed if you want to execute > several queries and mix them with complex logic, but I think that methods > that explicitly cause TX begin/start/commit are confusing and provide bad > DevX. > What I propose is to use lambda-controlled TXs (slightly different from > the above). I think that an explicit rollback/commit of TX by the user is > similar to the goto operator in old programming languages; it gives finer > control over TXs, but overall only adds confusion and can be completely > avoided in modern languages. > > Explicit control on TXs is proposed to be implemented using methods > similar to above, with a different signature, namely: > gSource.executeInTx(graphTraversalSupplier -> { > var g = graphTraversalSupplier.get(); > var v = g.V().next(); > g.close() > > v.property("p", "val"); > int i = v.value("int"); > > if(i > 0) { > g = graphTraversalSupplier.get(); > g.V().has("i2", i).property("i2", i + 1).iterate(); > } > }) > > In this case, the transaction is detached from GraphTraversal, and its > lifecycle is not controlled by GraphTraversal. > Instead, TX is committed if an exception is not thrown from the lambda > method. > > To inform the user if GraphTraversal controls TX or not, a simple > isTxAttached method is proposed to be introduced > gSource.<V>computeInTX(graphTraversalSupplier -> {}) works in similar way > but returns value. > > > So the final diagram could look like the following: > > [image: image.png] > > Please note that in this model, there is no need for an explicit > transaction object. > > On Fri, Dec 5, 2025 at 10:18 AM Andrii Lomakin < > [email protected]> wrote: > >> Good day, Ken. >> >> >This could map to multiple threads on the provider side. It's probably >> OK if something like "g.tx()..." returns the same transaction. We could >> require users to open multiple DriverRemoteConnections/Clients to achieve >> this and that would be similar to other data connectivity APIs. >> >> Unfortunately I do not follow, could you elaborate more or give me >> example how do you see it ? >> >> On Thu, Dec 4, 2025 at 7:06 PM Ken Hu <[email protected]> wrote: >> >>> I might be thinking about this too much for just the remote case and not >>> considering the embedded case enough. In my mind, there just needs to be an >>> easy way for users of the language variants (in particular JavaScript >>> because of its single threaded nature) to start multiple transactions from >>> a single thread. This could map to multiple threads on the provider side. >>> It's probably OK if something like "g.tx()..." returns the same >>> transaction. We could require users to open multiple >>> DriverRemoteConnections/Clients to achieve this and that would be similar >>> to other data connectivity APIs. >>> >>> On Thu, Dec 4, 2025 at 2:37 AM Andrii Lomakin via dev < >>> [email protected]> wrote: >>> >>>> The point about having multiple concurrent transactions within the same >>>> thread warrants a more detailed discussion. >>>> >>>> To compare this behavior, consider how major SQL databases handle >>>> attempts >>>> to start a second transaction when one is already active: >>>> >>>> 1. Error out: The database prevents the operation, signaling that a >>>> transaction is already active (e.g., "Transaction already active"). >>>> 2. Create a nested transaction (savepoint): The database creates a >>>> sub-scope of the first transaction, not an independent transaction. If >>>> the >>>> parent transaction fails, the child scope also fails. >>>> 3. Implicitly commit the first: The database automatically commits the >>>> first transaction to open the second one (a common behavior in MySQL >>>> DDL). >>>> >>>> As far as I know, only Firebird has historically allowed several truly >>>> independent transactions concurrently on the same thread. I suspect >>>> support >>>> for simultaneous transactions may be even more limited among graph >>>> database >>>> vendors. >>>> >>>> On Thu, Dec 4, 2025 at 10:47 AM Andrii Lomakin < >>>> [email protected]> >>>> wrote: >>>> >>>> > Good day. >>>> > As for questions from Ken >>>> > >What OGMs are you referring to here? Regular transactions are >>>> complicated >>>> > enough by itself, I would only want to introduce nested transactions >>>> if >>>> > there were well maintained OGMs that would have a demonstrable >>>> benefit from >>>> > this. >>>> > >>>> > We have such an OGM using TinkerPop that is well maintained, if you >>>> mean >>>> > by this years of support and thousands of instances running on OS >>>> version. >>>> > However, I have given it thought and believe that it is a relatively >>>> rare >>>> > case, and OGM is so complex that it can manage nested transactions >>>> itself >>>> > if needed. >>>> > I think we can support nested transactions on pause for now, till we >>>> > receive more feedback from users. >>>> > >>>> > There are many inconsistencies to discuss in the TX lifecycle, even >>>> > without nested transactions. >>>> > >>>> > >Sessions won't exist moving forward (4.x+) as they are tied to remote >>>> > Groovy execution. So let's continue this conversation as simply >>>> > transactions and not refer to sessions at all to prevent confusion, >>>> unless >>>> > you intend to have these changes in the 3.8.x line? >>>> > Yeah, we already removed Groovy support in our fork, so it is even >>>> now in >>>> > our distribution TX, and the session is the same. >>>> > >>>> > >>>> > >Could you expand a bit more on "Transaction scope is limited to the >>>> > current thread only". >>>> > I mean that TX visibility and manipulation are limited to a single >>>> thread >>>> > where it is created. >>>> > If some of the providers support TX visibility between several >>>> threads, >>>> > that will only add a bonus for them if such a need arises, of course, >>>> > without breaking the API. >>>> > >>>> > > I find it limiting that an attempt to open multiple transactions >>>> from >>>> > the same thread ends up returning the same Transaction instance >>>> > Do you propose to support several transactions in the same thread? >>>> > I am afraid not all current vendors can support this model. >>>> > >>>> > On Wed, Dec 3, 2025 at 1:23 PM Stephen Mallette <[email protected] >>>> > >>>> > wrote: >>>> > >>>> >> Andrii, I'd like to just quickly clarify Ken's point stating " >>>> Sessions >>>> >> won't exist moving forward (4.x+) ". He's saying what you said: " >>>> >> 'session' >>>> >> and 'tx' are essentially the same concepts". We acknowledged that and >>>> >> decided to stop using "session" terminology and instead use >>>> "transaction" >>>> >> terminology only to avoid confusion. >>>> >> >>>> >> Much of what you wrote is in the spirit of what has been considered >>>> for >>>> >> 4.x, so that is nice alignment. I do think the goal though is to >>>> have a >>>> >> consistent transaction API/system that works for both embedded and >>>> remote >>>> >> cases. It will be interesting to see how that is achieved. >>>> >> >>>> >> On Tue, Dec 2, 2025 at 3:53 PM Ken Hu <[email protected]> wrote: >>>> >> >>>> >> > Hi Andrii, >>>> >> > >>>> >> > Good day to you as well. I have a couple questions about this that >>>> I >>>> >> would >>>> >> > like some more details on. >>>> >> > >>>> >> > 1. What OGMs are you referring to here? Regular transactions are >>>> >> > complicated enough by itself, I would only want to introduce nested >>>> >> > transactions if there were well maintained OGMs that would have a >>>> >> > demonstrable benefit from this. >>>> >> > 2. I like some of your ideas about removing Transaction as its own >>>> >> class, >>>> >> > but I'm not sure GraphTraversalSource is where it should be >>>> instead. Do >>>> >> you >>>> >> > have some more details about what you think the Transaction API >>>> should >>>> >> look >>>> >> > like? >>>> >> > 3. Could you expand a bit more on "Transaction scope is limited to >>>> the >>>> >> > current thread only". I always get confused by TinkerPop's >>>> threaded vs >>>> >> > multithreaded transaction. I find it limiting that an attempt to >>>> open >>>> >> > multiple transactions from the same thread ends up returning the >>>> same >>>> >> > Transaction instance. Maybe I didn't quite understand what you >>>> meant by >>>> >> > this. >>>> >> > 4. Sessions won't exist moving forward (4.x+) as they are tied to >>>> remote >>>> >> > Groovy execution. So let's continue this conversation as simply >>>> >> > transactions and not refer to sessions at all to prevent confusion, >>>> >> unless >>>> >> > you intend to have these changes in the 3.8.x line? >>>> >> > >>>> >> > Regards, >>>> >> > Ken >>>> >> > >>>> >> > >>>> >> > On Tue, Dec 2, 2025 at 3:33 AM Andrii Lomakin via dev < >>>> >> > [email protected]> wrote: >>>> >> > >>>> >> > > Small clarification. >>>> >> > > Implementation notes are related to changes in the remote >>>> protocol. >>>> >> > > >>>> >> > > On Tue, Dec 2, 2025 at 8:55 AM Andrii Lomakin < >>>> >> > > [email protected]> >>>> >> > > wrote: >>>> >> > > >>>> >> > > > To clarify: in this model, 'session' and 'tx' are essentially >>>> the >>>> >> same >>>> >> > > > concepts. >>>> >> > > > >>>> >> > > > I also think it is essential to have a common denominator that >>>> >> makes TP >>>> >> > > > applications portable across implementations. >>>> >> > > > That is why I also propose to restrict transaction scope to the >>>> >> thread >>>> >> > > > scope. It will be a common denominator that will not harm user >>>> >> > experience >>>> >> > > > but will ensure uniform and expected API behavior. >>>> >> > > > There should not be both `begin` and `open` methods because the >>>> >> > contract >>>> >> > > > of the beginning method is vague and unpredictable. >>>> >> > > > >>>> >> > > > On Tue, Dec 2, 2025 at 8:18 AM Andrii Lomakin < >>>> >> > > > [email protected]> wrote: >>>> >> > > > >>>> >> > > >> Good day, >>>> >> > > >> >>>> >> > > >> I have a proposal to enhance the transaction handling logic >>>> in TP, >>>> >> > > >> specifically regarding nested transaction calls, which are >>>> common >>>> >> in >>>> >> > EE >>>> >> > > >> development, particularly when utilizing OGM. >>>> >> > > >> >>>> >> > > >> Currently, calling open multiple times causes an exception. >>>> This >>>> >> > > behavior >>>> >> > > >> is restrictive, and the term "open" can be confusing. >>>> >> > > >> >>>> >> > > >> I propose a design that improves the developer experience by >>>> using >>>> >> an >>>> >> > > >> internal counter for transaction calls: >>>> >> > > >> >>>> >> > > >> Proposed transaction counter logic >>>> >> > > >> 1. begin/commit counter: >>>> >> > > >> - begin would increment an internal counter. >>>> >> > > >> - commit would decrement the internal counter, causing the >>>> actual >>>> >> > > >> transaction commit only when the counter reaches zero. >>>> >> > > >> 2. rollback behavior: >>>> >> > > >> - rollback would immediately force a transaction rollback and >>>> >> > > decrements >>>> >> > > >> the internal counter >>>> >> > > >> - the subsequent calls to rollback on the same transaction >>>> object >>>> >> > > should >>>> >> > > >> be allowed until the stack trace reaches the initial begin >>>> method >>>> >> > call, >>>> >> > > >> after which further rollback calls would throw an exception. >>>> >> > > >> 3. Error handling: >>>> >> > > >> - A commit without a matching begin (e.g., counter is already >>>> zero) >>>> >> > > >> should throw an exception. >>>> >> > > >> - A rollback without a matching begin should throw an >>>> exception >>>> >> > > >> >>>> >> > > >> I would also consider hiding the Transaction object >>>> altogether and >>>> >> > > >> delegating the logic of transaction management to >>>> >> GraphTraversalSource >>>> >> > > it >>>> >> > > >> will simplify the API a lot, both for users and vendors. >>>> >> > > >> >>>> >> > > >> I also propose to restrict transaction scope to the thread >>>> scope, >>>> >> it >>>> >> > > will >>>> >> > > >> be a common denominator that will not harm user experience >>>> but will >>>> >> > > ensure >>>> >> > > >> uniform and expected API behavior across all implementations. >>>> >> > > >> >>>> >> > > >> Proposed API changes >>>> >> > > >> - I propose deprecating open and using begin with the >>>> semantics >>>> >> > > >> described above to avoid confusion. >>>> >> > > >> - Transaction object is deprecated, and TX control >>>> functionality is >>>> >> > > >> delegated to the GraphTraversal/Graph instances. >>>> >> > > >> - tx() method is deprecated. >>>> >> > > >> - Transaction scope is limited to the current thread only. >>>> >> > > >> >>>> >> > > >> Behavior for non-direct calls >>>> >> > > >> When begin/commit/rollback methods are not called directly, >>>> the >>>> >> > > >> transaction should be automatically committed by a terminal >>>> >> operation >>>> >> > > >> (e.g., when hasNext returns false) in both remote and embedded >>>> >> modes. >>>> >> > > >> >>>> >> > > >> Implementation notes: >>>> >> > > >> 1. The begin command should be added. Relying on the implicit >>>> >> start of >>>> >> > > TX >>>> >> > > >> by traversal can lead to non-controlled side effects. >>>> >> > > >> 2. For the sake of optimization of the remote protocol, while >>>> >> multiple >>>> >> > > >> calls of begin/commit/rollback are allowed in the remote >>>> protocol, >>>> >> > > >> practically, it will mean that we will track the counter >>>> locally on >>>> >> > the >>>> >> > > >> client and will send begin/commit/rollback only once for the >>>> TX >>>> >> > > lifecycle. >>>> >> > > >> >>>> >> > > >> This model should be uniform across all deployment types. >>>> >> > > >> What are your thoughts on this approach? >>>> >> > > >> >>>> >> > > >> ----- >>>> >> > > >> Andrii Lomakin >>>> >> > > >> YouTrackDB development lead >>>> >> > > >> >>>> >> > > > >>>> >> > > > >>>> >> > > > -- >>>> >> > > > Andrii Lomakin >>>> >> > > > YouTrackDB development lead >>>> >> > > > >>>> >> > > >>>> >> > > >>>> >> > > -- >>>> >> > > Andrii Lomakin >>>> >> > > YouTrackDB development lead >>>> >> > > >>>> >> > >>>> >> >>>> > >>>> > >>>> > -- >>>> > Andrii Lomakin >>>> > YouTrackDB development lead >>>> > >>>> >>>> >>>> -- >>>> Andrii Lomakin >>>> YouTrackDB development lead >>>> >>> >> >> -- >> Andrii Lomakin >> YouTrackDB development lead >> > > > -- > Andrii Lomakin > YouTrackDB development lead > -- Andrii Lomakin YouTrackDB development lead
