Hi all, We merged the Transaction PR. Here is an update on the performance metrics:
Machine: EC2 c5.8xlarge Neo4jGraph run on tmpfs All times in milliseconds. Scenarios tested against Kelvin Lawrence's air-routes-latest data. See https://github.com/Bit-Quill/tinkerpop/blob/ken/txntester/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/Tester.java for more details on each Scenario | Scenario | TinkerGraph | TinkerTransactionGraph | Neo4jGraph | TinkerGraph | | -------- | -------- | -------- | -------- | -------- | | Read the graph with both() | 25 500 | 111 087 | 39 118 | 23 528 | | Drop all vertices based on descending centrality | 260 | 309 | 1 011 | 134 | | Drop all vertices in random order | 228 | 298 | 806 | 75 | | Drop all edges in random order | 940 | 1101 | 3 003 | 677 | | Move edges to different vertices randomly | 9 453 | 10 572 | 13 976 | 9 246 | | Update properties on random elements | 47 424 | 50 057 | 49 820 | 47 542 | | Add properties to vertices with MergeV | 73 | 96 | 138 | 78 | | Upsert edges with MergeE | 11 037 | 16 229 | 18 532 | 10 602 | | Add edges with a power law distribution to vertices | 390 | 2 934 | 3 465 | 363 | | Rollback edges after attempting to add them | N/A | 2 576 | 2680 | N/A | | Rollback vertices after attempting to drop them | N/A | 354 | 250 | N/A | | Move edges while committing half and rollbacking half | N/A | 9 838 | 12 514 | N/A | | Add edge between nodes that are two degrees away | 40 482 | 281 893 | 25 505 | 41 017 | | Drop edges from the most central vertices | 18 570 | 70 889 | 32 756 | 18 267 | All known limitations have been listed in the reference documentations. We plan to implement further optimizations for reads and writes in 3.7.1 by creating transaction specific implementations of all TinkerElement classes. We expect these optimizations to bring TinkerTransactionGraph performance close to the non-transactional variant. Regards, Valentyn On Fri, Jul 7, 2023 at 5:25 PM Valentyn Kahamlyk <valent...@bitquilltech.com> wrote: > Hi all, > > I'm happy to announce that the transactional PR is ready for review > https://github.com/apache/tinkerpop/pull/2087. > I'm still planning a couple of small changes, but nothing big. > > Regards, Valentyn > > > On Thu, Jun 15, 2023 at 10:18 PM Valentyn Kahamlyk < > valent...@bitquilltech.com> wrote: > >> Hi all, >> >> I'm working on transactions implementation and I would like to discuss >> testing of this feature. >> >> Currently available options for tests in TinkerPop: >> 1. Unit tests. >> For development I wrote bunch of unit tests for the newly introduced >> TinkerTransactionGraph because they are easier to make and much more >> convenient to debug. However, these tests are very complicated to be reused >> by other providers. >> 2. Gherkin tests. >> This is the best option for providers to test some features. However, the >> current implementation does not support many of the abilities needed to test >> transactions, for example multi-step tests and multi-threaded tests. The >> ideal solution would be to add such features, but unfortunately it takes a >> lot of time and effort. >> For basic checks of TinkerTransactionGraph existing Gherkin tests are >> already used. >> 3. Integration tests. >> We have 2 kinds of integration tests in `gremlin-test` >> a) process tests are mirror of Gherkin tests with some small differences. >> b) structure tests are probably the best candidate to use for >> TinkerTransactionGraph. There is already a set of tests for transactions in >> TransactionTest and I can add more cases there. These tests will be used in >> a similar way to TinkerGraphStructureStandardTest. >> 4. Gremlin Server tests. >> I think these tests have a different responsibility. >> 5. GLV tests. >> All GLV's already have transaction tests and we can extend them to use >> TinkerTransactionGraph at server-side. >> >> To summarize I suggest using structure tests to verify transaction >> implementation. >> >> Regards, Valentyn >> >> >> On Wed, May 10, 2023 at 2:59 PM Valentyn Kahamlyk < >> valent...@bitquilltech.com> wrote: >> >>> Hi all, >>> >>> I opened PR with a detailed proposal for transaction implementation: >>> https://github.com/apache/tinkerpop/pull/2054. >>> >>> Regards, Valentyn >>> >>> On Mon, Feb 27, 2023 at 1:22 PM Valentyn Kahamlyk < >>> valent...@bitquilltech.com> wrote: >>> >>>> Hi all, >>>> >>>> Now, if you need to use transactions in tinkerpop, the only solution is >>>> to use the Neo4J plugin. Unfortunately, this plugin has not been updated >>>> for a long time and is only compatible with Neo4J version 3.4, which >>>> reached end of life in March, 2020. >>>> >>>> I propose to implement native transaction support in TinkerGraph. >>>> >>>> The most interesting question to discuss is the level of isolation in >>>> SQL 92 terms. There are 5 options to choose from: `Read uncommitted`, `Read >>>> committed`, `Repeatable read`, `Snapshot` and `Serializable`. >>>> >>>> In terms of balancing complexity of implementation and functionality, I >>>> think the most optimal option is `Read committed` or `Repeatable read`. >>>> >>>> I hope to implement `Read committed`, it is enough to add versions to >>>> the Vertices and Edges, add list of updated Elements in Graph. >>>> Write lock can be Vertex-level and Edge-level. >>>> >>>> Regards, Valentyn >>>> >>>
