IMO the use case makes sense. However, as Sergi already mentioned, the problem is far more complex, than simply passing TX state over a wire. Most probably a kind of coordinator will be required still to manage all kinds of failures. This task should be started with clean design proposal explaining how we handle all these concurrent events. And only then, when we understand all implications, we should move to development stage.
On Wed, Mar 15, 2017 at 10:38 AM, ALEKSEY KUZNETSOV < [email protected]> wrote: > Right > > ср, 15 мар. 2017 г. в 10:35, Sergi Vladykin <[email protected]>: > > > Good! Basically your orchestrator just takes some predefined graph of > > distributed services to be invoked, calls them by some kind of RPC and > > passes the needed parameters between them, right? > > > > Sergi > > > > 2017-03-14 22:46 GMT+03:00 ALEKSEY KUZNETSOV <[email protected]>: > > > > > orchestrator is a custom thing. He is responsible for managing business > > > scenarios flows. Many nodes are involved in scenarios. They exchange > data > > > and folow one another. If you acquinted with BPMN framework, so > > > orchestrator is like bpmn engine. > > > > > > вт, 14 Мар 2017 г., 18:56 Sergi Vladykin <[email protected]>: > > > > > > > What is Orchestrator for you? Is it a thing from Microsoft or your > > custom > > > > in-house software? > > > > > > > > Sergi > > > > > > > > 2017-03-14 18:00 GMT+03:00 ALEKSEY KUZNETSOV < > [email protected] > > >: > > > > > > > > > Fine. Let's say we've got multiple servers which fulfills custom > > logic. > > > > > This servers compound oriented graph (BPMN process) which > controlled > > by > > > > > Orchestrator. > > > > > For instance, *server1 *creates *variable A *with value 1, > persists > > it > > > > to > > > > > IGNITE cache and creates *variable B *and sends it to* server2. > *The > > > > > latests receives *variable B*, do some logic with it and stores to > > > > IGNITE. > > > > > All the work made by both servers must be fulfilled in *one* > > > transaction. > > > > > Because we need all information done, or nothing(rollbacked). The > > > > scenario > > > > > is managed by orchestrator. > > > > > > > > > > вт, 14 мар. 2017 г. в 17:31, Sergi Vladykin < > > [email protected] > > > >: > > > > > > > > > > > Ok, it is not a business case, it is your wrong solution for it. > > > > > > Lets try again, what is the business case? > > > > > > > > > > > > Sergi > > > > > > > > > > > > 2017-03-14 16:42 GMT+03:00 ALEKSEY KUZNETSOV < > > > [email protected] > > > > >: > > > > > > > > > > > > > The case is the following, One starts transaction in one node, > > and > > > > > commit > > > > > > > this transaction in another jvm node(or rollback it remotely). > > > > > > > > > > > > > > вт, 14 мар. 2017 г. в 16:30, Sergi Vladykin < > > > > [email protected] > > > > > >: > > > > > > > > > > > > > > > Because even if you make it work for some simplistic > scenario, > > > get > > > > > > ready > > > > > > > to > > > > > > > > write many fault tolerance tests and make sure that you TXs > > work > > > > > > > gracefully > > > > > > > > in all modes in case of crashes. Also make sure that we do > not > > > have > > > > > any > > > > > > > > performance drops after all your changes in existing > > benchmarks. > > > > All > > > > > in > > > > > > > all > > > > > > > > I don't believe these conditions will be met and your > > > contribution > > > > > will > > > > > > > be > > > > > > > > accepted. > > > > > > > > > > > > > > > > Better solution to what problem? Sending TX to another node? > > The > > > > > > problem > > > > > > > > statement itself is already wrong. What business case you are > > > > trying > > > > > to > > > > > > > > solve? I'm sure everything you need can be done in a much > more > > > > simple > > > > > > and > > > > > > > > efficient way at the application level. > > > > > > > > > > > > > > > > Sergi > > > > > > > > > > > > > > > > 2017-03-14 16:03 GMT+03:00 ALEKSEY KUZNETSOV < > > > > > [email protected] > > > > > > >: > > > > > > > > > > > > > > > > > Why wrong ? You know the better solution? > > > > > > > > > > > > > > > > > > вт, 14 мар. 2017 г. в 15:46, Sergi Vladykin < > > > > > > [email protected] > > > > > > > >: > > > > > > > > > > > > > > > > > > > Just serializing TX object and deserializing it on > another > > > node > > > > > is > > > > > > > > > > meaningless, because other nodes participating in the TX > > have > > > > to > > > > > > know > > > > > > > > > about > > > > > > > > > > the new coordinator. This will require protocol changes, > we > > > > > > > definitely > > > > > > > > > will > > > > > > > > > > have fault tolerance and performance issues. IMO the > whole > > > idea > > > > > is > > > > > > > > wrong > > > > > > > > > > and it makes no sense to waste time on it. > > > > > > > > > > > > > > > > > > > > Sergi > > > > > > > > > > > > > > > > > > > > 2017-03-14 10:57 GMT+03:00 ALEKSEY KUZNETSOV < > > > > > > > [email protected] > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > IgniteTransactionState implememntation contains > > > > IgniteTxEntry's > > > > > > > which > > > > > > > > > is > > > > > > > > > > > supposed to be transferable > > > > > > > > > > > > > > > > > > > > > > пн, 13 мар. 2017 г. в 19:32, Dmitriy Setrakyan < > > > > > > > > [email protected] > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > It sounds a little scary to me that we are passing > > > > > transaction > > > > > > > > > objects > > > > > > > > > > > > around. Such object may contain all sorts of Ignite > > > > context. > > > > > If > > > > > > > > some > > > > > > > > > > data > > > > > > > > > > > > needs to be passed across, we should create a special > > > > > transfer > > > > > > > > object > > > > > > > > > > in > > > > > > > > > > > > this case. > > > > > > > > > > > > > > > > > > > > > > > > D. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Mar 13, 2017 at 9:10 AM, ALEKSEY KUZNETSOV < > > > > > > > > > > > > [email protected] > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > well, there a couple of issues preventing > transaction > > > > > > > proceeding. > > > > > > > > > > > > > At first, After transaction serialization and > > > > > deserialization > > > > > > > on > > > > > > > > > the > > > > > > > > > > > > remote > > > > > > > > > > > > > server, there is no txState. So im going to put it > in > > > > > > > > > > > > > writeExternal()\readExternal() > > > > > > > > > > > > > > > > > > > > > > > > > > The last one is Deserialized transaction lacks of > > > shared > > > > > > cache > > > > > > > > > > context > > > > > > > > > > > > > field at TransactionProxyImpl. Perhaps, it must be > > > > injected > > > > > > by > > > > > > > > > > > > > GridResourceProcessor ? > > > > > > > > > > > > > > > > > > > > > > > > > > пн, 13 мар. 2017 г. в 17:27, ALEKSEY KUZNETSOV < > > > > > > > > > > > [email protected] > > > > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > > > while starting and continuing transaction in > > > different > > > > > jvms > > > > > > > in > > > > > > > > > run > > > > > > > > > > > into > > > > > > > > > > > > > > serialization exception in writeExternalMeta : > > > > > > > > > > > > > > > > > > > > > > > > > > > > @Override public void writeExternal(ObjectOutput > > out) > > > > > > throws > > > > > > > > > > > > IOException > > > > > > > > > > > > > { > > > > > > > > > > > > > > writeExternalMeta(out); > > > > > > > > > > > > > > > > > > > > > > > > > > > > some meta is cannot be serialized. > > > > > > > > > > > > > > пт, 10 мар. 2017 г. в 17:25, Alexey Goncharuk < > > > > > > > > > > > > > [email protected] > > > > > > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > > > > Aleksey, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think I am starting to get what you want, but I > > > have > > > > a > > > > > > few > > > > > > > > > > > concerns: > > > > > > > > > > > > > > - What is the API for the proposed change? In > your > > > > test, > > > > > > you > > > > > > > > > pass > > > > > > > > > > an > > > > > > > > > > > > > > instance of transaction created on ignite(0) to > the > > > > > ignite > > > > > > > > > instance > > > > > > > > > > > > > > ignite(1). This is obviously not possible in a > > truly > > > > > > > > distributed > > > > > > > > > > > > > > (multi-jvm) environment. > > > > > > > > > > > > > > - How will you synchronize cache update actions > and > > > > > > > transaction > > > > > > > > > > > commit? > > > > > > > > > > > > > > Say, you have one node that decided to commit, > but > > > > > another > > > > > > > node > > > > > > > > > is > > > > > > > > > > > > still > > > > > > > > > > > > > > writing within this transaction. How do you make > > sure > > > > > that > > > > > > > two > > > > > > > > > > nodes > > > > > > > > > > > > will > > > > > > > > > > > > > > not call commit() and rollback() simultaneously? > > > > > > > > > > > > > > - How do you make sure that either commit() or > > > > > rollback() > > > > > > is > > > > > > > > > > called > > > > > > > > > > > if > > > > > > > > > > > > > an > > > > > > > > > > > > > > originator failed? > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2017-03-10 15:38 GMT+03:00 Дмитрий Рябов < > > > > > > > > [email protected] > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Alexey Goncharuk, heh, my initial understanding > > was > > > > > that > > > > > > > > > > > transferring > > > > > > > > > > > > > of > > > > > > > > > > > > > > tx > > > > > > > > > > > > > > > ownership from one node to another will be > > happened > > > > > > > > > automatically > > > > > > > > > > > > when > > > > > > > > > > > > > > > originating node is gone down. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2017-03-10 15:36 GMT+03:00 ALEKSEY KUZNETSOV < > > > > > > > > > > > > [email protected] > > > > > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Im aiming to span transaction on multiple > > > threads, > > > > > > nodes, > > > > > > > > > > > > jvms(soon). > > > > > > > > > > > > > > So > > > > > > > > > > > > > > > > every node is able to rollback, or commit > > common > > > > > > > > > transaction.It > > > > > > > > > > > > > turned > > > > > > > > > > > > > > > up i > > > > > > > > > > > > > > > > need to transfer tx between nodes in order to > > > > commit > > > > > > > > > > transaction > > > > > > > > > > > in > > > > > > > > > > > > > > > > different node(in the same jvm). > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > пт, 10 мар. 2017 г. в 15:20, Alexey > Goncharuk < > > > > > > > > > > > > > > > [email protected] > > > > > > > > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Aleksey, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Do you mean that you want a concept of > > > > transferring > > > > > > of > > > > > > > tx > > > > > > > > > > > > ownership > > > > > > > > > > > > > > > from > > > > > > > > > > > > > > > > > one node to another? My initial > understanding > > > was > > > > > > that > > > > > > > > you > > > > > > > > > > want > > > > > > > > > > > > to > > > > > > > > > > > > > be > > > > > > > > > > > > > > > > able > > > > > > > > > > > > > > > > > to update keys in a transaction from > multiple > > > > > threads > > > > > > > in > > > > > > > > > > > > parallel. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --AG > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2017-03-10 15:01 GMT+03:00 ALEKSEY > KUZNETSOV > > < > > > > > > > > > > > > > > [email protected] > > > > > > > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Well. Consider transaction started in one > > > node, > > > > > and > > > > > > > > > > continued > > > > > > > > > > > > in > > > > > > > > > > > > > > > > another > > > > > > > > > > > > > > > > > > one. > > > > > > > > > > > > > > > > > > The following test describes my idea: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Ignite ignite1 = ignite(0); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > IgniteTransactions transactions = > > > > > > > > ignite1.transactions(); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > IgniteCache<String, Integer> cache = > > > > > > > > > > > ignite1.getOrCreateCache(" > > > > > > > > > > > > > > > > > > testCache"); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Transaction tx = transactions.txStart( > > > > > concurrency, > > > > > > > > > > > isolation); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > cache.put("key1", 1); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > cache.put("key2", 2); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > tx.stop(); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > IgniteInternalFuture<Boolean> fut = > > > > > > > > > > GridTestUtils.runAsync(() > > > > > > > > > > > > -> > > > > > > > > > > > > > { > > > > > > > > > > > > > > > > > > IgniteTransactions ts = > > > > > > ignite(1).transactions(); > > > > > > > > > > > > > > > > > > Assert.assertNull(ts.tx()); > > > > > > > > > > > > > > > > > > Assert.assertEquals( > > > > > TransactionState.STOPPED, > > > > > > > > > > > tx.state()); > > > > > > > > > > > > > > > > > > ts.txStart(tx); > > > > > > > > > > > > > > > > > > > > > > Assert.assertEquals(TransactionState.ACTIVE, > > > > > > > > > > > tx.state()); > > > > > > > > > > > > > > > > > > cache.put("key3", 3); > > > > > > > > > > > > > > > > > > > > Assert.assertTrue(cache.remove("key2")); > > > > > > > > > > > > > > > > > > tx.commit(); > > > > > > > > > > > > > > > > > > return true; > > > > > > > > > > > > > > > > > > }); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > fut.get(); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Assert.assertEquals( > > > TransactionState.COMMITTED, > > > > > > > > > > tx.state()); > > > > > > > > > > > > > > > > > > Assert.assertEquals((long)1, > > > > > > > (long)cache.get("key1")); > > > > > > > > > > > > > > > > > > Assert.assertEquals((long)3, > > > > > > > (long)cache.get("key3")); > > > > > > > > > > > > > > > > > > Assert.assertFalse(cache. > > > containsKey("key2")); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In method *ts.txStart(...)* we just > rebind > > > *tx* > > > > > to > > > > > > > > > current > > > > > > > > > > > > > thread: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > public void txStart(Transaction tx) { > > > > > > > > > > > > > > > > > > TransactionProxyImpl > transactionProxy = > > > > > > > > > > > > > > (TransactionProxyImpl)tx; > > > > > > > > > > > > > > > > > > cctx.tm().reopenTx( > > > transactionProxy.tx()); > > > > > > > > > > > > > > > > > > transactionProxy. > bindToCurrentThread(); > > > > > > > > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In method *reopenTx* we alter *threadMap* > > so > > > > that > > > > > > it > > > > > > > > > binds > > > > > > > > > > > > > > > transaction > > > > > > > > > > > > > > > > > > to current thread. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > How do u think about it ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > вт, 7 мар. 2017 г. в 22:38, Denis Magda < > > > > > > > > > [email protected] > > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Alexey, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please share the rational behind this > and > > > the > > > > > > > > thoughts, > > > > > > > > > > > > design > > > > > > > > > > > > > > > ideas > > > > > > > > > > > > > > > > > you > > > > > > > > > > > > > > > > > > > have in mind. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > — > > > > > > > > > > > > > > > > > > > Denis > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mar 7, 2017, at 3:19 AM, ALEKSEY > > > > > KUZNETSOV < > > > > > > > > > > > > > > > > > > [email protected]> > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi all! Im designing distributed > > > > transaction > > > > > > > which > > > > > > > > > can > > > > > > > > > > be > > > > > > > > > > > > > > started > > > > > > > > > > > > > > > > at > > > > > > > > > > > > > > > > > > one > > > > > > > > > > > > > > > > > > > > node, and continued at other one. Has > > > > anybody > > > > > > > > > thoughts > > > > > > > > > > on > > > > > > > > > > > > it > > > > > > > > > > > > > ? > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *Best Regards,* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *Kuznetsov Aleksey* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *Best Regards,* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *Kuznetsov Aleksey* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *Best Regards,* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *Kuznetsov Aleksey* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > > > > > *Best Regards,* > > > > > > > > > > > > > > > > > > > > > > > > > > > > *Kuznetsov Aleksey* > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > > > *Best Regards,* > > > > > > > > > > > > > > > > > > > > > > > > > > *Kuznetsov Aleksey* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > *Best Regards,* > > > > > > > > > > > > > > > > > > > > > > *Kuznetsov Aleksey* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > *Best Regards,* > > > > > > > > > > > > > > > > > > *Kuznetsov Aleksey* > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > *Best Regards,* > > > > > > > > > > > > > > *Kuznetsov Aleksey* > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > *Best Regards,* > > > > > > > > > > *Kuznetsov Aleksey* > > > > > > > > > > > > -- > > > > > > *Best Regards,* > > > > > > *Kuznetsov Aleksey* > > > > > > -- > > *Best Regards,* > > *Kuznetsov Aleksey* >
