Alexey, Every job is executed in transaction having same semantics as any other transaction, but transaction is managed by Ignite transparently.
Job's transaction is rolled back on throwing any unchecked exception or special TransactionRollbackException. Concurrency issues between jobs are solved using entry locking or currency primitives. I think the closest similar thing is JTA transaction enlisting. Maybe I even'll implement such logic using current Ignite JTA module, haven't tried yet. But I don't like JTA overhead and a requirement to have JTA coordinator. 2017-01-27 13:40 GMT+03:00 Alexey Goncharuk <alexey.goncha...@gmail.com>: > Alexei, > > Can you describe the semantics of the task in greater detail? > * What if two jobs on different nodes access and try to put the same key? > (this can be resolved by allowing a job only access local primary keys) > This is prefectly valid. Second transaction'll wait for completion of first. > * How do you define the lock acquisition order and prevent deadlocks? I > assume that running such a compute task will involve a lot of keys, so it > is quite unlikely that OPTIMISTIC SERIALIZABLE transactions are applicable > here > Same as any other transaction - user is responsible for avoiding deadlock. > * Currently a started transaction locks the cache topology, so job > failover in the case of node crash will be very hard to implement > Could you provide more details? Current job transactions must be rolled back all together and task must be routed to other node. > > 2017-01-26 23:36 GMT+03:00 Alexei Scherbakov <alexey.scherbak...@gmail.com > >: > > > Guys, > > > > How difficult would be to support transactional tasks? > > > > What means every job in task executed in it's own transaction. > > > > In case of single job failure or reduce phase failure all transaction > > started by jobs are rolled back. > > > > Only if all jobs are successfully executed, corresponding transactions > are > > commited. > > > > Also it would be very desirable to implement tasks failover in the > similar > > way how jobs failover is implemented. > > > > In case of master's failure jobs are rolled back, and task is restarted > on > > another node. > > > > This should greatly simplify implementing complex business processes. > > > > -- > > > > Best regards, > > Alexei Scherbakov > > > -- Best regards, Alexei Scherbakov
