Re: [saga-actuator]consider about adding StreamBasedSaga engine to integrate with shardingsphere

[Willem Jiang]

We could do some enhancement on it.
>From my understanding, We could create a calling graph which has three
sub graph of it, or we could let the SQL executor running the command
one by one.
Any thoughts?

Willem Jiang

Twitter: willemjiang
Weibo: 姜宁willem

On Wed, May 1, 2019 at 4:29 PM zhaojun  wrote:
>
> But saga-actuator don’t support 3 logic SQL is a global transaction while you 
> submit 3 times separately.
>
> --
> Zhao Jun
> Apache Sharding-Sphere & ServiceComb
>
> > On May 1, 2019, at 8:38 AM, Willem Jiang  wrote:
> >
> > Willem Jiang
> >
> > Twitter: willemjiang
> > Weibo: 姜宁willem
> >
> > On Mon, Apr 29, 2019 at 7:58 PM zhaojun  wrote:
> >>
> >> Hi, Willem
> >>
> >> Saga actuator could not compatible with ShardingSphere currently,  there 
> >> are 2 main problem exist.
> >> 1. If sql execute in saga transport instead of ShardingSphere, we can not 
> >> see the result of logic-sql even in one transaction, it is like this:
> >>update t_order set status=’start’ where usr_id=’tom’;
> >>select status from t_order where usr_id=’tom’;  
> >> —> we can’t query ’start’ record as actuator haven’t executed.
> >>Insert into t_order values(?,?,?) ;
> >
> > I think we can break the whole SQL interactions into smaller steps,
> > and let saga-actuator do the job one by one.
> > In this case, you can break the whole calling graph into several sub
> > calling graphs.
> > Any thought?
> >
> >> 2. If logic-sql execute in ShardingSphere, we cannot handle recovery 
> >> before submit graph result, as event log only wrote at saga engine.
> >>
> >> so we should integrate saga-transaction like omega send event log step by 
> >> step.
> >> It is better to make every instance do recovery Independently, instead of 
> >> providing another coordinator center service.
> >> I feel that embed saga should have following  capability.
> >>  1).It can provide service in jar package independent
> >>  2).each embed saga only recovery their own transaction-data of this 
> >> instance.
> >>   If one instance crashed, we can introduce external service to do 
> >> failover.
> >>
> >> so we consider about exending saga-acutator, if it can support submit task 
> >> step by step in one transaction, it is a good choice.
> >> of course, there have many things we should do.
> >>
> >> --
> >> Zhao Jun
> >> Apache Sharding-Sphere & ServiceComb
> >>
> >>> On Apr 29, 2019, at 5:17 PM, Willem Jiang  wrote:
> >>>
> >>> First Saga actuator need to build up the calling grapha before sending
> >>> out the request.  I don't think you can do the step by step SQL
> >>> invocation with Saga actuator.
> >>> If you want to call the SQL execution step by step , you may need to
> >>> switch to ServiceComb Pack project which has a coordinator to take
> >>> care of the distributed transaction. But that introduce another
> >>> endpoint(Alpha) to shardingsphere.
> >>>
> >>> From my understanding, Saga actuator is most efficient way to execute
> >>> the SQL across different data nodes.
> >>>
> >>>
> >>> Willem Jiang
> >>>
> >>> Twitter: willemjiang
> >>> Weibo: 姜宁willem
> >>>
> >>> On Fri, Apr 26, 2019 at 6:41 PM zhaojun  wrote:
> 
>  Hi, all
> 
>  currently, we have integrated with saga using graph based engine in 
>  shardingsphere[1]
>  it need us to collect all participated actual SQL, then submit to saga 
>  actuator in commit/rollback phase.
>  if application crashed before invoking saga actuator, undo log of branch 
>  transaction SQL will not be saved,
>  so recovery thread will not be executed correctly.
> 
>  it's better that encapsulating every actual SQL as a saga task in 
>  shardingsphere side,
>  then submit to saga actuator realtime instead of batch processing all 
>  the SQLs at commit/rollback phase.
>  this architecture will make the boundary more clear between 
>  shardingsphre and saga, currently we have done some additional works for 
>  integrating saga.
> 
>  any thought?
> 
>  [1]:https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
>   
>  
> 
> 
> 
>  --
>  Zhao Jun
>  Apache Sharding-Sphere & ServiceComb
> 
> >>
>

Re: [saga-actuator]consider about adding StreamBasedSaga engine to integrate with shardingsphere

[zhaojun]

But saga-actuator don’t support 3 logic SQL is a global transaction while you 
submit 3 times separately.

--
Zhao Jun
Apache Sharding-Sphere & ServiceComb

> On May 1, 2019, at 8:38 AM, Willem Jiang  wrote:
> 
> Willem Jiang
> 
> Twitter: willemjiang
> Weibo: 姜宁willem
> 
> On Mon, Apr 29, 2019 at 7:58 PM zhaojun  wrote:
>> 
>> Hi, Willem
>> 
>> Saga actuator could not compatible with ShardingSphere currently,  there are 
>> 2 main problem exist.
>> 1. If sql execute in saga transport instead of ShardingSphere, we can not 
>> see the result of logic-sql even in one transaction, it is like this:
>>update t_order set status=’start’ where usr_id=’tom’;
>>select status from t_order where usr_id=’tom’;  
>> —> we can’t query ’start’ record as actuator haven’t executed.
>>Insert into t_order values(?,?,?) ;
> 
> I think we can break the whole SQL interactions into smaller steps,
> and let saga-actuator do the job one by one.
> In this case, you can break the whole calling graph into several sub
> calling graphs.
> Any thought?
> 
>> 2. If logic-sql execute in ShardingSphere, we cannot handle recovery before 
>> submit graph result, as event log only wrote at saga engine.
>> 
>> so we should integrate saga-transaction like omega send event log step by 
>> step.
>> It is better to make every instance do recovery Independently, instead of 
>> providing another coordinator center service.
>> I feel that embed saga should have following  capability.
>>  1).It can provide service in jar package independent
>>  2).each embed saga only recovery their own transaction-data of this 
>> instance.
>>   If one instance crashed, we can introduce external service to do 
>> failover.
>> 
>> so we consider about exending saga-acutator, if it can support submit task 
>> step by step in one transaction, it is a good choice.
>> of course, there have many things we should do.
>> 
>> --
>> Zhao Jun
>> Apache Sharding-Sphere & ServiceComb
>> 
>>> On Apr 29, 2019, at 5:17 PM, Willem Jiang  wrote:
>>> 
>>> First Saga actuator need to build up the calling grapha before sending
>>> out the request.  I don't think you can do the step by step SQL
>>> invocation with Saga actuator.
>>> If you want to call the SQL execution step by step , you may need to
>>> switch to ServiceComb Pack project which has a coordinator to take
>>> care of the distributed transaction. But that introduce another
>>> endpoint(Alpha) to shardingsphere.
>>> 
>>> From my understanding, Saga actuator is most efficient way to execute
>>> the SQL across different data nodes.
>>> 
>>> 
>>> Willem Jiang
>>> 
>>> Twitter: willemjiang
>>> Weibo: 姜宁willem
>>> 
>>> On Fri, Apr 26, 2019 at 6:41 PM zhaojun  wrote:
 
 Hi, all
 
 currently, we have integrated with saga using graph based engine in 
 shardingsphere[1]
 it need us to collect all participated actual SQL, then submit to saga 
 actuator in commit/rollback phase.
 if application crashed before invoking saga actuator, undo log of branch 
 transaction SQL will not be saved,
 so recovery thread will not be executed correctly.
 
 it's better that encapsulating every actual SQL as a saga task in 
 shardingsphere side,
 then submit to saga actuator realtime instead of batch processing all the 
 SQLs at commit/rollback phase.
 this architecture will make the boundary more clear between shardingsphre 
 and saga, currently we have done some additional works for integrating 
 saga.
 
 any thought?
 
 [1]:https://github.com/sharding-sphere/shardingsphere-spi-impl/tree/master/sharding-transaction-spi-impl/sharding-transaction-base-spi-impl/sharding-transaction-base-saga
  
 
 
 
 
 --
 Zhao Jun
 Apache Sharding-Sphere & ServiceComb
 
>>