Hi Shimin, It can be guaranteed to be an atomic operation. This is ensured by the RPC framework. You could take a look at RpcEndpoint for more details.
Regards, Dian > 在 2019年9月6日，下午2:35，shimin yang <ysmcl...@gmail.com> 写道： > > Hi Fu, > > Thank you for the remind. I think it would work in my case as long as it's > an atomic operation. > > Dian Fu <dian0511...@gmail.com> 于2019年9月6日周五 下午2:22写道： > >> Hi Jingsong, >> >> Thanks for bring up this discussion. You can try to look at the >> GlobalAggregateManager to see if it can meet your requirements. It can be >> got via StreamingRuntimeContext#getGlobalAggregateManager(). >> >> Regards, >> Dian >> >>> 在 2019年9月6日，下午1:39，shimin yang <ysmcl...@gmail.com> 写道： >>> >>> Hi Jingsong, >>> >>> Big fan of this idea. We faced the same problem and resolved by adding a >>> distributed lock. It would be nice to have this feature in JobMaster, >> which >>> can replace the lock. >>> >>> Best, >>> Shimin >>> >>> JingsongLee <lzljs3620...@aliyun.com.invalid> 于2019年9月6日周五 下午12:20写道： >>> >>>> Hi devs: >>>> >>>> I try to implement streaming file sink for table[1] like >> StreamingFileSink. >>>> If the underlying is a HiveFormat, or a format that updates visibility >>>> through a metaStore, I have to update the metaStore in the >>>> notifyCheckpointComplete, but this operation occurs on the task side, >>>> which will lead to distributed access to the metaStore, which will >>>> lead to bottleneck. >>>> >>>> So I'm curious if we can support notifyOnMaster for >>>> notifyCheckpointComplete like FinalizeOnMaster. >>>> >>>> What do you think? >>>> >>>> [1] >>>> >> https://docs.google.com/document/d/15R3vZ1R_pAHcvJkRx_CWleXgl08WL3k_ZpnWSdzP7GY/edit?usp=sharing >>>> >>>> Best, >>>> Jingsong Lee >> >>
