Re: Slides of exactly once/effectively once processing in Heron

2018-03-20 Thread Karthik Ramasamy
Thanks Sree. On Mon, Mar 19, 2018 at 7:22 AM, Sree V wrote: > Great work, Karthik & Sanjeev. > > Thanks > Sree > > On Saturday, March 10, 2018, 11:58:21 AM PST, Karthik Ramasamy < > kart...@streaml.io> wrote: > > Thanks Dave - more is on the way! > > cheers

Re: Heron OSS Sync

2018-03-20 Thread Fu Maosong
Hi Dave - I added Ning Wang from Twitter with Admin permission and he has started working on it. 2018-03-16 10:31 GMT-07:00 Dave Fisher : > Hi Maosong - > > Infra is waiting. > > Daniel Takamori commented on INFRA-16117: > - > >

Re: Fault tolerance

2018-03-20 Thread Ning Wang
Two features I can think of: Dhalion https://blog.acolyer.org/2017/06/30/dhalion-self-regulating-stream-processing-in-heron/ Backpressure Streammanager is responsible for all data transactions and it monitors the data consumption. If a process runs slow and incoming data buffer is full, it

Fault tolerance

2018-03-20 Thread Thuvarakan Tharmarajasingam
Hello, I am currently working on a distributed fault tolerance system for big data toolkit. Could you please give me an overview of how Heron achieved the fault tolerance?

Re: Heron OSS Sync

2018-03-20 Thread Dave Fisher
Hi Ning Wang, You can reach out to Infra on their HipChat at https://www.hipchat.com/gh6DJhKrN Regards, Dave > On Mar 20, 2018, at 2:55 PM, Fu Maosong wrote: > > Hi Dave - > > I added Ning Wang from Twitter with Admin permission and