Proposal of Alternatively Transferring State via Disk

2018-07-03 Thread Neng Lu
Hi all, During our test at twitter, when a state is very large (GBs), the current checkpointing mechanism doesn't work well. Here's a proposal of transferring large checkpointing state through local disk to solve the problem we've found.

Re: Heron OSS Sync

2018-07-03 Thread Ning Wang
Today's brief notes: - We are testing DSL API in Twitter. Jerry mentioned some potential improvement on builder. - Found a Dhalion issue (2927). - Shipped new integration tests for topology structures - The current model in the queuing model (M/M/1) might not be useful. Trying new ones. - Added

Heron OSS Sync

2018-07-03 Thread Ning Wang
Hi, Sorry for the late notice The heron OSS sync meeting will be happening today at 2.00 pm PDT. Please use the following hangout link: https://hangouts.google.com/hangouts/_/streaml.io/oss-heron-sync?authuser=0 See you all then.