Proxy Client - Batch Ordering / Commit

2016-10-04 Thread Cameron Hatfield
I have a question about the Proxy Client. Basically, for our use cases, we want to guarantee ordering at the key level, irrespective of the ordering of the partition it may be assigned to as a whole. Due to the source of the data (HBase Replication), we cannot guarantee that a single partition will

Re: Proxy Client - Batch Ordering / Commit

2016-10-05 Thread Cameron Hatfield
would be the relevant places to add the ability > > to do this, and would the project be interested in a pull request? > > > Does the example linked above meet your requirements? > > Thx > > On Tue, Oct 4, 2016 at 12:39 PM, Cameron Hatfield > wrote: > > > I have

Re: Proxy Client - Batch Ordering / Commit

2016-10-05 Thread Cameron Hatfield
ate. > > In failure scenarios you would need to recover by reading the latest id, > and retry based on this. > > Would that work? > > On Wed, Oct 5, 2016 at 2:02 PM, Cameron Hatfield wrote: > > > Main question I have then is what happens in the case of retry due to >

Re: Proxy Client - Batch Ordering / Commit

2016-10-14 Thread Cameron Hatfield
> > at it can before throwing exceptions to users. The code you are looking > for > > - it is on BKLogSegmentWriter for the proxy handling writes and it is on > > DistributedLogClientImpl for the proxy client handling responses from > > proxies. Does this help you? >

Re: Proxy Client - Batch Ordering / Commit

2016-10-17 Thread Cameron Hatfield
he core concern here > is almost 'exact-once' write (or a way to explicit tell if a write can be > retried or not). > > Comments inline. > > On Fri, Oct 14, 2016 at 11:17 AM, Cameron Hatfield > wrote: > > > > Ah- yes good point (to be clear we're not

Re: Proxy Client - Batch Ordering / Commit

2016-10-25 Thread Cameron Hatfield
he process of getting a new epoch. -Cameron On Wed, Oct 19, 2016 at 7:43 PM, Sijie Guo wrote: > On Wed, Oct 19, 2016 at 7:17 PM, Sijie Guo wrote: > > > > > > > On Monday, October 17, 2016, Cameron Hatfield wrote: > > > >> Answer inline: > >

Re: Proxy Client - Batch Ordering / Commit

2016-11-13 Thread Cameron Hatfield
> > proposed idea here. > > > > > > > > > > > > On Wed, Oct 26, 2016 at 4:53 PM, Sijie Guo > > > > wrote: > > > > > > > On Tue, Oct 25, 2016 at 11:30 AM, Cameron Hatfield > > > > > wrote: > > > &

Re: Proxy Client - Batch Ordering / Commit

2016-11-16 Thread Cameron Hatfield
on On Tue, Nov 15, 2016 at 2:01 AM, Xi Liu wrote: > Thank you, Cameron. Look forward to your comments. > > - Xi > > On Sun, Nov 13, 2016 at 1:21 PM, Cameron Hatfield > wrote: > > > Sorry, I've been on vacation for the past week, and heads down for a > >

Re: Proxy Client - Batch Ordering / Commit

2016-11-16 Thread Cameron Hatfield
Also, would it be possible for me to get wiki access so I will be able to update it / etc? -Cameron On Wed, Nov 16, 2016 at 11:59 AM, Cameron Hatfield wrote: > "A couple of questions" is what I originally wrote, and then the following > happened. Sorry about the large swath

Re: Proxy Client - Batch Ordering / Commit

2016-11-16 Thread Cameron Hatfield
016 at 12:11 PM, Cameron Hatfield > wrote: > > > Also, would it be possible for me to get wiki access so I will be able to > > update it / etc? > > > > -Cameron > > > > On Wed, Nov 16, 2016 at 11:59 AM, Cameron Hatfield > > wrote: > > > &