Re: Review Request 51633: SAMZA-1013: Add YARN Node label support

2016-10-06 Thread Jagadish Venkatraman
> On Oct. 5, 2016, 6:18 p.m., Jagadish Venkatraman wrote: > > Overall, the patch looks great! This is exciting given that Samza can > > support scheduling based on tags. For example, jobs with rocksdb can be > > assigned to nodes with SSDs. > > > > > > Can you please add some detail on

Re: Review Request 51633: SAMZA-1013: Add YARN Node label support

2016-10-06 Thread Maxim Logvinenko
> On Oct. 5, 2016, 6:18 p.m., Jagadish Venkatraman wrote: > > samza-yarn/src/main/java/org/apache/samza/job/yarn/YarnClusterResourceManager.java, > > line 187 > > > > > > nit: containerLabel so that we distinguish

Re: Review Request 51633: SAMZA-1013: Add YARN Node label support

2016-10-06 Thread Maxim Logvinenko
> On Oct. 5, 2016, 6:18 p.m., Jagadish Venkatraman wrote: > > Overall, the patch looks great! This is exciting given that Samza can > > support scheduling based on tags. For example, jobs with rocksdb can be > > assigned to nodes with SSDs. > > > > > > Can you please add some detail on

Re: Review Request 51633: SAMZA-1013: Add YARN Node label support

2016-10-06 Thread Maxim Logvinenko
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51633/ --- (Updated Oct. 7, 2016, 12:08 a.m.) Review request for samza. Changes ---

Re: RecordTooLargeException recovery

2016-10-06 Thread David Yu
Xinyu, Thanks for the answers. Those suggestions are helpful as well. David On Thu, Oct 6, 2016 at 12:48 PM xinyu liu wrote: > Hi, David, > > For your questions: > > 1) In this case Samza recovered but the changelog message was lost. In > 0.10.1 KafkaSystemProducer has

[VOTE] Apache Samza 0.11.0 RC1

2016-10-06 Thread xinyu liu
Hey all, This is a call for a vote on a release of Apache Samza 0.11.0. Thanks to everyone who has contributed to this release. We are very glad to see some new contributors in this release. Note: this release candidate fixed the version issue in integration test previously found in 0.11.0 RC0.

Re: RecordTooLargeException recovery

2016-10-06 Thread xinyu liu
Hi, David, For your questions: 1) In this case Samza recovered but the changelog message was lost. In 0.10.1 KafkaSystemProducer has a race condition: there is small chance the later send success might override the previous failure. The bug is fixed in the upcoming 0.11.0 release (SAMZA-1019).

RecordTooLargeException recovery

2016-10-06 Thread David Yu
Hi, Our Samza job (0.10.1) throws RecordTooLargeExceptions when flushing the KV store change to the changelog topic, as well as sending outputs to Kafka. We have two questions to this problem: 1. It seems that after the affected containers failed multiple times, the job was able to recover and

Re: Review for SAMZA-1012

2016-10-06 Thread Tommy Becker
Thanks! On 10/04/2016 11:18 AM, Jacob Maes wrote: I'm on it! On Tue, Oct 4, 2016 at 6:22 AM, Tommy Becker wrote: Could I bother a committer for a review of this patch? https://reviews.apache.org/r/51689/ -- Tommy Becker Senior Software