Re: Review Request 47835: SAMZA-914: Initial draft for Java programming APIs on operators supporting DAGs

2016-09-01 Thread Jake Maes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47835/#review147607 --- A few more comments. This is all for now.

Re: Review Request 51142: SAMZA-967: HDFS System Consumer

2016-09-01 Thread Navina Ramesh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51142/#review147596 --- @lhaiesp: Your patch looks awesome. Happy to review again once

Re: Review Request 51346: SAMZA-974 - Support finite datasources in Samza that have a notion of End-Of-Stream

2016-09-01 Thread Chris Pettitt
> On Aug. 26, 2016, 7:51 p.m., Chris Pettitt wrote: > > samza-api/src/main/java/org/apache/samza/system/IncomingMessageEnvelope.java, > > line 31 > > > > > > How likely are we to collide with this? Thats the problem

Periodic cleanup of unused local stores

2016-09-01 Thread santhosh venkat
Currently in Samza, to enable reuse of local store between restarts, local store is persisted outside of the YARN’s working directory. However, there is no mechanism currently available to periodically clean up the unused local stores. Here is a proposal detailing a possible way to accomplish

Re: Review Request 47835: SAMZA-914: Initial draft for Java programming APIs on operators supporting DAGs

2016-09-01 Thread Jake Maes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47835/#review147578 ---

Re: Review Request 51516: SAMZA-702: Document the significance of all the different metrics emitted by Samza out of the box

2016-09-01 Thread Jagadish Venkatraman
> On Sept. 1, 2016, 1:50 a.m., Navina Ramesh wrote: > > Overall, the patch looks pretty good! The only roadblock I see is that with > > SAMZA-680, we introduced "ContainerProcessManagerMetrics" which would > > replace "SamzaAppMasterMetrics". This is kind of problematic because it > > seems

Re: [DISCUSS] End of Stream Feature in Samza

2016-09-01 Thread Renato Marroquín Mogrovejo
Thanks Jagadish! This is great! Can you share some thoughts/opinions on how this feature relates on using punctuations (at some point) in Samza? I mean do you think that using punctuated streams could be seen as a generalization of this problem? And if so, could this feature be used later on as a