Re: Samza hung after bootstrapping

2015-06-21 Thread Roger Hoover
Thanks, Yan. I'll give it a try. Sent from my iPhone > On Jun 21, 2015, at 10:02 PM, Yan Fang wrote: > > Hi Roger, > > I will try to look at the issue tomorrow if my time allows. > > First thing first: > > The build has some unexpected results. A quick fix: > > 1. apply https://issues.apac

Re: Samza hung after bootstrapping

2015-06-21 Thread Yan Fang
Hi Roger, I will try to look at the issue tomorrow if my time allows. First thing first: The build has some unexpected results. A quick fix: 1. apply https://issues.apache.org/jira/browse/SAMZA-712 2. add sourceSets.main.scala.srcDir "src/main/java" sourceSets.main.java.srcDirs = [] at line 1

Re: Samza hung after bootstrapping

2015-06-21 Thread Roger Hoover
Was looking through the code a little and it looks like the BootstrappingChooser could use the list of SSPs passed into it's register() method to figure out which partitions it need to monitor. I wanted to try to build Samza to play around with it but I'm getting error trying to build off of both

Re: Samza hung after bootstrapping

2015-06-21 Thread Roger Hoover
I think I see what's happening. When there are 8 tasks and I set yarn.container.count=8, then each container is responsible for a single task. However, the systemStreamLagCounts map ( https://github.com/apache/samza/blob/0.9.0/samza-core/src/main/scala/org/apache/samza/system/chooser/Bootstrappin

Re: Samza hung after bootstrapping

2015-06-21 Thread Roger Hoover
Hi Yan, I've uploaded a file with TRACE level logging here: http://filebin.ca/261yhsTZcZQZ/samza-container-0.log.gz I really appreciate your help as this is a critical issue for me. Thanks, Roger On Fri, Jun 19, 2015 at 12:05 PM, Yan Fang wrote: > Hi Roger, > > " but it only spawns one conta

Re: Samza hung after bootstrapping

2015-06-20 Thread Roger Hoover
Thank you, Yan. I'll get a trace level log as soon as I can. Sent from my iPhone > On Jun 19, 2015, at 12:05 PM, Yan Fang wrote: > > Hi Roger, > > " but it only spawns one container and still hangs after bootstrap" >-- this probably is due to your local machine does not have enough > reso

Re: Samza hung after bootstrapping

2015-06-19 Thread Yan Fang
Hi Roger, " but it only spawns one container and still hangs after bootstrap" -- this probably is due to your local machine does not have enough resource for the second container. Because I checked your log file, each container is about 4GB. "When I run it on our YARN cluster with a single co

Samza hung after bootstrapping

2015-06-18 Thread Roger Hoover
I need some help. I have a job which bootstraps one stream and then is supposed to read from two. When I run it on our YARN cluster with a single container, it works correctly. When I tried it with 5 containers, it gets hung after consuming the bootstrap topic. I ran it with the grid script on