Hi Lukas, We use similar docker based deployment. Would be glad if you could post here if ThreadJobFactory worked well for you.
Thanks, Michael On Sat, May 30, 2015 at 12:22 AM, Lukas Steiblys <lu...@doubledutch.me> wrote: > Yes, I think switching to ThreadJobFactory is a good solution. I think the > reasons why I switched to ProcessJobFactory earlier no longer hold true. > > Thanks. > > Lukas > > -----Original Message----- From: Yi Pan > Sent: Friday, May 29, 2015 2:05 PM > > To: dev@samza.apache.org > Subject: Re: ProcessJobFactory parent process > > Hi, Lukas, > > Yes. That's exactly part of the feature to allow > health-check/failure-detection of containers. Another short-term option is > trying to use ThreadJobFactory, which has the JobCoordinator and containers > in the same process. Does that work for your use case? > > -Yi > > On Fri, May 29, 2015 at 12:59 PM, Lukas Steiblys <lu...@doubledutch.me> > wrote: > > Yes, I'm talking about the child process crashing. I'd like the parent to >> die as well if the child crashes so Docker can understand that the process >> failed and restart the container. >> >> Lukas >> >> -----Original Message----- From: Yi Pan >> Sent: Friday, May 29, 2015 12:47 PM >> To: dev@samza.apache.org >> Subject: Re: ProcessJobFactory parent process >> >> >> Hi, Lukas, >> >> I assume that when you say "the job crashes", you were referring to the >> child process running the container, not the parent process? If yes, we >> were actually talking about adding container >> health-check/failure-detection >> in the JobCoordinator. SAMZA-680 would be the good place to start these >> kind of discussion. >> >> Thanks! >> >> -Yi >> >> On Fri, May 29, 2015 at 8:39 AM, Lukas Steiblys <lu...@doubledutch.me> >> wrote: >> >> Hi Yan, >> >>> >>> The memory usage is not very high, but I'm trying to cut the usage any >>> way >>> I can. >>> >>> The bigger problem is when the job crashes and the parent process stays >>> active preventing an auto restart by the Docker supervisor. >>> >>> Lukas >>> >>> On Thursday, May 28, 2015, Yan Fang <yanfang...@gmail.com> wrote: >>> >>> > Hi Lukas, >>> > >>> > The parent process is used to manage the lifecycle of the actual >>> process. I >>> > am curious how much memory the parent process takes? >>> > >>> > Thanks, >>> > >>> > Fang, Yan >>> > yanfang...@gmail.com <javascript:;> >>> > >>> > On Thu, May 28, 2015 at 2:30 PM, Lukas Steiblys <lu...@doubledutch.me >>> > <javascript:;>> >>> > wrote: >>> > >>> > > Hello, >>> > > >>> > > I’m running Samza tasks using ProcessJobFactory and after I start the >>> > job, >>> > > the initial process spawns a new process that is the actual process >>> where >>> > > the code is run. The problem is that the parent process stays active >>> even >>> > > after the job is started and that messes with the way I deploy Samza >>> (in >>> > > Docker containers) and consumes memory while not doing anything. >>> > > >>> > > My question: is it possible to kill the parent process while still >>> > leaving >>> > > the Samza tasks to process messages? >>> > > >>> > > Lukas >>> > >>> >>> >>> >> >
