I think Log4j includes a Syslog appender—the log4j config included with Flink just logs to the logs/ dir, but you should just be able to modify it (log4j.properties) to suit your needs.
-- Patrick Lucas On Thu, Mar 30, 2017 at 2:39 PM, Chakravarthy varaga < chakravarth...@gmail.com> wrote: > Hi, > > With regards to logging (both Flink & application specific logs) > within the container, are there best practices that you know of to get the > logs to a centralized locations.. > For e.g. the flink TM's log are local inside the container and I don't > wish to write to shared/mounted volumes, this means that I have to run > separate daemon running inside the container along with the TM to transport > these logs to another server or so... > + I don't see that flink provides support for syslog to be able to > connect to rsyslog etc., > > Can you please advice a way to go here? > > Best Regards > CVP > > On Wed, Mar 29, 2017 at 11:33 AM, Patrick Lucas <patr...@data-artisans.com > > wrote: > >> For 1., I think the standard approach would be to specify from without >> what the heap size should be. If you want an *x* MB heap, you could set >> your container memory limit to 1.3 * *x* or so (to account for overhead) >> and set taskmanager.heap.mb: *x* in your config. >> >> The other way around—e.g. from inside the container determine its memory >> limit and divide it by 1.3—sounds interesting though, so please share if >> you have success with that. >> >> For 2. I don't think there's really a good way yet to monitor the health >> of containerized jobs directly, so probably your best bet is to watch the >> job's metrics from outside the Flink cluster. >> >> -- >> Patrick Lucas >> >> On Wed, Mar 29, 2017 at 10:58 AM, Chakravarthy varaga < >> chakravarth...@gmail.com> wrote: >> >>> Hi, >>> >>> Any updates here? I'm sure many would have faced similar issues like >>> these, any help here is highly appreciated. >>> >>> Best Regards >>> CVP >>> >>> On Tue, Mar 28, 2017 at 5:47 PM, Chakravarthy varaga < >>> chakravarth...@gmail.com> wrote: >>> >>>> Hi Team, >>>> >>>> If the flink cluster is conainerized and managed through by a >>>> container orchestrator, >>>> >>>> 1. the orchestrator allocates resources for each JM. TM etc., say >>>> if the container (TM) needs to run with 2G RAM, how should this allocation >>>> be honoured by the TM when its JVM starts. I'm thinking of wrapping up a >>>> script that determines the resource allocation for the container and writes >>>> the flink-conf.yaml before the TM starts the process. Is this the way to >>>> go? >>>> >>>> 2. The container orchestrator looks at health of the containers and >>>> is however unaware of the job health status runnning inside the >>>> container/cluster. How should this be determined? >>>> >>>> Best Regards >>>> CVP >>>> >>>> >>>> >>> >> >
