Re: Mesos logging

[Vinod Kone]

Did you figure this out? AFAICT, the LOG(INFO) line should be printed in
agent logs. What agent flags are you using?

On Tue, Aug 9, 2016 at 8:19 AM, Hendrik Haddorp 
wrote:

> I saw a few "Running ..." log entries from the docker support code but
> they seem to be all from VLOG(1) calls while for some reason the code
> that does the actual "docker run" call uses LOG(INFO) and that does not
> seem to come out by default, or I don't see it. But I'll try on ...
>
> On 09/08/16 11:38, haosdent wrote:
> > Hi, @Hendrik You could see INFO log when running Mesos Agent in
> > default level. Some docker run logs may exist in the stdout/stderr of
> > executor.
> >
> > On Tue, Aug 9, 2016 at 12:27 PM, Hendrik Haddorp
> > > wrote:
> >
> > I would like to see the "docker run" trace from
> > https://github.com/apache/mesos/blob/master/src/docker/docker.cpp
> > 
> > line 811.
> > What verbosity level does INFO map to?
> >
> > On 09/08/16 05:06, Charles Allen wrote:
> > > Which glog are you trying to capture? You can set the verbosity
> > level
> > > with the environment variable GLOG_v
> > >
> > > And you can also set it through things like Spark. So if you want a
> > > lot of ZK chatter at the mesos level in your spark logs, add
> > >
> > > spark.executorEnv.GLOG_v=9
> > >
> > > to your spark context
> > >
> > > On Mon, Aug 8, 2016 at 2:53 PM Hendrik Haddorp
> > > 
> > >>
> > wrote:
> > >
> > > Hi,
> > >
> > > the Mesos code contains log statements using LOG(INFO) and
> > > VLOG(1), for
> > > example. So far I found that Mesos is using the Google Logging
> > > Library.
> > > Looking in the logs I only seem to be able to find output
> > from VLOG
> > > statements. What do I need to do to get the output from the LOG
> > > statements? Where would I typically find the output? I'm
> > using CentOS
> > > 7.2 and found the output so far in the files below
> > /var/log/mesos.
> > >
> > > thanks,
> > > Hendrik
> > >
> >
> >
> >
> >
> > --
> > Best Regards,
> > Haosdent Huang
>
>

Re: Mesos logging

[Hendrik Haddorp]

I saw a few "Running ..." log entries from the docker support code but
they seem to be all from VLOG(1) calls while for some reason the code
that does the actual "docker run" call uses LOG(INFO) and that does not
seem to come out by default, or I don't see it. But I'll try on ...

On 09/08/16 11:38, haosdent wrote:
> Hi, @Hendrik You could see INFO log when running Mesos Agent in
> default level. Some docker run logs may exist in the stdout/stderr of
> executor.
>
> On Tue, Aug 9, 2016 at 12:27 PM, Hendrik Haddorp
> > wrote:
>
> I would like to see the "docker run" trace from
> https://github.com/apache/mesos/blob/master/src/docker/docker.cpp
> 
> line 811.
> What verbosity level does INFO map to?
>
> On 09/08/16 05:06, Charles Allen wrote:
> > Which glog are you trying to capture? You can set the verbosity
> level
> > with the environment variable GLOG_v
> >
> > And you can also set it through things like Spark. So if you want a
> > lot of ZK chatter at the mesos level in your spark logs, add
> >
> > spark.executorEnv.GLOG_v=9
> >
> > to your spark context
> >
> > On Mon, Aug 8, 2016 at 2:53 PM Hendrik Haddorp
> > 
> >>
> wrote:
> >
> > Hi,
> >
> > the Mesos code contains log statements using LOG(INFO) and
> > VLOG(1), for
> > example. So far I found that Mesos is using the Google Logging
> > Library.
> > Looking in the logs I only seem to be able to find output
> from VLOG
> > statements. What do I need to do to get the output from the LOG
> > statements? Where would I typically find the output? I'm
> using CentOS
> > 7.2 and found the output so far in the files below
> /var/log/mesos.
> >
> > thanks,
> > Hendrik
> >
>
>
>
>
> -- 
> Best Regards,
> Haosdent Huang

Re: Mesos logging

[haosdent]

Hi, @Hendrik You could see INFO log when running Mesos Agent in default
level. Some docker run logs may exist in the stdout/stderr of executor.

On Tue, Aug 9, 2016 at 12:27 PM, Hendrik Haddorp 
wrote:

> I would like to see the "docker run" trace from
> https://github.com/apache/mesos/blob/master/src/docker/docker.cpp line
> 811.
> What verbosity level does INFO map to?
>
> On 09/08/16 05:06, Charles Allen wrote:
> > Which glog are you trying to capture? You can set the verbosity level
> > with the environment variable GLOG_v
> >
> > And you can also set it through things like Spark. So if you want a
> > lot of ZK chatter at the mesos level in your spark logs, add
> >
> > spark.executorEnv.GLOG_v=9
> >
> > to your spark context
> >
> > On Mon, Aug 8, 2016 at 2:53 PM Hendrik Haddorp
> > > wrote:
> >
> > Hi,
> >
> > the Mesos code contains log statements using LOG(INFO) and
> > VLOG(1), for
> > example. So far I found that Mesos is using the Google Logging
> > Library.
> > Looking in the logs I only seem to be able to find output from VLOG
> > statements. What do I need to do to get the output from the LOG
> > statements? Where would I typically find the output? I'm using CentOS
> > 7.2 and found the output so far in the files below /var/log/mesos.
> >
> > thanks,
> > Hendrik
> >
>
>


-- 
Best Regards,
Haosdent Huang

Re: Mesos logging

[Hendrik Haddorp]

I would like to see the "docker run" trace from
https://github.com/apache/mesos/blob/master/src/docker/docker.cpp line 811.
What verbosity level does INFO map to?

On 09/08/16 05:06, Charles Allen wrote:
> Which glog are you trying to capture? You can set the verbosity level
> with the environment variable GLOG_v
>
> And you can also set it through things like Spark. So if you want a
> lot of ZK chatter at the mesos level in your spark logs, add 
>
> spark.executorEnv.GLOG_v=9
>
> to your spark context
>
> On Mon, Aug 8, 2016 at 2:53 PM Hendrik Haddorp
> > wrote:
>
> Hi,
>
> the Mesos code contains log statements using LOG(INFO) and
> VLOG(1), for
> example. So far I found that Mesos is using the Google Logging
> Library.
> Looking in the logs I only seem to be able to find output from VLOG
> statements. What do I need to do to get the output from the LOG
> statements? Where would I typically find the output? I'm using CentOS
> 7.2 and found the output so far in the files below /var/log/mesos.
>
> thanks,
> Hendrik
>

Re: Mesos logging

[Charles Allen]

Which glog are you trying to capture? You can set the verbosity level with
the environment variable GLOG_v

And you can also set it through things like Spark. So if you want a lot of
ZK chatter at the mesos level in your spark logs, add

spark.executorEnv.GLOG_v=9

to your spark context

On Mon, Aug 8, 2016 at 2:53 PM Hendrik Haddorp 
wrote:

> Hi,
>
> the Mesos code contains log statements using LOG(INFO) and VLOG(1), for
> example. So far I found that Mesos is using the Google Logging Library.
> Looking in the logs I only seem to be able to find output from VLOG
> statements. What do I need to do to get the output from the LOG
> statements? Where would I typically find the output? I'm using CentOS
> 7.2 and found the output so far in the files below /var/log/mesos.
>
> thanks,
> Hendrik
>

Mesos logging

[Hendrik Haddorp]

Hi,

the Mesos code contains log statements using LOG(INFO) and VLOG(1), for
example. So far I found that Mesos is using the Google Logging Library.
Looking in the logs I only seem to be able to find output from VLOG
statements. What do I need to do to get the output from the LOG
statements? Where would I typically find the output? I'm using CentOS
7.2 and found the output so far in the files below /var/log/mesos.

thanks,
Hendrik

Re: Mesos logging configuration questions

[Benjamin Mahler]

I'm afraid that document is out of date, please ignore the comments related
to MESOS_HOME in the interim of us fixing that document.
https://issues.apache.org/jira/browse/MESOS-934

--log_dir is a parameter of the mesos-master and mesos-slave binaries.

The directory you linked is to an executor sandbox, the note about
MESOS_HOME/work is no longer correct.


On Tue, Jan 14, 2014 at 2:48 PM, Jim Freeman ji...@sbcglobal.net wrote:

 Referring to
 http://mesos.apache.org/documentation/latest/logging-and-debugging/  :

 I have not set MESOS_HOME, so it should default per the doc.

 1) Doc says Mesos uses the Google Logging library and writes logs to
 MESOS_HOME/logs by default, where MESOS_HOME is the location where Mesos is
 installed.

 How do I determine where Mesos is installed?  The 'make' put Mesos files
 in various places.

 I do see mesos-log.* files directly in /tmp (not in a */logs directory)

 2) Doc says The log directory can be configured using the log_dir
 parameter.

 log_dir parameter to what?  'configured' is a web link that might tell me,
 but it links to a nonexistent web page.

 3) Doc says Frameworks that run on Mesos have their output stored to a
 work directory on each machine. By default, this is MESOS_HOME/work.

 Yet I only find stderr and stdout text files in paths like this (note
 there is no work component):


 /tmp/mesos/slaves/201401131343-16842879-5050-5210-0/frameworks/201401131343-16842879-5050-5210-/executors/default/runs/43bf587e-63df-4f26-b698-ab5c9e8ed385

Re: Mesos logging configuration questions

[Vinod Kone]

If --log_dir is not specified nothing is written to disk.

➜  build git:(master) ✗ ./bin/mesos-master.sh --help
...
...
  --log_dir=VALUELocation to put log files (no default, nothing
  is written to disk unless specified;
  does not affect logging to stderr)
...
...


@vinodkone


On Tue, Jan 21, 2014 at 10:59 AM, Jim Freeman jim.free...@oracle.comwrote:

 So if you don't supply --log_dir, then logs go in the /tmp executor
 sandbox?



 *From:* Benjamin Mahler [mailto:benjamin.mah...@gmail.com]
 *Sent:* Tuesday, January 21, 2014 10:45 AM
 *To:* user@mesos.apache.org; Jim Freeman
 *Subject:* Re: Mesos logging configuration questions



 I'm afraid that document is out of date, please ignore the comments
 related to MESOS_HOME in the interim of us fixing that document.

 https://issues.apache.org/jira/browse/MESOS-934



 --log_dir is a parameter of the mesos-master and mesos-slave binaries.



 The directory you linked is to an executor sandbox, the note about
 MESOS_HOME/work is no longer correct.



 On Tue, Jan 14, 2014 at 2:48 PM, Jim Freeman ji...@sbcglobal.net wrote:

 Referring to
 http://mesos.apache.org/documentation/latest/logging-and-debugging/  :



 I have not set MESOS_HOME, so it should default per the doc.



 1) Doc says Mesos uses the Google Logging library and writes logs to
 MESOS_HOME/logs by default, where MESOS_HOME is the location where Mesos is
 installed.



 How do I determine where Mesos is installed?  The 'make' put Mesos files
 in various places.



 I do see mesos-log.* files directly in /tmp (not in a */logs directory)



 2) Doc says The log directory can be *configured *using the log_dir
 parameter.



 log_dir parameter to what?  'configured' is a web link that might tell me,
 but it links to a nonexistent web page.



 3) Doc says Frameworks that run on Mesos have their output stored to a
 work directory on each machine. By default, this is MESOS_HOME/work.



 Yet I only find stderr and stdout text files in paths like this (note
 there is no work component):




 /tmp/mesos/slaves/201401131343-16842879-5050-5210-0/frameworks/201401131343-16842879-5050-5210-/executors/default/runs/43bf587e-63df-4f26-b698-ab5c9e8ed385

Re: Mesos logging configuration questions

[Jim Freeman]

I'm not specifying --log_dir for master and slave startup, yet master and slave 
logs are being written to /tmp.  For example 
/tmp/mesos-master.jifreem-VirtualBox.jifreem.log.INFO.20140121-152544.3064



 From: Vinod Kone vi...@twitter.com
To: user@mesos.apache.org user@mesos.apache.org 
Sent: Tuesday, January 21, 2014 11:06 AM
Subject: Re: Mesos logging configuration questions
 


If --log_dir is not specified nothing is written to disk.

➜  build git:(master) ✗ ./bin/mesos-master.sh --help
...
...
  --log_dir=VALUE        Location to put log files (no default, nothing
                                  is written to disk unless specified;
                                  does not affect logging to stderr)
...
...



@vinodkone


On Tue, Jan 21, 2014 at 10:59 AM, Jim Freeman jim.free...@oracle.com wrote:

So if you don't supply --log_dir, then logs go in the /tmp executor sandbox?
 
From:Benjamin Mahler [mailto:benjamin.mah...@gmail.com] 
Sent: Tuesday, January 21, 2014 10:45 AM
To: user@mesos.apache.org; Jim Freeman
Subject: Re: Mesos logging configuration questions
 
I'm afraid that document is out of date, please ignore the comments related to 
MESOS_HOME in the interim of us fixing that document.
https://issues.apache.org/jira/browse/MESOS-934
 
--log_dir is a parameter of the mesos-master and mesos-slave binaries.
 
The directory you linked is to an executor sandbox, the note about 
MESOS_HOME/work is no longer correct.
 
On Tue, Jan 14, 2014 at 2:48 PM, Jim Freeman ji...@sbcglobal.net wrote:
Referring to 
http://mesos.apache.org/documentation/latest/logging-and-debugging/  :
 
I have not set MESOS_HOME, so it should default per the doc.
 
1) Doc says Mesos uses the Google Logging library and writes logs to 
MESOS_HOME/logs by default, where MESOS_HOME is the location where Mesos is 
installed.
 
How do I determine where Mesos is installed?  The 'make' put Mesos files in 
various places.
 
I do see mesos-log.* files directly in /tmp (not in a */logs directory)
 
2) Doc says The log directory can be configured using the log_dir parameter.
 
log_dir parameter to what?  'configured' is a web link that might tell me, but 
it links to a nonexistent web page.
 
3) Doc says Frameworks that run on Mesos have their output stored to a work 
directory on each machine. By default, this is MESOS_HOME/work.
 
Yet I only find stderr and stdout text files in paths like this (note there is 
no work component):
 
/tmp/mesos/slaves/201401131343-16842879-5050-5210-0/frameworks/201401131343-16842879-5050-5210-/executors/default/runs/43bf587e-63df-4f26-b698-ab5c9e8ed385