Re: Mapped diagnostics context - Adding internal Mesos IDs as context to the logs

[Joris Van Remoortere]

It's adding *an* extension to the header of the log. In this case after the
file name.
In the comment above the code you can see that it's:

> often used to append the port we're listening on to the logfile

That means we can tag our own IDs in there such as Agent ID, Executor ID,
Master ID, etc.

—
*Joris Van Remoortere*
Mesosphere

On Wed, Oct 5, 2016 at 11:34 AM, Benjamin Hindman 
wrote:

> How does adding the filename extension help with the context problem?
>
> On Wed, Oct 5, 2016 at 6:14 AM, Joris Van Remoortere 
> wrote:
>
>> This is technically feasible:
>>> //
>>> // Specify an "extension" added to the filename specified via
>>> // SetLogDestination.  This applies to all severity levels.  It's
>>> // often used to append the port we're listening on to the logfile
>>> // name.  Thread-safe.
>>> //
>>> GOOGLE_GLOG_DLL_DECL void SetLogFilenameExtension(
>>> const char* filename_extension);
>>
>>
>> @benm @vinod @benh what are your thoughts on actually doing it?
>>
>> —
>> *Joris Van Remoortere*
>> Mesosphere
>>
>> On Tue, Oct 4, 2016 at 2:39 PM, Zameer Manji  wrote:
>>
>>> I don't know if this is feasible or not, but I would be a strong +1 to
>>> this.
>>>
>>> This would make tracing failures much easier.
>>>
>>> On Tue, Oct 4, 2016 at 6:12 AM, Frank Scholten 
>>> wrote:
>>>
>>> > Hi,
>>> >
>>> > On JIRA I found several issues about making logs less spammy as well
>>> > as making them easier to understand for day to day operations.
>>> >
>>> > https://issues.apache.org/jira/browse/MESOS-4432 Condense (redundant)
>>> > log messages related to task launch/status/finish
>>> > https://issues.apache.org/jira/browse/MESOS-5467 offer DECLINE /
>>> > ACCEPT + Recovered resource messages are spammy
>>> > https://issues.apache.org/jira/browse/MESOS-4430 Identify and change
>>> > logging level for message that don't contain specific
>>> > task/framework/slave info
>>> >
>>> > Besides reducing the logs would it be possible to add more context? In
>>> > the Java world the technique of 'Mapped diagnostic context' is used
>>> > with Logback where each log line contains a few fields with context.
>>> > See http://logback.qos.ch/manual/mdc.html
>>> >
>>> > To translate this to Mesos how about adding the internal IDs such as
>>> > agent, framework, and task IDs at the beginning of the log, so this
>>> > information is separated from the textual, human readble log message.
>>> > At the moment this information is tangled which makes it hard to
>>> > interpret, especially when there are so many logs for each task.
>>> >
>>> > For example, change this
>>> >
>>> > I1004 12:24:12.118803  3780 status_update_manager.cpp:320] Received
>>> > status update TASK_FAILED (UUID: a1d03948-30bf-46b3-9599-cfcfc7cbc27b)
>>> > for task weave-demo_database_catalogue-db.6d415c17-8a2d-11e6-90a7-
>>> > 0242458f9469
>>> > of framework f1546295-ab46-496a-8cf9-91756fece4ed-
>>> >
>>> > to
>>> >
>>> > I1004 12:24:12.118803  3780 status_update_manager.cpp:320
>>> > A:agent12318032910 F:f1546295-ab46-496a-8cf9-91756fece4ed-
>>> > T:xyz-a1d03948-30bf-46b3-9599-cfcfc7cbc27b] Received status update
>>> > TASK_FAILED for task 'xyz'
>>> >
>>> > In this case the header contains A:$AGENT_ID, F:$FRAMEWORK_ID,
>>> > T:$TASK_ID and as the context and lines with similar context can be
>>> > more easily correlated visually.
>>> >
>>> > Is this feasible?
>>> >
>>> > Cheers,
>>> >
>>> > Frank
>>> >
>>> > --
>>> > Zameer Manji
>>> >
>>>
>>
>>
>
>
> --
> Benjamin Hindman
> Founder of Mesosphere and Co-Creator of Apache Mesos
> Mesosphere Inc.  
>
> Follow us on Twitter: @mesosphere 
>
> [image: Watch the Video]
> 
>  [image:
> .]See how DC/OS elastically
> powers the modern application
> 
>
>

[GitHub] mesos pull request #170: typo

[The-smooth-operator]

GitHub user The-smooth-operator opened a pull request:

https://github.com/apache/mesos/pull/170

typo

Corrected small typo.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/The-smooth-operator/mesos typo

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/mesos/pull/170.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #170


commit f5a8c9d7c02a2bc121aa84e5eb85409b7acfd6c2
Author: Alberto 
Date:   2016-10-06T14:59:33Z

typo




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] mesos pull request #170: typo

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/mesos/pull/170


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: LXC Container Support

[Tom Barber]

Also just as an aside, being able to run system containers and not
precanned Docker applications would seem to certainly be a missing feature
that Mesos could benefit from, as it would allow sysadmins to spin up
vanilla Ubuntu, Centos, Debian etc images without doing something funky
like using the Docker image and tailing /dev/null. :)

On Thu, Oct 6, 2016 at 2:11 AM, Tom Barber  wrote:

> Thanks guys.  I do a lot of work with the juju ecosystem and lxc containers
> to deploy workloads with that, for example multi node big top clusters etc.
> It's far more flexible using LXC system containers than pre configured app
> containers. This works on cloud and bare metal but I'd also like to be able
> to deploy containers to Mesos using the same technology stack.
>
> Cheers
> Tom
>
> On 6 Oct 2016 00:51, "Jie Yu"  wrote:
>
> > Tom,
> >
> > I'm looking to add LXC/LXD support, POC initially then hopefully bake it
> > > into some Mesos libraries. From what I can tell Mesos doesn't have
> hooks
> > > for LXC in the new container setup and also the External Shim that was
> > > underdevelopment is Deprecated. Sound roughly accurate so far?
> >
> >
> > Yup. External containerizer has been deprecated and removed. The
> direction
> > is to maintain only one containerizer in the future, and make that
> > containerizer better and supports various containerization requests.
> >
> > If so, would extending the Unified Containerizer be the correct way to go
> > > about adding LXC/LXD support?
> >
> >
> > Yes, I would prefer exploring this direction. But even before that, it
> > would be nice to understand the motivation for using lxd. I'd prefer we
> > find out the feature gap and augment unified containerizer to fill the
> gap.
> >
> > - Jie
> >
> > On Wed, Oct 5, 2016 at 3:10 PM, Tom Barber 
> > wrote:
> >
> > > Hello folks,
> > >
> > > I've been Googling Mesos container support a bit recently, I see
> support
> > > for Mesos(AppC) and Docker in the latest versions.
> > >
> > > I'm looking to add LXC/LXD support, POC initially then hopefully bake
> it
> > > into some Mesos libraries. From what I can tell Mesos doesn't have
> hooks
> > > for LXC in the new container setup and also the External Shim that was
> > > underdevelopment is Deprecated. Sound roughly accurate so far?
> > >
> > > If so, would extending the Unified Containerizer be the correct way to
> go
> > > about adding LXC/LXD support? (
> > > http://mesos.apache.org/documentation/latest/container-image/)
> > >
> > > Thanks
> > >
> > > Tom
> > >
> >
>

Re: Notification: Community Meeting @ Thu Oct 6, 2016 3pm - 4pm PST (Apache Mesos)

[haosdent]

Did it cancel?

On Oct 6, 2016 6:09 AM, "Michael Park"  wrote:

>
>