Hi Zhitao,
Is there a JIRA for this? I looked at
http://search-hadoop.com/?project=Mesos&type=issue&q=log but didn't see
anything matching 1).
I'd love for Logagent to ship with log parser/pattern for Mesos OOTB.
Thanks,
Otis
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr &
Hi Otis,
Thanks for the good summary. The conversation is mostly about 1) in this
thread, because right now Mesos logs are not really structured, or at least
most of it.
On Tue, Dec 20, 2016 at 6:57 AM, Otis Gospodnetić <
otis.gospodne...@gmail.com> wrote:
> Hi Zhitao,
>
> When people talk about
Hi Zhitao,
When people talk about structure and logging it typically means two things:
1) make the log format a known/standard format where all its elements are
known, and thus it's easy to parse them; a log event can still be a single
line, but it can also be multi-line or JSON or some other (ev
Why not `logstash`? I think it's the target case of `logstash`.
On Tue, Dec 20, 2016 at 7:35 AM Zhitao Li wrote:
> Great.
>
> I also found this old thread
> http://search-hadoop.com/m/Mesos/0Vlr6meKs116T2k1?subj=Mapped+diagnostics+context+Adding+internal+Mesos+IDs+as+context+to+the+logs
> on
>
Great.
I also found this old thread
http://search-hadoop.com/m/Mesos/0Vlr6meKs116T2k1?subj=Mapped+diagnostics+context+Adding+internal+Mesos+IDs+as+context+to+the+logs
on
dev list, which seems no consensus has been made.
Maybe we can talk about this in the next community sync?
On Mon, Dec 19, 201
> On Dec 19, 2016, at 2:54 PM, Zhitao Li wrote:
>
> Hi James,
>
> Stitching events together is only one possible use cases, and I'm not exactly
> sure what you meant by directly event logging.
>
> Taking the hierarchical allocator for example. In a multi-framework cluster,
> sometimes I want
Charles,
Thanks for sharing the pattern. If my reading is right, this will extract
the entire message line as one string. What I'm looking for is: on top of
extracting the entire message line, also break it into structured fields
automatically.
On Mon, Dec 19, 2016 at 1:59 PM, Charles Allen <
c
Joris,
I am particular looking for structure. We have mechanism to add static tags
easily to log collected into ELK.
If there is a way to dynamically inject tags like "framework_id" at actual
logging call, it might be a starting point for me.
I cannot find a good reference on how to add tagging
Hi James,
Stitching events together is only one possible use cases, and I'm not
exactly sure what you meant by directly event logging.
Taking the hierarchical allocator for example. In a multi-framework
cluster, sometimes I want to comb through various loggings and present a
trace on how allocati
For what its worth we use SumoLogic and the magic parsing search looks like
this:
parse regex field=message "^(?[IWE])(?[0-9]{4}
[0-9:.]*) [0-9]*
(?[0-9a-zA-Z.]*):(?[0-9]*)]
(?.*)$"
On Mon, Dec 19, 2016 at 11:15 AM Joris Van Remoortere
wrote:
> @Zhitao are you looking specifically for structu
@Zhitao are you looking specifically for structure or just for tagging?
glog does already have support for custom tags in the header. I don't know
if this is enough for your use case though.
—
*Joris Van Remoortere*
Mesosphere
On Mon, Dec 19, 2016 at 9:58 AM, James Peach wrote:
>
> > On Dec 19,
> On Dec 19, 2016, at 9:43 AM, Zhitao Li wrote:
>
> Hi,
>
> I'm looking at how to better utilize ElasticSearch to perform log analysis
> for logs from Mesos. It seems like ElasticSearch would generally work better
> for structured logging, but Mesos still uses glog thus all logs produced are
Hi,
I'm looking at how to better utilize ElasticSearch to perform log analysis
for logs from Mesos. It seems like ElasticSearch would generally work
better for structured logging, but Mesos still uses glog thus all logs
produced are old-school unstructured lines.
I wonder whether anyone has broug
13 matches
Mail list logo
