Hi Zhitao, Is there a JIRA for this? I looked at http://search-hadoop.com/?project=Mesos&type=issue&q=log but didn't see anything matching 1).
I'd love for Logagent to ship with log parser/pattern for Mesos OOTB. Thanks, Otis -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ On Tue, Dec 20, 2016 at 12:06 PM, Zhitao Li <zhitaoli...@gmail.com> wrote: > Hi Otis, > > Thanks for the good summary. The conversation is mostly about 1) in this > thread, because right now Mesos logs are not really structured, or at least > most of it. > > On Tue, Dec 20, 2016 at 6:57 AM, Otis Gospodnetić < > otis.gospodne...@gmail.com> wrote: > >> Hi Zhitao, >> >> When people talk about structure and logging it typically means two >> things: >> >> 1) make the log format a known/standard format where all its elements are >> known, and thus it's easy to parse them; a log event can still be a single >> line, but it can also be multi-line or JSON or some other (even binary) >> format. As long as the format/structure is known, the log event *is* >> structured. >> >> 2) I want tools/configs/patterns that will let me easily parse this log >> event structure and send it somewhere (e.g. Elasticsearch or Logsene >> <http://sematext.com/logsene> or ...) where this structure will be >> handled in the way that lets me easy filtering/slicing and dicing by one or >> more attributes/fields extracted from the log event structure. >> >> *For 1*): >> I'm assuming Mesos logs already are structured. I assume their format is >> either widely known (like Apache common log format, for example), or >> well-documented (again like Apache common log format). If that is not >> true, then yes, Mesos devs will want to do document the structure. I've >> looked at https://mesos.apache.org/documentation/latest/logging/ but saw >> nothing mentioning the structure. Maybe this info is somewhere else? >> >> *For 2)* >> This is where modern log shippers come in. We open-sourced our Logagent >> <https://github.com/sematext/logagent-js> (more info here >> <http://sematext.com/logagent/>), which has log parsing (and thus >> structuring) built-in. It ships with a bunch of log patterns/parsers, and >> one can add new ones (e.g. for Mesos). Elasticsearch, mentioned in this >> thread, is one of the outputs. It's sort of like Filebeat+Logstash in one, >> and it's often used in Dockerized deployments, as part of this Docker >> agent <https://sematext.com/docker/>. One could also use Logstash for >> parsing/structuring, but Logstash is a bit heavy. >> >> I hope this helps. >> >> Otis >> -- >> Monitoring - Log Management - Alerting - Anomaly Detection >> Solr & Elasticsearch Consulting Support Training - http://sematext.com/ >> >> >> On Mon, Dec 19, 2016 at 6:03 PM, Zhitao Li <zhitaoli...@gmail.com> wrote: >> >>> Charles, >>> >>> Thanks for sharing the pattern. If my reading is right, this will >>> extract the entire message line as one string. What I'm looking for is: on >>> top of extracting the entire message line, also break it into structured >>> fields automatically. >>> >>> >>> >>> On Mon, Dec 19, 2016 at 1:59 PM, Charles Allen < >>> charles.al...@metamarkets.com> wrote: >>> >>>> For what its worth we use SumoLogic and the magic parsing search looks >>>> like >>>> this: >>>> >>>> parse regex field=message "^(?<glog_severity>[IWE])(?<gl >>>> og_date>[0-9]{4} >>>> [0-9:.]*) [0-9]* >>>> (?<glog_source_file>[0-9a-zA-Z.]*):(?<glog_source_line>[0-9]*)] >>>> (?<glog_message>.*)$" >>>> >>>> >>>> >>>> On Mon, Dec 19, 2016 at 11:15 AM Joris Van Remoortere < >>>> jo...@mesosphere.io> >>>> wrote: >>>> >>>> > @Zhitao are you looking specifically for structure or just for >>>> tagging? >>>> > glog does already have support for custom tags in the header. I don't >>>> know >>>> > if this is enough for your use case though. >>>> > >>>> > — >>>> > *Joris Van Remoortere* >>>> > Mesosphere >>>> > >>>> > On Mon, Dec 19, 2016 at 9:58 AM, James Peach <jor...@gmail.com> >>>> wrote: >>>> > >>>> > >>>> > > On Dec 19, 2016, at 9:43 AM, Zhitao Li <zhitaoli...@gmail.com> >>>> wrote: >>>> > > >>>> > > Hi, >>>> > > >>>> > > I'm looking at how to better utilize ElasticSearch to perform log >>>> > analysis for logs from Mesos. It seems like ElasticSearch would >>>> generally >>>> > work better for structured logging, but Mesos still uses glog thus >>>> all logs >>>> > produced are old-school unstructured lines. >>>> > > >>>> > > I wonder whether anyone has brought the conversation of making Mesos >>>> > logs easier to process, or if anyone has experience to share. >>>> > >>>> > Are you trying to stitch together sequences of events? I that case, >>>> would >>>> > direct event logging be more useful? >>>> > >>>> > J >>>> > >>>> > >>>> > >>>> >>> >>> >>> >>> -- >>> Cheers, >>> >>> Zhitao Li >>> >> >> > > > -- > Cheers, > > Zhitao Li >
