Hi Otis, Thanks for the good summary. The conversation is mostly about 1) in this thread, because right now Mesos logs are not really structured, or at least most of it.
On Tue, Dec 20, 2016 at 6:57 AM, Otis Gospodnetić < otis.gospodne...@gmail.com> wrote: > Hi Zhitao, > > When people talk about structure and logging it typically means two things: > > 1) make the log format a known/standard format where all its elements are > known, and thus it's easy to parse them; a log event can still be a single > line, but it can also be multi-line or JSON or some other (even binary) > format. As long as the format/structure is known, the log event *is* > structured. > > 2) I want tools/configs/patterns that will let me easily parse this log > event structure and send it somewhere (e.g. Elasticsearch or Logsene > <http://sematext.com/logsene> or ...) where this structure will be > handled in the way that lets me easy filtering/slicing and dicing by one or > more attributes/fields extracted from the log event structure. > > *For 1*): > I'm assuming Mesos logs already are structured. I assume their format is > either widely known (like Apache common log format, for example), or > well-documented (again like Apache common log format). If that is not > true, then yes, Mesos devs will want to do document the structure. I've > looked at https://mesos.apache.org/documentation/latest/logging/ but saw > nothing mentioning the structure. Maybe this info is somewhere else? > > *For 2)* > This is where modern log shippers come in. We open-sourced our Logagent > <https://github.com/sematext/logagent-js> (more info here > <http://sematext.com/logagent/>), which has log parsing (and thus > structuring) built-in. It ships with a bunch of log patterns/parsers, and > one can add new ones (e.g. for Mesos). Elasticsearch, mentioned in this > thread, is one of the outputs. It's sort of like Filebeat+Logstash in one, > and it's often used in Dockerized deployments, as part of this Docker > agent <https://sematext.com/docker/>. One could also use Logstash for > parsing/structuring, but Logstash is a bit heavy. > > I hope this helps. > > Otis > -- > Monitoring - Log Management - Alerting - Anomaly Detection > Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > > > On Mon, Dec 19, 2016 at 6:03 PM, Zhitao Li <zhitaoli...@gmail.com> wrote: > >> Charles, >> >> Thanks for sharing the pattern. If my reading is right, this will extract >> the entire message line as one string. What I'm looking for is: on top of >> extracting the entire message line, also break it into structured fields >> automatically. >> >> >> >> On Mon, Dec 19, 2016 at 1:59 PM, Charles Allen < >> charles.al...@metamarkets.com> wrote: >> >>> For what its worth we use SumoLogic and the magic parsing search looks >>> like >>> this: >>> >>> parse regex field=message "^(?<glog_severity>[IWE])(?<glog_date>[0-9]{4} >>> [0-9:.]*) [0-9]* >>> (?<glog_source_file>[0-9a-zA-Z.]*):(?<glog_source_line>[0-9]*)] >>> (?<glog_message>.*)$" >>> >>> >>> >>> On Mon, Dec 19, 2016 at 11:15 AM Joris Van Remoortere < >>> jo...@mesosphere.io> >>> wrote: >>> >>> > @Zhitao are you looking specifically for structure or just for tagging? >>> > glog does already have support for custom tags in the header. I don't >>> know >>> > if this is enough for your use case though. >>> > >>> > — >>> > *Joris Van Remoortere* >>> > Mesosphere >>> > >>> > On Mon, Dec 19, 2016 at 9:58 AM, James Peach <jor...@gmail.com> wrote: >>> > >>> > >>> > > On Dec 19, 2016, at 9:43 AM, Zhitao Li <zhitaoli...@gmail.com> >>> wrote: >>> > > >>> > > Hi, >>> > > >>> > > I'm looking at how to better utilize ElasticSearch to perform log >>> > analysis for logs from Mesos. It seems like ElasticSearch would >>> generally >>> > work better for structured logging, but Mesos still uses glog thus all >>> logs >>> > produced are old-school unstructured lines. >>> > > >>> > > I wonder whether anyone has brought the conversation of making Mesos >>> > logs easier to process, or if anyone has experience to share. >>> > >>> > Are you trying to stitch together sequences of events? I that case, >>> would >>> > direct event logging be more useful? >>> > >>> > J >>> > >>> > >>> > >>> >> >> >> >> -- >> Cheers, >> >> Zhitao Li >> > > -- Cheers, Zhitao Li
