Proposal: Adding json logging

2018-04-13 Thread David Arnold
xoe.solutions/> DAVID ARNOLD Gerente General xoe.solutions dar@xoe.solutions +57 (315) 304 13 68 *Confidentiality Note: * This email may contain confidential and/or private information. If you received this email in error please delete and notify sender. *Environmental Consideration: * Please avoid pri

Re: Proposal: Adding json logging

2018-04-14 Thread David Arnold
> Plus it's likely only a short-lived interchange format, not something to be retained for a long period. Absolutely. There might be an argument that it's not easy on the eyes in the case it would be consumed by a pair of them. It's absolutely valid. Golang community has found a solution for that

Re: Proposal: Adding json logging

2018-04-14 Thread David Arnold
>I'm dubious that JSON is "easier on machines" than CSV. Under common paradigms you are right, but if we talk of line-by-line streaming with subsequent processing, then it's a show stopper. Of course, some log aggregators have buffers for that and can do Multiline parsing on that buffer, but 1. No

Re: Proposal: Adding json logging

2018-04-14 Thread David Arnold
. But I'm in favor to step back with that idea in favour of prioritizing JSON. El sáb., 14 abr. 2018, 11:03 a.m., David Arnold escribió: > >I'm dubious that JSON is "easier on machines" than CSV. > > Under common paradigms you are right, but if we talk of line-by-lin

Re: Proposal: Adding json logging

2018-04-14 Thread David Arnold
;a newline?" > > - ... > > > JSON, whatever gripes I have about > > the format[1] is extremely well specified, and hence has excellent > > parsing libraries. > > It has, if nothing else, the benefit of coming around later and seeing > what happened with CSV. >

Re: Proposal: Adding json logging

2018-04-15 Thread David Arnold
>A slightly larger lift would include escaping newlines and ensuring that JSON output is always single lines, however long. I think that's necessary, actually I was implicitly assuming that as a prerequisite. I cannot imagine anything else beeing actually useful. Alternatively, I'm sure logfmt ha

Re: Proposal: Adding json logging

2018-04-15 Thread David Arnold
> Exactly - arrays, maps, nested json objects. It's more organized and easier to reason about. As postgresql becomes more and more sophisticated over time, I see flat logging becoming more unwieldy. With tools like jq, reading and querying json on the command line is simple and user friendly, and u

Re: Proposal: Adding json logging

2018-04-15 Thread David Arnold
te intention to take a step forward. Best Regards, David Arnold El dom., 15 abr. 2018, 10:49 a.m., David Arnold escribió: > > Exactly - arrays, maps, nested json objects. It's more organized and > easier to reason about. As postgresql becomes more and more sophisticated > over t

Re: Proposal: Adding json logging

2018-04-15 Thread David Arnold
fore a SPOF) First point is definitely a strong one. Did I miss any additional arguments? El dom., 15 abr. 2018 a las 12:24, Christophe Pettus () escribió: > > > On Apr 15, 2018, at 10:07, Christophe Pettus wrote: > > > > > >> On Apr 15, 2018, at 09:51, David Arnold

Re: Proposal: Adding json logging

2018-04-15 Thread David Arnold
12:46, Christophe Pettus () escribió: > > > On Apr 15, 2018, at 10:39, David Arnold wrote: > > > > In the light of the specific use case / problem for this thread to be > born, what exactly would you suggest? > > It looks like the thread skipped over the problem space for

Re: Proposal: Adding json logging

2018-04-15 Thread David Arnold
nes of unknown cardinality. El dom., 15 abr. 2018 a las 13:00, David Arnold () escribió: > >It looks like the thread skipped over the problem space for the solution > space pretty fast > > OK, I apologize, it seemed to me from the feedback that the problem was > already uncontested

Re: Proposal: Adding json logging

2018-04-15 Thread David Arnold
tbit.io/documentation/0.13/input/tail.html#multiline> support configuration feature." https://fluentbit.io/documentation/0.13/parser/regular_expression.html El dom., 15 abr. 2018 a las 13:08, David Arnold () escribió: > >This would appear to solve multiline issues within Fluent..

Re: Proposal: Adding json logging

2018-04-15 Thread David Arnold
.io/documentation/current/getting_started/parser.html El dom., 15 abr. 2018 a las 13:29, John W Higgins () escribió: > On Sun, Apr 15, 2018 at 11:08 AM, David Arnold wrote: > >> >This would appear to solve multiline issues within Fluent. >> >https://docs.fluentd.org/v

Re: Proposal: Adding json logging

2018-04-15 Thread David Arnold
issues/564 Let's see what Eduardo has to inform us about this... El dom., 15 abr. 2018 a las 16:05, Christophe Pettus () escribió: > > > On Apr 15, 2018, at 12:16, David Arnold wrote: > > > > Core-Problem: "Multi line logs are unnecessarily inconvenient to parse

Re: Proposal: Adding json logging

2018-04-16 Thread David Arnold
6 abr. 2018 a las 9:41, David Fetter () escribió: > On Mon, Apr 16, 2018 at 10:06:29AM -0400, Andrew Dunstan wrote: > > On 04/15/2018 05:05 PM, Christophe Pettus wrote: > > >> On Apr 15, 2018, at 12:16, David Arnold wrote: > > >> > > >> Core-Problem: &quo

Re: Proposal: Adding json logging

2018-04-16 Thread David Arnold
ould be equally happy with that solution. The second order problem, after all is just that: of second order... El lun., 16 abr. 2018, 1:28 p.m., Daniel Verite escribió: > David Arnold wrote: > > > Not claiming this assumption does imply parsing of a *rolling* set > >

Re: Proposal: Adding json logging

2018-04-17 Thread David Arnold
lid, seems sometimes is tought it's limits in very mondane practicability and efficiency needs. El mar., 17 abr. 2018, 6:55 a.m., Daniel Verite escribió: > David Arnold wrote: > > > Interesting, does that implicitly mean the whole log event would get > > transmitted

Re: Proposal: Adding json logging

2018-04-17 Thread David Arnold
the problem definition is "unnecessary", which implies judgment on responsibilities and ecosystems working together, rather than a broken system. El mar., 17 abr. 2018 a las 7:31, David Arnold () escribió: > >To me it's implied by the doc at: > > https://www.postgresql.o

Re: Proposal: Adding json logging

2018-04-17 Thread David Arnold
hat these things are > always UTF8? > > -- > Álvaro Herrerahttps://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services > -- [image: XOE Solutions] <http://xoe.solutions/> DAVID ARNOLD Gerente General xoe.solutions dar

Re: Proposal: Adding json logging

2018-04-18 Thread David Arnold
t; > > For the record, I'm tentatively in favor of including something like > > this in contrib. > > I'm much less fussed by this in contrib/ (with the same concern you > noted), at minimum as an example of how to do logging in other formats. > > -- > -- Ch