Re: [HACKERS] Similar to csvlog but not really, json logs?

2014-08-28 Thread Josh Berkus
On 08/27/2014 09:53 AM, Andres Freund wrote: Perhaps instead of doing this in-core it would be better to make log handling more extensible? I'm thinking add a specific binary format and an external tool that can parse that and do whatever the user wants with it. That means we don't have

Re: [HACKERS] Similar to csvlog but not really, json logs?

2014-08-27 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Stephen Frost wrote: To try to clarify that a bit, as it comes across as rather opaque even on my re-reading, consider a case where you can't have the credit_card_number field ever exported to an audit or log file, but you're required to

Re: [HACKERS] Similar to csvlog but not really, json logs?

2014-08-27 Thread Jim Nasby
On 8/26/14, 8:45 PM, Michael Paquier wrote: Hi all, As mentioned here, we support multiple logging format: http://www.postgresql.org/docs/devel/static/runtime-config-logging.html Now what about a json format logging with one json object per log entry? A single json entry would need more space

Re: [HACKERS] Similar to csvlog but not really, json logs?

2014-08-27 Thread Andres Freund
On 2014-08-26 23:04:48 -0500, Jim Nasby wrote: On 8/26/14, 8:45 PM, Michael Paquier wrote: Hi all, As mentioned here, we support multiple logging format: http://www.postgresql.org/docs/devel/static/runtime-config-logging.html Now what about a json format logging with one json object per log

Re: [HACKERS] Similar to csvlog but not really, json logs?

2014-08-27 Thread Petr Jelinek
On 27/08/14 18:53, Andres Freund wrote: On 2014-08-26 23:04:48 -0500, Jim Nasby wrote: On 8/26/14, 8:45 PM, Michael Paquier wrote: Hi all, As mentioned here, we support multiple logging format: http://www.postgresql.org/docs/devel/static/runtime-config-logging.html Now what about a json

Re: [HACKERS] Similar to csvlog but not really, json logs?

2014-08-26 Thread Peter Geoghegan
On Tue, Aug 26, 2014 at 6:45 PM, Michael Paquier michael.paqu...@gmail.com wrote: Thoughts? I think that it would be a good beginner's project to make pprint() print JSON. I spend enough time staring at its output that I've often wished I could expand and collapse each part using my text

Re: [HACKERS] Similar to csvlog but not really, json logs?

2014-08-26 Thread Tom Lane
Michael Paquier michael.paqu...@gmail.com writes: Now what about a json format logging with one json object per log entry? A single json entry would need more space than a csv one as we need to track the field names with their values. Also, there is always the argument that if an application

Re: [HACKERS] Similar to csvlog but not really, json logs?

2014-08-26 Thread Tom Lane
Peter Geoghegan p...@heroku.com writes: I think that it would be a good beginner's project to make pprint() print JSON. There's something to be said for that (or, really, for any standardized widely-popular textual data format; but JSON is a perfectly reasonable candidate). The existing

Re: [HACKERS] Similar to csvlog but not really, json logs?

2014-08-26 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Michael Paquier michael.paqu...@gmail.com writes: Now what about a json format logging with one json object per log entry? A single json entry would need more space than a csv one as we need to track the field names with their values. Also, there is

Re: [HACKERS] Similar to csvlog but not really, json logs?

2014-08-26 Thread Peter Geoghegan
On Tue, Aug 26, 2014 at 7:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: I think that it would be a good beginner's project to make pprint() print JSON. There's something to be said for that (or, really, for any standardized widely-popular textual data format; but JSON is a perfectly reasonable

Re: [HACKERS] Similar to csvlog but not really, json logs?

2014-08-26 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: I think the extra representational overhead is already a good reason to say no. There is not any production scenario I've ever heard of where log output volume isn't a consideration. The flip side is that there

Re: [HACKERS] Similar to csvlog but not really, json logs?

2014-08-26 Thread Alvaro Herrera
Stephen Frost wrote: The flip side is that there are absolutely production cases where what we output is either too little or too much- being able to control that and then have the (filtered) result in JSON would be more-or-less exactly what a client of ours is looking for. My impression is

Re: [HACKERS] Similar to csvlog but not really, json logs?

2014-08-26 Thread Stephen Frost
Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: The flip side is that there are absolutely production cases where what we output is either too little or too much- being able to control that and then have the (filtered) result in JSON would be

Re: [HACKERS] Similar to csvlog but not really, json logs?

2014-08-26 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Stephen Frost wrote: The flip side is that there are absolutely production cases where what we output is either too little or too much- being able to control that and then have the (filtered) result in JSON would be more-or-less exactly

Re: [HACKERS] Similar to csvlog but not really, json logs?

2014-08-26 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: Consider an audit system where which columns end up in the audit log are controlled by issuing ALTER TABLE .. ALTER COLUMN type statements. blink I'd like to consider such a thing, but it seems like utter pie in the sky. Do you really believe that

Re: [HACKERS] Similar to csvlog but not really, json logs?

2014-08-26 Thread Tom Lane
I wrote: Stephen Frost sfr...@snowman.net writes: Consider an audit system where which columns end up in the audit log are controlled by issuing ALTER TABLE .. ALTER COLUMN type statements. I'd like to consider such a thing, but it seems like utter pie in the sky. On further thought: the

Re: [HACKERS] Similar to csvlog but not really, json logs?

2014-08-26 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: Consider an audit system where which columns end up in the audit log are controlled by issuing ALTER TABLE .. ALTER COLUMN type statements. blink I'd like to consider such a thing, but it seems like utter

Re: [HACKERS] Similar to csvlog but not really, json logs?

2014-08-26 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: I wrote: Stephen Frost sfr...@snowman.net writes: Consider an audit system where which columns end up in the audit log are controlled by issuing ALTER TABLE .. ALTER COLUMN type statements. I'd like to consider such a thing, but it seems like utter

Re: [HACKERS] Similar to csvlog but not really, json logs?

2014-08-26 Thread Fujii Masao
On Wed, Aug 27, 2014 at 12:48 PM, Tom Lane t...@sss.pgh.pa.us wrote: I wrote: Stephen Frost sfr...@snowman.net writes: Consider an audit system where which columns end up in the audit log are controlled by issuing ALTER TABLE .. ALTER COLUMN type statements. I'd like to consider such a