Re: pgsql: Revert "Allow on-line enabling and disabling of data checksums"

Re: Magnus Hagander 2018-04-09 > Revert "Allow on-line enabling and disabling of data checksums" > > Modified Files > -- > doc/src/sgml/reference.sgml | 1 - This removed "" which causes pg_verify_checksums.1 not to be

Re: pgsql: Revert "Allow on-line enabling and disabling of data checksums"

On Sun, Apr 15, 2018 at 11:08 AM, Christoph Berg wrote: > Re: Magnus Hagander 2018-04-09 > > Revert "Allow on-line enabling and disabling of data checksums" > > > > Modified Files > > -- > > doc/src/sgml/reference.sgml

Re: Gotchas about pg_verify_checksums

On Sun, Apr 15, 2018 at 9:49 AM, Daniel Gustafsson wrote: > > On 15 Apr 2018, at 09:36, Michael Paquier wrote: > > > > On Tue, Apr 10, 2018 at 10:27:19PM +0200, Daniel Gustafsson wrote: > >> Thinking more on this, I don’t think the -f option should be in

Re: Online enabling of checksums

On Tue, Apr 10, 2018 at 6:18 PM, Robert Haas wrote: > On Fri, Apr 6, 2018 at 8:59 PM, Andres Freund wrote: > > This is PROPARALLEL_RESTRICTED. That doesn't strike me right, shouldn't > > they be PROPARALLEL_UNSAFE? It might be fine, but I'd not want to

Re: using index or check in ALTER TABLE SET NOT NULL

Hello Attached updated patch follows recent Reorganize partitioning code commit. regards, Sergeidiff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index bd22627..db98a98 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -215,8

Re: Gotchas about pg_verify_checksums

On Tue, Apr 10, 2018 at 10:27:19PM +0200, Daniel Gustafsson wrote: > Thinking more on this, I don’t think the -f option should be in the tool until > we have the ability to turn on/off checksums. Since checksums are always on, > or always off, -f is at best confusing IMO. The attached patch

Re: pgsql: Validate page level checksums in base backups

On Tue, Apr 10, 2018 at 11:45 PM, David Steele wrote: > On 4/10/18 5:24 PM, Tomas Vondra wrote: > >> >> I think there's a bug in sendFile(). We do check checksums on all pages >> that pass this LSN check: >> >> /* >> * Only check pages which have not been modified

Re: Gotchas about pg_verify_checksums

> On 15 Apr 2018, at 09:36, Michael Paquier wrote: > > On Tue, Apr 10, 2018 at 10:27:19PM +0200, Daniel Gustafsson wrote: >> Thinking more on this, I don’t think the -f option should be in the tool >> until >> we have the ability to turn on/off checksums. Since checksums

Re: Proposal: Adding json logging

> > I would suggest that the community consider whether postgres will log > multidimensional data. That will weigh into the decision of json vs. > another format quite significantly. I am a fan of the json5 spec ( > https://json5.org/), though adoption of this is quite poor. > > What do you mean

Re: Proposal: Adding json logging

> Exactly - arrays, maps, nested json objects. It's more organized and easier to reason about. As postgresql becomes more and more sophisticated over time, I see flat logging becoming more unwieldy. With tools like jq, reading and querying json on the command line is simple and user friendly, and

Re: Proposal: Adding json logging

> On Apr 15, 2018, at 10:07, Christophe Pettus wrote: > > >> On Apr 15, 2018, at 09:51, David Arnold wrote: >> >> 1. Throughout this vivid discussion a good portion of support has already >> been manifested for the need of a more structured (machine

Re: Proposal: Adding json logging

>This would appear to solve multiline issues within Fluent. >https://docs.fluentd.org/v0.12/articles/parser_multiline I definitely looked at that, but what guarantees do I have that the sequence is always ERROR/STATEMENT/DETAIL? And not the other way round? And it only works with tail logging

Re: Proposal: Adding json logging

>A slightly larger lift would include escaping newlines and ensuring that JSON output is always single lines, however long. I think that's necessary, actually I was implicitly assuming that as a prerequisite. I cannot imagine anything else beeing actually useful. Alternatively, I'm sure logfmt

Re: Proposal: Adding json logging

> Exactly what are you logging here ??? Why would I need to see a > multi-dimensional array in the log ? If I wanted to capture the location of errors my clients are encountering on their postgres clusters in detail, I would need to parse the 'LOCATION' string in their log entries, parse out

Re: Proposal: Adding json logging

> On Apr 15, 2018, at 09:51, David Arnold wrote: > > 1. Throughout this vivid discussion a good portion of support has already > been manifested for the need of a more structured (machine readable) logging > format. There has been no substantial objection to this need. I'm

Re: Proposal: Adding json logging

>More specifically, JSON logging does seem to be a solution in search of a problem. PostgreSQL's CSV logs are very easy to machine-parse, and if there are corrupt lines being emitted there, the first step should be to fix those, rather than introduce a new "this time, for sure" logging method.

Re: Proposal: Adding json logging

> On Apr 15, 2018, at 10:39, David Arnold wrote: > > In the light of the specific use case / problem for this thread to be born, > what exactly would you suggest? It looks like the thread skipped over the problem space for the solution space pretty fast; I see your note:

Re: Proposal: Adding json logging

On Sun, Apr 15, 2018 at 10:39 AM, David Arnold wrote: > >More specifically, JSON logging does seem to be a solution in search of a > problem. PostgreSQL's CSV logs are very easy to machine-parse, and if > there are corrupt lines being emitted there, the first step should be

Re: Proposal: Adding json logging

>It looks like the thread skipped over the problem space for the solution space pretty fast OK, I apologize, it seemed to me from the feedback that the problem was already uncontested. To verify/falsify that was the objective of my previous mail :) >Can you elaborate? Sure. CSV Logs:

Re: Proposal: Adding json logging

>Why? The newlines aren't meaningfully different from other characters you need to parse? The data isn't actually stored in a newline separated fashion, that's just one byte with that meaning I miss the details, but I believe that stdout is usually parsed and streamed simply line by line. Like

Re: Proposal: Adding json logging

On 15 April 2018 at 11:27, Jordan Deitch wrote: > > > I would suggest that the community consider whether postgres will log > > multidimensional data. That will weigh into the decision of json vs. > > another format quite significantly. I am a fan of the json5 spec ( > >

Re: Proposal: Adding json logging

Does everyone more or less agree with the following intermediate résumé? 1. Throughout this vivid discussion a good portion of support has already been manifested for the need of a more structured (machine readable) logging format. There has been no substantial objection to this need. 2. It has

Re: Proposal: Adding json logging

On 2018-04-15 18:00:05 +, David Arnold wrote: > CSV shows line breaks, STDOUT shows ERROR/FATAL and detail on different > lines, not an easy problem to stream-parse reliably (without some kind of a > buffer, etc)... Why? The newlines aren't meaningfully different from other characters you

Re: Proposal: Adding json logging

On Sun, Apr 15, 2018 at 11:08 AM, David Arnold wrote: > >This would appear to solve multiline issues within Fluent. > >https://docs.fluentd.org/v0.12/articles/parser_multiline > > I definitely looked at that, but what guarantees do I have that the > sequence is always

Re: Proposal: Adding json logging

> On Apr 15, 2018, at 11:00, David Arnold wrote: > > CSV Logs: https://pastebin.com/uwfmRdU7 Is the issue that there are line breaks in things like lines 7-9? -- -- Christophe Pettus x...@thebuild.com

Re: Deadlock in multiple CIC.

Awhile back, Alvaro Herrera wrote: >> Pushed to all affected branches, along with a somewhat lame >> isolationtester test for the condition (since we've already broken this >> twice and not noticed for long). > Buildfarm member okapi just failed this test in 9.4: okapi has continued to fail that

Re: Proposal: Adding json logging

>Have you asked that question? You seem to at least have opened the source code - did you try to figure out what the logging format is? 1. -> No. 2. -> Yes. I might be wrong, but something in my head tells me to have them seen the other way round. Unfortunately, I'm not experienced enough to be

Re: Proposal: Adding json logging

> I'd argue that the first line of attack on that should be to explain to those consumers of logs that they are making some unwarranted assumptions about the kind of inputs they'll be seeing. PostgreSQL's CSV log formats are not a particular bizarre format, or very difficult to parse. The

Re: submake-errcodes

Hi Tom, On Thu, 2018-04-12 at 14:36 -0400, Tom Lane wrote: > I don't have F27 at hand, but I tried F26 and F28, and I can't reproduce > on either one. I uploaded the SRPM for you: https://gunduz.org/temp/postgresql11-11.0-20180415_1PGDG.f27.src.rpm I built this SRPM using daily snapshot at:

Re: Proposal: Adding json logging

> On Apr 15, 2018, at 12:16, David Arnold wrote: > > Core-Problem: "Multi line logs are unnecessarily inconvenient to parse and > are not compatible with the design of some (commonly used) logging > aggregation flows." I'd argue that the first line of attack on that should

Re: WIP: Covering + unique indexes.

On Thu, Apr 12, 2018 at 9:21 AM, Teodor Sigaev wrote: > Agree, I prefer to add more Assert, even. may be, more than actually needed. > Assert-documented code :) Absolutely. The danger with a feature like this is that we'll miss one place. I suppose that you could say that I am

Re: Gotchas about pg_verify_checksums

On Sun, Apr 15, 2018 at 01:54:13PM +0200, Magnus Hagander wrote: > I have applied this patch along with the documentation patch from Michael. Thanks, I saw the commits. That's fine by me. -- Michael signature.asc Description: PGP signature

Re: [HACKERS] lseek/read/write overhead becomes visible at scale ..

On Fri, Jun 23, 2017 at 4:50 AM, Andres Freund wrote: > On 2017-06-22 12:43:16 -0400, Robert Haas wrote: >> On Wed, Jan 25, 2017 at 2:52 PM, Andres Freund wrote: >> > You'll, depending on your workload, still have a lot of lseeks even if >> > we were to

Re: Gotchas about pg_verify_checksums

On Thu, Apr 12, 2018 at 05:47:29AM +0900, Michael Paquier wrote: > On Wed, Apr 11, 2018 at 10:21:29PM +0200, Daniel Gustafsson wrote: >> Naming it pg_checksums, with only verification as an option, seems to me to >> imply future direction for 12 more than what pg_verify_checksums does. I >>

Re: Setting rpath on llvmjit.so?

2018-04-15 6:13 GMT+09:00 Andres Freund : > On 2018-04-14 17:10:21 -0400, Tom Lane wrote: > > Andres Freund writes: > > > On April 14, 2018 1:56:08 PM PDT, Tom Lane wrote: > > >> The project policy is to use exactly the GNU

Re: Reopen logfile on SIGHUP

At Thu, 12 Apr 2018 17:23:42 +0300, Alexander Kuzmenkov wrote in > On 11.04.2018 00:00, Tom Lane wrote: > > So we need a mechanism that's narrowly targeted > > to reopening the logfile, without SIGHUP'ing the