On Sun, Jan 22, 2017 at 3:27 AM, Stephen Frost wrote:
> All,
>
> * Simon Riggs (si...@2ndquadrant.com) wrote:
>> On 12 August 2016 at 01:01, Tom Lane wrote:
>> > Michael Paquier writes:
>> >> In short, autovacuum will need to scan by itself the VM of each
>> >> relation and decide based on that.
Jim Nasby writes:
> The other (possibly naive) question I have is how useful negative
> entries really are? Will Postgres regularly incur negative lookups, or
> will these only happen due to user activity?
It varies depending on the particular syscache, but in at least some
of them, negative ca
On Sun, Jan 22, 2017 at 4:06 AM, Petr Jelinek
wrote:
> Because we don't have intermediate steps in logical replication, writes
> happen immediately and in whole transactions so whatever was received by
> the time we send reply is already written (it might not necessarily be
> that way forever so t
On 12/26/16 2:31 AM, Kyotaro HORIGUCHI wrote:
The points of discussion are the following, I think.
1. The first patch seems working well. It costs the time to scan
the whole of a catcache that have negative entries for other
reloids. However, such negative entries are created by rather
On 1/21/17 10:02 AM, Tom Lane wrote:
Magnus Hagander writes:
Is it time to enable checksums by default, and give initdb a switch to turn
it off instead?
Have we seen *even one* report of checksums catching problems in a useful
way?
I've experienced multiple corruption events that were ultima
Tom,
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> Not at all; I just think that it's not clear that they are a net win
> for the average user, and so I'm unconvinced that turning them on by
> default is a good idea. I could be convinced otherwise by suitable
> evidence. What I'm objecting to is turn
On 1/20/17 5:38 PM, Nico Williams wrote:
If these triggers could be automatically generated, that sure would be
nice, but some control would be needed over when to update the MV vs.
mark it as needing a refresh.
FWIW, pg_classy[1], which is still a work in progress, would allow for
that. The i
Thomas,
* Thomas Munro (thomas.mu...@enterprisedb.com) wrote:
> On Sun, Jan 22, 2017 at 7:37 AM, Stephen Frost wrote:
> > Exactly, and that awareness will allow a user to prevent further data
> > loss or corruption. Slow corruption over time is a very much known and
> > accepted real-world case
On Sun, Jan 22, 2017 at 7:37 AM, Stephen Frost wrote:
> Exactly, and that awareness will allow a user to prevent further data
> loss or corruption. Slow corruption over time is a very much known and
> accepted real-world case that people do experience, as well as bit
> flipping enough for someone
All,
* Simon Riggs (si...@2ndquadrant.com) wrote:
> On 12 August 2016 at 01:01, Tom Lane wrote:
> > Michael Paquier writes:
> >> In short, autovacuum will need to scan by itself the VM of each
> >> relation and decide based on that.
> >
> > That seems like a worthwhile approach to pursue. The V
On Sat, Jan 21, 2017 at 10:16 PM, Michael Banck
wrote:
> On Sat, Jan 21, 2017 at 09:02:25PM +0200, Ants Aasma wrote:
>> On Sat, Jan 21, 2017 at 6:41 PM, Andreas Karlsson wrote:
>> > It might be worth looking into using the CRC CPU instruction to reduce this
>> > overhead, like we do for the WAL c
David Bowen writes:
> If the intent of the test is to compare strings, you should be using ne not
> !=.
Sure. We were just confused as to how it had ever appeared to work.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make
Peter Eisentraut writes:
> On 1/21/17 12:50 PM, Tom Lane wrote:
>> I have to question the decision to make "no locales" a hard error.
>> What's the point of that? In fact, should we even be bothering with
>> a warning, considering how often initdb runs unattended these days?
> Hmm, it was a warn
-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160
Tom Lane points out:
> Yeah, and there's a bunch of usability tooling that we don't have,
> centered around "what do you do after you get a checksum error?".
I've asked myself this as well, and came up with a proof of conecpt
repair tool calle
On 2017-01-21 17:12, Tom Lane wrote:
Erik Rijkers writes:
With the pdf increasingly in readability-decline (more and more text
parts fall off the (right) side of the page and quite a few tables
contain unreable bits) I would like to have a single page html.
Given the size of our manual, I fin
On 1/21/17 12:50 PM, Tom Lane wrote:
> I have to question the decision to make "no locales" a hard error.
> What's the point of that? In fact, should we even be bothering with
> a warning, considering how often initdb runs unattended these days?
Hmm, it was a warning in initdb, so making it an er
On Sat, Jan 21, 2017 at 09:02:25PM +0200, Ants Aasma wrote:
> On Sat, Jan 21, 2017 at 6:41 PM, Andreas Karlsson wrote:
> > It might be worth looking into using the CRC CPU instruction to reduce this
> > overhead, like we do for the WAL checksums. Since that is a different
> > algorithm it would be
-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160
tl;dr +1 from me for changing the default, it is worth it.
Tom Lane wrote:
> Have we seen *even one* report of checksums catching
> problems in a usefuld way?
Sort of chicken-and-egg, as most places don't have it enabled.
Which leads us to:
If the intent of the test is to compare strings, you should be using ne not
!=.
David Bowen
On Sat, Jan 21, 2017 at 10:38 AM, Pavan Deolasee
wrote:
>
>
> On Sat, Jan 21, 2017 at 9:39 PM, Tom Lane wrote:
>
>> Pavan Deolasee writes:
>> > On Sat, Jan 21, 2017 at 9:09 PM, Tom Lane wrote:
>> >> H
* Ants Aasma (ants.aa...@eesti.ee) wrote:
> On Sat, Jan 21, 2017 at 7:39 PM, Petr Jelinek
> wrote:
> > So in summary "postgresql.conf options are easy to change" while "initdb
> > options are hard to change", I can see this argument used both for
> > enabling or disabling checksums by default. As
On Sat, Jan 21, 2017 at 7:39 PM, Petr Jelinek
wrote:
> So in summary "postgresql.conf options are easy to change" while "initdb
> options are hard to change", I can see this argument used both for
> enabling or disabling checksums by default. As I said I would be less
> worried if it was easy to t
On Sat, Jan 21, 2017 at 6:41 PM, Andreas Karlsson wrote:
> On 01/21/2017 04:48 PM, Stephen Frost wrote:
>>
>> * Fujii Masao (masao.fu...@gmail.com) wrote:
>>>
>>> If the performance overhead by the checksums is really negligible,
>>> we may be able to get rid of wal_log_hints parameter, as well.
>
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> Andres Freund writes:
> > Sure, it might be easy, but we don't have it. Personally I think
> > checksums just aren't even ready for prime time. If we had:
> > - ability to switch on/off at runtime (early patches for that have IIRC
> > been posted)
> > -
* Andres Freund (and...@anarazel.de) wrote:
> On 2017-01-21 13:03:52 -0500, Stephen Frost wrote:
> > * Andres Freund (and...@anarazel.de) wrote:
> > > On 2017-01-21 12:46:05 -0500, Stephen Frost wrote:
> > > > Do you run with all defaults in those environments?
> > >
> > > Irrelevant - changing re
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> Stephen Frost writes:
> > Because I see having checksums as, frankly, something we always should
> > have had (as most other databases do, for good reason...) and because
> > they will hopefully prevent data loss. I'm willing to give us a fair
> > bit to m
Andres Freund writes:
> Sure, it might be easy, but we don't have it. Personally I think
> checksums just aren't even ready for prime time. If we had:
> - ability to switch on/off at runtime (early patches for that have IIRC
> been posted)
> - *builtin* tooling to check checksums for everything
On 2017-01-21 13:04:18 -0500, Tom Lane wrote:
> Andres Freund writes:
> > On 2017-01-21 12:46:05 -0500, Stephen Frost wrote:
> >> Do you run with all defaults in those environments?
>
> > Irrelevant - changing requires re-initdb'ing. That's unrealistic.
>
> If you can't turn checksums *off* with
On 2017-01-21 13:03:52 -0500, Stephen Frost wrote:
> * Andres Freund (and...@anarazel.de) wrote:
> > On 2017-01-21 12:46:05 -0500, Stephen Frost wrote:
> > > Do you run with all defaults in those environments?
> >
> > Irrelevant - changing requires re-initdb'ing. That's unrealistic.
>
> I disagre
Stephen Frost writes:
> Because I see having checksums as, frankly, something we always should
> have had (as most other databases do, for good reason...) and because
> they will hopefully prevent data loss. I'm willing to give us a fair
> bit to minimize the risk of losing data.
To be perfectly
* Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote:
> > Do you run with all defaults in those environments?
>
> For initdb? Mostly yes.
Ok, fine, but you probably wouldn't if this change went in.
For me, it's the other way- I have to go enable checksums at initdb time
unless there's an excuse n
Andres Freund writes:
> On 2017-01-21 12:46:05 -0500, Stephen Frost wrote:
>> Do you run with all defaults in those environments?
> Irrelevant - changing requires re-initdb'ing. That's unrealistic.
If you can't turn checksums *off* without re-initdb, that raises the
stakes for this enormously.
On 2017-01-21 12:46:05 -0500, Stephen Frost wrote:
> > I stand by the opinion that changing default which affect performance
> > without any benchmark is bad idea.
>
> I'd be surprised if the performance impact has really changed all that
> much since the code went in. Perhaps that's overly optim
On 21/01/17 18:46, Stephen Frost wrote:
> * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote:
>> As we don't know the performance impact is (there was no benchmark done
>> on reasonably current code base) I really don't understand how you can
>> judge if it's worth it or not.
>
> Because I see ha
Peter Eisentraut writes:
> Add function to import operating system collations
BTW, hamster's been failing since this went in:
running bootstrap script ... ok
performing post-bootstrap initialization ... 2017-01-19 03:17:38.748 JST
[14656] FATAL: no usable system locales were found
2017-01-19 0
* Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote:
> As we don't know the performance impact is (there was no benchmark done
> on reasonably current code base) I really don't understand how you can
> judge if it's worth it or not.
Because I see having checksums as, frankly, something we always s
On 21/01/17 18:15, Stephen Frost wrote:
> * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote:
>> On 21/01/17 17:51, Stephen Frost wrote:
>>> I'm quite sure that the performance numbers for the CREATE TABLE + COPY
>>> case with wal_level=minimal would have been *far* better than for
>>> wal_level >
Andres Freund writes:
> What wouldn't hurt is enabling it by default in pg_regress on master for
> a while. That seems like a good thing to do independent of flipping the
> default.
Yeah, I could get behind that. I'm not certain how much the regression
tests really stress checksumming: most of t
* Andres Freund (and...@anarazel.de) wrote:
> On 2017-01-21 12:09:53 -0500, Tom Lane wrote:
> > Also, if we do decide to do that, there's the question of timing.
> > As I mentioned, one of the chief risks I see is the possibility of
> > false-positive checksum failures due to bugs; I think that cod
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> Stephen Frost writes:
> > As for checksums, I do see value in them and I'm pretty sure that the
> > author of that particular feature did as well, or we wouldn't even have
> > it as an option. You seem to be of the opinion that we might as well
> > just ri
* Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote:
> On 21/01/17 17:51, Stephen Frost wrote:
> > I'm quite sure that the performance numbers for the CREATE TABLE + COPY
> > case with wal_level=minimal would have been *far* better than for
> > wal_level > minimal.
>
> Which is random usecase very
On 2017-01-21 12:09:53 -0500, Tom Lane wrote:
> Also, if we do decide to do that, there's the question of timing.
> As I mentioned, one of the chief risks I see is the possibility of
> false-positive checksum failures due to bugs; I think that code has seen
> sufficiently little field use that we s
Stephen Frost writes:
> As for checksums, I do see value in them and I'm pretty sure that the
> author of that particular feature did as well, or we wouldn't even have
> it as an option. You seem to be of the opinion that we might as well
> just rip all of that code and work out as being useless.
On 21/01/17 17:51, Stephen Frost wrote:
> * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote:
>> On 21/01/17 17:31, Stephen Frost wrote:
>>> This is just changing the *default*, not requiring checksums to always
>>> be enabled. We do not hold the same standards for our defaults as we do
>>> for a
On 2017-01-22 00:41:55 +0900, Fujii Masao wrote:
> On Sun, Jan 22, 2017 at 12:18 AM, Petr Jelinek
> wrote:
> > On 21/01/17 11:39, Magnus Hagander wrote:
> >> Is it time to enable checksums by default, and give initdb a switch to
> >> turn it off instead?
> >
> > I'd like to see benchmark first, bo
On 2017-01-21 11:39:18 +0100, Magnus Hagander wrote:
> Is it time to enable checksums by default, and give initdb a switch to turn
> it off instead?
-1 - the WAL overhead is quite massive, and in contrast to the other
GUCs recently changed you can't just switch this around.
Andres
--
Sent via
* Andreas Karlsson (andr...@proxel.se) wrote:
> On 01/21/2017 04:48 PM, Stephen Frost wrote:
> >* Fujii Masao (masao.fu...@gmail.com) wrote:
> >>If the performance overhead by the checksums is really negligible,
> >>we may be able to get rid of wal_log_hints parameter, as well.
> >
> >Prior benchma
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> Stephen Frost writes:
> > * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote:
> >> The change of wal_level was supported by benchmark, I think it's
> >> reasonable to ask for this to be as well.
>
> > No, it wasn't, it was that people felt the cases where
* Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote:
> On 21/01/17 17:31, Stephen Frost wrote:
> > This is just changing the *default*, not requiring checksums to always
> > be enabled. We do not hold the same standards for our defaults as we do
> > for always-enabled code, for clear reasons- not
Stephen Frost writes:
> * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote:
>> The change of wal_level was supported by benchmark, I think it's
>> reasonable to ask for this to be as well.
> No, it wasn't, it was that people felt the cases where changing
> wal_level would seriously hurt performa
On 01/21/2017 04:48 PM, Stephen Frost wrote:
* Fujii Masao (masao.fu...@gmail.com) wrote:
If the performance overhead by the checksums is really negligible,
we may be able to get rid of wal_log_hints parameter, as well.
Prior benchmarks showed it to be on the order of a few percent, as I
recal
On 21/01/17 17:31, Stephen Frost wrote:
> Petr,
>
> * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote:
>> On 21/01/17 16:40, Stephen Frost wrote:
>>> * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote:
On 21/01/17 11:39, Magnus Hagander wrote:
> Is it time to enable checksums by defaul
On Sat, Jan 21, 2017 at 9:39 PM, Tom Lane wrote:
> Pavan Deolasee writes:
> > On Sat, Jan 21, 2017 at 9:09 PM, Tom Lane wrote:
> >> Hm, but what of the "null" value? Also, I get
> >>
> >> $ perl -e 'use warnings; use Test::More; ok("2017-01-01" != "null",
> "ok");'
> >> Argument "null" isn't n
Stephen Frost writes:
> * Tom Lane (t...@sss.pgh.pa.us) wrote:
>> Have we seen *even one* report of checksums catching problems in a useful
>> way?
> This isn't the right question.
I disagree. If they aren't doing something useful for people who have
turned them on, what's the reason to think t
Petr,
* Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote:
> On 21/01/17 16:40, Stephen Frost wrote:
> > * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote:
> >> On 21/01/17 11:39, Magnus Hagander wrote:
> >>> Is it time to enable checksums by default, and give initdb a switch to
> >>> turn it of
On 21/01/17 16:40, Stephen Frost wrote:
> Petr,
>
> * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote:
>> On 21/01/17 11:39, Magnus Hagander wrote:
>>> Is it time to enable checksums by default, and give initdb a switch to
>>> turn it off instead?
>>
>> I'd like to see benchmark first, both in t
Erik Rijkers writes:
> With the pdf increasingly in readability-decline (more and more text
> parts fall off the (right) side of the page and quite a few tables
> contain unreable bits) I would like to have a single page html.
Given the size of our manual, I find it hard to believe that most
re
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> Magnus Hagander writes:
> > Is it time to enable checksums by default, and give initdb a switch to turn
> > it off instead?
>
> Have we seen *even one* report of checksums catching problems in a useful
> way?
This isn't the right question.
The right ques
Pavan Deolasee writes:
> On Sat, Jan 21, 2017 at 9:09 PM, Tom Lane wrote:
>> Hm, but what of the "null" value? Also, I get
>>
>> $ perl -e 'use warnings; use Test::More; ok("2017-01-01" != "null", "ok");'
>> Argument "null" isn't numeric in numeric ne (!=) at -e line 1.
>> Argument "2017-01-01"
On Sat, Jan 21, 2017 at 9:09 PM, Tom Lane wrote:
>
>
> Hm, but what of the "null" value? Also, I get
>
> $ perl -e 'use warnings; use Test::More; ok("2017-01-01" != "null", "ok");'
> Argument "null" isn't numeric in numeric ne (!=) at -e line 1.
> Argument "2017-01-01" isn't numeric in numeric n
Magnus Hagander writes:
> Is it time to enable checksums by default, and give initdb a switch to turn
> it off instead?
Have we seen *even one* report of checksums catching problems in a useful
way?
I think this will be making the average user pay X% for nothing.
regards
* Fujii Masao (masao.fu...@gmail.com) wrote:
> On Sun, Jan 22, 2017 at 12:18 AM, Petr Jelinek
> wrote:
> > On 21/01/17 11:39, Magnus Hagander wrote:
> >> Is it time to enable checksums by default, and give initdb a switch to
> >> turn it off instead?
> >
> > I'd like to see benchmark first, both i
On Sun, Jan 22, 2017 at 12:18 AM, Petr Jelinek
wrote:
> On 21/01/17 11:39, Magnus Hagander wrote:
>> Is it time to enable checksums by default, and give initdb a switch to
>> turn it off instead?
>
> I'd like to see benchmark first, both in terms of CPU and in terms of
> produced WAL (=network tra
Petr,
* Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote:
> On 21/01/17 11:39, Magnus Hagander wrote:
> > Is it time to enable checksums by default, and give initdb a switch to
> > turn it off instead?
>
> I'd like to see benchmark first, both in terms of CPU and in terms of
> produced WAL (=net
Pavan Deolasee writes:
> I think I understand why it's only affecting me and not others.
> I've PGDATESTYLE set to "Postgres, MDY" in my bashrc and that formats the
> commit timestamp as "Fri Jan 20 07:59:52.322811 2017 PST". If I unset that,
> the result comes in a format such as "2017-01-20 21:
On 21/01/17 11:39, Magnus Hagander wrote:
> Is it time to enable checksums by default, and give initdb a switch to
> turn it off instead?
I'd like to see benchmark first, both in terms of CPU and in terms of
produced WAL (=network traffic) given that it turns on logging of hint bits.
--
Petr J
On 21/01/17 01:34, Thomas Munro wrote:
> Hi,
>
> In src/backend/replication/logical/worker.c:
>
> pq_sendbyte(reply_message, 'r');
> pq_sendint64(reply_message, recvpos); /* write */
> pq_sendint64(reply_message, flushpos); /* flush */
> pq_sendint64(reply_message, wri
Magnus,
* Magnus Hagander (mag...@hagander.net) wrote:
> On Sat, Jan 21, 2017 at 3:05 PM, Michael Paquier
> wrote:
>
> > On Sat, Jan 21, 2017 at 7:39 PM, Magnus Hagander
> > wrote:
> > > Is it time to enable checksums by default, and give initdb a switch to
> > turn
> > > it off instead?
> > >
* Michael Paquier (michael.paqu...@gmail.com) wrote:
> On Sat, Jan 21, 2017 at 7:39 PM, Magnus Hagander wrote:
> > Is it time to enable checksums by default, and give initdb a switch to turn
> > it off instead?
> >
> > I keep running into situations where people haven't enabled it, because (a)
> >
On Sat, Jan 21, 2017 at 3:05 PM, Michael Paquier
wrote:
> On Sat, Jan 21, 2017 at 7:39 PM, Magnus Hagander
> wrote:
> > Is it time to enable checksums by default, and give initdb a switch to
> turn
> > it off instead?
> >
> > I keep running into situations where people haven't enabled it, becaus
On Sat, Jan 21, 2017 at 7:39 PM, Magnus Hagander wrote:
> Is it time to enable checksums by default, and give initdb a switch to turn
> it off instead?
>
> I keep running into situations where people haven't enabled it, because (a)
> they didn't know about it, or (b) their packaging system ran ini
* Magnus Hagander (mag...@hagander.net) wrote:
> Is it time to enable checksums by default, and give initdb a switch to turn
> it off instead?
Yes, please.
We've already agreed to make changes to have a better user experience
and ask those who really care about certain performance aspects to have
Hello Jeff!
>Review of the code itself:
How do you think, is there anything else to improve in that patch or
we can mark it as 'Ready for committer'?
I've checked the concurrency algorithm with original work of Lehman
and Yao on B-link-tree. For me everything looks fine (safe, deadlock
free and n
Hi,
With the pdf increasingly in readability-decline (more and more text
parts fall off the (right) side of the page and quite a few tables
contain unreable bits) I would like to have a single page html. (cf the
bash scipting guide at http://tldp.org/guides.html even if that is
smaller than
Is it time to enable checksums by default, and give initdb a switch to turn
it off instead?
I keep running into situations where people haven't enabled it, because (a)
they didn't know about it, or (b) their packaging system ran initdb for
them so they didn't even know they could. And of course th
74 matches
Mail list logo
