inful MultiXact problems) will find the relminmxid thing very
compelling.
--
Peter Geoghegan
>
> Okay, thanks Bruce.
--
Peter Geoghegan
pporting AIX will
happen (say) at some time in the next decade, no matter what happens
today.
--
Peter Geoghegan
diately. Obviously these users will be able to use a
supported version of Postgres for several more years.
--
Peter Geoghegan
>
> I don't think this is an appropriate detail for the release notes.
Okay. What about saying something about relminmxid advancement where
the database consumes lots of multixacts?
--
Peter Geoghegan
en with features that we think of
as high impact major features. So evidently users really value these
kinds of quality of life improvements.
--
Peter Geoghegan
E 1: ...lict (bar) do update set bar = excluded.bar where barr != 5;
^
HINT: Perhaps you meant to reference the column "foo.bar" or the
column "excluded.bar".
--
Peter Geoghegan
lved here, since EXCLUDED is the thing that creates the ambiguity,
but that seems almost incidental to me. This user would probably not
have been confused if they didn't need to use a WHERE clause (very
much the common case), even when expression evaluation involving
EXCLUDED in the SET was still used (also common).
--
Peter Geoghegan
upsert. MySQL's INSERT
... ON DUPLICATE KEY UPDATE feature has a magical UPSERT-only
expression instead of "excluded".
--
Peter Geoghegan
to an from_item
alias used within an UPDATE FROM ... where the target table is
also the from_item table (i.e. there is a self-join). There is only
one table involved.
--
Peter Geoghegan
in Postgres 15, for the
first time -- even though that's very basic information. The revised
definition of scanned_pages enabled that enhancement. We are no longer
encumbered by the existence of a no-cleanup-lock-page special case.
--
Peter Geoghegan
at's the problem with not reporting on that?
--
Peter Geoghegan
On Tue, Jun 28, 2022 at 1:35 PM Bruce Momjian wrote:
> Okay, text updated, thanks. Applied patch attached.
I have some notes on these items:
1. "Allow vacuum to be more aggressive in setting the oldest frozenxid
(Peter Geoghegan)"
2. "Add additional information
oints via a PRNG. During CREATE TABLE, for
example. This approach could make it easier to reproduce failures on the
buildfarm.
--
Peter Geoghegan
today, but you can't expect to avoid freezing
altogether (without significant work elsewhere). My general sense is
that freezing isn't a particularly good thing to try to do lazily --
even if we ignore the risk of an eventual wraparound failure.
--
Peter Geoghegan
m tuplesorts for
'SELECT(DISTINCT mycol) ...' style queries on low cardinality columns
extremely fast. We're not really sorting so much as bucketing. This is
based on Dijkstra's Dutch national flag problem.
--
Peter Geoghegan
On Wed, Jun 22, 2022 at 8:45 AM Bruce Momjian wrote:
> That is a big help for committers who want to email a URL of the new
> doc commit output.
Yes, it's a nice "quality of life" improvement.
Thanks for finally getting this done!
--
Peter Geoghegan
n working on B-Tree deduplication).
It's quite straightforward to set up and run.
[1] http://smalldatum.blogspot.com/2017/06/the-insert-benchmark.html
--
Peter Geoghegan
oesn't matter much now (since you have all but conceded
that using a bit for this makes sense), but FWIW that's the main
reason why I almost took it for granted that we'd need to use a status
bit (or bits) for this.
--
Peter Geoghegan
On Tue, Jun 14, 2022 at 7:21 PM Robert Haas wrote:
> On Tue, Jun 14, 2022 at 9:56 PM Peter Geoghegan wrote:
> > Technically we don't already do that today, with the 16-bit checksums
> > that are stored in PageHeaderData.pd_checksum. But we do something
> > equivalent: low-
ch seems absolutely fine. They *definitely*
have no value if nobody ever uses them for anything.
--
Peter Geoghegan
On Tue, Jun 14, 2022 at 1:32 PM Peter Geoghegan wrote:
> On Tue, Jun 14, 2022 at 1:22 PM Robert Haas wrote:
> > I still am not clear on precisely what you are proposing here. I do
> > agree that there is significant bit space available in pd_flags and
> > that consumin
, as you
have suggested. We're still discussing new approaches to checksumming
in the scope of this work, which of course includes many individual
cases that don't involve any encryption. Plus even with encryption
there are things like defensive assertions that can be added by using
a flag bit for this.
--
Peter Geoghegan
for performance reasons
(though it's unclear how much it matters). In which case the status
bit is technically redundant information as far as the code is
concerned. That may well be fine.
--
Peter Geoghegan
it will in fact affect a lot of callers to the
bufpage.h functions.
--
Peter Geoghegan
in which that assumption no longer
holds. Like when you do finally get around to making TDE something
that can work at the relation level, for example. Even if there is
only a small chance of that ever happening, why wouldn't we be
prepared for it, just on general principle?
--
Peter Geoghegan
pg_upgrade? I would argue that this would
be the first, since it is the moral equivalent of extending the size
of the generic page header.
For all I know the overhead will be perfectly fine, and everybody
wins. I just want to be adamant that we're making the right
trade-offs, and maximizing the benefit from any new cost imposed on
access method code.
--
Peter Geoghegan
ion selectively, perhaps at the
relation level?
--
Peter Geoghegan
On Tue, Jun 14, 2022 at 8:48 AM Robert Haas wrote:
> On Mon, Jun 13, 2022 at 6:26 PM Peter Geoghegan wrote:
> > Anyway, I can see how it would be useful to be able to know the offset
> > of a nonce or of a hash digest on any given page, without access to a
> > running serv
gt; I wasn't aware myself of all the gotchas described there.
I didn't realize that it was that bad. Even if it's only 10% as bad as
you say, it would still be very valuable to do something about it
(ideally with an approach that is non-invasive).
--
Peter Geoghegan
arer in a doc patch that recently became
commit 8ac700ac.
--
Peter Geoghegan
ystem lib C to be authoritative for the OS as a whole, in
the way that Postgres supposes. At least in the case of Mac OS, which
is after all purely a desktop operating system.
--
Peter Geoghegan
make
all the difference.
--
Peter Geoghegan
compose with other things?
--
Peter Geoghegan
On Mon, Jun 13, 2022 at 2:54 PM Bruce Momjian wrote:
> On Mon, Jun 13, 2022 at 02:44:41PM -0700, Peter Geoghegan wrote:
> > Is that the how block-level encryption feature from EDB Advanced Server
> > does it?
>
> Uh, EDB Advanced Server doesn't have a block-level encryption
e checksum that's going to tell you
> whether the page contents are messed up. Perhaps this could be worked
> around if you tried hard enough, but I don't see what we get out of
> it.
Is that the how block-level encryption feature from EDB Advanced Server does it?
--
Peter Geoghegan
n't forced to use the same ancient ICU
version.
--
Peter Geoghegan
surely any related indexes must have been rebuilt too. The
interesting part may be what that upgrade looks like in detail.
--
Peter Geoghegan
k out of it once it begins, but mostly
it's an inflexible process that just does what we need it to do.
Does something like that seem sensible to you?
--
Peter Geoghegan
ning the problem as a problem with old
indexes/constraints only seems like it might make things a lot easier.
--
Peter Geoghegan
ustly
> and simply implementable. Hmm.
That may be a decisive reason to go with your proposal. I really don't know.
--
Peter Geoghegan
r their logical collation). So
directly tackling that seems natural to me.
--
Peter Geoghegan
irection
of decoupling the concerns that we have as database people from the
concerns of natural language experts. Let's not step on their toes,
and let's avoid having our toes trampled on.
--
Peter Geoghegan
ustom collation for
"natural sorting", often used for things like alphanumeric invoice
numbers. That might break if you downgrade ICU version, and maybe even
if you upgrade ICU version.
--
Peter Geoghegan
tation detail, perhaps only used to
unambiguously identify which specific ICU version and locale string
relate to which on-disk relfilenode structure currently.
--
Peter Geoghegan
On Thu, Jun 9, 2022 at 2:33 PM Peter Geoghegan wrote:
> My preference is for an approach that builds on that, or at least
> doesn't significantly complicate it. So a cryptographic hash or nonce
> can go in the special area proper (structs like BTPageOpaqueData don't
> need
nges), but at a page offset before the special area proper
-- not after.
What disadvantages does that approach have, if any, from your point of view?
--
Peter Geoghegan
atively common.
That's what the search_path case seems like to me.
If somebody else wants to write another patch that adds on that,
great. If not, then having this much still seems useful.
--
Peter Geoghegan
store the index metapage. (Actually unlogged
indexes that run on a standby don't, but that's accounted for
directly.)
--
Peter Geoghegan
not to do.
--
Peter Geoghegan
t;compelling feature" in its own right.
I believe that EDB adopted ICU many years ago, and stuck with one
vendored version for quite a few years. And eventually being on a very
old version of ICU became a real problem.
--
Peter Geoghegan
On Thu, Jun 9, 2022 at 6:40 AM Robert Haas wrote:
> Are you going to code up a patch?
I can, but feel free to fix it yourself if you prefer. Your analysis
seems sound.
--
Peter Geoghegan
On Wed, Jun 8, 2022 at 10:39 PM Peter Geoghegan wrote:
> They simply REINDEX, without changing anything. The details are still
> fuzzy, but at least that's what I was thinking of.
As I said before, BCP47 format tags are incredibly forgiving by
design. So it should be reasonable to
(which is likely), you can avoid reindexing,
and so reserve the option of backing out of a complex upgrade until
very late in the process. You're going to have to do it eventually,
but it can probably just be an afterthought.
--
Peter Geoghegan
cessitates thinking about
multiple evaluation hazards, which is enough to discourage good
defensive coding practices.
--
Peter Geoghegan
So the limit on internal pages never
changed, even in Postgres 12. There was no separate leaf page limit
prior to 12. Only the rules on the leaf level ever really changed.
Note also that amcheck has tests for this stuff. Though that probably
doesn't matter at all.
--
Peter Geoghegan
s or more to run the
test.
I think that we should fix this on HEAD, on general principle. There
is no reason to believe that this is a live bug, so a backpatch seems
unnecessary.
--
Peter Geoghegan
h to implementing strxfrm() is another example of the same
thing. (The Apple strxfrm() produces huge low entropy binary strings,
unlike the glibc version, which is pretty well optimized.)
--
Peter Geoghegan
ance all-rounder that can still be
beaten by specialized techniques that come with their own downsides.
--
Peter Geoghegan
ant to generalize the definition of a collation, so
that it's associated with one particular ICU version and collation for
the purposes of on-disk compatibility, but isn't necessarily tied to
the same ICU version in other contexts, such as on a dump and restore?
--
Peter Geoghegan
hen I can easily think of far better
examples of software bloat. Defining "stable behavior for collations"
as "uses exactly the same software artifact over time" is defensive
(compared to always linking to one ICU version that does it all), but
we have plenty that we need to defend against here.
--
Peter Geoghegan
throwing a "locale not
found" error message usually isn't helpful from the point of view of
the end user. Note that this is a broader standard than ICU or CLDR or
even Unicode.
[1] https://www.ietf.org/rfc/rfc6067.txt
--
Peter Geoghegan
ect job of versioning, I still think that their priorities are
very much unlike our priorities, and that that should be a relevant
consideration for us.
--
Peter Geoghegan
still be something that we could influence. We could have a back
and forth conversation about what's not working in the field.
--
Peter Geoghegan
ng its use in new databases). This isn't the same
thing as forking ICU. It's a compromise between that extreme, and
the current situation.
--
Peter Geoghegan
ever get to hear about already. That's rather
beside the point. The index must be assumed to be corrupt.
--
Peter Geoghegan
ex-only scans that scan a significant fraction of the entire index,
or even the entire index.
--
Peter Geoghegan
es of scalar types.
The abbreviated keys optimization is very much something that comes
from the world of databases, not the world of sorting. It's pretty much a
domain-specific technique. That seems relevant to me.
--
Peter Geoghegan
otQuicksort.pdf
At one point quite a few years back I planned on investigating it
myself, but never followed through.
--
Peter Geoghegan
hat's pretty standard practice when summarizing a set of
benchmark results that are expressed as ratios to some baseline.
If I tweak your spreadsheet to use the geometric mean, the patch looks
slightly better -- 89%.
--
Peter Geoghegan
On Fri, May 27, 2022 at 11:59 AM Andres Freund wrote:
> On 2022-05-27 11:48:45 -0700, Peter Geoghegan wrote:
> > I find it hard to believe that there wasn't even a cursory effort at
> > performance validation before this was committed, but that's what it
> > looks like.
>
nything useful! There's not a
> single benchmark for the patch.
I find it hard to believe that there wasn't even a cursory effort at
performance validation before this was committed, but that's what it
looks like.
--
Peter Geoghegan
On Thu, May 19, 2022 at 1:12 PM Justin Pryzby wrote:
> Should these debug lines be removed ?
>
> elog(DEBUG1, "qsort_tuple");
I agree -- DEBUG1 seems too chatty for something like this. DEBUG2
would be more appropriate IMV. Though I don't feel very strongly about
it.
--
Peter Geoghegan
st obviously false.
+1
--
Peter Geoghegan
which boil down to "the current FSM design
cannot be totally trusted, so we verify redundantly".
--
Peter Geoghegan
ks, where Postgres does entirely the wrong thing.
It's especially interesting to me as somebody that focuses on B-Tree
indexing.
--
Peter Geoghegan
ible to overlook), then I
would object -- why even take a small chance? Fortunately I don't
believe that we're even taking a small chance here, all things
considered. And so I agree; this issue isn't a concern.
--
Peter Geoghegan
On Thu, Apr 21, 2022 at 4:28 PM Peter Geoghegan wrote:
> I don't think that there is any risk of one user of either variable
> "clobbering" some other user -- the current values of the variables
> are not actually meaningful at all. They're only useful as a way that
> an a
On Wed, Apr 20, 2022 at 8:00 PM Peter Geoghegan wrote:
> I knew about pgBufferUsage, and I knew about
> VacuumPage{Hit,Miss,Dirty} for a long time. But somehow I didn't make
> the very obvious connection between the two until today. I am probably
> not the only one.
added the others too?
I knew about pgBufferUsage, and I knew about
VacuumPage{Hit,Miss,Dirty} for a long time. But somehow I didn't make
the very obvious connection between the two until today. I am probably
not the only one.
--
Peter Geoghegan
On Tue, Apr 12, 2022 at 11:01 AM Peter Geoghegan wrote:
> Attached patch fixes the issue, and includes the test case that you posted.
Pushed a similar patch just now. Backpatched to all supported branches.
--
Peter Geoghegan
ields like
pgBufferUsage.shared_blks_hit (i.e. those that don't have a
time/duration component) officially okay to rely on across the board?
It looks like they are okay to rely on (even when track_io_timing is
off), but it would be nice to put that on a formal footing, if it
isn't already.
--
Peter Geoghegan
ith using DBT5 on a modern
Linux distribution. Perhaps I gave up too easily at the time, but I'm
definitely still interested. Has there been work on that since?
Thanks
--
Peter Geoghegan
e
spec.
--
Peter Geoghegan
On Mon, Apr 18, 2022 at 1:12 PM Peter Geoghegan wrote:
> I would argue that it would be correct for the first time -- at least
> if we take the behavior within heapam_index_build_range_scan (and
> everywhere else) as authoritative. That's a feature, not a bug.
Attached draft patch sho
would argue that it would be correct for the first time -- at least
if we take the behavior within heapam_index_build_range_scan (and
everywhere else) as authoritative. That's a feature, not a bug.
--
Peter Geoghegan
ue
in the amvacuumcleanup path (instead of new_rel_tuples). That way the
rule about IndexVacuumInfo.num_heap_tuples is simple: it's always
taken from pg_class.reltuples (for the heap rel). Either the existing
value, or the new value.
--
Peter Geoghegan
es, which is related. Granted,
that won't be used to update pg_class for the index in the case where
it's just an estimate anyway.
--
Peter Geoghegan
.reltuples is -1 even after a
VACUUM" case is completely impossible following the Postgres 15 work
on VACUUM, but we should still clamp for safety in
update_relstats_all_indexes (though not in the amvacuumcleanup path).
--
Peter Geoghegan
oesn't affect "dead but not yet removable".
But OldestXmin affects FreezeLimit.
Anyway, I'm not opposed to showing the age at the start as well. But
from the point of view of issues like this tenk1 issue, it would be
more useful to just report on new_rel_allvisible. It would also be
more useful to users.
--
Peter Geoghegan
FreezeLimit makes little sense.
--
Peter Geoghegan
ly in the kinds of extreme cases I'm thinking about.
--
Peter Geoghegan
currently running.
As well as the age of OldestXmin at the start of VACUUM.
--
Peter Geoghegan
though. Your new wording is fine.
I'll update the log output some time today.
--
Peter Geoghegan
les of both. This could easily be changed to "XIDs".
--
Peter Geoghegan
nd did so in the
script as well.
That seems like it definitely could matter.
--
Peter Geoghegan
nner on here). Just a wild guess at this point.
--
Peter Geoghegan
yncronous_commit thing was new
information that made that worth revisiting.
--
Peter Geoghegan
that effect, whose
commit fits with our timeline for the problems seen on wrasse?
--
Peter Geoghegan
On Thu, Apr 14, 2022 at 3:28 PM Peter Geoghegan wrote:
> A bunch of autovacuums that ran between "2022-04-14 22:49:16.274" and
> "2022-04-14 22:49:19.088" all have the same "removable cutoff".
Are you aware of Andres' commit 02fea8fd? That work preven
801 - 900 of 3084 matches
Mail list logo
