> 25 июля 2022 г., в 14:29, Bharath Rupireddy
> написал(а):
>
> Hm, after thinking for a while, I tend to agree with the above
> approach - meaning, query cancel interrupt processing can completely
> be disabled in SyncRepWaitForLSN() and process proc die interrupt
> immediately, this
> On 26 Jun 2022, at 00:10, Andrey Borodin wrote:
>
> I will split the patch in 3 steps:
> 1. extract generic functions to amcheck.c
> 2. add gist functions
> 3. add gin functions
>
> I'll fix other notes too in the next version.
Done. PFA attached patchset.
Thank
in 16?
Best regards, Andrey Borodin.
you think?
Best regards, Andrey Borodin.
le only
holding shared lock? I think it is, per Alexander's comment, but maybe let's
document it?
Thank you!
Thanks! Best regards, Andrey Borodin.
> On 29 Jun 2022, at 23:07, Justin Pryzby wrote:
>
> On Wed, Jun 29, 2022 at 10:58:44PM +0500, Andrey Borodin wrote:
>>> On 28 Jun 2022, at 04:30, Justin Pryzby wrote:
>>>
>>> Nope, it's as I said: this would break pg_upgrade from older version
routinely.
Besides this the patch seems to be RfC.
Best regards, Andrey Borodin.
!
> On 23 Jun 2022, at 04:29, Andres Freund wrote:
Thanks for looking into the patch, Andres!
> On 2022-06-22 20:40:56 +0300, Andrey Borodin wrote:
>>> diff --git a/contrib/amcheck/amcheck.c b/contrib/amcheck/amcheck.c
>> new file mode 100644
>> index 00..7a
superuser.
Best regards, Andrey Borodin.
eeded too:
// "AND aggtranstype='anyarray'::regtype
Thank you!
Best regards, Andrey Borodin.
on about found objects. For example,
operator will look like "operator: ||". At least we can get nspname and oid.
And, maybe return type for aggregator and leftarg\rightarg types for operator?
BTW comment /* Before v11, used proisagg=true, and afterwards uses prokind='a'
*/ seems interesting, but irrelevant. We join with pg_aggregate anyway.
Thanks!
Best regards, Andrey Borodin.
> On 24 Jun 2022, at 18:30, Justin Pryzby wrote:
>
> On Fri, Jun 24, 2022 at 04:09:46PM +0500, Andrey Borodin wrote:
>> Hi everyone!
>>
>> Sorry for bumping old thread.
>
> Please find this newer thread+patch here ;)
> https://www.postgresql.org/message-id/
> On 24 Jun 2022, at 16:09, Andrey Borodin wrote:
>
> Would it be backpatchable to add this check to pg_upgrade?
Just to be clear of what exactly I propose I drafted a patch. PFA.
I've tested it with PG13 and
CREATE AGGREGATE public.array_accum(anyelement) (
SFUNC = arr
, 'array_prepend','array_cat',
'array_position','array_positions', 'array_remove', 'array_replace',
'width_bucket') and pg_operator.oid >= 16384;
if pg_upgrade is executed with --check option.
Best regards, Andrey Borodin.
16 pages. And I believe 128KB to be optimal for most storages. And
having no GUCs sounds great.
But storage systems might be different, far beyond benchmarks.
All in all, I don't have strong opinion on having 1 or 0 GUCs to configure this.
I've added patch to the CF.
Thanks!
Best regards, Andrey Borodin.
> On 30 May 2022, at 12:40, Andrey Borodin wrote:
>
> What do you think?
Hi Andrey!
Here's a version with better tests. I've made sure that GiST tests actually
trigger page reuse after deletion. And enhanced comments in both GiST and GIN
test scripts. I hope you'll like
Best regards, Andrey Borodin.
ld you please elaborate how have you got the results with that big
> difference? (Though I don't against fadvise usage, at worst it is expected to
> be useless).
FWIW we with Kirill observed drastically reduced lag on a production server
when running patched version. Fidvise surely works :) The question is how to
use it optimally.
Best regards, Andrey Borodin.
f defined(USE_POSIX_FADVISE) && defined(POSIX_FADV_WILLNEED)
if ((readOff % (8 * XLOG_BLCKSZ)) == 0)
posix_fadvise(readFile, readOff + XLOG_BLCKSZ, XLOG_BLCKSZ * 8,
POSIX_FADV_WILLNEED);
#endif
and maybe define\reuse the some GUC to control number of prefetched pages at
once.
Best regards, Andrey Borodin.
OS readahead affects index scan performance.
Best regards, Andrey Borodin.
g to solve the problem of
reading WAL segments that are our of OS page cache.
Best regards, Andrey Borodin.
ypes support.
On a quick glance your version lacks support of abbreviated sort, so I think
benchmarks can be pushed event further :)
Let's merge our efforts and create combined patch?
Please, create a new entry for the patch on Commitfest.
Thank you!
Best regards, Andrey Borodin
hecksum. It's much
easier to obtain fresh block copy from HA standby.
Best regards, Andrey Borodin.
it will not deadlock, because split is doing the same locking.
What do you think?
Best regards, Andrey Borodin.
[0]
https://www.postgresql.org/message-id/flat/CAF3eApa07-BajjG8%2BRYx-Dr_cq28ZA0GsZmUQrGu5b2ayRhB5A%40mail.gmail.com
[1]
https://www.postgresql.org/message-id/flat/9fdbb584-1e10-6a55
ere standby is promoted you
definitely want infinite wait for cancels. Yet once upon a time you want to
shutdown postgres without coredump - thus proc die needs to be processed.
Thanks!
Best regards, Andrey Borodin.
inite until synchrous_commit\synchronous_standby_names are satisfied )
And yes, we need additional complexity - but in some other place. Transaction
can also be locally committed in presence of a server crash. But this another
difficult problem. Crashed server must not allow data queries until LSN of
timeline end is successfully replicated to synchronous_standby_names.
Best regards, Andrey Borodin.
ver analyzed table.
But now it is working just as you would expect. There's an interesting relevant
discussion linked to the commit message.
Best regards, Andrey Borodin.
[0]
https://github.com/postgres/postgres/commit/3d351d916b20534f973eda760cde17d96545d4c4
ang
version 11.0.3 (clang-1103.0.32.62), 64-bit
(1 row)
Without "ANALYZE test1;" table_block_relation_estimate_size() assumes relation
size is 10 blocks.
Best regards, Andrey Borodin.
> 25 апр. 2022 г., в 21:48, Nathan Bossart
> написал(а):
>
> I'm personally in
> favor of just adding a GUC that can be enabled to block canceling
> synchronous replication waits
+1. I think it's the only option to provide quorum commit guarantees.
Best regards, Andrey Borodin.
> 22 апр. 2022 г., в 19:15, Bharath Rupireddy
> написал(а):
>
> On Sat, Apr 9, 2022 at 10:21 PM Robert Haas wrote:
>>
>> On Sat, Apr 9, 2022 at 12:25 PM Andrey Borodin wrote:
>>> Please excuse me if I'm not attentive enough. I've read this threa
during TLI
switch.
Either way we can rename the function to
pg_walfile_name_as_if_on_timeline_of_last_wal_replayed().
Thanks!
Best regards, Andrey Borodin.
r input besides reviews and Andres's -0.2.
Is there a chance to proceed?
Best regards, Andrey Borodin.
r to understand.
Thanks! Best regards, Andrey Borodin.
v4-0001-Use-linked-list-to-improve-KnownAssignedXids-perf.patch
Description: Binary data
s a must-have” referring
to real pathological performance degradation inflicted by SLRU cache
starvation. And I can remember dozen of other incidents that would not happen
if the patch was applied, e.g. this post is referring to the patch as a cure
[1].
Best regards, Andrey Borodin.
[0]
https://w
PG version it will land.
And there is 100% consensus that first patch is useful and helps to prevent big
issues. Maybe let's commit 1'st step without lifting default xact_buffers
limit? Or 1st patch as-is with any simple technique that prevents linear search
in SLRU buffers.
Best regards, Andrey Borodin.
while Zstd might make things slower.
Thanks!
Best regards, Andrey Borodin.
n error" [0]. Thank you!
Now it’s obvious that this is not a real data corruption. Maybe let’s remove
corruption error code from the error? I had been literally woken at night by
this code few times in January.
And do you have a plan how to fix the actual issue?
Best regards, And
> 19 янв. 2022 г., в 09:31, Andrey Borodin написал(а):
>>
>> I've addressed Andrey Borodin's concerns about v2 of this patch by Aliaksandr
>> Kalenik in attached version.
>
> Thank you! I'll make a new iteration of review. From a first glan
the name BufTableFreeDeleted() confuses me a bit. You know, in C we usually
free(), but in C++ we delete [], and here we do both... Just to be sure.
Thanks!
Best regards, Andrey Borodin.
> 21 янв. 2022 г., в 05:19, Shawn Debnath написал(а):
>
> On Thu, Jan 20, 2022 at 09:21:24PM +0500, Andrey Borodin wrote:
>> CAUTION: This email originated from outside of the organization. Do not
>> click links or open attachments unless you can confirm the sender and
tended to do make thing better on default buffer sizes. It
must save the perfromance in case of really huge SLRU buffers.
Thanks!
Best regards, Andrey Borodin.
ything
looks good, but gist_sorted_build_page_buffer_size haven't any documentation....
Best regards, Andrey Borodin.
sk...
Thank you for raising this topic, I think I should rebase and refresh that
patch too...
Best regards, Andrey Borodin.
[0]
https://www.postgresql.org/message-id/flat/7780A07B-4D04-41E2-B228-166B41D07EEE%40yandex-team.ru
!
Best regards, Andrey Borodin.
s the latest version.
>
> This is failing to compile under linux and windows due to bitfield syntax.
> http://cfbot.cputube.org/andrey-borodin.html
Uh, sorry, I formatted a patch from wrong branch.
Just tested Cirrus. It's wonderful, thanks! Really faster than doing stuff on
my machines...
Bes
on Author.
>>
>
> I was planning on running a set of stress tests on these patches. Could
> we confirm which ones we plan to include in the commitfest?
Many thanks for your interest. Here's the latest version.
Best regards, Andrey Borodin.
v19-0003-Pack-SLRU-page_number-page_sta
erent on Standby. E.g.
default_transaction_isolation [1]. This is expected behaviour, AFAIK.
Thanks!
Best regards, Andrey Borodin.
[0] https://www.postgresql.org/docs/current/transaction-iso.html
[1] https://www.postgresql.org/docs/current/runtime-config-client.html
ed previously and addressed.
What else do we need to decide architecturally to make protocol compression
happen in 15? As far as I can see - only HBA\GUC part.
Best regards, Andrey Borodin.
this before on clusters with clean amcheck and heapcheck results.
Alvaro, did you observe this on binaries from August 13 minor release or older?
Thanks!
Best regards, Andrey Borodin.
> Andrey Borodin writes:
>
>> I've tried to toy with the patch and remembered one related caveat.
>> If we have index for both returnable and nonreturnable attributes, IOS will
>> not be choosen:
>
>> postgres=# create index on t using gist(a gist_t
to better compress current bytes.
Best regards, Andrey Borodin.
But maybe we would like to have equal behavior
in both cases...
Thanks!
Best regards, Andrey Borodin.
page_statuses, but from my POV here complexity does not worth possible
performance gain.
PFA rebase of the patchset. Also I've added a patch to combine page_number,
page_status, and page_dirty together to touch less cachelines.
Best regards, Andrey Borodin.
From ea59d2ebde818ddc2a9111858b3d956cbcc7
GistSortedBuildPageState is no more page state. It's Level state or
something like that.
6. The patch desperately needs comments.
Thanks!
Best regards, Andrey Borodin.
[0]
https://www.postgresql.org/message-id/flat/59D0DA6B-1652-4D44-B0EF-A582D5824F83%40yandex-team.ru
gist (point );
CREATE INDEX
Time: 1973,248 ms (00:01,973)
postgres=# create index ON x using gist (point );
CREATE INDEX
Time: 1970,578 ms (00:01,971)
Thanks!
Best regards, Andrey Borodin.
> 30 нояб. 2021 г., в 17:19, Simon Riggs
> написал(а):
>
> On Mon, 30 Aug 2021 at 11:25, Andrey Borodin wrote:
>>
>> Hi Pengcheng!
>>
>> You are solving important problem, thank you!
>>
>>> 30 авг. 2021 г., в 13:43, Pengchengliu напис
IOS vs pinned buffers issue[0].
And, probably, sort items on leaf pages. And, maybe, split pages more
intelligently.
I hope to get to this in December.
I'll post rebased version ASAP.
Best regards, Andrey Borodin.
[0]
https://www.postgresql.org/message-id/CAH2-Wz=PqOziyRSrnN5jAtfXWXY7-BJcHz9S355LH8Dt=5q...@mail.gmail.com
> 21 нояб. 2021 г., в 23:58, Michail Nikolaev
> написал(а):
>
>
Write barrier must be issued after write, not before.
Don't we need to issue read barrier too?
Best regards, Andrey Borodin.
) and subsequent CoW memory allocations.
And these expenses would be not necessary if we could just send a new Startup
message after the Terminate ('X') message.
But this effectively would empty out out all backend caches.
Yet the feature seems useful from my PoV.
Best regards, Andrey Borodin.
> 18 нояб. 2021 г., в 12:05, Noah Misch написал(а):
>
> What else might help?
Let's add more tests that check survival of 2PC through crash recovery? We do
now only one restart. Maybe it worth to do 4 or 8?
Best regards, Andrey Borodin.
> 15 нояб. 2021 г., в 19:32, Rushabh Lathia
> написал(а):
>
> Open for suggestions and thoughts.
How about walapplier ?
Similar to walsender, walreciver..
Best regards, Andrey Borodin.
hat discussion, sorry for the
noise. And thanks for the pointer.
Best regards, Andrey Borodin.
n tcp keepalives, but they did not fire for some reason.
Seems like the client was alive but sluggish.
Does it make sense to look for infinite timeouts in communication and replace
them with a loop checking for interrupts?
Best regards, Andrey Borodin.
> On 11/10/21 16:54, Andrey Borodin wrote:
>
>> Compression is crucial for highly available setups. Replication traffic is
>> often billed. Or route has bandwidth limits.
>> An entropy added by WAL headers makes CRIME attack against replication
>> encryption
errormsg ? errormsg : "(no details available)")));
>
Noah recently proposed the same [0]. But responders like me started to complain
about other places without error reporting around xlog reading.
Probably I had to be more clear on the matter. Error logging certainly worth
doing, a
reindex any concurrently-built indexes.
>>
>> Done.
>
> As far as I know (correct me if I'm mistaken), all of the CIC/RC bugs
> are detectable using amcheck's heapallindexed verification feature.
Yes, but only for B-trees.
Best regards, Andrey Borodin.
ilable setups. Replication traffic is
often billed. Or route has bandwidth limits.
An entropy added by WAL headers makes CRIME attack against replication
encryption impractical.
Best regards, Andrey Borodin.
> 7 нояб. 2021 г., в 06:31, Noah Misch написал(а):
>
> As a first step, let's report the actual XLogReadRecord() error message.
> Attached. All the other sites that expect no error already do this.
BTW some time ago I've spotted a good number of related unreported errors [0].
[0]
Sorry for so late reply. I've been thinking about possible approaches.
KnownAssignedXids over hashtable in fact was implemented long before and
rejected [0].
> 3 авг. 2021 г., в 22:35, Andres Freund написал(а):
>
> On 2021-08-03 10:33:50 +0500, Andrey Borodin wrote:
>>> 3 авг
?
I'd say next to impossible, yet not impossible. Or, perhaps, I do not see
protection from this.
Moreover there's a "microvacuum". It kills tuples with BUFFER_LOCK_SHARE. AFAIU
it should take cleanup lock on buffer too?
Best regards, Andrey Borodin.
> 4 нояб. 2021 г., в 17:55, Daniel Gustafsson написал(а):
>
> The patch no longer applies, can you submit a rebased version please?
Thanks, Daniel! PFA rebase.
Best regards, Andrey Borodin.
v3-0001-Allow-providing-restore_command-as-a-command-line.patch
Description: Binary data
independently. That's mostly the reason why I didn't know what to reply for so
long.
> 5 нояб. 2021 г., в 01:47, Tomas Vondra
> написал(а):
>
> Andrey, can you update the patch per Mark's review? I'll do my best to get it
> committed sometime in this CF.
Cool! Here's the patch.
Best regards, Andrey Borodin.
v6-0001-Reorganize-pglz-compression-code.patch
Description: Binary data
h checking visibility, and Backend2 must do LockBufferForCleanup(index_page1). Do I get things right? Best regards, Andrey Borodin.
> 3 нояб. 2021 г., в 14:08, Alexander Korotkov
> написал(а):
>
> ( a.On Wed, Nov 3, 2021 at 11:44 AM Andrey Borodin
> wrote:
>>> 21 окт. 2021 г., в 09:01, Kyotaro Horiguchi
>>> написал(а):
>>>
>>> If the discussion so far is
erflowedXid = InvalidTransactionId;
>LWLockRelease(ProcArrayLock);
> }
The patch seems correct bugfix to me. The only question I have: is it right
place from modularity standpoint? procArray->lastOverflowedXid is not a part of
KnownAssignedTransactionIds?
Best regards, Andrey Borodin.
low == false. I can't find a reason not to do so.
Best regards, Andrey Borodin.
wrong assumption
somewhere. My wild guess is that metapage is read before it was written.
Best regards, Andrey Borodin.
like system restart to
enable shared library.
I think we need a design that legalises already existing de-facto standard
features in archive tools. Or event better - enables these tools to be more
efficient, reliable etc. Either way we will create legacy code from the scratch.
Thanks!
Best regards, Andrey Borodin.
-target-wal, and that's not
> really instinctive to pass down a command via another command..
Currently we know that --restore-target-wal is not enough if postgresql.conf
does not reside within PGDATA.
Best regards, Andrey Borodin.
ta exploration [0,1]. This
is a kind of data aggregation which protects data from deducing single row by
means of aggregation. Implementation could be resemblant to MuchPIR.
Thanks!
Best regards, Andrey Borodin.
[0] https://en.wikipedia.org/wiki/Differential_privacy
[1] https://cs.uwaterloo.ca/~ilyas/papers/GeSIGMOD2019.pdf
rely on
data spilled to disk.
Are there any known good random access compressed file libs? So we could avoid
reinventing the wheel.
Maybe someone tried this approach before?
Thanks!
Best regards, Andrey Borodin.
[0]
https://github.com/x4m/postgres_g/commit/426cd767694b88e64f5e6bee99fc653c45eb5abd
el of parallelism will adapt to the workload.
Best regards, Andrey Borodin.
Existing archive_commands will continue as is.
Currently information about what is archived is stored on filesystem in
archive_status dir. We do not need to change anything.
If archive_command exits (with any exit code) we will restart it if there are
WAL files that still were not archived.
Best regards, Andrey Borodin.
> 10 сент. 2021 г., в 11:11, Julien Rouhaud написал(а):
>
> On Fri, Sep 10, 2021 at 2:03 PM Andrey Borodin wrote:
>>
>>> 10 сент. 2021 г., в 10:52, Julien Rouhaud написал(а):
>>>
>>> Yes, but it also means that it's up to every single archivin
> 10 сент. 2021 г., в 10:52, Julien Rouhaud написал(а):
>
> On Fri, Sep 10, 2021 at 1:28 PM Andrey Borodin wrote:
>>
>> It's OK if external tool is responsible for concurrency. Do we want this
>> complexity in core? Many users do not enable archiving at all.
&g
pe such tool swiftly :)
E.g. postgres runs ```wal-g wal-archiver``` and pushes ready segment filenames
on stdin. And no more listing of archive_status and hacky algorithms to predict
next WAL name and completition time!
Thoughts?
Best regards, Andrey Borodin.
[0]
https://www.postgresql.org/mes
ch bank (in
v17-0002-Divide-SLRU-buffers-into-n-associative-banks.patch). With this
approach we will have to rename s/bank/partition/g for consistency with locks
and buffers partitions. I really liked having my own banks, but consistency
worth it anyway.
Thanks!
Best regards, Andrey Borodin.
ers-into-n-associative-banks.patch). With this approach we will have to rename s/bank/partition/g for consistency with locks and buffers partitions. I really liked having my own banks, but consistency worth it anyway.Thanks!Best regards, Andrey Borodin.
was a community request to move SLRUs to
shared_buffers [2].
Thanks!
Best regards, Andrey Borodin.
[0] https://commitfest.postgresql.org/34/2627/
[1]
https://www.postgresql.org/message-id/flat/BE73A0BB-5929-40F4-BAF8-55323DE39561%40yandex-team.ru
[2]
https://www.postgresql.org/message-id/fla
> 29 июня 2021 г., в 19:34, Alexey Kondratov
> написал(а):
>
> On Fri, Jun 18, 2021 at 10:06 PM Alexey Kondratov
> wrote:
>> On Fri, Jun 18, 2021 at 5:42 PM Andrey Borodin wrote:
>>>
>>> If we run 'pg_rewind --restore-target-wal' there must be r
ions in signal
> handlers?
In my view GUC backtrace_functions is expected to be used for debug purposes.
Not for enabling on production server for bgworker_die() or
FloatExceptionHandler().
Are there any way to call backtrace_symbols() without touching
backtrace_functions?
Best regards, Andrey Borodin.
t?
- Where to seek a manual for recovery?
But I don't think we can have this logged in case of "ERROR: t_xmin is
uncommitted in tuple to be updated"
Thanks!
Best regards, Andrey Borodin.
tch does not incur
penalty on normal workload.
Thank you!
Best regards, Andrey Borodin.
[0]
https://gitlab.com/postgres-ai/postgresql-consulting/tests-and-benchmarks/-/issues/20
fsync was lost before restart)
Best regards, Andrey Borodin.
EX_CORRUPTED is used. Iif it's
possible to extract this information, of cause. This is needed especially in
amcheck functions.
Best regards, Andrey Borodin.
balance (reviews_done - reviews_owed).
+1 for tracking this.
BTW when review is done? When first revision is published? Or when patch is
committed\rollbacked?
When the review is owed? At the moment when patch is submitted? Or when it is
committed?
Best regards, Andrey Borodin.
en
adding\removing single transactions. But much worse when doing
KnownAssignedXidsRemove().
Maybe use a tree? (AVL\RB or something like that) It will be slightly better,
because it does not need eventual compression like exiting array.
Best regards, Andrey Borodin.
es. For me it's not a single bit easier to read comment then code. If this
comment is that important, maybe refactor this assignment into function and
document once?
Thanks!
Best regards, Andrey Borodin.
> 25 июля 2021 г., в 05:29, Andres Freund написал(а):
>
> Hi,
>
> On 2021-07-24 15:53:15 +0500, Andrey Borodin wrote:
>> Are there any other problems with blocking cancels?
>
> Unless you have commandline access to the server, it's not hard to get
> into a si
> 23 июля 2021 г., в 22:54, Bossart, Nathan написал(а):
>
> On 7/23/21, 4:33 AM, "Andrey Borodin" wrote:
>> Thanks for you interest in the topic. I think in the thread [0] we almost
>> agreed on general design.
>> The only left question is that w
101 - 200 of 609 matches
Mail list logo
