Re: [HACKERS] New gist vacuum.

2017-11-12 Thread Andrey Borodin
ash table is not very good option if we have all pages there: we should either use array or do not fill table for every page. If author and community do not object, I want to continue work on Konstantin's patch. Best regards, Andrey Borodin. 0001-GiST-VACUUM-rebase.patch Des

Re: [HACKERS] WIP: Covering + unique indexes.

2017-11-12 Thread Andrey Borodin
otpath, anyway, so I propose ignoring this for current version. 2. I've done grammarly checking :) This comma seems redundant [2] I don't think something of these items require fixing. Thanks for working on this, I believe it is important. Best regards, Andrey Borodin.

Re: [HACKERS] Index only scan for cube and seg

2017-10-29 Thread Andrey Borodin
o just remove both compress/decompress methods from > existing opclass. Alexander, Tom, you are absolutely right. I was sure there is toasting code in cube's compress, but it was not ever there. Here is patch for cube that drops functions. Best regards, Andrey Borodin. 0001-Enable-Index-

Re: [HACKERS] Reading timeline from pg_control on replication slave

2017-10-28 Thread Andrey Borodin
see that the safest option is to check timeline from pg_control after start and after stop. If this timelines differ - invalidate backup entirely. This does not seem too hard condition for invalidation, does it? Best regards, Andrey Borodin. -- Sent via pgsql-hackers mailing list (pgsql-hackers@po

[HACKERS] Reading timeline from pg_control on replication slave

2017-10-27 Thread Andrey Borodin
Hi, hackers! I'm working on backups from replication salve in WAL-G [0] Backups used to use result of pg_walfile_name(pg_start_backup(...)). Call to pg_start_backup() works nice, but "pg_walfile_name() cannot be executed during recovery." This function has LSN as argument and reads TimeLineId fr

[HACKERS] Index only scan for cube and seg

2017-10-26 Thread Andrey Borodin
Hi hackers! Here are patches enabling Index Only Scan for cube and seg extensions. These patches follow this discussion [0]. For cube there is new default opclass. We cannot drop old opclass, because it could TOAST come cube values in rare occasions. Index Only Scan is enabled only for newly c

Re: [HACKERS] Allow GiST opcalsses without compress\decompres functions

2017-10-23 Thread Andrey Borodin
> 22 окт. 2017 г., в 21:21, Tom Lane написал(а): > > Andrey Borodin writes: >> I was looking for a way to correctly drop compress\decompress functions from >> opclasses. > > Making a new opclass seems like a pretty grotty answer; it won't > help existi

Re: [HACKERS] Allow GiST opcalsses without compress\decompres functions

2017-10-22 Thread Andrey Borodin
Hi, Tom! > 20 сент. 2017 г., в 8:38, Tom Lane написал(а): > > Andrey Borodin writes: >> [ 0001-Allow-uncompressed-GiST-4.patch ] > > Pushed, with a bit more work on the documentation and some minor > cosmetic changes. > > I did not like the fact that the new

Re: [HACKERS] On markers of changed data

2017-10-15 Thread Andrey Borodin
Hello! > 9 окт. 2017 г., в 10:23, Andrey Borodin написал(а): > > Thanks, Stephen, this actually pointed what to look for > VM is WAL-logged [0] > FSM is not [1] > > [0] > https://github.com/postgres/postgres/blob/113b0045e20d40f726a0a30e33214455e4f1385e

Re: [HACKERS] On markers of changed data

2017-10-09 Thread Andrey Borodin
, FSM cannot. But the design you are describing resembles PTRACK[0]: fork for page changes tracking, not WAL-logged, but crash safe due to recovery from others WALs. Best regards, Andrey Borodin. [0] https://gist.github.com/stalkerg/ab833d94e2f64df241f1835651e06e4b <https://gist.github.com/stalkerg/ab833d94e2f64df241f1835651e06e4b>

Re: [HACKERS] On markers of changed data

2017-10-08 Thread Andrey Borodin
> 8 окт. 2017 г., в 20:11, Stephen Frost написал(а): > * Andrey Borodin (x4...@yandex-team.ru) wrote: >> But my other question still seems unanswered: can I use LSN logic for >> incrementing FSM and VM? Seems like most of the time there is valid LSN > > I haven'

Re: [HACKERS] On markers of changed data

2017-10-08 Thread Andrey Borodin
by Stephen are taken. But my other question still seems unanswered: can I use LSN logic for incrementing FSM and VM? Seems like most of the time there is valid LSN * I like the idea of using something for both incr(diff) backups and VACUUM, it worth thinking about. Best regards, Andrey Borodin

[HACKERS] On markers of changed data

2017-10-06 Thread Andrey Borodin
ck this "capacity" of FS? Thank you for your attention. I'll be glad to receive any information\pointers on this matter. Best regards, Andrey Borodin, Yandex. [0] https://github.com/wal-g/wal-g/pull/29 [1] https://github.com/wal-g/wal-g/pull/29/files#diff-d77406e827f5f947d4d4a1

Re: [HACKERS] Allow GiST opcalsses without compress\decompres functions

2017-09-20 Thread Andrey Borodin
Hello Tom! Thanks for committing the patch! > 20 сент. 2017 г., в 8:38, Tom Lane написал(а): > > Andrey Borodin writes: >> [ 0001-Allow-uncompressed-GiST-4.patch ] > > ... There's still room > to improve the contrib opclasses I have one important question

Re: [HACKERS] Index Only Scan support for cube

2017-09-19 Thread Andrey Borodin
, Tom! : ) "Index Only Scan support for cube" patch now is obsolete. I'm working on another similar patch for contribs to support GiST IOS and remove no-op support functions. Best regards, Andrey Borodin. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To m

Re: [HACKERS] Allow GiST opcalsses without compress\decompres functions

2017-09-15 Thread Andrey Borodin
tch look suspicious for me > > We are typically evade changing formatting in fragments of codes not directly > touched by the patch. Thanks for spotting this out. Here's fixed version. Best regards, Andrey Borodin. 0001-Allow-uncompressed-GiST-4.patch Description: Bin

Re: [HACKERS] Hooks to track changed pages for backup purposes

2017-09-13 Thread Andrey Borodin
Hi! Thank you for your interest and experiment results. > 13 сент. 2017 г., в 15:43, Ants Aasma написал(а): > > On Thu, Aug 31, 2017 at 9:02 AM, Andrey Borodin wrote: >> When we have accumulated diff blocknumbers for most of segments we can >> significantly speed up metho

Re: [HACKERS] Hooks to track changed pages for backup purposes

2017-09-12 Thread Andrey Borodin
encrypt backups and send to S3. And there is no system with CPUs that can handle that WAL parsing. Currently, I'm considering mocking prototype for wal-g, which works exactly this. Your comments were very valuable, thank you for looking into the patch and joining the discussion. Best regards, Andrey Borodin.

Re: [HACKERS] Allow GiST opcalsses without compress\decompres functions

2017-09-11 Thread Andrey Borodin
o the patch! Best regards, Andrey Borodin. 0001-Allow-uncompressed-GiST-3.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hooks to track changed pages for backup purposes

2017-08-31 Thread Andrey Borodin
al takes vanishingly small part of backend performance, but, indeed, nonzero part. Again, thank you for your time and comments. Best regards, Andrey Borodin. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Hooks to track changed pages for backup purposes

2017-08-30 Thread Andrey Borodin
nable more efficient backups. How do you think? Any ideas will be appreciated. This patch is influenced by the code of PTRACK (Yury Zhuravlev and Postgres Professional). Best regards, Andrey Borodin. [0] https://www.postgresql.org/message-id/flat/20051502087457%40webcorp01e.yandex-team.ru#2

Re: [HACKERS] Adding hook in BufferSync for backup purposes

2017-08-28 Thread Andrey Borodin
f each critical section. But I want to avoid intervention into critical sections. Thank you for your attention, any thoughts will be appreciated. Best regards, Andrey Borodin. [0] https://gist.github.com/stalkerg/ab833d94e2f64df241f1835651e06e4b 0001-hooks-to-watch-for-changed-pages.patch Descript

Re: [HACKERS] [PROPOSAL] Use SnapshotAny in get_actual_variable_range

2017-08-18 Thread Andrey Borodin
x27;s correct checklist: installcheck-world passes, documented as expected. Feature is there. Best regards, Andrey Borodin. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PROPOSAL] Use SnapshotAny in get_actual_variable_range

2017-08-17 Thread Andrey Borodin
"give me actual values, but only if it is on first index page", like conditional locks. But I think this patch is a step to right direction. Best regards, Andrey Borodin. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] GSoC 2017 : Patch for predicate locking in Gist index

2017-08-16 Thread Andrey Borodin
. Patch applies and installcheck-world passes 2. Patch contains new isolation tests 3. These tests fail if indexes do not implement predicate locking 4. Patch updates documentation Shubham also had checked that patches work (install check-world) on 1Kb, 8Kb and 32Kb pages. Best regards, Andrey Borodin, Yandex.

Re: [HACKERS] Adding hook in BufferSync for backup purposes

2017-08-07 Thread Andrey Borodin
by > BufferSync to schedule buffer writes has changed multiple times > before and doubtless will again; if we're going to have a hook > here it should depend as little as possible on those details. OK, now I see that «buf_internals.h» had word internals for a reason. Thanks for

Re: [HACKERS] pg_background contrib module proposal

2017-01-19 Thread Andrey Borodin
. Summarizing these points, appropriate statuses of the patch are ‘Ready for committer’ or ‘Rejected’. Between these two I choose ‘Ready for committer’, I think patch is committable (after bg sessions). Best regards, Andrey Borodin. The new status of this patch is: Ready for Committer -- Sent via pgsql

Re: [HACKERS] autonomous transactions

2017-01-07 Thread Andrey Borodin
_link patches, but this review does not apply to them. I suppose db_link refactoring would be useful and functionality is added, so I think these patches deserve separate commitfest entry. Best regards, Andrey Borodin. The new status of this patch is: Ready for Committer -- Sent via pgsql-ha

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-08 Thread Andrey Borodin
Thank you for your attention to details, Mikhail. pack_float_good() looks good. But I'm not sure inline strict init is allowed under ansi C. Converting to regular ancient form b.fp = v; won't change compile result, would it? Regards, Andrey Borodin. -- Sent via pgsql-hackers ma

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

2016-09-08 Thread Andrey Borodin
That storage assertion fired during usual update table set x=random() without conditions. Also Make check fails without it (for brin usage, gist is ok with it). Cannot type quotation properly, I'm on a phone for a long time. Regards, Andrey Borodin. -- Sent via pgsql-hackers mailing

Re: [HACKERS] WIP: Covering + unique indexes.

2016-08-14 Thread Andrey Borodin
asure, STD = 0) A. 317 MB B. 509 MB C. 399 MB Index size is in the middle between full index and minimal index. I think this numbers agree with expectation from the feature. CONCLUSION== This patch brings useful and important feature. Build shall be repaired; other my suggestions

Re: [HACKERS] Optimizing numeric SUM() aggregate

2016-08-03 Thread Andrey Borodin
> 3 авг. 2016 г., в 22:47, Andrey Borodin написал(а): > > make installcheck-world: tested, failed > Implements feature: tested, failed > Spec compliant: tested, failed > Documentation:tested, failed > > This is a review of a patch

Re: [HACKERS] Optimizing numeric SUM() aggregate

2016-08-03 Thread Andrey Borodin
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, failed Spec compliant: tested, failed Documentation:tested, failed This is a review of a patch "Optimizing numeric SUM() aggrega