Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-25 Thread Alexander Korotkov
id you use for these tests: power-8 or x86? How long was single run? Per-run variation seems quite high. It also seems that it depends on which version runs first. But that could be a coincidence. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Ordering in guc.c vs. config.sgml vs. postgresql.sample.conf

2016-04-25 Thread Alexander Korotkov
usly not a 9.6 > topic at this point, but it'd probably be good to that early in 9.7. > +1 Also, what do you think about validation script which checks consistency between them? -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Autovacuum to prevent wraparound tries to consume xid

2016-05-19 Thread Alexander Korotkov
On Mon, Mar 28, 2016 at 2:05 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > After some debugging I found that vac_truncate_clog consumes xid just to > produce warning. I wrote simple patch which replaces > GetCurrentTransactionId() with ShmemVariabl

Re: [HACKERS] Autovacuum to prevent wraparound tries to consume xid

2016-05-22 Thread Alexander Korotkov
On Sun, May 22, 2016 at 12:39 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Mon, Mar 28, 2016 at 4:35 PM, Alexander Korotkov < > a.korot...@postgrespro.ru> wrote: > >> Hackers, >> >> one our customer meet near xid wraparound situation. xid cou

Re: [HACKERS] Is the unfair lwlock behavior intended?

2016-05-24 Thread Alexander Korotkov
scalability issues including high ProcArrayLock contention. References. 1. http://www.postgresql.org/message-id/CAPpHfdsytkTFMy3N-zfSo+kAuUx=u-7jg6q2byb6fpuw2cd...@mail.gmail.com 2. http://www.postgresql.org/message-id/20151211130413.go14...@awork2.anarazel.de 3. http://www.po

Re: [HACKERS] atomic pin/unpin causing errors

2016-05-04 Thread Alexander Korotkov
could package up a > data directory after the issue hit? > FWIW, I'm also trying to reproduce it on big x86 machine on 9888b34f. I'll write about results when get any. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] GIN logging GIN_SEGMENT_UNMODIFIED actions?

2016-05-10 Thread Alexander Korotkov
> I've checked GIN code. Have no idea of how such wal record could be generated... The only idea I have is to add check that we're inserting valid WAL record immediately before XLogInsert(). -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Declarative partitioning

2016-04-19 Thread Alexander Korotkov
m the syntax side it very looks like defining something boxes regions for two keys which could be replacement for subpartitioning. But it isn't so. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-14 Thread Alexander Korotkov
1116256 1167681 130 1066475 1120891 140 1040379 1085904 150 974064 1022160 160 938396 976487 170 953636 978120 180 920772 953843 We can see small but certain regression after snapshot too old feature. -- Alexander Korotkov Postgres Professional: http://www.postgre

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-14 Thread Alexander Korotkov
there is small regression. I think this regression could be caused by alignment of other data structures. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Alexander Korotkov
ould take a hit. But I think that would be > acceptable. Oh, I found that I underestimated complexity of async commit... :) Do I understand right that now async commit right as follows? 1) Async transaction confirms commit before flushing WAL. 2) Other transactions sees effect of async transaction only when its WAL flushed. 3) In the session which just committed async transaction, effect of this transaction is visible immediately (before WAL flushed). Is it true? -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Alexander Korotkov
an XID array. FYI, I'm still fan of idea to rewrite XID with CSN in tuple in the same way we're writing hint bits now. I'm going to make prototype of this approach which would be enough for performance measurements. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Alexander Korotkov
ake async commits to work. > I didn't get async commits problem at first glance. AFAICS, the difference between sync commit and async is only that async commit doesn't wait WAL log to flush. But async commit still receives LSN. Could you describe it in more details? -- A

Re: [HACKERS] Wait events monitoring future development

2016-08-10 Thread Alexander Korotkov
s just an idea without any proof so soon. 2) We already have functionality which trades insight into database with way more huge overhead. auto_explain.log_analyze = true can slowdown queries *in times*. Do you think we should remove it? -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Wait events monitoring future development

2016-08-10 Thread Alexander Korotkov
at we do not have such accurate > tests now? > Do you think that large shared buffers is a kind a stress test for wait events monitoring? If so, why? -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Alexander Korotkov
On Wed, Aug 10, 2016 at 8:26 PM, Greg Stark <st...@mit.edu> wrote: > On Wed, Aug 10, 2016 at 5:54 PM, Alexander Korotkov > <a.korot...@postgrespro.ru> wrote: > > Oh, I found that I underestimated complexity of async commit... :) > > Indeed. I think Tom's attitude

Re: [HACKERS] Pluggable storage

2016-08-15 Thread Alexander Korotkov
indexes and so on). But imperative was that it should have its own executor nodes, and it doesn't have to compatible with existing index access methods. Therefore, I think my design presented at PGCon and your current proposal are about orthogonal features which could coexist. -- Alexander

[HACKERS] Relocation of tuple between release and re-acquire of tuple lock

2016-08-12 Thread Alexander Korotkov
a buffer pin and vacuum would wait for pin release. But other heap operations could still call heap_page_prune() which correspondingly can relocate tuple. Probably, I'm missing something... -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Relocation of tuple between release and re-acquire of tuple lock

2016-08-12 Thread Alexander Korotkov
On Fri, Aug 12, 2016 at 3:15 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > I'm now exploring code working with heap tuples. The following code > in heap_update() catch my eyes. > > if (DoesMultiXactIdConflict((MultiXactId) xwait, infomask, >> *lockmode)) &

[HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

2016-07-03 Thread Alexander Korotkov
inserts > faster and to enable new advanced page layouts in indices. > +1 for PageReplaceItem() Even if item is not the same size, we can move the tail of page once instead of twice. I think you should implement PageReplaceItem() version and add it to the commitfest. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Index Onlys Scan for expressions

2016-08-16 Thread Alexander Korotkov
r index scan too. > Probably it > > worth to discuss the way to normalize index expressions and clauses and > work > > out more convenient way to match them. > > pg_operator.oprcommutative ? Do you mean pg_operator.oprcom? In these examples we're also lacking of pg_operator.oprasso

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-02-03 Thread Alexander Korotkov
On Fri, Feb 3, 2017 at 8:01 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > Unfortunately, I have no big enough Power machine at hand to reproduce > that results. Actually, I have no Power machine at hand at all. So, > lwlock-power-2.patch was written "bli

[HACKERS] LWLock optimization for multicore Power machines

2017-02-03 Thread Alexander Korotkov
eone would help me with testing and benchmarking. 1. https://www.postgresql.org/message-id/CAPpHfdsogj38HTDhNMLE56uJy9N8- =gya2nnuwbpujgp2n1...@mail.gmail.com -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company lwlock-power-1.patch Descripti

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-02-06 Thread Alexander Korotkov
because that machine is used for tries to make Postgres work properly on AIX. So, benchmarking help is very relevant. I would very appreciate that. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company lwlock-power-3.patch Description: Binary d

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-02-07 Thread Alexander Korotkov
On Tue, Feb 7, 2017 at 3:16 PM, Bernd Helmle <maili...@oopsware.de> wrote: > Am Montag, den 06.02.2017, 22:44 +0300 schrieb Alexander Korotkov: > >2. Also could you run each test longer: 3-5 mins, and run them with >variety of clients count? > > > So here a

Re: [HACKERS] GSoC 2017 - LAST CALL

2017-02-07 Thread Alexander Korotkov
r project ideas > submitted so we can include them. > Thank you for reminder. I added couple more project ideas and added me, Oleg and Teodor to possible mentors of some projects. That seems to be all that I have for now. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-02-06 Thread Alexander Korotkov
On Mon, Feb 6, 2017 at 8:28 PM, Bernd Helmle <maili...@oopsware.de> wrote: > Am Montag, den 06.02.2017, 16:45 +0300 schrieb Alexander Korotkov: > > I tried lwlock-power-2.patch on multicore Power machine we have in > > PostgresPro. > > I realized that using labels in as

Re: [HACKERS] PinBuffer() no longer makes use of strategy

2017-02-04 Thread Alexander Korotkov
ogic back unless we have proof that this change make it better. Patch is attached. I tried to make some comments, but probably they are not enough. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company put-buffer-usagecount–logic–back.patch

Re: [HACKERS] Should we cacheline align PGXACT?

2017-01-30 Thread Alexander Korotkov
nly pgbench results on 72 physically cores Intel server are attached. It quite similar to results I posted before, but it's curious that performance degradation of master on high concurrency became larger. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Pos

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-21 Thread Alexander Korotkov
WAL writes in a new > cluster are a more expensive than in an old one, because of segment > recycling. > +1 -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-21 Thread Alexander Korotkov
s me think that probably there is something wrong in testing methodology. Are you doing re-initdb and pgbench -i before each run? I would ask you to do the both. Also, if regression would still exist, let's change the order of versions. Thus, if you run master before patched version, I would ask

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-02-21 Thread Alexander Korotkov
On Tue, Feb 21, 2017 at 1:47 PM, Bernd Helmle <maili...@oopsware.de> wrote: > Am Samstag, den 11.02.2017, 15:42 +0300 schrieb Alexander Korotkov: > > I think it would make sense to run more kinds of tests. Could you > > try set > > of tests provided by Tomas Vondra?

[HACKERS] [PATCH] Incremental sort (was: PoC: Partial sort)

2017-02-18 Thread Alexander Korotkov
Also patch has set of assorted code and comments improvements. Links 1. https://www.postgresql.org/message-id/CA+TgmoZapyHRm7NVyuyZ+yAV=u1a070bogre7pkgyraegr4...@mail.gmail.com 2. https://www.postgresql.org/message-id/cam3swzql4yd2sndhemcgl0q2b2otdkuvv_l6zg_fcgoluwm...@mail.gmail.com -- Alexand

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-14 Thread Alexander Korotkov
On Mon, Feb 13, 2017 at 7:07 PM, Alvaro Herrera <alvhe...@2ndquadrant.com> wrote: > Alexander Korotkov wrote: > > > Yes, influence seems to be low. But nevertheless it's important to > insure > > that there is no regression here. > > Despite pg_prewarm'ing and r

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-14 Thread Alexander Korotkov
On Tue, Feb 14, 2017 at 3:57 PM, Alvaro Herrera <alvhe...@2ndquadrant.com> wrote: > Alexander Korotkov wrote: > > On Mon, Feb 13, 2017 at 7:07 PM, Alvaro Herrera < > alvhe...@2ndquadrant.com> > > wrote: > > > > Re the coding of the padding computat

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-02-14 Thread Alexander Korotkov
On Mon, Feb 13, 2017 at 10:17 PM, Tomas Vondra <tomas.von...@2ndquadrant.com > wrote: > On 02/13/2017 03:16 PM, Bernd Helmle wrote: > >> Am Samstag, den 11.02.2017, 15:42 +0300 schrieb Alexander Korotkov: >> >>> Thus, I see reasons why in your tests absolute

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-15 Thread Alexander Korotkov
/www.postgresql.org/message-id/20160822021747.u5bqx2xwwjzac...@alap3.anarazel.de -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company pgxact-align-3.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgre

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-13 Thread Alexander Korotkov
ee this variation on the graphs. Another issue with graphs is that we can't see details of read and write TPS variation on the same scale, because write TPS values are too low. I think you should draw write benchmark on the separate graph. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-11 Thread Alexander Korotkov
On Sat, Feb 11, 2017 at 4:17 PM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: > On 02/11/2017 01:21 PM, Alexander Korotkov wrote: > >> Hi, Tomas! >> >> On Sat, Feb 11, 2017 at 2:28 AM, Tomas Vondra >> <tomas.von...@2ndquadrant.com <mail

Re: [HACKERS] removing tsearch2

2017-02-10 Thread Alexander Korotkov
tarting a new > thread to make sure we collect all the relevant votes, but I really, > really think it's past time for this to go away. The last actual > change to tsearch2 which wasn't part of a wider cleanup was > 3ca7eddbb7c4803729d385a0c9535d8a972ee03f in January 2009, so it's been > 7 years since t

Re: [HACKERS] [PATCH] kNN for btree

2017-02-16 Thread Alexander Korotkov
if needed). This way wouldn't be that brief, but it is much more correct. Also this like catch my eyes. > info->amcanorderbyop = (void (*)()) amroutine->amcanorderbyop; It's not necessary to use cast here. For instance, we don't use cast for amcostestimate. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-15 Thread Alexander Korotkov
On Wed, Feb 15, 2017 at 11:49 AM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Mon, Feb 13, 2017 at 8:02 PM, Ashutosh Sharma <ashu.coe...@gmail.com> > wrote: > >> I too have performed benchmarking of this patch on a large machine >> (with 12

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-11 Thread Alexander Korotkov
FROM pg_class WHERE relkind IN ('i', 'r') ORDER BY oid) x; In my experience pg_prewarm both takes less time and leaves less variation afterwards. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-02-11 Thread Alexander Korotkov
On Wed, Feb 8, 2017 at 5:00 PM, Bernd Helmle <maili...@oopsware.de> wrote: > Am Dienstag, den 07.02.2017, 16:48 +0300 schrieb Alexander Korotkov: > > But win isn't > > as high as I observed earlier. And I wonder why absolute numbers are > > lower > > than in our

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-17 Thread Alexander Korotkov
On Fri, Feb 17, 2017 at 12:44 PM, Ashutosh Sharma <ashu.coe...@gmail.com> wrote: > >> On Thu, Feb 16, 2017 at 5:07 AM, Alexander Korotkov > >> <a.korot...@postgrespro.ru> wrote: > >> > On Wed, Feb 15, 2017 at 8:49 PM, Alvaro Herrera >

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-17 Thread Alexander Korotkov
On Thu, Feb 16, 2017 at 4:58 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Feb 16, 2017 at 5:07 AM, Alexander Korotkov > <a.korot...@postgrespro.ru> wrote: > > On Wed, Feb 15, 2017 at 8:49 PM, Alvaro Herrera < > alvhe...@2ndquadrant.com> > &g

[HACKERS] Should we cacheline align PGXACT?

2016-08-19 Thread Alexander Korotkov
performance regression in other cases. At least we should test read-write and smaller machines. Any other ideas? 1. https://www.postgresql.org/message-id/20160411214029.ce3fw6zxim5k6...@alap3.anarazel.de -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres

Re: [HACKERS] Should we cacheline align PGXACT?

2016-08-19 Thread Alexander Korotkov
On Fri, Aug 19, 2016 at 4:46 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Alexander Korotkov <a.korot...@postgrespro.ru> writes: > > originally this idea was proposed by Andres Freund while experimenting > with > > lockfree Pin/UnpinBuffer [1]. > > The

Re: [HACKERS] Should we cacheline align PGXACT?

2016-08-19 Thread Alexander Korotkov
On Fri, Aug 19, 2016 at 3:19 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Fri, Aug 19, 2016 at 11:42 AM, Alexander Korotkov > <a.korot...@postgrespro.ru> wrote: > > Hackers, > > > > originally this idea was proposed by Andres Freund while experimenting

Re: [HACKERS] Tracking wait event for latches

2016-08-22 Thread Alexander Korotkov
On Mon, Aug 22, 2016 at 12:09 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > I took a look at your patch. Couple of notes from me. > > const char * >> GetEventIdentifier(uint16 eventId) >> { >> const char *res; >> switch (eventId) &

Re: [HACKERS] Tracking wait event for latches

2016-08-22 Thread Alexander Korotkov
EventIdentifier seems too general name for me, isn't it? Could we name it WaitEventIdentifier? Or WaitEventId for shortcut? -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] GSoC 2017

2017-02-27 Thread Alexander Korotkov
Hi all! It seems that PostgreSQL has passed to GSoC mentoring organizations this year! https://summerofcode.withgoogle.com/organizations/4558465230962688/ Congratulations! -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-24 Thread Alexander Korotkov
; Attached is a graph with the results. Full results are available at >> https://hlinnaka.iki.fi/temp/csn-4-results/. In short, the patch >> improved throughput, measured in TPS, with >= 32 or so clients. The >> biggest difference was with 44 clients, which saw about 5% improvement. >&g

Re: [HACKERS] PoC: Partial sort

2016-09-13 Thread Alexander Korotkov
On Fri, Apr 8, 2016 at 10:09 PM, Peter Geoghegan <p...@heroku.com> wrote: > On Wed, Mar 30, 2016 at 8:02 AM, Alexander Korotkov > <aekorot...@gmail.com> wrote: > > Hmm... I'm not completely agree with that. In typical usage partial sort > > should definitely use

Re: [HACKERS] Pluggable storage

2016-08-18 Thread Alexander Korotkov
flexibility yet without adding a branch > point in the code? > It's impossible without branch point in code. The question is where this branch should be located. In particular, be can put this branch point into planner by defining distinct executor nodes for each pluggable storage. In this

Re: [HACKERS] Block level parallel vacuum WIP

2016-08-23 Thread Alexander Korotkov
vacuum of single index. But pointing each worker at separate index seems to be fair enough for majority of cases. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Indirect indexes

2016-10-19 Thread Alexander Korotkov
On Wed, Oct 19, 2016 at 12:53 PM, Simon Riggs <si...@2ndquadrant.com> wrote: > On 18 October 2016 at 23:46, Alexander Korotkov > <a.korot...@postgrespro.ru> wrote: > > > Then vacuum removes (0;1) from heap, reference to (0;1) from tbl_pk_idx. > > But ho

Re: [HACKERS] Indirect indexes

2016-10-19 Thread Alexander Korotkov
But it seems like rather better design for me. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Indirect indexes

2016-10-18 Thread Alexander Korotkov
ted this yet. > Imagine another situation: PK column was not updated, but indirect indexed column was updated. Thus, for single heap tuple we would have single PK tuple and two indirect index tuples (correct me if I'm wrong). How are we going to delete old indirect index tuple? --

Re: [HACKERS] Indirect indexes

2016-10-18 Thread Alexander Korotkov
On Wed, Oct 19, 2016 at 12:21 AM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Tue, Oct 18, 2016 at 9:28 PM, Alvaro Herrera <alvhe...@2ndquadrant.com> > wrote: > >> Vacuuming presents an additional challenge: in order to remove index >> items from

Re: [HACKERS] UNDO and in-place update

2016-11-29 Thread Alexander Korotkov
are just different kind of maps. And they could implement the same interface. Imagine you can select between heap-organized table and index-organized table just by choosing its primary access method. If you select heap for primary access method, indexes would refer TID. If you select btree o

[HACKERS] PgConf.Russia 2017 Call for Papers

2016-12-08 Thread Alexander Korotkov
kers will get free admission to the conference (all days), tickets to Moscow and accommodation booked and paid by the organizers. We will be glad to see you in Moscow! ------ Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Declarative partitioning - another take

2016-12-08 Thread Alexander Korotkov
ly update) tuple routing for foreign partitions - hash partitioning - not scanning the parent Time is growing short, but it would > be great to polish this a little more before we ship it. > Yes. Getting at least some of this features committed to v10 would be great and improve partitioning usabi

Re: [HACKERS] UNDO and in-place update

2016-12-02 Thread Alexander Korotkov
. But I don't like this approach. If we have large enough index scan, then we could either run out of cache or consume too much memory for that cache. So, I think storing visibility information in index tuple is great for regular index scans too. At least, it allow us to evade #2 problem. That is great by itself. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

[HACKERS] GSoC 2017

2017-01-10 Thread Alexander Korotkov
-0mW6M03St7A%40mail.gmail.com [2] https://www.postgresql.org/message-id/flat/CALxAEPuGpAjBSN-PTuxHfuLLqDS47BEbO_ZYxUYQR3ud1nwbww%40mail.gmail.com [3] https://developers.google.com/open-source/gsoc/timeline -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Crash on promotion when recovery.conf is renamed

2017-03-27 Thread Alexander Korotkov
nly connections 2017-03-27 17:32:16.920 MSK [30788] LOG: started streaming WAL from primary at 0/900 on timeline 1 done server started # select pg_is_in_recovery(); pg_is_in_recovery ─── t (1 row) Thus, I think patch is working as expected in this case. Also,

Re: [HACKERS] Unused argument in PinBuffer function

2017-03-27 Thread Alexander Korotkov
On Thu, Aug 4, 2016 at 2:28 PM, Ildar Musin <i.mu...@postgrespro.ru> wrote: > I was looking through the buffer manager's code and have noticed that > function PinBuffer has an unused 'strategy' argument. It's seems that after > refactoring made by Alexander Korotkov an

Re: [HACKERS] Index usage for elem-contained-by-const-range clauses

2017-03-27 Thread Alexander Korotkov
On Sat, Mar 18, 2017 at 12:41 AM, Pritam Baral <pri...@pritambaral.com> wrote: > On Friday 10 March 2017 07:59 PM, Alexander Korotkov wrote: > >> Hi, Pritam! > > I've assigned to review this patch. > > On Thu, Feb 23, >> 2017 at 2:17 AM, Pritam

Re: [HACKERS] Re: proposal - psql: possibility to specify sort for describe commands, when size is printed

2017-03-27 Thread Alexander Korotkov
On Fri, Mar 10, 2017 at 6:06 PM, Pavel Stehule <pavel.steh...@gmail.com> wrote: > 2017-03-10 16:00 GMT+01:00 Alexander Korotkov <a.korot...@postgrespro.ru>: > >> On Fri, Mar 10, 2017 at 5:16 PM, Stephen Frost <sfr...@snowman.net> >> wrote: >

Re: [HACKERS] Re: proposal - psql: possibility to specify sort for describe commands, when size is printed

2017-03-30 Thread Alexander Korotkov
On Tue, Mar 28, 2017 at 10:12 AM, Pavel Stehule <pavel.steh...@gmail.com> wrote: > 2017-03-27 13:59 GMT+02:00 Alexander Korotkov <a.korot...@postgrespro.ru>: > >> On Fri, Mar 10, 2017 at 6:06 PM, Pavel Stehule <pavel.steh...@gmail.com> >> wrote: >>

Re: [HACKERS] [PATCH] Incremental sort

2017-03-28 Thread Alexander Korotkov
On Tue, Mar 28, 2017 at 5:27 PM, David Steele <da...@pgmasters.net> wrote: > Hi Alexander, > > On 3/20/17 10:19 AM, Heikki Linnakangas wrote: > >> On 03/20/2017 11:33 AM, Alexander Korotkov wrote: >> >>> Please, find rebased patch in the a

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-03-25 Thread Alexander Korotkov
On Thu, Mar 16, 2017 at 8:35 PM, David Steele <da...@pgmasters.net> wrote: > On 2/21/17 9:54 AM, Bernd Helmle wrote: > > Am Dienstag, den 14.02.2017, 15:53 +0300 schrieb Alexander Korotkov: > >> +1 > >> And you could try to use pg_wait_sampling >

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-03-25 Thread Alexander Korotkov
On Sat, Mar 25, 2017 at 8:44 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Alexander Korotkov <a.korot...@postgrespro.ru> writes: > > [ lwlock-power-3.patch ] > > I experimented with this patch a bit. I can't offer help on testing it > on large PPC machines, b

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-03-25 Thread Alexander Korotkov
On Sat, Mar 25, 2017 at 11:32 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Alexander Korotkov <a.korot...@postgrespro.ru> writes: > > I moved PPC implementation of pg_atomic_fetch_mask_add_u32() into > > port/atomics/arch-ppc.h. I also had to declare pg_atomic_uint32

Re: [HACKERS] Crash on promotion when recovery.conf is renamed

2017-03-27 Thread Alexander Korotkov
re end of online backup"), > errhint("Online backup started with pg_start_backup() must be ended with > pg_stop_backup(), and all WAL up to that point must be available at > recovery."))); > else > ereport(FATAL, > (errmsg("WAL ends before consistent recovery point"))); Should we consider fixing them? -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-03-31 Thread Alexander Korotkov
On Sun, Mar 26, 2017 at 12:29 AM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Sat, Mar 25, 2017 at 11:32 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > >> Alexander Korotkov <a.korot...@postgrespro.ru> writes: >> > I moved PPC impleme

Re: [HACKERS] GSOC - TOAST'ing in slices

2017-03-15 Thread Alexander Korotkov
approaches you're going to try, and specify them in proposal. Schedule should include prototyping each of these approaches, then testing and benchmarking to selecting between them, then work on bringing selected approach to committable shape. -- Alexander Korotkov Postgres Professional: ht

Re: [HACKERS] PinBuffer() no longer makes use of strategy

2017-03-20 Thread Alexander Korotkov
> > if (BUF_STATE_GET_USAGECOUNT(buf_state) != BM_MAX_USAGE_COUNT) > > being prone to paranoia, I prefer the first, but I've seen both styles in > the code so I don't know if it's worth futzing with. > Ok, let's be paranoic and do this same way as before. Revised patch is attac

Re: [HACKERS] PinBuffer() no longer makes use of strategy

2017-03-20 Thread Alexander Korotkov
48354581a49c30f5757c203415aa8412d85b0f70 at April, 10. Does it mean the > fix should be backpatched too? I think so. This patch reverts unintentional change and can be considered as bug fix. BTW, sorry for unicode filename in previous letter. Patch with normal ASCII name is attached. -- Alexander

Re: [HACKERS] Merge join for GiST

2017-04-11 Thread Alexander Korotkov
On Tue, Apr 11, 2017 at 5:46 PM, Jeff Davis <pg...@j-davis.com> wrote: > On Tue, Apr 11, 2017 at 2:17 AM, Alexander Korotkov > <a.korot...@postgrespro.ru> wrote: > > FYI, I've implemented this algorithm for pgsphere. See following branch. > > https://gith

Re: [HACKERS] [PATCH] Incremental sort

2017-04-03 Thread Alexander Korotkov
On Mon, Apr 3, 2017 at 10:05 PM, Andres Freund <and...@anarazel.de> wrote: > On April 3, 2017 12:03:56 PM PDT, Alexander Korotkov < > a.korot...@postgrespro.ru> wrote: > >On Mon, Apr 3, 2017 at 9:34 PM, Andres Freund <and...@anarazel.de> > >wrote: > > &g

Re: [HACKERS] [PATCH] Incremental sort

2017-04-03 Thread Alexander Korotkov
On Tue, Apr 4, 2017 at 12:09 AM, Andres Freund <and...@anarazel.de> wrote: > On 2017-04-04 00:04:09 +0300, Alexander Korotkov wrote: > > > >Thank you! > > > >I already sent version of patch after David's reminder. > > > >Please find rebased patch

Re: [HACKERS] Should we cacheline align PGXACT?

2017-04-04 Thread Alexander Korotkov
On Mon, Apr 3, 2017 at 9:58 PM, Andres Freund <and...@anarazel.de> wrote: > On 2017-03-25 19:35:35 +0300, Alexander Korotkov wrote: > > On Wed, Mar 22, 2017 at 12:23 AM, David Steele <da...@pgmasters.net> > wrote: > > > > > Hi Alexander > > > &g

Re: [HACKERS] Possible problem in Custom Scan API

2017-04-11 Thread Alexander Korotkov
y own, but it uses replace_nestloop_params() (again, static), > and the problem goes further and further. > As I understand, making build_path_tlist a non-static function would solve the problem. Tom, do you think it's possible? -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] index-only count(*) for indexes supporting bitmap scans

2017-04-12 Thread Alexander Korotkov
On Wed, Apr 12, 2017 at 12:40 AM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Tue, Apr 11, 2017 at 8:24 PM, Alexander Kuzmenkov < > a.kuzmen...@postgrespro.ru> wrote: > >> I would like to propose a patch that speeds up the queries of the form >&

Re: [HACKERS] index-only count(*) for indexes supporting bitmap scans

2017-04-11 Thread Alexander Korotkov
scan returns all rechecks, alternative to Bitmap Count is still Aggregate + Bitmap Heap Scan. Thus, I think Bitmap Count would have the same performance or even slightly faster. That's worth testing. Also, what is planning overhead of this patch? That's worth testing too. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] GSoC 2017 Proposal

2017-04-06 Thread Alexander Korotkov
ve that “<@( is > contained by)” can be used in this scope. What exactly you're going to do in this item? Check that GIN index over array support given operator, do the performance benchmark or something else? -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

2017-04-06 Thread Alexander Korotkov
do not hit a broken line. I think that situation when backend is directly reading file during COPY is not typical. More typical case is \copy psql command. In that case "COPY ... FROM stdin;" is actually executed while psql is streaming the data. How can we apply parallel COPY in th

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-04-06 Thread Alexander Korotkov
On Thu, Apr 6, 2017 at 5:37 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Thu, Apr 6, 2017 at 2:16 AM, Andres Freund <and...@anarazel.de> wrote: > >> On 2017-04-03 11:56:13 -0700, Andres Freund wrote: >> > Have you done x86 benchmarking? >&

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-04-06 Thread Alexander Korotkov
+ } > > > + pg_unreachable(); > > > +} > > > +#endif > > > > Have you done x86 benchmarking? > > I think unless such benchmarking is done in the next 24h we need to move > this patch to the next CF... > Thank you for your comments. I didn't x86 benchmarking. I even didn't manage to reproduce previous results on Power8. Presumably, it's because previous benchmarks were done on bare metal, while now I have to some kind of virtual machine on IBM E880 where I can't reproduce any win of Power8 LWLock optimization. But probably there is another reason. Thus, I'm moving this patch to the next CF. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] [GSoC] Push-based query executor discussion

2017-04-06 Thread Alexander Korotkov
; I'd love to see a comment from Andres Freund who is leading executor performance improvements. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Merge join for GiST

2017-04-11 Thread Alexander Korotkov
//github.com/akorotkov/pgsphere/tree/crossmatch_cnode You also can find some experimental evaluation here: http://www.adass2016.inaf.it/images/presentations/10_Korotkov.pdf Feel free to experiment with this code and/or ask any question. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Should we cacheline align PGXACT?

2017-03-10 Thread Alexander Korotkov
to the > alignment patch. Do we have evidence to commit both? > > Well, We have seen some regression in read-write test with pgxact > alignment patch - [1]. I may need to run the test with both the > patches to see the combined effect on performance. > > [1]

Re: [HACKERS] Page Scan Mode in Hash Index

2017-03-10 Thread Alexander Korotkov
CT a FROM GENERATE_SERIES(1, 1000) a; You can achieve the same result by execution of single SQL statement. INSERT INTO con_hash_index_table (keycol) SELECT (a - 1) % 1000 + 1 FROM GENERATE_SERIES(1, 1002000) a; Unless you have some special idea of doing this in 1002 separate transactions. -

Re: [HACKERS] [PATCH] kNN for btree

2017-03-09 Thread Alexander Korotkov
tes: > >>> On Thu, Feb 16, 2017 at 8:05 AM, Alexander Korotkov > >>> <a.korot...@postgrespro.ru> wrote: > >>>> My idea is that we need more general redesign of specifying ordering > which > >>>> index can produce. Ideally, we should repla

Re: [HACKERS] [PATCH] kNN for SP-GiST

2017-03-09 Thread Alexander Korotkov
letely separate feature. And it should be posted and discussed separately. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] [PATCH]: fix bug in SP-GiST box_ops

2017-03-09 Thread Alexander Korotkov
mark it as "ready for committer"? -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Re: proposal - psql: possibility to specify sort for describe commands, when size is printed

2017-03-10 Thread Alexander Korotkov
between different sortings quickly. For instance, after seeing tables sorted by name, user can require them sorted by size descending, then sorted by size ascending, etc... Therefore, I find user-defined SQL clause to be cumbersome. Even psql variable itself seems to be cumbersome for me. I would propose t

Re: [HACKERS] Index usage for elem-contained-by-const-range clauses

2017-03-10 Thread Alexander Korotkov
Node *leftop, Oid expr_op, Oid opfamily, ^ Could you please recheck that you published right version of patch? Also, I noticed that patch haven't regression tests. Some mention of this optimization in docs is also nice to have. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Should we cacheline align PGXACT?

2017-03-10 Thread Alexander Korotkov
196 27628 28511 3.19603301 > 256 28467 28529 0.2177960445 > Results look good for me. Idea of committing both of patches looks attractive. We have pretty much acceleration for read-only case and small acceleration for read-write case. I'll run benchmark on 72-cores machine as well. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

<    4   5   6   7   8   9   10   11   >