[HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-04 Thread Aleksander Alekseev
Hello all, Current implementation of ResourceOwner uses arrays to store resources like TupleDescs, Snapshots, etc. When we want to release one of these resources ResourceOwner finds it with linear scan. Granted, resource array are usually small so its not a problem most of the time. But it

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-09 Thread Aleksander Alekseev
e ResourceArray with array-based implementation (abstraction layer). Then we apply second patch which change ResourceArray implementation to hashing based (optimization). Best regards, Aleksander On Tue, 8 Dec 2015 17:30:33 -0500 Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Dec 4, 201

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-11 Thread Aleksander Alekseev
>> To be honest, I think this patch is really ugly. [...] I'm not sure >> exactly what to do about that, but it seems like a problem. I have an idea. There are actually two types of resources - int-like (buffers, files) and void*-like (RelationRef, TupleDesc, ...). What if I split ResourceArray

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-11 Thread Aleksander Alekseev
> It would be InvalidBuffer for buffers, -1 for files and NULL for all > void*-types. Does such solution sounds OK? On second thought I believe there is no reason for storing anything for void*-like types. I could just hardcode NULL in PointerResourceArray. Best regards, Aleksander -- Sent

[HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-11 Thread Aleksander Alekseev
Hello all, Consider following stacktrace: (gdb) bt #0 0x7f77c81fae87 in semop () syscall-template.S:81 #1 0x0063b721 in PGSemaphoreLock pg_sema.c:387 #2 0x00692e1b in LWLockAcquire lwlock.c:1026 #3 0x0068d4c5 in LockAcquireExtended lock.c:881 #4

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-11 Thread Aleksander Alekseev
Hello, Tom I see your point, but I would like to clarify a few things. 1. Do we consider described measurement method good enough to conclude that sometimes PostgreSQL really spends 3 ms in a spinlock (like a RTT between two Internet hosts in the same city)? If not, what method should be used to

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-14 Thread Aleksander Alekseev
Hello, Robert Here is my fix for item 4. Best regards, Aleksander On Thu, 10 Dec 2015 11:37:23 -0500 Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Dec 9, 2015 at 8:30 AM, Aleksander Alekseev > <a.aleks...@postgrespro.ru> wrote: > > Hello, Robert > > > &g

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-15 Thread Aleksander Alekseev
p I believe would be to remove init_size parameter in ShmemInitHash procedure since in practice it always equals max_size. On Fri, 11 Dec 2015 10:30:30 -0500 Tom Lane <t...@sss.pgh.pa.us> wrote: > Aleksander Alekseev <a.aleks...@postgrespro.ru> writes: > > Tu

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-11 Thread Aleksander Alekseev
Oops. s/approve or disapprove/confirm or deny/ On Fri, 11 Dec 2015 19:14:41 +0300 Aleksander Alekseev <a.aleks...@postgrespro.ru> wrote: > Hello, Tom > > I see your point, but I would like to clarify a few things. > > 1. Do we consider described measurement method go

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-17 Thread Aleksander Alekseev
> It'd really like to see it being replaced by a queuing lock > (i.e. lwlock) before we go there. And then maybe partition the > freelist, and make nentries an atomic. I believe I just implemented something like this (see attachment). The idea is to partition PROCLOCK hash table manually into

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-18 Thread Aleksander Alekseev
> Oops, 3.5-4 _times_ more TPS, i.e. 2206 TPS vs 546 TPS. In fact these numbers are for similar but a little bit different benchmark (same schema without checks on child tables). Here are exact numbers for a benchmark described above. Before: $ pgbench -j 64 -c 64 -f pgbench.sql -T 30

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-18 Thread Aleksander Alekseev
> This idea can improve the situation with ProcLock hash table, but I > think IIUC what Andres is suggesting would reduce the contention > around dynahash freelist and can be helpful in many more situations > including BufMapping locks. I agree. But as I understand PostgreSQL community doesn't

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-18 Thread Aleksander Alekseev
> What's in pgbench.sql? It's from first message of this thread: http://www.postgresql.org/message-id/20151211170001.78ded9d7@fujitsu -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-29 Thread Aleksander Alekseev
Good news, everyone! I discovered that NUM_LOCK_OF_PARTITIONS is a bottleneck for a last patch. Long story short - NUM_LOCK_OF_PARTITIONS = (1 << 7) gives best performance: PN = 16, AN = 8, NUM_LOCK_PARTITIONS = (1 << 7): 4782.9 PN = 16, AN = 4, NUM_LOCK_PARTITIONS = (1 << 7): 4089.9 PN = 16,

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-22 Thread Aleksander Alekseev
> > Actually, I'd like to improve all partitioned hashes instead of > > improve only one case. > > Yeah. I'm not sure that should be an LWLock rather than a spinlock, > but we can benchmark it both ways. I would like to share some preliminary results. I tested four implementations: - no

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-18 Thread Aleksander Alekseev
> Could we split one freelist in hash to NUM_LOCK_PARTITIONS freelists? > Each partition will have its own freelist and if freelist is empty > then partition should search an entry in freelists of other > partitions. To prevent concurrent access it's needed to add one > LWLock to hash, each

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-18 Thread Aleksander Alekseev
> This idea can improve the situation with ProcLock hash table, but I > think IIUC what Andres is suggesting would reduce the contention > around dynahash freelist and can be helpful in many more situations > including BufMapping locks. I agree. But as I understand PostgreSQL community doesn't

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-28 Thread Aleksander Alekseev
Here is another preliminary result I would like to share. As always you will find corresponding patch in attachment. It has work in progress quality. The idea was suggested by colleague of mine Aleksander Lebedev. freeList is partitioned like in "no lock" patch. When there is no enough free

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-24 Thread Aleksander Alekseev
Oops, wrong patches - here are correct ones.diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c index 0e7acbf..3330c8d 100644 --- a/src/backend/utils/resowner/resowner.c +++ b/src/backend/utils/resowner/resowner.c @@ -29,6 +29,156 @@ #include

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-24 Thread Aleksander Alekseev
I believe I fixed all flaws mentioned so far (see attachment). Also I did a new benchmark to make sure that new patch makes PostgreSQL work faster and doesn't cause performance degradation in some cases. "usual pgbench" row corresponds to `pgbench -j 8 -c 8 -T 30 pgbench` performed on a 4-core

Re: [HACKERS] WIP: bloom filter in Hash Joins with batches

2015-12-24 Thread Aleksander Alekseev
Hello, Tomas. Great idea! Did you consider to cache bloom filter or at least part(s) of it somehow? I think this way we could gain some more TPS. This of course assuming that creating a bloom filter is really a bottleneck here, which would be nice be investigated first. Best regards, Aleksander

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-30 Thread Aleksander Alekseev
Here is a clean version of the patch. Step 1 is an optimization. Step 2 refactors this: HTAB * ShmemInitHash(const char *name, /* table string name for shmem index */ - long init_size, /* initial table size */ + long init_size, /* initial table size XXX ignored,

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-30 Thread Aleksander Alekseev
wrote: > Andres Freund wrote: > > On 2015-12-30 11:37:19 -0300, Alvaro Herrera wrote: > > > Aleksander Alekseev wrote: > > > > > > > Here is a funny thing - benchmark results I shared 22.12.2015 > > > > are wrong because I forgot to run `make clean` a

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-17 Thread Aleksander Alekseev
> Oh, that's an interesting idea. I guess the problem is that if the > freelist is unshared, then users might get an error that the lock > table is full when some other partition still has elements remaining. True, but I don't believe it should be a real problem assuming we have a strong hash

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-01-12 Thread Aleksander Alekseev
Hello, Álvaro > So you were saying some posts upthread that the new hash tables would > "borrow" AN elements from the freelist then put AN elements back when > too many got unused. Is that idea embodied in this patch? Right. It didn't satisfy "use all memory" requirement anyway. It's better to

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-02-08 Thread Aleksander Alekseev
Hello, Robert. > So: do we have clear evidence that we need 128 partitions here, or > might, say, 16 work just fine? Yes, this number of partitions was chosen based on this benchmark (see "spinlock array" column): http://www.postgresql.org/message-id/20151229184851.1bb7d1bd@fujitsu In fact we

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-02-12 Thread Aleksander Alekseev
Hello, Robert > It also strikes me that it's probably quite likely that slock_t > mutex[NUM_FREELISTS] is a poor way to lay out this data in memory. > For example, on a system where slock_t is just one byte, most likely > all of those mutexes are going to be in the same cache line, which > means

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-02-10 Thread Aleksander Alekseev
Hello, Robert > Basically, the burden for you to impose a new coding rule on everybody > who uses shared hash tables in the future is very high. I fixed an issue you described. Number of spinlocks doesn't depend of NUM_LOCK_PARTITIONS anymore and could be specified for each hash table on a

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-02-11 Thread Aleksander Alekseev
Hello, Robert > Thanks, this looks way better. Some more comments: > > - I don't think there's any good reason for this patch to change the > calling convention for ShmemInitHash(). Maybe that's a good idea and > maybe it isn't, but it's a separate issue from what this patch is > doing, and if

Re: [HACKERS] [WIP] Effective storage of duplicates in B-tree index.

2016-01-29 Thread Aleksander Alekseev
I tested this patch on x64 and ARM servers for a few hours today. The only problem I could find is that INSERT works considerably slower after applying a patch. Beside that everything looks fine - no crashes, tests pass, memory doesn't seem to leak, etc. > Okay, now for some badness. I've

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-01-22 Thread Aleksander Alekseev
Hi, > First of all, why not merge both patches into one? They aren't too > big anyway. Agree. > I think comments should be changed, to be more informative here. > Add a comment here too. > Maybe you should explain this magic number 7 in the comment above? Done. > Then, this thread became

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-01-22 Thread Aleksander Alekseev
...@mail.gmail.com On Thu, 21 Jan 2016 16:49:12 +0530 Dilip Kumar <dilipbal...@gmail.com> wrote: > On Tue, Jan 12, 2016 at 1:50 PM, Aleksander Alekseev < > a.aleks...@postgrespro.ru> wrote: > > > > > increasing number of lock partitions (see columns "no locks", >

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2016-01-25 Thread Aleksander Alekseev
> Um, that's not too surprising, because they're exactly the same patch? Wrong diff. Here is correct one. Everything else is right. I just re-checked :) step2a: number of transactions actually processed: 16325 latency average: 49.008 ms latency stddev: 8.780 ms tps = 163.182594 (including

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2016-01-25 Thread Aleksander Alekseev
Hello, Tom > Also, there are defined ways to convert between Datum format and > other formats (DatumGetPointer() etc). This code isn't using 'em. Fixed. But I was not sure how to deal with File and Buffer types since they are ints (see fd.h and buf.h) and there is no DatumGetInt macro, only

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-01-27 Thread Aleksander Alekseev
> > This patch affects header files. By any chance didn't you forget to > > run `make clean` after applying it? As we discussed above, when you > > change .h files autotools doesn't rebuild dependent .c files: > > > > Yes, actually i always compile using "make clean;make -j20; make > install" If

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2016-01-27 Thread Aleksander Alekseev
Hello, Tom. I'm a bit concerned regarding assumption that sizeof int never exceeds 4 bytes. While this could be true today for most C compilers, standard [1][2] doesn't guarantee that. Perhaps we should add something like: StaticAssertStmt(sizeof(int) <= sizeof(int32), "int size exceeds

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-01-27 Thread Aleksander Alekseev
> This comment certainly requires some changes. Fixed. > BTW, could you explain why init_table_size was two times less than > max_table_size? I have no clue. My best guess is that it was a reasonable thing to do in the past. Then somebody changed a code and now there is little reason to use

[HACKERS] Small patch for pgstat.c: fix comment + pgindent

2016-03-10 Thread Aleksander Alekseev
Hello I noticed: http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=b6fb6471f6afaf649e52f38269fd8c5c60647669 ... that comments for procedures pgstat_progress_update_param and pgstat_progress_end_command are identical. Here is a patch that fixes this. Also one empty line added

[HACKERS] Small patch: fix warnings during compilation on FreeBSD

2016-03-10 Thread Aleksander Alekseev
Hello I noticed that master branch of PostgreSQL currently compiles with warnings on FreeBSD 10.2 RELEASE: ``` pg_locale.c:1290:12: warning: implicit declaration of function 'wcstombs_l' is invalid in C99 [-Wimplicit-function-declaration] result = wcstombs_l(to, from, tolen, locale);

Re: [HACKERS] Small patch: fix warnings during compilation on FreeBSD

2016-03-15 Thread Aleksander Alekseev
tdlib.h ... wtf? Same on FreeBSD. -- Best regards, Aleksander Alekseev http://eax.me/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Small patch: fix comments in contrib/pg_trgm/

2016-03-18 Thread Aleksander Alekseev
Here are a few more patches. On Wed, 16 Mar 2016 18:11:04 +0300 Aleksander Alekseev <a.aleks...@postgrespro.ru> wrote: > Typos for the most part. > -- Best regards, Aleksander Alekseev http://eax.me/ diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/times

Re: [HACKERS] Small patch: fix warnings during compilation on FreeBSD

2016-03-15 Thread Aleksander Alekseev
LARGE_OFF_T off_t) 1 << 31) << 31) - 1 + (((off_t) 1 << 31) << 31)) ``` ... were generated but `autoreconf -iv`. I was not sure what to do about them. Eventually I decided to keep them. Still these changes could be safely discarded. -- Best regards, Aleksander Alekseev htt

Re: [HACKERS] Small patch: fix warnings during compilation on FreeBSD

2016-03-11 Thread Aleksander Alekseev
Hello, Tom > I think what we need is configure logic to find out where wcstombs_l() > is declared, and then #include only if it's necessary to > get that definition. I haven't experimented but probably you could > make such a check with nested uses of AC_CHECK_DECL. Sounds like quite a dirty

[HACKERS] Small patch: fix double variable initializations in policy.c

2016-03-16 Thread Aleksander Alekseev
pgindent'ed. Second patch fixes this too. Naturally I checked that after applying these patches PostgreSQL still compiles and pass all regression tests. -- Best regards, Aleksander Alekseev http://eax.me/ diff --git a/src/backend/commands/policy.c b/src/backend/commands/policy.c index bb735ac..dfc6f00

Re: [HACKERS] OOM in libpq and infinite loop with getCopyStart()

2016-03-09 Thread Aleksander Alekseev
Hello, Michael Thanks a lot for steps to reproduce you provided. I tested your path on Ubuntu Linux 14.04 LTS (GCC 4.8.4) and FreeBSD 10.2 RELEASE (Clang 3.4.1). In both cases patch applies cleanly, there are no warnings during compilation and all regression tests pass. A few files are not

Re: [HACKERS] Yet another small patch - reorderbuffer.c:1099

2016-04-06 Thread Aleksander Alekseev
ion, surely that will be in a separate stanza with > > its own comment; and if we ever want to remove the list deletion > > from one of the two cases (something that strikes me as unlikely) > > then we will need to fix the comment, too. > > +1 to everything here except for the way b

[HACKERS] Small patch for snapmgr.c

2016-04-08 Thread Aleksander Alekseev
yzers happy and makes intention of this code a little more obvious. -- Best regards, Aleksander Alekseev http://eax.me/ diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index b88e012..329bbba 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/s

[HACKERS] Small patch: fix comments in contrib/pg_trgm/

2016-03-19 Thread Aleksander Alekseev
Typos for the most part. -- Best regards, Aleksander Alekseev http://eax.me/ diff --git a/contrib/pg_trgm/trgm_gin.c b/contrib/pg_trgm/trgm_gin.c index ea8edef..2e41a9f 100644 --- a/contrib/pg_trgm/trgm_gin.c +++ b/contrib/pg_trgm/trgm_gin.c @@ -204,7 +204,7 @@ gin_trgm_consistent

[HACKERS] Patch: typo, s/espaced/escaped/

2016-03-19 Thread Aleksander Alekseev
There is a typo in one word in this commit: http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=9a206d063c410df7cd5da01b169b23bff413fef5 Patch attached. -- Best regards, Aleksander Alekseev http://eax.me/ diff --git a/contrib/unaccent/generate_unaccent_rules.py b/contrib/unaccent

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-03-21 Thread Aleksander Alekseev
eeling that we are just wasting our time here. -- Best regards, Aleksander Alekseev http://eax.me/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: PROPOSAL: make PostgreSQL sanitizers-friendly (and prevent information disclosure)

2016-03-22 Thread Aleksander Alekseev
> > It's possible that memset() would be more convincing. > > +1 OK, here is corrected patch. -- Best regards, Aleksander Alekseev http://eax.me/ diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index b48e97c..273e0b0 100644 --- a/src/backend

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-03-23 Thread Aleksander Alekseev
or doubts regarding these patches please don't hesitate to ask. -- Best regards, Aleksander Alekseev http://eax.me/ diff --git a/src/backend/utils/hash/dynahash.c b/src/backend/utils/hash/dynahash.c index 24a53da..06c413c 100644 --- a/src/backend/utils/hash/dynahash.c +++ b/src/backend/uti

[HACKERS] Small patch: Change calling convention for ShmemInitHash (and fix possible bug)

2016-03-24 Thread Aleksander Alekseev
regards, Aleksander Alekseev http://eax.me/ diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c index 3d9b8e4..4213c3a 100644 --- a/contrib/pg_stat_statements/pg_stat_statements.c +++ b/contrib/pg_stat_statements/pg_stat_statements.c

Re: [HACKERS] Small patch: Change calling convention for ShmemInitHash (and fix possible bug)

2016-03-25 Thread Aleksander Alekseev
realized - there is none. -- Best regards, Aleksander Alekseev http://eax.me/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Small patch: fix code duplication in heapam.c

2016-03-25 Thread Aleksander Alekseev
it become a _real_ problem. As we know from experience, to that time it's usually much harder to fix anything than it is now. -- Best regards, Aleksander Alekseev http://eax.me/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Small patch: fix code duplication in heapam.c

2016-03-24 Thread Aleksander Alekseev
ation there is less change that somebody someday will change say heap_openrv without updating heap_openrv_extended accordingly. -- Best regards, Aleksander Alekseev http://eax.me/ diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 34ba385..c55e6a7 100644 --- a

Re: [HACKERS] Small patch: Change calling convention for ShmemInitHash (and fix possible bug)

2016-03-25 Thread Aleksander Alekseev
point of suggesting a patch that splits it into two or three separate implementations for each use case, right? :) and I misunderstood how it actually works. My apologies. -- Best regards, Aleksander Alekseev http://eax.me/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make c

Re: [HACKERS] Re: PROPOSAL: make PostgreSQL sanitizers-friendly (and prevent information disclosure)

2016-03-21 Thread Aleksander Alekseev
this case here is a patch that fixes "use of uninitialized value" reports by MemorySanitizer I managed to catch so far. -- Best regards, Aleksander Alekseev http://eax.me/ diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index b48e97c..3e81865 100644 ---

[HACKERS] PROPOSAL: make PostgreSQL sanitizers-friendly (and prevent information disclosure)

2016-03-21 Thread Aleksander Alekseev
you think? [1] http://clang.llvm.org/docs/MemorySanitizer.html [2] http://clang.llvm.org/docs/AddressSanitizer.html [3] http://clang.llvm.org/docs/LeakSanitizer.html [4] http://clang.llvm.org/docs/ThreadSanitizer.html -- Best regards, Aleksander Alekseev http://eax.me/ -- Sent via pgsql-hackers

Re: [HACKERS] Re: PROPOSAL: make PostgreSQL sanitizers-friendly (and prevent information disclosure)

2016-03-21 Thread Aleksander Alekseev
0 0x00 0x00 0x00 0x00 (gdb) quit Naturally we could use memset() as well. But I personally find it a bit less readable. And in theory it doesn't prevent some _very_ "smart" C compiler from not cleaning the whole structure anyway. -- Best regards, Aleksander Alekseev http://eax.me/

Re: [HACKERS] Add something in struct PGPROC

2016-03-23 Thread Aleksander Alekseev
header files dependent files are not recompiled automatically. -- Best regards, Aleksander Alekseev http://eax.me/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-03-23 Thread Aleksander Alekseev
> Thanks! I can't think of anything else to worry about with regards to > that version, so I have committed it. > Thanks, Robert. And thanks everyone for contributing to this patch. -- Best regards, Aleksander Alekseev http://eax.me/ -- Sent via pgsql-hackers mailing list (pgsq

[HACKERS] Small patch: fix code duplication in heapam.c

2016-03-24 Thread Aleksander Alekseev
Hello I discovered that there is a lot of code duplication in heapam.c. In particular relation_openrv and relation_openrv_extended procedures and also heap_openrv and heap_openrv_extended procedures are almost the same. Here is a patch that fixes this. -- Best regards, Aleksander Alekseev http

Re: [HACKERS] pgbench small bug fix

2016-03-04 Thread Aleksander Alekseev
> Attached is a v3 which test integers more logically. I'm a lazy > programmer who tends to minimize the number of key strokes. Well. From what I can tell this patch is Ready for Committer. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-03-03 Thread Aleksander Alekseev
> Won't it always use the same freelist to remove and add the entry from > freelist as for both cases it will calculate the freelist_idx in same > way? No. If "our" freelist is empty when we try to remove an item from it we borrow item from another freelist. Then this borrowed item will be

Re: [HACKERS] OOM in libpq and infinite loop with getCopyStart()

2016-03-03 Thread Aleksander Alekseev
Hello, Michael > The easiest way to perform tests with this patch is to take a debugger > and enforce the malloc'd pointers to NULL in the code paths. I see. Still I don't think it's an excuse to not provide clear steps to reproduce an issue. As I see it anyone should be able to easily check

Re: [HACKERS] OOM in libpq and infinite loop with getCopyStart()

2016-03-03 Thread Aleksander Alekseev
Hello, Michael I didn't checked your patch in detail yet but here is a thought I would like to share. In my experience usually it takes number of rewrites before patch will be accepted. To make sure that after every rewrite your patch still solves an issue you described you should probably

Re: [HACKERS] pgbench small bug fix

2016-03-03 Thread Aleksander Alekseev
> time pgbench -T 5 -R 0.1 -P 1 -c 2 -j 2 On my laptop this command executes 25 seconds instead of 5. I'm pretty sure it IS a bug. Probably a minor one though. I tested this patch on Ubuntu 14.04 LTS with GCC 4.8. It applies cleanly on master branch (c7111d11) and solves a described problem. No

[HACKERS] PROPOSAL: Fast temporary tables

2016-03-01 Thread Aleksander Alekseev
Hello There are applications that create and delete a lot of temporary tables. Currently PostgreSQL doesn't handle such a use case well. Consider the following benchmark/example. postgresql.conf: ``` autovacuum = on log_min_messages = debug2 ``` temp-table.pgbench: ``` create temporary table

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-03-01 Thread Aleksander Alekseev
Hello, Amit > I am not sure, if this is exactly what has been suggested by Robert, > so it is not straightforward to see if his suggestion can allow us to > use NUM_FREELISTS as 8 rather than 32. I think instead of trying to > use FREELISTBUFF, why not do it as Robert has suggested and try with

[HACKERS] Tiny patch: sigmask.diff

2016-04-04 Thread Aleksander Alekseev
iteration we are doing `<< -1`. I think it's a bug. Patch attached. -- Best regards, Aleksander Alekseev http://eax.me/ diff --git a/src/backend/port/win32/signal.c b/src/backend/port/win32/signal.c index 36c6ebd..3724aa3 100644 --- a/src/backend/port/win32/signal.c +++ b/src/backend/port/win32/

Re: [HACKERS] Yet another small patch - reorderbuffer.c:1099

2016-04-05 Thread Aleksander Alekseev
ntly. I see. If this it true I think there should be a comment that explains it. When you read such a code you suspect a bug. Not mentioning that static code analyzers (I'm currently experimenting with Clang and PVS Studio) complain about code like this. -- Best regards, Aleksander Alekseev http://eax.me

Re: [HACKERS] OOM in libpq and infinite loop with getCopyStart()

2016-03-30 Thread Aleksander Alekseev
e > reported problem. There is one similar problem [2] reported by Heikki > which I think can be fixed separately. > [...] > Let me know if you think anything more from myside can help in moving > patch. +1, same here. Lets see whats committer's opinion. -- Best regards, Aleksa

Re: [HACKERS] WIP: Access method extendability

2016-03-30 Thread Aleksander Alekseev
ard to reason about. You could do it much simpler choosing only one thing to do --- either allocate a new structure or use a provided one. 5) Code is not pgindent'ed and has many trailing spaces. [1] http://www.postgresql.org/message-id/56eff347.20...@anastigmatix.net -- Best regards, Aleks

Re: [HACKERS] WIP: Access method extendability

2016-03-30 Thread Aleksander Alekseev
stgreSQL sanitizers-friendly or not in a corresponding thread. So far no one spoke against this idea. Thus I don't think that new patches should complicate implementing it. Especially considering that it's very simple to do and even is considered a good practice according to PostgreSQL documentation. -- Best regar

Re: [HACKERS] Small patch: --disable-setproctitle flag

2016-04-01 Thread Aleksander Alekseev
Hello, Andres > Seems more appropriate to simply manually add a #undef > HAVE_SETPROCTITLE to pg_config_manual.h in that case. Adding > configure flags for ephemeral debugger issues seems like a high churn > activity. I think you are right. OK. -- Best regards, Aleksander A

Re: [HACKERS] WIP: Access method extendability

2016-04-01 Thread Aleksander Alekseev
ng against goto. But are you sure that using it here is really justified? -- Best regards, Aleksander Alekseev http://eax.me/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Two division by 0 errors in optimizer/plan/planner.c and optimizer/path/costsize.c

2016-03-28 Thread Aleksander Alekseev
ork, no stacktraces in gdb. Could you please provide more concrete steps to reproduce these issues i.e, OS and compiler version, compilation flags (or package version), cluster config, database schema, etc? These steps are required at least to make sure that fixed code really fixes a problem. Also it

Re: [HACKERS] WIP: Access method extendability

2016-04-01 Thread Aleksander Alekseev
Hello > Fixed. Thanks. I don't have any other suggestions at the moment. Let see whats committers opinion on this. -- Best regards, Aleksander Alekseev http://eax.me/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: h

[HACKERS] Yet another small patch - reorderbuffer.c:1099

2016-04-04 Thread Aleksander Alekseev
ed in one commit b89e1510. I added author and committer of this code to CC since they have much better understanding of it than I do. -- Best regards, Aleksander Alekseev http://eax.me/ diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffe

Re: [HACKERS] Tiny patch: sigmask.diff

2016-04-04 Thread Aleksander Alekseev
any event waiting there. Agreed. -- Best regards, Aleksander Alekseev http://eax.me/ diff --git a/src/backend/port/win32/signal.c b/src/backend/port/win32/signal.c index 36c6ebd..0ba2817 100644 --- a/src/backend/port/win32/signal.c +++ b/src/backend/port/win32/signal.c @@ -113,7 +113,7 @@ pgw

[HACKERS] Small patch: --disable-setproctitle flag

2016-03-31 Thread Aleksander Alekseev
ing patch is attached. -- Best regards, Aleksander Alekseev http://eax.me/ diff --git a/configure b/configure index 24655dc..87e845e 100755 --- a/configure +++ b/configure @@ -819,6 +819,7 @@ with_wal_segsize with_CC enable_depend enable_cassert +enable_setproctitle enable_thread_safety with_

Re: [HACKERS] Parser extensions (maybe for 10?)

2016-04-22 Thread Aleksander Alekseev
apologies to anyone who was offended in any way. -- Best regards, Aleksander Alekseev http://eax.me/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parser extensions (maybe for 10?)

2016-04-19 Thread Aleksander Alekseev
> On 2016-04-19 12:20:03 +0300, Aleksander Alekseev wrote: > > Can we guarantee that extensions don't conflict? In fact we can > > since we already do it. If all tests pass there is no conflict. > > How does that follow? Even if you were to test all possible extensions &

[HACKERS] Should we remove unused code?

2016-04-18 Thread Aleksander Alekseev
regards, Aleksander Alekseev http://eax.me/ diff --git a/src/backend/utils/hash/dynahash.c b/src/backend/utils/hash/dynahash.c index 1a9f70c..9b32ceb 100644 --- a/src/backend/utils/hash/dynahash.c +++ b/src/backend/utils/hash/dynahash.c @@ -452,16 +452,6 @@ hash_create(const char *tabname, long nelem

Re: [HACKERS] Parser extensions (maybe for 10?)

2016-04-18 Thread Aleksander Alekseev
implement full SQL - only few > statements. > > Is it this idea more workable? What if there are two or more contribs that extend the parser? Can we be sure that these contribs will not conflict? -- Best regards, Aleksander Alekseev http://eax.me/ -- Sent via pgsql-hackers mai

Re: [HACKERS] Wire protocol compression

2016-04-21 Thread Aleksander Alekseev
o > add that. It just doesn't sound like a feature that should be implemented separately for every single application that uses TCP. Granted TCP proxy is not the most convenient way to solve a task. Maybe it could be implemented in OpenVPN or on Linux TCP/IP stack level. -- Best regards, Aleksand

Re: [HACKERS] Wire protocol compression

2016-04-21 Thread Aleksander Alekseev
ion and encryption. Just as an example. -- Best regards, Aleksander Alekseev http://eax.me/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parser extensions (maybe for 10?)

2016-04-19 Thread Aleksander Alekseev
default priority of parser extension. Extensions with higher priority are applied first. Also user can resolve conflicts by manually overriding these priorities: ``` select pg_parser_extension_priorities(); select pg_override_parser_extension_priority('some_extension', 100500); ``` I think it should

Re: [HACKERS] Coverage report

2016-04-19 Thread Aleksander Alekseev
% (179603 of 284239 lines) functions..: 69.5% (10550 of 15183 functions) ``` Maybe we should send an e-mail to pgsql-hackers@ automatically if these numbers drops. -- Best regards, Aleksander Alekseev http://eax.me/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.o

Re: [HACKERS] Should we remove unused code?

2016-04-18 Thread Aleksander Alekseev
.h files considered a public API and should be backward compatible? In this case - for how long, forever or until next major release? Or lets say there is some API that is not used neither internally nor by any package available on PGXN. Can we break API then? -- Best regards, Aleksander Alek

Re: [HACKERS] Parser extensions (maybe for 10?)

2016-04-18 Thread Aleksander Alekseev
". Lets say extension A allows "SELECT ASAP ..." queries and extension B --- "... ORDER BY RANK". What happens when user executes "SELECT ASAP ... ORDER BY RANK" query? -- Best regards, Aleksander Alekseev http://eax.me/ -- Sent via pgsql-hackers mailing lis

Re: [HACKERS] Parser extensions (maybe for 10?)

2016-04-19 Thread Aleksander Alekseev
without any changes in PostgreSQL core. -- Best regards, Aleksander Alekseev http://eax.me/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Wire protocol compression

2016-04-21 Thread Aleksander Alekseev
xist. -- Best regards, Aleksander Alekseev http://eax.me/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] [Patch] RBTree iteration interface improvement

2016-07-27 Thread Aleksander Alekseev
current interface since in theory some extensions can use it. Still I believe such a move is worth considering. -- Best regards, Aleksander Alekseev diff --git a/src/backend/access/gin/ginbulk.c b/src/backend/access/gin/ginbulk.c index d6422ea..eac76c4 100644 --- a/src/backend/access/gin/ginbulk.c

Re: [HACKERS] [Patch] RBTree iteration interface improvement

2016-07-27 Thread Aleksander Alekseev
And as always, I didn't re-check a patch before sending it :) Here is a corrected version without any fprintf's. -- Best regards, Aleksander Alekseev diff --git a/src/backend/access/gin/ginbulk.c b/src/backend/access/gin/ginbulk.c index d6422ea..eac76c4 100644 --- a/src/backend/access/gin

Re: [HACKERS] [Patch] RBTree iteration interface improvement

2016-07-28 Thread Aleksander Alekseev
m.c procedures. Exact order may depend on user's query so you don't even control it. -- Best regards, Aleksander Alekseev -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-07-29 Thread Aleksander Alekseev
y feedback on this patch will be much appreciated! -- Best regards, Aleksander Alekseev diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index 0689cc9..940210b 100644 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -1693,7 +1693,7 @@

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-01 Thread Aleksander Alekseev
r. I look forward to receive more comments and questions regarding this patch! -- Best regards, Aleksander Alekseev -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-04 Thread Aleksander Alekseev
managed to correct this flaw. If you believe that some parts of the code are still poorly commented or could be improved in any other way please let me know. And as usual, any other comments, remarks or questions are highly appreciated! -- Best regards, Aleksander Alekseev diff --git a/doc/src

  1   2   3   >