Re: [HACKERS] Secondary index access optimizations

2017-12-04 Thread Konstantin Knizhnik
On 04.12.2017 19:44, Alvaro Herrera wrote: Konstantin Knizhnik wrote: On 30.11.2017 05:16, Michael Paquier wrote: On Mon, Nov 6, 2017 at 10:13 PM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: Concerning broken partition_join test: it is "expected" failure: my pa

Re: [HACKERS] Surjective functional indexes

2017-12-15 Thread Konstantin Knizhnik
On 15.12.2017 01:21, Michael Paquier wrote: On Fri, Dec 15, 2017 at 6:15 AM, Alvaro Herrera <alvhe...@alvh.no-ip.org> wrote: Konstantin Knizhnik wrote: If you still thing that additional 16 bytes per relation in statistic is too high overhead, then I will also remove autotune. I thin

Re: [HACKERS] Surjective functional indexes

2017-12-13 Thread Konstantin Knizhnik
Thank you for review. On 13.12.2017 14:29, Simon Riggs wrote: On 4 December 2017 at 15:35, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: On 30.11.2017 05:02, Michael Paquier wrote: On Wed, Sep 27, 2017 at 4:07 PM, Simon Riggs <si...@2ndquadrant.com> wrote: On 15 Septembe

Re: [HACKERS] Surjective functional indexes

2017-12-14 Thread Konstantin Knizhnik
On 13.12.2017 14:29, Simon Riggs wrote: On 4 December 2017 at 15:35, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: On 30.11.2017 05:02, Michael Paquier wrote: On Wed, Sep 27, 2017 at 4:07 PM, Simon Riggs <si...@2ndquadrant.com> wrote: On 15 September 2017 at 16:34

AS OF queries

2017-12-20 Thread Konstantin Knizhnik
| 2017-12-20 14:59:22.933753 | insert   3 | 2017-12-20 14:59:27.87712  | insert   1 | 2017-12-20 15:09:17.046047 | upd (3 rows) Comments and feedback are welcome:) -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company diff --git a/contrib

Re: [HACKERS] Partition-wise aggregation/grouping

2017-11-15 Thread Konstantin Knizhnik
work for inherited tables, although there seems to be not so much difference between this two cases. Do you think that sometimes it will be also supported for standard inheritance mechanism or there is no sense in it? -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Cached plans and statement generalization

2017-12-04 Thread Konstantin Knizhnik
On 30.11.2017 04:59, Michael Paquier wrote: On Wed, Sep 13, 2017 at 2:11 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: One more patch passing all regression tests with autoprepare_threshold=1. I still do not think that it should be switch on by default... This patc

Re: [HACKERS] Surjective functional indexes

2017-12-04 Thread Konstantin Knizhnik
On 30.11.2017 05:02, Michael Paquier wrote: On Wed, Sep 27, 2017 at 4:07 PM, Simon Riggs <si...@2ndquadrant.com> wrote: On 15 September 2017 at 16:34, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: Attached please find yet another version of the patch. Thanks. I'm review

Re: Postgres with pthread

2017-12-07 Thread Konstantin Knizhnik
Hi On 06.12.2017 20:08, Andres Freund wrote: 4. Rewrite file descriptor cache to be global (shared by all threads). That one I'm very unconvinced of, that's going to add a ton of new contention. Do you mean lock contention because of mutex I used to synchronize access to shared file

Re: Postgres with pthread

2017-12-07 Thread Konstantin Knizhnik
to __thread variables is not more expensive then access to static variable or access to fields of session context structure through current_session.  And there is no extra space overhead for them. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Postgres with pthread

2017-12-06 Thread Konstantin Knizhnik
Postgres for this architecture). I just want to receive some feedback and know if community is interested in any further work in this direction. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: Postgres with pthread

2017-12-08 Thread konstantin knizhnik
On Dec 7, 2017, at 10:41 AM, Simon Riggs wrote: >> But it is a theory. The main idea of this prototype was to prove or disprove >> this expectation at practice. > >> But please notice that it is very raw prototype. A lot of stuff is not >> working yet. > >> And supporting all of exited

Re: Having query cache in core

2018-05-07 Thread Konstantin Knizhnik
will be similar with other shared caches implementation. If we have infrastructure for building efficient shared caches (lockless algorithms, smart invalidation, ...) then it will be not so difficult to implement result cache on top of it. -- Konstantin Knizhnik Postgres Professional: http

Re: Having query cache in core

2018-05-07 Thread Konstantin Knizhnik
On 07.05.2018 11:24, Tsunakawa, Takayuki wrote: From: Konstantin Knizhnik [mailto:k.knizh...@postgrespro.ru] But I think it is better to start first with 1. Global prepared statements cache 2. Global catalog cache 3. Global relation cache May I ask why prepared statements need to precede

Re: Built-in connection pooling

2018-05-07 Thread Konstantin Knizhnik
content... Or make it possible to change lifetime of temporary tables from session to transaction... -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: Built-in connection pooling

2018-05-04 Thread Konstantin Knizhnik
caused by differences in pg_prepared_statements caused by "mangled" prepared names. Failures of isolation tests are caused by unsupported advisory locks. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: Built-in connection pooling

2018-04-27 Thread Konstantin Knizhnik
On 27.04.2018 16:49, Merlin Moncure wrote: On Thu, Apr 26, 2018 at 6:04 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: On 25.04.2018 20:02, Merlin Moncure wrote: Yep. The main workaround today is to disable them. Having said that, it's not that difficult to imagine h

Re: Built-in connection pooling

2018-04-27 Thread Konstantin Knizhnik
On 27.04.2018 18:33, Merlin Moncure wrote: On Fri, Apr 27, 2018 at 10:05 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: On 27.04.2018 16:49, Merlin Moncure wrote: *) How are you pinning client connections to an application managed transaction? (IMNSHO, this feature is u

Re: Built-in connection pooling

2018-04-26 Thread Konstantin Knizhnik
with GUCs. Most of them are really associated with backend, not with session. So them should not be handled by reschedule. But there may be some variables which are intended to be session specific. And locating this variables is really non trivial task. -- Konstantin Knizhnik Postgres Professional

Re: Built-in connection pooling

2018-04-28 Thread Konstantin Knizhnik
On 27.04.2018 23:43, Merlin Moncure wrote: On Fri, Apr 27, 2018 at 11:44 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: On 27.04.2018 18:33, Merlin Moncure wrote: On Fri, Apr 27, 2018 at 10:05 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: On 27.0

Re: [HACKERS] Surjective functional indexes

2018-05-11 Thread Konstantin Knizhnik
On 11.05.2018 07:48, David G. Johnston wrote: On Thursday, February 1, 2018, Konstantin Knizhnik <k.knizh...@postgrespro.ru <mailto:k.knizh...@postgrespro.ru>> wrote: Old + New for check = 2 plus calculate again in index = 3 Yes, we have to calculate the va

Re: Built-in connection pooling

2018-05-08 Thread Konstantin Knizhnik
porary table, advisory locks,... with built-in connection pooler you can use sessions but with some restrictions (lack of advisory locks, for example). It is better than nothing, isn't it? -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: Removing unneeded self joins

2018-05-17 Thread Konstantin Knizhnik
performance and relation size can not dramatically changed during this time. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Possible bug in logical replication.

2018-05-17 Thread Konstantin Knizhnik
lity of it seems to be quite small... I am not so familiar with wal iteration code and logical decoding, so I will be pleased to receive confirmation orconfutation of my analysis of the problem. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: Possible bug in logical replication.

2018-05-17 Thread Konstantin Knizhnik
On 17.05.2018 10:45, Konstantin Knizhnik wrote: We got the following assertion failure at our buildfarm of master branch of Postgres in contrib/test_decoding regression tests: 2018-05-07 19:50:07.241 MSK [5af083bf.54ae:49] DETAIL: Streaming transactions committing after 0/2A0, reading

Re: libpq compression

2018-05-16 Thread Konstantin Knizhnik
On 16.05.2018 18:09, Grigory Smolkin wrote: Hello! I have noticed that psql --help lack -Z|--compression option. Also it would be nice to have option like --compression-level in psql and pgbench. Thank you for this notice. Updated and rebased patch is attached. Concerning specification of

Re: libpq compression

2018-05-15 Thread Konstantin Knizhnik
On 15.05.2018 13:23, Dmitry Dolgov wrote: > On 30 March 2018 at 14:53, Konstantin Knizhnik <k.knizh...@postgrespro.ru <mailto:k.knizh...@postgrespro.ru>> wrote: > Hi hackers, > One of our customers was managed to improve speed about 10 times by using SSL compression

Re: libpq compression

2018-06-06 Thread Konstantin Knizhnik
On 06.06.2018 10:53, Michael Paquier wrote: On Tue, Jun 05, 2018 at 06:58:42PM +0300, Konstantin Knizhnik wrote: I have considered this patch mostly as prototype to estimate efficiency of libpq protocol compression and compare it with SSL compression. So I agree with you that there are a lot

Re: libpq compression

2018-06-06 Thread Konstantin Knizhnik
On 06.06.2018 02:03, Thomas Munro wrote: On Wed, Jun 6, 2018 at 2:06 AM, Konstantin Knizhnik wrote: Thank you for review. Updated version of the patch fixing all reported problems is attached. Small problem on Windows[1]: C:\projects\postgresql\src\include\common/zpq_stream.h(17): error

Re: WAL prefetch

2018-06-15 Thread Konstantin Knizhnik
On 16.06.2018 06:33, Amit Kapila wrote: On Fri, Jun 15, 2018 at 11:31 PM, Andres Freund wrote: On 2018-06-14 10:13:44 +0300, Konstantin Knizhnik wrote: On 14.06.2018 09:52, Thomas Munro wrote: On Thu, Jun 14, 2018 at 1:09 AM, Konstantin Knizhnik wrote: pg_wal_prefetch function

Re: WAL prefetch

2018-06-15 Thread Konstantin Knizhnik
On 16.06.2018 06:30, Amit Kapila wrote: On Fri, Jun 15, 2018 at 8:45 PM, Konstantin Knizhnik wrote: On 15.06.2018 18:03, Amit Kapila wrote: wal_prefetch is prefetching blocks referenced by WAL records. But in case of "full page writes" such prefetch is not needed and even

Re: WAL prefetch

2018-06-14 Thread Konstantin Knizhnik
. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: WAL prefetch

2018-06-16 Thread Konstantin Knizhnik
On 16.06.2018 22:02, Andres Freund wrote: On 2018-06-16 11:38:59 +0200, Tomas Vondra wrote: On 06/15/2018 08:01 PM, Andres Freund wrote: On 2018-06-14 10:13:44 +0300, Konstantin Knizhnik wrote: On 14.06.2018 09:52, Thomas Munro wrote: On Thu, Jun 14, 2018 at 1:09 AM, Konstantin Knizhnik

Re: WAL prefetch

2018-06-16 Thread Konstantin Knizhnik
On 16.06.2018 22:23, Andres Freund wrote: Hi, On 2018-06-13 16:09:45 +0300, Konstantin Knizhnik wrote: Usage: 1. At master: create extension wal_prefetch 2. At replica: Call pg_wal_prefetch() function: it will not return until you interrupt it. FWIW, I think the proper design would rather

WAL prefetch

2018-06-13 Thread Konstantin Knizhnik
. It is possible to explicitly specify start LSN for pg_wal_prefetch() function. Otherwise, WAL redo position will be used as start LSN. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company #include #include #include #include #include #include

Re: WAL prefetch

2018-06-14 Thread Konstantin Knizhnik
On 14.06.2018 16:25, Robert Haas wrote: On Thu, Jun 14, 2018 at 9:23 AM, Konstantin Knizhnik wrote: Speed of random HDD access is limited by speed of disk head movement. By running several IO requests in parallel we just increase probability of head movement, so actually parallel access

Re: WAL prefetch

2018-06-15 Thread Konstantin Knizhnik
On 15.06.2018 18:03, Amit Kapila wrote: On Fri, Jun 15, 2018 at 1:08 PM, Konstantin Knizhnik wrote: On 15.06.2018 07:36, Amit Kapila wrote: On Fri, Jun 15, 2018 at 12:16 AM, Stephen Frost wrote: I have tested wal_prefetch at two powerful servers with 24 cores, 3Tb NVME RAID 10 storage

Re: WAL prefetch

2018-06-17 Thread Konstantin Knizhnik
On 17.06.2018 03:00, Andres Freund wrote: On 2018-06-16 23:25:34 +0300, Konstantin Knizhnik wrote: On 16.06.2018 22:02, Andres Freund wrote: On 2018-06-16 11:38:59 +0200, Tomas Vondra wrote: On 06/15/2018 08:01 PM, Andres Freund wrote: On 2018-06-14 10:13:44 +0300, Konstantin Knizhnik

Re: WAL prefetch

2018-06-19 Thread Konstantin Knizhnik
On 19.06.2018 18:50, Andres Freund wrote: On 2018-06-19 12:08:27 +0300, Konstantin Knizhnik wrote: I do not think that prefetching in shared buffers requires much more efforts and make patch more envasive... It even somehow simplify it, because there is no to maintain own cache of prefetched

Re: WAL prefetch

2018-06-14 Thread Konstantin Knizhnik
On 14.06.2018 09:52, Thomas Munro wrote: On Thu, Jun 14, 2018 at 1:09 AM, Konstantin Knizhnik wrote: pg_wal_prefetch function will infinitely traverse WAL and prefetch block references in WAL records using posix_fadvise(WILLNEED) system call. Hi Konstantin, Why stop at the page cache

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2018-06-05 Thread Konstantin Knizhnik
On 04.06.2018 21:42, Andres Freund wrote: Hi, On 2018-06-04 16:47:29 +0300, Konstantin Knizhnik wrote: We in PostgresProc were faced with lock extension contention problem at two more customers and tried to use this patch (v13) to address this issue. Unfortunately replacing heavy lock

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2018-06-05 Thread Konstantin Knizhnik
On 05.06.2018 07:22, Masahiko Sawada wrote: On Mon, Jun 4, 2018 at 10:47 PM, Konstantin Knizhnik wrote: On 26.04.2018 09:10, Masahiko Sawada wrote: On Thu, Apr 26, 2018 at 3:30 AM, Robert Haas wrote: On Tue, Apr 10, 2018 at 9:08 PM, Masahiko Sawada wrote: Never mind. There was a lot

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2018-06-05 Thread Konstantin Knizhnik
On 05.06.2018 13:29, Masahiko Sawada wrote: On Tue, Jun 5, 2018 at 6:47 PM, Konstantin Knizhnik wrote: On 05.06.2018 07:22, Masahiko Sawada wrote: On Mon, Jun 4, 2018 at 10:47 PM, Konstantin Knizhnik wrote: On 26.04.2018 09:10, Masahiko Sawada wrote: On Thu, Apr 26, 2018 at 3:30 AM

Re: libpq compression

2018-06-05 Thread Konstantin Knizhnik
On 05.06.2018 08:26, Thomas Munro wrote: On Thu, May 17, 2018 at 3:54 AM, Konstantin Knizhnik wrote: Thank you for this notice. Updated and rebased patch is attached. Hi Konstantin, Seems very useful. +1. + rc = inflate(>rx, Z_SYNC_FLUSH); + if (rc != Z_OK) + { + ret

Re: libpq compression

2018-06-05 Thread Konstantin Knizhnik
On 05.06.2018 10:09, Michael Paquier wrote: On Tue, Jun 05, 2018 at 06:04:21PM +1200, Thomas Munro wrote: On Thu, May 17, 2018 at 3:54 AM, Konstantin Knizhnik Speaking of configuration, are you planning to support multiple compression libraries at the same time? It looks like the current

Re: libpq compression

2018-06-05 Thread Konstantin Knizhnik
On 05.06.2018 09:04, Thomas Munro wrote: On Thu, May 17, 2018 at 3:54 AM, Konstantin Knizhnik wrote: Concerning specification of compression level: I have made many experiments with different data sets and both zlib/zstd and in both cases using compression level higher than default doesn't

Re: I'd like to discuss scaleout at PGCon

2018-06-06 Thread Konstantin Knizhnik
h provides efficient extraction and processing of data. With abstract storage API Postgres is also given a chance to implement efficient storage for OLAP data processing. But huge amount of work has to be done here. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russia

Re: libpq compression

2018-06-06 Thread Konstantin Knizhnik
is with pg_dump which establish connection with server to fetch data from the database and is able to compress output data. So here we may need two options: compress input and compress output.  But I do not think that because of it -Z option should be removed from psql and pgbench. -- Konstantin

Re: libpq compression

2018-06-06 Thread Konstantin Knizhnik
On 06.06.2018 19:33, Konstantin Knizhnik wrote: On 05.06.2018 20:06, Peter Eisentraut wrote: On 6/5/18 03:09, Michael Paquier wrote: I just had a quick look at this patch, lured by the smell of your latest messages...  And it seems to me that this patch needs a heavy amount of work

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2018-06-04 Thread Konstantin Knizhnik
transaction doesn't release this lock (unlike heavy weight lock) and the relation is kept locked. So database is actually stalled and server has to be restarted. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: Global shared meta cache

2018-07-02 Thread Konstantin Knizhnik
surprises here: negative effect of shared cache is the largest for the case of non-prepared selects (because selects themselves are much faster than updates and during compilation we have to access relations multiple times). -- Konstantin Knizhnik Postgres Professi

Re: libpq compression

2018-06-22 Thread Konstantin Knizhnik
On 22.06.2018 19:05, Nico Williams wrote: On Fri, Jun 22, 2018 at 10:18:12AM +0300, Konstantin Knizhnik wrote: On 22.06.2018 00:34, Nico Williams wrote: So I think you just have to have lengths. Now, this being about compression, I understand that you might now want to have 4-byte lengths

Re: libpq compression

2018-06-23 Thread Konstantin Knizhnik
On 22.06.2018 20:56, Robbie Harwood wrote: Konstantin Knizhnik writes: On 22.06.2018 18:59, Robbie Harwood wrote: Konstantin Knizhnik writes: On 21.06.2018 20:14, Robbie Harwood wrote: Konstantin Knizhnik writes: On 21.06.2018 17:56, Robbie Harwood wrote: Konstantin Knizhnik writes

Re: libpq compression

2018-06-22 Thread Konstantin Knizhnik
On 22.06.2018 18:59, Robbie Harwood wrote: Konstantin Knizhnik writes: On 21.06.2018 20:14, Robbie Harwood wrote: Konstantin Knizhnik writes: On 21.06.2018 17:56, Robbie Harwood wrote: Konstantin Knizhnik writes: On 20.06.2018 23:34, Robbie Harwood wrote: Konstantin Knizhnik writes

Re: libpq compression

2018-06-25 Thread Konstantin Knizhnik
to set up compressed vs. uncompressed connections - similarly to how we've documentation on setting up TLS connection (though presumably compressed connection documentation will be shorter). Document protocol changes needed for libpq compression. -- Konstantin Knizhnik Postgres Professional: http

Re: WAL prefetch

2018-06-19 Thread Konstantin Knizhnik
ability/3 -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: libpq compression

2018-06-20 Thread Konstantin Knizhnik
On 20.06.2018 00:04, Robbie Harwood wrote: Konstantin Knizhnik writes: On 18.06.2018 23:34, Robbie Harwood wrote: I also don't like that you've injected into the *startup* path - before authentication takes place. Fundamentally, authentication (if it happens) consists of exchanging some

Re: libpq compression

2018-06-21 Thread Konstantin Knizhnik
On 20.06.2018 23:34, Robbie Harwood wrote: Konstantin Knizhnik writes: My idea was the following: client want to use compression. But server may reject this attempt (for any reasons: it doesn't support it, has no proper compression library, do not want to spend CPU for decompression

Unusable index

2018-07-03 Thread Konstantin Knizhnik
so dramatically increases query execution time in another database also seems to be very disappointing. Thanks in advance, -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: Built-in connection pooling

2018-04-26 Thread Konstantin Knizhnik
to why we would take another approach than 'don't use the non-xact variants of them in a pooling environment'. merlin -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: AS OF queries

2017-12-27 Thread Konstantin Knizhnik
On 27.12.2017 00:52, Jeff Janes wrote: On Thu, Dec 21, 2017 at 6:00 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru <mailto:k.knizh...@postgrespro.ru>> wrote: There is still one significant difference of my prototype implementation with SQL standard: it associat

Re: AS OF queries

2017-12-27 Thread Konstantin Knizhnik
On 27.12.2017 17:14, PostgreSQL - Hans-Jürgen Schönig wrote: On 12/20/2017 01:45 PM, Konstantin Knizhnik wrote: I wonder if Postgres community is interested in supporting time travel queries in PostgreSQL (something like AS OF queries in Oracle: https://docs.oracle.com/cd/B14117_01/appdev

Re: AS OF queries

2017-12-29 Thread Konstantin Knizhnik
On 28.12.2017 20:28, Peter Eisentraut wrote: On 12/28/17 11:36, Konstantin Knizhnik wrote: Attached please find new version of AS OF patch which allows to specify time travel period. Older versions outside this period may be reclaimed by autovacuum. This behavior is controlled

Re: AS OF queries

2017-12-23 Thread konstantin knizhnik
On Dec 23, 2017, at 2:08 AM, Greg Stark wrote: > On 20 December 2017 at 12:45, Konstantin Knizhnik > <k.knizh...@postgrespro.ru> wrote: > >> It seems to me that it will be not so difficult to implement them in >> Postgres - we already have versions of tuples. >&

Re: AS OF queries

2017-12-25 Thread Konstantin Knizhnik
On 25.12.2017 06:26, Craig Ringer wrote: On 24 December 2017 at 04:53, konstantin knizhnik <k.knizh...@postgrespro.ru <mailto:k.knizh...@postgrespro.ru>> wrote: But what if I just forbid to change recent_global_xmin? If it is stalled at FirstNormalTransactionId and n

Re: AS OF queries

2018-01-10 Thread Konstantin Knizhnik
xample some column was added or removed which is not used in this query), then should we really throw error in this case? -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Secondary index access optimizations

2018-01-10 Thread Konstantin Knizhnik
On 09.01.2018 19:48, Antonin Houska wrote: Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: On 14.08.2017 19:33, Konstantin Knizhnik wrote: On 14.08.2017 12:37, Konstantin Knizhnik wrote: Hi hackers, I am trying to compare different ways of optimizing work with huge

Re: [HACKERS] Secondary index access optimizations

2018-01-11 Thread Konstantin Knizhnik
On 11.01.2018 12:34, Antonin Houska wrote: Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: On 09.01.2018 19:48, Antonin Houska wrote: Have you considered using the range types (internally in operator_predicate_proof()) instead of hard-coding operator OIDs? The range types d

Re: [HACKERS] Surjective functional indexes

2018-01-10 Thread Konstantin Knizhnik
On 07.01.2018 01:59, Stephen Frost wrote: Greetings, * Konstantin Knizhnik (k.knizh...@postgrespro.ru) wrote: On 15.12.2017 01:21, Michael Paquier wrote: On Fri, Dec 15, 2017 at 6:15 AM, Alvaro Herrera <alvhe...@alvh.no-ip.org> wrote: Konstantin Knizhnik wrote: If you still

Re: AS OF queries

2018-01-10 Thread Konstantin Knizhnik
? Attached please find new version of the AS OF patch which throws error if specified AS OF timestamp is older that time travel horizon and "check_asof_timestamp" parameter is set to true (by default it is switched off). -- Konstantin Knizhnik Postgres Professi

Builtin connection polling

2018-01-17 Thread Konstantin Knizhnik
ing socket descriptors between processes was implemented only for Unix and tested only at Linux, although is expected to work also as MacOS and other Unix dialects. Windows is not supported now. I will be glad to receive an feedback and suggestion concerning perspectives of embedded conne

Re: Built-in connection pooling

2018-01-18 Thread Konstantin Knizhnik
On 17.01.2018 19:09, Konstantin Knizhnik wrote: Hi hackers, My recent experiments with pthread version of Postgres show that although pthread offers some performance advantages comparing with processes for large number of connections, them still can not eliminate need in connection pooling

Re: Built-in connection pooling

2018-01-19 Thread Konstantin Knizhnik
On 18.01.2018 18:00, Claudio Freire wrote: On Thu, Jan 18, 2018 at 11:48 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru <mailto:k.knizh...@postgrespro.ru>> wrote: Attached please find new version of the patch with few fixes. And more results at NUMA system wi

Re: [HACKERS] Secondary index access optimizations

2018-01-19 Thread Konstantin Knizhnik
On 19.01.2018 16:14, Antonin Houska wrote: Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: On 11.01.2018 12:34, Antonin Houska wrote: Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: I haven't thought that much about details, so just one comment: you sh

Re: Built-in connection pooling

2018-01-19 Thread Konstantin Knizhnik
m not sure that finally it will allow to reach better performance. More flexible solution in many cases doesn't mean more efficient solution. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: Built-in connection pooling

2018-01-19 Thread Konstantin Knizhnik
On 19.01.2018 18:53, Tomas Vondra wrote: On 01/19/2018 10:52 AM, Konstantin Knizhnik wrote: On 18.01.2018 18:02, Tomas Vondra wrote: Hi Konstantin, On 01/18/2018 03:48 PM, Konstantin Knizhnik wrote: On 17.01.2018 19:09, Konstantin Knizhnik wrote: Hi hackers, ... I haven't looked

Re: Built-in connection pooling

2018-01-19 Thread Konstantin Knizhnik
On 19.01.2018 20:01, Pavel Stehule wrote: 2018-01-19 17:53 GMT+01:00 Konstantin Knizhnik <k.knizh...@postgrespro.ru <mailto:k.knizh...@postgrespro.ru>>: On 19.01.2018 19:28, Pavel Stehule wrote: When I've been thinking about adding a built-in

Re: Built-in connection pooling

2018-01-19 Thread Konstantin Knizhnik
millions of TPS. It is illustrated by the results I have sent in the previous mail: by spawning 10 instances of pgbouncer I was able to receive 7 times bigger speed. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: Built-in connection pooling

2018-01-19 Thread Konstantin Knizhnik
On 19.01.2018 20:03, Claudio Freire wrote: On Fri, Jan 19, 2018 at 1:53 PM, Konstantin Knizhnik <k.knizh...@postgrespro.ru <mailto:k.knizh...@postgrespro.ru>> wrote: On 19.01.2018 19:28, Pavel Stehule wrote: When I've been thinking about ad

Re: JIT compiling with LLVM v9.0

2018-01-26 Thread Konstantin Knizhnik
erElim = 1;         LLVMCreateMCJITCompilerForModule(, mod, , sizeof(options),                                          );     }     ... But you are compiling code using LLVMOrcAddEagerlyCompiledIR and I find no way to pass no-omit-frame pointer option here. -- Konstantin Knizhnik Postgres Professional: http://www.postg

Re: JIT compiling with LLVM v9.0

2018-01-25 Thread Konstantin Knizhnik
with combining JIT and parallel execution? -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: AS OF queries

2018-01-25 Thread Konstantin Knizhnik
On 26.01.2018 03:55, Bruce Momjian wrote: On Sat, Dec 23, 2017 at 11:53:19PM +0300, konstantin knizhnik wrote: On Dec 23, 2017, at 2:08 AM, Greg Stark wrote: On 20 December 2017 at 12:45, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: It seems to me that it will be

Re: [HACKERS] Surjective functional indexes

2018-01-18 Thread Konstantin Knizhnik
On 18.01.2018 11:38, Simon Riggs wrote: On 10 January 2018 at 09:54, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: Sorry, issue with documentation is fixed. OK, thanks. Patch appears to work cleanly now. I'm wondering now about automatically inferring "recheck_on_up

Re: JIT compiling with LLVM v9.0

2018-01-25 Thread Konstantin Knizhnik
library name some signature of the compiled expression and allow to share this libraries between backends. So before starting code generation, ExecReadyCompiledExpr can first build signature and check if correspondent library is already present. Also it will be easier to control space used by comp

Re: JIT compiling with LLVM v9.0

2018-01-31 Thread Konstantin Knizhnik
ns of GCC: I have to upgrade GCC to 7.2 to make it possible to compile this code. The problem in not in compiler itself, but in libc++ headers. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: Built-in connection pooling

2018-02-01 Thread Konstantin Knizhnik
formance caused by such contention. But unfortunately it is not a silver bullet fixing all Postgres scalability problems. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: Built-in connection pooling

2018-02-02 Thread Konstantin Knizhnik
261.0 [UPDATE], 99thPercentileLatency(us), 87039.0 [UPDATE], Return=OK, 1515510 -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: Built-in connection pooling

2018-01-29 Thread Konstantin Knizhnik
nternal connection poolers: https://momjian.us/main/blogs/pgblog/2017.html#April_21_2017 I completely agree with your arguments in this post. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Secondary index access optimizations

2018-01-29 Thread Konstantin Knizhnik
On 29.01.2018 07:34, Thomas Munro wrote: On Sat, Jan 20, 2018 at 5:41 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: On 19.01.2018 16:14, Antonin Houska wrote: you should test the operator B-tree strategy: BTLessStrategyNumber, BTLessEqualStrategyNumber, etc. The operator

Re: JIT compiling with LLVM v9.0

2018-01-29 Thread Konstantin Knizhnik
%20JIT.pdf) speedup Q1 is 5.5x times. May be it is because them are using double type to calculate aggregates while as far as I understand you are using standard Postgres aggregate functions? Or may be because ISPRAS version is not checking for NULL values... -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: Built-in connection pooling

2018-02-01 Thread Konstantin Knizhnik
degradation of performance with increasing number of competing transactions. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Surjective functional indexes

2018-02-01 Thread Konstantin Knizhnik
On 01.02.2018 03:10, Simon Riggs wrote: On 10 January 2018 at 09:54, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: (new version attached) Why this comment? Current implementation of projection optimization has to calculate index expression twice in case of hit (value of

Re: Built-in connection pooling

2018-02-01 Thread Konstantin Knizhnik
and may be implement some contention aware scheduler as described here: http://www.vldb.org/pvldb/vol11/p648-tian.pdf But this is a different story, not related to built-in connection pooling. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Secondary index access optimizations

2018-01-29 Thread Konstantin Knizhnik
On 29.01.2018 16:24, Konstantin Knizhnik wrote: On 29.01.2018 07:34, Thomas Munro wrote: On Sat, Jan 20, 2018 at 5:41 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: On 19.01.2018 16:14, Antonin Houska wrote: you should test the operator B-tree strategy: BTLessStrategy

Using scalar function as set-returning: bug or feature?

2018-02-08 Thread Konstantin Knizhnik
(number of attributes is zero). May be there are some reasons for such behavior, but I find it quite confusing and unnatural. I prefer to report error in this case or return tuple with single column, containing return value of the function. -- Konstantin Knizhnik Postgres Professional: http

Re: Using scalar function as set-returning: bug or feature?

2018-02-09 Thread Konstantin Knizhnik
pected. But I wonder if the original statement should be considered as error or at least we should produce warning for such empty projections? -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: Built-in connection pooling

2018-02-09 Thread Konstantin Knizhnik
Attached please find new version of built-in connection pooling supporting temporary tables and session GUCs. Also Win32 support was added. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company diff --git a/src/backend/catalog/namespace.c b/src

Re: Using scalar function as set-returning: bug or feature?

2018-02-09 Thread Konstantin Knizhnik
On 09.02.2018 11:58, Konstantin Knizhnik wrote: On 09.02.2018 11:02, Konstantin Knizhnik wrote: On 09.02.2018 10:47, Sergei Kornilov wrote: Hello select into b from my_insert('from func atx'); You missed select something into b. For example, select ret into b from my_insert('from func

Re: Using scalar function as set-returning: bug or feature?

2018-02-09 Thread Konstantin Knizhnik
On 09.02.2018 11:02, Konstantin Knizhnik wrote: On 09.02.2018 10:47, Sergei Kornilov wrote: Hello select into b from my_insert('from func atx'); You missed select something into b. For example, select ret into b from my_insert('from func atx') as ret; Using scalar function in from

Re: Cached/global query plans, autopreparation

2018-02-15 Thread Konstantin Knizhnik
se engine. Language binding can compare costs of generic and specialized plans, shared prepared statements between all database connections,... But I think that it is more natural and efficient to implement this logic in one place, rather than try to reimplement it several times for different

  1   2   3   4   5   6   7   >