Re: [HACKERS] Sum aggregate calculation for single precsion real

2017-02-13 Thread Konstantin Knizhnik
On 13.02.2017 19:20, Tom Lane wrote: Konstantin Knizhnik <k.knizh...@postgrespro.ru> writes: I wonder why SUM aggregate is calculated for real (float4) type using floating point accumulator? If you can't deal with the vagaries of floating-point arithmetic, you shouldn't be storing you

Re: [HACKERS] [WIP]Vertical Clustered Index (columnar store extension)

2017-02-13 Thread Konstantin Knizhnik
hich can demonstrate advantages of using VCI index? On 06.02.2017 04:26, Haribabu Kommi wrote: On Fri, Feb 3, 2017 at 8:28 PM, Konstantin Knizhnik <k.knizh...@postgrespro.ru <mailto:k.knizh...@postgrespro.ru>> wrote: On 30.12.2016 06:55, Haribabu Kommi wrote: Hi All, Fu

[HACKERS] VOPS: vectorized executor for Postgres: how to speedup OLAP queries more than 10 times without changing anything in Postgres executor

2017-02-13 Thread Konstantin Knizhnik
found here: https://github.com/postgrespro/vops.git -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mail

[HACKERS] Sum aggregate calculation for single precsion real

2017-02-13 Thread Konstantin Knizhnik
natual) way. Are there are reasons of using float4pl function for SUM aggregate instead of float4_accum? Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes t

Re: [HACKERS] Active zombies at AIX

2017-02-06 Thread Konstantin Knizhnik
#define USE_ANONYMOUS_SHMEM #endif I wonder why do we prohibit now configuration of Postgres without mmap? On 06.02.2017 12:47, Konstantin Knizhnik wrote: Last update on the problem. Using kdb tool (thank's to Tony Reix for advice and help) we get the following trace of Poastgres backend

Re: [HACKERS] Active zombies at AIX

2017-02-06 Thread Konstantin Knizhnik
settings for lgpg_size, lgpg_regions and v_pinshm already seem correct. On 24.01.2017 18:08, Konstantin Knizhnik wrote: Hi hackers, Yet another story about AIX. For some reasons AIX very slowly cleaning zombie processes. If we launch pgbench with -C parameter then very soon limit for maximal

Re: [HACKERS] logical decoding of two-phase transactions

2017-02-03 Thread Konstantin Knizhnik
de slower than without it. So autonomous transaction can be used for audits (its the primary goal of using ATX in Oracle PL/SQL applications) but this mechanism is not efficient for concurrent execution of multiple transaction in one backend. -- Konstantin Knizhnik Postgres Professional: http

Re: [HACKERS] [WIP]Vertical Clustered Index (columnar store extension)

2017-02-03 Thread Konstantin Knizhnik
he patches to the latest master, and share them with community. Any Comments on the approach? Regards, Hari Babu Fujitsu Australia -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Deadlock in XLogInsert at AIX

2017-02-02 Thread Konstantin Knizhnik
Eyecatcher/version mismatch in RWA -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Deadlock in XLogInsert at AIX

2017-02-02 Thread Konstantin Knizhnik
0 Feb 2 09:09 826_vm -rw---1 pgstbf staff 8192 Feb 2 09:09 827 -rw---1 pgstbf staff 8192 Feb 2 09:09 828 -rw---1 pgstbf staff 4 Feb 2 09:09 PG_VERSION -rw---1 pgstbf staff 512 Feb 2 09:09 pg_filenode.map -rw---

Re: [HACKERS] Deadlock in XLogInsert at AIX

2017-02-02 Thread Konstantin Knizhnik
d to debug it. Heikki, I will be pleased if you have a chance to login at the system and look at it yourself. May be you will have some idea what's happening... Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgs

Re: [HACKERS] Deadlock in XLogInsert at AIX

2017-02-01 Thread Konstantin Knizhnik
On 02/01/2017 08:28 PM, Heikki Linnakangas wrote: > > But if there's no pressing reason to change it, let's leave it alone. It's > not related to the problem at hand, right? > Yes, I agree with you: we should better leave it as it is. -- Konstantin Knizhnik Postgres Profes

Re: [HACKERS] Deadlock in XLogInsert at AIX

2017-02-01 Thread Konstantin Knizhnik
ver, that should be > invisible with the "check-world" tests I guess. > > Regards, > > Tony > > > Le 01/02/2017 à 16:59, Konstantin Knizhnik a écrit : >> Hi Tony, >> >> On 01.02.2017 18:42, REIX, Tony wrote: >>> >>> Hi Konstanti

Re: [HACKERS] logical decoding of two-phase transactions

2017-02-01 Thread Konstantin Knizhnik
wing for a kind of * distributed 2PC. */ For some scenarios it works well, but if we really need prepared state at replica (as in case of multimaster), then it is not enough. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- S

Re: [HACKERS] Deadlock in XLogInsert at AIX

2017-02-01 Thread Konstantin Knizhnik
of your patch. I verified that it is correctly applied, build and postgres normally works with it. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company diff --git a/src/include/port/atomics/arch-ppc.h b/src/include/port/atomics/arch-ppc.h index

Re: [HACKERS] Deadlock in XLogInsert at AIX

2017-02-01 Thread Konstantin Knizhnik
on AIX ? We still have one open issue at AIX: see https://www.mail-archive.com/pgsql-hackers@postgresql.org/msg303094.html It will be great if you can somehow help to fix this problem. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Deadlock in XLogInsert at AIX

2017-02-01 Thread Konstantin Knizhnik
--with-libxml --with-libxslt --enable-nls --enable-thread-safety --sysconfdir=/etc/sysconfig/postgresql Am I missing some option for more optimization on AIX ? Thanks Regards, Tony Le 01/02/2017 à 12:07, Konstantin Knizhnik a écrit : Attached please f

Re: [HACKERS] Deadlock in XLogInsert at AIX

2017-02-01 Thread Konstantin Knizhnik
On 01.02.2017 15:39, Heikki Linnakangas wrote: On 02/01/2017 01:07 PM, Konstantin Knizhnik wrote: Attached please find my patch for XLC/AIX. The most critical fix is adding __sync to pg_atomic_fetch_add_u32_impl. The comment in this file says that: * __fetch_and_add() emits a leading

Re: [HACKERS] Deadlock in XLogInsert at AIX

2017-02-01 Thread Konstantin Knizhnik
re_exchange_u32_impl. I have fixed it by using __sync() built-in function instead. Thanks to everybody who helped me to locate and fix this problem. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company diff --git a/src/include/port/atomics/arch

Re: [HACKERS] Deadlock in XLogInsert at AIX

2017-01-31 Thread Konstantin Knizhnik
0,r6,0x stwr0,0(r3) mfcr r0 rlwinm r0,r0,3,31,31 rldicl r0,r0,0,56 stbr0,112(SP) isync lbzr3,112(SP) addi SP,SP,128 bclr BO_ALWAYS,CR0_LT sync is here! -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Deadlock in XLogInsert at AIX

2017-01-31 Thread Konstantin Knizhnik
On 30.01.2017 19:21, Heikki Linnakangas wrote: On 01/24/2017 04:47 PM, Konstantin Knizhnik wrote: Interesting.. What should happen here is that for the backend's own insertion slot, the "insertingat" value should be greater than the requested flush point ('upto' variable). That

Re: [HACKERS] logical decoding of two-phase transactions

2017-01-31 Thread Konstantin Knizhnik
PRECOMMIT+COMMIT_PREPARED either just COMMIT_PREPARED. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailp

Re: [HACKERS] Deadlock in XLogInsert at AIX

2017-01-24 Thread Konstantin Knizhnik
is different for GCC and XLC. But ... if I rebuild Postgres without spinlocks, then the problem is still reproduced. On 24.01.2017 17:47, Konstantin Knizhnik wrote: Hi Hackers, We are running Postgres at AIX and encoountered two strqange problems: active zombies process and deadlock in XLOG

Re: [HACKERS] Active zombies at AIX

2017-01-24 Thread Konstantin Knizhnik
On 24.01.2017 18:26, Tom Lane wrote: Konstantin Knizhnik <k.knizh...@postgrespro.ru> writes: But ps shows that status of process is [14:46:02]root@postgres:~ # ps -elk | grep 25691556 * A - 25691556 - - - - - As far as I could find by googling, this means that the p

[HACKERS] Active zombies at AIX

2017-01-24 Thread Konstantin Knizhnik
to defunct process, it is not clear how to understand what's going on. I wonder if somebody has encountered similar problems at AIX and may be can suggest some solution to solve this problem. Thanks in advance -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian

[HACKERS] Deadlock in XLogInsert at AIX

2017-01-24 Thread Konstantin Knizhnik
ust call this function * *before* acquiring WALWriteLock, to avoid deadlocks. This function might * need to wait for an insertion to finish (or at least advance to next * uninitialized page), and the inserter might need to evict an old WAL buffer * to make room for a new one, which in turn requires WALWriteLock. Which contradicts to the observed stack trace. I wonder if it is really synchronization bug in xlog.c or there is something wrong in this stack trace and it can not happen in case of normal work? Thanks in advance, -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Unlogged tables cleanup

2016-11-09 Thread konstantin knizhnik
database and tablespace to reproduce this issue. So actually the whole sequence is mkdir fs initdb -D pgsql pg_ctl -D pgsql -l logfile start psql postgres # create tablespace fs location '/home/knizhnik/dtm-data/fs'; # set default_tablespace=fs; # create unlogged table foo(x integer); # insert into f

[HACKERS] Unlogged tables cleanup

2016-11-09 Thread Konstantin Knizhnik
Hi, hackers I wonder if such behavior can be considered as a bug: knizhnik@knizhnik:~/dtm-data$ psql postgres psql (10devel) Type "help" for help. postgres=# create tablespace fs location '/home/knizhnik/dtm-data/fs'; CREATE TABLESPACE postgres=# set default_tablespace=fs; SE

Re: [HACKERS] On conflict update & hint bits

2016-10-24 Thread Konstantin Knizhnik
LockBuffer(buffer, BUFFER_LOCK_UNLOCK); + /* * Make tuple and any needed join variables available to ExecQual and * ExecProject. The EXCLUDED tuple is installed in ecxt_innertuple, while -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Post

Re: [HACKERS] On conflict update & hint bits

2016-09-30 Thread Konstantin Knizhnik
On 30.09.2016 19:37, Peter Geoghegan wrote: On Fri, Sep 30, 2016 at 5:33 PM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: Later we try to check tuple visibility: ExecCheckHeapTupleVisible(estate, , buffer); and inside HeapTupleSatisfiesMVCC try to set hint bit. So,

[HACKERS] On conflict update & hint bits

2016-09-30 Thread Konstantin Knizhnik
re or exclusive lock is OK */ Assert(LWLockHeldByMe(BufferDescriptorGetContentLock(bufHdr))); So the question is whether it is correct that ExecOnConflictUpdate tries to access and update tuple without holding lock on the buffer? Thank in advance, -- Konstantin Knizhnik Postgres Professional

[HACKERS] Handling dropped attributes in pglogical_proto

2016-09-28 Thread Konstantin Knizhnik
t->atttypid == InvalidOid) { + continue; + } + kind = pq_getmsgbyte(in); + switch (kind) { case 'n': /* null */ -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com

Re: [HACKERS] UPSERT strange behavior

2016-08-25 Thread Konstantin Knizhnik
what is the best workaround for the problem if we really need to have to separate indexes and want to enforce unique constraint for both keys? -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hacker

Re: [HACKERS] Logical decoding restart problems

2016-08-20 Thread konstantin knizhnik
Thank you for answers. > No, you don't need to recreate them. Just advance your replication identifier > downstream and request a replay position in the future. Let the existing slot > skip over unwanted data and resume where you want to start replay. > > You can advance the replication

Re: [HACKERS] Logical decoding restart problems

2016-08-19 Thread Konstantin Knizhnik
started investigation of logical decoding code and found several things which I do not understand. Never seen this happen. Do you have more details about what exactly is happening? This is transaction at primary node: root@knizhnik:/home/knizhnik/postgres_cluster/contrib/mmts# docker exec -ti

[HACKERS] Logical decoding restart problems

2016-08-19 Thread konstantin knizhnik
ism in xlog which can enforce consistent decoding of transaction (so that no transaction records are missed)? May be I missed something but I didn't find any "record_number" or something else which can identify first record of transaction. Thanks in advance, Konstantin Knizhnik, Postgres P

Re: [HACKERS] WIP: Barriers

2016-08-15 Thread Konstantin Knizhnik
ATABASE_CONNECTION, BgWorkerStart_ConsistentState, BGW_NEVER_RESTART, PongLatch }; static void PingPong() { RegisterBackgroundWorker(); RegisterBackgroundWorker(); } -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- S

Re: [HACKERS] WIP: Barriers

2016-08-14 Thread konstantin knizhnik
Hi Thomas, Barriers are really very simple and convenient mechanism for process synchronization. But it is actually a special case of semaphores: having semaphore primitive it is trivial to implement a barrier. We have semaphores in Postgres, but ... them can not be used by extensions: there

Re: [HACKERS] Restriction of windows functions

2016-06-17 Thread Konstantin Knizhnik
t possible just to locate "-" or "+ operator for this type? I do not see any difference here with locating comparison operator needed for sorting. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers ma

[HACKERS] Restriction of windows functions

2016-06-17 Thread Konstantin Knizhnik
nt row is less or equal than specified range value. Looks like not something too complex to implement, doesn't it? Are there some caveats? Certainly it assumes that window is ordered by key and the key type supports subtraction, so "text" can not be used here. Something else? -- Konstantin Kn

Re: [HACKERS] array of domain types

2016-06-03 Thread Konstantin Knizhnik
On 03.06.2016 02:02, Rod Taylor wrote: On Thu, Jun 2, 2016 at 10:42 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru <mailto:k.knizh...@postgrespro.ru>> wrote: On 02.06.2016 17:22, Tom Lane wrote: konstantin knizhnik <k.knizh...@postgrespro.ru

Re: [HACKERS] XTM & parallel search

2016-06-03 Thread Konstantin Knizhnik
On 03.06.2016 16:05, Amit Kapila wrote: On Fri, Jun 3, 2016 at 1:34 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru <mailto:k.knizh...@postgrespro.ru>> wrote: We have to add three more functions to eXtensible Transaction Manager API (XTM): /*

[HACKERS] XTM & parallel search

2016-06-02 Thread Konstantin Knizhnik
ext. I wonder if not only DTM will be interested in sharing some common state between workers and should we provide some way of replicating user defined context between workers? From my point of view XTM seems to be good place for it... -- Konstantin Knizhnik Postgres Professional

Re: [HACKERS] array of domain types

2016-06-02 Thread Konstantin Knizhnik
On 02.06.2016 17:22, Tom Lane wrote: konstantin knizhnik <k.knizh...@postgrespro.ru> writes: Attached please find patch for DefineDomain function. You didn't attach the patch, Sorry, but I did attached the patch - I see the attachment in my mail received from the group. Multidimen

Re: [HACKERS] array of domain types

2016-06-02 Thread konstantin knizhnik
On Jun 2, 2016, at 12:29 PM, Thom Brown wrote:On 2 June 2016 at 10:13, konstantin knizhnik <k.knizh...@postgrespro.ru> wrote:Yes, it doesn't work:# CREATE DOMAIN teenager AS int CHECK (VALUE BETWEEN 13 AND 19);CREATE DOMAIN# SELECT 14::teenager; teenager --   14(1 row)# SEL

Re: [HACKERS] array of domain types

2016-06-02 Thread konstantin knizhnik
On Jun 1, 2016, at 4:37 PM, Thom Brown wrote: > On 1 June 2016 at 14:20, Konstantin Knizhnik <k.knizh...@postgrespro.ru> > wrote: > I wonder why domain types can not be used for specification of array element: > > create domain objref as bigint; > create table foo(x o

[HACKERS] array of domain types

2016-06-01 Thread Konstantin Knizhnik
mplemented? -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Logical replication & oldest XID.

2016-05-31 Thread Konstantin Knizhnik
quiredXmin), I want to understand if it is real problem of logical replication or we are doing something wrong? BDR should be faced with the same problem if all updates are performed from one node... -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Com

Re: [HACKERS] Inheritance

2016-05-24 Thread Konstantin Knizhnik
f this. Feature-wise, we need to be doing *more* of this kind of work, not less. Lack of support for OO paradigms was one of the drivers for NoSQL. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Just-in-time compiling things

2016-05-14 Thread Konstantin Knizhnik
On 05/14/2016 12:10 PM, Andreas Seltenreich wrote: Konstantin Knizhnik writes: Latest information from ISP RAS guys: them have made good progress since February: them have rewritten most of methods of Scan, Aggregate and Join to LLVM API. Is their work available somewhere? I'm experimenting

Re: [HACKERS] Academic help for Postgres

2016-05-12 Thread konstantin knizhnik
On May 12, 2016, at 6:16 AM, Rajeev rastogi wrote: > On 11 May 2016 19:50, Bruce Momjian Wrote: > > >> I am giving a keynote at an IEEE database conference in Helsinki next >> week (http://icde2016.fi/). (Yes, I am not attending PGCon Ottawa >> because I accepted the Helsinki conference

Re: [HACKERS] Academic help for Postgres

2016-05-11 Thread Konstantin Knizhnik
On 11.05.2016 17:32, Bruce Momjian wrote: On Wed, May 11, 2016 at 05:31:10PM +0300, Konstantin Knizhnik wrote: On 11.05.2016 17:20, Bruce Momjian wrote: I am giving a keynote at an IEEE database conference in Helsinki next week (http://icde2016.fi/). (Yes, I am not attending PGCon Ottawa

Re: [HACKERS] Academic help for Postgres

2016-05-11 Thread Konstantin Knizhnik
Sharding Any others? Incremental materialized views? -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] asynchronous and vectorized execution

2016-05-11 Thread Konstantin Knizhnik
made good progress since February: them have rewritten most of methods of Scan, Aggregate and Join to LLVM API. Also then implemented automatic translation of PostgreSQL backend functions to LLVM API. As a result time of TPC-H Q1 query is reduced four times. -- Konstantin Knizhnik Postgres Pr

Re: [HACKERS] asynchronous and vectorized execution

2016-05-11 Thread Konstantin Knizhnik
On 11.05.2016 17:00, Robert Haas wrote: On Tue, May 10, 2016 at 3:42 PM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: Doesn't this actually mean that we need to have normal job scheduler which is given queue of jobs and having some pool of threads will be able to orginize eff

Re: [HACKERS] asynchronous and vectorized execution

2016-05-10 Thread Konstantin Knizhnik
On 05/10/2016 08:26 PM, Robert Haas wrote: On Tue, May 10, 2016 at 3:00 AM, konstantin knizhnik <k.knizh...@postgrespro.ru> wrote: What's wrong with it that worker is blocked? You can just have more workers (more than CPU cores) to let other of them continue to do useful work. Not

Re: [HACKERS] asynchronous and vectorized execution

2016-05-10 Thread konstantin knizhnik
Hi, > 1. asynchronous execution, It seems to me that asynchronous execution can be considered as alternative to multithreading model (in case of PostgreSQL the roles of threads are played by workers). Async. operations are used to have smaller overhead but have scalability problems (because

Re: [HACKERS] Batch update of indexes

2016-04-02 Thread Konstantin Knizhnik
On 04/02/2016 09:57 PM, Tom Lane wrote: Konstantin Knizhnik <k.knizh...@postgrespro.ru> writes: Attached please find patch for "ALTER INDEX ... WHERE ..." clause. It is now able to handle all three possible situations: 1. Making index partial (add WHERE condition to the o

[HACKERS] Very small patch for decode.c

2016-03-30 Thread Konstantin Knizhnik
i; -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] avg,first,last,median in one query

2016-03-24 Thread Konstantin Knizhnik
g user-defined aggregate function which uses array_appendand so materialize all values in memory: https://wiki.postgresql.org/wiki/Aggregate_Median 3. Using percentile aggregate: http://blog.jooq.org/2015/01/06/how-to-emulate-the-median-aggregate-function-using-inverse-distribution-functions/ Th

Re: [HACKERS] Applying logical replication changes by more than one process

2016-03-22 Thread konstantin knizhnik
On Mar 22, 2016, at 11:14 AM, Petr Jelinek wrote: > > And each slot means connection with logical decoding attached to it so you > don't really want to have thousands of those anyway. I think you'll hit other > problems faster than loop over slots becomes problem if you plan to keep all > of

Re: [HACKERS] Applying logical replication changes by more than one process

2016-03-22 Thread konstantin knizhnik
On Mar 22, 2016, at 10:10 AM, Craig Ringer wrote: > On 22 March 2016 at 14:32, konstantin knizhnik <k.knizh...@postgrespro.ru> > wrote: > >> Ah you mean because with wal_log=true the origin advance is in different WAL >> record than commit? OK yeah you migh

Re: [HACKERS] Applying logical replication changes by more than one process

2016-03-22 Thread konstantin knizhnik
On Mar 21, 2016, at 4:30 PM, Petr Jelinek wrote: > On 21/03/16 14:25, Andres Freund wrote: >> On 2016-03-21 14:18:27 +0100, Petr Jelinek wrote: >>> On 21/03/16 14:15, Andres Freund wrote: > Only when the origin is actually setup for the current session. You > need > to call the

Re: [HACKERS] Applying logical replication changes by more than one process

2016-03-21 Thread Konstantin Knizhnik
On 21.03.2016 15:10, Petr Jelinek wrote: Hi, On 19/03/16 11:46, Konstantin Knizhnik wrote: Hi, I am trying to use logical replication mechanism in implementation of PostgreSQL multimaster and faced with one conceptual problem. Originally logical replication was intended to support

[HACKERS] Applying logical replication changes by more than one process

2016-03-19 Thread Konstantin Knizhnik
done using current logical replication mechanism when changes of each slot are applied by more than one process? Or the only alternative is to write/read origin LSNs in WAL myself, for example using custom WAL records? Thanks in advance! -- Konstantin Knizhnik Postgres Professional: http://

Re: [HACKERS] Batch update of indexes

2016-03-14 Thread Konstantin Knizhnik
Hi David, Rebased patch is attached. On 14.03.2016 15:09, David Steele wrote: Hi Konstantin, On 2/3/16 11:47 AM, Konstantin Knizhnik wrote: Attached please find patch for "ALTER INDEX ... WHERE ..." clause. It is now able to handle all three possible situations: 1. Making index pa

Re: [HACKERS] eXtensible Transaction Manager API (v2)

2016-03-11 Thread Konstantin Knizhnik
g this 'fest. I think we'd be well advised to boot it to the 2016-09 CF and focus our efforts on other stuff that has a better chance of getting finished this month. regards, tom lane -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian P

Re: [HACKERS] Optimizer questions

2016-03-10 Thread konstantin knizhnik
On Mar 10, 2016, at 1:56 AM, Tom Lane wrote: > Konstantin Knizhnik <k.knizh...@postgrespro.ru> writes: >> I think that the best approach is to generate two different paths: >> original one, when projection is always done before sort and another one >> with postpone

Re: [HACKERS] Optimizer questions

2016-03-09 Thread Konstantin Knizhnik
f the path instead of comparing costs of full paths. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mail

Re: [HACKERS] Optimizer questions

2016-03-08 Thread Konstantin Knizhnik
On 03/08/2016 07:01 AM, Tom Lane wrote: Konstantin Knizhnik <k.knizh...@postgrespro.ru> writes: Attached please find improved version of the optimizer patch for LIMIT clause. This patch isn't anywhere close to working after 3fc6e2d7f5b652b4. (TBH, the reason I was negative about this up

Re: [HACKERS] The plan for FDW-based sharding

2016-03-07 Thread Konstantin Knizhnik
is area to be part of core. None of that means I would support any particular hook proposal, of course. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] The plan for FDW-based sharding

2016-03-02 Thread Konstantin Knizhnik
and simple OLAP. For OLTP we definitely need transaction manager to provide global consistency. And we have actually prototype of integration postgres_fdw with out pg_dtm and pg_tsdtm transaction managers. The results are quite IMHO promising (see attached diagram). -- Konstantin Knizhnik Postgres

Re: [HACKERS] PROPOSAL: Fast temporary tables

2016-03-01 Thread Konstantin Knizhnik
. Actually translating that into relcache and everything else would be a serious amount of work. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] The plan for FDW-based sharding

2016-03-01 Thread Konstantin Knizhnik
es mapping between local XIDs and global CSNs. Visibility checking rules looks on CSNs, not on XIDs. In both cases if system is for some reasons restarted and DTM plugin failed to be loaded, you can still access database locally. No data can be lost. -- Konstantin Knizhnik Postgres Professi

Re: [HACKERS] The plan for FDW-based sharding

2016-03-01 Thread Konstantin Knizhnik
uspect that a transaction manager API would end up similarly situated. IMHO non-stable API is better than lack of API. Just because it makes it possible to implement features in modular way. And refactoring of API is not so difficult thing... -- Konstantin Knizhnik Postgres Professional: http://www.postgresp

Re: [HACKERS] The plan for FDW-based sharding

2016-03-01 Thread Konstantin Knizhnik
Thank you very much for you comments. On 01.03.2016 18:19, Robert Haas wrote: On Sat, Feb 27, 2016 at 2:29 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: How do you prevent clock skew from causing serialization anomalies? If node receives message from "feature&quo

Re: [HACKERS] The plan for FDW-based sharding

2016-02-28 Thread Konstantin Knizhnik
have not (yet) reviewed the literature for it. The reference to the article is at our WiKi pages explaining our DTM: https://wiki.postgresql.org/wiki/DTM http://research.microsoft.com/en-us/people/samehe/clocksi.srds2013.pdf -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com

Re: [HACKERS] The plan for FDW-based sharding

2016-02-27 Thread Konstantin Knizhnik
visibility check. On 02/27/2016 01:48 AM, Kevin Grittner wrote: On Fri, Feb 26, 2016 at 2:19 PM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: pg_tsdtm is based on another approach: it is using system time as CSN Which brings up an interesting point, if we want logical replic

Re: [HACKERS] The plan for FDW-based sharding

2016-02-27 Thread Konstantin Knizhnik
On 02/27/2016 06:54 AM, Robert Haas wrote: On Fri, Feb 26, 2016 at 10:56 PM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: We do not have formal prove that proposed XTM is "general enough" to handle all possible transaction manager implementations. But there are

Re: [HACKERS] Relation cache invalidation on replica

2016-02-26 Thread Konstantin Knizhnik
016-02-26 18:05:55 +0300, Konstantin Knizhnik wrote: > The reason of the problem is that invalidation messages are not delivered to > replica after the end of concurrent create index. > Invalidation messages are included in xlog as part of transaction commit >

Re: [HACKERS] The plan for FDW-based sharding

2016-02-26 Thread Konstantin Knizhnik
On 02/27/2016 06:57 AM, Robert Haas wrote: On Sat, Feb 27, 2016 at 1:49 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: pg_tsdtm is based on another approach: it is using system time as CSN and doesn't require arbiter. In theory there is no limit for scalability. But diffe

Re: [HACKERS] The plan for FDW-based sharding

2016-02-26 Thread Konstantin Knizhnik
On 02/26/2016 09:30 PM, Alvaro Herrera wrote: Konstantin Knizhnik wrote: Yes, it is certainly possible to develop cluster by cloning PostgreSQL. But it cause big problems both for developers, which have to permanently synchronize their branch with master, and, what is more important

Re: [HACKERS] The plan for FDW-based sharding

2016-02-26 Thread Konstantin Knizhnik
people", who has different ideas on postgres future. That's why we propose this patch, let's play the game ! I don't like to play games with the architecture of PostgreSQL. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via

Re: [HACKERS] Relation cache invalidation on replica

2016-02-26 Thread Konstantin Knizhnik
ev Postgres Professional: http://www.postgrespro.com Russian Postgres Company -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] The plan for FDW-based sharding

2016-02-24 Thread Konstantin Knizhnik
ostgres XC/XL will probably remain a separate fork of Postgres. I don't think anyone knows the answer to this question, and I don't know how to find the answer except to keep going with our current FDW sharding approach. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com T

Re: [HACKERS] pglogical - logical replication contrib module

2016-02-17 Thread Konstantin Knizhnik
. On 17.02.2016 12:16, Craig Ringer wrote: On 17 February 2016 at 16:24, Konstantin Knizhnik <k.knizh...@postgrespro.ru <mailto:k.knizh...@postgrespro.ru>> wrote: Thanks for your explanation. I have to agree with your arguments that in general case replication of DDL statement u

Re: [HACKERS] pglogical - logical replication contrib module

2016-02-17 Thread Konstantin Knizhnik
astics done by ALTER TABLE, etc. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Clock with Adaptive Replacement

2016-02-12 Thread Konstantin Knizhnik
address this problem because there are not counters - justs single bit per page. On 12.02.2016 18:55, Robert Haas wrote: On Thu, Feb 11, 2016 at 4:02 PM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: What do you think about improving cache replacement clock-sweep alg

[HACKERS] Clock with Adaptive Replacement

2016-02-11 Thread Konstantin Knizhnik
ARC (CAR is inspired by ARC, but it is different algorithm). As far as I know there are several problems with current clock-sweep algorithm in PostgreSQL, especially for very large caches. May be CAR can address some of them? -- Konstantin Knizhnik Postgres Professional: http

[HACKERS] eXtensible Transaction Manager API (v2)

2016-02-10 Thread Konstantin Knizhnik
its usage. The complete PostgreSQL branch with all our changes can be found here: https://github.com/postgrespro/postgres_cluster -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company diff --git a/contrib/pg_tsdtm/Makefile b/contrib/pg_tsdtm/Makefil

Re: [HACKERS] Batch update of indexes

2016-02-03 Thread Konstantin Knizhnik
h together with "index-only scans with partial indexes" patch: http://www.postgresql.org/message-id/560c7213.3010...@2ndquadrant.com only in this case regression test will produce expected output. On 27.01.2016 23:15, Robert Haas wrote: On Wed, Jan 20, 2016 at 4:28 AM, Konstant

Re: [HACKERS] Batch update of indexes

2016-02-03 Thread konstantin knizhnik
On Feb 4, 2016, at 2:00 AM, Jim Nasby wrote: > > My suspicion is that it would be useful to pre-order the new data before > trying to apply it to the indexes. Sorry, but ALTER INDEX is expected to work for all indexes, not only B-Tree, and for them sorting may not be possible... But for

Re: [HACKERS] PATCH: index-only scans with partial indexes

2016-02-02 Thread Konstantin Knizhnik
.2016 01:11, Alvaro Herrera wrote: Konstantin Knizhnik wrote: I am very interested in this patch because it allows to use partial indexes to ... speed up inserts. I have implemented "ALTER INDEX ... WHERE ..." construction which allows to change predicate of partial index without necessity

Re: [HACKERS] PATCH: index-only scans with partial indexes

2016-01-31 Thread Konstantin Knizhnik
re if either Kyotaro or Tomas should be considered the patch author ... maybe both?) -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to you

Re: [HACKERS] Optimizer questions

2016-01-30 Thread Konstantin Knizhnik
34 rows=1 width=8) (actual time=0.007..0.007 rows=1 loops=10) Index Cond: (t1.k = k) Planning time: 0.537 ms Execution time: 0.241 ms (7 rows) On 01/30/2016 01:01 AM, Alexander Korotkov wrote: On Fri, Jan 8, 2016 at 11:58 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru <

Re: [HACKERS] Batch update of indexes

2016-01-26 Thread Konstantin Knizhnik
ed to be interesting by community, I will try to address these issues. On 20.01.2016 12:28, Konstantin Knizhnik wrote: Hi hackers, I want to know opinion of community about possible ways of solving quite common problem: increasing insert speed while still providing indexes for efficient executi

Re: [HACKERS] Batch update of indexes

2016-01-21 Thread Konstantin Knizhnik
On 21.01.2016 10:14, Simon Riggs wrote: On 21 January 2016 at 06:41, konstantin knizhnik <k.knizh...@postgrespro.ru <mailto:k.knizh...@postgrespro.ru>> wrote: Certainly for B-Tree we can organize insert buffer (or pending list) as sorted array or also as a tree. But

Re: [HACKERS] Batch update of indexes

2016-01-21 Thread Konstantin Knizhnik
proper recovery of main index in case of failure (assuming that pending list is maintained in memory and is lost after the fault). -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresq

Re: [HACKERS] Batch update of indexes

2016-01-20 Thread konstantin knizhnik
On Jan 21, 2016, at 5:14 AM, Simon Riggs wrote: > On 20 January 2016 at 14:55, Konstantin Knizhnik <k.knizh...@postgrespro.ru> > wrote: > Hi, > > Hi, I glad to see that you interested in that too. > I think this is a good feature and I think it will be very useful t

Re: [HACKERS] Batch update of indexes

2016-01-20 Thread Konstantin Knizhnik
. So small patch... Why it was not accepted? I do no see any problems with it... -- Anastasia Lubennikova Postgres Professional:http://www.postgrespro.com The Russian Postgres Company -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

<    1   2   3   >