Re: [HACKERS] eXtensible Transaction Manager API

On 11/08/2015 02:46 PM, Michael Paquier wrote: On Sun, Nov 8, 2015 at 1:53 AM, Konstantin Knizhnik wrote: In tsDTM approach two phase commit is performed by coordinator and currently is using standard PostgreSQL two phase commit: Code in GO performing two phase commit: exec(conn1

Re: [HACKERS] eXtensible Transaction Manager API

I am sorry, looks like I had incorrectly interpret Michael's comment: Not always. If COMMIT PREPARED fails at some of the nodes but succeeds on others, the transaction is already partially acknowledged as committed in the cluster. Hence it makes more sense for the coordinator to commit

Re: [HACKERS] Question concerning XTM (eXtensible Transaction Manager API)

On 11/16/2015 10:54 PM, Alvaro Herrera wrote: Konstantin Knizhnik wrote: But you may notice that original TransactionIdSetTreeStatus function is void - it is not intended to return anything. It is called in RecordTransactionCommit in critical section where it is not expected that commit may

[HACKERS] Question concerning XTM (eXtensible Transaction Manager API)

Hello, Some time ago at PgConn.Vienna we have proposed eXtensible Transaction Manager API (XTM). The idea is to be able to provide custom implementation of transaction managers as standard Postgres extensions, primary goal is implementation of distritibuted transaction manager. It should not

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On 06.11.2015 21:37, Robert Haas wrote: On Wed, Aug 12, 2015 at 6:25 AM, Ashutosh Bapat wrote: The previous patch would not compile on the latest HEAD. Here's updated patch. Perhaps unsurprisingly, this doesn't apply any more. But we have bigger things to

Re: [HACKERS] eXtensible Transaction Manager API

On 09.11.2015 07:46, Amit Kapila wrote: On Sat, Nov 7, 2015 at 10:23 PM, Konstantin Knizhnik <k.knizh...@postgrespro.ru <mailto:k.knizh...@postgrespro.ru>> wrote: Lock manager is one of the tasks we are currently working on. There are still a lot of open questions:

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On 09.11.2015 09:59, Ashutosh Bapat wrote: Since the foreign server (referred to in the slides as secondary server) requires to call "create extension pg_dtm" and select dtm_join_transaction(xid);, I assume that the foreign server has to be a PostgreSQL server and one which has this

[HACKERS] eXtensible Transaction Manager API

clarify problems with recovery and performance with this approach. Also we have patch for pg_shard to work with DTM, which also will be published soon. Waiting for your feedback Konstantin Knizhnik, Postgres Professional -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] eXtensible Transaction Manager API

Hi, Thank you for your feedback. My comments are inside. On 11/07/2015 05:11 PM, Amit Kapila wrote: Today, while studying your proposal and related material, I noticed that in both the approaches DTM and tsDTM, you are talking about committing a transaction and acquiring the snapshot

Re: [HACKERS] eXtensible Transaction Manager API

On 02.11.2015 12:01, Simon Riggs wrote: At first I was concerned about recovery, but that looks to be covered. Yes, we have not considered all possible scenarios of working with PostgreSQL. We have tested work with different isolation levels: repeatable read and read committed, but we

Re: [HACKERS] eXtensible Transaction Manager API

On 02.11.2015 06:17, Craig Ringer wrote: On 31 October 2015 at 17:22, konstantin knizhnik <k.knizh...@postgrespro.ru> wrote: Waiting for your feedback For anyone wondering about performance impact, there are some graphs on page 23 of the PDF presentation. I didn't see anythin

Re: [HACKERS] Question concerning XTM (eXtensible Transaction Manager API)

Thank you for your response. On Nov 16, 2015, at 11:21 PM, Kevin Grittner wrote: > I'm not entirely clear on what you're saying here. I admit I've > not kept in close touch with the distributed processing discussions > lately -- is there a write-up and/or diagram to give an overview of > where

Re: [HACKERS] Question concerning XTM (eXtensible Transaction Manager API)

On Nov 17, 2015, at 10:44 AM, Amit Kapila wrote: > > I think the general idea is that if Commit is WAL logged, then the > operation is considered to committed on local node and commit should > happen on any node, only once prepare from all nodes is successful. > And after that transaction is

Re: [HACKERS] Logical replication and multimaster

Thank you for reply. On 12/02/2015 08:30 PM, Robert Haas wrote: Logical decoding only begins decoding a transaction once the transaction is complete. So I would guess that the sequence of operations here is something like this - correct me if I'm wrong: 1. Do the transaction. 2. PREPARE. 3.

Re: [HACKERS] Logical replication and multimaster

On Dec 3, 2015, at 4:09 AM, Craig Ringer wrote: > On 1 December 2015 at 00:20, Konstantin Knizhnik <k.knizh...@postgrespro.ru> > wrote: > > > We have implemented ACID multimaster based on logical replication and our > > DTM (distributed transaction manager) plug

Re: [HACKERS] Logical replication and multimaster

On Dec 3, 2015, at 4:18 AM, Craig Ringer wrote: > Excellent. > > It should be possible to make that a separate extension. You can use C > functions from other extensions by exposing a single pg_proc function with > 'internal' return type that populates a struct of function pointers for the >

Re: [HACKERS] Logical replication and multimaster

On Dec 3, 2015, at 10:34 AM, Craig Ringer wrote: > On 3 December 2015 at 14:54, konstantin knizhnik <k.knizh...@postgrespro.ru> > wrote: > >> I'd really like to collaborate using pglogical_output if at all possible. >> Petr's working really hard to get the pglogical

Re: [HACKERS] Logical replication and multimaster

nctions? May be it is not intended way of passing custom data to this functions... Certainly it is possible to use static variables for this purpose. But I think that passing user specific data through PGLogicalOutputData is safer and more flexible solution. On 12/03/2015 04:53 PM, Craig Ringer

Re: [HACKERS] W-TinyLfu for cache eviction

On 03.12.2015 10:27, Vladimir Sitnikov wrote: I've recently noticed W-TinyLfu cache admission policy (see [1]) being used for caffeine "high performance caching library for Java 8". It demonstrates high cache hit ratios (see [2]) and enables to build high-throughput caches (see caffeine in

[HACKERS] Tee for COPY

Hi, I am trying to create version of COPY command which can scatter/replicate data to different nodes based on some distribution method. There is some master process, having information about data distribution, to which all clients are connected. This master process should receive copied data

Re: [HACKERS] Logical replication and multimaster

tm, pg_tsdtm, bdr (sorry for plagiarism, it is just a toy, lightweight version of multimaster with asynchronous replication, used to compare performance). On 13.12.2015 15:46, Konstantin Knizhnik wrote: On 12/13/2015 12:19 PM, Simon Riggs wrote: On 6 December 2015 at 17:39, Konstantin Knizhn

Re: [HACKERS] Logical replication and multimaster

On 12/13/2015 12:19 PM, Simon Riggs wrote: On 6 December 2015 at 17:39, Konstantin Knizhnik <k.knizh...@postgrespro.ru <mailto:k.knizh...@postgrespro.ru>> wrote: I have integrated pglogical_output in multimaster, using bdr_apply from BDR as template for implementation of r

Re: [HACKERS] Optimizer questions

On 01/06/2016 12:03 PM, David Rowley wrote: Konstantin, are you thinking of looking into this more, with plans to implement code to improve this? I am not familiar with PostgreSQL optimizer, but I now looking inside its code and trying to find a way to fix this problem. But if there is

[HACKERS] Optimizer questions

Hi hackers, I want to ask two questions about PostgreSQL optimizer. I have the following query: SELECT o.id as id,s.id as sid,o.owner,o.creator,o.parent_id as dir_id,s.mime_id,m.c_type,s.p_file,s.mtime,o.ctime,o.name,o.title,o.size,o.deleted,la.otime,la.etime,uo.login as owner_login,uc.login

Re: [HACKERS] Optimizer questions

. On 01/06/2016 12:03 PM, David Rowley wrote: On 6 January 2016 at 13:13, Alexander Korotkov <a.korot...@postgrespro.ru <mailto:a.korot...@postgrespro.ru>> wrote: On Wed, Jan 6, 2016 at 12:08 AM, Tom Lane <t...@sss.pgh.pa.us <mailto:t...@sss.pgh.pa.us>> wrote:

Re: [HACKERS] On columnar storage (2)

Hi Alvaro, May be you know, that I have implemented IMCS (in-memory-columnar-store) as PostgreSQL extension. It was not so successful, mostly because people prefer to use standard SQL rather than using some special functions for accessing columnar storage (CS). Now I am thinking about second

Re: [HACKERS] Threads in PostgreSQL

Hi, PostgreSQL is not using threads but it is possible to spawn thread in your PostgreSQL extensions. For example, I have used pool of threads in my IMCS extension. But you need to build your extension with -pthread: CUSTOM_COPT = -pthread Also, please take in account that many PostgreSQL

[HACKERS] Logical replication and multimaster

Hello all, We have implemented ACID multimaster based on logical replication and our DTM (distributed transaction manager) plugin. Good news is that it works and no inconsistency is detected. But unfortunately it is very very slow... At standalone PostgreSQL I am able to achieve about 3

[HACKERS] SPI and transactions

Hello, SPI was originally developed for execution SQL statements from C user defined functions in context of existed transaction. This is why it is not possible to execute any transaction manipulation statement (BEGIN, COMMIT, PREPARE,...) using SPI_execute:SPI_ERROR_TRANSACTION is returned.

[HACKERS] Selective logical replication

Hi, I want to use logical replication for implementing multimaster (so all nodes are both sending and receiving changes). But there is one "stupid" problem: how to prevent infinite recursion and not to rereplicate replicated data. I.e. node receives change set from some other node, applies it

[HACKERS] XTM & parallel search

ext. I wonder if not only DTM will be interested in sharing some common state between workers and should we provide some way of replicating user defined context between workers? From my point of view XTM seems to be good place for it... -- Konstantin Knizhnik Postgres Professional

Re: [HACKERS] array of domain types

On 03.06.2016 02:02, Rod Taylor wrote: On Thu, Jun 2, 2016 at 10:42 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru <mailto:k.knizh...@postgrespro.ru>> wrote: On 02.06.2016 17:22, Tom Lane wrote: konstantin knizhnik <k.knizh...@postgrespro.ru

Re: [HACKERS] XTM & parallel search

On 03.06.2016 16:05, Amit Kapila wrote: On Fri, Jun 3, 2016 at 1:34 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru <mailto:k.knizh...@postgrespro.ru>> wrote: We have to add three more functions to eXtensible Transaction Manager API (XTM): /*

Re: [HACKERS] array of domain types

On Jun 1, 2016, at 4:37 PM, Thom Brown wrote: > On 1 June 2016 at 14:20, Konstantin Knizhnik <k.knizh...@postgrespro.ru> > wrote: > I wonder why domain types can not be used for specification of array element: > > create domain objref as bigint; > create table foo(x o

Re: [HACKERS] array of domain types

On Jun 2, 2016, at 12:29 PM, Thom Brown wrote:On 2 June 2016 at 10:13, konstantin knizhnik <k.knizh...@postgrespro.ru> wrote:Yes, it doesn't work:# CREATE DOMAIN teenager AS int CHECK (VALUE BETWEEN 13 AND 19);CREATE DOMAIN# SELECT 14::teenager; teenager --   14(1 row)# SEL

Re: [HACKERS] array of domain types

On 02.06.2016 17:22, Tom Lane wrote: konstantin knizhnik <k.knizh...@postgrespro.ru> writes: Attached please find patch for DefineDomain function. You didn't attach the patch, Sorry, but I did attached the patch - I see the attachment in my mail received from the group. Multidimen

[HACKERS] array of domain types

mplemented? -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Logical replication & oldest XID.

quiredXmin), I want to understand if it is real problem of logical replication or we are doing something wrong? BDR should be faced with the same problem if all updates are performed from one node... -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Com

Re: [HACKERS] Restriction of windows functions

t possible just to locate "-" or "+ operator for this type? I do not see any difference here with locating comparison operator needed for sorting. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers ma

[HACKERS] Limit and inherited tables

optimizer patch for it. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Limit and inherited tables

rows=1 loops=1) But if sort is performed by non-indexed fields, then current behaviour will be inefficient and can be significantly improved by pushing limits to remote hosts. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-

Re: [HACKERS] Optimizer questions

) AND m.mime_id = o.mime_id AND o.owner = uo.user_id AND o.creator = uc.user_id ORDER BY s.mtime LIMIT 9; -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing

[HACKERS] Restriction of windows functions

nt row is less or equal than specified range value. Looks like not something too complex to implement, doesn't it? Are there some caveats? Certainly it assumes that window is ordered by key and the key type supports subtraction, so "text" can not be used here. Something else? -- Konstantin Kn

[HACKERS] eXtensible Transaction Manager API (v2)

its usage. The complete PostgreSQL branch with all our changes can be found here: https://github.com/postgrespro/postgres_cluster -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company diff --git a/contrib/pg_tsdtm/Makefile b/contrib/pg_tsdtm/Makefil

Re: [HACKERS] Clock with Adaptive Replacement

address this problem because there are not counters - justs single bit per page. On 12.02.2016 18:55, Robert Haas wrote: On Thu, Feb 11, 2016 at 4:02 PM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: What do you think about improving cache replacement clock-sweep alg

[HACKERS] Clock with Adaptive Replacement

ARC (CAR is inspired by ARC, but it is different algorithm). As far as I know there are several problems with current clock-sweep algorithm in PostgreSQL, especially for very large caches. May be CAR can address some of them? -- Konstantin Knizhnik Postgres Professional: http

Re: [HACKERS] Batch update of indexes

h together with "index-only scans with partial indexes" patch: http://www.postgresql.org/message-id/560c7213.3010...@2ndquadrant.com only in this case regression test will produce expected output. On 27.01.2016 23:15, Robert Haas wrote: On Wed, Jan 20, 2016 at 4:28 AM, Konstant

Re: [HACKERS] Batch update of indexes

On Feb 4, 2016, at 2:00 AM, Jim Nasby wrote: > > My suspicion is that it would be useful to pre-order the new data before > trying to apply it to the indexes. Sorry, but ALTER INDEX is expected to work for all indexes, not only B-Tree, and for them sorting may not be possible... But for

Re: [HACKERS] Optimizer questions

34 rows=1 width=8) (actual time=0.007..0.007 rows=1 loops=10) Index Cond: (t1.k = k) Planning time: 0.537 ms Execution time: 0.241 ms (7 rows) On 01/30/2016 01:01 AM, Alexander Korotkov wrote: On Fri, Jan 8, 2016 at 11:58 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru <

Re: [HACKERS] PATCH: index-only scans with partial indexes

.2016 01:11, Alvaro Herrera wrote: Konstantin Knizhnik wrote: I am very interested in this patch because it allows to use partial indexes to ... speed up inserts. I have implemented "ALTER INDEX ... WHERE ..." construction which allows to change predicate of partial index without necessity

Re: [HACKERS] PATCH: index-only scans with partial indexes

re if either Kyotaro or Tomas should be considered the patch author ... maybe both?) -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to you

Re: [HACKERS] The plan for FDW-based sharding

On 02/26/2016 09:30 PM, Alvaro Herrera wrote: Konstantin Knizhnik wrote: Yes, it is certainly possible to develop cluster by cloning PostgreSQL. But it cause big problems both for developers, which have to permanently synchronize their branch with master, and, what is more important

Re: [HACKERS] The plan for FDW-based sharding

have not (yet) reviewed the literature for it. The reference to the article is at our WiKi pages explaining our DTM: https://wiki.postgresql.org/wiki/DTM http://research.microsoft.com/en-us/people/samehe/clocksi.srds2013.pdf -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com

Re: [HACKERS] Relation cache invalidation on replica

ev Postgres Professional: http://www.postgrespro.com Russian Postgres Company -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] The plan for FDW-based sharding

On 02/27/2016 06:54 AM, Robert Haas wrote: On Fri, Feb 26, 2016 at 10:56 PM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: We do not have formal prove that proposed XTM is "general enough" to handle all possible transaction manager implementations. But there are

Re: [HACKERS] The plan for FDW-based sharding

On 02/27/2016 06:57 AM, Robert Haas wrote: On Sat, Feb 27, 2016 at 1:49 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: pg_tsdtm is based on another approach: it is using system time as CSN and doesn't require arbiter. In theory there is no limit for scalability. But diffe

Re: [HACKERS] Relation cache invalidation on replica

016-02-26 18:05:55 +0300, Konstantin Knizhnik wrote: > The reason of the problem is that invalidation messages are not delivered to > replica after the end of concurrent create index. > Invalidation messages are included in xlog as part of transaction commit >

Re: [HACKERS] The plan for FDW-based sharding

visibility check. On 02/27/2016 01:48 AM, Kevin Grittner wrote: On Fri, Feb 26, 2016 at 2:19 PM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: pg_tsdtm is based on another approach: it is using system time as CSN Which brings up an interesting point, if we want logical replic

Re: [HACKERS] The plan for FDW-based sharding

people", who has different ideas on postgres future. That's why we propose this patch, let's play the game ! I don't like to play games with the architecture of PostgreSQL. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via

Re: [HACKERS] The plan for FDW-based sharding

ostgres XC/XL will probably remain a separate fork of Postgres. I don't think anyone knows the answer to this question, and I don't know how to find the answer except to keep going with our current FDW sharding approach. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com T

Re: [HACKERS] Batch update of indexes

On Jan 21, 2016, at 5:14 AM, Simon Riggs wrote: > On 20 January 2016 at 14:55, Konstantin Knizhnik <k.knizh...@postgrespro.ru> > wrote: > Hi, > > Hi, I glad to see that you interested in that too. > I think this is a good feature and I think it will be very useful t

Re: [HACKERS] Batch update of indexes

On 21.01.2016 10:14, Simon Riggs wrote: On 21 January 2016 at 06:41, konstantin knizhnik <k.knizh...@postgrespro.ru <mailto:k.knizh...@postgrespro.ru>> wrote: Certainly for B-Tree we can organize insert buffer (or pending list) as sorted array or also as a tree. But

Re: [HACKERS] Batch update of indexes

proper recovery of main index in case of failure (assuming that pending list is maintained in memory and is lost after the fault). -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresq

Re: [HACKERS] Batch update of indexes

. So small patch... Why it was not accepted? I do no see any problems with it... -- Anastasia Lubennikova Postgres Professional:http://www.postgrespro.com The Russian Postgres Company -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Proposal for UPDATE: do not insert new tuple on heap if update does not change data

est case. Kind regards, Gasper Zejn -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] Batch update of indexes

ed to be interesting by community, I will try to address these issues. On 20.01.2016 12:28, Konstantin Knizhnik wrote: Hi hackers, I want to know opinion of community about possible ways of solving quite common problem: increasing insert speed while still providing indexes for efficient executi

[HACKERS] Batch update of indexes

g idx1 on t (cost=0.00..4.13 rows=12263 width=0) Index Cond: (c1 < '10'::double precision) What do you think about this approach? Will it be useful to work in this direction? Or there are some better solutions for the problem? -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] pglogical - logical replication contrib module

astics done by ALTER TABLE, etc. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: [HACKERS] pglogical - logical replication contrib module

. On 17.02.2016 12:16, Craig Ringer wrote: On 17 February 2016 at 16:24, Konstantin Knizhnik <k.knizh...@postgrespro.ru <mailto:k.knizh...@postgrespro.ru>> wrote: Thanks for your explanation. I have to agree with your arguments that in general case replication of DDL statement u

Re: [HACKERS] The plan for FDW-based sharding

es mapping between local XIDs and global CSNs. Visibility checking rules looks on CSNs, not on XIDs. In both cases if system is for some reasons restarted and DTM plugin failed to be loaded, you can still access database locally. No data can be lost. -- Konstantin Knizhnik Postgres Professi

Re: [HACKERS] PROPOSAL: Fast temporary tables

. Actually translating that into relcache and everything else would be a serious amount of work. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Optimizer questions

On Mar 10, 2016, at 1:56 AM, Tom Lane wrote: > Konstantin Knizhnik <k.knizh...@postgrespro.ru> writes: >> I think that the best approach is to generate two different paths: >> original one, when projection is always done before sort and another one >> with postpone

Re: [HACKERS] Optimizer questions

f the path instead of comparing costs of full paths. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mail

Re: [HACKERS] eXtensible Transaction Manager API (v2)

g this 'fest. I think we'd be well advised to boot it to the 2016-09 CF and focus our efforts on other stuff that has a better chance of getting finished this month. regards, tom lane -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian P

[HACKERS] Applying logical replication changes by more than one process

done using current logical replication mechanism when changes of each slot are applied by more than one process? Or the only alternative is to write/read origin LSNs in WAL myself, for example using custom WAL records? Thanks in advance! -- Konstantin Knizhnik Postgres Professional: http://

Re: [HACKERS] Batch update of indexes

Hi David, Rebased patch is attached. On 14.03.2016 15:09, David Steele wrote: Hi Konstantin, On 2/3/16 11:47 AM, Konstantin Knizhnik wrote: Attached please find patch for "ALTER INDEX ... WHERE ..." clause. It is now able to handle all three possible situations: 1. Making index pa

[HACKERS] Very small patch for decode.c

i; -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Applying logical replication changes by more than one process

On Mar 21, 2016, at 4:30 PM, Petr Jelinek wrote: > On 21/03/16 14:25, Andres Freund wrote: >> On 2016-03-21 14:18:27 +0100, Petr Jelinek wrote: >>> On 21/03/16 14:15, Andres Freund wrote: > Only when the origin is actually setup for the current session. You > need > to call the

Re: [HACKERS] Applying logical replication changes by more than one process

On Mar 22, 2016, at 10:10 AM, Craig Ringer wrote: > On 22 March 2016 at 14:32, konstantin knizhnik <k.knizh...@postgrespro.ru> > wrote: > >> Ah you mean because with wal_log=true the origin advance is in different WAL >> record than commit? OK yeah you migh

Re: [HACKERS] Applying logical replication changes by more than one process

On Mar 22, 2016, at 11:14 AM, Petr Jelinek wrote: > > And each slot means connection with logical decoding attached to it so you > don't really want to have thousands of those anyway. I think you'll hit other > problems faster than loop over slots becomes problem if you plan to keep all > of

[HACKERS] avg,first,last,median in one query

g user-defined aggregate function which uses array_appendand so materialize all values in memory: https://wiki.postgresql.org/wiki/Aggregate_Median 3. Using percentile aggregate: http://blog.jooq.org/2015/01/06/how-to-emulate-the-median-aggregate-function-using-inverse-distribution-functions/ Th

Re: [HACKERS] Applying logical replication changes by more than one process

On 21.03.2016 15:10, Petr Jelinek wrote: Hi, On 19/03/16 11:46, Konstantin Knizhnik wrote: Hi, I am trying to use logical replication mechanism in implementation of PostgreSQL multimaster and faced with one conceptual problem. Originally logical replication was intended to support

Re: [HACKERS] Optimizer questions

On 03/08/2016 07:01 AM, Tom Lane wrote: Konstantin Knizhnik <k.knizh...@postgrespro.ru> writes: Attached please find improved version of the optimizer patch for LIMIT clause. This patch isn't anywhere close to working after 3fc6e2d7f5b652b4. (TBH, the reason I was negative about this up

Re: [HACKERS] The plan for FDW-based sharding

is area to be part of core. None of that means I would support any particular hook proposal, of course. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] The plan for FDW-based sharding

and simple OLAP. For OLTP we definitely need transaction manager to provide global consistency. And we have actually prototype of integration postgres_fdw with out pg_dtm and pg_tsdtm transaction managers. The results are quite IMHO promising (see attached diagram). -- Konstantin Knizhnik Postgres

Re: [HACKERS] The plan for FDW-based sharding

Thank you very much for you comments. On 01.03.2016 18:19, Robert Haas wrote: On Sat, Feb 27, 2016 at 2:29 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: How do you prevent clock skew from causing serialization anomalies? If node receives message from "feature&quo

Re: [HACKERS] The plan for FDW-based sharding

uspect that a transaction manager API would end up similarly situated. IMHO non-stable API is better than lack of API. Just because it makes it possible to implement features in modular way. And refactoring of API is not so difficult thing... -- Konstantin Knizhnik Postgres Professional: http://www.postgresp

Re: [HACKERS] Batch update of indexes

On 04/02/2016 09:57 PM, Tom Lane wrote: Konstantin Knizhnik <k.knizh...@postgrespro.ru> writes: Attached please find patch for "ALTER INDEX ... WHERE ..." clause. It is now able to handle all three possible situations: 1. Making index partial (add WHERE condition to the o

Re: [HACKERS] Inheritance

f this. Feature-wise, we need to be doing *more* of this kind of work, not less. Lack of support for OO paradigms was one of the drivers for NoSQL. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list

Re: [HACKERS] asynchronous and vectorized execution

On 05/10/2016 08:26 PM, Robert Haas wrote: On Tue, May 10, 2016 at 3:00 AM, konstantin knizhnik <k.knizh...@postgrespro.ru> wrote: What's wrong with it that worker is blocked? You can just have more workers (more than CPU cores) to let other of them continue to do useful work. Not

Re: [HACKERS] asynchronous and vectorized execution

made good progress since February: them have rewritten most of methods of Scan, Aggregate and Join to LLVM API. Also then implemented automatic translation of PostgreSQL backend functions to LLVM API. As a result time of TPC-H Q1 query is reduced four times. -- Konstantin Knizhnik Postgres Pr

Re: [HACKERS] asynchronous and vectorized execution

On 11.05.2016 17:00, Robert Haas wrote: On Tue, May 10, 2016 at 3:42 PM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: Doesn't this actually mean that we need to have normal job scheduler which is given queue of jobs and having some pool of threads will be able to orginize eff

Re: [HACKERS] Academic help for Postgres

On 11.05.2016 17:32, Bruce Momjian wrote: On Wed, May 11, 2016 at 05:31:10PM +0300, Konstantin Knizhnik wrote: On 11.05.2016 17:20, Bruce Momjian wrote: I am giving a keynote at an IEEE database conference in Helsinki next week (http://icde2016.fi/). (Yes, I am not attending PGCon Ottawa

Re: [HACKERS] Academic help for Postgres

Sharding Any others? Incremental materialized views? -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] Academic help for Postgres

On May 12, 2016, at 6:16 AM, Rajeev rastogi wrote: > On 11 May 2016 19:50, Bruce Momjian Wrote: > > >> I am giving a keynote at an IEEE database conference in Helsinki next >> week (http://icde2016.fi/). (Yes, I am not attending PGCon Ottawa >> because I accepted the Helsinki conference

Re: [HACKERS] Just-in-time compiling things

On 05/14/2016 12:10 PM, Andreas Seltenreich wrote: Konstantin Knizhnik writes: Latest information from ISP RAS guys: them have made good progress since February: them have rewritten most of methods of Scan, Aggregate and Join to LLVM API. Is their work available somewhere? I'm experimenting

Re: [HACKERS] asynchronous and vectorized execution

Hi, > 1. asynchronous execution, It seems to me that asynchronous execution can be considered as alternative to multithreading model (in case of PostgreSQL the roles of threads are played by workers). Async. operations are used to have smaller overhead but have scalability problems (because

Re: [HACKERS] WIP: Barriers

Hi Thomas, Barriers are really very simple and convenient mechanism for process synchronization. But it is actually a special case of semaphores: having semaphore primitive it is trivial to implement a barrier. We have semaphores in Postgres, but ... them can not be used by extensions: there

Re: [HACKERS] WIP: Barriers

ATABASE_CONNECTION, BgWorkerStart_ConsistentState, BGW_NEVER_RESTART, PongLatch }; static void PingPong() { RegisterBackgroundWorker(); RegisterBackgroundWorker(); } -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- S

Re: [HACKERS] Deadlock in XLogInsert at AIX

d to debug it. Heikki, I will be pleased if you have a chance to login at the system and look at it yourself. May be you will have some idea what's happening... Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgs

  1   2   3   >