Re: [HACKERS] Declarative partitioning - another take

2016-10-04 Thread Petr Jelinek
On 05/10/16 03:50, Robert Haas wrote: > On Tue, Oct 4, 2016 at 4:02 AM, Amit Langote > wrote: >> [ latest patch set ] > > Reviewing 0003: > > + with matching types and no more. Also, it must have all the matching > + constraints as the target table.

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2016-10-04 Thread Michael Paquier
On Mon, Oct 3, 2016 at 12:48 PM, Craig Ringer wrote: > On 3 October 2016 at 10:10, Michael Paquier wrote: >> On Tue, Sep 6, 2016 at 8:01 PM, Craig Ringer wrote: >>> On 6 September 2016 at 16:10, Daniel Verite

[HACKERS] Aggregation push down, reorder of join and group by

2016-10-04 Thread Chenxi Li
Hello, I'm reading some papers about aggregation push down like in " https://ub-madoc.bib.uni-mannheim.de/37228/1/main.pdf; and " http://www.vldb.org/conf/1995/P345.PDF;. I think it is very useful but very complex to implement. In some complex queries, it can be a lot faster. Is there any plan to

Re: [HACKERS] postgres_fdw : altering foreign table not invalidating prepare statement execution plan.

2016-10-04 Thread Ashutosh Bapat
> > I looked at your patch closely. You added code to track dependencies on > FDW-related objects to createplan.c, but I think it would be more > appropriate to put that code in setrefs.c like the attached. I think > record_foreignscan_plan_dependencies in your patch would be a bit > inefficient

Re: [HACKERS] Is the last 9.1 release planned?

2016-10-04 Thread Magnus Hagander
On Oct 5, 2016 5:42 AM, "Tsunakawa, Takayuki" < tsunakawa.ta...@jp.fujitsu.com> wrote: > > From: pgsql-hackers-ow...@postgresql.org > > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Jaime Casanova > > Well, no. We normally don't give special treatment to any minor release > > not even

Re: [HACKERS] Proposal: ON UPDATE REMOVE foreign key action

2016-10-04 Thread Vitaly Burovoy
On 10/4/16, Kirill Berezin wrote: > Disclaimer: sorry, i dont understand, should i reply to each of you > personally, or just answer to channel. Some feedbacks were sended in > personal, and some include channel copy. Usually discussions are in the list, therefore you should

Re: [HACKERS] Aggregation push down, reorder of join and group by

2016-10-04 Thread Amit Langote
Hi, On 2016/10/05 14:19, Chenxi Li wrote: > Hello, > > I'm reading some papers about aggregation push down like in " > https://ub-madoc.bib.uni-mannheim.de/37228/1/main.pdf; and " > http://www.vldb.org/conf/1995/P345.PDF;. I think it is very useful but very > complex to implement. In some

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-10-04 Thread Petr Jelinek
On 05/10/16 03:11, Tomas Vondra wrote: > On 10/04/2016 09:44 PM, John Gorman wrote: >> >> Remind me again why we cannot realloc in place for sizes >> smaller than chunkSize? GenSlab is happy to allocate smaller >> sizes and put them into the fixed size chunks. >> >> Maybe SlabAlloc can be happy

Re: [HACKERS] Hash tables in dynamic shared memory

2016-10-04 Thread Thomas Munro
On Wed, Oct 5, 2016 at 11:22 AM, Andres Freund wrote: >> Potential use cases for DHT include caches, in-memory database objects >> and working state for parallel execution. > > Is there a more concrete example, i.e. a user we'd convert to this at > the same time as introducing

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-10-04 Thread John Gorman
SlabContext has a parent context. It can delegate requests that it cannot handle to the parent context which is GenSlab. Genslab can send them to the sister aset context. This could handle all reallocs so there will be no surprises. Remind me again why we cannot realloc in place for sizes

Re: [HACKERS] Misidentification of Python shared library

2016-10-04 Thread Robert Haas
On Tue, Oct 4, 2016 at 3:38 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Oct 4, 2016 at 12:57 PM, Tom Lane wrote: >>> Sure, but it will only matter given a more-or-less-broken Python >>> installation. Anyway, if you have a

Re: [HACKERS] Misidentification of Python shared library

2016-10-04 Thread Tom Lane
Aidan Van Dyk writes: > https://packages.debian.org/jessie/amd64/libpython3.4/filelist > The ".so." is in /usr/lib/, but ".so" symlink is in the > config directory... Hah. I guess that explains why they are setting LIBDIR to /usr/lib. Thanks for the info!

Re: [HACKERS] [PATCH] Generic type subscription

2016-10-04 Thread Oleg Bartunov
On Sat, Oct 1, 2016 at 12:52 PM, Dmitry Dolgov <9erthali...@gmail.com> wrote: > > I've tried to compile this patch with current state of master (commit > > 51c3e9fade76c12) and found out that, when configured with > --enable-cassert, > > it doesn't pass make check. > > Thanks for the feedback.

Re: [HACKERS] Misidentification of Python shared library

2016-10-04 Thread Tom Lane
Robert Haas writes: > On Tue, Oct 4, 2016 at 12:57 PM, Tom Lane wrote: >> Sure, but it will only matter given a more-or-less-broken Python >> installation. Anyway, if you have a better idea, let's hear it. > Sorry, no clue So I pushed that, and

Re: [HACKERS] condition variables

2016-10-04 Thread Thomas Munro
On Mon, Aug 15, 2016 at 5:58 PM, Thomas Munro wrote: > Please find attached a -v2 of your patch which includes suggestions > 1-3 above. Here's a rebased patch. ConditionVariableSleep now takes wait_event_info. Anyone using this in patches for core probably needs

Re: [HACKERS] Misidentification of Python shared library

2016-10-04 Thread Tom Lane
Robert Haas writes: > On Tue, Oct 4, 2016 at 3:38 PM, Tom Lane wrote: >> So I pushed that, and most of the Debian-based buildfarm critters >> don't like it. Where does Debian keep the python shlib, pray tell? >

Re: [HACKERS] Tracking wait event for latches

2016-10-04 Thread Thomas Munro
On Wed, Oct 5, 2016 at 4:59 AM, Robert Haas wrote: > On Mon, Oct 3, 2016 at 8:43 PM, Michael Paquier > wrote: >> The rest looks good to me. Thanks for the feedback and the time! > > Thanks for the fixes. I committed this with an additional

[HACKERS] longfin

2016-10-04 Thread Robert Haas
So, buildfarm member longfin is failing with: test python3/hstore_plpython ... FAILED (test process exited with exit code 2) It claims that it's been failing since 6f3bd98 Tue Oct 4 15:01:42 2016 UTC Extend framework from commit 53be0b1ad to report latch waits. which would tend to pointer

Re: [HACKERS] pgbench more operators & functions

2016-10-04 Thread Fabien COELHO
Hello Robert, I think it's pretty clear that this patch is not Ready for Committer, As a reviewer, I do not know when to decide to put something as "ready". My opinion is that it is a matter of the reviewer to decide. Whether some consensus is actually reached, or whether a committer is

Re: [HACKERS] Misidentification of Python shared library

2016-10-04 Thread Tom Lane
Robert Haas writes: > On Tue, Oct 4, 2016 at 11:45 AM, Tom Lane wrote: >> "$python_libdir /usr/lib64 /usr/lib" (too bad there's no easy >> platform-independent way to find out the linker's default search path). > "too bad" sounds like a grave

Re: [HACKERS] Question / requests.

2016-10-04 Thread Robert Haas
On Mon, Oct 3, 2016 at 5:44 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Fri, Sep 30, 2016 at 11:20 AM, Francisco Olarte >> wrote: >> > After some messages due to vacuumdb auto-deadlocking itself on the >> > system tables when doing

Re: [HACKERS] Cache Hash Index meta page.

2016-10-04 Thread Jeff Janes
On Fri, Jul 22, 2016 at 3:02 AM, Mithun Cy wrote: > I have created a patch to cache the meta page of Hash index in > backend-private memory. This is to save reading the meta page buffer every > time when we want to find the bucket page. In “_hash_first” call, we try >

Re: [HACKERS] Misidentification of Python shared library

2016-10-04 Thread Robert Haas
On Tue, Oct 4, 2016 at 12:57 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Oct 4, 2016 at 11:45 AM, Tom Lane wrote: >>> "$python_libdir /usr/lib64 /usr/lib" (too bad there's no easy >>> platform-independent way to find out

Re: [HACKERS] longfin

2016-10-04 Thread Tom Lane
Robert Haas writes: > So, buildfarm member longfin is failing with: > test python3/hstore_plpython ... FAILED (test process exited with exit code > 2) Yeah, this is a side effect of what I'm working on, don't sweat it. (The only reason it's passed at all today is that I

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-10-04 Thread Robert Haas
On Tue, Oct 4, 2016 at 1:48 AM, Michael Paquier wrote: > Yes, pg_wal is fine as new name in replacement of pg_xlog. Now for the > pg_clog... Of course the irony here is that "WAL" stands for "Write Ahead Log". So we're renaming a directly that has "log" in the name

Re: [HACKERS] Declarative partitioning - another take

2016-10-04 Thread Robert Haas
On Tue, Oct 4, 2016 at 4:02 AM, Amit Langote wrote: >> Even if we leave the empty relfilenode around for now -- in the long >> run I think it should die -- I think we should prohibit the creation >> of subsidiary object on the parent which is only sensible if it has

[HACKERS] Hash tables in dynamic shared memory

2016-10-04 Thread Thomas Munro
Hi hackers, Here is an experimental hash table implementation that uses DSA memory so that hash tables can be shared by multiple backends and yet grow dynamically. Development name: "DHT". It's impossible to write a general purpose hash table that will be suitable for every use case,

[HACKERS] Idea: Tell compiler palloc() et al return aligned values

2016-10-04 Thread Andres Freund
Hi, Just saw a fair amount of memset/memcpy in a profile, with the symbol name indicating that the compiler assumes the data is unaligned. After pondering it for a few moments, that's not too surprising: Even if the memory is allocated in the same place, the compiler has no way knowing that

Re: [HACKERS] Stopping logical replication protocol

2016-10-04 Thread Vladimir Gordiychuk
> Vladimir, can I have your opinion on the latest patch or if you want to make changes, an updated patch? I am satisfied with all our changes and I thinks it enough to complete this PR. 2016-10-03 6:51 GMT+03:00 Craig Ringer : > On 3 Oct. 2016 10:15, "Michael Paquier"

Re: [HACKERS] Stopping logical replication protocol

2016-10-04 Thread Andres Freund
Hi, On 2016-09-23 13:01:27 +0800, Craig Ringer wrote: > From f98f2388c57d938ebbe07ccd2dbe02138312858f Mon Sep 17 00:00:00 2001 > From: Vladimir Gordiychuk > Date: Wed, 7 Sep 2016 00:39:18 +0300 > Subject: [PATCH 2/4] Client-initiated CopyDone during transaction decoding in >

[HACKERS] Creating a DSA area to provide work space for parallel execution

2016-10-04 Thread Thomas Munro
Hi hackers, A couple of months ago I proposed dynamic shared areas[1]. DSA areas are dynamically sized shared memory heaps that backends can use to share data, building on top of the existing DSM infrastructure. One target use case for DSA areas is to provide work space for parallel query

Re: [HACKERS] proposal: psql \setfileref

2016-10-04 Thread Gilles Darold
Le 04/10/2016 à 17:29, Pavel Stehule a écrit : > > > 2016-10-04 9:18 GMT+02:00 Gilles Darold >: > > Le 03/10/2016 à 23:23, Gilles Darold a écrit : > > Le 03/10/2016 à 23:03, Robert Haas a écrit : > >> On Mon, Oct 3, 2016 at

Re: [HACKERS] Tracking wait event for latches

2016-10-04 Thread Michael Paquier
On Wed, Oct 5, 2016 at 4:28 AM, Thomas Munro wrote: > On Wed, Oct 5, 2016 at 4:59 AM, Robert Haas wrote: >> On Mon, Oct 3, 2016 at 8:43 PM, Michael Paquier >> wrote: >>> The rest looks good to me. Thanks for the

Re: [HACKERS] Hash tables in dynamic shared memory

2016-10-04 Thread Andres Freund
Hi, > It's impossible to write a general purpose hash table that will be > suitable for every use case, considering all the combinations of > design trade offs and subsets of functionality someone might want. Very much agreed. > There is other work being done in this space: I'm aware of

Re: [HACKERS] Question / requests.

2016-10-04 Thread Francisco Olarte
On Mon, Oct 3, 2016 at 11:44 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Fri, Sep 30, 2016 at 11:20 AM, Francisco Olarte >> What messages are you seeing, exactly? "auto-deadlocking" isn't a thing. >

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2016-10-04 Thread Amit Langote
On 2016/10/04 16:10, Ashutosh Bapat wrote: >>> Heuristics can not become the default behavior. A user should be given >>> an option to choose a heuristic, and he should be aware of the >>> pitfalls when using this heuristic. I guess, first, we need to get a >>> solution which ensures that the

Re: [HACKERS] [PATCH] Generic type subscription

2016-10-04 Thread Victor Wagner
On Sat, 1 Oct 2016 16:52:34 +0700 Dmitry Dolgov <9erthali...@gmail.com> wrote: > > I've tried to compile this patch with current state of master > > (commit 51c3e9fade76c12) and found out that, when configured with > --enable-cassert, > > it doesn't pass make check. > > Thanks for the

Re: [HACKERS] multivariate statistics (v19)

2016-10-04 Thread Dean Rasheed
On 4 October 2016 at 04:25, Michael Paquier wrote: > OK. A second thing was related to the use of schemas in the new system > catalogs. As mentioned in [1], those could be removed. > [1]: >

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2016-10-04 Thread Gavin Flower
On 04/10/16 20:15, Michael Paquier wrote: On Mon, Oct 3, 2016 at 11:52 PM, Daniel Verite wrote: Wouldn't pgbench benefit from it? It was mentioned some time ago [1], in relationship to the \into construct, how client-server latency was important enough to justify the

Re: [HACKERS] multivariate statistics (v19)

2016-10-04 Thread Dean Rasheed
On 4 October 2016 at 09:15, Heikki Linnakangas wrote: > However, for tables and views, each object you store in those views is a > "table" or "view", but with this thing, the object you store is > "statistics". Would you have a catalog table called "pg_scissor"? > No, probably

Re: [HACKERS] multivariate statistics (v19)

2016-10-04 Thread Gavin Flower
On 04/10/16 20:37, Dean Rasheed wrote: On 4 October 2016 at 04:25, Michael Paquier wrote: OK. A second thing was related to the use of schemas in the new system catalogs. As mentioned in [1], those could be removed. [1]:

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2016-10-04 Thread Craig Ringer
On 4 Oct. 2016 15:15, "Michael Paquier" wrote: > > On Mon, Oct 3, 2016 at 11:52 PM, Daniel Verite wrote: > > Wouldn't pgbench benefit from it? > > It was mentioned some time ago [1], in relationship to the > > \into construct, how client-server

Re: [HACKERS] multivariate statistics (v19)

2016-10-04 Thread Dean Rasheed
On 30 September 2016 at 12:10, Heikki Linnakangas wrote: > I fear that using "statistics" as the name of the new object might get a bit > awkward. "statistics" is a plural, but we use it as the name of a single > object, like "pants" or "scissors". Not sure I have any better

Re: [HACKERS] multivariate statistics (v19)

2016-10-04 Thread Heikki Linnakangas
On 10/04/2016 10:49 AM, Dean Rasheed wrote: On 30 September 2016 at 12:10, Heikki Linnakangas wrote: I fear that using "statistics" as the name of the new object might get a bit awkward. "statistics" is a plural, but we use it as the name of a single object, like "pants" or

Re: [HACKERS] [RFC] Should we fix postmaster to avoid slow shutdown?

2016-10-04 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Tom Lane FWIW, I'm pretty much -1 on messing with the timing of the socket close > actions. I broke that once within recent memory, so maybe I'm gun-shy, > but I think that the odds of unpleasant

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2016-10-04 Thread Masahiko Sawada
On Tue, Oct 4, 2016 at 8:29 PM, Ashutosh Bapat wrote: > On Tue, Oct 4, 2016 at 1:11 PM, Amit Langote > wrote: >> On 2016/10/04 16:10, Ashutosh Bapat wrote: > Heuristics can not become the default behavior. A user should be given

Re: [HACKERS] WIP: Covering + unique indexes.

2016-10-04 Thread Amit Kapila
On Tue, Sep 27, 2016 at 7:51 PM, Robert Haas wrote: > On Mon, Sep 26, 2016 at 11:17 AM, Anastasia Lubennikova > wrote: >>> Is there any fundamental problem in storing them in ordered way? I >>> mean to say, you need to anyway store all the

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-10-04 Thread David Steele
On 10/4/16 1:48 AM, Michael Paquier wrote: > So this is still open for votes. Here are the candidates and who > voiced for what: > - pg_transaction: Michael P, Thomas M. => Current 0002 is doing that. > - pg_xact: David S, Robert > - pg_trans: Tom > - pg_transaction_status: Peter E. Christoph

Re: [HACKERS] pnstrdup considered armed and dangerous

2016-10-04 Thread Geoff Winkless
On 4 October 2016 at 14:12, Geoff Winkless wrote: > Well I wouldn't say it's wrong, exactly. It might produce a segfault > if relationName[NAMEDATALEN] is outside readable memory for the > process, but otherwise it will behave as defined. Finger slippage. Of course I meant

Re: [HACKERS] pnstrdup considered armed and dangerous

2016-10-04 Thread Robert Haas
On Mon, Oct 3, 2016 at 5:55 PM, Andres Freund wrote: > /* > * pnstrdup > * Like pstrdup(), but append null byte to a > * not-necessarily-null-terminated input string. > */ > char * > pnstrdup(const char *in, Size len) > { > char *out

Re: [HACKERS] pnstrdup considered armed and dangerous

2016-10-04 Thread Geoff Winkless
On 3 October 2016 at 22:55, Andres Freund wrote: > A colleage of me just wrote innocent looking code like > char *shardRelationName = pnstrdup(relationName, NAMEDATALEN); > which is at the moment wrong if relationName isn't preallocated to > NAMEDATALEN size. [snip] >

Re: [HACKERS] asynchronous execution

2016-10-04 Thread Amit Khandekar
On 4 October 2016 at 02:30, Robert Haas wrote: > On Wed, Sep 28, 2016 at 12:30 AM, Amit Khandekar > wrote: >> On 24 September 2016 at 06:39, Robert Haas wrote: >>> Since Kyotaro Horiguchi found that my previous design had a

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2016-10-04 Thread Ashutosh Bapat
On Tue, Oct 4, 2016 at 1:11 PM, Amit Langote wrote: > On 2016/10/04 16:10, Ashutosh Bapat wrote: Heuristics can not become the default behavior. A user should be given an option to choose a heuristic, and he should be aware of the pitfalls when using

[HACKERS] Logical tape pause/resume

2016-10-04 Thread Heikki Linnakangas
One of the patches in Peter Geoghegan's Parallel tuplesort patch set [1] is to put a cap on the number of tapes to use for the sort. The reason is that each tape consumes 3 * BLCKSZ worth of memory, for the buffers. We decide the max number of tapes upfront, so if we decide that e.g. the max

Re: [HACKERS] Logical tape pause/resume

2016-10-04 Thread Simon Riggs
On 4 October 2016 at 11:47, Heikki Linnakangas wrote: > When we finish writing an initial run to a tape, we keep the tape buffers > around. That's the reason we need to reserve that memory. But there's no > fundamental reason we have to do that. As I suggested in [2], we could

[HACKERS] Incorrect comment/doc for poll_query_until

2016-10-04 Thread Daniel Gustafsson
Commit 2a0f89cd717ce6d raised the timeout in poll_query_until() to 180 seconds but left the documentation and comments at the previous value of 90 sec. Patch attached to update documentation and comment to match reality. cheers ./daniel poll_query_until_timeout_doc.diff Description: Binary

Re: [HACKERS] postgres_fdw : altering foreign table not invalidating prepare statement execution plan.

2016-10-04 Thread Etsuro Fujita
On 2016/09/30 1:09, Ashutosh Bapat wrote: You wrote: The attached patch adds the dependencies from create_foreignscan_plan() which is called for any foreign access. I have also added testcases to test the functionality. Let me know your comments on the patch. I wrote: Hmm. I'm not sure that

Re: [HACKERS] [PATCH] pgpassfile connection option

2016-10-04 Thread Julian Markwort
On 09/26/2016 07:51 PM, Robert Haas wrote: However, they don't have to accept the possibility that arbitrary local files readable by the user ID will be used for authentication and/or disclosed; this patch would force them to accept that risk. I do agree with you, however we might have to take a

Re: [HACKERS] Logical tape pause/resume

2016-10-04 Thread Heikki Linnakangas
On 10/04/2016 02:09 PM, Simon Riggs wrote: On 4 October 2016 at 11:47, Heikki Linnakangas wrote: When we finish writing an initial run to a tape, we keep the tape buffers around. That's the reason we need to reserve that memory. But there's no fundamental reason we have to do

[HACKERS] Cardinality estimation for group by

2016-10-04 Thread Chenxi Li
Friends, How is cardinality estimation for "group by" is done and where is the code doing that? Best Regards, Chenxi Li

Re: [HACKERS] Rename max_parallel_degree?

2016-10-04 Thread Julien Rouhaud
On 03/10/2016 21:27, Robert Haas wrote: > On Fri, Sep 30, 2016 at 12:23 PM, Julien Rouhaud > wrote: >> I've already fixed every other issues mentioned upthread, but I'm facing >> a problem for this one. Assuming that the bgworker classes are supposed >> to be mutually

Re: [HACKERS] WIP: Covering + unique indexes.

2016-10-04 Thread Robert Haas
On Tue, Oct 4, 2016 at 9:20 AM, Amit Kapila wrote: >>> I'd say that the reason for not using included columns in any >>> operations which require comparisons, is that they don't have opclass. >>> Let's go back to the example of points. >>> This data type don't have any

[HACKERS] Misidentification of Python shared library

2016-10-04 Thread Tom Lane
While chasing down last night's failure on buildfarm member longfin, I came across an interesting factoid. If you just do ./configure --enable-shared make make install in unmodified Python sources, what you will get is an install tree in which libpython.so (or local

Re: [HACKERS] WIP: Covering + unique indexes.

2016-10-04 Thread Anastasia Lubennikova
03.10.2016 15:29, Anastasia Lubennikova: 03.10.2016 05:22, Michael Paquier: On Tue, Sep 27, 2016 at 12:17 AM, Anastasia Lubennikova wrote: Ok, I'll write it in a few days. Marked as returned with feedback per last emails exchanged. The only complaint about

Re: [HACKERS] Move allocation size overflow handling to MemoryContextAllocExtended()?

2016-10-04 Thread Michael Paquier
On Wed, Oct 5, 2016 at 8:56 AM, Andres Freund wrote: > That made me wonder if it's not actually a mistake for > MemoryContextAllocExtended() size parameter to be declared > Size/size_t. That prevents it from detecting such overflows, forcing > code like the above on callsites.

Re: [HACKERS] Move allocation size overflow handling to MemoryContextAllocExtended()?

2016-10-04 Thread Andres Freund
On 2016-10-05 09:38:15 +0900, Michael Paquier wrote: > On Wed, Oct 5, 2016 at 8:56 AM, Andres Freund wrote: > > That made me wonder if it's not actually a mistake for > > MemoryContextAllocExtended() size parameter to be declared > > Size/size_t. That prevents it from

[HACKERS] Feature suggestion: ON CONFLICT DO NOTHING RETURNING which returns existing row data

2016-10-04 Thread Tom Dunstan
Hi all We recently moved to using 9.5 and were hoping to use the new upsert functionality, but unfortunately it doesn’t quite do what we need. Our setup is something like this: CREATE TABLE t1 ( id BIGSERIAL NOT NULL PRIMARY KEY, bk1 INT, bk2 UUID — other columns ); CREATE UNIQUE INDEX

Re: [HACKERS] Hash tables in dynamic shared memory

2016-10-04 Thread Thomas Munro
On Wed, Oct 5, 2016 at 12:11 PM, Thomas Munro wrote: > On Wed, Oct 5, 2016 at 11:22 AM, Andres Freund wrote: >>> Potential use cases for DHT include caches, in-memory database objects >>> and working state for parallel execution. >> >> Is there

[HACKERS] Move allocation size overflow handling to MemoryContextAllocExtended()?

2016-10-04 Thread Andres Freund
Hi, I was working on making sure that allocations for [1] don't overflow size_t for large hash tables, when created on 32bit systems. I started to write code like if (sizeof(SH_CONTAINS) * (uint64) tb->size) != sizeof(SH_CONTAINS) * (size_t) tb->size)) { elog(ERROR, "hash table too large

Re: [HACKERS] Logical tape pause/resume

2016-10-04 Thread Simon Riggs
On 4 October 2016 at 12:47, Heikki Linnakangas wrote: >> Why not just make each new run start at a block boundary? >> That way we waste on average BLCKSZ/2 disk space per run, which is >> negligible but we avoid any need to have code to read back in the last >> block. > > > Hmm.

Re: [HACKERS] Proposal: ON UPDATE REMOVE foreign key action

2016-10-04 Thread Kirill Berezin
Disclaimer: sorry, i dont understand, should i reply to each of you personally, or just answer to channel. Some feedbacks were sended in personal, and some include channel copy. Thanks for responses, you understand it correctly. When i said "anybody", i mean inclusive owner himself. For example

Re: [HACKERS] Commit fest 2016-09 is now closed

2016-10-04 Thread Fabrízio de Royes Mello
On Mon, Oct 3, 2016 at 2:05 AM, Michael Paquier wrote: > > On Mon, Oct 3, 2016 at 2:01 PM, Michael Paquier > wrote: > > There was something like 60~70 patches still listed as active in the > > CF app on Friday my time, so doing the vacuum

Re: [HACKERS] Misidentification of Python shared library

2016-10-04 Thread Andrew Dunstan
On 10/04/2016 10:43 AM, Tom Lane wrote: While chasing down last night's failure on buildfarm member longfin, I came across an interesting factoid. If you just do ./configure --enable-shared make make install in unmodified Python sources, what you will get is an

Re: [HACKERS] proposal: psql \setfileref

2016-10-04 Thread Pavel Stehule
2016-10-04 9:18 GMT+02:00 Gilles Darold : > Le 03/10/2016 à 23:23, Gilles Darold a écrit : > > Le 03/10/2016 à 23:03, Robert Haas a écrit : > >> On Mon, Oct 3, 2016 at 3:54 PM, Gilles Darold > wrote: > >>> 4) An other problem is that like this this

Re: [HACKERS] [COMMITTERS] pgsql: Extend framework from commit 53be0b1ad to report latch waits.

2016-10-04 Thread Tom Lane
Robert Haas writes: > Apparently, 'make world' does not build worker_spi. I thought 'make > world' was supposed to build everything? You'd have thunk, yeah. It looks like the issue is that src/Makefile is selective about recursing into certain subdirectories of test/,

Re: [HACKERS] Is the last 9.1 release planned?

2016-10-04 Thread Tsunakawa, Takayuki
From: Michael Paquier [mailto:michael.paqu...@gmail.com] > 9.1.24 will be the last in the 9.1 series as far as I know. And it is still > to come at the beginning of November: > https://www.postgresql.org/developer/roadmap/ But the release note for 9.1.23 says: "The PostgreSQL community will stop

Re: [HACKERS] Is the last 9.1 release planned?

2016-10-04 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Jaime Casanova > Well, no. We normally don't give special treatment to any minor release > not even if it is going to die. > What normally happens is that all minor releases are released the same

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-10-04 Thread Tomas Vondra
On 10/04/2016 09:44 PM, John Gorman wrote: SlabContext has a parent context. It can delegate requests that it cannot handle to the parent context which is GenSlab. Genslab can send them to the sister aset context. But Slab may also be used separately, not just as part of GenSlab (actually,

[HACKERS] Is the last 9.1 release planned?

2016-10-04 Thread Tsunakawa, Takayuki
Hello, (Please point me to the appropriate ML if this is not the right one.) According to the following mail, I thought one more release for 9.1 (9.1.24) was scheduled in September. Is there any release plan for the 9.1 last release? If there's, I want to wait for it, and apply 9.1.23

Re: [HACKERS] Move allocation size overflow handling to MemoryContextAllocExtended()?

2016-10-04 Thread Tom Lane
Andres Freund writes: > On 2016-10-05 09:38:15 +0900, Michael Paquier wrote: >> The existing interface of MemoryContextAlloc do not care much about >> anything except Size, so I'd just give the responsability to the >> caller to do checks like queue != (Size) queue when queue

Re: [HACKERS] Is the last 9.1 release planned?

2016-10-04 Thread Jaime Casanova
On 4 October 2016 at 22:00, Tsunakawa, Takayuki wrote: > > and 9.0.23 was released in October 8. So I guessed 9.1.24 will be released > in a week or so. > Well, no. We normally don't give special treatment to any minor release not even if it is going to die.

Re: [HACKERS] [PATCH] Generic type subscription

2016-10-04 Thread Dmitry Dolgov
On 5 October 2016 at 03:00, Oleg Bartunov wrote: > > have you ever run 'make check' ? > > = > 53 of 168 tests failed. > = > > Sure, how else could I write tests for this feature? But right now on my machine everything is ok

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-10-04 Thread Tomas Vondra
Hi, attached is v3 of the patches, with a few minor fixes in Slab, and much larger fixes in GenSlab. Slab (minor fixes) -- - Removed the unnecessary memset() of new blocks in SlabAlloc(), although we still need to zero the free bitmap at the end of the block. - Renamed

Re: [HACKERS] Is the last 9.1 release planned?

2016-10-04 Thread Michael Paquier
On Wed, Oct 5, 2016 at 10:09 AM, Tsunakawa, Takayuki wrote: > [Excerpt] > PostgreSQL version 9.1 will be End-of-Life in September 2016. The project > expects to only release one more update for that version. 9.1.24 will be the last in the 9.1 series as far as I

Re: [HACKERS] Declarative partitioning - another take

2016-10-04 Thread Robert Haas
On Tue, Oct 4, 2016 at 4:02 AM, Amit Langote wrote: > [ latest patch set ] Reviewing 0003: + This form attaches an existing table (partitioned or otherwise) as (which might itself be partitioned) + partition of the target table. Partition bound

Re: [HACKERS] Misidentification of Python shared library

2016-10-04 Thread Tom Lane
Andrew Dunstan writes: > On 10/04/2016 10:43 AM, Tom Lane wrote: >> In short: I propose replacing all of this logic with "if there's something >> in $python_libdir that has the right name to be a python shared library, >> use that, else try the same in $python_configdir, else

Re: [HACKERS] Tracking wait event for latches

2016-10-04 Thread Robert Haas
On Mon, Oct 3, 2016 at 8:43 PM, Michael Paquier wrote: > The rest looks good to me. Thanks for the feedback and the time! Thanks for the fixes. I committed this with an additional compile fix, but the buildfarm turned up a few more problems that my 'make check-world'

Re: [HACKERS] Logical tape pause/resume

2016-10-04 Thread Heikki Linnakangas
On 10/04/2016 05:58 PM, Simon Riggs wrote: On 4 October 2016 at 12:47, Heikki Linnakangas wrote: Why not just make each new run start at a block boundary? That way we waste on average BLCKSZ/2 disk space per run, which is negligible but we avoid any need to have code to read

Re: [HACKERS] [RFC] Should we fix postmaster to avoid slow shutdown?

2016-10-04 Thread Robert Haas
On Tue, Oct 4, 2016 at 5:05 AM, Tsunakawa, Takayuki wrote: > From: pgsql-hackers-ow...@postgresql.org >> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Tom Lane > FWIW, I'm pretty much -1 on messing with the timing of the socket close >> actions. I broke

Re: [HACKERS] Logical tape pause/resume

2016-10-04 Thread Claudio Freire
On Tue, Oct 4, 2016 at 7:47 AM, Heikki Linnakangas wrote: > 1. The first patch changes the way we store the logical tapes on disk. > Instead of using indirect blocks, HFS style, the blocks form a linked list. > Each block has a trailer, with the block numbers of the previous and

Re: [HACKERS] Re: [COMMITTERS] pgsql: Extend framework from commit 53be0b1ad to report latch waits.

2016-10-04 Thread Alvaro Herrera
Robert Haas wrote: > On Tue, Oct 4, 2016 at 11:29 AM, Tom Lane wrote: > > Robert Haas writes: > >> Apparently, 'make world' does not build worker_spi. I thought 'make > >> world' was supposed to build everything? > > > > You'd have thunk, yeah. It

Re: [HACKERS] Logical tape pause/resume

2016-10-04 Thread Heikki Linnakangas
On 10/04/2016 07:06 PM, Claudio Freire wrote: On Tue, Oct 4, 2016 at 7:47 AM, Heikki Linnakangas wrote: 1. The first patch changes the way we store the logical tapes on disk. Instead of using indirect blocks, HFS style, the blocks form a linked list. Each block has a trailer,

[HACKERS] Re: [COMMITTERS] pgsql: Extend framework from commit 53be0b1ad to report latch waits.

2016-10-04 Thread Robert Haas
On Tue, Oct 4, 2016 at 11:29 AM, Tom Lane wrote: > Robert Haas writes: >> Apparently, 'make world' does not build worker_spi. I thought 'make >> world' was supposed to build everything? > > You'd have thunk, yeah. It looks like the issue is that

Re: [HACKERS] Misidentification of Python shared library

2016-10-04 Thread Robert Haas
On Tue, Oct 4, 2016 at 11:45 AM, Tom Lane wrote: > "$python_libdir /usr/lib64 /usr/lib" (too bad there's no easy > platform-independent way to find out the linker's default search path). "too bad" sounds like a grave understatement of the perils involved using a fixed list of

Re: [HACKERS] pgbench more operators & functions

2016-10-04 Thread Robert Haas
On Sun, Oct 2, 2016 at 9:41 AM, Michael Paquier wrote: > On Sat, Oct 1, 2016 at 11:35 PM, Fabien COELHO wrote: >> Attached version changes: >> - removes C operators not present in psql >> - document operators one per line > > Moved to next CF

Re: [HACKERS] pgbench more operators & functions

2016-10-04 Thread Fabien COELHO
Hello Stephen, I'm pretty sure we should hold off on adding 'xor' [...] So I removed "xor" in the attached version. [...] I certainly understand how the if() function could be useful Indeed, some kind of "if" is needed, for instance to implement "tpc-b" correctly. That's an interesting

[HACKERS] Transactions involving multiple postgres foreign servers

2016-10-04 Thread Masahiko Sawada
On Tue, Oct 4, 2016 at 1:26 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com > wrote: >>> >>> Why always rollback any dangling transaction? There can be a case that >>> a foreign server has a dangling transaction which needs to be >>> committed because the portions of that

Re: [HACKERS] pg_basebackup stream xlog to tar

2016-10-04 Thread Michael Paquier
(Squashing two emails into one) On Fri, Sep 30, 2016 at 11:16 PM, Magnus Hagander wrote: > And here's yet another version, now rebased on top of the fsync and nosync > changes that got applied. My fault :p > In particular, this conflicted with pretty much every single

Re: [HACKERS] proposal: psql \setfileref

2016-10-04 Thread Gilles Darold
Le 03/10/2016 à 23:23, Gilles Darold a écrit : > Le 03/10/2016 à 23:03, Robert Haas a écrit : >> On Mon, Oct 3, 2016 at 3:54 PM, Gilles Darold wrote: >>> 4) An other problem is that like this this patch will allow anyone to >>> upload into a >>> column the content of any

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2016-10-04 Thread Michael Paquier
On Mon, Oct 3, 2016 at 11:52 PM, Daniel Verite wrote: > Wouldn't pgbench benefit from it? > It was mentioned some time ago [1], in relationship to the > \into construct, how client-server latency was important enough to > justify the use of a "\;" separator between

  1   2   >