Re: [HACKERS] Whether to back-patch fix for aggregate transtype width estimates

On Fri, Jun 17, 2016 at 10:41 PM, Tom Lane wrote: > While fixing the recent unpleasantness over parallel polymorphic > aggregates, I discovered that count_agg_clauses_walker's consideration > of an aggregate argument's typmod in estimating transition space > consumption has

Re: [HACKERS] 10.0

On 06/17/2016 10:04 AM, Alvaro Herrera wrote: > Merlin Moncure wrote: > >> Ugliness is a highly subjective qualifier. OTOH, Backwards >> compatibility, at least when the checks are properly written :-), is a >> very objective benefit. > > This is the argument that made us kept the PostgreSQL

Re: [HACKERS] New design for FK-based join selectivity estimation

Hi, A few more comments, about re-reading the patch more thoroughly. I wouldn't say any of those qualify as bugs, but rather as discussion about some of the design choices: 1) NULL handling I'd argue that we should do something about this, although I agree it's non-trivial to estimate - at

[HACKERS] Assert(LWLockHeldByMeInMode(lock, LW_EXCLUSIVE))

Hi hackers, Several times now when reading, debugging and writing code I've wished that LWLockHeldByMe assertions specified the expected mode, especially where exclusive locking is required. What do you think about something like the attached? See also an example of use. I will add this to the

[HACKERS] Whether to back-patch fix for aggregate transtype width estimates

While fixing the recent unpleasantness over parallel polymorphic aggregates, I discovered that count_agg_clauses_walker's consideration of an aggregate argument's typmod in estimating transition space consumption has been broken since commit 34d26872e (which changed Aggref.args from a simple

Re: [HACKERS] MultiXactId error after upgrade to 9.3.4

Alvaro Herrera wrote: > Andrew Gierth wrote: > > Why is the correct rule not "check for and ignore pre-upgrade mxids > > before even trying to fetch members"? > > I propose something like the attached patch, which implements that idea. Here's a backpatch of that to 9.3 and 9.4. I tested this

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

On Sat, Jun 18, 2016 at 3:43 AM, Tom Lane wrote: > DSM already exists, and for many purposes its lack of a > within-a-shmem-segment dynamic allocator is irrelevant; the same purpose > is served (with more speed, more reliability, and less code) by releasing > the whole DSM

Re: [HACKERS] forcing a rebuild of the visibility map

Since time is short and it seems better to get the xlog page magic bump done before beta2, I've gone ahead and committed this, but there are two points here that seem to warrant some input from other senior hackers. Andres and I have discussed these points off list but without reaching a meeting

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

Aleksey Demakov writes: > On Sat, Jun 18, 2016 at 12:45 AM, Tom Lane wrote: >> You're right, but that doesn't mean that the community is going to take >> much interest in an unportable replacement for code that already exists. > Excuse me, what code

Re: [HACKERS] Parallelized polymorphic aggs, and aggtype vs aggoutputtype

I wrote: > Meanwhile, I'll go see if I can work out exactly what's broken about the > polymorphic type-slinging. That I would like to see working in beta2. OK, I've got at least the core of the problem. fix_combine_agg_expr changes the upper aggregate's Aggref node so that its args list is a

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

On Sat, Jun 18, 2016 at 12:45 AM, Tom Lane wrote: > Aleksey Demakov writes: >> On Fri, Jun 17, 2016 at 10:54 PM, Robert Haas wrote: >>> In my opinion, that's not going to fly. If I thought otherwise, I >>> would not have developed

Re: [HACKERS] Parallelized polymorphic aggs, and aggtype vs aggoutputtype

Tom Lane wrote: > Alvaro Herrera writes: > > Can't we have them be ExtensibleNode? > > No, these are data types and values of data types, not parsetree nodes. Ah, right. > The core problem is to teach the type system to prevent you from > sticking foo(internal) into

Re: [HACKERS] Should phraseto_tsquery('simple', 'blue blue') @@ to_tsvector('simple', 'blue') be true ?

On Fri, Jun 17, 2016 at 11:07 AM, Teodor Sigaev wrote: >>> Such algorithm finds closest pair of (Lpos, Rpos) but satisfying pair >>> could be >>> not closest, example: to_tsvector('simple', '1 2 1 2') @@ '1 <3> 2'; >> >> >> Oh ... the indexes in the lists don't have much to do

Re: [HACKERS] Parallelized polymorphic aggs, and aggtype vs aggoutputtype

Alvaro Herrera writes: > Tom Lane wrote: >> Robert Haas writes: >>> I think we should break up internal into various kinds of internal >>> depending on what kind of a thing we've actually got a pointer to. >> Not a bad long-term project, but it's

Re: [HACKERS] Parallelized polymorphic aggs, and aggtype vs aggoutputtype

On Fri, Jun 17, 2016 at 3:14 PM, Tom Lane wrote: > The concern I have is that you could stick it into an aggregate that isn't > the one it's expecting to be used in, or into a slot in that aggregate > that isn't deserialize(), and the run-time test can't detect either of >

Re: [HACKERS] Parallelized polymorphic aggs, and aggtype vs aggoutputtype

Tom Lane wrote: > Robert Haas writes: > > I think we should break up internal into various kinds of internal > > depending on what kind of a thing we've actually got a pointer to. > > Not a bad long-term project, but it's not happening in this cycle. > I'm not very sure

Re: [HACKERS] Parallelized polymorphic aggs, and aggtype vs aggoutputtype

Robert Haas writes: > On Fri, Jun 17, 2016 at 1:45 PM, Tom Lane wrote: >> there's a far bigger problem which is that "deserialize(bytea) returns >> internal" is a blatant security hole, which I see you ignored the defense >> against in opr_sanity.sql.

Re: [HACKERS] Parallel safety tagging of extension functions

On Thu, Jun 16, 2016 at 9:15 AM, Andreas Karlsson wrote: >> That was the only clear mistake I found, but I tend >> to think that changing the markings on anything defined by >> UNSUPPORTED_FUNCTION() is pretty silly, because there's no point in >> going to extra planner effort

Re: [HACKERS] forcing a rebuild of the visibility map

On Fri, Jun 17, 2016 at 2:48 PM, Andres Freund wrote: >> From 010e99b403ec733d50c71a7d4ef646b1b446ef07 Mon Sep 17 00:00:00 2001 >> From: Robert Haas >> Date: Wed, 15 Jun 2016 22:52:58 -0400 >> Subject: [PATCH 2/2] Add VACUUM (DISABLE_PAGE_SKIPPING) for

Re: [HACKERS] forcing a rebuild of the visibility map

Hi, On 2016-06-15 23:06:37 -0400, Robert Haas wrote: > After having been scared out of my mind by the discovery of > longstanding breakage in heap_update[1], it occurred to me that this > is an excellent example of a case in which the option for which Andres > was agitating - specifically

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

Aleksey Demakov writes: > On Fri, Jun 17, 2016 at 10:54 PM, Robert Haas wrote: >> On Fri, Jun 17, 2016 at 12:34 PM, Aleksey Demakov wrote: >>> I believe it would be perfectly okay to allocate huge amount of address >>> space with

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

On Fri, Jun 17, 2016 at 2:23 PM, Aleksey Demakov wrote: > Essentially this is pessimizing for the lowest common denominator > among OSes. I totally agree. That's how we make the server portable. > Having a contiguous address space makes things so > much simpler that

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

On Fri, Jun 17, 2016 at 2:23 PM, Aleksey Demakov wrote: > On Fri, Jun 17, 2016 at 10:54 PM, Robert Haas > wrote: > > On Fri, Jun 17, 2016 at 12:34 PM, Aleksey Demakov > wrote: > >>> I expect that to be useful for parallel query and

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

On Sat, Jun 18, 2016 at 12:31 AM, Andres Freund wrote: > On 2016-06-18 00:23:14 +0600, Aleksey Demakov wrote: >> Finally, it's possible to repeatedly mmap >> and munmap on portions of a contiguous address space providing >> a given addr argument for both of them. The last

Re: [HACKERS] Parallelized polymorphic aggs, and aggtype vs aggoutputtype

On Fri, Jun 17, 2016 at 1:45 PM, Tom Lane wrote: > I wrote: >> Robert Haas writes: >>> I don't mind if you feel the need to refactor this. > >> I'm not sure yet. What I think we need to work out first is exactly >> how polymorphic parallel aggregates

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

Sorry for unclear language. Late Friday evening in my place is to blame. On Sat, Jun 18, 2016 at 12:23 AM, Aleksey Demakov wrote: > On Fri, Jun 17, 2016 at 10:54 PM, Robert Haas wrote: >> On Fri, Jun 17, 2016 at 12:34 PM, Aleksey Demakov

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

On 2016-06-18 00:23:14 +0600, Aleksey Demakov wrote: > Finally, it's possible to repeatedly mmap > and munmap on portions of a contiguous address space providing > a given addr argument for both of them. The last option might, of > course, is susceptible to hijacking this portion of the address by

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

On Fri, Jun 17, 2016 at 10:54 PM, Robert Haas wrote: > On Fri, Jun 17, 2016 at 12:34 PM, Aleksey Demakov wrote: >>> I expect that to be useful for parallel query and anything else where >>> processes need to share variable-size data. However, that's

Re: [HACKERS] Parallelized polymorphic aggs, and aggtype vs aggoutputtype

I wrote: > Robert Haas writes: >> I don't mind if you feel the need to refactor this. > I'm not sure yet. What I think we need to work out first is exactly > how polymorphic parallel aggregates ought to identify which concrete > data types they are using. There were

Re: [HACKERS] MultiXactId error after upgrade to 9.3.4

Andrew Gierth wrote: > > "Alvaro" == Alvaro Herrera writes: > > >> (It can, AFAICT, be inside the currently valid range due to > >> wraparound, i.e. without there being a valid pg_multixact entry for > >> it, because AFAICT in 9.2, once the mxid is hinted dead it

Re: [HACKERS] 10.0

On Fri, Jun 17, 2016 at 1:04 PM, Alvaro Herrera wrote: > Merlin Moncure wrote: > >> Ugliness is a highly subjective qualifier. OTOH, Backwards >> compatibility, at least when the checks are properly written :-), is a >> very objective benefit. > > This is the argument

Re: [HACKERS] 10.0

Merlin Moncure wrote: > Ugliness is a highly subjective qualifier. OTOH, Backwards > compatibility, at least when the checks are properly written :-), is a > very objective benefit. This is the argument that made us kept the PostgreSQL name instead of renaming back to Postgres. I'm not a fan

Re: [HACKERS] Typo in parallel comment of heap_delete()

On Thu, Jun 16, 2016 at 12:32 AM, Thomas Munro wrote: > On Tue, Jun 7, 2016 at 12:00 AM, Robert Haas wrote: >> On Sun, Jun 5, 2016 at 4:39 PM, Jim Nasby wrote: >>> I'm pretty sure this is a typo... >> >> Sure is.

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

On Fri, Jun 17, 2016 at 12:34 PM, Aleksey Demakov wrote: >> I expect that to be useful for parallel query and anything else where >> processes need to share variable-size data. However, that's different >> from this because ours can grown to arbitrary size and shrink again by

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

On Fri, Jun 17, 2016 at 10:18 PM, Robert Haas wrote: > On Fri, Jun 17, 2016 at 11:30 AM, Tom Lane wrote: > But I'm a bit confused about where it gets the bytes it wants to > manage. There's no call to dsm_create() or ShmemAlloc() anywhere in > the

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

On Fri, Jun 17, 2016 at 9:29 AM, Robert Haas wrote: > On Fri, Jun 17, 2016 at 9:04 AM, Amit Kapila wrote: >> Attached, please find updated patch based on above lines. Due to addition >> of projection path on top of partial paths, some small

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

On Fri, Jun 17, 2016 at 9:30 PM, Tom Lane wrote: > Aleksey Demakov writes: >> I have some very experimental code to enable dynamic memory allocation >> of shared memory for postgresql backend processes. > > Um ... what's this do that the existing DSM stuff

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

On Fri, Jun 17, 2016 at 11:30 AM, Tom Lane wrote: > Aleksey Demakov writes: >> I have some very experimental code to enable dynamic memory allocation >> of shared memory for postgresql backend processes. > > Um ... what's this do that the existing DSM

Re: [HACKERS] Experimental dynamic memory allocation of postgresql shared memory

Aleksey Demakov writes: > I have some very experimental code to enable dynamic memory allocation > of shared memory for postgresql backend processes. Um ... what's this do that the existing DSM stuff doesn't do? regards, tom lane -- Sent via

[HACKERS] Experimental dynamic memory allocation of postgresql shared memory

Hi all, I have some very experimental code to enable dynamic memory allocation of shared memory for postgresql backend processes. The source code in the repository is not complete yet. Moreover it is not immediately useful by itself. However it might serve as the basis to implement higher-level

Re: [HACKERS] Restriction of windows functions

Konstantin Knizhnik writes: > On 17.06.2016 17:01, Tom Lane wrote: >> IIRC, the sticking point was defining a reasonably datatype-independent >> (i.e. extensible) notion of distance. > Why it is not possible just to locate "-" or "+ operator for this type? Because

Re: [HACKERS] Should phraseto_tsquery('simple', 'blue blue') @@ to_tsvector('simple', 'blue') be true ?

Such algorithm finds closest pair of (Lpos, Rpos) but satisfying pair could be not closest, example: to_tsvector('simple', '1 2 1 2') @@ '1 <3> 2'; Oh ... the indexes in the lists don't have much to do with the distances, do they. OK, maybe it's not quite as easy as I was thinking. I'm okay

Re: [HACKERS] New design for FK-based join selectivity estimation

Tomas Vondra writes: > Thanks for getting the patch into a much better shape. I've quickly > reviewed the patch this morning before leaving to the airport - I do > plan to do additional review/testing once in the air or perhaps over the > weekend. So at the moment

Re: [HACKERS] Restriction of windows functions

On 17.06.2016 17:01, Tom Lane wrote: Certainly it assumes that window is ordered by key and the key type supports subtraction, so "text" can not be used here. IIRC, the sticking point was defining a reasonably datatype-independent (i.e. extensible) notion of distance. Why it is not possible

Re: [HACKERS] Should phraseto_tsquery('simple', 'blue blue') @@ to_tsvector('simple', 'blue') be true ?

Teodor Sigaev writes: > Tom Lane wrote: >> Hmm, couldn't the loop logic be simplified a great deal if this is the >> definition? Or are you leaving it like that with the idea that we might >> later introduce another operator with the less-than-or-equal behavior? > Do you

Re: [HACKERS] Should phraseto_tsquery('simple', 'blue blue') @@ to_tsvector('simple', 'blue') be true ?

Tom Lane wrote: Teodor Sigaev writes: So I think there's a reasonable case for decreeing that should only match lexemes *exactly* N apart. If we did that, we would no longer have the misbehavior that Jean-Pierre is complaining about, and we'd not need to argue about

Re: [HACKERS] Restriction of windows functions

Konstantin Knizhnik writes: > ERROR: RANGE PRECEDING is only supported with UNBOUNDED > Is there some principle problem in implementing such kind of window? There was such code in the original windowagg patch and it got rejected as being too broken. Please consult

Re: [HACKERS] MultiXactId error after upgrade to 9.3.4

> "Robert" == Robert Haas writes: >> Why is the correct rule not "check for and ignore pre-upgrade mxids >> before even trying to fetch members"? Robert> I entirely believe that's the correct rule, but doesn't Robert> implementing it require a crystal balll? Why

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

On Fri, Jun 17, 2016 at 9:04 AM, Amit Kapila wrote: > Attached, please find updated patch based on above lines. Due to addition > of projection path on top of partial paths, some small additional cost is > added for parallel paths. In one of the tests in

Re: [HACKERS] parallel.c is not marked as test covered

On Wed, Jun 15, 2016 at 5:10 PM, Robert Haas wrote: > I'm comfortable with that. Feel free to make it so, unless you want > me to do it for some reason. Time is short, so I did this. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL

Re: [HACKERS] 10.0

On Fri, Jun 17, 2016 at 1:01 AM, Craig Ringer wrote: > On 17 June 2016 at 08:34, Greg Stark wrote: >> >> So we would release 10.0.0 and 10.0.1 and the next major release would be >> 11.0.0. >> >> This would have two benefits: >> >> 1) It emphasises that

Re: [HACKERS] 10.0

On Fri, Jun 17, 2016 at 2:01 AM, Craig Ringer wrote: > On 17 June 2016 at 08:34, Greg Stark wrote: > >> So we would release 10.0.0 and 10.0.1 and the next major release would be >> 11.0.0. >> >> This would have two benefits: >> >> 1) It emphasises that

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

On Fri, Jun 17, 2016 at 8:18 AM, Amit Kapila wrote: > > On Fri, Jun 17, 2016 at 3:20 AM, Robert Haas wrote: > > > > > Something like what you have there might work if you use > > create_projection_path instead of apply_projection_to_path. > >

Re: [HACKERS] [sqlsmith] Failed assertion in postgres_fdw/deparse.c:1116

On Thu, Jun 16, 2016 at 10:44 PM, Etsuro Fujita wrote: > On 2016/06/16 22:00, Robert Haas wrote: >> On Thu, Jun 16, 2016 at 7:05 AM, Etsuro Fujita >> wrote: >>> >>> ISTM that a robuster solution to this is to push down the ft1-ft2-ft3 >>>

Re: [HACKERS] MultiXactId error after upgrade to 9.3.4

On Thu, Jun 16, 2016 at 4:50 AM, Andrew Gierth wrote: > Why is the correct rule not "check for and ignore pre-upgrade mxids > before even trying to fetch members"? I entirely believe that's the correct rule, but doesn't implementing it require a crystal balll? --

[HACKERS] Restriction of windows functions

Hi, One of the popular queries in financial analytic systems is to calculate some moving aggregate within some time interval, i.e. moving average of trade price within 5 minutes window. Unfortunately this case is not supported by PostgreSQL: select symbol,date,avg(price) over (order by date

Re: [HACKERS] MultiXactId error after upgrade to 9.3.4

> "Alvaro" == Alvaro Herrera writes: >> (It can, AFAICT, be inside the currently valid range due to >> wraparound, i.e. without there being a valid pg_multixact entry for >> it, because AFAICT in 9.2, once the mxid is hinted dead it is never >> again either

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

På fredag 17. juni 2016 kl. 08:14:39, skrev Amit Kapila >: On Fri, Jun 17, 2016 at 11:39 AM, Andreas Joseph Krogh > wrote: På torsdag 16. juni 2016 kl. 20:19:44, skrev Tom Lane

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

On Fri, Jun 17, 2016 at 11:39 AM, Andreas Joseph Krogh wrote: > På torsdag 16. juni 2016 kl. 20:19:44, skrev Tom Lane : > > Amit Kapila writes: > > On Mon, Jun 13, 2016 at 10:36 PM, Tom Lane wrote: > >>

Re: [HACKERS] ERROR: ORDER/GROUP BY expression not found in targetlist

På torsdag 16. juni 2016 kl. 20:19:44, skrev Tom Lane >: Amit Kapila writes: > On Mon, Jun 13, 2016 at 10:36 PM, Tom Lane wrote: >> min_parallel_relation_size, or min_parallelizable_relation_size,

Re: [HACKERS] pg_isready features

On 16 June 2016 at 00:40, Jimmy wrote: > > (1) I can (and do) use psql, pg_isready seems lighter and since it already > supports credentials and DB name, I see very little harm for pg_isready > to say back whether user logged IN or not using these credentials. > > > (2)

Re: [HACKERS] 10.0

On 17 June 2016 at 08:34, Greg Stark wrote: > So we would release 10.0.0 and 10.0.1 and the next major release would be > 11.0.0. > > This would have two benefits: > > 1) It emphasises that minor releases continue to be safe minor updates > that offer the same stability