Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Joshua D. Drake
On 08/10/2016 08:28 AM, Robert Haas wrote: On Wed, Aug 10, 2016 at 11:09 AM, Heikki Linnakangas wrote: Still, having to invent CSNs seems like a huge loss for this design. Personally I'd give up async commit first. If we had only sync commit, the rule could be "xact LSN less

Re: [HACKERS] Is there a way around function search_path killing SQL function inlining?

2016-08-10 Thread Tom Lane
Michael Banck writes: > As I've been bitten by this problem recently, I thought I'd take a look > at editing the PostGIS extension SQL file to this end, but contrary to > the above, the @extschema@ feature only applies to non-relocatable > extensions, from

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Joshua D. Drake
On 08/10/2016 09:04 AM, Stephen Frost wrote: * Joshua D. Drake (j...@commandprompt.com) wrote: +1 for Robert here, removing async commit is a non-starter. It is PostgreSQL performance 101 that you disable synchronous commit unless you have a specific data/business requirement that needs it.

Re: [HACKERS] Slowness of extended protocol

2016-08-10 Thread Vladimir Sitnikov
Robert>But that makes it the job of every driver to implement some sort of cache, which IMHO isn't a very reasonable position Let's wait what Shay decides on implementing query cache in npgsql ? Here's the issue: https://github.com/npgsql/npgsql/issues/1237 He could change his mind when it

Re: [HACKERS] No longer possible to query catalogs for index capabilities?

2016-08-10 Thread Andrew Gierth
> "Tom" == Tom Lane writes: >> - this still has everything in amapi.c rather than creating any new >> files. Also, the regression tests are in create_index.sql for lack >> of any obviously better place. Tom> This more than doubles the size of amapi.c, so it has a

Re: [HACKERS] Is there a way around function search_path killing SQL function inlining?

2016-08-10 Thread Michael Banck
On Thu, Mar 10, 2016 at 11:48:41AM -0500, Tom Lane wrote: > If you're worried about preserving relocatability of an extension > containing such functions, the @extschema@ feature might help. As I've been bitten by this problem recently, I thought I'd take a look at editing the PostGIS extension

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Stephen Frost
* Joshua D. Drake (j...@commandprompt.com) wrote: > +1 for Robert here, removing async commit is a non-starter. It is > PostgreSQL performance 101 that you disable synchronous commit > unless you have a specific data/business requirement that needs it. > Specifically because of how much faster Pg

Re: [HACKERS] Assertion failure in REL9_5_STABLE

2016-08-10 Thread Tom Lane
Marko Tiikkaja writes: > The failure is a number of levels down a call stack of PL/PgSQL > functions, but I can reproduce it at will by calling the function. I > unfortunately can't narrow it down to a smaller test case, but attached > is an xlogdump of the session. The query

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Alexander Korotkov
On Wed, Aug 10, 2016 at 6:09 PM, Heikki Linnakangas wrote: > On 08/10/2016 05:51 PM, Tom Lane wrote: > >> Heikki Linnakangas writes: >> >>> On 08/10/2016 05:09 PM, Tom Lane wrote: >>> Uh, what? That's not the semantics we have today, and I don't see why

Re: [HACKERS] Assertion failure in REL9_5_STABLE

2016-08-10 Thread Alvaro Herrera
Marko Tiikkaja wrote: > Hi, > > Running one specific test from our application against REL9_5_STABLE > (current as of today) gives me this: > > #2 0x7effb59595be in ExceptionalCondition ( > conditionName=conditionName@entry=0x7effb5b27a88 "!(CritSectionCount > 0 > ||

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Ants Aasma
On Wed, Aug 10, 2016 at 7:54 PM, Alexander Korotkov wrote: > Oh, I found that I underestimated complexity of async commit... :) > > Do I understand right that now async commit right as follows? > 1) Async transaction confirms commit before flushing WAL. > 2) Other

Re: [HACKERS] Slowness of extended protocol

2016-08-10 Thread Tom Lane
Robert Haas writes: > Sure, but I don't want the application to have to know about that, and > I don't really think the driver should need to know about that either. > Your point, as I understand it, is that sufficiently good query > caching in the driver can ameliorate the

Re: [HACKERS] Slowness of extended protocol

2016-08-10 Thread Stephen Frost
* Vladimir Sitnikov (sitnikov.vladi...@gmail.com) wrote: > It works completely transparent to the application, and it does use > server-prepared statements even though application builds "brand new" sql > text every time. And is the source of frequent complaints on various mailing lists along the

Re: [HACKERS] Slowness of extended protocol

2016-08-10 Thread Robert Haas
On Tue, Aug 9, 2016 at 5:07 PM, Vladimir Sitnikov wrote: > I do not buy that "dynamically generated queries defeat server-prepared > usage" argument. It is just not true (see below). > > Do you mean "in language X, where X != Java it is impossible to implement a >

Re: [HACKERS] No longer possible to query catalogs for index capabilities?

2016-08-10 Thread Tom Lane
Andrew Gierth writes: > - this still has everything in amapi.c rather than creating any new > files. Also, the regression tests are in create_index.sql for lack > of any obviously better place. This more than doubles the size of amapi.c, so it has a

Re: [HACKERS] Slowness of extended protocol

2016-08-10 Thread Robert Haas
On Wed, Aug 10, 2016 at 11:50 AM, Tom Lane wrote: > Robert Haas writes: >> Sure, but I don't want the application to have to know about that, and >> I don't really think the driver should need to know about that either. >> Your point, as I understand

[HACKERS] Assertion failure in REL9_5_STABLE

2016-08-10 Thread Marko Tiikkaja
Hi, Running one specific test from our application against REL9_5_STABLE (current as of today) gives me this: #2 0x7effb59595be in ExceptionalCondition ( conditionName=conditionName@entry=0x7effb5b27a88 "!(CritSectionCount > 0 || TransactionIdIsCurrentTransactionId((

Re: [HACKERS] Slowness of extended protocol

2016-08-10 Thread Vladimir Sitnikov
Stephen>I encourage you to look through the archives The thing is pl/pgsql suffers from exactly the same problem. pl/pgsql is not a typical language of choice (e.g. see Tiobe index and alike), so the probability of running into "prepared statement issues" was low. As more languages would use

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Heikki Linnakangas
On 08/10/2016 07:54 PM, Alexander Korotkov wrote: Do I understand right that now async commit right as follows? 1) Async transaction confirms commit before flushing WAL. Yes. 2) Other transactions sees effect of async transaction only when its WAL flushed. No. Other transactions also see

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Robert Haas
On Wed, Aug 10, 2016 at 11:09 AM, Heikki Linnakangas wrote: >> Still, having to invent CSNs seems like a huge loss for this design. >> Personally I'd give up async commit first. If we had only sync commit, >> the rule could be "xact LSN less than snapshot threshold and less than

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Ants Aasma
On Wed, Aug 10, 2016 at 6:09 PM, Heikki Linnakangas wrote: > Hmm. There's one more possible way this could all work. Let's have CSN == > LSN, also for asynchronous commits. A snapshot is the current insert > position, but also make note of the current flush position, when you

Re: [HACKERS] Slowness of extended protocol

2016-08-10 Thread Stephen Frost
* Vladimir Sitnikov (sitnikov.vladi...@gmail.com) wrote: > 3) "suddently get slow the 6th time" is a PostgreSQL bug that both fails to > estimate cardinality properly, and it does not provide administrator a way > to disable the feature (generic vs specific plan). Dropping and recreating the

Re: [HACKERS] Slowness of extended protocol

2016-08-10 Thread Vladimir Sitnikov
Stephen> While it may have good results in many cases, it's not accurate to say that using prepared statements will always be faster than not. There's no silver bullet. <-- that is accurate, but it is useless for end-user applications I've never claimed that "server prepared statement" is a

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Greg Stark
On Wed, Aug 10, 2016 at 5:54 PM, Alexander Korotkov wrote: > Oh, I found that I underestimated complexity of async commit... :) Indeed. I think Tom's attitude was right even if the specific conclusion was wrong. While I don't think removing async commit is viable I

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Alexander Korotkov
On Wed, Aug 10, 2016 at 2:10 PM, Heikki Linnakangas wrote: > Yeah, if the csnlog access turns out to be too expensive, we could do > something like this. In theory, you can always convert a CSN snapshot into > an old-style list of XIDs, by scanning the csnlog between the xmin

Re: [HACKERS] Wait events monitoring future development

2016-08-10 Thread Bruce Momjian
On Wed, Aug 10, 2016 at 11:37:36PM +0900, Satoshi Nagayasu wrote: > Agreed. > > If people are facing with some difficult situation in terms of performance, > they may accept some (one-time) overhead to resolve the issue. > But if they don't have (recognize) any issue, they may not. > > That's

Re: [HACKERS] Wait events monitoring future development

2016-08-10 Thread Bruce Momjian
On Wed, Aug 10, 2016 at 05:14:52PM +0300, Alexander Korotkov wrote: > On Tue, Aug 9, 2016 at 5:37 AM, Bruce Momjian wrote: > > On Tue, Aug  9, 2016 at 02:06:40AM +, Tsunakawa, Takayuki wrote: > > I hope wait event monitoring will be on by default even if the

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Tom Lane
Heikki Linnakangas writes: > Imagine that you have a stream of normal, synchronous, commits. They get > assigned LSNs: 1, 2, 3, 4. They become visible to other transactions in > that order. > The way I described this scheme in the first emails on this thread, was > to use the

Re: [HACKERS] Wait events monitoring future development

2016-08-10 Thread Satoshi Nagayasu
2016/08/10 23:22 "Bruce Momjian" : > > On Wed, Aug 10, 2016 at 05:14:52PM +0300, Alexander Korotkov wrote: > > On Tue, Aug 9, 2016 at 5:37 AM, Bruce Momjian wrote: > > > > On Tue, Aug 9, 2016 at 02:06:40AM +, Tsunakawa, Takayuki wrote: > > > I hope

Re: [HACKERS] multivariate statistics (v19)

2016-08-10 Thread Ants Aasma
On Wed, Aug 3, 2016 at 4:58 AM, Tomas Vondra wrote: > 2) combining multiple statistics > > I think the ability to combine multivariate statistics (covering different > subsets of conditions) is important and useful, but I'm starting to think > that the current

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Heikki Linnakangas
On 08/10/2016 04:34 PM, Alexander Korotkov wrote: On Tue, Aug 9, 2016 at 3:16 PM, Heikki Linnakangas wrote: I switched to using a separate counter for CSNs. CSN is no longer the same as the commit WAL record's LSN. While I liked the conceptual simplicity of CSN == LSN a lot,

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Heikki Linnakangas
On 08/10/2016 05:09 PM, Tom Lane wrote: Heikki Linnakangas writes: Imagine that you have a stream of normal, synchronous, commits. They get assigned LSNs: 1, 2, 3, 4. They become visible to other transactions in that order. The way I described this scheme in the first

Re: [HACKERS] Curing plpgsql's memory leaks for statement-lifespan values

2016-08-10 Thread Pavel Stehule
2016-08-10 11:25 GMT+02:00 Pavel Stehule : > Hi > > 2016-07-27 16:49 GMT+02:00 Tom Lane : > >> Robert Haas writes: >> > On Mon, Jul 25, 2016 at 6:04 PM, Tom Lane wrote: >> >> I suppose that a fix based on

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Alexander Korotkov
On Tue, Aug 9, 2016 at 3:16 PM, Heikki Linnakangas wrote: > (Reviving an old thread) > > I spent some time dusting off this old patch, to implement CSN snapshots. > Attached is a new patch, rebased over current master, and with tons of > comments etc. cleaned up. There's more

Re: [HACKERS] Wait events monitoring future development

2016-08-10 Thread Alexander Korotkov
On Tue, Aug 9, 2016 at 5:37 AM, Bruce Momjian wrote: > On Tue, Aug 9, 2016 at 02:06:40AM +, Tsunakawa, Takayuki wrote: > > I hope wait event monitoring will be on by default even if the overhead > is not > > almost zero, because the data needs to be readily available for

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Heikki Linnakangas
On 08/10/2016 05:51 PM, Tom Lane wrote: Heikki Linnakangas writes: On 08/10/2016 05:09 PM, Tom Lane wrote: Uh, what? That's not the semantics we have today, and I don't see why it's necessary or a good idea. Once the commit is in the WAL stream, any action taken on the

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Tom Lane
Heikki Linnakangas writes: > On 08/10/2016 05:09 PM, Tom Lane wrote: >> Uh, what? That's not the semantics we have today, and I don't see why >> it's necessary or a good idea. Once the commit is in the WAL stream, >> any action taken on the basis of seeing the commit must be

Re: [HACKERS] No longer possible to query catalogs for index capabilities?

2016-08-10 Thread Andrew Gierth
Updated patch. Changes: - returns NULL rather than "cache lookup failed" - added pg_index_column_has_property (incl. docs) - added regression tests Not changed / need consideration: - this still has everything in amapi.c rather than creating any new files. Also, the regression

Re: [HACKERS] Wait events monitoring future development

2016-08-10 Thread Alexander Korotkov
On Tue, Aug 9, 2016 at 12:47 AM, Ilya Kosmodemiansky < ilya.kosmodemian...@postgresql-consulting.com> wrote: > On Mon, Aug 8, 2016 at 7:03 PM, Bruce Momjian wrote: > > It seems asking users to run pg_test_timing before deploying to check > > the overhead would be sufficient. >

Re: [HACKERS] Set log_line_prefix and application name in test drivers

2016-08-10 Thread Peter Eisentraut
On 8/10/16 5:18 PM, Tom Lane wrote: > Or in short: I don't want to be seeing one prefix format in some buildfarm > logs and a different format in others. Sure. My patch has log_line_prefix = '%t [%p]: [%l] %qapp=%a ' which is modeled after the pgfouine recommendation, which is I believe a

Re: Improved ICU patch - WAS: [HACKERS] Implementing full UTF-8 support (aka supporting 0x00)

2016-08-10 Thread Peter Geoghegan
On Wed, Aug 10, 2016 at 1:42 PM, Palle Girgensohn wrote: > They've been used for the FreeBSD ports since 2005, and have served us well. > I have of course updated them regularly. In this latest version, I've removed > support for other encodings beside UTF-8, mostly since I

Re: [HACKERS] Is there a way around function search_path killing SQL function inlining?

2016-08-10 Thread Regina Obe
> Michael Banck writes: >> As I've been bitten by this problem recently, I thought I'd take a >> look at editing the PostGIS extension SQL file to this end, but >> contrary to the above, the @extschema@ feature only applies to >> non-relocatable extensions, from

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Alexander Korotkov
On Wed, Aug 10, 2016 at 8:26 PM, Greg Stark wrote: > On Wed, Aug 10, 2016 at 5:54 PM, Alexander Korotkov > wrote: > > Oh, I found that I underestimated complexity of async commit... :) > > Indeed. I think Tom's attitude was right even if the specific >

Re: [HACKERS] No longer possible to query catalogs for index capabilities?

2016-08-10 Thread Kevin Grittner
On Wed, Aug 10, 2016 at 12:31 PM, Andrew Gierth wrote: > These could be limited to being testable only on a specified index, and > not AM-wide: > predicate_locks (? maybe) That one seems like it should either be at the AM level or not included at all. Where it

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-10 Thread Claudio Freire
On Tue, Aug 9, 2016 at 11:39 PM, Jim Nasby wrote: > On 8/9/16 6:44 PM, Claudio Freire wrote: >> >> Since we can lookup all occurrences of k1=a index=0 and k2=a index=0, >> and in fact we probably did so already as part of the update logic > > > That's a change from what

Re: [HACKERS] multivariate statistics (v19)

2016-08-10 Thread Tomas Vondra
On 08/10/2016 02:24 PM, Michael Paquier wrote: On Wed, Aug 10, 2016 at 8:50 PM, Petr Jelinek wrote: On 10/08/16 13:33, Tomas Vondra wrote: On 08/10/2016 06:41 AM, Michael Paquier wrote: On Wed, Aug 3, 2016 at 10:58 AM, Tomas Vondra 2) combining multiple statistics

Re: [HACKERS] pg_ctl promote wait

2016-08-10 Thread Peter Eisentraut
On 8/7/16 9:44 PM, Michael Paquier wrote: >>> This is not a good >>> >> idea, and the idea of putting a wait argument in get_controlfile does >>> >> not seem a good interface to me. I'd rather see get_controlfile be >>> >> extended with a flag saying no_error_on_failure and keep the wait >>> >>

Re: [HACKERS] Set log_line_prefix and application name in test drivers

2016-08-10 Thread Peter Eisentraut
On 8/9/16 12:16 PM, Tom Lane wrote: > Peter Eisentraut writes: >> > Here is a small patch that sets log_line_prefix and application name in >> > pg_regress and the TAP tests, to make analyzing the server log output >> > easier. > How would this interact with the

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Kevin Grittner
On Wed, Aug 10, 2016 at 12:26 PM, Greg Stark wrote: > Complexity like this makes it hard to implement other features such as > CSNs. IIRC this already bit hot standby as well. I think it would be a > big improvement if we had a clear, well defined commit order that was > easy to

Re: [HACKERS] multivariate statistics (v19)

2016-08-10 Thread Tomas Vondra
On 08/10/2016 02:23 PM, Michael Paquier wrote: On Wed, Aug 10, 2016 at 8:33 PM, Tomas Vondra wrote: On 08/10/2016 06:41 AM, Michael Paquier wrote: Patch 0001: there have been comments about that before, and you have put the checks on RestrictInfo in a couple of

Re: [HACKERS] multivariate statistics (v19)

2016-08-10 Thread Tomas Vondra
On 08/10/2016 03:29 PM, Ants Aasma wrote: On Wed, Aug 3, 2016 at 4:58 AM, Tomas Vondra wrote: 2) combining multiple statistics I think the ability to combine multivariate statistics (covering different subsets of conditions) is important and useful, but I'm

Re: [HACKERS] Parallel tuplesort, partitioning, merging, and the future

2016-08-10 Thread Robert Haas
On Mon, Aug 8, 2016 at 3:44 PM, Peter Geoghegan wrote: > I don't think partitioning is urgent for CREATE INDEX, and may be > inappropriate for CREATE INDEX under any circumstances, because: > > * Possible problems with parallel infrastructure and writes. > * Unbalanced B-Trees

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-10 Thread Simon Riggs
On 10 August 2016 at 03:45, Pavan Deolasee wrote: > > > On Tue, Aug 9, 2016 at 12:06 AM, Claudio Freire > wrote: >> >> On Mon, Aug 8, 2016 at 2:52 PM, Pavan Deolasee >> wrote: >> >> > Some heuristics and limits on

Re: [HACKERS] regression test for extended query protocol

2016-08-10 Thread Alvaro Herrera
Michael Paquier wrote: > On Fri, Aug 5, 2016 at 12:21 AM, Alvaro Herrera > wrote: > > If somebody had some spare time to devote to this, I would suggest to > > implement something in core that can be used to specify a list of > > commands to run, and a list of

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-10 Thread Claudio Freire
On Wed, Aug 10, 2016 at 4:37 PM, Simon Riggs wrote: > On 10 August 2016 at 03:45, Pavan Deolasee wrote: >> >> >> On Tue, Aug 9, 2016 at 12:06 AM, Claudio Freire >> wrote: >>> >>> On Mon, Aug 8, 2016 at 2:52 PM, Pavan

[HACKERS] new autovacuum criterion for visible pages

2016-08-10 Thread Jeff Janes
I wanted to create a new relopt named something like autovacuum_vacuum_pagevisible_factor which would cause autovacuum to vacuum a table once less than a certain fraction of the relation's pages are marked allvisible. I wanted some feedback on some things. 1) One issue is that pg_class.relpages

Re: [HACKERS] Wait events monitoring future development

2016-08-10 Thread Robert Haas
On Tue, Aug 9, 2016 at 12:07 AM, Tsunakawa, Takayuki wrote: > As another idea, we can stand on the middle ground. Interestingly, MySQL > also enables their event monitoring (Performance Schema) by default, but not > all events are collected. I guess highly

Re: [HACKERS] Assertion failure in REL9_5_STABLE

2016-08-10 Thread Marko Tiikkaja
On 2016-08-10 19:28, Alvaro Herrera wrote: Umm. AFAICS HeapTupleSatisfiesUpdate() only returns SelfUpdated after already calling HeapTupleHeaderGetCmax() (which obviously hasn't caught the same assertion). Something is odd there ... HeapTupleSatisfiesUpdate() returns HeapTupleBeingUpdated in

Re: [HACKERS] Slowness of extended protocol

2016-08-10 Thread Shay Rojansky
Some comments... For the record, I do find implicit/transparent driver-level query preparation interesting and potentially useful, and have opened https://github.com/npgsql/npgsql/issues/1237 to think about it - mostly based on arguments on this thread. One big question mark I have is whether

[HACKERS] new pgindent run before branch?

2016-08-10 Thread Robert Haas
Hi, Would anyone mind too much if I refreshed typedefs.list and re-indented the whole tree before we branch? There's not too much churn at the moment, and I feel like it would be nice to start the cycle in as clean a state as possible. Current results of this attached. -- Robert Haas

Re: [HACKERS] Slowness of extended protocol

2016-08-10 Thread Vladimir Sitnikov
Shay> it's important to note that query parsing and rewriting isn't an "inevitable evil". Ok, I stay corrected. ADO.NET have raw mode in the API. That's interesting. Let's say "lots of heavily used languages do have their own notion of bind placeholders". And for the reset, it is still not that

Improved ICU patch - WAS: [HACKERS] Implementing full UTF-8 support (aka supporting 0x00)

2016-08-10 Thread Palle Girgensohn
> 4 aug. 2016 kl. 02:40 skrev Bruce Momjian : > > On Thu, Aug 4, 2016 at 08:22:25AM +0800, Craig Ringer wrote: >> Yep, it does. But we've made little to no progress on integration of ICU >> support and AFAIK nobody's working on it right now. > > Uh, this email from July says

Re: [HACKERS] new pgindent run before branch?

2016-08-10 Thread Bruce Momjian
On Wed, Aug 10, 2016 at 04:02:27PM -0400, Robert Haas wrote: > Hi, > > Would anyone mind too much if I refreshed typedefs.list and > re-indented the whole tree before we branch? There's not too much > churn at the moment, and I feel like it would be nice to start the > cycle in as clean a state

Re: [HACKERS] Parallel tuplesort, partitioning, merging, and the future

2016-08-10 Thread Peter Geoghegan
On Wed, Aug 10, 2016 at 11:59 AM, Robert Haas wrote: > My view on this - currently anyway - is that we shouldn't conflate the > tuplesort with the subsequent index generation, but that we should try > to use parallelism within the tuplesort itself to the greatest extent >

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2016-08-10 Thread Michael Paquier
On Fri, Jun 3, 2016 at 8:51 PM, Dmitry Igrishin wrote: >> BTW, I've publushed the HTML-ified SGML docs to >> http://2ndquadrant.github.io/postgres/libpq-batch-mode.html as a preview. > Typo detected: "Returns 1 if the batch curently being received" -- "curently". I am looking

Re: [HACKERS] Wait events monitoring future development

2016-08-10 Thread Craig Ringer
On 10 August 2016 at 07:09, Jim Nasby wrote: > > The downside to leaving stuff like this off by default is users won't > remember it's there when they need it. At best, that means they spend more > time debugging something than they need to. At worse, it means they

Re: [HACKERS] handling unconvertible error messages

2016-08-10 Thread Victor Wagner
On Wed, 10 Aug 2016 11:08:43 +0900 (Tokyo Standard Time) Kyotaro HORIGUCHI wrote: > Hello, > > (I've recovered the lost Cc recipients so far) > > At Mon, 8 Aug 2016 12:52:11 +0300, Victor Wagner > wrote in

Re: [HACKERS] Parallel tuplesort, partitioning, merging, and the future

2016-08-10 Thread Peter Geoghegan
On Wed, Aug 10, 2016 at 12:08 PM, Claudio Freire wrote: > I think it's a great design, but for that, per-worker final tapes have > to always be random-access. Thanks. I don't think I need to live with the randomAccess restriction, because I can be clever about reading

Re: [HACKERS] Set log_line_prefix and application name in test drivers

2016-08-10 Thread Tom Lane
Peter Eisentraut writes: > On 8/9/16 12:16 PM, Tom Lane wrote: >> Peter Eisentraut writes: >>> Here is a small patch that sets log_line_prefix and application name in >>> pg_regress and the TAP tests, to make analyzing the

Re: [HACKERS] per-statement-level INSTEAD OF triggers

2016-08-10 Thread Emre Hasegeli
> It might be more useful after we get the infrastructure that Kevin's been > working on to allow collecting all the updates into a tuplestore that > could be passed to a statement-level trigger. Right now I tend to agree > that there's little point. Maybe, this can be used to re-implement

Re: [HACKERS] Parallel tuplesort, partitioning, merging, and the future

2016-08-10 Thread Peter Geoghegan
On Wed, Aug 10, 2016 at 11:59 AM, Robert Haas wrote: > I think that last part is a very important property; my intuition is > that dividing up the work between cooperating processes in a way that > should come out equal will often fail to do so, either due to the >

Re: [HACKERS] new pgindent run before branch?

2016-08-10 Thread Tom Lane
Robert Haas writes: > On Wed, Aug 10, 2016 at 5:04 PM, Tom Lane wrote: >> Robert Haas writes: >>> Would anyone mind too much if I refreshed typedefs.list and >>> re-indented the whole tree before we branch? >> +1, I was planning

Re: [HACKERS] new pgindent run before branch?

2016-08-10 Thread Bruce Momjian
On Wed, Aug 10, 2016 at 10:31:35PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > I am confused --- are you pgindenting just HEAD and not 9.6? Why? > > The point is to pgindent while they're still the same. So I read this: > >> +1, I was planning to do that myself. > >

Re: [HACKERS] Btree Index on PostgreSQL and Wiredtiger (MongoDB3.2)

2016-08-10 Thread Lukas Fittl
On Wed, Aug 10, 2016 at 4:24 PM, Kisung Kim wrote: > > When I used the index bloating estimation script in > https://github.com/ioguix/pgsql-bloat-estimation, > the result is as follows: > Regardless of the issue at hand, it might make sense to verify these statistics using

Re: [HACKERS] new pgindent run before branch?

2016-08-10 Thread Tom Lane
Bruce Momjian writes: > I am confused --- are you pgindenting just HEAD and not 9.6? Why? The point is to pgindent while they're still the same. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] No longer possible to query catalogs for index capabilities?

2016-08-10 Thread Robert Haas
On Wed, Aug 10, 2016 at 6:14 PM, Tom Lane wrote: > Kevin Grittner writes: >> That one seems like it should either be at the AM level or not >> included at all. Where it would be interesting to know is if you >> are a hacker looking for an AM to enhance

Re: [HACKERS] phrase search TS_phrase_execute code readability patch

2016-08-10 Thread Robert Haas
On Tue, Aug 9, 2016 at 3:35 PM, David G. Johnston wrote: > I don't follow why LposStart is needed so I removed it... That doesn't seem very reasonable. > Not compiled or in any way tested... Please do not bother submitting patches that you aren't prepared to compile

Re: [HACKERS] new pgindent run before branch?

2016-08-10 Thread Robert Haas
On Wed, Aug 10, 2016 at 5:04 PM, Tom Lane wrote: > Robert Haas writes: >> Would anyone mind too much if I refreshed typedefs.list and >> re-indented the whole tree before we branch? > > +1, I was planning to do that myself. Great. Are you handling

Re: [HACKERS] new pgindent run before branch?

2016-08-10 Thread Bruce Momjian
On Wed, Aug 10, 2016 at 10:35:44PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > It sounds like you are saying that the branch is to happen before the > > pgindent run. Am I missing something? > > I read it as pgindent then branch. OK, good. -- Bruce Momjian

Re: [HACKERS] new pgindent run before branch?

2016-08-10 Thread Robert Haas
On Wed, Aug 10, 2016 at 10:23 PM, Tom Lane wrote: >> Great. Are you handling creating the new branch? > > Yeah, I guess. I've made most of them lately. I'll do it if you want. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company --

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-08-10 Thread Venkata Balaji N
On Tue, Aug 9, 2016 at 1:02 AM, Robert Haas wrote: > On Mon, Aug 8, 2016 at 10:10 AM, Rahila Syed > wrote: > > Thank you for inputs everyone. > > > > The opinions on this thread can be classified into following > > 1. Commit > > 2. Rollback > > 3.

Re: [HACKERS] new pgindent run before branch?

2016-08-10 Thread Bruce Momjian
On Wed, Aug 10, 2016 at 10:23:14PM -0400, Tom Lane wrote: > Robert Haas writes: > > On Wed, Aug 10, 2016 at 5:04 PM, Tom Lane wrote: > >> Robert Haas writes: > >>> Would anyone mind too much if I refreshed typedefs.list and > >>>

Re: [HACKERS] new pgindent run before branch?

2016-08-10 Thread Tom Lane
Bruce Momjian writes: > It sounds like you are saying that the branch is to happen before the > pgindent run. Am I missing something? I read it as pgindent then branch. regards, tom lane -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Btree Index on PostgreSQL and Wiredtiger (MongoDB3.2)

2016-08-10 Thread Kisung Kim
Thank you for your information. Here is the result: After insertions: ycsb=# select * from pgstatindex('usertable_pkey'); version | tree_level | index_size | root_block_no | internal_pages | leaf_pages | empty_pages | deleted_pages | avg_leaf_density | leaf_fragmentation

Re: [HACKERS] Declarative partitioning - another take

2016-08-10 Thread Robert Haas
On Wed, Aug 10, 2016 at 7:09 AM, Amit Langote wrote: > Attached is the latest set of patches to implement declarative > partitioning. Cool. I would encourage you to give some thought to what is the least committable subset of these patches, and think about whether

Re: [HACKERS] Parallel tuplesort, partitioning, merging, and the future

2016-08-10 Thread Claudio Freire
On Mon, Aug 8, 2016 at 4:44 PM, Peter Geoghegan wrote: > The basic idea I have in mind is that we create runs in workers in the > same way that the parallel CREATE INDEX patch does (one output run per > worker). However, rather than merging in the leader, we use a > splitting

Re: [HACKERS] new pgindent run before branch?

2016-08-10 Thread Tom Lane
Robert Haas writes: > Would anyone mind too much if I refreshed typedefs.list and > re-indented the whole tree before we branch? +1, I was planning to do that myself. regards, tom lane -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Assertion failure in REL9_5_STABLE

2016-08-10 Thread Alvaro Herrera
Marko Tiikkaja wrote: > On 2016-08-10 19:28, Alvaro Herrera wrote: > >Umm. AFAICS HeapTupleSatisfiesUpdate() only returns SelfUpdated after > >already calling HeapTupleHeaderGetCmax() (which obviously hasn't caught > >the same assertion). Something is odd there ... > >

Re: [HACKERS] Assertion failure in REL9_5_STABLE

2016-08-10 Thread Alvaro Herrera
BTW this is not a regression, right? It's been broken all along. Or am I mistaken? -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] No longer possible to query catalogs for index capabilities?

2016-08-10 Thread Tom Lane
Kevin Grittner writes: > That one seems like it should either be at the AM level or not > included at all. Where it would be interesting to know is if you > are a hacker looking for an AM to enhance with support, or (when > there is more than just btree supported, so it is not

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-10 Thread Amit Kapila
On Mon, Aug 8, 2016 at 9:56 PM, Bruce Momjian wrote: > On Mon, Aug 8, 2016 at 06:34:46PM +0530, Amit Kapila wrote: >> I think here expensive part would be recheck for the cases where the >> index value is changed to a different value (value which doesn't exist >> in WARM

Re: [HACKERS] multivariate statistics (v19)

2016-08-10 Thread Michael Paquier
On Thu, Aug 11, 2016 at 3:34 AM, Tomas Vondra wrote: > On 08/10/2016 02:23 PM, Michael Paquier wrote: >> >> On Wed, Aug 10, 2016 at 8:33 PM, Tomas Vondra >> wrote: >>> The idea is that the syntax should work even for statistics built on

Re: [HACKERS] Small issues in syncrep.c

2016-08-10 Thread Michael Paquier
On Wed, Aug 10, 2016 at 4:29 PM, Simon Riggs wrote: > 14e8803f1 was only a partial patch for the syncrep code, so I don't > see any reason to keep the code as it currently is in 9.5/9.6. > > Any objections to backpatching this to 9.5 and 9.6? None from here. -- Michael

Re: [HACKERS] Small issues in syncrep.c

2016-08-10 Thread Julien Rouhaud
On 10/08/2016 09:43, Michael Paquier wrote: > On Wed, Aug 10, 2016 at 4:29 PM, Simon Riggs wrote: >> >> I've updated Julien's patch to reflect Michael's suggestion. Thanks to you and Michael. >> 14e8803f1 was only a partial patch for the syncrep code, so I don't >> see

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-10 Thread Heikki Linnakangas
On 08/10/2016 01:03 PM, Ants Aasma wrote: On Tue, Aug 9, 2016 at 3:16 PM, Heikki Linnakangas wrote: (Reviving an old thread) I spent some time dusting off this old patch, to implement CSN snapshots. Attached is a new patch, rebased over current master, and with tons of

Re: [HACKERS] Small issues in syncrep.c

2016-08-10 Thread Simon Riggs
On 10 August 2016 at 06:24, Michael Paquier wrote: > On Tue, Aug 9, 2016 at 5:34 PM, Julien Rouhaud > wrote: >> Since 14e8803f1, it's not necessary to acquire the SyncRepLock to see up >> to date data. But it looks like this commit didn't

Re: [HACKERS] No longer possible to query catalogs for index capabilities?

2016-08-10 Thread Tom Lane
Andrew Gierth writes: > So these properties (I've changed all the names here, suggestions > welcome for better ones) I think should be testable on the AM, each with > an example of why: > can_order > can_unique > can_multi_col > can_exclude Check, flags

Re: [HACKERS] Assertion failure in REL9_5_STABLE

2016-08-10 Thread Marko Tiikkaja
On 2016-08-10 11:01 PM, Alvaro Herrera wrote: Oh, I see ... so there's an update chain, and you're updating a earlier tuple. But the later tuple has a multixact and one of the members is the current transaction. I wonder how can you lock a tuple that's not the latest, where that update chain

Re: [HACKERS] Assertion failure in REL9_5_STABLE

2016-08-10 Thread Marko Tiikkaja
On 2016-08-11 12:09 AM, Alvaro Herrera wrote: BTW this is not a regression, right? It's been broken all along. Or am I mistaken? You're probably right. I just hadn't realized I could run our app against 9.5 with --enable-cassert until last week. .m -- Sent via pgsql-hackers mailing

Re: [HACKERS] Declarative partitioning

2016-08-10 Thread Ashutosh Bapat
FOR VALUE clause of a partition does not allow a constant expression like (1/5 -1). It gives syntax error regression=# create table pt1_p1 partition of pt1 for values start (0) end ((1/5) - 1); ERROR: syntax error at or near "(" LINE 1: ...pt1_p1 partition of pt1 for values start (0) end

  1   2   >