Re: [HACKERS] Allow SQL/plpgsql functions to accept record

2015-04-23 Thread Jim Nasby
On 4/23/15 8:33 AM, Pavel Stehule wrote: I agree that that would be useful. I think the problem with an expression like rowvar.something is that PL/pgsql cannot infer the type of the result, and nothing else works without that. I doubt that it's practical to lift that

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-23 Thread Geoff Winkless
On 23 April 2015 at 14:50, Andres Freund and...@anarazel.de wrote: ​Maybe I'm misreading it, but isn't index_predicate meant to be inside the brackets? http://postgres-benchmarks.s3-website-us-east-1.amazonaws.com/on-conflict-docs/sql-insert.html That has changed since. ​Oh,

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Bruce Momjian
On Thu, Apr 23, 2015 at 09:13:52AM -0400, Robert Haas wrote: On Wed, Apr 22, 2015 at 10:41 PM, Bruce Momjian br...@momjian.us wrote: What is a real problem is that we don't block creating tablespaces anywhere at all, including in obviously problematic places like the transaction log

Re: [HACKERS] Add CINE for ALTER TABLE ... ADD COLUMN

2015-04-23 Thread Fabrízio de Royes Mello
On Wed, Apr 22, 2015 at 3:48 PM, Payal Singh pa...@omniti.com wrote: The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: not tested Spec compliant: not tested Documentation:not

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-23 Thread Jim Nasby
On 4/23/15 8:42 AM, Robert Haas wrote: On Thu, Apr 23, 2015 at 4:19 AM, Simon Riggs si...@2ndquadrant.com wrote: We were talking about having an incremental backup map also. Which sounds a lot like the freeze map. Yeah, possibly. I think we should try to set things up so that the backup map

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Bruce Momjian
On Thu, Apr 23, 2015 at 05:05:14PM +0200, Andres Freund wrote: On 2015-04-23 11:00:43 -0400, Bruce Momjian wrote: On Thu, Apr 23, 2015 at 09:13:52AM -0400, Robert Haas wrote: I think this is a good thing to do, but I sure wish we could go further and block it completely. That may require

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-23 Thread Bruce Momjian
On Thu, Apr 23, 2015 at 10:42:59AM +0300, Heikki Linnakangas wrote: On 04/22/2015 09:24 PM, Robert Haas wrote: I would feel safer if we added a completely new epoch counter to the page header, instead of reusing LSNs. But as we all know, changing the page format is a problem for in-place

Re: [HACKERS] anole - test case sha2 fails on all branches

2015-04-23 Thread Tom Lane
I wrote: Given that anole is the only one reporting this, I'm not sure that we should immediately blame Postgres itself. I have a vague recollection that we've seen this symptom before and traced it to a bug in some supporting library. Is anole using any particularly out-of-date versions of

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-23 Thread Bruce Momjian
On Thu, Apr 23, 2015 at 09:45:38AM -0400, Robert Haas wrote: Right. My point is that either you do X 2M times to maintain that fork and the overhead of the file existence, or you do one VACUUM FREEZE. I am saying that 2M is a large number and adding all those X's might exceed the cost of

Re: [HACKERS] anole - test case sha2 fails on all branches

2015-04-23 Thread Alvaro Herrera
Robert Haas wrote: There are lots of machines failing in pg_upgradeCheck, but I don't see details of the failures in the logs. Yeah, I think the buildfarm script is failing to save the error log. Anyway AFAIR this is related to the move from contrib to src/bin; machines that have updated to

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-23 Thread Petr Jelinek
On 23/04/15 17:24, Heikki Linnakangas wrote: On 04/23/2015 05:52 PM, Jim Nasby wrote: On 4/23/15 2:42 AM, Heikki Linnakangas wrote: On 04/22/2015 09:24 PM, Robert Haas wrote: Yeah. We have a serious need to reduce the size of our on-disk format. On a TPC-C-like workload Jan Wieck recently

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-23 Thread Heikki Linnakangas
On 04/23/2015 06:39 PM, Petr Jelinek wrote: On 23/04/15 17:24, Heikki Linnakangas wrote: On 04/23/2015 05:52 PM, Jim Nasby wrote: I've often wondered if there was some way we could consolidate XMIN/XMAX from multiple tuples at the page level; that could be a big win for OLAP environments where

Re: [HACKERS] anole - test case sha2 fails on all branches

2015-04-23 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Apr 23, 2015 at 6:13 AM, Sandeep Thakkar sandeep.thak...@enterprisedb.com wrote: The test case sha2 in contrib pgcrypto module is failing with a diff on anole because the results file contains the additional lines as: + WARNING: detected

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-23 Thread Andres Freund
On 2015-04-23 15:52:40 +0100, Geoff Winkless wrote: When I set out I was really only hoping to express a preference as a user; on balance I would really rather not have DO IGNORE, if it were possible to avoid, because it's really ugly, but DO UPDATE/DO NOTHING I could just about cope with (and

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Jim Nasby
On 4/22/15 9:41 PM, Bruce Momjian wrote: The case this doesn't catch is referencing a symbolic link that points to the same directory. We can't make it an error so people can use pg_upgrade these setups. Couldn't we make it an ERROR unless IsBinaryUpgrade? -- Jim Nasby, Data Architect, Blue

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-23 Thread Heikki Linnakangas
On 04/23/2015 05:52 PM, Jim Nasby wrote: On 4/23/15 2:42 AM, Heikki Linnakangas wrote: On 04/22/2015 09:24 PM, Robert Haas wrote: Yeah. We have a serious need to reduce the size of our on-disk format. On a TPC-C-like workload Jan Wieck recently tested, our data set was 34% larger than

Re: [HACKERS] PL/pgSQL, RAISE and error context

2015-04-23 Thread Pavel Stehule
2015-04-23 16:12 GMT+02:00 Robert Haas robertmh...@gmail.com: On Thu, Apr 23, 2015 at 9:55 AM, Pavel Stehule pavel.steh...@gmail.com wrote: On Thu, Apr 23, 2015 at 4:56 AM, Pavel Stehule pavel.steh...@gmail.com wrote: I don't see a contradiction. There is clean agreement, so ERROR

Re: [HACKERS] Split the 'Server Programming' chapter into two?

2015-04-23 Thread Alvaro Herrera
Robert Haas wrote: On Thu, Apr 23, 2015 at 8:21 AM, Andres Freund and...@anarazel.de wrote: To me at least 44 - 47 don't really fit well to the rest. I think we either should invent a new category for them, or move them to 'Internals'. Maybe we could introduce 'Extending the Server'

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-23 Thread Jim Nasby
On 4/23/15 2:42 AM, Heikki Linnakangas wrote: On 04/22/2015 09:24 PM, Robert Haas wrote: Yeah. We have a serious need to reduce the size of our on-disk format. On a TPC-C-like workload Jan Wieck recently tested, our data set was 34% larger than another database at the beginning of the test,

Re: [HACKERS] Split the 'Server Programming' chapter into two?

2015-04-23 Thread Jim Nasby
On 4/23/15 9:23 AM, Alvaro Herrera wrote: Robert Haas wrote: On Thu, Apr 23, 2015 at 8:21 AM, Andres Freund and...@anarazel.de wrote: To me at least 44 - 47 don't really fit well to the rest. I think we either should invent a new category for them, or move them to 'Internals'. Maybe we could

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-23 Thread Bruce Momjian
On Thu, Apr 23, 2015 at 05:02:19PM +0200, Andres Freund wrote: On 2015-04-23 15:52:40 +0100, Geoff Winkless wrote: When I set out I was really only hoping to express a preference as a user; on balance I would really rather not have DO IGNORE, if it were possible to avoid, because it's

Re: [HACKERS] Shouldn't CREATE TABLE LIKE copy the relhasoids property?

2015-04-23 Thread Bruce Momjian
On Thu, Apr 23, 2015 at 09:26:50AM -0400, Robert Haas wrote: What I meant was - I didn't see an attachment on that message. I didn't attach it as people have told me they can just as easily see the patch via git, and since it was so similar, I didn't repost it. Should I have? I can

Re: [HACKERS] Reducing tuple overhead

2015-04-23 Thread Joshua D. Drake
On 04/23/2015 09:42 AM, Jim Nasby wrote: On 4/23/15 11:24 AM, Andres Freund wrote: I do wonder what, in realistic cases, is actually the bigger contributor to the overhead. The tuple header or the padding we liberally add in many cases... Assuming you're talking about padding between

Re: [HACKERS] Reducing tuple overhead

2015-04-23 Thread Petr Jelinek
On 23/04/15 18:24, Andres Freund wrote: Whether that's feasible complexity wise is debatable, but it's certainly possible. I do wonder what, in realistic cases, is actually the bigger contributor to the overhead. The tuple header or the padding we liberally add in many cases... The logical

Re: [HACKERS] [BUGS] Failure to coerce unknown type to specific type

2015-04-23 Thread Jeff Davis
On Thu, Apr 23, 2015 at 1:49 AM, David G. Johnston david.g.johns...@gmail.com wrote: Reading and writing all this I'm convinced you have gotten the idea in your mind an expectation of equivalency and consistency where there really is little or none from an overall design perspective. And none

Re: [HACKERS] improving speed of make check-world

2015-04-23 Thread Jeff Janes
On Thu, Aug 14, 2014 at 10:45 PM, Peter Eisentraut pete...@gmx.net wrote: make check-world creates a temporary installation in every subdirectory it runs a test in, which is stupid: it's very slow and uses a lot of disk space. It's enough to do this once per run. That is the essence of what

Re: [HACKERS] Reducing tuple overhead

2015-04-23 Thread Alvaro Herrera
Thanks for posting this. Joshua D. Drake wrote: First of all I should note that I discussed the approach mentioned above to pgsql-hackers and got a very interesting comment from Tom Lane that adding sorting info to Var and TargetEntry nodes was not a very good idea because it'd break stored

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-23 Thread Jim Nasby
On 4/23/15 8:25 AM, Robert Haas wrote: Some users are partitioning tables just so that each partition can be autovac'd separately. That really shouldn't be required. Are they doing this for improved heap scan performance? Index scan performance? If the table wasn't partitioned, would they

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Andres Freund
On 2015-04-23 11:00:43 -0400, Bruce Momjian wrote: On Thu, Apr 23, 2015 at 09:13:52AM -0400, Robert Haas wrote: I think this is a good thing to do, but I sure wish we could go further and block it completely. That may require more thought than we have time to put in at this stage of the

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-23 Thread Bruce Momjian
On Thu, Apr 23, 2015 at 06:24:00PM +0300, Heikki Linnakangas wrote: I've often wondered if there was some way we could consolidate XMIN/XMAX from multiple tuples at the page level; that could be a big win for OLAP environments where most of your tuples belong to a pretty small range of XIDs.

Re: [HACKERS] adding more information about process(es) cpu and memory usage

2015-04-23 Thread Heikki Linnakangas
On 04/23/2015 08:00 PM, Radovan Jablonovsky wrote: During current encounters with amazon web services - RDS, the DBA does not have access to OS/linux shell of underlying instance. That render some postgresql monitoring technique of process CPU and memory usage, not useful. Even if the AWS

[HACKERS] Moving ExecInsertIndexTuples and friends to new file

2015-04-23 Thread Heikki Linnakangas
While looking at Peter's INSERT ... ON CONFLICT patch, I started to feel that ExecInsertIndexTuples() and friends would deserve a file of their own, and not be buried in the middle of execUtils.c. I propose that we split execUtils.c into two, moving ExecOpenIndices(), ExecCloseIndices()

[HACKERS] pg_dump: largeobject behavior issues (possible bug)

2015-04-23 Thread Joshua D. Drake
Hello, I have been working a problem with Andrew Gierth (sp?) in regards to pg_dump. Here is the basic breakdown: FreeBSD 10.1 PostgreSQL 9.3.6 64GB ~ memory 500GB database 228G of largeobjects (106M objects) The database dumps fine as long as we don't dump large objects. However, if we

Re: [HACKERS] [BUGS] Failure to coerce unknown type to specific type

2015-04-23 Thread Jim Nasby
On 4/23/15 5:07 AM, Kyotaro HORIGUCHI wrote: This is because parsing of UNION immediately converts constants of unknown type in the UNION's both arms to text so the top level select won't be bothered by this problem. But the problematic query doesn't have appropriate timing to do that until the

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-23 Thread Heikki Linnakangas
On 04/23/2015 10:53 PM, Andres Freund wrote: On 2015-04-23 12:45:59 -0700, Peter Geoghegan wrote: On Thu, Apr 23, 2015 at 12:55 AM, Andres Freund and...@anarazel.de wrote: I think you misread my statement: I'm saying we don't need the new argument anymore, even if we still do the

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Andres Freund
On 2015-04-23 15:17:55 -0500, Jim Nasby wrote: Yes, but only after creating a brand new cluster from scratch, which would then disallow them from putting tablespaces in $PGDATA. pg_dumpall output includes tablespaces. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread David Steele
On 4/23/15 4:30 PM, Andres Freund wrote: On 2015-04-23 16:26:09 -0400, Robert Haas wrote: But pg_upgrade automates all that, so you can't use pg_upgrade in that case. If we add a GUC as I suggested, you can still use pg_upgrade. But we also have to live with data directories being in a shit

Re: [HACKERS] Moving ExecInsertIndexTuples and friends to new file

2015-04-23 Thread Peter Geoghegan
On Thu, Apr 23, 2015 at 12:05 PM, Heikki Linnakangas hlinn...@iki.fi wrote: While looking at Peter's INSERT ... ON CONFLICT patch, I started to feel that ExecInsertIndexTuples() and friends would deserve a file of their own, and not be buried in the middle of execUtils.c. I propose that we

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Robert Haas
On Thu, Apr 23, 2015 at 11:00 AM, Bruce Momjian br...@momjian.us wrote: I have developed the attached patch to warn about creating tablespaces inside the data directory. The case this doesn't catch is referencing a symbolic link that points to the same directory. We can't make it an

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Andres Freund
On 2015-04-23 15:46:20 -0400, Robert Haas wrote: Well, we've made backward-incompatible changes before. Not to this specific thing, but in general. I don't think there's anything preventing us from doing so here, except that we don't want to annoy too many users. I think the number of users

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-23 Thread Andres Freund
On 2015-04-23 12:45:59 -0700, Peter Geoghegan wrote: On Thu, Apr 23, 2015 at 12:55 AM, Andres Freund and...@anarazel.de wrote: I think you misread my statement: I'm saying we don't need the new argument anymore, even if we still do the super-deletion in heap_delete(). Now that the

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Jim Nasby
On 4/23/15 11:01 AM, Andres Freund wrote: On April 23, 2015 6:12:05 PM GMT+03:00, Jim Nasby jim.na...@bluetreble.com wrote: On 4/22/15 9:41 PM, Bruce Momjian wrote: The case this doesn't catch is referencing a symbolic link that points to the same directory. We can't make it an error so

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-23 Thread Andres Freund
On 2015-04-23 15:40:36 -0400, Robert Haas wrote: The issue is that you have to vacuum a table frequently enough to avoid accumulating bloat. The frequency with which you need to vacuum varies depending on the size of the table and how frequently it's updated. However, a large,

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-23 Thread Peter Geoghegan
On Thu, Apr 23, 2015 at 12:55 AM, Andres Freund and...@anarazel.de wrote: I think you misread my statement: I'm saying we don't need the new argument anymore, even if we still do the super-deletion in heap_delete(). Now that the speculative insertion will not be visible (as in seen on a tuple

Re: [HACKERS] pg_dump: largeobject behavior issues (possible bug)

2015-04-23 Thread Andrew Gierth
Joshua == Joshua D Drake j...@commandprompt.com writes: Joshua The database dumps fine as long as we don't dump large Joshua objects. However, if we try to dump the large objects, FreeBSD Joshua will kill pg_dump as it will consume all free memory and Joshua swap. With Andrew's help we were

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-23 Thread Peter Geoghegan
On Thu, Apr 23, 2015 at 12:53 PM, Andres Freund and...@anarazel.de wrote: Unconvinced. Not breaking an API has its worth. Yeah, and I acknowledge that - but it's not something that it's appropriate to encapsulate IMV. Let's just leave it to Heikki...I'd say he has the deciding vote, especially

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-23 Thread Andres Freund
On 2015-04-23 23:08:34 +0300, Heikki Linnakangas wrote: The heapam API is not that stable, we've added arguments to those functions every once in a while, and I don't recall any complaints. I heard some, but not too many, that's true. I know that I've written code that'd be broken/needed even

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Andres Freund
On 2015-04-23 16:26:09 -0400, Robert Haas wrote: But pg_upgrade automates all that, so you can't use pg_upgrade in that case. If we add a GUC as I suggested, you can still use pg_upgrade. But we also have to live with data directories being in a shit state forever onward. We won't really be

Re: [HACKERS] pg_dump: largeobject behavior issues (possible bug)

2015-04-23 Thread Andrew Dunstan
On 04/23/2015 04:04 PM, Andrew Gierth wrote: Joshua == Joshua D Drake j...@commandprompt.com writes: Joshua The database dumps fine as long as we don't dump large Joshua objects. However, if we try to dump the large objects, FreeBSD Joshua will kill pg_dump as it will consume all free

[HACKERS] adding more information about process(es) cpu and memory usage

2015-04-23 Thread Radovan Jablonovsky
During current encounters with amazon web services - RDS, the DBA does not have access to OS/linux shell of underlying instance. That render some postgresql monitoring technique of process CPU and memory usage, not useful. Even if the AWS provide internal tools/programming interface for

Re: [HACKERS] [BUGS] Failure to coerce unknown type to specific type

2015-04-23 Thread David G. Johnston
On Thursday, April 23, 2015, Jeff Davis pg...@j-davis.com wrote: On Thu, Apr 23, 2015 at 1:49 AM, David G. Johnston david.g.johns...@gmail.com javascript:; wrote: Reading and writing all this I'm convinced you have gotten the idea in your mind an expectation of equivalency and consistency

Re: [HACKERS] [committers] pgsql: RLS fixes, new hooks, and new test module

2015-04-23 Thread Stephen Frost
Christian, * Christian Ullrich (ch...@chrullrich.net) wrote: * Stephen Frost wrote: RLS fixes, new hooks, and new test module The buildfarm says that with -DCLOBBER_CACHE_ALWAYS, the RLS violations get blamed on the wrong tables. Mostly, they are catalogs (I have seen pg_opclass, pg_am,

Re: [HACKERS] Moving ExecInsertIndexTuples and friends to new file

2015-04-23 Thread Stephen Frost
* Peter Geoghegan (p...@heroku.com) wrote: On Thu, Apr 23, 2015 at 12:05 PM, Heikki Linnakangas hlinn...@iki.fi wrote: While looking at Peter's INSERT ... ON CONFLICT patch, I started to feel that ExecInsertIndexTuples() and friends would deserve a file of their own, and not be buried in

Re: [HACKERS] Reducing tuple overhead

2015-04-23 Thread Amit Kapila
On Fri, Apr 24, 2015 at 1:03 AM, Jim Nasby jim.na...@bluetreble.com wrote: On 4/23/15 11:45 AM, Petr Jelinek wrote: On 23/04/15 18:24, Andres Freund wrote: Whether that's feasible complexity wise is debatable, but it's certainly possible. I do wonder what, in realistic cases, is actually

Re: [HACKERS] [BUGS] Failure to coerce unknown type to specific type

2015-04-23 Thread Kyotaro HORIGUCHI
Hello, At Thu, 23 Apr 2015 14:49:29 -0500, Jim Nasby jim.na...@bluetreble.com wrote in 55394cc9.5050...@bluetreble.com On 4/23/15 5:07 AM, Kyotaro HORIGUCHI wrote: This is because parsing of UNION immediately converts constants of unknown type in the UNION's both arms to text so the top

Re: [HACKERS] forward vs backward slashes in msvc build code

2015-04-23 Thread Michael Paquier
On Fri, Mar 13, 2015 at 6:04 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Fri, Mar 13, 2015 at 6:20 AM, Alvaro Herrera wrote: Peter Eisentraut wrote: This is contrib/chkpass not finding the crypt symbol, which is presumably in some library. But I can't see how it would normally

[HACKERS] Improving vacuum/VM/etc

2015-04-23 Thread Jim Nasby
I mentioned this idea in the other[1] vacuum thread [2], but I think it got lost. Kevin Grittner pointed out that there's a potentially huge number of writes we incur over the life of a tuple [3]: (1) WAL log the insert. (2) Write the tuple. (3) Hint and rewrite the tuple. (4) WAL log the

Re: [HACKERS] Reducing tuple overhead

2015-04-23 Thread Jim Nasby
On 4/23/15 11:45 AM, Petr Jelinek wrote: On 23/04/15 18:24, Andres Freund wrote: Whether that's feasible complexity wise is debatable, but it's certainly possible. I do wonder what, in realistic cases, is actually the bigger contributor to the overhead. The tuple header or the padding we

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-23 Thread Robert Haas
On Thu, Apr 23, 2015 at 10:44 AM, Jim Nasby jim.na...@bluetreble.com wrote: On 4/23/15 8:25 AM, Robert Haas wrote: Some users are partitioning tables just so that each partition can be autovac'd separately. That really shouldn't be required. Are they doing this for improved heap scan

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Robert Haas
On Thu, Apr 23, 2015 at 3:57 PM, Andres Freund and...@anarazel.de wrote: On 2015-04-23 15:46:20 -0400, Robert Haas wrote: Well, we've made backward-incompatible changes before. Not to this specific thing, but in general. I don't think there's anything preventing us from doing so here, except

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-23 Thread Michael Paquier
On Thu, Apr 23, 2015 at 10:42 PM, Robert Haas wrote: On Thu, Apr 23, 2015 at 4:19 AM, Simon Riggs wrote: We only need a freeze/backup map for larger relations. So if we map 1000 blocks per map page, we skip having a map at all when size 1000. Agreed. We might also want to map multiple

Re: [HACKERS] [committers] pgsql: RLS fixes, new hooks, and new test module

2015-04-23 Thread Christian Ullrich
* Stephen Frost wrote: RLS fixes, new hooks, and new test module The buildfarm says that with -DCLOBBER_CACHE_ALWAYS, the RLS violations get blamed on the wrong tables. Mostly, they are catalogs (I have seen pg_opclass, pg_am, and pg_amproc), but some also come up with binary garbage

Re: [HACKERS] adding more information about process(es) cpu and memory usage

2015-04-23 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: On Thu, Apr 23, 2015 at 10:17 AM, Heikki Linnakangas hlinn...@iki.fi wrote: In a nutshell, I don't think PostgreSQL should get involved in that... I have often wanted an SQL function which would expose the back-end's rusage statistics to the front-end.

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-23 Thread Peter Geoghegan
On Thu, Apr 23, 2015 at 12:45 PM, Peter Geoghegan p...@heroku.com wrote: * We need to figure out the tuple lock strength details. I think this is doable, but it is the greatest challenge to committing ON CONFLICT UPDATE at this point. Andres feels that we should require no greater lock

Re: [HACKERS] PL/pgSQL, RAISE and error context

2015-04-23 Thread Pavel Stehule
2015-04-23 15:47 GMT+02:00 Robert Haas robertmh...@gmail.com: On Thu, Apr 23, 2015 at 4:56 AM, Pavel Stehule pavel.steh...@gmail.com wrote: I don't see a contradiction. There is clean agreement, so ERROR level should to show the context. NOTICE and WARNINGs doesn't need it - and there is a

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-23 Thread Andres Freund
On 2015-04-23 14:34:02 +0100, Geoff Winkless wrote: A syntax error. DO is a reserved keyword. Update is just unreserved (and thus can be used as a column label). Ignore is unreserved with the patch and was unreserved before. We obviously can make both reserved, but of so we have to do it

Re: [HACKERS] Code paths where LWLock should be released on failure

2015-04-23 Thread Kevin Grittner
Michael Paquier michael.paqu...@gmail.com wrote: I have also been surprised by the inconsistencies particularly in predicate.c or other places regarding LWLock releases. Sometimes they are released on failure, sometimes not. Those are not needed for correctness; they are a micro-optimization

Re: [HACKERS] anole - test case sha2 fails on all branches

2015-04-23 Thread Robert Haas
On Thu, Apr 23, 2015 at 6:13 AM, Sandeep Thakkar sandeep.thak...@enterprisedb.com wrote: The test case sha2 in contrib pgcrypto module is failing with a diff on anole because the results file contains the additional lines as: -- + WARNING: detected write past chunk end in ExprContext

Re: [HACKERS] Split the 'Server Programming' chapter into two?

2015-04-23 Thread Robert Haas
On Thu, Apr 23, 2015 at 8:21 AM, Andres Freund and...@anarazel.de wrote: While playing around with where exactly to put the replication origin/progress docs I once more noticed that the 'Server Programming' book is a mix of different topics. It currently contains: 35. Extending SQL

Re: [HACKERS] PL/pgSQL, RAISE and error context

2015-04-23 Thread Robert Haas
On Thu, Apr 23, 2015 at 9:55 AM, Pavel Stehule pavel.steh...@gmail.com wrote: On Thu, Apr 23, 2015 at 4:56 AM, Pavel Stehule pavel.steh...@gmail.com wrote: I don't see a contradiction. There is clean agreement, so ERROR level should to show the context. NOTICE and WARNINGs doesn't need it

Re: [HACKERS] Code paths where LWLock should be released on failure

2015-04-23 Thread Michael Paquier
On Thu, Apr 23, 2015 at 2:46 PM, Tom Lane t...@sss.pgh.pa.us wrote: Michael Paquier michael.paqu...@gmail.com writes: After looking at bug #13128, I have been looking at the code around LWLockAcquire/Release to see if there are similar issues elsewhere. Here are my findings: IIRC, we

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-23 Thread Sawada Masahiko
On Thu, Apr 23, 2015 at 3:24 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Apr 22, 2015 at 12:39 PM, Heikki Linnakangas hlinn...@iki.fi wrote: The thing that made me nervous about that approach is that it made the LSN of each page critical information. If you somehow zeroed out the LSN,

Re: [HACKERS] [BUGS] Failure to coerce unknown type to specific type

2015-04-23 Thread Jeff Davis
On Wed, 2015-04-22 at 20:35 -0700, David G. Johnston wrote: But the fact that column b has the data type unknown is only a warning - not an error. I get an error: postgres=# SELECT ' '::text = 'a'; ?column? -- f (1 row) postgres=# SELECT a=b FROM (SELECT ''::text, ' ') x(a,b);

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-04-23 Thread Robert Haas
On Wed, Apr 22, 2015 at 10:41 PM, Bruce Momjian br...@momjian.us wrote: What is a real problem is that we don't block creating tablespaces anywhere at all, including in obviously problematic places like the transaction log directory: josh=# create tablespace tbl2 location

Re: [HACKERS] [BUGS] Failure to coerce unknown type to specific type

2015-04-23 Thread Kyotaro HORIGUCHI
Sorry, the patch had obvious bug.. -+ Int32GetDatum(inputTypeMod), ++ Int32GetDatum(targetTypeMod), regards, Hello, I

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-23 Thread Simon Riggs
On 21 April 2015 at 22:21, Robert Haas robertmh...@gmail.com wrote: I'm not saying those ideas don't have problems, because they do. But I think they are worth further exploring. The main reason I gave up on that is because Heikki was working on the XID-to-LSN mapping stuff. That seemed

Re: [HACKERS] [BUGS] Failure to coerce unknown type to specific type

2015-04-23 Thread David G. Johnston
On Thu, Apr 23, 2015 at 1:35 AM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Very sorry for the trash.. === Now I found a comment at just where I patched, * XXX if the typinput function is not immutable, we really ought to * postpone evaluation of the function call until

Re: [HACKERS] [BUGS] Failure to coerce unknown type to specific type

2015-04-23 Thread David G. Johnston
On Thu, Apr 23, 2015 at 1:07 AM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Hello, I think this is a bug. The core of this problem is that coerce_type() fails for Var of type UNKNOWNOID. The comment for the function says that, * The caller should already have determined

Re: [HACKERS] Code paths where LWLock should be released on failure

2015-04-23 Thread Andres Freund
Hi,m On 2015-04-23 13:51:57 +0900, Michael Paquier wrote: After looking at bug #13128, I have been looking at the code around LWLockAcquire/Release to see if there are similar issues elsewhere. Here are my findings: Afaics all of these should actually be handled by the paths that release

Re: [HACKERS] [BUGS] Failure to coerce unknown type to specific type

2015-04-23 Thread Kyotaro HORIGUCHI
Hello, I think this is a bug. The core of this problem is that coerce_type() fails for Var of type UNKNOWNOID. The comment for the function says that, * The caller should already have determined that the coercion is possible; * see can_coerce_type. But can_coerce_type() should say it's

Re: [HACKERS] [BUGS] Failure to coerce unknown type to specific type

2015-04-23 Thread David G. Johnston
On Wednesday, April 22, 2015, Jeff Davis pg...@j-davis.com wrote: On Wed, 2015-04-22 at 20:35 -0700, David G. Johnston wrote: But the fact that column b has the data type unknown is only a warning - not an error. I get an error: postgres=# SELECT ' '::text = 'a'; ?column?

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-23 Thread Heikki Linnakangas
On 04/22/2015 09:24 PM, Robert Haas wrote: I would feel safer if we added a completely new epoch counter to the page header, instead of reusing LSNs. But as we all know, changing the page format is a problem for in-place upgrade, and takes some space too. Yeah. We have a serious need to reduce

Re: [HACKERS] Streaming replication and WAL archive interactions

2015-04-23 Thread Heikki Linnakangas
On 04/22/2015 11:58 PM, Robert Haas wrote: On Wed, Apr 22, 2015 at 3:34 PM, Heikki Linnakangas hlinn...@iki.fi wrote: On 04/22/2015 10:21 PM, Robert Haas wrote: On Wed, Apr 22, 2015 at 3:01 PM, Heikki Linnakangas hlinn...@iki.fi wrote: For example, imagine that perform point-in-time recovery

Re: [HACKERS] [BUGS] Failure to coerce unknown type to specific type

2015-04-23 Thread Kyotaro HORIGUCHI
Very sorry for the trash.. === Now I found a comment at just where I patched, * XXX if the typinput function is not immutable, we really ought to * postpone evaluation of the function call until runtime. But there * is no way to represent a typinput function call as an expression * tree,

Re: [HACKERS] [BUGS] Failure to coerce unknown type to specific type

2015-04-23 Thread David G. Johnston
On Thu, Apr 23, 2015 at 1:49 AM, David G. Johnston david.g.johns...@gmail.com wrote: On Wednesday, April 22, 2015, Jeff Davis pg...@j-davis.com wrote: On Wed, 2015-04-22 at 20:35 -0700, David G. Johnston wrote: ​My gut reaction is if you feel strongly enough to add some additional

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-23 Thread Andres Freund
On 2015-04-22 15:23:16 -0700, Peter Geoghegan wrote: On Tue, Apr 21, 2015 at 7:57 AM, Andres Freund and...@anarazel.de wrote: * Iff we're going to have the XLOG_HEAP_AFFIRM record, I'd rather have that guide the logical decoding code. Seems slightly cleaner. I thought that you didn't

Re: [HACKERS] PL/pgSQL, RAISE and error context

2015-04-23 Thread Marko Tiikkaja
On 4/2/15 9:37 AM, Pavel Stehule wrote: estate-err_text = stmt-elog_level == ERROR ? estate-err_text : raise_skip_msg ; Can we do this simple change? It will produce a stackinfo for exceptions and it will not to make mad developers by lot of useless content. I'm not sure everyone agrees

Re: [HACKERS] [BUGS] Failure to coerce unknown type to specific type

2015-04-23 Thread Kyotaro HORIGUCHI
Now I found a comment at just where I patched, * XXX if the typinput function is not immutable, we really ought to * postpone evaluation of the function call until runtime. But there * is no way to represent a typinput function call as an expression * tree, because C-string values are not

Re: [HACKERS] PL/pgSQL, RAISE and error context

2015-04-23 Thread Pavel Stehule
2015-04-23 9:53 GMT+02:00 Marko Tiikkaja ma...@joh.to: On 4/2/15 9:37 AM, Pavel Stehule wrote: estate-err_text = stmt-elog_level == ERROR ? estate-err_text : raise_skip_msg ; Can we do this simple change? It will produce a stackinfo for exceptions and it will not to make mad developers

[HACKERS] anole - test case sha2 fails on all branches

2015-04-23 Thread Sandeep Thakkar
Hi, The test case sha2 in contrib pgcrypto module is failing with a diff on anole because the results file contains the additional lines as: -- + WARNING: detected write past chunk end in ExprContext 6021cbb0 -- Ex: The log for REL9_2_STABLE can be seen at

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-23 Thread Sawada Masahiko
On Mon, Apr 20, 2015 at 10:17 PM, David Steele da...@pgmasters.net wrote: On 4/20/15 4:40 AM, Sawada Masahiko wrote: Thank you for updating the patch. One question about regarding since v7 (or later) patch is; What is the different between OBJECT logging and SESSION logging? In brief,

Re: [HACKERS] [BUGS] Failure to coerce unknown type to specific type

2015-04-23 Thread Kyotaro HORIGUCHI
Hello, Very sorry for the trash.. === Now I found a comment at just where I patched, * XXX if the typinput function is not immutable, we really ought to * postpone evaluation of the function call until runtime. But there * is no way to represent a typinput function call as

Re: [HACKERS] [BUGS] Failure to coerce unknown type to specific type

2015-04-23 Thread Kyotaro HORIGUCHI
Hello, Hello, I think this is a bug. The core of this problem is that coerce_type() fails for Var of type UNKNOWNOID. The comment for the function says that, * The caller should already have determined that the coercion is possible; * see can_coerce_type. But

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-23 Thread Andres Freund
On 2015-04-22 16:40:07 -0700, Peter Geoghegan wrote: On Wed, Apr 22, 2015 at 3:23 PM, Peter Geoghegan p...@heroku.com wrote: * We need to sort out those issues with the grammar, since that only really applies to the inference specification. Maybe the WHERE clause that the inference

[HACKERS] Split the 'Server Programming' chapter into two?

2015-04-23 Thread Andres Freund
Hi, While playing around with where exactly to put the replication origin/progress docs I once more noticed that the 'Server Programming' book is a mix of different topics. It currently contains: 35. Extending SQL 36. Triggers 37. Event Triggers 38. The Rule System 39.

Re: [HACKERS] Replication identifiers, take 4

2015-04-23 Thread Andres Freund
On 2015-03-24 22:22:29 +0100, Petr Jelinek wrote: Perhaps we should have some Logical replication developer documentation section and put all those three as subsections of that? So I just played around with this and it didn't find it worthwhile. Primarily because there's lots of uses of logical

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-23 Thread Geoff Winkless
Apologies for butting in but can I (as a user) express a preference as a user against DO? Firstly, it looks horrible. And what's to stop me having SELECT true AS do in the where clause (as per your UPDATE objection)? Shouldn't UPDATE be a reserved keyword anyway? AIUI ANSI suggests so.

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-23 Thread Petr Jelinek
On 23/04/15 14:34, Geoff Winkless wrote: Apologies for butting in but can I (as a user) express a preference as a user against DO? Firstly, it looks horrible. And what's to stop me having SELECT true AS do in the where clause (as per your UPDATE objection)? DO is already reserved keyword.

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-23 Thread Andres Freund
On April 23, 2015 3:34:07 PM GMT+03:00, Geoff Winkless pgsqlad...@geoff.dj wrote: Apologies for butting in but can I (as a user) express a preference as a user against DO? Sure. If you propose an alternative ;) Firstly, it looks horrible. And what's to stop me having SELECT true AS do in the

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-23 Thread Robert Haas
On Wed, Apr 22, 2015 at 5:17 PM, Bruce Momjian br...@momjian.us wrote: On Wed, Apr 22, 2015 at 06:07:00PM -0300, Alvaro Herrera wrote: Good point, but doesn't vacuum remove the need for pruning as it removes all the old rows? Sure. The point, I think, is to make autovacuum runs of some

  1   2   >