Re: [HACKERS] The question about the type numeric

2014-04-17 Thread sure.postgres
I am so sorry for that. This is my fault. My mail client does not receive any e-mail. So I tried to re-send the e-mail again. My problem has been resolved. Thank you for your reply. 2014-04-16 wangshuo HighGo Software Co.,Ltd. Address: A203 Block D QILU Soft Park, High-Tech Zone, Lixia

Re: [HACKERS] [COMMITTERS] pgsql: Include planning time in EXPLAIN ANALYZE output.

2014-04-17 Thread Oleg Bartunov
I found a bit confusing, when planning time is greater total time, so +1 for execution time. On Thu, Apr 17, 2014 at 3:35 AM, Tom Lane t...@sss.pgh.pa.us wrote: Bruce Momjian br...@momjian.us writes: Where are we on this? I still see: test= EXPLAIN ANALYZE SELECT 1;

Re: [HACKERS] Question about optimising (Postgres_)FDW

2014-04-17 Thread Etsuro Fujita
(2014/04/16 22:16), Hannu Krosing wrote: On 04/16/2014 01:35 PM, Etsuro Fujita wrote: Maybe I'm missing something, but I think that you can do what I think you'd like to do by the following procedure: No, what I'd like PostgreSQL to do is to 1. select the id+set from local table 2. select

[HACKERS] New windows compiler warning from 585bca39

2014-04-17 Thread David Rowley
I've attached a tiny patch that fixes a new compiler warning on the windows build... Perhaps the #ifndef could be placed in a nicer spot in the patch, but the attached should at least describe where the problem lies... (ClCompile target) - src\interfaces\libpq\fe-connect.c(3365): warning

Re: [HACKERS] slow startup due to LWLockAssign() spinlock

2014-04-17 Thread Andres Freund
On 2014-04-16 19:33:52 -0400, Bruce Momjian wrote: On Tue, Feb 4, 2014 at 12:58:49AM +0100, Andres Freund wrote: On 2014-02-03 11:22:45 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On larger, multi-socket, machines, startup takes a fair bit of time. As I was

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2014-04-17 Thread Andres Freund
On 2014-04-16 19:18:02 -0400, Bruce Momjian wrote: On Thu, Feb 6, 2014 at 09:40:32AM +0100, Andres Freund wrote: On 2014-02-05 12:36:42 -0500, Robert Haas wrote: It may well be that your proposal is spot on. But I'd like to see some data-structure-by-data-structure measurements,

Re: [HACKERS] [COMMITTERS] pgsql: Include planning time in EXPLAIN ANALYZE output.

2014-04-17 Thread Andreas Karlsson
On 04/17/2014 01:35 AM, Tom Lane wrote: I'll go change it. Thanks for fixing this. The new name Execution time is much clearer. -- Andreas Karlsson -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Patch: iff - if

2014-04-17 Thread Nicolas Barbier
2014-04-17 Michael Paquier michael.paqu...@gmail.com: Is there no equivalent in German? For example in French there is ssi. gdw (genau dann, wenn) Nicolas -- A. Because it breaks the logical sequence of discussion. Q. Why is top posting bad? -- Sent via pgsql-hackers mailing list

Re: [HACKERS] How can we make beta testing better?

2014-04-17 Thread Greg Stark
On Wed, Apr 16, 2014 at 12:53 AM, Rod Taylor rod.tay...@gmail.com wrote: 4) Plays queries from the CSV logs starting from $TIME mimicking actual timing and transaction boundaries This ^^ But I recall a number of previous attempts including plugins for general load testing systems, what

Re: [HACKERS] Buildfarm master-next branch? (was: Dynamic Shared Memory stuff)

2014-04-17 Thread Robert Haas
On Wed, Apr 16, 2014 at 8:24 PM, Craig Ringer cr...@2ndquadrant.com wrote: On 04/17/2014 12:08 AM, Robert Haas wrote: On Tue, Apr 15, 2014 at 10:46 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Wed, Apr 16, 2014 at 3:01 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Apr 15, 2014 at

Re: [HACKERS] Need Multixact Freezing Docs

2014-04-17 Thread Robert Haas
On Wed, Apr 16, 2014 at 4:39 PM, Josh Berkus j...@agliodbs.com wrote: Hmm, are you sure it's INT_MAX and not 4244967297? Heikki reported that: http://www.postgresql.org/message-id/52401aea.9000...@vmware.com The absolute value is not important; I think that's mostly harmless. I don't think

Re: [HACKERS] Minor performance improvement in transition to external sort

2014-04-17 Thread Robert Haas
On Wed, Apr 16, 2014 at 7:38 PM, Bruce Momjian br...@momjian.us wrote: On Thu, Apr 10, 2014 at 06:03:15PM +0100, Simon Riggs wrote: On 6 February 2014 18:21, Jeff Janes jeff.ja...@gmail.com wrote: On Tue, Feb 4, 2014 at 2:22 PM, Jeremy Harris j...@wizmail.org wrote: The attached patch

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Greg Stark
On Wed, Apr 16, 2014 at 12:44 AM, Robert Haas robertmh...@gmail.com wrote: This isn't a fundamental property of the usage-count idea; it's an artifact of the fact that usage count decreases are tied to eviction pressure rather than access pressure. For example, suppose we made a rule that if

Re: [HACKERS] four minor proposals for 9.5

2014-04-17 Thread Pavel Stehule
2014-04-17 7:12 GMT+02:00 Amit Kapila amit.kapil...@gmail.com: On Mon, Apr 14, 2014 at 6:27 PM, Robert Haas robertmh...@gmail.com wrote: I agree. I don't think the idea of pushing this into the log_line_prefix stuff as a one-off is a very good one. Sure, we could wedge it in there, but

Re: [HACKERS] BGWorkers, shared memory pointers, and postmaster restart

2014-04-17 Thread Robert Haas
On Wed, Apr 16, 2014 at 8:46 PM, Craig Ringer cr...@2ndquadrant.com wrote: On 04/17/2014 12:16 AM, Robert Haas wrote: On Wed, Apr 16, 2014 at 7:11 AM, Craig Ringer cr...@2ndquadrant.com wrote: - A flag like BGW_UNREGISTER_ON_RESTART; I would be OK with this, maybe modulo the name. - To

Re: [HACKERS] four minor proposals for 9.5

2014-04-17 Thread Tom Lane
Pavel Stehule pavel.steh...@gmail.com writes: We can introduce new feature without hard dependency on CSV format Look, the long and the short of it is that there is not consensus that this measurement is worth creating a new CSV log column for. And from that, there is also not consensus that

Re: [HACKERS] assertion failure 9.3.4

2014-04-17 Thread Andrew Dunstan
On 04/16/2014 10:28 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 04/16/2014 07:19 PM, Tom Lane wrote: Yeah, it would be real nice to see a self-contained test case for this. Well, that might be hard to put together, but I did try running without pg_stat_statements and

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Robert Haas
On Thu, Apr 17, 2014 at 9:40 AM, Greg Stark st...@mit.edu wrote: On Wed, Apr 16, 2014 at 12:44 AM, Robert Haas robertmh...@gmail.com wrote: This isn't a fundamental property of the usage-count idea; it's an artifact of the fact that usage count decreases are tied to eviction pressure rather

Re: [HACKERS] Buildfarm master-next branch?

2014-04-17 Thread Andrew Dunstan
On 04/17/2014 09:17 AM, Robert Haas wrote: In terms of improving the buildfarm infrastructure, the thing I would most like to have is more frequent runs. It would be great if pushing a commit to the master repository triggered an immediate build on every buildfarm animal so that you could

Re: [HACKERS] Verbose output of pg_dump not show schema name

2014-04-17 Thread Fabrízio de Royes Mello
On Thu, Apr 17, 2014 at 2:14 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Apr 17, 2014 at 11:41 AM, Fabrízio de Royes Mello fabriziome...@gmail.com wrote: Hi all, There are some reason to verbose output of pg_dump don't show schema name? A output example of using

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Bruce Momjian
On Thu, Apr 17, 2014 at 10:18:43AM -0400, Robert Haas wrote: I also believe this to be the case on first principles and my own experiments. Suppose you have a workload that fits inside shared_buffers. All of the usage counts will converge to 5. Then, somebody accesses a table that is not

Re: [HACKERS] Verbose output of pg_dump not show schema name

2014-04-17 Thread Bruce Momjian
On Thu, Apr 17, 2014 at 11:29:03AM -0300, Fabrízio de Royes Mello wrote: This database have a lot of different schemas with same structure and if I need do view the status of dump I don't know what schema the table are dump from. Yes this may be helpful. The attached quick'n dirty

Re: [HACKERS] Buildfarm master-next branch?

2014-04-17 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 04/17/2014 09:17 AM, Robert Haas wrote: In terms of improving the buildfarm infrastructure, the thing I would most like to have is more frequent runs. IMO the best single thing that could happen for the buildfarm is if we had more critters (at

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Robert Haas
On Thu, Apr 17, 2014 at 10:32 AM, Bruce Momjian br...@momjian.us wrote: On Thu, Apr 17, 2014 at 10:18:43AM -0400, Robert Haas wrote: I also believe this to be the case on first principles and my own experiments. Suppose you have a workload that fits inside shared_buffers. All of the usage

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Bruce Momjian
On Thu, Apr 17, 2014 at 10:40:40AM -0400, Robert Haas wrote: On Thu, Apr 17, 2014 at 10:32 AM, Bruce Momjian br...@momjian.us wrote: On Thu, Apr 17, 2014 at 10:18:43AM -0400, Robert Haas wrote: I also believe this to be the case on first principles and my own experiments. Suppose you have

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Robert Haas
On Thu, Apr 17, 2014 at 10:48 AM, Bruce Momjian br...@momjian.us wrote: I understand now. If there is no memory pressure, every buffer gets the max usage count, and when a new buffer comes in, it isn't the max so it is swiftly removed until the clock sweep has time to decrement the old

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Andres Freund
On 2014-04-17 10:48:15 -0400, Bruce Momjian wrote: On Thu, Apr 17, 2014 at 10:40:40AM -0400, Robert Haas wrote: That can happen, but the real problem I was trying to get at is that when all the buffers get up to max usage count, they all appear equally important. But in reality they're

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Greg Stark
On Thu, Apr 17, 2014 at 10:18 AM, Robert Haas robertmh...@gmail.com wrote: Because all the usage counts are the same, the eviction at this point is completely indiscriminate. We're just as likely to kick out a btree root page or a visibility map page as we are to kick out a random heap page,

Re: [HACKERS] Verbose output of pg_dump not show schema name

2014-04-17 Thread Fabrízio de Royes Mello
On Thu, Apr 17, 2014 at 11:36 AM, Bruce Momjian br...@momjian.us wrote: On Thu, Apr 17, 2014 at 11:29:03AM -0300, Fabrízio de Royes Mello wrote: This database have a lot of different schemas with same structure and if I need do view the status of dump I don't know what schema the table

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Greg Stark
On Tue, Apr 15, 2014 at 7:30 PM, Peter Geoghegan p...@heroku.com wrote: Frankly, there doesn't need to be any research on this, because it's just common sense that probabilistically, leaf pages are much more useful than heap pages in servicing index scan queries if we assume a uniform

Re: [HACKERS] Verbose output of pg_dump not show schema name

2014-04-17 Thread Bruce Momjian
On Thu, Apr 17, 2014 at 12:07:39PM -0300, Fabrízio de Royes Mello wrote: Can you get that to _conditionally_ double-quote the strings?  Sorry, I didn't understand what you means? Your idea is to check if the namespace is available and then don't show the double-quote, is that? The idea is

Re: [HACKERS] [GENERAL] pg_upgrade tablespaces

2014-04-17 Thread Bruce Momjian
On Wed, Apr 16, 2014 at 01:49:20PM -0400, Bruce Momjian wrote: On Sun, Jan 12, 2014 at 11:04:41PM -0500, Bruce Momjian wrote: In the pgsql_old installation you have symlinks pointing back to the current default location. As well pg_tablespace points back to /usr/local/pgsql/data/ The

Re: [HACKERS] Verbose output of pg_dump not show schema name

2014-04-17 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: The idea is that we only need quotes when there are odd characters in the identifier. We do that right now in some places, though I can't find them in pg_dump. I know psql does that, see quote_ident(). I think our general style rule is that identifiers

Re: [HACKERS] Verbose output of pg_dump not show schema name

2014-04-17 Thread Bruce Momjian
On Thu, Apr 17, 2014 at 11:44:37AM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: The idea is that we only need quotes when there are odd characters in the identifier. We do that right now in some places, though I can't find them in pg_dump. I know psql does that, see

Re: [HACKERS] shouldn't we log permission errors when accessing the configured trigger file?

2014-04-17 Thread Bruce Momjian
On Wed, Apr 16, 2014 at 06:55:14PM -0400, Bruce Momjian wrote: Seems reasonable. It could lead to quite a bit of log spam, I suppose, but the way things are now could be pretty mystifying if you've located your trigger file somewhere outside $PGDATA, and a parent directory

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: several orders of magnitude more often. That's clearly bad. On systems that are not too heavily loaded it doesn't matter too much because we just fault the page right back in from the OS pagecache. Ehhh. No. If it's a hot page that we've been

Re: [HACKERS] Fix memset usage in pgcrypto

2014-04-17 Thread Bruce Momjian
On Mon, Mar 31, 2014 at 09:03:41PM -0400, Bruce Momjian wrote: On Thu, Dec 26, 2013 at 03:42:12PM +0200, Marko Kreen wrote: http://www.viva64.com/en/b/0227/ reported that on-stack memset()s might be optimized away by compilers. Fix it. * Replace memset() with px_memset() * Add

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-04-17 Thread Bruce Momjian
On Mon, Feb 10, 2014 at 06:40:30PM +, Peter Geoghegan wrote: On Sun, Jan 19, 2014 at 2:17 AM, Peter Geoghegan p...@heroku.com wrote: I'm just throwing an error when locking the tuple returns HeapTupleInvisible, and the xmin of the tuple is our xid. I would like some feedback on this

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Greg Stark
On Thu, Apr 17, 2014 at 12:21 PM, Stephen Frost sfr...@snowman.net wrote: Ehhh. No. If it's a hot page that we've been holding in *our* cache long enough, the kernel will happily evict it as 'cold' from *its* cache, leading to... This is a whole nother problem. It is worrisome that we

Re: [HACKERS] WAL replay bugs

2014-04-17 Thread Heikki Linnakangas
On 04/08/2014 06:41 AM, Michael Paquier wrote: On Tue, Apr 8, 2014 at 3:16 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I've been playing with a little hack that records a before and after image of every page modification that is WAL-logged, and writes the images to a file along with

Re: [HACKERS] WAL replay bugs

2014-04-17 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: Two things that are not bugs, but I'd like to change just to make this tool easier to maintain, and to generally clean things up: 1. When creating a sequence, we first use simple_heap_insert() to insert the sequence tuple, which creates a

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Peter Geoghegan
On Thu, Apr 17, 2014 at 9:21 AM, Stephen Frost sfr...@snowman.net wrote: * Robert Haas (robertmh...@gmail.com) wrote: several orders of magnitude more often. That's clearly bad. On systems that are not too heavily loaded it doesn't matter too much because we just fault the page right back in

Re: [HACKERS] How can we make beta testing better?

2014-04-17 Thread Josh Berkus
On 04/17/2014 05:39 AM, Greg Stark wrote: On Wed, Apr 16, 2014 at 12:53 AM, Rod Taylor rod.tay...@gmail.com wrote: 4) Plays queries from the CSV logs starting from $TIME mimicking actual timing and transaction boundaries This ^^ But I recall a number of previous attempts including plugins

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Stephen Frost
* Greg Stark (st...@mit.edu) wrote: On Thu, Apr 17, 2014 at 12:21 PM, Stephen Frost sfr...@snowman.net wrote: Ehhh. No. If it's a hot page that we've been holding in *our* cache long enough, the kernel will happily evict it as 'cold' from *its* cache, leading to... This is a whole

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-04-17 Thread Peter Geoghegan
On Thu, Apr 17, 2014 at 9:52 AM, Bruce Momjian br...@momjian.us wrote: Where are we on this? My hope is that I can get agreement on a way forward during pgCon. Or, at the very least, explain the issues as I see them in a relatively accessible and succinct way to those interested. -- Peter

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Andres Freund
On 2014-04-17 21:44:47 +0300, Heikki Linnakangas wrote: On 04/17/2014 09:38 PM, Stephen Frost wrote: * Greg Stark (st...@mit.edu) wrote: On Thu, Apr 17, 2014 at 12:21 PM, Stephen Frost sfr...@snowman.net wrote: Ehhh. No. If it's a hot page that we've been holding in *our* cache long enough,

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: Note that if we somehow come up with a page replacement algorithm that tends to evict pages that are in the OS cache, we have effectively solved the double buffering problem. When a page is cached in both caches, evicting it from one of them

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Heikki Linnakangas
On 04/17/2014 09:38 PM, Stephen Frost wrote: * Greg Stark (st...@mit.edu) wrote: On Thu, Apr 17, 2014 at 12:21 PM, Stephen Frost sfr...@snowman.net wrote: Ehhh. No. If it's a hot page that we've been holding in *our* cache long enough, the kernel will happily evict it as 'cold' from *its*

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Merlin Moncure
On Thu, Apr 17, 2014 at 1:48 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-04-17 21:44:47 +0300, Heikki Linnakangas wrote: On 04/17/2014 09:38 PM, Stephen Frost wrote: * Greg Stark (st...@mit.edu) wrote: On Thu, Apr 17, 2014 at 12:21 PM, Stephen Frost sfr...@snowman.net wrote: Ehhh.

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Peter Geoghegan
On Thu, Apr 17, 2014 at 11:53 AM, Merlin Moncure mmonc...@gmail.com wrote: No. but if you were very judicious, maybe you could hint the o/s (posix_fadvise) about pages that are likely to stay hot that you don't need them. Mitsumasa KONDO wrote a patch like that. I don't think the results were

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Stephen Frost
* Merlin Moncure (mmonc...@gmail.com) wrote: I doubt that's necessary though -- if the postgres caching algorithm improves such that there is a better tendency for hot pages to stay in s_b, Eventually the O/S will deschedule the page for something else that needs it. In other words,

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Merlin Moncure
On Thu, Apr 17, 2014 at 2:00 PM, Stephen Frost sfr...@snowman.net wrote: * Merlin Moncure (mmonc...@gmail.com) wrote: I doubt that's necessary though -- if the postgres caching algorithm improves such that there is a better tendency for hot pages to stay in s_b, Eventually the O/S will

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: I wonder if it would help to actually tell the OS to read in buffers that we're *evicting*... On the general notion that if the OS already has them buffered then it's almost a no-op, and if it doesn't and it's actually a 'hot' buffer that we're gonna

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Stephen Frost
* Merlin Moncure (mmonc...@gmail.com) wrote: I don't think this would work unless we would keep some kind of tracking information on the page itself which seems not worth a write operation to do (maybe if the page is dirtied it could be snuck in there though...). IOW, it would only make

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: I wonder if it would help to actually tell the OS to read in buffers that we're *evicting*... On the general notion that if the OS already has them buffered then it's almost a no-op, and if it doesn't and it's

Re: [HACKERS] How can we make beta testing better?

2014-04-17 Thread Merlin Moncure
On Tue, Apr 15, 2014 at 4:47 PM, Josh Berkus j...@agliodbs.com wrote: Hackers, I think 9.3 has given us evidence that our users aren't giving new versions of PostgreSQL substantial beta testing, or if they are, they aren't sharing the results with us. How can we make beta testing better and

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Merlin Moncure
On Thu, Apr 17, 2014 at 2:16 PM, Stephen Frost sfr...@snowman.net wrote: * Merlin Moncure (mmonc...@gmail.com) wrote: I don't think this would work unless we would keep some kind of tracking information on the page itself which seems not worth a write operation to do (maybe if the page is

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Stephen Frost
On Thursday, April 17, 2014, Merlin Moncure mmonc...@gmail.com wrote: yeah -- the thing is, we are already too spendy already on supplemental write i/o (hint bits, visible bits, freezing, etc) and likely not worth it to throw something else on the pile unless the page is already dirty; the

Re: [HACKERS] Verbose output of pg_dump not show schema name

2014-04-17 Thread Fabrízio de Royes Mello
On Thu, Apr 17, 2014 at 12:46 PM, Bruce Momjian br...@momjian.us wrote: On Thu, Apr 17, 2014 at 11:44:37AM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: The idea is that we only need quotes when there are odd characters in the identifier. We do that right now in some

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Merlin Moncure
On Thu, Apr 17, 2014 at 2:28 PM, Stephen Frost sfr...@snowman.net wrote: On Thursday, April 17, 2014, Merlin Moncure mmonc...@gmail.com wrote: yeah -- the thing is, we are already too spendy already on supplemental write i/o (hint bits, visible bits, freezing, etc) and likely not worth it

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Stephen Frost
On Thursday, April 17, 2014, Merlin Moncure mmonc...@gmail.com wrote: no -- I got you. My point was, that's a pure guess unless you base it on evidence recorded on the page itself. Without that evidence, (which requires writing) the operating is in a a better place to make that guess so it's

Re: [HACKERS] New windows compiler warning from 585bca39

2014-04-17 Thread Tom Lane
David Rowley dgrowle...@gmail.com writes: I've attached a tiny patch that fixes a new compiler warning on the windows build... Applied, thanks. Perhaps the #ifndef could be placed in a nicer spot in the patch, but the attached should at least describe where the problem lies... Yeah, I

[HACKERS] assertion in 9.4 with wal_level=logical

2014-04-17 Thread Steve Singer
With master/9.4 from today (52e757420fa98a76015c2c88432db94269f3e8f4) I am getting an assertion when doing a truncate via SPI when I have wal_level=logical. Stack trace is below. I am just replicating a table with normal slony (2.2) I don't need to establish any replication slots to get

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Peter Geoghegan
On Thu, Apr 17, 2014 at 8:10 AM, Greg Stark st...@mit.edu wrote: I don't think common sense is compelling. I think you need to pin down exactly what it is about btree intermediate pages that the LRU isn't capturing and not just argue they're more useful. The LRU is already capturing which

Re: [HACKERS] assertion in 9.4 with wal_level=logical

2014-04-17 Thread Andres Freund
Hi, On 2014-04-17 16:23:54 -0400, Steve Singer wrote: With master/9.4 from today (52e757420fa98a76015c2c88432db94269f3e8f4) I am getting an assertion when doing a truncate via SPI when I have wal_level=logical. Stack trace is below. I am just replicating a table with normal slony (2.2)

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Andres Freund
On 2014-04-17 13:33:27 -0700, Peter Geoghegan wrote: Just over 99.6% of pages (leaving aside the meta page) in the big 10 GB pgbench_accounts_pkey index are leaf pages. That's a rather nice number. I knew it was big, but I'd have guessed it'd be a percent lower. Do you happen to have the same

Re: [HACKERS] assertion in 9.4 with wal_level=logical

2014-04-17 Thread Alvaro Herrera
Steve Singer wrote: With master/9.4 from today (52e757420fa98a76015c2c88432db94269f3e8f4) I am getting an assertion when doing a truncate via SPI when I have wal_level=logical. Stack trace is below. I am just replicating a table with normal slony (2.2) I don't need to establish any

[HACKERS] DISCARD ALL (Again)

2014-04-17 Thread Joshua D. Drake
Hello, Over at my quaint establishment we have been working on some plython work that makes use of GD. We wrote this code with the assumption (per the docs) that when you issued a DISCARD ALL, the GD would be cleared. Apparently this is not the case. The docs themselves are clearly wrong,

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Peter Geoghegan
On Thu, Apr 17, 2014 at 1:39 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-04-17 13:33:27 -0700, Peter Geoghegan wrote: Just over 99.6% of pages (leaving aside the meta page) in the big 10 GB pgbench_accounts_pkey index are leaf pages. That's a rather nice number. I knew it was big,

Re: [HACKERS] assertion in 9.4 with wal_level=logical

2014-04-17 Thread Andres Freund
On 2014-04-17 17:40:01 -0300, Alvaro Herrera wrote: For once, this looks more like a problem in logical decoding, which is trying to assert about the tuple being updated; the assertion failing is the one added a week ago about not palloc'ing in a critical section. It's this (older) assertion

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Peter Geoghegan
On Thu, Apr 17, 2014 at 1:33 PM, Peter Geoghegan p...@heroku.com wrote: I can't imagine that this is much of a problem in practice. Although I will add that not caching highly useful inner pages for the medium term, because that index isn't being used at all for 5 minutes probably is very bad.

Re: [HACKERS] assertion in 9.4 with wal_level=logical

2014-04-17 Thread Steve Singer
On 04/17/2014 04:33 PM, Andres Freund wrote: Hi, On 2014-04-17 16:23:54 -0400, Steve Singer wrote: With master/9.4 from today (52e757420fa98a76015c2c88432db94269f3e8f4) I am getting an assertion when doing a truncate via SPI when I have wal_level=logical. Stack trace is below. I am just

Re: [HACKERS] DISCARD ALL (Again)

2014-04-17 Thread Josh Berkus
On 04/17/2014 01:44 PM, Joshua D. Drake wrote: Does it seem reasonable based on the docs: DISCARD ALL: Releases all temporary resources associated with the current session and resets the session to its initial state. That we should also release the GD? It does, but that's a feature

Re: [HACKERS] Patch: iff - if

2014-04-17 Thread Andreas 'ads' Scherbaum
On 04/17/2014 12:33 PM, Nicolas Barbier wrote: 2014-04-17 Michael Paquier michael.paqu...@gmail.com: Is there no equivalent in German? For example in French there is ssi. gdw (genau dann, wenn) More likely that you see \equiv or: \leftrightarrow Regards, --

Re: [HACKERS] How can we make beta testing better?

2014-04-17 Thread Jeff Janes
On Tue, Apr 15, 2014 at 2:47 PM, Josh Berkus j...@agliodbs.com wrote: Hackers, I think 9.3 has given us evidence that our users aren't giving new versions of PostgreSQL substantial beta testing, or if they are, they aren't sharing the results with us. A lot of the bugs that turned up are

Re: [HACKERS] How can we make beta testing better?

2014-04-17 Thread Jan Wieck
On 04/17/14 15:16, Merlin Moncure wrote: On Tue, Apr 15, 2014 at 4:47 PM, Josh Berkus j...@agliodbs.com wrote: Hackers, I think 9.3 has given us evidence that our users aren't giving new versions of PostgreSQL substantial beta testing, or if they are, they aren't sharing the results with us.

Re: [HACKERS] DISCARD ALL (Again)

2014-04-17 Thread Joshua D. Drake
On 04/17/2014 02:17 PM, Josh Berkus wrote: On 04/17/2014 01:44 PM, Joshua D. Drake wrote: Does it seem reasonable based on the docs: DISCARD ALL: Releases all temporary resources associated with the current session and resets the session to its initial state. That we should also release

Re: [HACKERS] DISCARD ALL (Again)

2014-04-17 Thread Alvaro Herrera
Joshua D. Drake wrote: On 04/17/2014 02:17 PM, Josh Berkus wrote: On 04/17/2014 01:44 PM, Joshua D. Drake wrote: Does it seem reasonable based on the docs: DISCARD ALL: Releases all temporary resources associated with the current session and resets the session to its initial state.

Re: [HACKERS] DISCARD ALL (Again)

2014-04-17 Thread David G Johnston
Alvaro Herrera-9 wrote Are we going to backpatch a doc change that says releases all temporary resources, except for plptyhon's and plperl's GD? Surely not ... GD = Global Dictionary I don't see why something like the following wouldn't have value. For those languages that make use of a

Re: [HACKERS] DISCARD ALL (Again)

2014-04-17 Thread Fabrízio de Royes Mello
On Thu, Apr 17, 2014 at 6:51 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: It does sounds a legitimate feature request to me. I don't remember if we honored the request to add resetting of cached sequences, though; if we didn't, this one is probably going to be tough too. +1 Another

Re: [HACKERS] How can we make beta testing better?

2014-04-17 Thread Josh Berkus
On 04/15/2014 09:53 PM, Rod Taylor wrote: A documented beta test process/toolset which does the following would help: 1) Enables full query logging 2) Creates a replica of a production DB, record $TIME when it stops. 3) Allow user to make changes (upgrade to 9.4, change hardware, change

Re: [HACKERS] assertion failure 9.3.4

2014-04-17 Thread Josh Berkus
All, So have encountered a 2nd report of this issue, or of an issue which sounds very similar: - corruption in two queue tables - the tables are written in a high-concurrency, lock-contested environment - user uses SELECT FOR UPDATE with these tables. - pg_stat_statements .so is loaded, but

Re: [HACKERS] Verbose output of pg_dump not show schema name

2014-04-17 Thread Michael Paquier
On Fri, Apr 18, 2014 at 4:29 AM, Fabrízio de Royes Mello fabriziome...@gmail.com wrote: On Thu, Apr 17, 2014 at 12:46 PM, Bruce Momjian br...@momjian.us wrote: On Thu, Apr 17, 2014 at 11:44:37AM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: The idea is that we only need

Re: [HACKERS] DISCARD ALL (Again)

2014-04-17 Thread Tom Lane
Joshua D. Drake j...@commandprompt.com writes: Does it seem reasonable based on the docs: DISCARD ALL: Releases all temporary resources associated with the current session and resets the session to its initial state. That we should also release the GD? There are a couple of reasons why this

Re: [HACKERS] DISCARD ALL (Again)

2014-04-17 Thread Joshua D. Drake
On 04/17/2014 05:24 PM, Tom Lane wrote: Joshua D. Drake j...@commandprompt.com writes: Does it seem reasonable based on the docs: DISCARD ALL: Releases all temporary resources associated with the current session and resets the session to its initial state. That we should also release the GD?

Re: [HACKERS] DISCARD ALL (Again)

2014-04-17 Thread Tom Lane
Joshua D. Drake j...@commandprompt.com writes: On 04/17/2014 05:24 PM, Tom Lane wrote: 1. The things that DISCARD ALL resets are explicitly enumerated in its documentation page; it is not an open-ended promise to clean up anything anybody happens to think of. Actually, it is. Unless we

Re: [HACKERS] assertion failure 9.3.4

2014-04-17 Thread Peter Geoghegan
On Thu, Apr 17, 2014 at 7:15 AM, Andrew Dunstan and...@dunslane.net wrote: track_activity_query_size = 10240 shared_preload_libraries = 'auto_explain,pg_stat_statements' As you can see, auto_explain's log_min_duration hasn't been set, so it shouldn't be doing anything very much, I should

Re: [HACKERS] Buildfarm master-next branch?

2014-04-17 Thread Craig Ringer
On 04/17/2014 10:38 PM, Tom Lane wrote: IMO the best single thing that could happen for the buildfarm is if we had more critters (at least twice as many) running a wider variety of platforms, compilers, and configuration options than there are today. More frequent runs would come out of that

Re: [HACKERS] How can we make beta testing better?

2014-04-17 Thread Greg Stark
On Thu, Apr 17, 2014 at 5:26 PM, Jeff Janes jeff.ja...@gmail.com wrote: A lot of the bugs that turned up are not the kind I would expect to have been found in most beta testing done by non-hacking users. Weren't they mostly around rare race conditions, crash recovery, and freezing? Actually I

Re: [HACKERS] [bug fix] pg_ctl always uses the same event source

2014-04-17 Thread Amit Kapila
On Sat, Apr 12, 2014 at 1:21 PM, MauMau maumau...@gmail.com wrote: Hello, Amit san, Tom san, I'm sorry for my late response. I've just caught up with the discussion. I'm almost convinced. Please find attached the revised patch. I'd like to follow the idea of adding a switch to pg_ctl.

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Greg Stark
On Thu, Apr 17, 2014 at 4:48 PM, Peter Geoghegan p...@heroku.com wrote: Although I will add that not caching highly useful inner pages for the medium term, because that index isn't being used at all for 5 minutes probably is very bad. Using the 4,828 buffers that it would take to store all the

Re: [HACKERS] Clock sweep not caching enough B-Tree leaf pages?

2014-04-17 Thread Peter Geoghegan
On Thu, Apr 17, 2014 at 6:50 PM, Greg Stark st...@mit.edu wrote: On Thu, Apr 17, 2014 at 4:48 PM, Peter Geoghegan p...@heroku.com wrote: Although I will add that not caching highly useful inner pages for the medium term, because that index isn't being used at all for 5 minutes probably is very

Re: [HACKERS] DISCARD ALL (Again)

2014-04-17 Thread David G Johnston
On 04/17/2014 05:24 PM, Tom Lane wrote: On the whole I'm not sure this is something we ought to get into. If you really need a fresh session, maybe you should start a fresh session. Isn't the whole point to avoid the reconnection overhead, especially for connection poolers? DISCARD ALL

Re: [HACKERS] DISCARD ALL (Again)

2014-04-17 Thread Stephen Frost
* David G Johnston (david.g.johns...@gmail.com) wrote: On 04/17/2014 05:24 PM, Tom Lane wrote: On the whole I'm not sure this is something we ought to get into. If you really need a fresh session, maybe you should start a fresh session. Isn't the whole point to avoid the reconnection

Re: [HACKERS] DISCARD ALL (Again)

2014-04-17 Thread Joshua D. Drake
On 04/17/2014 07:07 PM, David G Johnston wrote: On 04/17/2014 05:24 PM, Tom Lane wrote: On the whole I'm not sure this is something we ought to get into. If you really need a fresh session, maybe you should start a fresh session. Isn't the whole point to avoid the

Re: [HACKERS] DISCARD ALL (Again)

2014-04-17 Thread David Johnston
On Thursday, April 17, 2014, Joshua D. Drake j...@commandprompt.com wrote: On 04/17/2014 07:07 PM, David G Johnston wrote: On 04/17/2014 05:24 PM, Tom Lane wrote: On the whole I'm not sure this is something we ought to get into. If you really need a fresh session, maybe you

Re: [HACKERS] assertion failure 9.3.4

2014-04-17 Thread Andrew Dunstan
On 04/17/2014 09:04 PM, Peter Geoghegan wrote: On Thu, Apr 17, 2014 at 7:15 AM, Andrew Dunstan and...@dunslane.net wrote: track_activity_query_size = 10240 shared_preload_libraries = 'auto_explain,pg_stat_statements' As you can see, auto_explain's log_min_duration hasn't been set, so it