Re: [HACKERS] [WIP] Better partial index-only scans

2014-03-17 Thread Joshua Yanovski
Here's a SQL script that (1) demonstrates the new index only scan functionality, and (2) at least on my machine, has a consistently higher planning time for the version with my change than without it. On Sun, Mar 16, 2014 at 5:08 AM, Joshua Yanovski pythones...@gmail.com wrote: Proof of concept

Re: [HACKERS] Fix typo in nbtree.h introduced by efada2b

2014-03-17 Thread Magnus Hagander
On Mon, Mar 17, 2014 at 6:06 AM, Michael Paquier michael.paqu...@gmail.comwrote: Hi, I found a small typo in nbtree.h, introduced by commit efada2b. Patch is attached. Applied, thanks. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/

Re: [HACKERS] inherit support for foreign tables

2014-03-17 Thread Kyotaro HORIGUCHI
Hi Fujita-san, Thank you for working this patch! No problem, but my point seems always out of the main target a bit:( | =# alter table passwd add column added int, add column added2 int; | NOTICE: This command affects foreign relation cf1 | NOTICE: This command affects foreign relation

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread KONDO Mitsumasa
Hi Heikki-san, (2014/03/17 14:39), KONDO Mitsumasa wrote: (2014/03/15 15:53), Fabien COELHO wrote: Hello Heikki, A couple of comments: * There should be an explicit \setrandom ... uniform option too, even though you get that implicitly if you don't specify the distribution Fix. We can use

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Heikki Linnakangas
On 03/15/2014 08:53 AM, Fabien COELHO wrote: * Does min and max really make sense for gaussian and exponential distributions? For gaussian, I would expect mean and standard deviation as the parameters, not min/max/threshold. Yes... and no:-) The aim is to draw an integer primary key from a

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Heikki Linnakangas
On 03/17/2014 10:40 AM, KONDO Mitsumasa wrote: By the way, you seem to want to remove --gaussian=NUM and --exponential=NUM command options. Can you tell me the objective reason? I think pgbench is the benchmark test on PostgreSQL and default benchmark is TPC-B-like benchmark. It is written in

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread KONDO Mitsumasa
(2014/03/17 17:46), Heikki Linnakangas wrote: On 03/15/2014 08:53 AM, Fabien COELHO wrote: * Does min and max really make sense for gaussian and exponential distributions? For gaussian, I would expect mean and standard deviation as the parameters, not min/max/threshold. Yes... and no:-) The

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread KONDO Mitsumasa
(2014/03/17 18:02), Heikki Linnakangas wrote: On 03/17/2014 10:40 AM, KONDO Mitsumasa wrote: By the way, you seem to want to remove --gaussian=NUM and --exponential=NUM command options. Can you tell me the objective reason? I think pgbench is the benchmark test on PostgreSQL and default

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Fujii Masao
On Mon, Mar 17, 2014 at 7:07 PM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: (2014/03/17 18:02), Heikki Linnakangas wrote: On 03/17/2014 10:40 AM, KONDO Mitsumasa wrote: By the way, you seem to want to remove --gaussian=NUM and --exponential=NUM command options. Can you tell me the

Re: [HACKERS] Patch: show relation and tuple infos of a lock to acquire

2014-03-17 Thread Christian Kruse
Hi Amit, I've been ill the last few days, so sorry for my late response. I have updated the patch to pass TID and operation information in error context and changed some of the comments in code. Let me know if the added operation information is useful, else we can use better generic message

[HACKERS] Various typos

2014-03-17 Thread Thom Brown
Attached is a small patch to fix various typos. Regards Thom diff --git a/contrib/pgcrypto/openssl.c b/contrib/pgcrypto/openssl.c index ad7fb9e..86c0fb0 100644 --- a/contrib/pgcrypto/openssl.c +++ b/contrib/pgcrypto/openssl.c @@ -429,7 +429,7 @@ bf_init(PX_Cipher *c, const uint8 *key, unsigned

Re: [HACKERS] Changeset Extraction v7.9.1

2014-03-17 Thread Robert Haas
On Wed, Mar 12, 2014 at 2:06 PM, Andres Freund and...@2ndquadrant.com wrote: Attached are the collected remaining patches. The docs might need further additions, but it seems better to add them now. A few questions about pg_recvlogical: - There doesn't seem to be any provision for this tool to

Re: [HACKERS] Changeset Extraction v7.9.1

2014-03-17 Thread Andres Freund
On 2014-03-17 06:55:28 -0400, Robert Haas wrote: On Wed, Mar 12, 2014 at 2:06 PM, Andres Freund and...@2ndquadrant.com wrote: Attached are the collected remaining patches. The docs might need further additions, but it seems better to add them now. A few questions about pg_recvlogical: -

[HACKERS] pg_dump without explicit table locking

2014-03-17 Thread Jürgen Strobel
Hi, at work at my company I inherited responsibility for a large PG 8.1 DB, with a an extreme number of tables (~30). Surprisingly this is working quite well, except for maintenance and backup. I am tasked with finding a way to do dump restore to 9.3 with as little downtime as possible.

Re: [HACKERS] Various typos

2014-03-17 Thread Fujii Masao
On Mon, Mar 17, 2014 at 7:52 PM, Thom Brown t...@linux.com wrote: Attached is a small patch to fix various typos. Thanks! Committed. Regards, -- Fujii Masao -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Changeset Extraction v7.9.1

2014-03-17 Thread Robert Haas
On Mon, Mar 17, 2014 at 7:27 AM, Andres Freund and...@2ndquadrant.com wrote: - There doesn't seem to be any provision for this tool to ever switch from one output file to the next. That seems like a practical need. One idea would be to have it respond to SIGHUP by reopening the

Re: [HACKERS] pg_dump without explicit table locking

2014-03-17 Thread Robert Haas
On Mon, Mar 17, 2014 at 7:52 AM, Jürgen Strobel juergen...@strobel.info wrote: at work at my company I inherited responsibility for a large PG 8.1 DB, with a an extreme number of tables (~30). Surprisingly this is working quite well, except for maintenance and backup. I am tasked with

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-17 Thread Fujii Masao
On Mon, Mar 17, 2014 at 10:20 AM, Robert Haas robertmh...@gmail.com wrote: On Sun, Mar 16, 2014 at 6:23 AM, MauMau maumau...@gmail.com wrote: The PostgreSQL documentation describes cp (on UNIX/Linux) or copy (on Windows) as an example for archive_command. However, cp/copy does not sync the

Re: [HACKERS] pg_dump without explicit table locking

2014-03-17 Thread Pavel Stehule
2014-03-17 12:52 GMT+01:00 Jürgen Strobel juergen...@strobel.info: Hi, at work at my company I inherited responsibility for a large PG 8.1 DB, with a an extreme number of tables (~30). Surprisingly this is working quite well, except for maintenance and backup. I am tasked with finding

Re: [HACKERS] Changeset Extraction v7.9.1

2014-03-17 Thread Andres Freund
On 2014-03-17 08:00:22 -0400, Robert Haas wrote: Yea. The reason it reports the flush position is that it allows to test sync rep. I don't think other usecases will appreciate frequent fsyncs... Maybe make it optional? Well, as I'm sure you recognize, if you're actually trying to build a

Re: [HACKERS] Changeset Extraction v7.9.1

2014-03-17 Thread Andres Freund
On 2014-03-17 08:00:22 -0400, Robert Haas wrote: On Mon, Mar 17, 2014 at 7:27 AM, Andres Freund and...@2ndquadrant.com wrote: - There doesn't seem to be any provision for this tool to ever switch from one output file to the next. That seems like a practical need. One idea would be to have

Re: [HACKERS] Changeset Extraction v7.9.1

2014-03-17 Thread Robert Haas
On Mon, Mar 17, 2014 at 8:29 AM, Andres Freund and...@2ndquadrant.com wrote: Perhaps there could be a switch for an fsync interval, or something like that. The default could be, say, to fsync every 10 seconds. And if you want to change it, then go ahead; 0 disables. Writing to standard

Re: [HACKERS] Changeset Extraction v7.9.1

2014-03-17 Thread Robert Haas
On Mon, Mar 17, 2014 at 8:58 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-03-17 08:00:22 -0400, Robert Haas wrote: On Mon, Mar 17, 2014 at 7:27 AM, Andres Freund and...@2ndquadrant.com wrote: - There doesn't seem to be any provision for this tool to ever switch from one output

Re: [HACKERS] Changeset Extraction v7.9.1

2014-03-17 Thread Andres Freund
On 2014-03-17 09:13:38 -0400, Robert Haas wrote: On Mon, Mar 17, 2014 at 8:29 AM, Andres Freund and...@2ndquadrant.com wrote: Perhaps there could be a switch for an fsync interval, or something like that. The default could be, say, to fsync every 10 seconds. And if you want to change it,

Re: [HACKERS] Changeset Extraction v7.9.1

2014-03-17 Thread Andres Freund
On 2014-03-17 09:14:51 -0400, Robert Haas wrote: On Mon, Mar 17, 2014 at 8:58 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-03-17 08:00:22 -0400, Robert Haas wrote: On Mon, Mar 17, 2014 at 7:27 AM, Andres Freund and...@2ndquadrant.com wrote: - There doesn't seem to be any

Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-03-17 Thread Fujii Masao
On Sun, Mar 16, 2014 at 7:15 AM, Alexander Korotkov aekorot...@gmail.com wrote: On Sat, Mar 15, 2014 at 11:27 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 03/15/2014 08:40 PM, Fujii Masao wrote: Hi, I executed the following statements in HEAD and 9.3, and compared the size of

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Tom Lane
KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp writes: (2014/03/17 18:02), Heikki Linnakangas wrote: On 03/17/2014 10:40 AM, KONDO Mitsumasa wrote: There is an infinite number of variants of the TPC-B test that we could include in pgbench. If we start adding every one of them, we're quickly

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-17 Thread Mitsumasa KONDO
2014-03-17 21:12 GMT+09:00 Fujii Masao masao.fu...@gmail.com: On Mon, Mar 17, 2014 at 10:20 AM, Robert Haas robertmh...@gmail.com wrote: On Sun, Mar 16, 2014 at 6:23 AM, MauMau maumau...@gmail.com wrote: The PostgreSQL documentation describes cp (on UNIX/Linux) or copy (on Windows) as an

Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-03-17 Thread Alvaro Herrera
Fujii Masao escribió: On Sun, Mar 16, 2014 at 7:15 AM, Alexander Korotkov aekorot...@gmail.com wrote: That could be optimized, but I figured we can live with it, thanks to the fastupdate feature. Fastupdate allows amortizing that cost over several insertions. But of course, you explicitly

Re: [HACKERS] pg_dump without explicit table locking

2014-03-17 Thread Tom Lane
Pavel Stehule pavel.steh...@gmail.com writes: 2014-03-17 12:52 GMT+01:00 Jürgen Strobel juergen...@strobel.info: I've googled the problem and there seem to be more people with similar problems, so I made this a command line option --no-table-locks and wrapped it up in as nice a patch against

Re: [HACKERS] Archive recovery won't be completed on some situation.

2014-03-17 Thread Heikki Linnakangas
On 03/15/2014 05:59 PM, Fujii Masao wrote: What about adding new option into pg_resetxlog so that we can reset the pg_control's backup start location? Even after we've accidentally entered into the situation that you described, we can exit from that by resetting the backup start location in

Re: [HACKERS] Patch: show relation and tuple infos of a lock to acquire

2014-03-17 Thread Amit Kapila
On Mon, Mar 17, 2014 at 4:20 PM, Christian Kruse christ...@2ndquadrant.com wrote: Hi Amit, I've been ill the last few days, so sorry for my late response. Sorry to hear and no problem for delay. I don't think that this fixes the translation guideline issues brought up by Robert. This

Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-03-17 Thread Heikki Linnakangas
On 03/17/2014 03:20 PM, Fujii Masao wrote: On Sun, Mar 16, 2014 at 7:15 AM, Alexander Korotkov aekorot...@gmail.com wrote: On Sat, Mar 15, 2014 at 11:27 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I ran pg_xlogdump | grep Gin and checked the size of GIN-related WAL, and then found

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Robert Haas
On Sat, Mar 15, 2014 at 4:50 AM, Mitsumasa KONDO kondo.mitsum...@gmail.com wrote: There are explanations and computations as comments in the code. If it is about the documentation, I'm not sure that a very precise mathematical definition will help a lot of people, and might rather hinder

Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-03-17 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: 2. Instead of storing the new compressed posting list in the WAL record, store only the new item pointers added to the page. WAL replay would then have to duplicate the work done in the main insertion code path: find the right posting lists

Re: [HACKERS] on_exit_reset fails to clear DSM-related exit actions

2014-03-17 Thread Robert Haas
On Mon, Mar 10, 2014 at 11:48 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Mar 7, 2014 at 2:40 PM, Robert Haas robertmh...@gmail.com wrote: Hmm. So the problematic sequence of events is where a postmaster child forks, and then exits without

Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-03-17 Thread Heikki Linnakangas
On 03/17/2014 04:33 PM, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: 2. Instead of storing the new compressed posting list in the WAL record, store only the new item pointers added to the page. WAL replay would then have to duplicate the work done in the main insertion

Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-03-17 Thread Robert Haas
On Mon, Mar 17, 2014 at 10:54 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Heap and B-tree WAL records also rely on PageAddItem etc. to reconstruct the page, instead of making a physical copy of the modified parts. And _bt_restore_page even inserts the items physically in different

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Greg Stark st...@mit.edu writes: This is not really accurate: This error allowed multiple versions of the same row to become visible to queries, resulting in apparent duplicates. Since the error is in WAL replay, it would only manifest during crash recovery or on standby servers. I think

Re: [HACKERS] on_exit_reset fails to clear DSM-related exit actions

2014-03-17 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: One option is to just change that function to also unmap the control segment, and maybe rename it to dsm_detach_all(), and then use that everywhere. The problem is that I'm not sure we really want to incur the overhead of an extra munmap() during

Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-03-17 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Mar 17, 2014 at 10:54 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Heap and B-tree WAL records also rely on PageAddItem etc. to reconstruct the page, instead of making a physical copy of the modified parts. And _bt_restore_page even

[HACKERS] Planner hints in Postgresql

2014-03-17 Thread Rajmohan C
I am implementing Planner hints in Postgresql to force the optimizer to select a particular plan for a query on request from sql input. I am having trouble in modifying the planner code. I want to create a path node of hint plan and make it the plan to be used by executor. How do I enforce this ?

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
On Mon, Mar 17, 2014 at 9:22 PM, Rajmohan C csrajmo...@gmail.com wrote: I am implementing Planner hints in Postgresql to force the optimizer to select a particular plan for a query on request from sql input. I am having trouble in modifying the planner code. I want to create a path node of

Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-03-17 Thread Heikki Linnakangas
On 03/17/2014 05:35 PM, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Mar 17, 2014 at 10:54 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: The imminent danger I see is if we change the logic on how the items are divided into posting lists, and end up in a situation

Re: [HACKERS] Portability issues in shm_mq

2014-03-17 Thread Robert Haas
On Sun, Mar 16, 2014 at 10:45 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: But I think there's another possible problem here. In order for reads from the buffer not to suffer alignment problems, the chunk size for reads and writes from the buffer needs to

Re: [HACKERS] Portability issues in shm_mq

2014-03-17 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Sun, Mar 16, 2014 at 10:45 PM, Tom Lane t...@sss.pgh.pa.us wrote: Well, it will result in padding space when you maxalign the length word, but I don't see why it wouldn't work; and it would certainly be no less efficient than what's there today.

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread David Johnston
Atri Sharma wrote On Mon, Mar 17, 2014 at 9:22 PM, Rajmohan C lt; csrajmohan@ gt; wrote: I am implementing Planner hints in Postgresql to force the optimizer to select a particular plan for a query on request from sql input. I am having trouble in modifying the planner code. I want to

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Tom Lane
David Johnston pol...@yahoo.com writes: Need to discuss the general why before any meaningful help on the how is going to be considered by hackers. Possibly worth noting is that in past discussions, we've concluded that the most sensible type of hint would not be use this plan at all, but

[HACKERS] warning when compiling utils/tqual.h

2014-03-17 Thread Alvaro Herrera
I noticed (by running cd src/include ; make check with the attached patch applied) that since commit b89e151054 (Introduce logical decoding.) tqual.h now emits this warning when compiled standalone: /pgsql/source/HEAD/src/include/utils/tqual.h:101:13: warning: ‘struct HTAB’ declared inside

Re: [HACKERS] warning when compiling utils/tqual.h

2014-03-17 Thread Andres Freund
Hi, On 2014-03-17 13:40:53 -0300, Alvaro Herrera wrote: I noticed (by running cd src/include ; make check with the attached patch applied) that since commit b89e151054 (Introduce logical decoding.) tqual.h now emits this warning when compiled standalone: I think we should add such a check

Re: [HACKERS] warning when compiling utils/tqual.h

2014-03-17 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: I noticed (by running cd src/include ; make check with the attached patch applied) that since commit b89e151054 (Introduce logical decoding.) tqual.h now emits this warning when compiled standalone:

Re: [HACKERS] warning when compiling utils/tqual.h

2014-03-17 Thread Andres Freund
On 2014-03-17 12:50:37 -0400, Tom Lane wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: I noticed (by running cd src/include ; make check with the attached patch applied) that since commit b89e151054 (Introduce logical decoding.) tqual.h now emits this warning when compiled

Re: [HACKERS] warning when compiling utils/tqual.h

2014-03-17 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-03-17 13:40:53 -0300, Alvaro Herrera wrote: There is of course a third choice which is to dictate that this function ought to be declared in reorderbuffer.h; but that would have the unpleasant side-effect that tqual.c would need to #include

Re: [HACKERS] warning when compiling utils/tqual.h

2014-03-17 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-03-17 12:50:37 -0400, Tom Lane wrote: I guess the real question is why such a prototype is in tqual.h in the first place. ISTM this should be pushed somewhere specific to reorderbuffer.c. I'm -1 on having struct HTAB bleed into tqual.h

Re: [HACKERS] warning when compiling utils/tqual.h

2014-03-17 Thread Andres Freund
On 2014-03-17 12:57:15 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-03-17 12:50:37 -0400, Tom Lane wrote: I guess the real question is why such a prototype is in tqual.h in the first place. ISTM this should be pushed somewhere specific to reorderbuffer.c.

Re: [HACKERS] warning when compiling utils/tqual.h

2014-03-17 Thread Andres Freund
On 2014-03-17 12:56:12 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-03-17 13:40:53 -0300, Alvaro Herrera wrote: There is of course a third choice which is to dictate that this function ought to be declared in reorderbuffer.h; but that would have the

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Josh Berkus
On 03/17/2014 08:28 AM, Tom Lane wrote: Greg Stark st...@mit.edu writes: The error causes some rows to disappear from indexes resulting in inconsistent query results on a hot standby depending on whether indexes are used. If the standby is subsequently activated or if it occurs during

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 10:03:52 -0700, Josh Berkus wrote: On 03/17/2014 08:28 AM, Tom Lane wrote: Greg Stark st...@mit.edu writes: The error causes some rows to disappear from indexes resulting in inconsistent query results on a hot standby depending on whether indexes are used. If the standby is

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
On Mon, Mar 17, 2014 at 9:45 PM, Tom Lane t...@sss.pgh.pa.us wrote: David Johnston pol...@yahoo.com writes: Need to discuss the general why before any meaningful help on the how is going to be considered by hackers. Possibly worth noting is that in past discussions, we've concluded that

Re: [HACKERS] on_exit_reset fails to clear DSM-related exit actions

2014-03-17 Thread Robert Haas
On Mon, Mar 17, 2014 at 11:32 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: One option is to just change that function to also unmap the control segment, and maybe rename it to dsm_detach_all(), and then use that everywhere. The problem is that I'm not sure

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Tom Lane
Atri Sharma atri.j...@gmail.com writes: On Mon, Mar 17, 2014 at 9:45 PM, Tom Lane t...@sss.pgh.pa.us wrote: Possibly worth noting is that in past discussions, we've concluded that the most sensible type of hint would not be use this plan at all, but here's what to assume about the selectivity

Re: [HACKERS] Portability issues in shm_mq

2014-03-17 Thread Robert Haas
On Mon, Mar 17, 2014 at 12:03 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Sun, Mar 16, 2014 at 10:45 PM, Tom Lane t...@sss.pgh.pa.us wrote: Well, it will result in padding space when you maxalign the length word, but I don't see why it wouldn't work; and

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Stephen Frost
* Atri Sharma (atri.j...@gmail.com) wrote: Isnt using a user given value for selectivity a pretty risky situation as it can horribly screw up the plan selection? Why not allow the user to specify an alternate plan and have the planner Uh, you're worried about the user given us a garbage

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread David Johnston
Atri Sharma wrote On Mon, Mar 17, 2014 at 9:45 PM, Tom Lane lt; tgl@.pa gt; wrote: David Johnston lt; polobo@ gt; writes: Need to discuss the general why before any meaningful help on the how is going to be considered by hackers. Possibly worth noting is that in past

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-15 16:02:19 -0400, Tom Lane wrote: First-draft release notes are committed, and should be visible at http://www.postgresql.org/docs/devel/static/release-9-3-4.html once guaibasaurus does its next buildfarm run a few minutes from now. Any suggestions? So, the current text is: This

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
On Mon, Mar 17, 2014 at 10:58 PM, Stephen Frost sfr...@snowman.net wrote: * Atri Sharma (atri.j...@gmail.com) wrote: Isnt using a user given value for selectivity a pretty risky situation as it can horribly screw up the plan selection? Why not allow the user to specify an alternate plan

Re: [HACKERS] on_exit_reset fails to clear DSM-related exit actions

2014-03-17 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: After mulling over a few possible approaches, I came up with the attached, which seems short and to the point. Looks reasonable in principle. I didn't run through all the existing PGSharedMemoryDetach calls to see if there are any other places to call

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Stephen Frost
* Atri Sharma (atri.j...@gmail.com) wrote: Of course, this is not a nice hack. Specifically after our discussion on IRC the other day, I am against planner hints, but if we are just discussing how it could be done, I could think of some ways which I listed. There's lots of ways to implement

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-03-17 10:03:52 -0700, Josh Berkus wrote: First, see suggested text in my first-draft release announcement. I don't think that text is any better, it's imo even wrong: The bug causes rows to vanish from indexes during recovery due to

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
The larger question to answer first is whether we want to implement something that is deterministic... How about just dropping the whole concept of hinting and provide a way for someone to say use this plan, or die trying. Maybe require it be used in conjunction with named PREPAREd

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
There's lots of ways to implement planner hints, but I fail to see the point in discussing how to implement something we actively don't want. +1. The original poster wanted a way to implement it as a personal project or something ( I think he only replied to me, not the entire list). Planner

Re: [HACKERS] jsonb status

2014-03-17 Thread Andrew Dunstan
On 03/16/2014 04:10 AM, Peter Geoghegan wrote: On Thu, Mar 13, 2014 at 2:00 PM, Andrew Dunstan and...@dunslane.net wrote: I'll be travelling a good bit of tomorrow (Friday), but I hope Peter has finished by the time I am back on deck late tomorrow and that I am able to commit this on Saturday.

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
On Mon, Mar 17, 2014 at 10:54 PM, Tom Lane t...@sss.pgh.pa.us wrote: Atri Sharma atri.j...@gmail.com writes: On Mon, Mar 17, 2014 at 9:45 PM, Tom Lane t...@sss.pgh.pa.us wrote: Possibly worth noting is that in past discussions, we've concluded that the most sensible type of hint would not

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Merlin Moncure
On Mon, Mar 17, 2014 at 11:15 AM, Tom Lane t...@sss.pgh.pa.us wrote: David Johnston pol...@yahoo.com writes: Need to discuss the general why before any meaningful help on the how is going to be considered by hackers. Possibly worth noting is that in past discussions, we've concluded that the

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 13:42:59 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-03-17 10:03:52 -0700, Josh Berkus wrote: First, see suggested text in my first-draft release announcement. I don't think that text is any better, it's imo even wrong: The bug causes rows

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 11:28:45 -0400, Tom Lane wrote: Hm ... rows disappearing from indexes might make people think that they could fix or mitigate the damage via REINDEX. Good point. I guess in some cases it will end up working because VACUUM/hot pruning have cleaned up the mess, but that's certainly

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Stephen Frost
* Merlin Moncure (mmonc...@gmail.com) wrote: Yeah -- the most common case I see is outlier culling where several repeated low non-deterministic selectivity quals stack reducing the row count estimate to 1. For example: SELECT * FROM foo WHERE length(bar) = 1000 AND length(bar) = 2; This is

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
There's a big difference between saying to the planner, Use plan X vs Here's some information describing the data supporting choosing plan X intelligently. The latter allows for better plans in the face of varied/changing data, integrates with the planner in natural way, and encourages users

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: That's much better, yes. Two things: * I'd change the warning about unique key violations into a more general one about constraints. Foreign key and exclusion constraint are also affected... I'll see what I can do. * I wonder if we should

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 14:01:03 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: * I wonder if we should make the possible origins a bit more general as it's perfectly possible to trigger the problem without foreign keys. Maybe: can arise when a table row that has been updated

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-03-17 14:01:03 -0400, Tom Lane wrote: IIUC, this case only occurs when using the new-in-9.3 types of nonexclusive row locks. I'm willing to bet that the number of applications using those is negligible; so I think it's all right to not

Re: [HACKERS] jsonb status

2014-03-17 Thread Oleg Bartunov
Alexander will take a look on TriConsistent function. On Mon, Mar 17, 2014 at 9:48 PM, Andrew Dunstan and...@dunslane.net wrote: On 03/16/2014 04:10 AM, Peter Geoghegan wrote: On Thu, Mar 13, 2014 at 2:00 PM, Andrew Dunstan and...@dunslane.net wrote: I'll be travelling a good bit of

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Merlin Moncure
On Mon, Mar 17, 2014 at 12:57 PM, Atri Sharma atri.j...@gmail.com wrote: There's a big difference between saying to the planner, Use plan X vs Here's some information describing the data supporting choosing plan X intelligently. The latter allows for better plans in the face of

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 14:16:41 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-03-17 14:01:03 -0400, Tom Lane wrote: IIUC, this case only occurs when using the new-in-9.3 types of nonexclusive row locks. I'm willing to bet that the number of applications using those

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: To me that looks sufficient to trigger the bug, because we're issuing a wal record about the row that was passed to heap_lock_update(), not the latest one in the ctid chain. When replaying that record, it will reset the t_ctid field, thus breaking

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 14:29:56 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: To me that looks sufficient to trigger the bug, because we're issuing a wal record about the row that was passed to heap_lock_update(), not the latest one in the ctid chain. When replaying that

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
On Mon, Mar 17, 2014 at 11:50 PM, Merlin Moncure mmonc...@gmail.com wrote: On Mon, Mar 17, 2014 at 12:57 PM, Atri Sharma atri.j...@gmail.com wrote: There's a big difference between saying to the planner, Use plan X vs Here's some information describing the data supporting choosing plan X

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Pavel Stehule
2014-03-17 19:35 GMT+01:00 Atri Sharma atri.j...@gmail.com: On Mon, Mar 17, 2014 at 11:50 PM, Merlin Moncure mmonc...@gmail.comwrote: On Mon, Mar 17, 2014 at 12:57 PM, Atri Sharma atri.j...@gmail.com wrote: There's a big difference between saying to the planner, Use plan X vs Here's

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 14:52:25 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-03-17 14:29:56 -0400, Tom Lane wrote: [ scratches head ... ] If that's what's happening, isn't it a bug in itself? Surely the WAL record ought to point at the tuple that was locked.

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-03-17 14:29:56 -0400, Tom Lane wrote: [ scratches head ... ] If that's what's happening, isn't it a bug in itself? Surely the WAL record ought to point at the tuple that was locked. There's a separate XLOG_HEAP2_LOCK_UPDATED record, for

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Tom Lane
Atri Sharma atri.j...@gmail.com writes: Wont this have scaling issues and issues over time as the data in the table changes? It can't possibly have worse problems of that sort than explicitly specifying a plan does. regards, tom lane -- Sent via pgsql-hackers

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Merlin Moncure
On Mon, Mar 17, 2014 at 1:45 PM, Pavel Stehule pavel.steh...@gmail.com wrote: I don't believe so SELECTIVITY can work well too. Slow queries are usually related to some strange points in data. I am thinking so well concept should be based on validity of estimations. Some plans are based on

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Alvaro Herrera
Andres Freund wrote: On 2014-03-17 14:01:03 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: * I wonder if we should make the possible origins a bit more general as it's perfectly possible to trigger the problem without foreign keys. Maybe: can arise when a table

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Uhm. But at the bottom of that block, right above the failed: label (heapam.c line 4527 in current master), we recheck the tuple for locked-only-ness; and fail the whole operation by returning HeapTupleUpdated, if it's not locked-only, no?

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Jim Nasby
On 3/17/14, 12:58 PM, Stephen Frost wrote: * Merlin Moncure (mmonc...@gmail.com) wrote: Yeah -- the most common case I see is outlier culling where several repeated low non-deterministic selectivity quals stack reducing the row count estimate to 1. For example: SELECT * FROM foo WHERE

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Merlin Moncure
On Mon, Mar 17, 2014 at 2:02 PM, Jim Nasby j...@nasby.net wrote: Just being able to detect that something has possibly gone wrong would be useful. We could log that to alert the DBA/user of a potential bad plan. We could even format this in such a fashion that it's suitable for emailing the

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 16:17:35 -0300, Alvaro Herrera wrote: Andres Freund wrote: On 2014-03-17 14:01:03 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: * I wonder if we should make the possible origins a bit more general as it's perfectly possible to trigger the problem

Re: [HACKERS] bpchar functinos

2014-03-17 Thread Noah Misch
On Sat, Mar 15, 2014 at 05:02:44PM +0330, Mohsen SM wrote: I want to fined when is used these functions(what query caused the call of these functions) : -char_bpchar() -bpchar_name() -name_bpchar() They implement casts. For example, select 'foo'::character(10)::name calls bpchar_name(). --

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Martijn van Oosterhout
On Mon, Mar 17, 2014 at 01:20:47PM -0500, Merlin Moncure wrote: A query plan is a complicated thing that is the result of detail analysis of the data. I bet there are less than 100 users on the planet with the architectural knowledge of the planner to submit a 'plan'. What users do have is

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Greg Stark
On Mon, Mar 17, 2014 at 3:28 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm thinking we'd better promote that Assert to a normal runtime elog. I wasn't sure about this but on further thought I think it's a really good idea and should be mentioned in the release notes. One of the things that's been

  1   2   >