Re: [HACKERS] [WIP] Better partial index-only scans

2014-03-17 Thread Joshua Yanovski
Here's a SQL script that (1) demonstrates the new index only scan functionality, and (2) at least on my machine, has a consistently higher planning time for the version with my change than without it. On Sun, Mar 16, 2014 at 5:08 AM, Joshua Yanovski wrote: > Proof of concept initial patch for ena

Re: [HACKERS] Fix typo in nbtree.h introduced by efada2b

2014-03-17 Thread Magnus Hagander
On Mon, Mar 17, 2014 at 6:06 AM, Michael Paquier wrote: > Hi, > > I found a small typo in nbtree.h, introduced by commit efada2b. Patch > is attached. > Applied, thanks. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/

Re: [HACKERS] inherit support for foreign tables

2014-03-17 Thread Kyotaro HORIGUCHI
Hi Fujita-san, > Thank you for working this patch! No problem, but my point seems always out of the main target a bit:( > > | =# alter table passwd add column added int, add column added2 int; > > | NOTICE: This command affects foreign relation "cf1" > > | NOTICE: This command affects foreign

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread KONDO Mitsumasa
Hi Heikki-san, (2014/03/17 14:39), KONDO Mitsumasa wrote: (2014/03/15 15:53), Fabien COELHO wrote: Hello Heikki, A couple of comments: * There should be an explicit "\setrandom ... uniform" option too, even though you get that implicitly if you don't specify the distribution Fix. We can us

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Heikki Linnakangas
On 03/15/2014 08:53 AM, Fabien COELHO wrote: >* Does min and max really make sense for gaussian and exponential >distributions? For gaussian, I would expect mean and standard deviation as >the parameters, not min/max/threshold. Yes... and no:-) The aim is to draw an integer primary key from a ta

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Heikki Linnakangas
On 03/17/2014 10:40 AM, KONDO Mitsumasa wrote: By the way, you seem to want to remove --gaussian=NUM and --exponential=NUM command options. Can you tell me the objective reason? I think pgbench is the benchmark test on PostgreSQL and default benchmark is TPC-B-like benchmark. It is written in doc

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread KONDO Mitsumasa
(2014/03/17 17:46), Heikki Linnakangas wrote: On 03/15/2014 08:53 AM, Fabien COELHO wrote: >* Does min and max really make sense for gaussian and exponential >distributions? For gaussian, I would expect mean and standard deviation as >the parameters, not min/max/threshold. Yes... and no:-) The

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread KONDO Mitsumasa
(2014/03/17 18:02), Heikki Linnakangas wrote: On 03/17/2014 10:40 AM, KONDO Mitsumasa wrote: By the way, you seem to want to remove --gaussian=NUM and --exponential=NUM command options. Can you tell me the objective reason? I think pgbench is the benchmark test on PostgreSQL and default benchmar

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Fujii Masao
On Mon, Mar 17, 2014 at 7:07 PM, KONDO Mitsumasa wrote: > (2014/03/17 18:02), Heikki Linnakangas wrote: >> >> On 03/17/2014 10:40 AM, KONDO Mitsumasa wrote: >>> >>> By the way, you seem to want to remove --gaussian=NUM and >>> --exponential=NUM >>> command options. Can you tell me the objective re

Re: [HACKERS] Patch: show relation and tuple infos of a lock to acquire

2014-03-17 Thread Christian Kruse
Hi Amit, I've been ill the last few days, so sorry for my late response. > I have updated the patch to pass TID and operation information in > error context and changed some of the comments in code. > Let me know if the added operation information is useful, else > we can use better generic messa

[HACKERS] Various typos

2014-03-17 Thread Thom Brown
Attached is a small patch to fix various typos. Regards Thom diff --git a/contrib/pgcrypto/openssl.c b/contrib/pgcrypto/openssl.c index ad7fb9e..86c0fb0 100644 --- a/contrib/pgcrypto/openssl.c +++ b/contrib/pgcrypto/openssl.c @@ -429,7 +429,7 @@ bf_init(PX_Cipher *c, const uint8 *key, unsigned kl

Re: [HACKERS] Changeset Extraction v7.9.1

2014-03-17 Thread Robert Haas
On Wed, Mar 12, 2014 at 2:06 PM, Andres Freund wrote: > Attached are the collected remaining patches. The docs might need > further additions, but it seems better to add them now. A few questions about pg_recvlogical: - There doesn't seem to be any provision for this tool to ever switch from one

Re: [HACKERS] Changeset Extraction v7.9.1

2014-03-17 Thread Andres Freund
On 2014-03-17 06:55:28 -0400, Robert Haas wrote: > On Wed, Mar 12, 2014 at 2:06 PM, Andres Freund wrote: > > Attached are the collected remaining patches. The docs might need > > further additions, but it seems better to add them now. > > A few questions about pg_recvlogical: > > - There doesn't

[HACKERS] pg_dump without explicit table locking

2014-03-17 Thread Jürgen Strobel
Hi, at work at my company I inherited responsibility for a large PG 8.1 DB, with a an extreme number of tables (~30). Surprisingly this is working quite well, except for maintenance and backup. I am tasked with finding a way to do dump & restore to 9.3 with as little downtime as possible. Us

Re: [HACKERS] Various typos

2014-03-17 Thread Fujii Masao
On Mon, Mar 17, 2014 at 7:52 PM, Thom Brown wrote: > Attached is a small patch to fix various typos. Thanks! Committed. Regards, -- Fujii Masao -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsq

Re: [HACKERS] Changeset Extraction v7.9.1

2014-03-17 Thread Robert Haas
On Mon, Mar 17, 2014 at 7:27 AM, Andres Freund wrote: >> - There doesn't seem to be any provision for this tool to ever switch >> from one output file to the next. That seems like a practical need. >> One idea would be to have it respond to SIGHUP by reopening the >> originally-named output file.

Re: [HACKERS] pg_dump without explicit table locking

2014-03-17 Thread Robert Haas
On Mon, Mar 17, 2014 at 7:52 AM, Jürgen Strobel wrote: > at work at my company I inherited responsibility for a large PG 8.1 DB, > with a an extreme number of tables (~30). Surprisingly this is > working quite well, except for maintenance and backup. I am tasked with > finding a way to do dump

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-17 Thread Fujii Masao
On Mon, Mar 17, 2014 at 10:20 AM, Robert Haas wrote: > On Sun, Mar 16, 2014 at 6:23 AM, MauMau wrote: >> The PostgreSQL documentation describes cp (on UNIX/Linux) or copy (on >> Windows) as an example for archive_command. However, cp/copy does not sync >> the copied data to disk. As a result, t

Re: [HACKERS] pg_dump without explicit table locking

2014-03-17 Thread Pavel Stehule
2014-03-17 12:52 GMT+01:00 Jürgen Strobel : > > Hi, > > at work at my company I inherited responsibility for a large PG 8.1 DB, > with a an extreme number of tables (~30). Surprisingly this is > working quite well, except for maintenance and backup. I am tasked with > finding a way to do dump

Re: [HACKERS] Changeset Extraction v7.9.1

2014-03-17 Thread Andres Freund
On 2014-03-17 08:00:22 -0400, Robert Haas wrote: > > Yea. The reason it reports the flush position is that it allows to test > > sync rep. I don't think other usecases will appreciate frequent > > fsyncs... Maybe make it optional? > > Well, as I'm sure you recognize, if you're actually trying to b

Re: [HACKERS] Changeset Extraction v7.9.1

2014-03-17 Thread Andres Freund
On 2014-03-17 08:00:22 -0400, Robert Haas wrote: > On Mon, Mar 17, 2014 at 7:27 AM, Andres Freund wrote: > >> - There doesn't seem to be any provision for this tool to ever switch > >> from one output file to the next. That seems like a practical need. > >> One idea would be to have it respond to

Re: [HACKERS] Changeset Extraction v7.9.1

2014-03-17 Thread Robert Haas
On Mon, Mar 17, 2014 at 8:29 AM, Andres Freund wrote: >> Perhaps there could be a switch for an fsync interval, or something >> like that. The default could be, say, to fsync every 10 seconds. And >> if you want to change it, then go ahead; 0 disables. Writing to >> standard output would be doc

Re: [HACKERS] Changeset Extraction v7.9.1

2014-03-17 Thread Robert Haas
On Mon, Mar 17, 2014 at 8:58 AM, Andres Freund wrote: > On 2014-03-17 08:00:22 -0400, Robert Haas wrote: >> On Mon, Mar 17, 2014 at 7:27 AM, Andres Freund >> wrote: >> >> - There doesn't seem to be any provision for this tool to ever switch >> >> from one output file to the next. That seems lik

Re: [HACKERS] Changeset Extraction v7.9.1

2014-03-17 Thread Andres Freund
On 2014-03-17 09:13:38 -0400, Robert Haas wrote: > On Mon, Mar 17, 2014 at 8:29 AM, Andres Freund wrote: > >> Perhaps there could be a switch for an fsync interval, or something > >> like that. The default could be, say, to fsync every 10 seconds. And > >> if you want to change it, then go ahead

Re: [HACKERS] Changeset Extraction v7.9.1

2014-03-17 Thread Andres Freund
On 2014-03-17 09:14:51 -0400, Robert Haas wrote: > On Mon, Mar 17, 2014 at 8:58 AM, Andres Freund wrote: > > On 2014-03-17 08:00:22 -0400, Robert Haas wrote: > >> On Mon, Mar 17, 2014 at 7:27 AM, Andres Freund > >> wrote: > >> >> - There doesn't seem to be any provision for this tool to ever swi

Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-03-17 Thread Fujii Masao
On Sun, Mar 16, 2014 at 7:15 AM, Alexander Korotkov wrote: > On Sat, Mar 15, 2014 at 11:27 PM, Heikki Linnakangas > wrote: >> >> On 03/15/2014 08:40 PM, Fujii Masao wrote: >>> >>> Hi, >>> >>> I executed the following statements in HEAD and 9.3, and compared >>> the size of WAL which were generate

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Tom Lane
KONDO Mitsumasa writes: > (2014/03/17 18:02), Heikki Linnakangas wrote: >> On 03/17/2014 10:40 AM, KONDO Mitsumasa wrote: >> There is an infinite number of variants of the TPC-B test that we could >> include >> in pgbench. If we start adding every one of them, we're quickly going to have >> hundr

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-17 Thread Mitsumasa KONDO
2014-03-17 21:12 GMT+09:00 Fujii Masao : > On Mon, Mar 17, 2014 at 10:20 AM, Robert Haas > wrote: > > On Sun, Mar 16, 2014 at 6:23 AM, MauMau wrote: > >> The PostgreSQL documentation describes cp (on UNIX/Linux) or copy (on > >> Windows) as an example for archive_command. However, cp/copy does

Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-03-17 Thread Alvaro Herrera
Fujii Masao escribió: > On Sun, Mar 16, 2014 at 7:15 AM, Alexander Korotkov > wrote: > >> That could be optimized, but I figured we can live with it, thanks to the > >> fastupdate feature. Fastupdate allows amortizing that cost over several > >> insertions. But of course, you explicitly disabled

Re: [HACKERS] pg_dump without explicit table locking

2014-03-17 Thread Tom Lane
Pavel Stehule writes: > 2014-03-17 12:52 GMT+01:00 Jürgen Strobel : >> I've googled the problem and there seem to be more people with similar >> problems, so I made this a command line option --no-table-locks and >> wrapped it up in as nice a patch against github/master as I can manage >> (and I

Re: [HACKERS] Archive recovery won't be completed on some situation.

2014-03-17 Thread Heikki Linnakangas
On 03/15/2014 05:59 PM, Fujii Masao wrote: What about adding new option into pg_resetxlog so that we can reset the pg_control's backup start location? Even after we've accidentally entered into the situation that you described, we can exit from that by resetting the backup start location in pg_co

Re: [HACKERS] Patch: show relation and tuple infos of a lock to acquire

2014-03-17 Thread Amit Kapila
On Mon, Mar 17, 2014 at 4:20 PM, Christian Kruse wrote: > Hi Amit, > > I've been ill the last few days, so sorry for my late response. Sorry to hear and no problem for delay. > I don't think that this fixes the translation guideline issues brought > up by Robert. This produces differing strin

Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-03-17 Thread Heikki Linnakangas
On 03/17/2014 03:20 PM, Fujii Masao wrote: On Sun, Mar 16, 2014 at 7:15 AM, Alexander Korotkov wrote: On Sat, Mar 15, 2014 at 11:27 PM, Heikki Linnakangas wrote: I ran "pg_xlogdump | grep Gin" and checked the size of GIN-related WAL, and then found its max seems more than 256B. Am I missing

Re: [HACKERS] gaussian distribution pgbench

2014-03-17 Thread Robert Haas
On Sat, Mar 15, 2014 at 4:50 AM, Mitsumasa KONDO wrote: >> There are explanations and computations as comments in the code. If it is >> about the documentation, I'm not sure that a very precise mathematical >> definition will help a lot of people, and might rather hinder understanding, >> so the d

Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-03-17 Thread Tom Lane
Heikki Linnakangas writes: > 2. Instead of storing the new compressed posting list in the WAL record, > store only the new item pointers added to the page. WAL replay would > then have to duplicate the work done in the main insertion code path: > find the right posting lists to insert to, decod

Re: [HACKERS] on_exit_reset fails to clear DSM-related exit actions

2014-03-17 Thread Robert Haas
On Mon, Mar 10, 2014 at 11:48 AM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Mar 7, 2014 at 2:40 PM, Robert Haas wrote: >>> Hmm. So the problematic sequence of events is where a postmaster >>> child forks, and then exits without exec-ing, perhaps because e.g. >>> exec fails? > >> I've att

Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-03-17 Thread Heikki Linnakangas
On 03/17/2014 04:33 PM, Tom Lane wrote: Heikki Linnakangas writes: 2. Instead of storing the new compressed posting list in the WAL record, store only the new item pointers added to the page. WAL replay would then have to duplicate the work done in the main insertion code path: find the right p

Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-03-17 Thread Robert Haas
On Mon, Mar 17, 2014 at 10:54 AM, Heikki Linnakangas wrote: > Heap and B-tree WAL records also rely on PageAddItem etc. to reconstruct the > page, instead of making a physical copy of the modified parts. And > _bt_restore_page even inserts the items physically in different order than > the normal

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Greg Stark writes: > This is not really accurate: > "This error allowed multiple versions of the same row to become > visible to queries, resulting in apparent duplicates. Since the error > is in WAL replay, it would only manifest during crash recovery or on > standby servers." > I think the idea

Re: [HACKERS] on_exit_reset fails to clear DSM-related exit actions

2014-03-17 Thread Tom Lane
Robert Haas writes: > One option is to just change that function to also unmap the control > segment, and maybe rename it to dsm_detach_all(), and then use that > everywhere. The problem is that I'm not sure we really want to incur > the overhead of an extra munmap() during every backend exit, ju

Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-03-17 Thread Tom Lane
Robert Haas writes: > On Mon, Mar 17, 2014 at 10:54 AM, Heikki Linnakangas > wrote: >> Heap and B-tree WAL records also rely on PageAddItem etc. to reconstruct the >> page, instead of making a physical copy of the modified parts. And >> _bt_restore_page even inserts the items physically in differ

[HACKERS] Planner hints in Postgresql

2014-03-17 Thread Rajmohan C
I am implementing Planner hints in Postgresql to force the optimizer to select a particular plan for a query on request from sql input. I am having trouble in modifying the planner code. I want to create a path node of hint plan and make it the plan to be used by executor. How do I enforce this ? S

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
On Mon, Mar 17, 2014 at 9:22 PM, Rajmohan C wrote: > I am implementing Planner hints in Postgresql to force the optimizer to > select a particular plan for a query on request from sql input. I am having > trouble in modifying the planner code. I want to create a path node of hint > plan and make

Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-03-17 Thread Heikki Linnakangas
On 03/17/2014 05:35 PM, Tom Lane wrote: Robert Haas writes: On Mon, Mar 17, 2014 at 10:54 AM, Heikki Linnakangas wrote: The imminent danger I see is if we change the logic on how the items are divided into posting lists, and end up in a situation where a master server adds an item to a page,

Re: [HACKERS] Portability issues in shm_mq

2014-03-17 Thread Robert Haas
On Sun, Mar 16, 2014 at 10:45 PM, Tom Lane wrote: > Robert Haas writes: >> But I think there's another possible problem here. In order for reads >> from the buffer not to suffer alignment problems, the chunk size for >> reads and writes from the buffer needs to be MAXIMUM_ALIGNOF (or some >> mul

Re: [HACKERS] Portability issues in shm_mq

2014-03-17 Thread Tom Lane
Robert Haas writes: > On Sun, Mar 16, 2014 at 10:45 PM, Tom Lane wrote: >> Well, it will result in padding space when you maxalign the length word, >> but I don't see why it wouldn't work; and it would certainly be no less >> efficient than what's there today. > Well, the problem is with this:

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread David Johnston
Atri Sharma wrote > On Mon, Mar 17, 2014 at 9:22 PM, Rajmohan C < > csrajmohan@ > > wrote: > >> I am implementing Planner hints in Postgresql to force the optimizer to >> select a particular plan for a query on request from sql input. I am >> having >> trouble in modifying the planner code. I wa

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Tom Lane
David Johnston writes: > Need to discuss the general "why" before any meaningful help on the "how" is > going to be considered by hackers. Possibly worth noting is that in past discussions, we've concluded that the most sensible type of hint would not be "use this plan" at all, but "here's what t

[HACKERS] warning when compiling utils/tqual.h

2014-03-17 Thread Alvaro Herrera
I noticed (by running "cd src/include ; make check" with the attached patch applied) that since commit b89e151054 ("Introduce logical decoding.") tqual.h now emits this warning when compiled standalone: /pgsql/source/HEAD/src/include/utils/tqual.h:101:13: warning: ‘struct HTAB’ declared inside pa

Re: [HACKERS] warning when compiling utils/tqual.h

2014-03-17 Thread Andres Freund
Hi, On 2014-03-17 13:40:53 -0300, Alvaro Herrera wrote: > I noticed (by running "cd src/include ; make check" with the attached > patch applied) that since commit b89e151054 ("Introduce logical > decoding.") tqual.h now emits this warning when compiled standalone: I think we should add such a ch

Re: [HACKERS] warning when compiling utils/tqual.h

2014-03-17 Thread Tom Lane
Alvaro Herrera writes: > I noticed (by running "cd src/include ; make check" with the attached > patch applied) that since commit b89e151054 ("Introduce logical > decoding.") tqual.h now emits this warning when compiled standalone: > /pgsql/source/HEAD/src/include/utils/tqual.h:101:13: warning: â

Re: [HACKERS] warning when compiling utils/tqual.h

2014-03-17 Thread Andres Freund
On 2014-03-17 12:50:37 -0400, Tom Lane wrote: > Alvaro Herrera writes: > > I noticed (by running "cd src/include ; make check" with the attached > > patch applied) that since commit b89e151054 ("Introduce logical > > decoding.") tqual.h now emits this warning when compiled standalone: > > > /pgsq

Re: [HACKERS] warning when compiling utils/tqual.h

2014-03-17 Thread Tom Lane
Andres Freund writes: > On 2014-03-17 13:40:53 -0300, Alvaro Herrera wrote: >> There is of course a third choice which is to dictate that this function >> ought to be declared in reorderbuffer.h; but that would have the >> unpleasant side-effect that tqual.c would need to #include that. > I am pr

Re: [HACKERS] warning when compiling utils/tqual.h

2014-03-17 Thread Tom Lane
Andres Freund writes: > On 2014-03-17 12:50:37 -0400, Tom Lane wrote: >> I guess the real question is why such a prototype is in tqual.h in >> the first place. ISTM this should be pushed somewhere specific to >> reorderbuffer.c. I'm -1 on having struct HTAB bleed into tqual.h >> via either of th

Re: [HACKERS] warning when compiling utils/tqual.h

2014-03-17 Thread Andres Freund
On 2014-03-17 12:57:15 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-03-17 12:50:37 -0400, Tom Lane wrote: > >> I guess the real question is why such a prototype is in tqual.h in > >> the first place. ISTM this should be pushed somewhere specific to > >> reorderbuffer.c. I'm -1 on h

Re: [HACKERS] warning when compiling utils/tqual.h

2014-03-17 Thread Andres Freund
On 2014-03-17 12:56:12 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-03-17 13:40:53 -0300, Alvaro Herrera wrote: > >> There is of course a third choice which is to dictate that this function > >> ought to be declared in reorderbuffer.h; but that would have the > >> unpleasant side-eff

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Josh Berkus
On 03/17/2014 08:28 AM, Tom Lane wrote: > Greg Stark writes: >> The error causes some rows to disappear from indexes resulting in >> inconsistent query results on a hot standby depending on whether >> indexes are used. If the standby is subsequently activated or if it >> occurs during recovery aft

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 10:03:52 -0700, Josh Berkus wrote: > On 03/17/2014 08:28 AM, Tom Lane wrote: > > Greg Stark writes: > >> The error causes some rows to disappear from indexes resulting in > >> inconsistent query results on a hot standby depending on whether > >> indexes are used. If the standby is su

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
On Mon, Mar 17, 2014 at 9:45 PM, Tom Lane wrote: > David Johnston writes: > > Need to discuss the general "why" before any meaningful help on the > "how" is > > going to be considered by hackers. > > Possibly worth noting is that in past discussions, we've concluded that > the most sensible type

Re: [HACKERS] on_exit_reset fails to clear DSM-related exit actions

2014-03-17 Thread Robert Haas
On Mon, Mar 17, 2014 at 11:32 AM, Tom Lane wrote: > Robert Haas writes: >> One option is to just change that function to also unmap the control >> segment, and maybe rename it to dsm_detach_all(), and then use that >> everywhere. The problem is that I'm not sure we really want to incur >> the ov

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Tom Lane
Atri Sharma writes: > On Mon, Mar 17, 2014 at 9:45 PM, Tom Lane wrote: >> Possibly worth noting is that in past discussions, we've concluded that >> the most sensible type of hint would not be "use this plan" at all, but >> "here's what to assume about the selectivity of this WHERE clause". >> Th

Re: [HACKERS] Portability issues in shm_mq

2014-03-17 Thread Robert Haas
On Mon, Mar 17, 2014 at 12:03 PM, Tom Lane wrote: > Robert Haas writes: >> On Sun, Mar 16, 2014 at 10:45 PM, Tom Lane wrote: >>> Well, it will result in padding space when you maxalign the length word, >>> but I don't see why it wouldn't work; and it would certainly be no less >>> efficient than

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Stephen Frost
* Atri Sharma (atri.j...@gmail.com) wrote: > Isnt using a user given value for selectivity a pretty risky situation as > it can horribly screw up the plan selection? > > Why not allow the user to specify an alternate plan and have the planner Uh, you're worried about the user given us a garbage s

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread David Johnston
Atri Sharma wrote > On Mon, Mar 17, 2014 at 9:45 PM, Tom Lane < > tgl@.pa > > wrote: > >> David Johnston < > polobo@ > > writes: >> > Need to discuss the general "why" before any meaningful help on the >> "how" is >> > going to be considered by hackers. >> >> Possibly worth noting is that in p

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-15 16:02:19 -0400, Tom Lane wrote: > First-draft release notes are committed, and should be visible at > http://www.postgresql.org/docs/devel/static/release-9-3-4.html > once guaibasaurus does its next buildfarm run a few minutes from > now. Any suggestions? So, the current text is: "T

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
On Mon, Mar 17, 2014 at 10:58 PM, Stephen Frost wrote: > * Atri Sharma (atri.j...@gmail.com) wrote: > > Isnt using a user given value for selectivity a pretty risky situation as > > it can horribly screw up the plan selection? > > > > Why not allow the user to specify an alternate plan and have t

Re: [HACKERS] on_exit_reset fails to clear DSM-related exit actions

2014-03-17 Thread Tom Lane
Robert Haas writes: > After mulling over a few possible approaches, I came up with the > attached, which seems short and to the point. Looks reasonable in principle. I didn't run through all the existing PGSharedMemoryDetach calls to see if there are any other places to call dsm_detach_all, but

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Stephen Frost
* Atri Sharma (atri.j...@gmail.com) wrote: > Of course, this is not a nice hack. Specifically after our discussion on > IRC the other day, I am against planner hints, but if we are just > discussing how it could be done, I could think of some ways which I listed. There's lots of ways to implement

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Andres Freund writes: > On 2014-03-17 10:03:52 -0700, Josh Berkus wrote: >> First, see suggested text in my first-draft release announcement. > I don't think that text is any better, it's imo even wrong: > "The bug causes rows to vanish from indexes during recovery due to > simultaneous updates o

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
The larger question to answer first is whether we want to implement > something that is deterministic... > > How about just dropping the whole concept of "hinting" and provide a way > for > someone to say "use this plan, or die trying." Maybe require it be used in > conjunction with named PREPARE

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
> There's lots of ways to implement planner hints, but I fail to see the > point in discussing how to implement something we actively don't want. > > > +1. The original poster wanted a way to implement it as a personal project or something ( I think he only replied to me, not the entire list). Pla

Re: [HACKERS] jsonb status

2014-03-17 Thread Andrew Dunstan
On 03/16/2014 04:10 AM, Peter Geoghegan wrote: On Thu, Mar 13, 2014 at 2:00 PM, Andrew Dunstan wrote: I'll be travelling a good bit of tomorrow (Friday), but I hope Peter has finished by the time I am back on deck late tomorrow and that I am able to commit this on Saturday. I asked Andrew to

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
On Mon, Mar 17, 2014 at 10:54 PM, Tom Lane wrote: > Atri Sharma writes: > > On Mon, Mar 17, 2014 at 9:45 PM, Tom Lane wrote: > >> Possibly worth noting is that in past discussions, we've concluded that > >> the most sensible type of hint would not be "use this plan" at all, but > >> "here's wha

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Merlin Moncure
On Mon, Mar 17, 2014 at 11:15 AM, Tom Lane wrote: > David Johnston writes: >> Need to discuss the general "why" before any meaningful help on the "how" is >> going to be considered by hackers. > > Possibly worth noting is that in past discussions, we've concluded that > the most sensible type of

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 13:42:59 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-03-17 10:03:52 -0700, Josh Berkus wrote: > >> First, see suggested text in my first-draft release announcement. > > > I don't think that text is any better, it's imo even wrong: > > "The bug causes rows to vanish fr

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 11:28:45 -0400, Tom Lane wrote: > Hm ... "rows disappearing from indexes" might make people think that > they could fix or mitigate the damage via REINDEX. Good point. I guess in some cases it will end up working because VACUUM/hot pruning have cleaned up the mess, but that's certain

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Stephen Frost
* Merlin Moncure (mmonc...@gmail.com) wrote: > Yeah -- the most common case I see is outlier culling where several > repeated low non-deterministic selectivity quals stack reducing the > row count estimate to 1. For example: > SELECT * FROM foo WHERE length(bar) <= 1000 AND length(bar) >= 2; This

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
There's a big difference between saying to the planner, "Use plan X" > vs "Here's some information describing the data supporting choosing > plan X intelligently". The latter allows for better plans in the face > of varied/changing data, integrates with the planner in natural way, > and encourages

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Andres Freund writes: > That's much better, yes. Two things: > * I'd change the warning about unique key violations into a more general > one about constraints. Foreign key and exclusion constraint are also > affected... I'll see what I can do. > * I wonder if we should make the possible or

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 14:01:03 -0400, Tom Lane wrote: > Andres Freund writes: > > * I wonder if we should make the possible origins a bit more > > general as it's perfectly possible to trigger the problem without > > foreign keys. Maybe: "can arise when a table row that has been updated > > is row l

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Andres Freund writes: > On 2014-03-17 14:01:03 -0400, Tom Lane wrote: >> IIUC, this case only occurs when using the new-in-9.3 types of >> nonexclusive row locks. I'm willing to bet that the number of >> applications using those is negligible; so I think it's all right to not >> mention that case

Re: [HACKERS] jsonb status

2014-03-17 Thread Oleg Bartunov
Alexander will take a look on TriConsistent function. On Mon, Mar 17, 2014 at 9:48 PM, Andrew Dunstan wrote: > > On 03/16/2014 04:10 AM, Peter Geoghegan wrote: >> >> On Thu, Mar 13, 2014 at 2:00 PM, Andrew Dunstan >> wrote: >>> >>> I'll be travelling a good bit of tomorrow (Friday), but I hope P

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Merlin Moncure
On Mon, Mar 17, 2014 at 12:57 PM, Atri Sharma wrote: > >> There's a big difference between saying to the planner, "Use plan X" >> vs "Here's some information describing the data supporting choosing >> plan X intelligently". The latter allows for better plans in the face >> of varied/changing data

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 14:16:41 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-03-17 14:01:03 -0400, Tom Lane wrote: > >> IIUC, this case only occurs when using the new-in-9.3 types of > >> nonexclusive row locks. I'm willing to bet that the number of > >> applications using those is negligibl

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Andres Freund writes: > To me that looks sufficient to trigger the bug, because we're issuing a > wal record about the row that was passed to heap_lock_update(), not the > latest one in the ctid chain. When replaying that record, it will reset > the t_ctid field, thus breaking the chain. [ scratc

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 14:29:56 -0400, Tom Lane wrote: > Andres Freund writes: > > To me that looks sufficient to trigger the bug, because we're issuing a > > wal record about the row that was passed to heap_lock_update(), not the > > latest one in the ctid chain. When replaying that record, it will reset

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
On Mon, Mar 17, 2014 at 11:50 PM, Merlin Moncure wrote: > On Mon, Mar 17, 2014 at 12:57 PM, Atri Sharma wrote: > > > >> There's a big difference between saying to the planner, "Use plan X" > >> vs "Here's some information describing the data supporting choosing > >> plan X intelligently". The l

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Pavel Stehule
2014-03-17 19:35 GMT+01:00 Atri Sharma : > > > > On Mon, Mar 17, 2014 at 11:50 PM, Merlin Moncure wrote: > >> On Mon, Mar 17, 2014 at 12:57 PM, Atri Sharma >> wrote: >> > >> >> There's a big difference between saying to the planner, "Use plan X" >> >> vs "Here's some information describing the da

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 14:52:25 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-03-17 14:29:56 -0400, Tom Lane wrote: > >> [ scratches head ... ] If that's what's happening, isn't it a bug in > >> itself? Surely the WAL record ought to point at the tuple that was > >> locked. > > > There's a

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Andres Freund writes: > On 2014-03-17 14:29:56 -0400, Tom Lane wrote: >> [ scratches head ... ] If that's what's happening, isn't it a bug in >> itself? Surely the WAL record ought to point at the tuple that was >> locked. > There's a separate XLOG_HEAP2_LOCK_UPDATED record, for every later tup

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Tom Lane
Atri Sharma writes: > Wont this have scaling issues and issues over time as the data in the > table changes? It can't possibly have worse problems of that sort than explicitly specifying a plan does. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hack

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Merlin Moncure
On Mon, Mar 17, 2014 at 1:45 PM, Pavel Stehule wrote: > I don't believe so SELECTIVITY can work well too. Slow queries are usually > related to some strange points in data. I am thinking so well concept should > be based on validity of estimations. Some plans are based on totally wrong > estimatio

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Alvaro Herrera
Andres Freund wrote: > On 2014-03-17 14:01:03 -0400, Tom Lane wrote: > > Andres Freund writes: > > > * I wonder if we should make the possible origins a bit more > > > general as it's perfectly possible to trigger the problem without > > > foreign keys. Maybe: "can arise when a table row that

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Alvaro Herrera writes: > Uhm. But at the bottom of that block, right above the "failed:" label > (heapam.c line 4527 in current master), we recheck the tuple for > "locked-only-ness"; and fail the whole operation by returning > HeapTupleUpdated, if it's not locked-only, no? Which would cause > E

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Jim Nasby
On 3/17/14, 12:58 PM, Stephen Frost wrote: * Merlin Moncure (mmonc...@gmail.com) wrote: Yeah -- the most common case I see is outlier culling where several repeated low non-deterministic selectivity quals stack reducing the row count estimate to 1. For example: SELECT * FROM foo WHERE length(ba

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Merlin Moncure
On Mon, Mar 17, 2014 at 2:02 PM, Jim Nasby wrote: > Just being able to detect that something has possibly gone wrong would be > useful. We could log that to alert the DBA/user of a potential bad plan. We > could even format this in such a fashion that it's suitable for emailing the > community wit

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 16:17:35 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > On 2014-03-17 14:01:03 -0400, Tom Lane wrote: > > > Andres Freund writes: > > > > * I wonder if we should make the possible origins a bit more > > > > general as it's perfectly possible to trigger the problem without

Re: [HACKERS] bpchar functinos

2014-03-17 Thread Noah Misch
On Sat, Mar 15, 2014 at 05:02:44PM +0330, Mohsen SM wrote: > I want to fined when is used these functions(what query caused the call of > these functions) : > -char_bpchar() > -bpchar_name() > -name_bpchar() They implement casts. For example, "select 'foo'::character(10)::name" calls bpchar_name(

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Martijn van Oosterhout
On Mon, Mar 17, 2014 at 01:20:47PM -0500, Merlin Moncure wrote: > A query plan is a complicated thing that is the result of detail > analysis of the data. I bet there are less than 100 users on the > planet with the architectural knowledge of the planner to submit a > 'plan'. What users do have i

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Greg Stark
On Mon, Mar 17, 2014 at 3:28 PM, Tom Lane wrote: > I'm thinking we'd better promote that Assert to a normal runtime elog. I wasn't sure about this but on further thought I think it's a really good idea and should be mentioned in the release notes. One of the things that's been bothering me about

  1   2   >