Re: [HACKERS] Per-Database Roles

2012-05-23 Thread Florian Pflug
On May23, 2012, at 00:21 , Josh Berkus wrote: If you could help me work out the semantics and the high-level issues, I'd love to spend time on this for 9.3... Syntax seems simple: CREATE LOCAL ROLE ... For that matter, let's keep other things simple: 1. local roles can inherit only from

[HACKERS] [RFC] Interface of Row Level Security

2012-05-23 Thread Kohei KaiGai
Let me have a discussion to get preferable interface for row-level security. My planned feature will perform to append additional conditions to WHERE clause implicitly, to restrict tuples being visible for the current user. For example, when row-level policy uname = getpgusername() is configured

Re: [HACKERS] pg_basebackup -x stream from the standby gets stuck

2012-05-23 Thread Magnus Hagander
On Fri, Mar 2, 2012 at 2:26 PM, Magnus Hagander mag...@hagander.net wrote: On Tue, Feb 28, 2012 at 09:22, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Feb 23, 2012 at 1:02 AM, Magnus Hagander mag...@hagander.net wrote: On Tue, Feb 7, 2012 at 12:30, Fujii Masao masao.fu...@gmail.com wrote:

Re: [HACKERS] Why is indexonlyscan so darned slow?

2012-05-23 Thread Greg Stark
On Mon, May 21, 2012 at 9:37 PM, Simon Riggs si...@2ndquadrant.com wrote: This is exactly what we do for VACUUM and it works faster there. The reason that's okay for vacuum is that vacuum doesn't care if it visits the same index tuple multiple times.  It will not work for real queries, unless

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-05-23 Thread Tom Lane
Kohei KaiGai kai...@kaigai.gr.jp writes: Let me have a discussion to get preferable interface for row-level security. My planned feature will perform to append additional conditions to WHERE clause implicitly, to restrict tuples being visible for the current user. For example, when row-level

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Amit Kapila
And besides if the decrements are decoupled from the allocation requests it's no longer obvious that the algorithm is even an approximation of LRU. I was trying to highlight that we can do the clocksweep in bgwriter and keep the backends logic as it is currently. The core idea is that it will

Re: [HACKERS] Why is indexonlyscan so darned slow?

2012-05-23 Thread Merlin Moncure
On Tue, May 22, 2012 at 11:33 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Mon, May 21, 2012 at 2:29 PM, Merlin Moncure mmonc...@gmail.com wrote: See here: http://www.devheads.net/database/postgresql/performance/index-all-necessary-columns-postgres-vs-mssql.htm for a 'in the wild' gripe

Re: [HACKERS] Archiver not exiting upon crash

2012-05-23 Thread Peter Eisentraut
On mån, 2012-05-21 at 12:52 -0400, Tom Lane wrote: I see Peter's commit d6de43099ac0bddb4b1da40088487616da892164 only touched postgres.c's quickdie(), and not all the *other* background processes with identical coding. That seems a clear oversight, so I will go fix it. None[*] of the other

Re: [HACKERS] Archiver not exiting upon crash

2012-05-23 Thread Peter Eisentraut
On mån, 2012-05-21 at 13:14 -0400, Tom Lane wrote: ... but having said that, I see Peter's commit d6de43099ac0bddb4b1da40088487616da892164 only touched postgres.c's quickdie(), and not all the *other* background processes with identical coding. That seems a clear oversight, so I will go

Re: [HACKERS] Archiver not exiting upon crash

2012-05-23 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On mån, 2012-05-21 at 13:14 -0400, Tom Lane wrote: ... wait, scratch that. AFAICS, that commit was totally useless, because BlockSig should always already contain SIGQUIT. No, because PostgresMain() deletes it from BlockSig. Ah. So potentially we

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Greg Smith
On 05/23/2012 11:36 AM, Amit Kapila wrote: Do you feel I can attempt to address this problem with some prototypes and discuss here after few days when I have some results ready. I don't think there is a clear picture yet of what benchmark to use for testing changes here. Items like Consider

Re: [HACKERS] Archiver not exiting upon crash

2012-05-23 Thread Fujii Masao
On Thu, May 24, 2012 at 1:26 AM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: On mån, 2012-05-21 at 13:14 -0400, Tom Lane wrote: ... wait, scratch that.  AFAICS, that commit was totally useless, because BlockSig should always already contain SIGQUIT. No,

Re: [HACKERS] Add primary key/unique constraint using prefix columns of an index

2012-05-23 Thread Robert Haas
On Tue, May 22, 2012 at 5:26 PM, Simon Riggs si...@2ndquadrant.com wrote:   A bigger problem is that creating such an index turns all of pgbench's write traffic from HOT updates into non-HOT updates, which means this is probably only going to be a win if the write volume is miniscule. Not

Re: [HACKERS] Add primary key/unique constraint using prefix columns of an index

2012-05-23 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I don't object to the feature, but I think it's real-world utility will be more limited than we might hope. When covering indexes are not in play, someone might choose to index only, say, the primary key. And maybe the primary key doesn't change

[HACKERS] spgist metapage

2012-05-23 Thread Robert Haas
A quick survey of our various index AMs reveals that: 1. btree and hash indexes have a metapage that begins with a 32-bit magic number, followed by a 32-bit version number. 2. spgist has a metapage with a version number, but no magic number. 3. gin has a metapage with a version number that is not

Re: [HACKERS] Exclusion Constraints on Arrays?

2012-05-23 Thread Peter Geoghegan
On 13 May 2012 23:45, Robert Haas robertmh...@gmail.com wrote: Hmm, it looks like GIN can't support exclusive constraints because amgettuple support is required, and unfortunately that got remove for GIN in this commit: commit ff301d6e690bb5581502ea3d8591a1600fd87acc That seems very

Re: [HACKERS] Exclusion Constraints on Arrays?

2012-05-23 Thread Robert Haas
On Wed, May 23, 2012 at 1:34 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 13 May 2012 23:45, Robert Haas robertmh...@gmail.com wrote: Hmm, it looks like GIN can't support exclusive constraints because amgettuple support is required, and unfortunately that got remove for GIN in this

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Amit Kapila
I don't think there is a clear picture yet of what benchmark to use for testing changes here. I will first try to generate such a scenario(benchmark). I have still not thought completely. However the idea in my mind is that scenario where buffer list is heavily operated upon. Operations where

Re: [HACKERS] Exclusion Constraints on Arrays?

2012-05-23 Thread David E. Wheeler
On May 23, 2012, at 10:35 AM, Robert Haas wrote: Exclusion constraints generalize uniqueness constraints by allowing arbitrary comparison operators, not just equalityThis is useful for time periods and other ranges, as well as arrays. I objected to it at the time, but lost the argument.

Re: [HACKERS] Add primary key/unique constraint using prefix columns of an index

2012-05-23 Thread Jeff Janes
On Tue, May 22, 2012 at 11:01 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, May 22, 2012 at 1:41 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Janes jeff.ja...@gmail.com writes: Now that there are index only scans, there is a use case for having a composite index which has the primary key

Re: [HACKERS] spgist metapage

2012-05-23 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Since we haven't actually released spgist yet, I'm wondering if it should be revised to look more like btree and hash - that is, add a 32-bit version number immediately following the magic number. All experience hath shown that it is often much more

Re: [HACKERS] Add primary key/unique constraint using prefix columns of an index

2012-05-23 Thread Robert Haas
On Wed, May 23, 2012 at 2:00 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Tue, May 22, 2012 at 11:01 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, May 22, 2012 at 1:41 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Janes jeff.ja...@gmail.com writes: Now that there are index only scans,

Re: [HACKERS] spgist metapage

2012-05-23 Thread Robert Haas
On Wed, May 23, 2012 at 2:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Since we haven't actually released spgist yet, I'm wondering if it should be revised to look more like btree and hash - that is, add a 32-bit version number immediately following the

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Jeff Janes
On Wed, May 23, 2012 at 10:33 AM, Amit Kapila amit.kap...@huawei.com wrote: I don't think there is a clear picture yet of what benchmark to use for testing changes here. I will first try to generate such a scenario(benchmark). I have still not thought completely. However the idea in my mind is

Re: [HACKERS] incorrect handling of the timeout in pg_receivexlog

2012-05-23 Thread Fujii Masao
On Tue, May 22, 2012 at 11:04 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, May 14, 2012 at 2:24 PM, Fujii Masao masao.fu...@gmail.com wrote: On Fri, May 11, 2012 at 11:43 PM, Magnus Hagander mag...@hagander.net wrote: Should we go down the easy way and just reject connections when the

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Jeff Janes
On Wed, May 23, 2012 at 8:36 AM, Amit Kapila amit.kap...@huawei.com wrote: And besides if the decrements are decoupled from the allocation requests it's no longer obvious that the algorithm is even an approximation of LRU. I was trying to highlight that we can do the clocksweep in bgwriter and

[HACKERS] Re: [COMMITTERS] pgsql: Send new protocol keepalive messages to standby servers.

2012-05-23 Thread Fujii Masao
On Sat, Dec 31, 2011 at 10:34 PM, Simon Riggs si...@2ndquadrant.com wrote: Send new protocol keepalive messages to standby servers. Allows streaming replication users to calculate transfer latency and apply delay via internal functions. No external functions yet. Is there plan to implement

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Amit Kapila
You need to have shared_buffers be much smaller than RAM, and have almost all the disk data resident in RAM but not in shared_buffers. Sure, this is better way to generate heavy activity buffers -Original Message- From: Jeff Janes [mailto:jeff.ja...@gmail.com] Sent: Wednesday, May 23,

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: One thing I wanted to play with is having newly read buffers get a usage count of 0 rather than 1. The problem is that there is no way to test it in enough different situations to convince people it would be a general improvement. Hmm ... ISTM that

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Amit Kapila
Do we have any evidence that this is actually a problem which needs to be solved? I don't have very clear evidence, didn't generate any profiling report still. However in the scenario Where there are many buffers in the list and backend has to traverse it, there should be reasonable work under

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Jeff Janes
On Wed, May 23, 2012 at 11:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Janes jeff.ja...@gmail.com writes: One thing I wanted to play with is having newly read buffers get a usage count of 0 rather than 1.  The problem is that there is no way to test it in enough different situations to

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Robert Haas
On Wed, May 23, 2012 at 2:09 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Wed, May 23, 2012 at 10:33 AM, Amit Kapila amit.kap...@huawei.com wrote: I don't think there is a clear picture yet of what benchmark to use for testing changes here. I will first try to generate such a

[HACKERS] adding and upgrading metapages

2012-05-23 Thread Robert Haas
On Wed, May 23, 2012 at 2:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: In any case, I would've expected we'd end up changing all these page formats if we go over to your concept of a metapage for everything. Fixing indexes that already have some kind of metapage will surely be trivial compared to

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-05-23 Thread Alastair Turner
On Wed, May 23, 2012 at 5:09 PM, Tom Lane t...@sss.pgh.pa.us wrote: Kohei KaiGai kai...@kaigai.gr.jp writes: Let me have a discussion to get preferable interface for row-level security. My planned feature will perform to append additional conditions to WHERE clause implicitly, to restrict

Re: [HACKERS] Changing the concept of a DATABASE

2012-05-23 Thread Peter Eisentraut
On tis, 2012-05-22 at 18:00 +0200, Susanne Ebrecht wrote: CREATE SCHEMA foo LC_COLLATE bar isn't supported so you went up a level and do it by creating a database. I would like to get default collation per schema / table in 9.2 or 9.3 but that is my personal wish, Another way I've been

Re: [HACKERS] External Open Standards

2012-05-23 Thread Peter Eisentraut
On mån, 2012-05-21 at 15:34 +1000, Brendan Jurd wrote: I'd be okay with just adding a note in the manual under Date/Time Output to the effect of Note: ISO 8601 specifies the use of uppercase letter 'T' to separate the date and time. Postgres uses a space for improved readability, in line with

Re: [HACKERS] Readme of Buffer Management seems to have wrong sentence

2012-05-23 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: On Wed, May 23, 2012 at 11:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: Hmm ... ISTM that that was discussed back when we instituted buffer usage counts, and rejected on the grounds that a newly-read buffer could then have negligible life expectancy.  The

Re: [HACKERS] Per-Database Roles

2012-05-23 Thread Stephen Frost
On May 22, 2012, at 12:18, Tom Lane t...@sss.pgh.pa.us wrote: Another objection is that it wouldn't scale up nicely to multiple levels of catalog hierarchy. But maybe local/global is enough. That would be a huge improvement and this wouldn't get in the way of any solution to the global oid

[HACKERS] should postgres -C print units?

2012-05-23 Thread Peter Eisentraut
The postgres -C option was added in 9.2. $ postgres -D ... -C shared_buffers 4096 Shouldn't that use units? We use them everywhere else in user interfaces. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] External Open Standards

2012-05-23 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On mån, 2012-05-21 at 15:34 +1000, Brendan Jurd wrote: I'd be okay with just adding a note in the manual under Date/Time Output to the effect of Note: ISO 8601 specifies the use of uppercase letter 'T' to separate the date and time. Postgres uses a

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-05-23 Thread Kohei KaiGai
2012/5/23 Tom Lane t...@sss.pgh.pa.us: Kohei KaiGai kai...@kaigai.gr.jp writes: Let me have a discussion to get preferable interface for row-level security. My planned feature will perform to append additional conditions to WHERE clause implicitly, to restrict tuples being visible for the

Re: [HACKERS] Archiver not exiting upon crash

2012-05-23 Thread Jeff Janes
On Mon, May 21, 2012 at 9:22 AM, Fujii Masao masao.fu...@gmail.com wrote: On Sat, May 19, 2012 at 1:23 AM, Jeff Janes jeff.ja...@gmail.com wrote: I've been testing the crash recovery of REL9_2_BETA1, using the same method I posted in the Scaling XLog insertion thread.  I have the checkpointer

Re: [HACKERS] incorrect handling of the timeout in pg_receivexlog

2012-05-23 Thread Magnus Hagander
On Wed, May 23, 2012 at 8:11 PM, Fujii Masao masao.fu...@gmail.com wrote: On Tue, May 22, 2012 at 11:04 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, May 14, 2012 at 2:24 PM, Fujii Masao masao.fu...@gmail.com wrote: On Fri, May 11, 2012 at 11:43 PM, Magnus Hagander mag...@hagander.net

Re: [HACKERS] should postgres -C print units?

2012-05-23 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: The postgres -C option was added in 9.2. $ postgres -D ... -C shared_buffers 4096 Shouldn't that use units? We use them everywhere else in user interfaces. Hm. But AFAIR, that feature is not meant to support user interfaces, it's for programs (such

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-05-23 Thread Kohei KaiGai
2012/5/23 Alastair Turner b...@ctrlf5.co.za: On Wed, May 23, 2012 at 5:09 PM, Tom Lane t...@sss.pgh.pa.us wrote: Kohei KaiGai kai...@kaigai.gr.jp writes: Let me have a discussion to get preferable interface for row-level security. My planned feature will perform to append additional conditions

Re: [HACKERS] Archiver not exiting upon crash

2012-05-23 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: It looks to me like the SIGQUIT from the postmaster is simply getting lost. And from what little I understand of signal handling, this is a known race with system(3). The archive_command, child of archiver, exits before it can receive the signal sent

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-05-23 Thread Robert Haas
On Wed, May 23, 2012 at 3:45 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: I wanted to have discussion to handle this problem. Unlike leaky-view problem, we don't need to worry about unexpected qualifier distribution on either side of join, because a scan on table never contains any join. Thus,

Re: [HACKERS] Changing the concept of a DATABASE

2012-05-23 Thread Josh Berkus
Simon, However, given sufficient people speaking against it, I'll leave this idea. Well, I *will* point out that you have your work cut out for you on 9.3 already ... -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Archiver not exiting upon crash

2012-05-23 Thread Tom Lane
I wrote: On my machine, man system(3) saith: system() ignores the SIGINT and SIGQUIT signals, and blocks the SIGCHLD signal, while waiting for the command to terminate. If this might cause the application to miss a signal that would have killed it, the application should

Re: [HACKERS] Archiver not exiting upon crash

2012-05-23 Thread Jeff Janes
On Wed, May 23, 2012 at 1:10 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Janes jeff.ja...@gmail.com writes: It looks to me like the SIGQUIT from the postmaster is simply getting lost.  And from what little I understand of signal handling, this is a known race with system(3).  The

Re: [HACKERS] Archiver not exiting upon crash

2012-05-23 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: On Wed, May 23, 2012 at 1:10 PM, Tom Lane t...@sss.pgh.pa.us wrote: On my machine, man system(3) saith: system() ignores the SIGINT and SIGQUIT signals, and blocks the SIGCHLD signal, while waiting for the command to terminate. If this

Re: [HACKERS] Changing the concept of a DATABASE

2012-05-23 Thread Simon Riggs
On 23 May 2012 21:15, Josh Berkus j...@agliodbs.com wrote: However, given sufficient people speaking against it, I'll leave this idea. Well, I *will* point out that you have your work cut out for you on 9.3 already ... Yes, we do. It would be best to conclude that things I do on hackers

Re: [HACKERS] Changing the concept of a DATABASE

2012-05-23 Thread David E. Wheeler
On May 23, 2012, at 1:55 PM, Simon Riggs wrote: Well, I *will* point out that you have your work cut out for you on 9.3 already ... Yes, we do. It would be best to conclude that things I do on hackers relate in some way to those goals, even if it isn't immediately clear how. Simon moves

Re: [HACKERS] Add primary key/unique constraint using prefix columns of an index

2012-05-23 Thread Simon Riggs
On 23 May 2012 18:13, Robert Haas robertmh...@gmail.com wrote: On Tue, May 22, 2012 at 5:26 PM, Simon Riggs si...@2ndquadrant.com wrote:   A bigger problem is that creating such an index turns all of pgbench's write traffic from HOT updates into non-HOT updates, which means this is probably

Re: [HACKERS] Archiver not exiting upon crash

2012-05-23 Thread Tom Lane
I wrote: Jeff Janes jeff.ja...@gmail.com writes: But what happens if the SIGQUIT is blocked before the system(3) is invoked? Does the ignore take precedence over the block, or does the block take precedence over the ignore, and so the signal is still waiting once the block is reversed after

Re: [HACKERS] WIP: parameterized function scan

2012-05-23 Thread Antonin Houska
On 05/22/2012 09:31 PM, Robert Haas wrote: This implementation looks different than I'd expect: I would have thought that it would work by generating paths with param_info set to the appropriate set of rels to provide the necessary values, rather than inventing its own mechanism for forcing a

Re: [HACKERS] WIP: parameterized function scan

2012-05-23 Thread Tom Lane
Antonin Houska antonin.hou...@gmail.com writes: On 05/22/2012 09:31 PM, Robert Haas wrote: Also, I think we will want something that implements the LATERAL() syntax, rather than just removing the prohibition on lateral references. So you think it's not good to first implement (implicitly)

Re: [HACKERS] External Open Standards

2012-05-23 Thread Brendan Jurd
On 24 May 2012 05:30, Peter Eisentraut pete...@gmx.net wrote: On mån, 2012-05-21 at 15:34 +1000, Brendan Jurd wrote: I'd be okay with just adding a note in the manual under Date/Time Output to the effect of Note: ISO 8601 specifies the use of uppercase letter 'T' to separate the date and time.

Re: [HACKERS] Patch: add conversion from pg_wchar to multibyte

2012-05-23 Thread Alexander Korotkov
On Tue, May 22, 2012 at 3:27 PM, Tatsuo Ishii is...@postgresql.org wrote: Thanks for your comments. They clarify a lot. But I still don't realize how can we distinguish IS_LCPRV2 and IS_LC2? Isn't it possible for them to produce same pg_wchar? If LB is in 0x90 - 0x99 range, then they are