Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Richard Huxton
Greg Smith wrote: Where I suspect this is all is going to settle down into is that if 1) the SE GUC is on and 2) one of the tables in a join has rows filtered, then you can expect that a) it's possible that the result will leak information, which certainly need to be documented, As far as I

Re: [HACKERS] log_duration_sample config option patch

2009-01-28 Thread Timo Savola
Euler Taveira de Oliveira wrote: IIRC pgFouine shows exact percentage of query by type; this new GUC implies we can not rely on it anymore. Also, you could skip us from logging long (and bad) running queries. Another statistics data in pgFouine will suffer from the same problem. I see your

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Peter Eisentraut
Greg Smith wrote: PostgreSQL advocacy point, one of the questions Tom asked about a bit upthread is still a bit hazy here. There are commercial database offerings selling into the trusted space already. While the use-cases you describe make perfect sense, I don't think it's clear to everyone

Re: [HACKERS] pg_upgrade project status

2009-01-28 Thread Zdenek Kotala
Heikki Linnakangas píše v st 28. 01. 2009 v 09:24 +0200: Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: That implies a fairly robust and configurable system for adding to and rewriting system catalogs, which today we haven't got. And we won't ever have, because it's

[HACKERS] Hot standby, recovery infra

2009-01-28 Thread Heikki Linnakangas
I've been reviewing and massaging the so called recovery infra patch. To recap, the goal is to: - start background writer during (archive) recovery - skip the shutdown checkpoint at the end of recovery. Instead, the database is brought up immediately, and the bgwriter performs a normal online

[HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Peter Eisentraut
I have re-reviewed the SE-PostgreSQL patch set. I don't want to talk about here whether the security model is appropriate, how foreign keys are to be handled etc. I want to discuss that I really don't like the architecture of this patch. I have said this before in previous review rounds,

Re: Commitfest infrastructure (was Re: [HACKERS] 8.4 release planning)

2009-01-28 Thread Magnus Hagander
Peter Eisentraut wrote: On Tuesday 27 January 2009 23:59:46 Magnus Hagander wrote: Marko Kreen wrote: On 1/27/09, Peter Eisentraut pete...@gmx.net wrote: On Tuesday 27 January 2009 15:51:02 Marko Kreen wrote: Such app already exists: http://ozlabs.org/~jk/projects/patchwork/

Re: Commitfest infrastructure (was Re: [HACKERS] 8.4 release planning)

2009-01-28 Thread Cédric Villemain
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Magnus Hagander a écrit : Peter Eisentraut wrote: On Tuesday 27 January 2009 23:59:46 Magnus Hagander wrote: Marko Kreen wrote: On 1/27/09, Peter Eisentraut pete...@gmx.net wrote: On Tuesday 27 January 2009 15:51:02 Marko Kreen wrote: Such app

[HACKERS] Column privileges for system catalogs

2009-01-28 Thread Peter Eisentraut
Is it now acceptable to use column privileges for system catalogs? For the new SQL/MED catalogs we have used the old system of revoking all permissions and having a filtered view on top of it (tradition since pg_shadow), but I figured we could do this properly now by just revoking permissions

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread KaiGai Kohei
Richard Huxton wrote: Greg Smith wrote: Where I suspect this is all is going to settle down into is that if 1) the SE GUC is on and 2) one of the tables in a join has rows filtered, then you can expect that a) it's possible that the result will leak information, which certainly need to be

Re: [HACKERS] Patch to add Windows 7 support

2009-01-28 Thread Magnus Hagander
Dave Page wrote: I don't think it's enough that we need to care about it really. I'm thinking we could perhaps even just never set that, and not bother with the version check... That was how I originally coded it, but figured we might as well set it if we can - it's not like it's expensive

Re: [HACKERS] Index Scan cost expression

2009-01-28 Thread Gregory Stark
Amit Gupta amit.pc.gu...@gmail.com writes: Moreover it only models a single index scan. It assumes nothing is cached prior to the index scan which is very much not true if we're repeatedly scanning similar ranges of keys. It's reasonable to assume that nothing is cached for estimating the

Re: [HACKERS] pg_upgrade project status

2009-01-28 Thread Stephen Frost
* Zdenek Kotala (zdenek.kot...@sun.com) wrote: And very important thing is that you need old version of postgreSQL installed, which is something what packagers does not want. Look on Oracle how does it. Just as a counter-point, Debian handles multiple concurrently installed versions of

Re: [HACKERS] Hot standby, recovery infra

2009-01-28 Thread Simon Riggs
On Wed, 2009-01-28 at 12:04 +0200, Heikki Linnakangas wrote: I've been reviewing and massaging the so called recovery infra patch. Thanks. I feel quite good about this patch now. Given the amount of code churn, it requires testing, and I'll read it through one more time after sleeping over

Re: [HACKERS] posix_fadvise v22

2009-01-28 Thread Gregory Stark
Tom Lane t...@sss.pgh.pa.us writes: What I intend to do over the next day or so is commit the prefetch infrastructure and the bitmap scan prefetch logic, but I'm bouncing the indexscan part back for rework. I think that it should be implemented in or near index_getnext() and pay attention

Re: [HACKERS] [COMMITTERS] pgsql: Silence compiler warning on win32.

2009-01-28 Thread Magnus Hagander
Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: Tom Lane wrote: Surely this patch is wrong. It is suppressing, not fixing, a critical warning about a datatype mismatch. You mean the signed vs unsigned part? Other than that, int and dword are always the same on win32... Hmm,

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread KaiGai Kohei
Sorry for long description again. Peter Eisentraut wrote: I have re-reviewed the SE-PostgreSQL patch set. I don't want to talk about here whether the security model is appropriate, how foreign keys are to be handled etc. I want to discuss that I really don't like the architecture of this

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Jonah H. Harris
On Wed, Jan 28, 2009 at 4:28 AM, Peter Eisentraut pete...@gmx.net wrote: Greg Smith wrote: PostgreSQL advocacy point, one of the questions Tom asked about a bit upthread is still a bit hazy here. There are commercial database offerings selling into the trusted space already. While the

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Magnus Hagander
Robert Treat wrote: The problem is that the pain point is extremely high for missing a given release cycle. If you don't make a specific release, you have a 12-14 month wait for feature arrival. Given that choice, someone who deperately need (aka wants) HS/SEPostgres/Win32/HOT/IPU/etc...

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Magnus Hagander
Josh Berkus wrote: That's modest. I've talked to several oracle and db2 shops that want a standby for reporting that has relatively easy setup/maintenance (handling ddl is a big part of this) and the HS feature your working on will give them something as good as what they are getting now. So

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Magnus Hagander
Robert Haas wrote: I think the best thing we could do overall is to set release dates and stick to them. If your patch is not ready, well, at least it will get out in a defined amount of time. Right now, the *real* problem with it being pushed to the next release is you don't know how

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Robert Haas
However, we have to make clear whether the PGACE architecture is incorrect, or not, at first. I think the name of PGACE is not important, but it is necessary to make SELinux's decision in similar strategic point in finally. I've been thinking about this issue as well. I think a framework of

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Gregory Stark
Magnus Hagander mag...@hagander.net writes: Josh Berkus wrote: One client is planning on deploying a rather complex FS cloning infrastructure just to have a bunch of reporting, testing and read-only search databases they need. They'd be thrilled with an HS feature which produced DBs which

Re: [HACKERS] Hot standby, recovery infra

2009-01-28 Thread Fujii Masao
Hi, On Wed, Jan 28, 2009 at 7:04 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I've been reviewing and massaging the so called recovery infra patch. Great! I feel quite good about this patch now. Given the amount of code churn, it requires testing, and I'll read it

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Andrew Sullivan
On Wed, Jan 28, 2009 at 11:31:25AM +0900, KaiGai Kohei wrote: As I noted before, there is a symmetrical structure between OS and DBMS. Well, you said that before. I think your analogy is awful. I don't think the similarities are nearly as great as you think, and I also think there are

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Stephen Frost
* KaiGai Kohei (kai...@kaigai.gr.jp) wrote: So, I cannot believe refactoring pg_xxx_aclcheck() is not acceptable. If vanilla PostgreSQL become to check ACLs on tables, independent from views, do you think it is acceptable? Well, just to be clear, ACLs are checked on tables under views, but

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Robert Haas
That being said, I'd think that if we do need different semantics from that for SE-PostgreSQL, we could implement it using a GUC or similar to keep the current behavior as well allow the SE-PostgreSQL behavior. I don't think a GUC is what you need because you need both behaviors

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread KaiGai Kohei
Robert Haas wrote: I haven't read the code, but from reading the docs, I have a feeling that right now the answer to both questions are NO, which means it doesn't really have a lot of value. One example of this is the pg_security system catalog. The catalog representation you're proposing is

Re: [HACKERS] Hot standby, recovery infra

2009-01-28 Thread Simon Riggs
On Wed, 2009-01-28 at 23:19 +0900, Fujii Masao wrote: @@ -355,6 +359,27 @@ BackgroundWriterMain(void) */ PG_SETMASK(UnBlockSig); + BgWriterRecoveryMode = IsRecoveryProcessingMode(); + + if (BgWriterRecoveryMode) + elog(DEBUG1, bgwriter starting during

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Robert Haas
Text represented security attribute is the most common format for any other security mechanism also. (For example, T-SOL also have its text representation.) In addition, TEXT is the most flexible format. If any other feature want to handle it as an array, it can be stored as a text

Re: [HACKERS] Hot standby, recovery infra

2009-01-28 Thread Fujii Masao
Hi, On Wed, Jan 28, 2009 at 11:47 PM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2009-01-28 at 23:19 +0900, Fujii Masao wrote: @@ -355,6 +359,27 @@ BackgroundWriterMain(void) */ PG_SETMASK(UnBlockSig); + BgWriterRecoveryMode = IsRecoveryProcessingMode(); + + if

Re: [HACKERS] Column privileges for system catalogs

2009-01-28 Thread Euler Taveira de Oliveira
Peter Eisentraut escreveu: Is it now acceptable to use column privileges for system catalogs? For the new SQL/MED catalogs we have used the old system of revoking all permissions and having a filtered view on top of it (tradition since pg_shadow), but I figured we could do this properly now

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread KaiGai Kohei
Stephen Frost wrote: * KaiGai Kohei (kai...@kaigai.gr.jp) wrote: So, I cannot believe refactoring pg_xxx_aclcheck() is not acceptable. If vanilla PostgreSQL become to check ACLs on tables, independent from views, do you think it is acceptable? Well, just to be clear, ACLs are checked on

Re: [HACKERS] [COMMITTERS] pgsql: Silence compiler warning on win32.

2009-01-28 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: Tom Lane wrote: Hmm, need more caffeine I guess. I was thinking dword == long. But in any case, I'd feel a lot more comfortable if the patch ifdef'd the declaration of exit_status to match, rather than forcing a cast of the pointer value. Hmm. I

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread KaiGai Kohei
KaiGai Kohei wrote: I'm not sure about KaiGai's feelings on this, but it strikes me that adding SELinux support for the existing levels of access control in PG might be straight-forward and small enough to include for 8.4 and would show some commitment to this approach of do it for PG, add

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Simon Riggs
On Wed, 2009-01-28 at 14:55 +0100, Magnus Hagander wrote: If the release is pushed back, maybe some other patch could also have been finished by the new deadline - should that also be included? What about a completely new feature that isn't even started yet, but that could easily be finished

Re: [HACKERS] mingw check hung

2009-01-28 Thread Magnus Hagander
Andrew Dunstan wrote: Andrew Dunstan wrote: Magnus Hagander wrote: Andrew Dunstan wrote: I've installed drmingw to handle exceptions instead, so we'll see if that gives us useful info. If not, I'll see what I can do with gdb. Hadn't heard of drwmingw, I see how that can be

Re: [HACKERS] Hot standby, recovery infra

2009-01-28 Thread Simon Riggs
On Wed, 2009-01-28 at 23:54 +0900, Fujii Masao wrote: Why is InitXLOGAccess() called also here when bgwriter is started after recovery? That is already called by AuxiliaryProcessMain(). InitXLOGAccess() sets the timeline and also gets the latest record pointer. If the bgwriter is

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread KaiGai Kohei
Even if I implement SE-PostgreSQL as a loadable module, core PostgreSQL has to provide proper hooks in strategic points and facilities to manage security attribute (pg_security system catalog and security_label system column). If you require to implement it without these facilities, I think it is

Re: [HACKERS] Hot standby, recovery infra

2009-01-28 Thread Heikki Linnakangas
Fujii Masao wrote: On Wed, Jan 28, 2009 at 7:04 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I feel quite good about this patch now. Given the amount of code churn, it requires testing, and I'll read it through one more time after sleeping over it. Simon, do you see

Re: [HACKERS] [COMMITTERS] pgsql: Silence compiler warning on win32.

2009-01-28 Thread Magnus Hagander
Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: Tom Lane wrote: Hmm, need more caffeine I guess. I was thinking dword == long. But in any case, I'd feel a lot more comfortable if the patch ifdef'd the declaration of exit_status to match, rather than forcing a cast of the

Re: [HACKERS] pg_upgrade project status

2009-01-28 Thread Tom Lane
Zdenek Kotala zdenek.kot...@sun.com writes: Heikki Linnakangas píše v st 28. 01. 2009 v 09:24 +0200: Right, the dump+initdb+reload approach works quite well in both pg_upgrade and pg-migrator. I believe the biggest issue with that ATM is supporting dropped columns, and maybe there's

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Joshua Brindle
KaiGai Kohei wrote: Stephen Frost wrote: * KaiGai Kohei (kai...@kaigai.gr.jp) wrote: So, I cannot believe refactoring pg_xxx_aclcheck() is not acceptable. If vanilla PostgreSQL become to check ACLs on tables, independent from views, do you think it is acceptable? Well, just to be clear, ACLs

Re: [HACKERS] mingw check hung

2009-01-28 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: Andrew Dunstan wrote: The suspect patch is quite definitely the source of the problem. I can't spot the error right off :-( Can you try to see if it's the putenv() or the unsetenv() that gets broken? Are we sure pgwin32_unsetenv works in this

Re: [HACKERS] Column privileges for system catalogs

2009-01-28 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: Is it now acceptable to use column privileges for system catalogs? Sure, to the same extent that table privileges work for them (ie, don't expect the C code to pay any attention ;)). For the new SQL/MED catalogs we have used the old system of revoking

Re: [HACKERS] pg_upgrade project status

2009-01-28 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Zdenek Kotala (zdenek.kot...@sun.com) wrote: And very important thing is that you need old version of postgreSQL installed, which is something what packagers does not want. Look on Oracle how does it. Just as a counter-point, Debian handles multiple

[HACKERS] using composite types in insert/update

2009-01-28 Thread Merlin Moncure
IMO, composite types on insert/update should work as they do on select: SELECT foo FROM foo; -- works, returning type foo INSERT INTO foo VALUES '(something)'::foo -- fails, but we have a workaround: INSERT INTO foo SELECT ('(something)'::foo).* -- expands foo into foo columns however no such

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Robert Treat
On Wednesday 28 January 2009 08:55:56 Magnus Hagander wrote: We're still going to have to pay the full cost of doing a release every time. With beta/rc management, release notes, announcements, postings, packaging and all those things. As I pointed out to Tom, by percentage the additional

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Tom Lane
Robert Treat xzi...@users.sourceforge.net writes: On Wednesday 28 January 2009 08:55:56 Magnus Hagander wrote: We're still going to have to pay the full cost of doing a release every time. With beta/rc management, release notes, announcements, postings, packaging and all those things. As I

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Tom Lane
Joshua Brindle met...@manicmethod.com writes: I'm not sure how much it would cut to remove row level access controls, but I do have some points here. To me, row level access controls are the most important part, this is the feature that lets us put secret and top secret data in the same table

Re: [HACKERS] pg_upgrade project status

2009-01-28 Thread Robert Haas
On Wed, Jan 28, 2009 at 10:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: Zdenek Kotala zdenek.kot...@sun.com writes: Heikki Linnakangas píše v st 28. 01. 2009 v 09:24 +0200: Right, the dump+initdb+reload approach works quite well in both pg_upgrade and pg-migrator. I believe the biggest issue with

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Joshua Brindle
Tom Lane wrote: Joshua Brindle met...@manicmethod.com writes: I'm not sure how much it would cut to remove row level access controls, but I do have some points here. To me, row level access controls are the most important part, this is the feature that lets us put secret and top secret data in

Re: [HACKERS] Output filter for psql

2009-01-28 Thread D'Arcy J.M. Cain
On Thu, 8 Jan 2009 17:22:04 -0500 D'Arcy J.M. Cain da...@druid.net wrote: So, I guess psql should pass XML to the user's filter and simply dump its output to the screen. Is XML the best format? It would allow us to pass meta information. For example, we could pass the border setting so that

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Gregory Stark
I skimmed through the Hot Standby patch for a preliminary review. I noted the following things, some minor tweaks, some just questions. None of the things I noted are big issues unless some of the questions uncover issues. 1) This code is obviously a cut-pasto: + else if (strcmp(tok1,

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Gregory Stark
Joshua Brindle met...@manicmethod.com writes: partitions don't help because, once again, the application would be making the determination about which partition to query. Not necessarily since the database can be programmed to automatically put the records into the right partition. Right now

Re: [HACKERS] pg_upgrade project status

2009-01-28 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Jan 28, 2009 at 10:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: Sure, but the other way is just a complete non-starter. Well, the goal of coding is to make such things easier. Already the solution you're advocating has one huge wart: the need to

Re: [HACKERS] Output filter for psql

2009-01-28 Thread Andrew Dunstan
D'Arcy J.M. Cain wrote: On Thu, 8 Jan 2009 17:22:04 -0500 D'Arcy J.M. Cain da...@druid.net wrote: So, I guess psql should pass XML to the user's filter and simply dump its output to the screen. Is XML the best format? It would allow us to pass meta information. For example, we could

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Simon Riggs
On Wed, 2009-01-28 at 18:55 +, Gregory Stark wrote: I skimmed through the Hot Standby patch for a preliminary review. I noted the following things, some minor tweaks, some just questions. None of the things I noted are big issues unless some of the questions uncover issues. Thanks for

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Joshua D. Drake
On Wed, 2009-01-28 at 19:27 +, Simon Riggs wrote: On Wed, 2009-01-28 at 18:55 +, Gregory Stark wrote: Agreed. As explained when I published that patch it is deliberately severe to allow testing of conflict resolution and feedback on it. I still *strongly* feel the default has to be

Re: [HACKERS] mingw check hung

2009-01-28 Thread Mark Cave-Ayland
Magnus Hagander wrote: Per discussion I looked at just reverting that part, but that won't work. If we do that, the call to SetEnvironmentVariable() will not be run, which certainly isn't right.. The problem has to be in win32env.c. I originally thought we accidentally called the putenv

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Tom Lane
Gregory Stark st...@enterprisedb.com writes: I don't think partitioning is really the same thing as row-level security. Of course not, but it seems to me that it can be used to accomplish most of the same practical use-cases. The main gripe about doing it via partitioning is that the user's

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Heikki Linnakangas
Gregory Stark wrote: 6) I still don't understand why you need unobserved_xids. We don't need this in normal running, an xid we don't know for certain is committed is exactly the same as a transaction we know is currently running or aborted. So why do you need it during HS? In normal operation,

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Andrew Sullivan
On Wed, Jan 28, 2009 at 01:49:21PM -0500, Joshua Brindle wrote: use. The people that need them understand the ramifications of row filtering and the absence of inaccessible rows won't be surprising. I wish there was even a little bit of evidence that users of a security system may be relied

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Simon Riggs
On Wed, 2009-01-28 at 18:55 +, Gregory Stark wrote: I still don't understand why you need unobserved_xids. We don't need this in normal running, an xid we don't know for certain is committed is exactly the same as a transaction we know is currently running or aborted. So why do you need

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Tom Lane
Joshua D. Drake j...@commandprompt.com writes: On Wed, 2009-01-28 at 19:27 +, Simon Riggs wrote: On Wed, 2009-01-28 at 18:55 +, Gregory Stark wrote: I still *strongly* feel the default has to be the non-destructive conservative -1. I don't. Primarily, we must support high

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Aidan Van Dyk
* Tom Lane t...@sss.pgh.pa.us [090128 15:02]: Well, those unexpectedly cancelled queries could have represented critical functionality too. I think this argument calls the entire approach into question. If there is no safe setting for the parameter then we need to find a way to not have

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Greg Stark
(sorry for top posting -- blame apple) I don't see anything dangerous with either setting. For use cases where the backup is the primary purpose then killing queries is fine. For use cases where the maching is a reporting machine then saving large amounts of archived logs is fine.

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Greg Stark
Put another way: your characterization is no more true than claiming there's no safe setting for statement_timeout since a large value means clog could overflow your disk and your tables could bloat. (I note we default statement_timeout to off though) -- Greg On 28 Jan 2009, at 19:56, Tom

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Simon Riggs
On Wed, 2009-01-28 at 21:41 +0200, Heikki Linnakangas wrote: So, you can think of the unobserved xids array as an extension of ProcArray. The entries are like light-weight PGPROC entries. In fact I proposed earlier to simply create dummy PGPROC entries instead. Which we don't do because we

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Jeff Davis
On Wed, 2009-01-28 at 19:27 +, Simon Riggs wrote: my failover was 12 hours behind when I needed it to be 10 seconds behind and I lost a $1 million because of downtime of Postgres The same could be said for warm standby right now. Or Slony-I, for that matter. I think that we can reasonably

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Heikki Linnakangas
Tom Lane wrote: Joshua D. Drake j...@commandprompt.com writes: On Wed, 2009-01-28 at 19:27 +, Simon Riggs wrote: On Wed, 2009-01-28 at 18:55 +, Gregory Stark wrote: I still *strongly* feel the default has to be the non-destructive conservative -1. I don't. Primarily, we must support

Re: [HACKERS] Output filter for psql

2009-01-28 Thread D'Arcy J.M. Cain
On Wed, 28 Jan 2009 14:08:54 -0500 Andrew Dunstan and...@dunslane.net wrote: D'Arcy J.M. Cain wrote: I suppose we could define another line with options that we could define for meta information such as the border setting and the table name and whatever we define later. E.g:

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Simon Riggs
On Wed, 2009-01-28 at 11:33 -0800, Joshua D. Drake wrote: On Wed, 2009-01-28 at 19:27 +, Simon Riggs wrote: On Wed, 2009-01-28 at 18:55 +, Gregory Stark wrote: Agreed. As explained when I published that patch it is deliberately severe to allow testing of conflict resolution and

Re: [HACKERS] mingw check hung

2009-01-28 Thread Magnus Hagander
Mark Cave-Ayland wrote: Magnus Hagander wrote: Per discussion I looked at just reverting that part, but that won't work. If we do that, the call to SetEnvironmentVariable() will not be run, which certainly isn't right.. The problem has to be in win32env.c. I originally thought we

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Simon Riggs
On Wed, 2009-01-28 at 14:56 -0500, Tom Lane wrote: Well, those unexpectedly cancelled queries could have represented critical functionality too. I think this argument calls the entire approach into question. If there is no safe setting for the parameter then we need to find a way to not

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Heikki Linnakangas
Simon Riggs wrote: The essential choice is What would you like the max failover time to be?. Some users want one server with max 5 mins behind, some want two servers, one with 0 seconds behind, one with 12 hours behind It's not quite that simple. Setting max_standby_delay=5mins means that

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Simon Riggs
On Wed, 2009-01-28 at 22:47 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: The essential choice is What would you like the max failover time to be?. Some users want one server with max 5 mins behind, some want two servers, one with 0 seconds behind, one with 12 hours behind It's

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Jeff Davis
On Wed, 2009-01-28 at 22:47 +0200, Heikki Linnakangas wrote: It's not quite that simple. Setting max_standby_delay=5mins means that you're willing to wait 5 minutes for each query to die. Which means that in worst case you have to stop for 5 minutes at every single vacuum record, and fall

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Tom Lane
Andrew Sullivan a...@crankycanuck.ca writes: On Wed, Jan 28, 2009 at 01:49:21PM -0500, Joshua Brindle wrote: The costs are nil for people who don't want this feature. That's also false, because developers who don't care about the feature have to continue to maintain it as part of the system.

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Robert Haas
I don't. Primarily, we must support high availability. It is much better if we get people saying I get my queries cancelled and we say RTFM and change parameter X, than if people say my failover was 12 hours behind when I needed it to be 10 seconds behind and I lost a $1 million because of

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Dimitri Fontaine
Sorry for top-posting and avoiding to paste online doc URL. Joshua, you sound like you're missing an important point. Sorry for misunderstanding if that's not true. Partitioning is supported in a way that a query does not need to know about it, be it a SELECT, INSERT, UPDATE or DELETE. And

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Gregory Stark
Simon Riggs si...@2ndquadrant.com writes: On Wed, 2009-01-28 at 14:56 -0500, Tom Lane wrote: Well, those unexpectedly cancelled queries could have represented critical functionality too. I think this argument calls the entire approach into question. If there is no safe setting for the

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I vote with Simon. The thing is that if you get some queries cancelled, you'll realize you have a problem. ... or if you don't, they couldn't have been all that critical. Having your failover be 12 hours behind (or 12 months behind) is something

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Dimitri Fontaine
Le 28 janv. 09 à 16:22, Simon Riggs si...@2ndquadrant.com a écrit : The only way to keep the dev window open longer is to overlap the start of the next cycle with the previous one. i.e. branch new version before final release. This is the second time the idea is raised and I like it. Do

Re: [HACKERS] Column privileges for system catalogs

2009-01-28 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: I don't have any objection to changing the catalog's own permissions that way, but the filtered view still has a usability advantage: you can just go select * from Is it reasonable to change the catalog permissions and keep the view too? I've fine

Re: [HACKERS] pg_upgrade project status

2009-01-28 Thread Zdenek Kotala
Tom Lane píše v st 28. 01. 2009 v 14:06 -0500: Trying to do catalog upgrade in-place is going to be a complete mess. I'd be interested to know, for example, how you imagine rearranging the contents of pg_class would work. You don't get to modify pg_class if you can't even find it, which

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Tom Lane
Dimitri Fontaine dfonta...@hi-media.com writes: Le 28 janv. 09 à 16:22, Simon Riggs si...@2ndquadrant.com a écrit : The only way to keep the dev window open longer is to overlap the start of the next cycle with the previous one. i.e. branch new version before final release. This is the

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Robert Haas
On Wed, Jan 28, 2009 at 3:58 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andrew Sullivan a...@crankycanuck.ca writes: On Wed, Jan 28, 2009 at 01:49:21PM -0500, Joshua Brindle wrote: The costs are nil for people who don't want this feature. That's also false, because developers who don't care about

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Bruce Momjian
Zdenek Kotala wrote: Bruce Momjian p??e v po 26. 01. 2009 v 23:02 -0500: OK, time for me to chime in. I think the outstanding commit-fest items can be broken down into four sections: o Log streaming o Hot standby o SE-PostgreSQL o Others You omit

Re: [HACKERS] pg_upgrade project status

2009-01-28 Thread Robert Haas
On Wed, Jan 28, 2009 at 5:00 PM, Zdenek Kotala zdenek.kot...@sun.com wrote: Tom Lane píše v st 28. 01. 2009 v 14:06 -0500: Trying to do catalog upgrade in-place is going to be a complete mess. I'd be interested to know, for example, how you imagine rearranging the contents of pg_class would

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Robert Haas
I considered pg_upgrade one of the others on the list; it is not as complex as the previous three. LOL. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: For me, the row-level access controls are really the sticking point. There is absolutely nothing you can say that will convince me that they don't break SQL in fundamental ways, and I also don't believe that it's going to be possible to implement them

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Ron Mayer
Joshua Brindle wrote: Tom Lane wrote: Joshua Brindle met...@manicmethod.com writes: I'm not sure how much it would cut to remove row level access controls, but I do have some points here. To me, row level access controls are the most important part, this is the feature that lets us put

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Ron Mayer
Stephen Frost wrote: And, just to go full circle, row-level access controls are exactly what the other enterprise RDBMSs have and is what is used in these security circles today. One of the major issues, as I understand it, is to be able to use stock applications with multiple security levels

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Robert Treat
On Wednesday 28 January 2009 12:35:42 Tom Lane wrote: Robert Treat xzi...@users.sourceforge.net writes: On Wednesday 28 January 2009 08:55:56 Magnus Hagander wrote: We're still going to have to pay the full cost of doing a release every time. With beta/rc management, release notes,

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Bruce Momjian
Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jan 27, 2009 at 12:52 PM, Tom Lane t...@sss.pgh.pa.us wrote: Right, but you expect that to be a small and predictable cost, say in the single-digits-percentage range. Plan optimizations that suddenly stop happening can

Re: [HACKERS] Output filter for psql

2009-01-28 Thread Andrew Dunstan
D'Arcy J.M. Cain wrote: On Wed, 28 Jan 2009 14:08:54 -0500 Andrew Dunstan and...@dunslane.net wrote: D'Arcy J.M. Cain wrote: I suppose we could define another line with options that we could define for meta information such as the border setting and the table name and whatever we

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Bruce Momjian
Simon Riggs wrote: On Tue, 2009-01-27 at 14:18 -0500, Tom Lane wrote: Joshua Brindle met...@manicmethod.com writes: Tom Lane wrote: Right, which is why it's bad for something like a foreign key constraint to expose the fact that the row does exist after all. Once again, this is

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Bruce Momjian
Robert Haas wrote: The flaw in that argument is that as you are doing it, the de-optimization only happens on queries that actually need the behavior. As the SEPostgres patch is constructed, the planner could *never* trust an FK for optimization since it would have no way to know whether

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: Tom Lane wrote: As the SEPostgres patch is constructed, the planner could *never* trust an FK for optimization since it would have no way to know whether row level permissions might be present (perhaps only for some rows) at execution time. You could

  1   2   >