[HACKERS] Hot standby, pausing recovery

2009-10-16 Thread Heikki Linnakangas
This if-block is misplaced:

   case RECOVERY_TARGET_STOP_IMMEDIATE:
   case RECOVERY_TARGET_STOP_XID:
   case RECOVERY_TARGET_STOP_TIME:
   paused = false;
   break;
 
   /*
* If we're paused, and mode has changed reset to allow 
 new settings
* to apply and maybe allow us to continue.
*/
   if (paused  prevRecoveryTargetMode != 
 recoveryTargetMode)
   paused = false;
 
   case RECOVERY_TARGET_PAUSE_XID:

Where was that supposed to go?

In advance-mode, we will merrilly skip over a WAL record that's a
recovery stop target. Is that a bug or a feature?

If you pause recovery, and then continue, we reset to target none
mode, even if a stopping point was set previously. That doesn't seem
right to me.

-- 
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Hot standby, pausing recovery

2009-10-16 Thread Simon Riggs
On Fri, 2009-10-16 at 09:33 +0300, Heikki Linnakangas wrote:

 In advance-mode, we will merrilly skip over a WAL record that's a
 recovery stop target. Is that a bug or a feature?

Merrily?!? I saw it more as a sombre stepping motion.

Advance currently means set target at next record and continue. Only one
target is currently supported, which means there is no other recovery
stop target.

 If you pause recovery, and then continue, we reset to target none
 mode, even if a stopping point was set previously. 

Yes, currently. Resetting the target mode will do what you want, rather
than continue.

 That doesn't seem right to me.

We can define these things various ways and I'm happy to change it to be
the way people find most intuitive and useful. You're the first person
to give me feedback on how these things should work, so I'm happy to
hear how you would like it to work.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] operator exclusion constraints [was: generalized index constraints]

2009-10-16 Thread Jeff Davis
On Sun, 2009-09-20 at 17:54 -0400, Tom Lane wrote:
 (I'm wondering a bit if anyone will want a WHERE clause, too, though
 adding that later shouldn't pose any big syntactic obstacles.)

Where should I put the WHERE clause? My current syntax (with patch) is:

[ CONSTRAINT constraint_name ]
  EXCLUSION [USING index_method] (expression CHECK WITH operator [, ...]) 
index_parameters }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
[ WHERE predicate ]

That's a little awkward to document, because WHERE is only supported for
operator exclusion constraints, so it doesn't just fit along side CHECK
and FOREIGN KEY. My only concern is that it would make the CREATE TABLE
syntax slightly harder to read.

We could allow the WHERE clause to be syntactically correct for all the
other constraints, but throw a not implemented error if they try to
use it. I'm not sure if that fits nicely with the spec or not.

I tried to move the WHERE clause right before or after the
index_parameters, but that resulted in shift/reduce conflicts.

Thoughts?

Regards,
Jeff Davis


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] inefficient use of relation extension?

2009-10-16 Thread Simon Riggs
On Thu, 2009-10-15 at 19:58 -0400, Tom Lane wrote:
 Alvaro Herrera alvhe...@commandprompt.com writes:
  Hmm ... this is something that had not occured to me earlier.  There is
  a connection pool here (JDBCConnectionPool I'm told; hadn't heard about
  that one) and there are about 100 backends permanently, not all of which
  are always busy.  Perhaps what's going on here is that some of them are
  idle for long enough that the sinval queue gets full.
 
 Hm, that's definitely possible, and 8.1 did not have very good code for
 coping with sinval overrun.  But it's not clear to me why that would
 affect the rel extension code path in particular.

I don't think this is an issue that affects the rel extension path
alone.

The typical behaviour is to attempt to assign work, if connection busy
then start a new connection and do work there. If the type of work being
done is similar then this behaviour means that contention leads to
additional contention. So *any* form of contention gets magnified.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Trigger with WHEN clause (WIP)

2009-10-16 Thread Simon Riggs
On Thu, 2009-10-15 at 16:54 -0500, Kevin Grittner wrote:
 Dimitri Fontaine dfonta...@hi-media.com wrote:
  
  It's pretty often the case (IME) that calling a trigger is the only
  point in the session where you fire plpgsql, and that's a visible
  cost.
  
 Wouldn't a connection pool solve this?

No

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Trigger with WHEN clause (WIP)

2009-10-16 Thread Simon Riggs
On Fri, 2009-10-16 at 10:14 +0900, Itagaki Takahiro wrote:
 Tom Lane t...@sss.pgh.pa.us wrote:
 
  Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp writes:
   I think there is a benefit to provide WHEN cluase at least
   for compatibility with other DBMSs, even through we can move
   the expressions into the body of trigger functions.
  
  This seems to me to be a lot of code to accomplish nothing useful.
  It will always be the case that any nontrivial logic has to be done
  inside the trigger.  And the compatibility argument is entirely
  pointless given the lack of compatibility in the trigger function
  itself.
 
 I see that WHEN cluase is not always needed,
 but I think there are several benefits to have it:
 
   * WHEN cluase is in SQL standard.
 
   * We could recheck trigger conditions when NEW tuple is modified.
 (not yet implemented in the patch, though)
 http://archives.postgresql.org/pgsql-hackers/2009-09/msg00286.php
 
   * As for compatibility, it is easy for typical users to move trigger
 bodies into a function, but they don't like to merge the condition
 and the bodies into a function in my experience.

+1 because

* It is SQL Standard
* Other RDBMS support it (Oracle, DB2)
* It will reduce size of in-memory pending trigger list (for large
statements) and avoid invoking run-time of function handlers, important
for heavier PLs (for small statements)

I also support addition of INSTEAD OF triggers for first two reasons and
because it may be an easier route to allow updateable views.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reworks for Access Control facilities (r2363)

2009-10-16 Thread Heikki Linnakangas
KaiGai Kohei wrote:
 The purpose of this patch is to provide function entrypoints for the
 upcoming SE-PostgreSQL feature, because I got a few comments that we
 hesitate to put sepgsql_xxx() hooks on the main routines directly in
 the first commit fest. In addition, I already tried to put SE-PG hooks
 within pg_xxx_aclchecks() in this CF, but it was failed due to the
 differences in the security models.

Can you elaborate that? It might well be that you need to adapt the
SE-PostgreSQL security model to the one that's there already. Putting
SE-PG hooks into existing pg_xxx_aclcheck functions is the only
low-impact way I can see to implement SE-PostgreSQL.

 * There are two special-purpose shims, ac_database_calculate_size and
 ac_tablespace_calculate_size, that got added for the benefit of
 utils/adt/dbsize.c.  What if that code were still in contrib?  How is it
 different from a lot of the code that is in contrib now, eg dblink or
 pgrowlocks, to say nothing of third-party modules?  Presuming that the
 shim layer can know explicitly about each individual permission-checking
 requirement is a dead-end design.
 
 Back to the definition of access controls (or reference monitor).
 It prevents violated accesses launched by user's requests (SQL).
 It is not a job to protect something from malicious internal modules.

The issue isn't malicious modules, but modules that have pg_xxx_aclcheck
calls in them and haven't been modified to do SE-pgsql checks like you
modified all the backend code. As the patch stands, they would perform
just the regular acl checks and bypass SE-pgsql.

-- 
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Unicode UTF-8 table formatting for psql text output

2009-10-16 Thread Peter Eisentraut
(HTML mail to preserve formatting; let's see if it works.)

I like the new Unicode tables, but the marking of continuation lines
looks pretty horrible:

 List of databases
 Name  │ Owner │ Encoding │ Collation │ Ctype │ Access
privileges
───┼───┼──┼───┼───┼───
 pl_regression │ peter │ LATIN2   │ cs_CZ │ cs_CZ │
 postgres  │ peter │ LATIN2   │ cs_CZ │ cs_CZ │
 template0 │ peter │ LATIN2   │ cs_CZ │ cs_CZ │ =c/peter
   ╷   ╷  ╷   ╷   ╎ peter=CTc/peter
 template1 │ peter │ LATIN2   │ cs_CZ │ cs_CZ │ =c/peter
   ╷   ╷  ╷   ╷   ╎ peter=CTc/peter
(4 rows)

This looks more like a rendering mistake than something sensible, and
also it doesn't actually help the viewer to tell what lines are
continued, which was the original purpose.

On a side note, I don't really understand why the access privileges need
to be broken up over multiple lines.  There is plenty of space left on
the line.

Note that the above is close to a default setup, so that is what many
people will see by default from now on.


Re: [HACKERS] DELETE not seeing expected rows before COPY on 8.4.1

2009-10-16 Thread Robert Haas
On Thu, Oct 15, 2009 at 11:44 PM, Greg Sabino Mullane g...@turnstep.com wrote:
 The DELETE returns 0 rows (and thus is the problem) according to the script.

I've had this problem before when there is a BEFORE DELETE trigger
that is inadvertently preventing the deletion, though this seems hard
to rationalize with the rest of what you're seeing...   still, any
triggers or rules on these tables?

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] DELETE not seeing expected rows before COPY on 8.4.1

2009-10-16 Thread Greg Sabino Mullane

-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160

 The DELETE returns 0 rows (and thus is the problem) according to the script.

 I've had this problem before when there is a BEFORE DELETE trigger
 that is inadvertently preventing the deletion, though this seems hard
 to rationalize with the rest of what you're seeing...   still, any
 triggers or rules on these tables?

There is a trigger on both sides that does an insert to another table,
but it should not prevent the deletion. Further, this session is
running 'replica' mode. Going to do some more debugging this morning...

- --
Greg Sabino Mullane g...@turnstep.com
End Point Corporation
PGP Key: 0x14964AC8 200910160720
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-BEGIN PGP SIGNATURE-

iEYEAREDAAYFAkrYVxYACgkQvJuQZxSWSsh8pgCdGtYA1tzFxeaq+Uwdij/BEsKf
recAmweTdVFQsZc19qqu8wJNfSjvTU2+
=eLya
-END PGP SIGNATURE-



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Trigger with WHEN clause (WIP)

2009-10-16 Thread Peter Eisentraut
On Fri, 2009-10-16 at 09:31 +0100, Simon Riggs wrote:
 * It will reduce size of in-memory pending trigger list (for large
 statements)

But this won't be the outcome when it's implemented the way it is being
proposed, which checks the where clause just before executing the
trigger function.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Unicode UTF-8 table formatting for psql text output

2009-10-16 Thread Roger Leigh
On Fri, Oct 16, 2009 at 01:38:15PM +0300, Peter Eisentraut wrote:
 (HTML mail to preserve formatting; let's see if it works.)
 
 I like the new Unicode tables, but the marking of continuation lines
 looks pretty horrible:

Yes, I'm not so keen myself.  The ASCII characters used are '|', ':' and
' ' for normal, wrapped and newlines.  Here, we are using vertical lines
with two or three dashes to replace the latter two.

As mentioned earlier in the thread, this patch will be followed up
with a futher patch to unify the folding/newline behaviour between
column header and data lines, which will replace these with normal
vertical lines in all cases, with the addition of characters in the
margin such as as a CR symbol or ellipsis to indicate newlines or
wrapping.  To preserve backward compatibility, the ASCII output will
remain unchanged.

I would have done this in the patch as applied, but was asked to hold
off doing that until the initial work was committed.

 Note that the above is close to a default setup, so that is what many
 people will see by default from now on.

I'll try to get the above proposed change done in the next week or so,
in time for the next CommitFest, if you agree with the general idea.


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?   http://gutenprint.sourceforge.net/
   `-GPG Public Key: 0x25BFB848   Please GPG sign your mail.


signature.asc
Description: Digital signature


[HACKERS] Application name patch - v2

2009-10-16 Thread Dave Page
I believe the attached patch is ready for review at the next
commitfest. It does the following:

- Adds a userset GUC called application_name.
- Allows application_name to be reported in log_line_prefix using %a.
- Includes application_name in CSV logs.
- Displays application_name in the output of pg_stat_activity and
pg_stat_get_activity().
- Adds a PQconnectdb connection string option called application_name
and corresponding envvar PGAPPLICATIONNAME to allow application_name
to be set during connection.
- Adds a PQconnectdb connection string option called
fallback_application_name to allow applications to specify a default
application_name value that can be overriden by PGAPPLICATIONNAME.
- Ensures that the application_name value is processed early in the
connection phase to ensure it can be used in connection log messages.
- Includes docs

I think that covers all the suggestions discussed over the last couple
of days, with the exception of the rejection of \n and similar
characters which I'm still not entirely convinced is worth the effort.
Any other opinions on that? Anything else that should be
added/changed?

-- 
Dave Page
EnterpriseDB UK:   http://www.enterprisedb.com


appname-v2.diff
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Trigger with WHEN clause (WIP)

2009-10-16 Thread Simon Riggs
On Fri, 2009-10-16 at 14:39 +0300, Peter Eisentraut wrote:
 On Fri, 2009-10-16 at 09:31 +0100, Simon Riggs wrote:
  * It will reduce size of in-memory pending trigger list (for large
  statements)
 
 But this won't be the outcome when it's implemented the way it is being
 proposed, which checks the where clause just before executing the
 trigger function.

Thanks for pointing that out.

I'm giving reasons why we'd want a WHEN clause. If the current
implementation doesn't do all that it could, then ISTM thats a reason to
reject patch for now, but not for all time.

Incidentally, re-accessing a data block at end of statement may have
caused to block to fall out of cache and then be re-accessed again. So
optimising that away can save on I/O as well.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Trigger with WHEN clause (WIP)

2009-10-16 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes:
 On Fri, 2009-10-16 at 09:31 +0100, Simon Riggs wrote:
 * It will reduce size of in-memory pending trigger list (for large
 statements)

 But this won't be the outcome when it's implemented the way it is being
 proposed, which checks the where clause just before executing the
 trigger function.

Hm, the feature could actually be worth something if it allows
conditions to be checked before an AFTER trigger event gets pushed
into the event list.  However, if it's not doing that ...

I note BTW that we have some ad-hoc logic already that arranges to
suppress queuing of AFTER events for FK triggers, if the FK column
value has not changed.  It might be interesting to look at whether
that hack could be unified with the user-accessible feature.  It's
essentially a WHEN OLD.x IS NOT DISTINCT FROM NEW.x test.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Trigger with WHEN clause (WIP)

2009-10-16 Thread Simon Riggs
On Fri, 2009-10-16 at 10:02 -0400, Tom Lane wrote:
 Peter Eisentraut pete...@gmx.net writes:
  On Fri, 2009-10-16 at 09:31 +0100, Simon Riggs wrote:
  * It will reduce size of in-memory pending trigger list (for large
  statements)
 
  But this won't be the outcome when it's implemented the way it is being
  proposed, which checks the where clause just before executing the
  trigger function.
 
 Hm, the feature could actually be worth something if it allows
 conditions to be checked before an AFTER trigger event gets pushed
 into the event list.  However, if it's not doing that ...
 
 I note BTW that we have some ad-hoc logic already that arranges to
 suppress queuing of AFTER events for FK triggers, if the FK column
 value has not changed.  It might be interesting to look at whether
 that hack could be unified with the user-accessible feature.  It's
 essentially a WHEN OLD.x IS NOT DISTINCT FROM NEW.x test.

If the function is idempotent then we can also optimise away multiple
calls by checking whether a similar call is already queued.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Trigger with WHEN clause (WIP)

2009-10-16 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes:
 If the function is idempotent then we can also optimise away multiple
 calls by checking whether a similar call is already queued.

But how would we know that?  It seems orthogonal to this patch,
anyway.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] operator exclusion constraints [was: generalized index constraints]

2009-10-16 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes:
 I tried to move the WHERE clause right before or after the
 index_parameters, but that resulted in shift/reduce conflicts.

You could avoid the conflicts in at least two ways:

* require parens around the WHERE expression

* stick the WHERE inside the EXCLUSION ( ... ) bit, so that its
expression is terminated by the outer right paren.

Not sure if either of these is less ugly than putting it at the end,
though :-(.  They both seem a bit surprising.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Trigger with WHEN clause (WIP)

2009-10-16 Thread Stephan Szabo
On Fri, 16 Oct 2009, Tom Lane wrote:

 I note BTW that we have some ad-hoc logic already that arranges to
 suppress queuing of AFTER events for FK triggers, if the FK column
 value has not changed.  It might be interesting to look at whether
 that hack could be unified with the user-accessible feature.  It's
 essentially a WHEN OLD.x IS NOT DISTINCT FROM NEW.x test.

Note that one of those cases (RI_TRIGGER_FK) is a bit special due to the
transaction id test. It might be worth seeing if a better solution is
possible to cover the case in the comment if the above becomes possible,
though.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Trigger with WHEN clause (WIP)

2009-10-16 Thread Tom Lane
Stephan Szabo ssz...@megazone.bigpanda.com writes:
 On Fri, 16 Oct 2009, Tom Lane wrote:
 I note BTW that we have some ad-hoc logic already that arranges to
 suppress queuing of AFTER events for FK triggers, if the FK column
 value has not changed.  It might be interesting to look at whether
 that hack could be unified with the user-accessible feature.  It's
 essentially a WHEN OLD.x IS NOT DISTINCT FROM NEW.x test.

 Note that one of those cases (RI_TRIGGER_FK) is a bit special due to the
 transaction id test. It might be worth seeing if a better solution is
 possible to cover the case in the comment if the above becomes possible,
 though.

The existing FK short-circuit code is presumably faster than generic
WHEN expression evaluation would be, so I wasn't really imagining that
we would take it out in favor of using the generic feature.  But
possibly we could clean things up to some extent by treating the FK case
as a variant of generic WHEN.  One particular thing I never much liked
about that hack is its dependence on specific function OIDs.  I could
see replacing that with some representation that says this trigger
has a special FK WHEN clause as opposed to providing an expression
tree.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Encoding issues in console and eventlog on win32

2009-10-16 Thread Bruce Momjian
Robert Haas wrote:
  Sure, the logfile will be filled with mixed encoding strings,
  that could happen in logfile and syslog on non-win32 platforms.
  I think UTF8 is better than UTF16 for logfile encoding because
  there are some text editors that do not support wide characters.
  At any rate, the logfile encoding feature will come from another patch,
  that might add log_encoding variable and work on any platforms.
 
 Magnus has promised me on a stack of instant messages that he will
 review this soon, but as he hasn't gotten to it yet, I am moving it to
 the next CommitFest.

I am with Magnus today and will make sure it gets done.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Rejecting weak passwords

2009-10-16 Thread Bruce Momjian
Tom Lane wrote:
 Robert Haas robertmh...@gmail.com writes:
  If we were using some kind of real public key system and someone
  suggested breaking it to add password complexity checking, I would
  understand the outrage here.  But I don't understand why everyone is
  so worked up about having an *optional* *flag* to force plaintext
  instead of MD5.  I might be wrong here, but can't a determined
  attacker brute-force an MD5 anyway?  The very fact that people are
  suggesting that password checking might be feasible even on a
  pre-MD5'd password by using a dictionary suggests that we're not
  getting a whole lot of real security here.  And even if not, dude,
  it's an *optional* *flag*.
 
 Yes, and it's an optional flag that could perfectly well be implemented
 in the plugin that I think we do have consensus to add a hook for.
 The argument is over why do we need to litter the core system with it.

So, are we agreed to provide a hook on the server side, but to use it
you have to configure your system with SSL and 'password'?  I can add
that to the TODO list.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Rejecting weak passwords

2009-10-16 Thread Bruce Momjian
Dave Page wrote:
 Too many of those caveats, and it's easy to see how we can be
 discounted early in the evaluation phase. It's not helped that often
 these lists will be drawn up by people used to working with the
 commercial DBMSs, so we probably wouldn't get extra points for having
 a dozen procedural languages, or other features that are largely
 unique to PostgreSQL, no matter how cool and useful they are.

Yep, this is illustrating something that is pretty basic to open source
--- that is open source often provides the tools for a solution, rather
than a complete solution.  I often think of open source as providing a
calculator with wires sticking out, rather than calculator buttons;  the
wires allow more flexibility, but they are harder to use.

Personally I think the calculator/wires approach is better from an
engineering perspective, but it can be a handicap in the user experience
and checkbox categories --- ease of use is perhaps not our strong point.
Much of our open source value is being different, in both cost,
reliability, and configurability.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Rejecting weak passwords

2009-10-16 Thread Tom Lane
Bruce Momjian br...@momjian.us writes:
 So, are we agreed to provide a hook on the server side, but to use it
 you have to configure your system with SSL and 'password'?  I can add
 that to the TODO list.

I think we're agreed to provide the hook.  What restrictions the hook
might enforce are not our concern.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Rejecting weak passwords

2009-10-16 Thread Mark Mielke

On 10/16/2009 11:28 AM, Bruce Momjian wrote:

Dave Page wrote:
   

Too many of those caveats, and it's easy to see how we can be
discounted early in the evaluation phase. It's not helped that often
these lists will be drawn up by people used to working with the
commercial DBMSs, so we probably wouldn't get extra points for having
a dozen procedural languages, or other features that are largely
unique to PostgreSQL, no matter how cool and useful they are.
 

Yep, this is illustrating something that is pretty basic to open source
--- that is open source often provides the tools for a solution, rather
than a complete solution.  I often think of open source as providing a
calculator with wires sticking out, rather than calculator buttons;  the
wires allow more flexibility, but they are harder to use.
   


Although often true - I think this is selling PostgreSQL a little short. 
It is a self-contained solution for what it does best, and for those 
that need more - there are better frameworks designed to be integrated 
that PostgreSQL is able to integrate with. PostgreSQL isn't a calculator 
with wires - if anything, I think PostgreSQL is an easy-to-use full 
functioned calculator whereas Oracle might be some advanced HP 
calculator that requires special training to learn how to use right... :-)



Personally I think the calculator/wires approach is better from an
engineering perspective, but it can be a handicap in the user experience
and checkbox categories --- ease of use is perhaps not our strong point.
Much of our open source value is being different, in both cost,
reliability, and configurability.


I found this true of a lot of tools. I still remember when the mutt 
developers argued against putting IMAP in their solution because they 
thought there might be a better IMAP component client out there. 
Eventually, such arguments are dropped, as the practical sense on the 
matter says that tight integration is a requirement.


I don't see how PostgreSQL has really failed in this regard. Maybe 
Oracle comes out-of-box with more features - but this doesn't make it 
necessarily a more complete solution - it just means it has more bells 
and whistles. A bicycle doesn't need a ticking card mounted through the 
spokes for it to be considered a complete solution. :-) Somebody might 
one day want that feature - but it's extra - it's not core.


Cheers,
mark

--
Mark Mielkem...@mielke.cc


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reworks for Access Control facilities (r2363)

2009-10-16 Thread Greg Stark
2009/10/16 KaiGai Kohei kai...@ak.jp.nec.com:
 . In addition, I already tried to put SE-PG hooks
 within pg_xxx_aclchecks() in this CF, but it was failed due to the
 differences in the security models.

I thought the last discussion ended with a pretty strong conclusion
that we didn't want differences in the security models.

The first step is to add hooks which don't change the security model
at all, just allow people to control the existing checks from their SE
configuration. Only as a second step we would look into making
incremental changes to the postgres security model to add support for
privileges SE users might expect to find, eventually possibly
including per-row permissions.

-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Deprecation

2009-10-16 Thread David Fetter
Folks,

We have some really silly legacy stuff in PostgreSQL, the silliest of
which, as far as I've found, is the add_missing_from GUC.

Since it's been deprecated, as in off by default, since 8.1, I'd like
to suggest that we remove it from both docs and code and throw an
error if someone tries to set it, just as if they'd set
add_flying_spaghetti_monster.

What say?

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Rejecting weak passwords

2009-10-16 Thread Greg Stark
On Fri, Oct 16, 2009 at 4:28 PM, Bruce Momjian br...@momjian.us wrote:
 Personally I think the calculator/wires approach is better from an
 engineering perspective, but it can be a handicap in the user experience
 and checkbox categories --- ease of use is perhaps not our strong point.
 Much of our open source value is being different, in both cost,
 reliability, and configurability.

I don't really see this as an either-or proposition. The calculator
with buttons still has wires inside, someone had to hack on the wiring
to get the functionality working. Someone else with an inclination
toward marketing put shiny buttons on it. I'm pretty happy to be in
the first group and since Postgres has a BSD license people in the
second group are free to work on shiny buttons.

The discussion at hand is an unusual situation where the priorities
conflict because supporting a particular shiny button requires ripping
out some useful wiring. In the long run I think having the wiring
right is more valuable than some shiny buttons today.

-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Deprecation

2009-10-16 Thread Tom Lane
David Fetter da...@fetter.org writes:
 We have some really silly legacy stuff in PostgreSQL, the silliest of
 which, as far as I've found, is the add_missing_from GUC.

Considering that we just had a discussion about a significant
application that's still using it, I'm not sure what's your hurry.
Is your intent specifically to break OpenACS in hopes of getting
their attention?  If so, you need to be a bit more up-front about that.

(I would actually not mind getting rid of it, because that would greatly
simplify a problem I'm wrestling with right now, namely how to put hooks
into the parser for resolution of plpgsql variables.  But we need to be
honest about what it's going to do to users.)

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reworks for Access Control facilities (r2363)

2009-10-16 Thread Tom Lane
Greg Stark gsst...@mit.edu writes:
 The first step is to add hooks which don't change the security model
 at all, just allow people to control the existing checks from their SE
 configuration.

This is in fact what the presented patch is meant to do.  The issue is
about whether the hook placement is sane/useful/extensible.  The main
problem I've got with the design is that it doesn't appear to work for
privilege checks made by add-on modules.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Rejecting weak passwords

2009-10-16 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian br...@momjian.us writes:
  So, are we agreed to provide a hook on the server side, but to use it
  you have to configure your system with SSL and 'password'?  I can add
  that to the TODO list.
 
 I think we're agreed to provide the hook.  What restrictions the hook
 might enforce are not our concern.

Great, added to TODO:

Allow server-side enforcement of password policies

Password checks might include password complexity or non-reuse of
passwords. This facility will require the client to send the password to
the server in plain-text, so SSL and 'password' authentication is
necessary to use this features.

* 
http://archives.postgresql.org/pgsql-hackers/2009-09/msg01766.php
* 
http://archives.postgresql.org/pgsql-hackers/2009-10/msg00025.php 

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Rejecting weak passwords

2009-10-16 Thread Bruce Momjian
Mark Mielke wrote:
  Personally I think the calculator/wires approach is better from an
  engineering perspective, but it can be a handicap in the user experience
  and checkbox categories --- ease of use is perhaps not our strong point.
  Much of our open source value is being different, in both cost,
  reliability, and configurability.
 
 I found this true of a lot of tools. I still remember when the mutt 
 developers argued against putting IMAP in their solution because they 
 thought there might be a better IMAP component client out there. 
 Eventually, such arguments are dropped, as the practical sense on the 
 matter says that tight integration is a requirement.
 
 I don't see how PostgreSQL has really failed in this regard. Maybe 
 Oracle comes out-of-box with more features - but this doesn't make it 
 necessarily a more complete solution - it just means it has more bells 
 and whistles. A bicycle doesn't need a ticking card mounted through the 
 spokes for it to be considered a complete solution. :-) Somebody might 
 one day want that feature - but it's extra - it's not core.

Agreed.  Many commercial database solutions start to look like
Frankenstein with all the bolted-on components.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reworks for Access Control facilities (r2363)

2009-10-16 Thread Robert Haas
On Fri, Oct 16, 2009 at 12:45 PM, Greg Stark gsst...@mit.edu wrote:
 2009/10/16 KaiGai Kohei kai...@ak.jp.nec.com:
 . In addition, I already tried to put SE-PG hooks
 within pg_xxx_aclchecks() in this CF, but it was failed due to the
 differences in the security models.

 I thought the last discussion ended with a pretty strong conclusion
 that we didn't want differences in the security models.

 The first step is to add hooks which don't change the security model
 at all, just allow people to control the existing checks from their SE
 configuration. Only as a second step we would look into making
 incremental changes to the postgres security model to add support for
 privileges SE users might expect to find, eventually possibly
 including per-row permissions.

I think we sort of came to the conclusion that even a basic
implementation of SE-PostgreSQL might have some requirements that
didn't quite square with the existing PostgreSQL security model.  The
charter of this patch AIUI was to refactor things so that they were
square up, but I the patch is substantially more complex and invasive
than what I thought would be necessary and it's not clear that it
solves the problem.  Rather than refactoring the existing checks to
provide a cleaner abstraction layer, it seems to provide a layer that,
if it's anything, is just a place-holder for an SE-PostgreSQL
implementation, and there's no guarantee that it's adequate even for
that, much less for anything else we might want to do.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Deprecation

2009-10-16 Thread David Fetter
On Fri, Oct 16, 2009 at 01:23:16PM -0400, Tom Lane wrote:
 David Fetter da...@fetter.org writes:
  We have some really silly legacy stuff in PostgreSQL, the silliest
  of which, as far as I've found, is the add_missing_from GUC.
 
 Considering that we just had a discussion about a significant
 application that's still using it, I'm not sure what's your hurry.
 Is your intent specifically to break OpenACS in hopes of getting
 their attention?  If so, you need to be a bit more up-front about
 that.
 
 (I would actually not mind getting rid of it, because that would
 greatly simplify a problem I'm wrestling with right now, namely how
 to put hooks into the parser for resolution of plpgsql variables.
 But we need to be honest about what it's going to do to users.)

Breaking legacy applications is a side effect, one we should probably
publish far and wide, of the code cleanup.

My hidden agenda, such as it is, is to make sure people don't get
the misapprehension that they can just fire and forget with
PostgreSQL, or any other software.  Interfaces change; ugly kludges
get removed, and they need to build their processes around this fact
rather than around wishful thinking.

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Deprecation

2009-10-16 Thread Greg Stark
On Fri, Oct 16, 2009 at 10:23 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 (I would actually not mind getting rid of it, because that would greatly
 simplify a problem I'm wrestling with right now, namely how to put hooks
 into the parser for resolution of plpgsql variables.  But we need to be
 honest about what it's going to do to users.)

I was actually expecting you to come down on the side of keep it
until it gets in the way. But if it's getting in the way already then
it seems reasonable to start talking about getting rid of it.

It only affects OpenACS if they want to upgrade to 8.5 which will
presumably mean other application changes as well. Though probaby none
requiring as much major code changes as this.


-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting the red out (of the buildfarm)

2009-10-16 Thread Peter Eisentraut
On Thu, 2009-10-15 at 00:43 +0300, Peter Eisentraut wrote:
 On Sun, 2009-10-04 at 10:48 -0400, Tom Lane wrote:
  Peter Eisentraut pete...@gmx.net writes:
   I understand the annoyance, but I think we do need to have an organized
   way to do testing of non-ASCII data and in particular UTF8 data, because
   there are an increasing number of special code paths for those.
  
  Well, if you want to keep the test, we should put in the variant with
  \200, because it is now clear that that is in fact the right answer
  in a nontrivial number of environments (arguably *more* cases than
  in which \u0080 is correct).
 
 I put in a new variant file.  Let's see if it works.

[http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/pl/plpython/expected/plpython_unicode_0.out]

Actually, what I committed was really the output I got.  Now with your
commit my tests started failing again.

The difference turns out to be caused by glibc.  When you print an
invalid UTF-8 byte sequence using %.*s when LC_CTYPE is a UTF-8 locale
(e.g., en_US.utf8), it prints nothing.  Presumably, it gets confused
counting the characters for aligning the field width.

Test program:

#include locale.h
#include stdio.h

int
main()
{
setlocale(LC_ALL, );
printf(%.*s, 1, \200);
return 0;
}

This prints nothing (check with od) when LC_CTYPE is en_US.utf8.

I think this can be filed under trouble caused by mismatching LC_CTYPE
and client encoding and doesn't need further fixing, but it's good to
keep in mind.

Let's see what the Solaris builds say now.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting the red out (of the buildfarm)

2009-10-16 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes:
 Actually, what I committed was really the output I got.  Now with your
 commit my tests started failing again.

Huh --- what I committed is what I got on a Fedora 11 machine.  Maybe
we need both variants?

 Let's see what the Solaris builds say now.

We'll know for sure in a couple hours, but it looks to me like their
results are matching mine.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting the red out (of the buildfarm)

2009-10-16 Thread Peter Eisentraut
On Fri, 2009-10-16 at 15:14 -0400, Tom Lane wrote:
 Peter Eisentraut pete...@gmx.net writes:
  Actually, what I committed was really the output I got.  Now with your
  commit my tests started failing again.
 
 Huh --- what I committed is what I got on a Fedora 11 machine.  Maybe
 we need both variants?

It depends on what LC_CTYPE is set to on the client side.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Deprecation

2009-10-16 Thread Tom Lane
Greg Stark gsst...@mit.edu writes:
 On Fri, Oct 16, 2009 at 10:23 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 (I would actually not mind getting rid of it, because that would greatly
 simplify a problem I'm wrestling with right now, namely how to put hooks
 into the parser for resolution of plpgsql variables.  But we need to be
 honest about what it's going to do to users.)

 I was actually expecting you to come down on the side of keep it
 until it gets in the way. But if it's getting in the way already then
 it seems reasonable to start talking about getting rid of it.

Yeah.  The problem is that I'd like to have plpgsql install a hook that
runs at the end of transformColumnRef and has an opportunity to provide
a reference to a plpgsql var if nothing was found in the normal SQL
lookups.  However, if the columnref looks like x.y where x happens to
match some table in the database (and not in the query) that doesn't
have a column y, the implicit-RTE code will have already modified the
querytree before finding out that column y doesn't exist.  While that
can probably be unwound somehow, it's going to be a major PITA, and
there would be no way to hide the fact that a lock got taken on x before
we changed our minds about including it in the query.  Considering that
this is all legacy behavior anyway it would be nice to not have to fix
it.

So, while I do think it's something we should leave alone until it gets
in the way, this is a sufficiently large value of in the way that I'm
willing to talk about removing add_missing_from.  I'm just concerned
about the impact of that, considering that an app that still depends on
it came up as recently as yesterday.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting the red out (of the buildfarm)

2009-10-16 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes:
 On Fri, 2009-10-16 at 15:14 -0400, Tom Lane wrote:
 Huh --- what I committed is what I got on a Fedora 11 machine.  Maybe
 we need both variants?

 It depends on what LC_CTYPE is set to on the client side.

I was testing the same case as the problematic Solaris tests, ie,
LANG=cs_CZ.iso88592
[ thinks ... ] although I don't remember if psql was seeing that value
too.  I might've just initdb'd with that and then run psql in my usual
C locale.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Deprecation

2009-10-16 Thread Guillaume Smet
On Fri, Oct 16, 2009 at 9:26 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 So, while I do think it's something we should leave alone until it gets
 in the way, this is a sufficiently large value of in the way that I'm
 willing to talk about removing add_missing_from.  I'm just concerned
 about the impact of that, considering that an app that still depends on
 it came up as recently as yesterday.

8.4 is going to be supported for several years so they can document
it's the last version supported by the current version of OpenACS.

If they want the benefits from 8.5 and higher (for themselves or their
users), they'll do the work needed to remove the various bad practices
from their code.

Note that the removal of the implicit casts to text was far more
problematic for a lot of apps than add_missing_from=off - which is the
de facto standard for several years now.

And as you can see in
http://openacs.org/forums/message-view?message_id=2471518 , they did
the ground work to support the tsearch2 changes from contrib to core.
A friendly email to them explaining why it needs to be fixed in their
code should be sufficient (and it might be already fixed, btw).

-- 
Guillaume

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [GENERAL] contrib/plantuner - enable PostgreSQL planner hints

2009-10-16 Thread decibel

On Oct 6, 2009, at 2:57 PM, Oleg Bartunov wrote:
this is an announcement of our new contribution module for  
PostgreSQL - Plantuner - enable planner hints

(http://www.sai.msu.su/~megera/wiki/plantuner).

=# set enable_seqscan=off;
=# set plantuner.forbid_index='id_idx2';



Out of curiosity, did you look at doing hints as comments in a query?  
I'm guessing you couldn't actually do that in just a contrib module,  
but it's how Oracle handles hints, and it seems to be *much* more  
convenient, because a hint only applies for a specific query. I think  
it's extremely unlikely you would intentionally want the same hint to  
apply to a bunch of queries, and extremely likely that you could  
accidentally forget to re-enable something.


That said, thanks for contributing this!
--
Decibel!, aka Jim C. Nasby, Database Architect  deci...@decibel.org
Give your computer some brain candy! www.distributed.net Team #1828



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [GENERAL] contrib/plantuner - enable PostgreSQL planner hints

2009-10-16 Thread Christophe Pettus


On Oct 16, 2009, at 10:04 AM, decibel wrote:

Out of curiosity, did you look at doing hints as comments in a query?


I don't think that a contrib module could change the grammar.

--
-- Christophe Pettus
   x...@thebuild.com


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [GENERAL] contrib/plantuner - enable PostgreSQL planner hints

2009-10-16 Thread Jeff Davis
On Fri, 2009-10-16 at 12:04 -0500, decibel wrote:
 I'm guessing you couldn't actually do that in just a contrib module,  
 but it's how Oracle handles hints, and it seems to be *much* more  
 convenient, because a hint only applies for a specific query.

If that's the only reason, that seems easy enough to solve by using SET
right before the query. SET LOCAL might be convenient if you want to
forget the setting after the query. Connection pool software will do a
RESET ALL anyway.

There are reasons that it might be convenient to use hints inside the
query itself -- for instance, if you want something to apply only to a
subquery. I'm still hoping that someone will come up with a more elegant
solution to solve that problem though.

Regards,
Jeff Davis


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Encoding issues in console and eventlog on win32

2009-10-16 Thread Magnus Hagander
2009/10/13 Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp:

 Magnus Hagander mag...@hagander.net wrote:

 One other question - you note that WriteConsoleW() could fail if
 stderr is redirected. Are you saying that it will always fail when
 stderr is redirected, or only sometimes? If ony sometimes, do you know
 under which conditions it happens?

 It will always fail if redirected. We can test the conditions using:
    pg_ctl start  result.log
 So, the comment should be:
    /* WriteConsoleW always fails if stderr is redirected. */

Ok, fair enough. We already have a variable for that though - it's
called redirection_done. I think it does what's necessary - I have
used that one in my version of the patch. Please verify that it works
in your environment.


 I cleaned up the patch per comments. I hope this will be the final one ;-).

  * Use in_error_recursion_trouble() instead of own implementation.
  * Use def_enc2name() macro to avoid adding the codepage field
    on non-win32 platforms.

Per previous email, I had done this in my version of the patch, so it
looks slightly different than yours, but it has the same
functionality.


  * Fix a bug of calculation of result length.

Where exactly is this one? I can't find it compared to my code, but
that could just be out-of-timezone-brain speaking :-)

  * Fix a memory leak on error handling path in pgwin32_toUTF16().

Missed that one, thanks!


 If it's always, I assume this just means that the logfile will be in
 the database encoding and not in UTF16? Is this what we want, or would
 we like the logfile to also be in UTF16? If we can convert it to
 UTF16, that would fix the case when you have different databases in
 different encodings, wouldn't it? (Even if your editor, unlike the
 console subsystem, can view the individual encoding you need, I bet it
 can't deal with multiple encodings in the same file)

 Sure, the logfile will be filled with mixed encoding strings,
 that could happen in logfile and syslog on non-win32 platforms.
 I think UTF8 is better than UTF16 for logfile encoding because
 there are some text editors that do not support wide characters.

Don't most text editors on Windows do UTF16? In particular, I'd expect
more of them to do UTF16 than UTF8, but I could be wrong?

 At any rate, the logfile encoding feature will come from another patch,
 that might add log_encoding variable and work on any platforms.

Ok, good. Particularly the other platform is the winning argument.

So, what I believe is the latest version of the patch applied. Please
point out if I made a mistake in my changes against yours.

Sorry about the delay :(


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Deprecation

2009-10-16 Thread Greg Stark
On Fri, Oct 16, 2009 at 12:26 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 However, if the columnref looks like x.y where x happens to
 match some table in the database (and not in the query) that doesn't
 have a column y, the implicit-RTE code will have already modified the
 querytree before finding out that column y doesn't exist.

Hm, so if you do nothing then really the only thing that doesn't work
is if you have add_missing_from then plpgsql record variables wouldn't
work when you tried to reference their columns?

Perhaps we could just document that add_missing_from breaks that case?
The worst thing about that is that someone could switch that option on
globally on their server and break code that's part of some
third-party module.

-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [GENERAL] contrib/plantuner - enable PostgreSQL planner hints

2009-10-16 Thread Greg Stark
On Fri, Oct 16, 2009 at 10:04 AM, decibel deci...@decibel.org wrote:
 Out of curiosity, did you look at doing hints as comments in a query? I'm
 guessing you couldn't actually do that in just a contrib module, but it's
 how Oracle handles hints, and it seems to be *much* more convenient, because
 a hint only applies for a specific query

Fwiw Oracle is moving away from this now. At OpenWorld the optimizer
folks were pleading with folks to get rid of all their hard-coded
hints by preparing plans for hinted queries and loading those as the
approved plans. In 11g there's a system which ensures the database
will not run any plan that isn't approved.

In fact it looks an *awful* lot like the system I sketched out 6 years
ago in: http://article.gmane.org/gmane.comp.db.postgresql.devel.general/11385

-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reworks for Access Control facilities (r2363)

2009-10-16 Thread KaiGai Kohei
Heikki Linnakangas wrote:
 KaiGai Kohei wrote:
 The purpose of this patch is to provide function entrypoints for the
 upcoming SE-PostgreSQL feature, because I got a few comments that we
 hesitate to put sepgsql_xxx() hooks on the main routines directly in
 the first commit fest. In addition, I already tried to put SE-PG hooks
 within pg_xxx_aclchecks() in this CF, but it was failed due to the
 differences in the security models.
 
 Can you elaborate that? It might well be that you need to adapt the
 SE-PostgreSQL security model to the one that's there already. Putting
 SE-PG hooks into existing pg_xxx_aclcheck functions is the only
 low-impact way I can see to implement SE-PostgreSQL.

We can show several examples that pg_xxx_aclcheck() routines are not
suitable to implement SELinux's security model.
Please note that it is not a defect of the default PG's security model
needless to say. It is just a different in standpoints.

1) creation of a database object

In SELinux model, when a user tries to create a new object (not limited
to database object, like a file or socket), a default security context
is assigned on the new object, then SELinux checks whether the user has
privileges to create a new object labeled with the security context, or not.

When we create a new table, the default PG model checks ACL_CREATE privilege
on the namespace which is supposed to own the new table. DefineRelation()
invokes pg_namespace_aclcheck() with OID of the namespace, but we cannot
see any properties of the new table from inside of pg_namespace_aclcheck().
It checks permissions on the couple of a user and a namespace.

On the other hand, SE-PG model follows the above principle. When we create
a new table, SE-PG compute a default security context to be assigned on,
then it checks the security policy whether the user is allowed to create
a new table labeled with the context, or not.
It checks permissions on the couple of a user and a new table itself.

The caller does not provide enough information to the pg_xxx_aclcheck(),
so we decided to create an abstraction layer which can provide enough
informations to both of security models. Then, the ac_xxx_*() routines
were implemented.


2) AND-condition for all the privileges

When a certain action requires multiple permissions at one time,
the principle of SELinux is that all the permissions have to be checked.
If one of them is not allowed, it disallows the required action.
In other word, all the conditions are chained by AND.

This principle enables us to analyze the data flows between users and
resources with the security policy, without implementation details.
If a certain permission (e.g db_table:{select}) can override any other
permission (e.g db_column:{select}), it also implicitly means a possibility
of infotmation leaks/manipulations, even if the security policy said this
user cannot read a data from the column.

On the other hand, the default PG model allows to bypass checks on
certain objects. For example, column-level privileges are only checked
when a user does not have enough permissions on the target table.
If SELECT a,b FROM t is given, pg_attribute_aclcheck() may not invoked
when user has needed privileges on the table t.


3) superuser is not an exception of access control.

It is the similar issue to the 2).
The following code is a part of AlterFunctionOwner_internal().


/* Superusers can always do it */
if (!superuser())
{
/* Otherwise, must be owner of the existing object */
if (!pg_proc_ownercheck(procOid, GetUserId()))
aclcheck_error(ACLCHECK_NOT_OWNER, ACL_KIND_PROC,
   NameStr(procForm-proname));

/* Must be able to become new owner */
check_is_member_of_role(GetUserId(), newOwnerId);

/* New owner must have CREATE privilege on namespace */
aclresult = pg_namespace_aclcheck(procForm-pronamespace,
  newOwnerId,
  ACL_CREATE);
if (aclresult != ACLCHECK_OK)
aclcheck_error(aclresult, ACL_KIND_NAMESPACE,
   get_namespace_name(procForm-pronamespace));
}


From perspective of the default PG model, this code perfectly correct.
Both of pg_proc_ownercheck() and pg_namespace_aclcheck() always returns
ACLCHECK_OK, so these invocations are bypassable.

However, if SE-PG's hooks are deployed on pg_xxx_aclcheck() routines,
it means that we cannot check correct MAC permissions when a client is
allowed to apply superuser privilege.
Please remind that SELinux requires AND-condition for all the privileges
required to a certain action. When a root user tries to read a certain
file without DAC permisions, it requires both of capability:{dac_override}
and file:{read} permissions in operating system.


These are a part of reasons why we had to design such a large patch.
If we stick around the common entrypoints of access 

Re: [HACKERS] Reworks for Access Control facilities (r2363)

2009-10-16 Thread KaiGai Kohei

Greg Stark wrote:

2009/10/16 KaiGai Kohei kai...@ak.jp.nec.com:

. In addition, I already tried to put SE-PG hooks
within pg_xxx_aclchecks() in this CF, but it was failed due to the
differences in the security models.


I thought the last discussion ended with a pretty strong conclusion
that we didn't want differences in the security models.


It is not a fact. Because the SE-PG patch is a bit large to review,
I got a suggestion to implement a part of permissions checks which
can be invoked from the pg_xxx_aclcheck() without any breaks for
SELinux's security model, at the first step.
In other word, I tried to implement only union part of the security
models.


The first step is to add hooks which don't change the security model
at all, just allow people to control the existing checks from their SE
configuration. Only as a second step we would look into making
incremental changes to the postgres security model to add support for
privileges SE users might expect to find, eventually possibly
including per-row permissions.


I already did it on the first CF...
However, most of permission checks had gone at the first step.
It was commented it is same as checks nothing.

Thanks,
--
KaiGai Kohei kai...@kaigai.gr.jp

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Deprecation

2009-10-16 Thread Marc G. Fournier

On Fri, 16 Oct 2009, Greg Stark wrote:

It only affects OpenACS if they want to upgrade to 8.5 which will 
presumably mean other application changes as well. Though probaby none 
requiring as much major code changes as this.


Being one that hosts alot of OACS sites, and has a fair experience with 
it, I agree ... as long as this isn't something that is going to be 
backpatched, there should be no reason why this can't go in for 8.5 ... 
the OACS guys will either fix the code, or stick to 8.4 ...



Marc G. FournierHub.Org Hosting Solutions S.A.
scra...@hub.org http://www.hub.org

Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Deprecation

2009-10-16 Thread Marc G. Fournier

On Fri, 16 Oct 2009, Tom Lane wrote:

So, while I do think it's something we should leave alone until it gets 
in the way, this is a sufficiently large value of in the way that I'm 
willing to talk about removing add_missing_from.  I'm just concerned 
about the impact of that, considering that an app that still depends on 
it came up as recently as yesterday.


As this should / would only affect 8.5+, just means that the app in 
question has to be stuck at 8.4 or fix the code.  Since, as David points 
out, this 'hack' has been in there since 8.1, I think we've given the app 
in question more then sufficient time to fix their code already, no?  3 
years, or so?



Marc G. FournierHub.Org Hosting Solutions S.A.
scra...@hub.org http://www.hub.org

Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers