Re: [HACKERS] Client application name
On Tue, Oct 13, 2009 at 10:25 PM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: On Tue, Oct 13, 2009 at 9:13 PM, Jaime Casanova jcasa...@systemguards.com.ec wrote: besides, as Robert mention, because of pooler connections using a GUC is more appropiate... I'd like both options to be available to the programmer. We have several things already that can be fed either from an environment variable or an option in the connection string. Is there any compelling reason why those two mechanisms aren't adequate for this? Err, yes - see above. And didn't you also say it was essential to be able to change it after the initial connection (for which the GUC seems like the obvious solution)? -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Buffer usage in EXPLAIN and pg_stat_statements (review)
Tom Lane t...@sss.pgh.pa.us wrote: 2. I do not understand the stuff with propagating counts into the top instrumentation node. That seems like it's going to double-count those counts. In any case it is 100% inconsistent to propagate only buffer counts that way and not any other resource usage. I think you should drop the TopInstrument variable and the logic that propagates counts up. It is required by contrib/pg_stat_statements. EXPLAIN wants per-node accumulation, but pg_stat_statements wants the total number. Is it enough to add a PG_TRY block to standard_ExecutorRun() to cleanup TopInstrument on error? I'm working on your other comments, but I cannot remove TopInstrument for pg_state_statements. I considerd other approaches, but all of them require node-dependent routines; for example, adding a function to walk through a plan tree and accumulate instrumentations in it at pg_stat_statements. But it is hard to be maintained on executor nodes changes. Are there any better idea? Regards, --- ITAGAKI Takahiro NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Hot standby, xlog_xact_assignment and unreported subxids.
In the hot standby patch, we have this comment in procarray.c: It is * important that the XLOG_XACT_ASSIGNMENT record contain *all* subxids * not just those so far unreported because the sole purpose is to ensure * we can remove the xids from KnownAssignedXids. As the patch stands, that's not what we do. We only include new subxids in the XLOG_XACT_ASSIGNMENT record that we haven't included in any previous ones. I think that comment is just obsolete and should be removed, but am I missing something? -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby, xlog_xact_assignment and unreported subxids.
On Wed, 2009-10-14 at 14:43 +0300, Heikki Linnakangas wrote: In the hot standby patch, we have this comment in procarray.c: It is * important that the XLOG_XACT_ASSIGNMENT record contain *all* subxids * not just those so far unreported because the sole purpose is to ensure * we can remove the xids from KnownAssignedXids. As the patch stands, that's not what we do. We only include new subxids in the XLOG_XACT_ASSIGNMENT record that we haven't included in any previous ones. That's the correct handling, in my view. I think that comment is just obsolete and should be removed, but am I missing something? I agree it looks wrong. (Did I write that??) -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Buffer usage in EXPLAIN and pg_stat_statements (review)
Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp writes: Tom Lane t...@sss.pgh.pa.us wrote: 2. I do not understand the stuff with propagating counts into the top instrumentation node. It is required by contrib/pg_stat_statements. EXPLAIN wants per-node accumulation, but pg_stat_statements wants the total number. Well, you need to find another way or risk getting the patch rejected altogether. Those global variables are the weakest part of the whole design, and I'm not going to commit a patch that destabilizes the entire system for the sake of a debatable requirement of a contrib module. If you went with the alternative definition I suggested (don't reset the static counters, so that every node includes its children's counts) then the behavior you want would fall out automatically. Or, at the price of running both resettable and non-resettable static counters, you could have pg_stat_statements obtain totals that are independent of any particular instrumentation node. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Mon, Sep 28, 2009 at 7:37 PM, Tom Lane t...@sss.pgh.pa.us wrote: IOW, having plaintext password in CREATE/ALTER time which can then checked for weaknesses is better that MD5 password, which actually does not increase security. This is not acceptable and will not happen. The case that ENCRYPTED protects against is database superusers finding out other users' original passwords, which is a security issue to the extent that those users have used the same/similar passwords for other systems. We're not going to give up protection for that in order to provide an option to do weak-password checking in a place that simply isn't the best place to do it anyway. This is an area in which we often get beaten up in in technical evaluations against other DBMSs. Password complexity checks are pretty much standard features in other products and it hurts our adoption not to be able to offer them, especially in large shops where the first phase of the eval may be a simple box-checking exercise. I would suggest that in addition to the proposed plugin, we add an suset GUC (defaulting to OFF) which rejects any use of WITH ENCRYPTED PASSWORD to ensure that the password complexity can be checked when roles are created or modified. In the default case, the current behaviour would not change. With the GUC turned on, passwords could be forced through the validation module. To allow dumps to remain secure, the GUC can be turned off by the DBA prior to loading, or in the dump itself. Logging of CREATE/ALTER users statements in this mode could be special-cased to prevent passwords going to the logs/stats (not sure what overhead that might incur though). In addition, the docs for the GUC would obviously point out that it should only be used in conjunction with SSL connections. This would allow us to remain secure-by-default, and yet offer an important option for many potential users. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] alpha 2 release notes
If someone wants to start working on the release notes for alpha 2, now would be a good time. In general, do we want the alpha 2 release notes as a separate section above alpha 1, or merged with the alpha 1 notes thereby giving a cumulative view of what happened since 8.4? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Client application name
Dave Page dp...@pgadmin.org writes: On Tue, Oct 13, 2009 at 10:25 PM, Tom Lane t...@sss.pgh.pa.us wrote: We have several things already that can be fed either from an environment variable or an option in the connection string. Is there any compelling reason why those two mechanisms aren't adequate for this? Err, yes - see above. And didn't you also say it was essential to be able to change it after the initial connection (for which the GUC seems like the obvious solution)? Sure. I'm envisioning that what the env variable or connection option actually does is cause libpq to include a SET command for a GUC variable in the initial connection request packet. Compare, say, PGCLIENTENCODING - client_encoding. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Client application name
On Wed, Oct 14, 2009 at 3:42 PM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: On Tue, Oct 13, 2009 at 10:25 PM, Tom Lane t...@sss.pgh.pa.us wrote: We have several things already that can be fed either from an environment variable or an option in the connection string. Is there any compelling reason why those two mechanisms aren't adequate for this? Err, yes - see above. And didn't you also say it was essential to be able to change it after the initial connection (for which the GUC seems like the obvious solution)? Sure. I'm envisioning that what the env variable or connection option actually does is cause libpq to include a SET command for a GUC variable in the initial connection request packet. Compare, say, PGCLIENTENCODING - client_encoding. Right - I think we're just misunderstanding each other whilst we're saying the same thing. I want the envvar/connection string option as ways to setup the value at connection time, and the use of SET available for later changes. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Unicode UTF-8 table formatting for psql text output
On Tue, Oct 13, 2009 at 05:08:20PM -0400, Tom Lane wrote: Roger Leigh rle...@codelibre.net writes: The attached updated patch renames all user-visible uses of utf8 to unicode. It also updates the documentation regarding locale to psql client character set encoding so the docs now match the code exactly. Applied with light editorialization. The main non-cosmetic change I made was to postpone selection of default line_style until runtime (see get_line_style() in the committed patch). The original coding required knowledge of the line_style default rule not only in three different places in psql, but in every other place using print.c, such as createlang/droplang -l (which dumped core with the patch as submitted). I changed it so that leaving line_style NULL implies the default encoding-driven behavior, so that we don't need to touch any of the callers. Thanks. I agree with your improvement in get_line_style(), it's cleaner this way; I didn't realise print.c was used by the other programs, sorry about that. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
Re: [HACKERS] Rejecting weak passwords
Dave Page dp...@pgadmin.org writes: I would suggest that in addition to the proposed plugin, we add an suset GUC (defaulting to OFF) which rejects any use of WITH ENCRYPTED PASSWORD to ensure that the password complexity can be checked when roles are created or modified. That's going to stop us from being beat up? A GUC that forcibly *weakens* security? I can't see it. If you're really intent on making that happen, you can have your password checker plugin reject crypted passwords; we don't need such a questionable rule in core. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Tom Lane wrote: Dave Page dp...@pgadmin.org writes: I would suggest that in addition to the proposed plugin, we add an suset GUC (defaulting to OFF) which rejects any use of WITH ENCRYPTED PASSWORD to ensure that the password complexity can be checked when roles are created or modified. That's going to stop us from being beat up? A GUC that forcibly *weakens* security? I can't see it. If you're really intent on making that happen, you can have your password checker plugin reject crypted passwords; we don't need such a questionable rule in core. And you could have the plugin do that depending on a custom GUC. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Wed, Oct 14, 2009 at 4:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: I would suggest that in addition to the proposed plugin, we add an suset GUC (defaulting to OFF) which rejects any use of WITH ENCRYPTED PASSWORD to ensure that the password complexity can be checked when roles are created or modified. That's going to stop us from being beat up? A GUC that forcibly *weakens* security? I can't see it. If the communications channel uses SSL, and passwords are prevented from hitting the logs then (assuming there are no other weaknesses I haven't thought of), then the net effect would surely be tighter overall security? In a very security-conscious shop, the DBA won't have access to the underlying system at all, so debugging tools etc would be out of the question. In most shops, he will have access and can already just set the auth method to 'password' and then break out the debugger (or even replace the executables), so I can't see that this option would open up any obvious new attack vectors. Users are almost always the biggest weak-point in any security system, so should naturally be the first hole we look at plugging, before the ones that are much harder to exploit effectively - especially when those are only open to exploit by people who already have superuser privileges! If you're really intent on making that happen, you can have your password checker plugin reject crypted passwords; we don't need such a questionable rule in core. Client software would need to have a standard way to know when to use ENCRYPTED PASSWORD or not. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Dave Page dp...@pgadmin.org writes: On Wed, Oct 14, 2009 at 4:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: If you're really intent on making that happen, you can have your password checker plugin reject crypted passwords; we don't need such a questionable rule in core. Client software would need to have a standard way to know when to use ENCRYPTED PASSWORD or not. Oh, so you want us to propagate extra support for this blatant security reduction all over the system too? No thank you. This whole line of discussion just proves the point that was made originally: it would be a lot better to do whatever checking you want done on the client side, rather than risk transmitting unencrypted passwords. If you are going to imagine that client-side software knows about such a GUC, you might as well imagine that they have cracklib built in. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On 10/14/09, Dave Page dp...@pgadmin.org wrote: On Mon, Sep 28, 2009 at 7:37 PM, Tom Lane t...@sss.pgh.pa.us wrote: IOW, having plaintext password in CREATE/ALTER time which can then checked for weaknesses is better that MD5 password, which actually does not increase security. This is not acceptable and will not happen. The case that ENCRYPTED protects against is database superusers finding out other users' original passwords, which is a security issue to the extent that those users have used the same/similar passwords for other systems. We're not going to give up protection for that in order to provide an option to do weak-password checking in a place that simply isn't the best place to do it anyway. This is an area in which we often get beaten up in in technical evaluations against other DBMSs. Password complexity checks are pretty much standard features in other products and it hurts our adoption not to be able to offer them, especially in large shops where the first phase of the eval may be a simple box-checking exercise. I would suggest that in addition to the proposed plugin, we add an suset GUC (defaulting to OFF) which rejects any use of WITH ENCRYPTED PASSWORD to ensure that the password complexity can be checked when roles are created or modified. Checking 1M common weak combinations against the md5 hashed password will take 1s. You cannot count the commas in password, but thats it. The usual problems: 'username', '1234' can be still detected. This would allow us to remain secure-by-default, and yet offer an important option for many potential users. -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Wed, Oct 14, 2009 at 4:30 PM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: On Wed, Oct 14, 2009 at 4:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: If you're really intent on making that happen, you can have your password checker plugin reject crypted passwords; we don't need such a questionable rule in core. Client software would need to have a standard way to know when to use ENCRYPTED PASSWORD or not. Oh, so you want us to propagate extra support for this blatant security reduction all over the system too? No thank you. You've twice asserted it's a reduction without providing any arguments to back that up. I argue that you *possibly* open a very hard to exploit hole, which is exploitable only by sysadmins/DBAs, in return for which you close a very large hole that allows users to reuse passwords or use common or easy to guess words. If I am incorrect or have missed an important point, please explain why or what. This whole line of discussion just proves the point that was made originally: it would be a lot better to do whatever checking you want done on the client side, rather than risk transmitting unencrypted passwords. If you are going to imagine that client-side software knows about such a GUC, you might as well imagine that they have cracklib built in. Surely you can see that it is *absolutely pointless* to put an password complexity checking in the client? All a user would need to do is grab a copy of psql to bypass it. If they can't do that, there's probably a scripting language or 12 that would make it similarly easy. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Wed, Oct 14, 2009 at 11:44 AM, Dave Page dp...@pgadmin.org wrote: On Wed, Oct 14, 2009 at 4:30 PM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: On Wed, Oct 14, 2009 at 4:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: If you're really intent on making that happen, you can have your password checker plugin reject crypted passwords; we don't need such a questionable rule in core. Client software would need to have a standard way to know when to use ENCRYPTED PASSWORD or not. Oh, so you want us to propagate extra support for this blatant security reduction all over the system too? No thank you. You've twice asserted it's a reduction without providing any arguments to back that up. I argue that you *possibly* open a very hard to exploit hole, which is exploitable only by sysadmins/DBAs, in return for which you close a very large hole that allows users to reuse passwords or use common or easy to guess words. If I am incorrect or have missed an important point, please explain why or what. This whole line of discussion just proves the point that was made originally: it would be a lot better to do whatever checking you want done on the client side, rather than risk transmitting unencrypted passwords. If you are going to imagine that client-side software knows about such a GUC, you might as well imagine that they have cracklib built in. Surely you can see that it is *absolutely pointless* to put an password complexity checking in the client? All a user would need to do is grab a copy of psql to bypass it. If they can't do that, there's probably a scripting language or 12 that would make it similarly easy. To all of the above, +1. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Largeobject access controls
KaiGai Kohei kai...@ak.jp.nec.com writes: Tom Lane wrote: The most serious problem is that you ripped out myLargeObjectExists(), apparently because you didn't understand what it's there for. The reason it's there is to ensure that pg_dump runs don't fail. pg_dump is expected to produce a consistent dump of all large objects that existed at the time of its transaction snapshot. With this code, pg_dump would get a large object doesn't exist error on any LO that is deleted between the time of the snapshot and the time pg_dump actually tries to dump it --- which could be quite a large window in a large database. The origin of this matter is the basis of existence was changed. Our current basis is whether pg_largeobject has one or more data chunk for the given loid in the correct snapshot, or not. The newer one is whether pg_largeobject_metadata has the entry for the given loid in the SnapshotNow, or not. Which is wrong. You can certainly switch your attention as to which catalog to look in, but you can't change the snapshot that the test is referenced to. The newer basis is same as other database objects, such as functions. But why pg_dump performs correctly for these database objects? The reason pg_dump works reasonably well is that it takes locks on tables, and the other objects it dumps (such as functions) have indivisible one-row representations in the catalogs. They're either there when pg_dump looks, or they're not. What you would have here is that pg_dump would see LO data chunks when it looks into pg_largeobject using its transaction snapshot, and then its attempts to open those LO OIDs would fail because the metadata is not there anymore according to SnapshotNow. There are certainly plenty of corner-case issues in pg_dump; I've complained before that it's generally a bad idea to be migrating pg_dump functionality into internal backend routines that tend to rely on SnapshotNow. But if we change LO dumping like this, it's not going to be a corner case --- it's going to be a common race-condition failure with a window measured in many minutes if not hours. That's more than sufficient reason to reject this patch, because the added functionality isn't worth it. And pg_dump isn't even the only client that depends on the assumption that a read-only open LO is static. Moving on to lesser but still significant problems, you probably already guessed my next one: there's no pg_dump support. Hmm. I planed to add support to the pg_dump next to the serve-side enhancement. We do not commit system changes that lack necessary pg_dump support. There are some things you can leave till later, but pg_dump is not negotiable. (Otherwise, testers would be left with databases they can't dump/reload across the next forced initdb.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Dave Page dp...@pgadmin.org writes: You've twice asserted it's a reduction without providing any arguments to back that up. You quoted two good arguments why it's insecure in your original message, neither of which your proposed GUC does anything to protect against; and you also admitted that there might be other leakage paths we haven't thought of. That seems to me to be more than sufficient reason to not encourage people to go back to passing unencrypted passwords around. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Wed, Oct 14, 2009 at 5:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: You've twice asserted it's a reduction without providing any arguments to back that up. You quoted two good arguments why it's insecure in your original message, neither of which your proposed GUC does anything to protect against; I see one, and I proposed masking passwords in any relevant queries before they were written to the stats or logs to mitigate that. and you also admitted that there might be other leakage paths we haven't thought of. That seems to me to be more than sufficient reason to not encourage people to go back to passing unencrypted passwords around. Yes. Which is why I asked your opinion as there's a far greater chance you would know of any such paths than I, *and* whether they represent a greater risk than the complete lack of control over the effectiveness of users passwords that we currently have. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] alpha 2 release notes
On Wed, 2009-10-14 at 17:35 +0300, Peter Eisentraut wrote: In general, do we want the alpha 2 release notes as a separate section above alpha 1, or merged with the alpha 1 notes thereby giving a cumulative view of what happened since 8.4? Cumulative makes sense to me. Even if people are trying out all of the alpha releases, they might have forgotten to try some of the features from alpha1. That being said, it's probably a good idea to highlight or mark the features new to alpha2 somehow, perhaps with a *. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Dave Page dp...@pgadmin.org writes: On Wed, Oct 14, 2009 at 5:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: I see one, and I proposed masking passwords in any relevant queries before they were written to the stats or logs to mitigate that. Let's see you do that (hint: CREATD USER ... PASSWORD is going to throw a syntax error before you realize there's anything there that might need to be protected). And you ignored the question of insecure transmission pathways, anyway. By the time the backend has figured out that it's got a CREATE USER ... PASSWORD command, it's already way too late if the client sent it over a non-SSL connection. Marko has pointed out repeatedly that a plugin can catch the worst cases of insecure passwords even when given a pre-md5'd password. So you can use a plugin that does it that way, or if you want you can use a plugin that throws error on a pre-md5'd password. I do not see a reason for us to add a boatload of questionable logic that favors the latter approach. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Wed, Oct 14, 2009 at 5:25 PM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: On Wed, Oct 14, 2009 at 5:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: I see one, and I proposed masking passwords in any relevant queries before they were written to the stats or logs to mitigate that. Let's see you do that (hint: CREATD USER ... PASSWORD is going to throw a syntax error before you realize there's anything there that might need to be protected). And you ignored the question of insecure transmission pathways, anyway. By the time the backend has figured out that it's got a CREATE USER ... PASSWORD command, it's already way too late if the client sent it over a non-SSL connection. These are exactly the sort of issues I've been trying to get you to comment on. Thank you. Marko has pointed out repeatedly that a plugin can catch the worst cases of insecure passwords even when given a pre-md5'd password. My issues with that approach is that you may have to maintain a huge library of passwords - which you're never going to do efficiently or effectively if you assume that users may use simple variations of their username for example - eg. t0mlane. So you can use a plugin that does it that way, or if you want you can use a plugin that throws error on a pre-md5'd password. My only concern with that idea is having some way for the client to know when to not hash the password. Figuring that out from a specific error code and then retrying would be inefficient, ugly, and in the case of pgAdmin, quite hard to do given the way that SQL is generated and then executed. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] ECPG: store own copy of the prepared statement name
Hi,
the attached patch makes ECPG more robust
against applications that free() strings by storing
its own copy of the prepared statement name.
Best regards,
Zoltán Böszörményi
--
Bible has answers for everything. Proof:
But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
than these cometh of evil. (Matthew 5:37) - basics of digital technology.
May your kingdom come - superficial description of plate tectonics
--
Zoltán Böszörményi
Cybertec Schönig Schönig GmbH
http://www.postgresql.at/
*** pgsql.4/src/interfaces/ecpg/ecpglib/prepare.c~ 2009-10-14 15:34:46.0 +0200
--- pgsql.4/src/interfaces/ecpg/ecpglib/prepare.c 2009-10-14 15:34:46.0 +0200
***
*** 152,158
replace_variables((stmt-command), lineno);
/* add prepared statement to our list */
! this-name = (char *) name;
this-stmt = stmt;
/* and finally really prepare the statement */
--- 152,158
replace_variables((stmt-command), lineno);
/* add prepared statement to our list */
! this-name = strdup(name);
this-stmt = stmt;
/* and finally really prepare the statement */
***
*** 160,165
--- 160,166
if (!ecpg_check_PQresult(query, stmt-lineno, stmt-connection-connection, stmt-compat))
{
ecpg_free(stmt-command);
+ ecpg_free(this-name);
ecpg_free(this);
ecpg_free(stmt);
return false;
***
*** 238,243
--- 239,245
/* okay, free all the resources */
ecpg_free(this-stmt-command);
ecpg_free(this-stmt);
+ ecpg_free(this-name);
if (prev != NULL)
prev-next = this-next;
else
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Client application name
On Oct 13, 2009, at 11:02 AM, Dave Page wrote: A useful feature found in other DBMSs such as MS SQL Server that has been requested on these lists a few times, is the ability for a client application to report its name to the server. I've been following this thread closely and haven't seen mention of including the setting as part of the process name, so a 'ps' (on Unix) would display it. Thoughts? eric -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Wed, Oct 14, 2009 at 18:25, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: On Wed, Oct 14, 2009 at 5:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: I see one, and I proposed masking passwords in any relevant queries before they were written to the stats or logs to mitigate that. Let's see you do that (hint: CREATD USER ... PASSWORD is going to throw a syntax error before you realize there's anything there that might need to be protected). I'm unsure if it's our responsibility to think about that. We can leak a *lot* of sensitive information to the logs through syntax errors, this is just one of them. We *do* need to worry about the statements when they are sent properly, of course. So throwing out a wild idea that's probably just wild enough to even consider, but one way to deal with the logging side of things would be to deprecate/remove ALTER USER/CREATE USER with password, and add a separate API call. With a separate wire protocol packet. That would certainly take care of the logging part ;) And you ignored the question of insecure transmission pathways, anyway. By the time the backend has figured out that it's got a CREATE USER ... PASSWORD command, it's already way too late if the client sent it over a non-SSL connection. Encryption policy of this type cannot be set at the server. Ever. And it shouldn't. If the sysadmin is worried someone might sniff the traffic, he should require SSL, or use a VPN, or similar things. If he's fine with letting the client decide, then it's not a problem, because it's up to the client not to send the password that way. And if the client did, well, the sysadmin just said it was ok to let the client decide, so we're ok. Marko has pointed out repeatedly that a plugin can catch the worst cases of insecure passwords even when given a pre-md5'd password. So you can use a plugin that does it that way, or if you want you can use a plugin that throws error on a pre-md5'd password. I do not see a reason for us to add a boatload of questionable logic that favors the latter approach. it lets you find some passwords, but it does not let you set a *policy*. As is often required by security policies etc. Does anybody know how other databases that *do* implement this solve the problem? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Client application name
On Wed, Oct 14, 2009 at 5:37 PM, Eric B. Ridge e...@tcdi.com wrote: On Oct 13, 2009, at 11:02 AM, Dave Page wrote: A useful feature found in other DBMSs such as MS SQL Server that has been requested on these lists a few times, is the ability for a client application to report its name to the server. I've been following this thread closely and haven't seen mention of including the setting as part of the process name, so a 'ps' (on Unix) would display it. Thoughts? Isn't that cluttered enough already? -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Client application name
On Wed, Oct 14, 2009 at 18:39, Dave Page dp...@pgadmin.org wrote: On Wed, Oct 14, 2009 at 5:37 PM, Eric B. Ridge e...@tcdi.com wrote: On Oct 13, 2009, at 11:02 AM, Dave Page wrote: A useful feature found in other DBMSs such as MS SQL Server that has been requested on these lists a few times, is the ability for a client application to report its name to the server. I've been following this thread closely and haven't seen mention of including the setting as part of the process name, so a 'ps' (on Unix) would display it. Thoughts? Isn't that cluttered enough already? +1 for pg_stat_activity but not 'os' output. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Dave Page dp...@pgadmin.org writes: On Wed, Oct 14, 2009 at 5:25 PM, Tom Lane t...@sss.pgh.pa.us wrote: So you can use a plugin that does it that way, or if you want you can use a plugin that throws error on a pre-md5'd password. My only concern with that idea is having some way for the client to know when to not hash the password. Figuring that out from a specific error code and then retrying would be inefficient, ugly, and in the case of pgAdmin, quite hard to do given the way that SQL is generated and then executed. I don't actually *want* client software deciding on its own hook to not hash the password. The only reason we have special client-side support for this command at all is to pre-hash the password, and people are accustomed to that behavior. If psql or pgAdmin takes a password and then sends it in the clear without telling me, that's a breach of trust with potentially serious consequences. I might not trust the DBA, for example, or I might be less confident of the network infrastructure than he is. Fix pgAdmin so that this is a user-controlled switch and let the user read the plugin's error message and decide if he wants to retry with a non-encrypted password. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Magnus Hagander mag...@hagander.net writes: On Wed, Oct 14, 2009 at 18:25, Tom Lane t...@sss.pgh.pa.us wrote: Let's see you do that (hint: CREATD USER ... PASSWORD is going to throw a syntax error before you realize there's anything there that might need to be protected). I'm unsure if it's our responsibility to think about that. We can leak a *lot* of sensitive information to the logs through syntax errors, this is just one of them. We *do* need to worry about the statements when they are sent properly, of course. Even if they're sent properly, this entire discussion misses the point. The reason to not want cleartext passwords in the logs is that the user doesn't trust the DBA. Why would a user who doesn't trust the DBA want to trust him to not be running a modified copy of the database with all this nice logic disabled? The real point of crypted passwords is to not let uncrypted passwords go anywhere outside the *user's* control. If the DBA wants to enforce a policy that is incompatible with that, it should be extremely obvious to all concerned that that's what he's doing. In particular it should be in the user's face that he's about to send an uncrypted password, so that he can think twice about the particular password he's choosing (and hopefully not use one that's also good for another service). For relatively smart clients like pgAdmin, there might also be an option to refuse to send such a command across an insecure connection, or at least nag the user about it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Client application name
Dave Page dp...@pgadmin.org writes: On Wed, Oct 14, 2009 at 5:37 PM, Eric B. Ridge e...@tcdi.com wrote: I've been following this thread closely and haven't seen mention of including the setting as part of the process name, so a 'ps' (on Unix) would display it. Thoughts? Isn't that cluttered enough already? Given the security arguments going on in other threads, it occurs to me that putting app names into ps output could be considered positively undesirable --- you could not put anything into the name that you'd not want exposed to everyone on the server machine. Of course, since we're exposing user and database names there anyhow, I'm not sure that this argument carries much weight. But I think it's not totally clear what people might choose to use the appname for, so we ought to consider the possibility that there's some somewhat-sensitive info in it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
nMagnus Hagander wrote: Marko has pointed out repeatedly that a plugin can catch the worst cases of insecure passwords even when given a pre-md5'd password. So you can use a plugin that does it that way, or if you want you can use a plugin that throws error on a pre-md5'd password. ?I do not see a reason for us to add a boatload of questionable logic that favors the latter approach. it lets you find some passwords, but it does not let you set a *policy*. As is often required by security policies etc. I see three checks we are trying to do on passwords: 1) Password complexity enforcement/policies 2) Password history - you can't reuse a password 3) Account disable after X incorrect attempts #2 can be done by doing an MD5 on the server of the old password and comparing it to what the client sent --- no need for a client change there. #3 can be implemented in the server too. #1 could be implemented in the server without client change if you are willing to expand any wildcards in the password policy pattern, MD5 those, and check them against what was sent from the client, and this only happens during password change. However, that only can check for simplistic patterns, like trailing digit, etc, but not for things like upper/lower case usage, etc. If you have such a policy, aren't you already using LDAP or PAM, and can't those enforce it? -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Wed, Oct 14, 2009 at 12:25 PM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: On Wed, Oct 14, 2009 at 5:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: I see one, and I proposed masking passwords in any relevant queries before they were written to the stats or logs to mitigate that. Let's see you do that (hint: CREATD USER ... PASSWORD is going to throw a syntax error before you realize there's anything there that might need to be protected). It seems to me incredibly rare for anyone to issue a manual CREATE USER command with an encrypted password. And if it is generated by a script, it will presumably not have a trivial typographical error. And you ignored the question of insecure transmission pathways, anyway. By the time the backend has figured out that it's got a CREATE USER ... PASSWORD command, it's already way too late if the client sent it over a non-SSL connection. Using a non-SSL connection over an untrusted network is incredibly stupid to begin with. I'm not sure we should be basing our design decisions around that scenario. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Triggers on columns
Peter Eisentraut pete...@gmx.net writes: Btw., I might not get a chance to commit this within the next 48 hours. If someone else wants to, go ahead. I will take it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Client application name
On Oct 14, 2009, at 12:39 PM, Dave Page wrote: Isn't that cluttered enough already? I find the ps output uninformative. Having it display something that gets generated from my application would start to make it useful. Maybe what I really want is a totally different feature: log_line_prefix, but for process name. eric -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Robert Haas robertmh...@gmail.com writes: On Wed, Oct 14, 2009 at 12:25 PM, Tom Lane t...@sss.pgh.pa.us wrote: Let's see you do that (hint: CREATD USER ... PASSWORD is going to throw a syntax error before you realize there's anything there that might need to be protected). It seems to me incredibly rare for anyone to issue a manual CREATE USER command with an encrypted password. And if it is generated by a script, it will presumably not have a trivial typographical error. Uh, this discussion was about cleartext passwords? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] What does this configure warning mean?
When I run ./configure, I get (...warning..) (If you are using the official distribution of PostgreSQL then you do not need to worry about this because the Flex output is pre-generated.) Well, I am am using the official distribution of PostreSQL and the output is not pre-generated. Clearly this message only comes when you build Postgres from source, in which case the output cannot be pre-generated (can it?). It seems like a very strange warning to me. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What does this configure warning mean?
Simon Riggs si...@2ndquadrant.com writes: When I run ./configure, I get (If you are using the official distribution of PostgreSQL then you do not need to worry about this because the Flex output is pre-generated.) Well, I am am using the official distribution of PostreSQL and the output is not pre-generated. Official distribution means a tarball, not a CVS pull. The message is correct but perhaps could be reworded a bit. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What does this configure warning mean?
Simon Riggs wrote: When I run ./configure, I get (...warning..) (If you are using the official distribution of PostgreSQL then you do not need to worry about this because the Flex output is pre-generated.) Well, I am am using the official distribution of PostreSQL and the output is not pre-generated. Clearly this message only comes when you build Postgres from source, in which case the output cannot be pre-generated (can it?). I take it official distribution means tarball as different from a copy obtained thru CVS. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Wed, Oct 14, 2009 at 10:28 AM, Bruce Momjian br...@momjian.us wrote: I see three checks we are trying to do on passwords: 1) Password complexity enforcement/policies 2) Password history - you can't reuse a password 3) Account disable after X incorrect attempts This whole discussion seems very strange to me. Surely any organization with rules like this will want them to be system-wide and will have already implemented them in their PAM and LDAP systems (assuming their not using Kerberos or something like that anyways). There's not much point in reinventing the wheel in the database when a) we'll never be remotely as complete as the existing authentication systems -- the above requirements only barely scratch the surface and b) even if we were as complete as existing systems it would never be integrated so there would be nothing stopping people from reusing passwords from their login account or trying passwords a limited number of times against each system to get many attempts in total. Incidentally I'm extremely dubious of systems that implement your goal #3. It seems like more of an obvious DOS attack vector than a security improvement to me. There are better defense mechanisms for such attacks such as preauth. One more argument why we shouldn't be reimplementing the wheel in an area where don't have particularly good experience. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Wed, Oct 14, 2009 at 7:42 PM, Greg Stark gsst...@mit.edu wrote: On Wed, Oct 14, 2009 at 10:28 AM, Bruce Momjian br...@momjian.us wrote: I see three checks we are trying to do on passwords: 1) Password complexity enforcement/policies 2) Password history - you can't reuse a password 3) Account disable after X incorrect attempts This whole discussion seems very strange to me. Surely any organization with rules like this will want them to be system-wide and will have already implemented them in their PAM and LDAP systems (assuming their not using Kerberos or something like that anyways). Because like it or not, this 'feature' is one that people *are* looking for in early stages of evaluations, and it counts against us and can hurt our adoption when we can't tick that box. As an example, after years of only offering password policy management via the NT domain/active directory authentication methods, even Microsoft finally gave in and added policy management for their SQL Server accounts with SQL 2k5. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Wed, Oct 14, 2009 at 6:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Wed, Oct 14, 2009 at 18:25, Tom Lane t...@sss.pgh.pa.us wrote: Let's see you do that (hint: CREATD USER ... PASSWORD is going to throw a syntax error before you realize there's anything there that might need to be protected). I'm unsure if it's our responsibility to think about that. We can leak a *lot* of sensitive information to the logs through syntax errors, this is just one of them. We *do* need to worry about the statements when they are sent properly, of course. Even if they're sent properly, this entire discussion misses the point. The reason to not want cleartext passwords in the logs is that the user doesn't trust the DBA. Why would a user who doesn't trust the DBA want to trust him to not be running a modified copy of the database with all this nice logic disabled? If you trust him that little, why would you use a password that you also use elsewhere? Besides, if he can run a modified version of the database, its game over anyway. Just set pg_hba.conf's auth method to password, and you don't even have to wait for the user to change their password - you can grab it the next time he logs in. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Wed, Oct 14, 2009 at 1:48 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Oct 14, 2009 at 12:25 PM, Tom Lane t...@sss.pgh.pa.us wrote: Let's see you do that (hint: CREATD USER ... PASSWORD is going to throw a syntax error before you realize there's anything there that might need to be protected). It seems to me incredibly rare for anyone to issue a manual CREATE USER command with an encrypted password. And if it is generated by a script, it will presumably not have a trivial typographical error. Uh, this discussion was about cleartext passwords? I understand that. The point is, you seemed to be worried that log-obfuscation wouldn't work because someone might type CREATD USER ... PASSWORD rather than CREATE USER ... PASSWORD. But this can happen today, too, can't it? The only difference is that today the password MIGHT be encrypted. But if the user is really entering the command manually, it's probably not. Sure, someone COULD pre-MD5 a string and then copy and paste it into a psql session, but I bet that's not too common. I suspect people using the pre-MD5 option are using a more sophisticated client of some sort anyway. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On 10/14/2009 12:25 PM, Tom Lane wrote: Let's see you do that (hint: CREATD USER ... PASSWORD is going to throw a syntax error before you realize there's anything there that might need to be protected). And you ignored the question of insecure transmission pathways, anyway. By the time the backend has figured out that it's got a CREATE USER ... PASSWORD command, it's already way too late if the client sent it over a non-SSL connection. It seems like the architectural problem here is that people think of SQL as being a valid way to change one's password. I never thought it was valid? What if, like createdb, or createuser, there was a pgpasswd that did all of the appropriate checks and provided the proper security (MD5 today?) during transit and when storing to a log file? createuser already does password prompting - maybe it should allow checking the password from the createuser client side? I think if we looked at the SQL commands as being a 'base operation not intended for everyday users', this entire debate would seem frivolous? :-) Does Oracle really do password checks on the base SQL commands used to change an Oracle password? That sounds silly. I'm totally on Tom's side. Having the server check passwords is the wrong solution to the problem. It makes things worse - not better. That they're approving a plugin capability on the server at all is generous, as it does lead to a sense of protection that may not be justifiable. Cheers, mark -- Mark Mielkem...@mielke.cc -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Dave Page dp...@pgadmin.org writes: On Wed, Oct 14, 2009 at 7:42 PM, Greg Stark gsst...@mit.edu wrote: This whole discussion seems very strange to me. Surely any organization with rules like this will want them to be system-wide and will have already implemented them in their PAM and LDAP systems (assuming their not using Kerberos or something like that anyways). Because like it or not, this 'feature' is one that people *are* looking for in early stages of evaluations, and it counts against us and can hurt our adoption when we can't tick that box. Okay, fine, so we're not looking for actual high-grade security, we're looking to tick off a checkbox in the minds of not terribly well-informed people. Then the plugin mechanism as currently proposed will do the job just fine. We do not need to put a whole bunch of dubious extra infrastructure in there, and we DEFINITELY do not need anything that can be painted as a backwards step security-wise. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Wed, Oct 14, 2009 at 9:00 PM, Tom Lane t...@sss.pgh.pa.us wrote: Okay, fine, so we're not looking for actual high-grade security, we're looking to tick off a checkbox in the minds of not terribly well-informed people. Then the plugin mechanism as currently proposed will do the job just fine. We do not need to put a whole bunch of dubious extra infrastructure in there, and we DEFINITELY do not need anything that can be painted as a backwards step security-wise. Nice exit strategy :-) I said up front this was a box-ticking exercise for these folks, however, rather than just tick the box and move on (meh - who cares if we can store 2009-02-31 - it stores all the valid dates which are the ones that matter :-p ) I prefer to discuss the issue and do the best job we can to make it a practical, usable and useful feature - which is kinda what we usually pride ourselves in doing! -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Could regexp_matches be immutable?
I tried making a functional index based on an expression containing the 2 argument regexp_matches() function. Is there a reason why this function is not marked immutable instead of normal? regards, Rod Taylor -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Dave Page dp...@pgadmin.org writes: I said up front this was a box-ticking exercise for these folks, however, rather than just tick the box and move on (meh - who cares if we can store 2009-02-31 - it stores all the valid dates which are the ones that matter :-p ) I prefer to discuss the issue and do the best job we can to make it a practical, usable and useful feature - which is kinda what we usually pride ourselves in doing! Well, sure. I just don't want to move backwards on other dimensions in order to move forward on this one. It's fair to argue that support of pre-crypted passwords closes only some holes that can be closed in other ways, but it's equally fair to argue that the limited capability of a plugin that has to check pre-crypted passwords also represents a corner case that can be solved in other ways. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Dave Page dp...@pgadmin.org wrote: I said up front this was a box-ticking exercise for these folks, Can they check the box if the provided clients include password strength checking? I'm just wondering if we're going at this the hard way, if that really is the main goal. From the point of view of usefulness, wouldn't it be OK if clients enforced the strength (or at least warned of weakness) *and* sent the md5sum? And, perhaps slightly off topic: if the login password is sent over a non-encrypted stream, md5sum or not, can't someone use it to log in if they're generating their own stream to connect? Discussions of which is the more secure way to change passwords seems a little silly if you're only worried about environments where someone can sniff any login sequence and spoof the user anyway. (meh - who cares if we can store 2009-02-31 - it stores all the valid dates which are the ones that matter :-p ) Oh, now that's just trolling -- you really don't want to open that can of worms again, do you? :-p -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could regexp_matches be immutable?
Rod Taylor rod.tay...@gmail.com writes: I tried making a functional index based on an expression containing the 2 argument regexp_matches() function. Is there a reason why this function is not marked immutable instead of normal? regex_flavor affects its result. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could regexp_matches be immutable?
On Wed, Oct 14, 2009 at 04:51:03PM -0400, Tom Lane wrote: Rod Taylor rod.tay...@gmail.com writes: I tried making a functional index based on an expression containing the 2 argument regexp_matches() function. Is there a reason why this function is not marked immutable instead of normal? regex_flavor affects its result. Speaking of which, can we see about deprecating and removing this GUC? I've yet to hear of anyone using a flavor other than the default. Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could regexp_matches be immutable?
David Fetter wrote: On Wed, Oct 14, 2009 at 04:51:03PM -0400, Tom Lane wrote: Rod Taylor rod.tay...@gmail.com writes: I tried making a functional index based on an expression containing the 2 argument regexp_matches() function. Is there a reason why this function is not marked immutable instead of normal? regex_flavor affects its result. Speaking of which, can we see about deprecating and removing this GUC? I've yet to hear of anyone using a flavor other than the default. You have now. I have a client who sadly uses a non-default setting. And on 8.4, what is more. There are more things under heaven and earth cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could regexp_matches be immutable?
David Fetter da...@fetter.org wrote: On Wed, Oct 14, 2009 at 04:51:03PM -0400, Tom Lane wrote: Rod Taylor rod.tay...@gmail.com writes: I tried making a functional index based on an expression containing the 2 argument regexp_matches() function. Is there a reason why this function is not marked immutable instead of normal? regex_flavor affects its result. Speaking of which, can we see about deprecating and removing this GUC? +1 It would seem to me to be more valuable to have the benefits of IMMUTABLE than preserve pre-7.4 compatibility forever. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Kevin Grittner kevin.gritt...@wicourts.gov writes: And, perhaps slightly off topic: if the login password is sent over a non-encrypted stream, md5sum or not, can't someone use it to log in if they're generating their own stream to connect? Not if they only capture a login exchange --- the password is doubly encrypted during that. If they see the md5'd password in a CREATE USER command, then yeah, they could pass a subsequent md5 challenge, using suitably modified client software that doesn't try to re-encrypt the given password. But the main point is to hide the cleartext password, in any case. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What does this configure warning mean?
On Wed, 2009-10-14 at 15:06 -0300, Alvaro Herrera wrote: Simon Riggs wrote: When I run ./configure, I get (...warning..) (If you are using the official distribution of PostgreSQL then you do not need to worry about this because the Flex output is pre-generated.) Well, I am am using the official distribution of PostreSQL and the output is not pre-generated. Clearly this message only comes when you build Postgres from source, in which case the output cannot be pre-generated (can it?). I take it official distribution means tarball as different from a copy obtained thru CVS. Maybe If you are using an official release tarball of PostgreSQL, or an official release source code archive of PostgreSQL if we want to avoid the term tarball. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could regexp_matches be immutable?
On 10/14/09 2:07 PM, David Fetter wrote: On Wed, Oct 14, 2009 at 04:51:03PM -0400, Tom Lane wrote: Rod Taylor rod.tay...@gmail.com writes: I tried making a functional index based on an expression containing the 2 argument regexp_matches() function. Is there a reason why this function is not marked immutable instead of normal? regex_flavor affects its result. Speaking of which, can we see about deprecating and removing this GUC? I've yet to hear of anyone using a flavor other than the default. Actually, *we* (PGX) have a client who does. You just haven't worked on their stuff. --Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Wed, Oct 14, 2009 at 9:50 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Dave Page dp...@pgadmin.org wrote: I said up front this was a box-ticking exercise for these folks, Can they check the box if the provided clients include password strength checking? I'm just wondering if we're going at this the hard way, if that really is the main goal. No. Any checks at the client are worthless, as they can be bypassed by 10 minutes worth of simple coding in any of a dozen or more languages. And, perhaps slightly off topic: if the login password is sent over a non-encrypted stream, md5sum or not, can't someone use it to log in if they're generating their own stream to connect? Discussions of which is the more secure way to change passwords seems a little silly if you're only worried about environments where someone can sniff any login sequence and spoof the user anyway. No - see Tom's reply. (meh - who cares if we can store 2009-02-31 - it stores all the valid dates which are the ones that matter :-p ) Oh, now that's just trolling -- you really don't want to open that can of worms again, do you? :-p Well, after 12+ years in these parts I figure anyone should get the privilege of a small dig once in a while :-) -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could regexp_matches be immutable?
On Wed, Oct 14, 2009 at 05:14:31PM -0400, Andrew Dunstan wrote: David Fetter wrote: On Wed, Oct 14, 2009 at 04:51:03PM -0400, Tom Lane wrote: Rod Taylor rod.tay...@gmail.com writes: I tried making a functional index based on an expression containing the 2 argument regexp_matches() function. Is there a reason why this function is not marked immutable instead of normal? regex_flavor affects its result. Speaking of which, can we see about deprecating and removing this GUC? I've yet to hear of anyone using a flavor other than the default. You have now. I have a client who sadly uses a non-default setting. And on 8.4, what is more. OK, now I've heard of one. I still think we should deprecate and remove. Say, deprecate this next release and remove for the following one? There are more things under heaven and earth My philosophy doesn't include infinite backward compatibility. Neither do heaven and earth, come to think of it. :) Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Getting the red out (of the buildfarm)
On Sun, 2009-10-04 at 10:48 -0400, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: I understand the annoyance, but I think we do need to have an organized way to do testing of non-ASCII data and in particular UTF8 data, because there are an increasing number of special code paths for those. Well, if you want to keep the test, we should put in the variant with \200, because it is now clear that that is in fact the right answer in a nontrivial number of environments (arguably *more* cases than in which \u0080 is correct). I put in a new variant file. Let's see if it works. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kevin.gritt...@wicourts.gov writes: if the login password is sent over a non-encrypted stream, md5sum or not, can't someone use it to log in if they're generating their own stream to connect? If they see the md5'd password in a CREATE USER command, then yeah So anyone in a truly security-conscious environment should only change the password through a trusted connection, whether or not it is md5'd? (Trusted here meaning that the connection has suitable encryption or it is local to the server, which is suitably secured.) But the main point is to hide the cleartext password, in any case. Sure, sorry to drift off-topic, but the thought kept nagging at me. :-) Anyway, I think it helped crystallize something for me -- the client-side md5 mangling protects against malicious DBAs or database server admins, while the trusted connection protects against network snooping. The former is more about keeping the bad guys, who would have other ways of compromising the database, from gleaning information about your password choices that they may be able to apply to other resources. The latter protects the database itself. Did I miss any other forms of attack either addresses? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Dave Page dp...@pgadmin.org wrote: No. Any checks at the client are worthless, as they can be bypassed by 10 minutes worth of simple coding in any of a dozen or more languages. Well, sure, but we're talking about a client going out of their way to wrestle the point of the gun toward their own foot, aren't we? If we're worried about the user compromising their own password, we have bigger problems, like that slip of paper in their desk drawer with the password written on it. I mean, I know some of these checklists can be pretty brain-dead (I've been on both sides of the RFP process many times), but it would seem over the top to say that client-side password strength checks aren't OK for the reason you give. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could regexp_matches be immutable?
+1 It would seem to me to be more valuable to have the benefits of IMMUTABLE than preserve pre-7.4 compatibility forever. Just create a shell function which calls it in a specific flavor, and make that immutable. --Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Wed, Oct 14, 2009 at 10:51 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Dave Page dp...@pgadmin.org wrote: No. Any checks at the client are worthless, as they can be bypassed by 10 minutes worth of simple coding in any of a dozen or more languages. Well, sure, but we're talking about a client going out of their way to wrestle the point of the gun toward their own foot, aren't we? If we're worried about the user compromising their own password, we have bigger problems, like that slip of paper in their desk drawer with the password written on it. I mean, I know some of these checklists can be pretty brain-dead (I've been on both sides of the RFP process many times), but it would seem over the top to say that client-side password strength checks aren't OK for the reason you give. See my previous comment about dates. Check-box items aside, I have absolutely no desire to try to give the illusion of a security feature, when in reality any user could easily bypass it. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could regexp_matches be immutable?
Andrew Dunstan and...@dunslane.net writes: David Fetter wrote: Speaking of which, can we see about deprecating and removing this GUC? I've yet to hear of anyone using a flavor other than the default. You have now. I have a client who sadly uses a non-default setting. And on 8.4, what is more. How critical is it to them? It would be nice to get rid of that source of variability. It would be possible to keep using old-style regexes even without the GUC, if they can interpose anything that can stick an embedded options prefix on the pattern strings. See 9.7.3.4: http://developer.postgresql.org/pgdocs/postgres/functions-matching.html regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Tom Lane wrote: Kevin Grittner kevin.gritt...@wicourts.gov writes: And, perhaps slightly off topic: if the login password is sent over a non-encrypted stream, md5sum or not, can't someone use it to log in if they're generating their own stream to connect? Not if they only capture a login exchange --- the password is doubly encrypted during that. If they see the md5'd password in a CREATE USER command, then yeah, they could pass a subsequent md5 challenge, using suitably modified client software that doesn't try to re-encrypt the given password. But the main point is to hide the cleartext password, in any case. What if we added a GUC that only allowed password changes via an SSL connection. You could say that is a security enhancement, and administrators could set up their systems to use 'password' authentication for SSL and check the password strength on the server because they come in clear-text. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Wed, 2009-10-14 at 12:59 -0400, Tom Lane wrote: If psql or pgAdmin takes a password and then sends it in the clear without telling me, that's a breach of trust with potentially serious consequences. I might not trust the DBA, for example, or I might be less confident of the network infrastructure than he is. Well, you would lose anyway if the DBA switches the pg_hba.conf setting from md5 to password without telling you. There is usually no straightforward way in client applications to guard against that. Something to think about. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Triggers on columns
Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp writes: column-trigger_20091014.patch is a full patch to the HEAD. Applied with assorted corrections, mostly cosmetic but not entirely. One thing you really should have caught was updating copyfuncs/equalfuncs for the parsenode field addition. Next time, try grepping for every reference to a struct type you change ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could regexp_matches be immutable?
On Wed, Oct 14, 2009 at 06:06:23PM -0400, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: David Fetter wrote: Speaking of which, can we see about deprecating and removing this GUC? I've yet to hear of anyone using a flavor other than the default. You have now. I have a client who sadly uses a non-default setting. And on 8.4, what is more. How critical is it to them? It would be nice to get rid of that source of variability. It would be possible to keep using old-style regexes even without the GUC, if they can interpose anything that can stick an embedded options prefix on the pattern strings. See 9.7.3.4: http://developer.postgresql.org/pgdocs/postgres/functions-matching.html Switching it to just embedded options solves the issue of leaving the feature in while cutting the surprises down for those not using it. :) The embedded options method is also doable by search-and-replace, as they only work in AREs, which such people wouldn't be using. Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On 10/14/2009 05:33 PM, Dave Page wrote: On Wed, Oct 14, 2009 at 9:50 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Can they check the box if the provided clients include password strength checking? I'm just wondering if we're going at this the hard way, if that really is the main goal. No. Any checks at the client are worthless, as they can be bypassed by 10 minutes worth of simple coding in any of a dozen or more languages. Why care? If the client is purposefully disabling passwords checks to use a weak password - this is an entirely different problem from somebody trying a weak password and being allowed. Circumvention of process is always a risk, and should be dealt with as a human resources problem. Why not stop the admin from disabling the security check when they create their pgadmin password too? We can't trust anybody - right? PAM does security checking client-side I think? I'm sure others do too? I'm not saying server checks are worthless - but I think you are exaggerating to say that client checks are worthless. Sending the password in cleartext via SQL seems bad. Sending it encoded seems only marginally better. Sending it in MD5 is good but means that password strength needs to be done by the client. You are saying that it's worth the loss of security in one area, to improve security in another. Providing client checks in the official clients is probably sufficient for your checkbox that you think is so important. Unless you think it is impossible to circumvent process in any of these other databases that do such a better job? Personally, I don't think PostgreSQL is the best place to manage passwords at this level anyways, beyond the basic usage. PostgreSQL shouldn't need to know the password, and the password should still be required to as strong as the organization requires it. Lots of other solutions here - PAM, LDAP, Kerberos, ... How much of these solutions should PostgreSQL re-implement? Cheers, mark -- Mark Mielkem...@mielke.cc -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Peter Eisentraut pete...@gmx.net writes: Well, you would lose anyway if the DBA switches the pg_hba.conf setting from md5 to password without telling you. True :-(. Anybody for a zero-knowledge protocol? (Realistically, non-password-based auth methods are the only real solution here, I fear. We should probably be doing more to encourage people to use SSL-cert-based authentication in low-trust situations.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Wed, 2009-10-14 at 13:08 -0400, Tom Lane wrote: The reason to not want cleartext passwords in the logs is that the user doesn't trust the DBA. I originally implemented the encrypt-on-the-client logic because the previous way of using ALTER USER ... PASSWORD would promiscuously leave the clear-text password all over the place, including the psql history, where it accidentally pops up again when you scroll up, or possibly the pgAdmin log, and the server log where the DBA can't avoid reading it even if he doesn't really want to, from where it goes through PgFouine and onto a web server. While guarding against a DBA you don't trust is a side-effect of this, it should not really be a goal. Most password systems that I can think of pass the cleartext password into address space controlled by the administrator at some point. The goal should be to keep it there for as little as possible and not spread it around randomly. But users should know that the password they enter into any system can be seen by the administrator of the system. The canonical solution for a situation where you don't trust anyone is SSL client certificates. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Wed, 2009-10-14 at 18:38 +0200, Magnus Hagander wrote: So throwing out a wild idea that's probably just wild enough to even consider, but one way to deal with the logging side of things would be to deprecate/remove ALTER USER/CREATE USER with password, and add a separate API call. With a separate wire protocol packet. That would certainly take care of the logging part ;) I think that would be the correct fix. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Bruce Momjian br...@momjian.us writes: Tom Lane wrote: But the main point is to hide the cleartext password, in any case. What if we added a GUC that only allowed password changes via an SSL connection. How's that help? The user has already exposed their new choice of password to any hypothetical eavesdropper. Of course, if they're smart, they'll pick a different password before they try again on a secure connection ... but good luck hoping for that. (And, again, there is ABSOLUTELY NO NEED for us to put such debatable policies into the core. Anyone who thinks that's a good idea can have his password-check plugin enforce it.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On 10/14/2009 06:02 PM, Dave Page wrote: On Wed, Oct 14, 2009 at 10:51 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Well, sure, but we're talking about a client going out of their way to wrestle the point of the gun toward their own foot, aren't we? If we're worried about the user compromising their own password, we have bigger problems, like that slip of paper in their desk drawer with the password written on it. I mean, I know some of these checklists can be pretty brain-dead (I've been on both sides of the RFP process many times), but it would seem over the top to say that client-side password strength checks aren't OK for the reason you give. See my previous comment about dates. Check-box items aside, I have absolutely no desire to try to give the illusion of a security feature, when in reality any user could easily bypass it. This is only true if you thing 'create/alter role ... password ...' is intended to be used by hand - and if you think this, are you not concerned that the password shows up on the screen as the user types this? It's not an illusion if the user requires effort to bypass it. Who are you protecting? If you are trying to protect the user from themselves - I'd start by disabling 'create/alter role ... password ...' altogether and requiring some other means of the user securely setting a password. At our company we have a single-sign on system where users must login to a central password server to change their password. The central password server provides the password strength test. The intent is for none of the applications to ever know or see the password. It's not about trust - it is about principle. The DBA and/or application has no need or requirement to know the password. They only need to know that it is valid. For web-based applications, the user authentication is accomplished via a forward to one of the central password authentication servers, which upon successful login, forwards the user back to their intended application with a token that proves they have authenticated. If you think security is such an important thing - why not go all the way? Disable the entry level security provided by PostgreSQL allowing SQL commands to 'set' a password entirely. Use a token-based system with a third party trusted authenticator. I'm not being facetious. I think it's silly to argue that 50% is insufficient, and that lots of effort should be spent on reaching 55%, when you are clearly nowhere near 100% in either case. Cheers, mark -- Mark Mielkem...@mielke.cc -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On 10/14/2009 06:25 PM, Peter Eisentraut wrote: On Wed, 2009-10-14 at 18:38 +0200, Magnus Hagander wrote: So throwing out a wild idea that's probably just wild enough to even consider, but one way to deal with the logging side of things would be to deprecate/remove ALTER USER/CREATE USER with password, and add a separate API call. With a separate wire protocol packet. That would certainly take care of the logging part ;) I think that would be the correct fix. Yep. +1. If we are really so paranoid. Cheers, mark -- Mark Mielkem...@mielke.cc -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What does this configure warning mean?
Peter Eisentraut pete...@gmx.net writes: On Wed, 2009-10-14 at 15:06 -0300, Alvaro Herrera wrote: I take it official distribution means tarball as different from a copy obtained thru CVS. Maybe If you are using an official release tarball of PostgreSQL, or an official release source code archive of PostgreSQL if we want to avoid the term tarball. How about officially released copy? I agree that tarball isn't going to do here because some people won't understand it, but source code archive is just going to annoy the hackers and probably still confuse those who don't know tarball. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
* Tom Lane (t...@sss.pgh.pa.us) wrote: Peter Eisentraut pete...@gmx.net writes: Well, you would lose anyway if the DBA switches the pg_hba.conf setting from md5 to password without telling you. True :-(. Anybody for a zero-knowledge protocol? (Realistically, non-password-based auth methods are the only real solution here, I fear. We should probably be doing more to encourage people to use SSL-cert-based authentication in low-trust situations.) Or GSSAPI.. Helping users understand how they can leverage their existing Kerberos or MS SSPI single-sign-on infrastructures to securely access PG would go a long way to reducing the password-based usage out there, imo. Of course, it'd be nice if we supported GSSAPI encrypted transport too. Separating the encryption into SSL is less than ideal. Stephen signature.asc Description: Digital signature
Re: [HACKERS] What does this configure warning mean?
On Thu, 2009-10-15 at 00:26 +0300, Peter Eisentraut wrote: On Wed, 2009-10-14 at 15:06 -0300, Alvaro Herrera wrote: Simon Riggs wrote: When I run ./configure, I get (...warning..) (If you are using the official distribution of PostgreSQL then you do not need to worry about this because the Flex output is pre-generated.) Well, I am am using the official distribution of PostreSQL and the output is not pre-generated. Clearly this message only comes when you build Postgres from source, in which case the output cannot be pre-generated (can it?). I take it official distribution means tarball as different from a copy obtained thru CVS. Maybe If you are using an official release tarball of PostgreSQL, or an official release source code archive of PostgreSQL if we want to avoid the term tarball. Something like... If you have downloaded a full copy of the source then If you are building from a repository checkout ... We don't need to use the word tarball, but we do need to say what we mean. Anyway, its a seldom-used error message and I wouldn't have hit it if my laptop hadn't needed recovery. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Inappropriate failure conditions in foreign_data regression test
Martin Pihlak escribió: By chance I noticed that the foreign_data regression test fails if run in an installation where bob is a live user. It appears to be assuming that half a dozen other fairly common names don't belong to real users, either. Could we make this a little less fragile? Maybe Attached is a patch that attempts to fix that -- user mappings are now created for regress_test_missing_role instead of bob or baz. Applied, thanks. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Largeobject access controls
On Wed, Oct 14, 2009 at 12:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: KaiGai Kohei kai...@ak.jp.nec.com writes: Tom Lane wrote: The most serious problem is that you ripped out myLargeObjectExists(), apparently because you didn't understand what it's there for. The reason it's there is to ensure that pg_dump runs don't fail. pg_dump is expected to produce a consistent dump of all large objects that existed at the time of its transaction snapshot. With this code, pg_dump would get a large object doesn't exist error on any LO that is deleted between the time of the snapshot and the time pg_dump actually tries to dump it --- which could be quite a large window in a large database. The origin of this matter is the basis of existence was changed. Our current basis is whether pg_largeobject has one or more data chunk for the given loid in the correct snapshot, or not. The newer one is whether pg_largeobject_metadata has the entry for the given loid in the SnapshotNow, or not. Which is wrong. You can certainly switch your attention as to which catalog to look in, but you can't change the snapshot that the test is referenced to. The newer basis is same as other database objects, such as functions. But why pg_dump performs correctly for these database objects? The reason pg_dump works reasonably well is that it takes locks on tables, and the other objects it dumps (such as functions) have indivisible one-row representations in the catalogs. They're either there when pg_dump looks, or they're not. What you would have here is that pg_dump would see LO data chunks when it looks into pg_largeobject using its transaction snapshot, and then its attempts to open those LO OIDs would fail because the metadata is not there anymore according to SnapshotNow. There are certainly plenty of corner-case issues in pg_dump; I've complained before that it's generally a bad idea to be migrating pg_dump functionality into internal backend routines that tend to rely on SnapshotNow. But if we change LO dumping like this, it's not going to be a corner case --- it's going to be a common race-condition failure with a window measured in many minutes if not hours. That's more than sufficient reason to reject this patch, because the added functionality isn't worth it. And pg_dump isn't even the only client that depends on the assumption that a read-only open LO is static. Moving on to lesser but still significant problems, you probably already guessed my next one: there's no pg_dump support. Hmm. I planed to add support to the pg_dump next to the serve-side enhancement. We do not commit system changes that lack necessary pg_dump support. There are some things you can leave till later, but pg_dump is not negotiable. (Otherwise, testers would be left with databases they can't dump/reload across the next forced initdb.) As part of closing out this CommitFest, I am marking this patch as Returned with Feedback. Because the amount of work that remains to be done is so substantial, that might have been a good idea anyway, but since the clock is expiring the point is moot. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Buffer usage in EXPLAIN and pg_stat_statements (review)
On Wed, Oct 14, 2009 at 9:56 AM, Tom Lane t...@sss.pgh.pa.us wrote: Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp writes: Tom Lane t...@sss.pgh.pa.us wrote: 2. I do not understand the stuff with propagating counts into the top instrumentation node. It is required by contrib/pg_stat_statements. EXPLAIN wants per-node accumulation, but pg_stat_statements wants the total number. Well, you need to find another way or risk getting the patch rejected altogether. Those global variables are the weakest part of the whole design, and I'm not going to commit a patch that destabilizes the entire system for the sake of a debatable requirement of a contrib module. If you went with the alternative definition I suggested (don't reset the static counters, so that every node includes its children's counts) then the behavior you want would fall out automatically. Or, at the price of running both resettable and non-resettable static counters, you could have pg_stat_statements obtain totals that are independent of any particular instrumentation node. I am marking this patch as Returned with Feedback. I hope that it will be resubmitted for a future CommitFest, because I think this could be pretty interesting feature. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Review of SQLDA support for ECPG
On Sat, Oct 10, 2009 at 2:09 PM, Boszormenyi Zoltan z...@cybertec.at wrote: Michael Meskes írta: On Thu, Oct 08, 2009 at 01:15:58PM +0200, Boszormenyi Zoltan wrote: What's the point of that? It can't be applied without documentation, and it just makes life more complicated to have two separate patch files floating around. It's easier to write the documentation for all changes at once. I would have the same situation that happened with the code, the patches with the documentation added would strictly depend on each other again. Also, Michael Meskes applied the string pseudo-type patch without the documentation, despite the patch had it, maybe at an improper place. With a tongue-in-cheek I don't get it. Are you blaming me for committing you patch although it had no documentation? No blaming, but sorry, it definitely had, in both these rounds of split-up ECPG patchsets: 2009-05-15: http://archives.postgresql.org/message-id/4a5e0f1d.7030...@cybertec.at 2009-08-03: http://archives.postgresql.org/message-id/4a770c7b.1060...@cybertec.at I assumed you have applied it from the second mail, this was the last version sent for the string patch. It is my fault that I haven't put it on the CommitFest page. Indeed, the version on the CF page doesn't have documentation. But the code you committed seems to be the one (or very close, with some editorialization) in the second mail quoted above. Since it doesn't seem that any of the patches are going to get committed RSN, I have moved all of the open ECPG patches to the next CommitFest and given them their own section. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Triggers on columns
Tom Lane t...@sss.pgh.pa.us wrote: Applied with assorted corrections, mostly cosmetic but not entirely. One thing you really should have caught was updating copyfuncs/equalfuncs for the parsenode field addition. Next time, try grepping for every reference to a struct type you change ... Thanks for the fix. I'll be careful about it. Regards, --- ITAGAKI Takahiro NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Largeobject access controls
Tom Lane wrote: The newer basis is same as other database objects, such as functions. But why pg_dump performs correctly for these database objects? The reason pg_dump works reasonably well is that it takes locks on tables, and the other objects it dumps (such as functions) have indivisible one-row representations in the catalogs. They're either there when pg_dump looks, or they're not. What you would have here is that pg_dump would see LO data chunks when it looks into pg_largeobject using its transaction snapshot, and then its attempts to open those LO OIDs would fail because the metadata is not there anymore according to SnapshotNow. It needs to break down the matter more simple. This patch adds the pg_largeobject_metadata catalog, and a certain entry within the catalog is refered by multiple entries within pg_largeobject. At the viewpoint of data model, it is equivalent that any pg_largeobject tuples which share same LOID always have common copy of the metadata. (If we tries to implement this actually, it needs to keep consistency of the part of metadata for each writer operations!) In the current design, when we access a certain large object with read-only mode, query's snapshot is used, not always SnapshotNow. If we assume any data chunk has its metadata part, the metadata should be also refered from the query's snapshot. In fact, this patch stores the part of metadata within pg_largeobject_metadata. But it seems to me the principle is that same snapshot which used for data chunks should be applied to scan the metadata. So, I can agree the following approach: So I'm kind of inclined to say that the least evil solution is to apply the permissions check using the query-snapshot-time metadata row. It's definitely a debatable question though. We'd also want to make sure that the aclcheck code doesn't fail if it finds a nonexistent role ID in the ACL. In this approach, we cannot apply the current pg_largeobject_aclcheck() because it does not have an argument to deliver the preferable snapshot. So, we need to extract acl field from the tuple being visible from the appropriate snapshot, then calls aclmask() routine. The aclmask() just compares identifiers in numeris representation, so it seems to me this routine does not raise an error if it finds a nonexistent role ID from the viewpoint of SnapshotNow. aclmask(const Acl *acl, Oid roleid, Oid ownerId, AclMode mask, AclMaskHow how) It requires five arguments. The acl and ownerId can be fetched from the metadata with query's snapshot. The roleid can be given by GetUserId(). The mask and how are constant. An I missing something? We do not commit system changes that lack necessary pg_dump support. There are some things you can leave till later, but pg_dump is not negotiable. (Otherwise, testers would be left with databases they can't dump/reload across the next forced initdb.) OK, I'll add support pg_dump soon. Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Largeobject access controls
Robert Haas wrote: On Wed, Oct 14, 2009 at 12:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: KaiGai Kohei kai...@ak.jp.nec.com writes: Tom Lane wrote: The most serious problem is that you ripped out myLargeObjectExists(), apparently because you didn't understand what it's there for. The reason it's there is to ensure that pg_dump runs don't fail. pg_dump is expected to produce a consistent dump of all large objects that existed at the time of its transaction snapshot. With this code, pg_dump would get a large object doesn't exist error on any LO that is deleted between the time of the snapshot and the time pg_dump actually tries to dump it --- which could be quite a large window in a large database. The origin of this matter is the basis of existence was changed. Our current basis is whether pg_largeobject has one or more data chunk for the given loid in the correct snapshot, or not. The newer one is whether pg_largeobject_metadata has the entry for the given loid in the SnapshotNow, or not. Which is wrong. You can certainly switch your attention as to which catalog to look in, but you can't change the snapshot that the test is referenced to. The newer basis is same as other database objects, such as functions. But why pg_dump performs correctly for these database objects? The reason pg_dump works reasonably well is that it takes locks on tables, and the other objects it dumps (such as functions) have indivisible one-row representations in the catalogs. They're either there when pg_dump looks, or they're not. What you would have here is that pg_dump would see LO data chunks when it looks into pg_largeobject using its transaction snapshot, and then its attempts to open those LO OIDs would fail because the metadata is not there anymore according to SnapshotNow. There are certainly plenty of corner-case issues in pg_dump; I've complained before that it's generally a bad idea to be migrating pg_dump functionality into internal backend routines that tend to rely on SnapshotNow. But if we change LO dumping like this, it's not going to be a corner case --- it's going to be a common race-condition failure with a window measured in many minutes if not hours. That's more than sufficient reason to reject this patch, because the added functionality isn't worth it. And pg_dump isn't even the only client that depends on the assumption that a read-only open LO is static. Moving on to lesser but still significant problems, you probably already guessed my next one: there's no pg_dump support. Hmm. I planed to add support to the pg_dump next to the serve-side enhancement. We do not commit system changes that lack necessary pg_dump support. There are some things you can leave till later, but pg_dump is not negotiable. (Otherwise, testers would be left with databases they can't dump/reload across the next forced initdb.) As part of closing out this CommitFest, I am marking this patch as Returned with Feedback. Because the amount of work that remains to be done is so substantial, that might have been a good idea anyway, but since the clock is expiring the point is moot. Could you wait for the next 24 hours please? It is unproductive to break off the discussion for a month, even if the patch will be actually commited at the December. Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Reworks for Access Control facilities (r2363)
2009/10/13 KaiGai Kohei kai...@ak.jp.nec.com: The attached patch is a revised one with the following updates: Despite two fairly explicit requests, this patch (and, with the exception of ECPG, only this patch) has not yet been reviewed by a committer. http://archives.postgresql.org/pgsql-hackers/2009-10/msg00591.php http://archives.postgresql.org/pgsql-hackers/2009-10/msg00652.php Are any of the committers willing to take a look at this? Tom? Alvaro, maybe? Bruce? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Buffer usage in EXPLAIN and pg_stat_statements (review)
Robert Haas robertmh...@gmail.com wrote: Well, you need to find another way or risk getting the patch rejected altogether. ?Those global variables are the weakest part of the whole design, and I'm not going to commit a patch that destabilizes the entire system for the sake of a debatable requirement of a contrib module. I am marking this patch as Returned with Feedback. I hope that it will be resubmitted for a future CommitFest, because I think this could be pretty interesting feature. Ok, I'll reconsider them and re-submit patches for the next commitfest. Maybe I need to split the patch into EXPLAIN-part and contrib-part. Regards, --- ITAGAKI Takahiro NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Reworks for Access Control facilities (r2363)
Robert Haas robertmh...@gmail.com writes: Are any of the committers willing to take a look at this? Tom? I do plan to look at it tomorrow. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CommitFest 2009-09, two weeks on
On Wed, Sep 30, 2009 at 12:24 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Do you have any sense of how soon you'll feel confident to commit either patch? I'm bad at estimating. Not this week for sure, and next week I'm traveling and won't be able to spend as much time on it as I am right now. If no new major issues are found, and all the outstanding issues are resolved by me or Simon by then, maybe the week after that. It's now the week after that, so unless you're in the process of typing that 'cvs commit' command, I'm going to move HS + SR out to the next CommitFest so that we can close this one out and stamp alpha2. brief pause for objections will now ensue ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Reworks for Access Control facilities (r2363)
Robert Haas wrote: 2009/10/13 KaiGai Kohei kai...@ak.jp.nec.com: The attached patch is a revised one with the following updates: Despite two fairly explicit requests, this patch (and, with the exception of ECPG, only this patch) has not yet been reviewed by a committer. http://archives.postgresql.org/pgsql-hackers/2009-10/msg00591.php http://archives.postgresql.org/pgsql-hackers/2009-10/msg00652.php Are any of the committers willing to take a look at this? Tom? Alvaro, maybe? Bruce? In actually, I cannot believe this patch to be perfectly commitable by the 15-Oct due to the remaining time, but it is necessary to be comittable at the head of the next commit fest. In other word, I strongly want to continue the discussion and revising the patch, even if it will be actually commited at the 15-Nov. Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Buffer usage in EXPLAIN and pg_stat_statements (review)
2009/10/14 Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp: Robert Haas robertmh...@gmail.com wrote: Well, you need to find another way or risk getting the patch rejected altogether. ?Those global variables are the weakest part of the whole design, and I'm not going to commit a patch that destabilizes the entire system for the sake of a debatable requirement of a contrib module. I am marking this patch as Returned with Feedback. I hope that it will be resubmitted for a future CommitFest, because I think this could be pretty interesting feature. Ok, I'll reconsider them and re-submit patches for the next commitfest. Maybe I need to split the patch into EXPLAIN-part and contrib-part. My (limited) experience is that it's usually better to get something incremental committed, even if it's not what you really want. You can always take another crack at the remaining issues later, but if the whole patch gets shot down then you are out of luck. In this case, I think that the auto_explain changes out to be part of the same patch as the core EXPLAIN changes, but if the pg_stat_statement stuff is severable it might make sense to push that off until later. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CommitFest 2009-09, two weeks on
Robert Haas wrote: On Wed, Sep 30, 2009 at 12:24 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Do you have any sense of how soon you'll feel confident to commit either patch? I'm bad at estimating. Not this week for sure, and next week I'm traveling and won't be able to spend as much time on it as I am right now. If no new major issues are found, and all the outstanding issues are resolved by me or Simon by then, maybe the week after that. It's now the week after that, so unless you're in the process of typing that 'cvs commit' command, I'm going to move HS + SR out to the next CommitFest so that we can close this one out and stamp alpha2. brief pause for objections will now ensue I have a question. When we register the postponed patches on the CF-Nov site again, which tag should be choosen? Needs Review? or Ready for Commiter? Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Buffer usage in EXPLAIN and pg_stat_statements (review)
Robert Haas robertmh...@gmail.com wrote: My (limited) experience is that it's usually better to get something incremental committed, even if it's not what you really want. You can always take another crack at the remaining issues later, but if the whole patch gets shot down then you are out of luck. Yeah, that makes sense. But the partial change should also be a long-term solution ;-). It is hard to determine whether the partial change is a good solution until the whole features works as expected (at least partially). Regards, --- ITAGAKI Takahiro NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Buffer usage in EXPLAIN and pg_stat_statements (review)
On Wed, Oct 14, 2009 at 9:38 PM, Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp wrote: Robert Haas robertmh...@gmail.com wrote: My (limited) experience is that it's usually better to get something incremental committed, even if it's not what you really want. You can always take another crack at the remaining issues later, but if the whole patch gets shot down then you are out of luck. Yeah, that makes sense. But the partial change should also be a long-term solution ;-). It is hard to determine whether the partial change is a good solution until the whole features works as expected (at least partially). Well, that's an indication that you've chosen too small a piece. But I don't really believe that a change that affects only core EXPLAIN and auto_explain is too small a piece to be independently useful. If it is, the whole feature is probably badly conceived in the first place... ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CommitFest 2009-09, two weeks on
2009/10/14 KaiGai Kohei kai...@ak.jp.nec.com: Robert Haas wrote: On Wed, Sep 30, 2009 at 12:24 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Do you have any sense of how soon you'll feel confident to commit either patch? I'm bad at estimating. Not this week for sure, and next week I'm traveling and won't be able to spend as much time on it as I am right now. If no new major issues are found, and all the outstanding issues are resolved by me or Simon by then, maybe the week after that. It's now the week after that, so unless you're in the process of typing that 'cvs commit' command, I'm going to move HS + SR out to the next CommitFest so that we can close this one out and stamp alpha2. brief pause for objections will now ensue I have a question. When we register the postponed patches on the CF-Nov site again, which tag should be choosen? Needs Review? or Ready for Commiter? Needs Review. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Largeobject access controls
2009/10/14 KaiGai Kohei kai...@ak.jp.nec.com: Robert Haas wrote: On Wed, Oct 14, 2009 at 12:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: KaiGai Kohei kai...@ak.jp.nec.com writes: Tom Lane wrote: The most serious problem is that you ripped out myLargeObjectExists(), apparently because you didn't understand what it's there for. The reason it's there is to ensure that pg_dump runs don't fail. pg_dump is expected to produce a consistent dump of all large objects that existed at the time of its transaction snapshot. With this code, pg_dump would get a large object doesn't exist error on any LO that is deleted between the time of the snapshot and the time pg_dump actually tries to dump it --- which could be quite a large window in a large database. The origin of this matter is the basis of existence was changed. Our current basis is whether pg_largeobject has one or more data chunk for the given loid in the correct snapshot, or not. The newer one is whether pg_largeobject_metadata has the entry for the given loid in the SnapshotNow, or not. Which is wrong. You can certainly switch your attention as to which catalog to look in, but you can't change the snapshot that the test is referenced to. The newer basis is same as other database objects, such as functions. But why pg_dump performs correctly for these database objects? The reason pg_dump works reasonably well is that it takes locks on tables, and the other objects it dumps (such as functions) have indivisible one-row representations in the catalogs. They're either there when pg_dump looks, or they're not. What you would have here is that pg_dump would see LO data chunks when it looks into pg_largeobject using its transaction snapshot, and then its attempts to open those LO OIDs would fail because the metadata is not there anymore according to SnapshotNow. There are certainly plenty of corner-case issues in pg_dump; I've complained before that it's generally a bad idea to be migrating pg_dump functionality into internal backend routines that tend to rely on SnapshotNow. But if we change LO dumping like this, it's not going to be a corner case --- it's going to be a common race-condition failure with a window measured in many minutes if not hours. That's more than sufficient reason to reject this patch, because the added functionality isn't worth it. And pg_dump isn't even the only client that depends on the assumption that a read-only open LO is static. Moving on to lesser but still significant problems, you probably already guessed my next one: there's no pg_dump support. Hmm. I planed to add support to the pg_dump next to the serve-side enhancement. We do not commit system changes that lack necessary pg_dump support. There are some things you can leave till later, but pg_dump is not negotiable. (Otherwise, testers would be left with databases they can't dump/reload across the next forced initdb.) As part of closing out this CommitFest, I am marking this patch as Returned with Feedback. Because the amount of work that remains to be done is so substantial, that might have been a good idea anyway, but since the clock is expiring the point is moot. Could you wait for the next 24 hours please? It is unproductive to break off the discussion for a month, even if the patch will be actually commited at the December. Well, I don't know that I have the deciding vote here. I don't have any super-powers to make Tom - or anyone else - review the next version of this patch for this CommitFest, the next CommitFest, or at all. In my ideal world, Tom would review every patch in his area of expertise the day it came in and commit it right away, especially the ones that implement features I happen to like. But unless I offer to pay Tom's salary, and maybe not even then, that isn't going to happen. Backing up a little bit, my understanding of the CommitFest process is that it is a time for everyone to stop working on their own patches and help review and commit the patches of others. Because everyone wants to get back to their own work, we try very hard to wrap CommitFests up in one month so that everyone can then have a full month to do their own work before the next CommitFest starts. I don't see any reason to believe that this patch is so important that we should make an exception to that policy on its behalf, and I think it's unfair of you to ask. Tom, I, and others have almost entirely put aside our own development work for the last month to focus on this CommitFest. You haven't offered to help, are now asking us to put aside our work for even longer for your benefit. Furthermore, you haven't offered to help with either of the previous two CommitFests in which I've been involved either, despite having submitted large, complex patches that required substantial reviewing effort. All that having been said, I have no intention of or desire to foreclose discussion on this patch, or any desire to
Re: [HACKERS] Reworks for Access Control facilities (r2363)
On Wed, Oct 14, 2009 at 9:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Are any of the committers willing to take a look at this? Tom? I do plan to look at it tomorrow. Oh, great. You've done an impressive job slogging through a bunch of big, complex patches in the last week. Thanks, ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could regexp_matches be immutable?
Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: David Fetter wrote: Speaking of which, can we see about deprecating and removing this GUC? I've yet to hear of anyone using a flavor other than the default. You have now. I have a client who sadly uses a non-default setting. And on 8.4, what is more. How critical is it to them? It would be nice to get rid of that source of variability. It would be possible to keep using old-style regexes even without the GUC, if they can interpose anything that can stick an embedded options prefix on the pattern strings. See 9.7.3.4: http://developer.postgresql.org/pgdocs/postgres/functions-matching.html They are probably quite open to changing it, but IIRC it is a setting imposed by OpenACS, which is what they are based on. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could regexp_matches be immutable?
Andrew Dunstan and...@dunslane.net writes: They are probably quite open to changing it, but IIRC it is a setting imposed by OpenACS, which is what they are based on. I seem to recall having asked this before ... but does OpenACS even know what they're doing here? The difference between ERE mode and ARE mode is awfully slight. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Largeobject access controls
Robert Haas wrote: 2009/10/14 KaiGai Kohei kai...@ak.jp.nec.com: Robert Haas wrote: On Wed, Oct 14, 2009 at 12:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: KaiGai Kohei kai...@ak.jp.nec.com writes: Tom Lane wrote: The most serious problem is that you ripped out myLargeObjectExists(), apparently because you didn't understand what it's there for. The reason it's there is to ensure that pg_dump runs don't fail. pg_dump is expected to produce a consistent dump of all large objects that existed at the time of its transaction snapshot. With this code, pg_dump would get a large object doesn't exist error on any LO that is deleted between the time of the snapshot and the time pg_dump actually tries to dump it --- which could be quite a large window in a large database. The origin of this matter is the basis of existence was changed. Our current basis is whether pg_largeobject has one or more data chunk for the given loid in the correct snapshot, or not. The newer one is whether pg_largeobject_metadata has the entry for the given loid in the SnapshotNow, or not. Which is wrong. You can certainly switch your attention as to which catalog to look in, but you can't change the snapshot that the test is referenced to. The newer basis is same as other database objects, such as functions. But why pg_dump performs correctly for these database objects? The reason pg_dump works reasonably well is that it takes locks on tables, and the other objects it dumps (such as functions) have indivisible one-row representations in the catalogs. They're either there when pg_dump looks, or they're not. What you would have here is that pg_dump would see LO data chunks when it looks into pg_largeobject using its transaction snapshot, and then its attempts to open those LO OIDs would fail because the metadata is not there anymore according to SnapshotNow. There are certainly plenty of corner-case issues in pg_dump; I've complained before that it's generally a bad idea to be migrating pg_dump functionality into internal backend routines that tend to rely on SnapshotNow. But if we change LO dumping like this, it's not going to be a corner case --- it's going to be a common race-condition failure with a window measured in many minutes if not hours. That's more than sufficient reason to reject this patch, because the added functionality isn't worth it. And pg_dump isn't even the only client that depends on the assumption that a read-only open LO is static. Moving on to lesser but still significant problems, you probably already guessed my next one: there's no pg_dump support. Hmm. I planed to add support to the pg_dump next to the serve-side enhancement. We do not commit system changes that lack necessary pg_dump support. There are some things you can leave till later, but pg_dump is not negotiable. (Otherwise, testers would be left with databases they can't dump/reload across the next forced initdb.) As part of closing out this CommitFest, I am marking this patch as Returned with Feedback. Because the amount of work that remains to be done is so substantial, that might have been a good idea anyway, but since the clock is expiring the point is moot. Could you wait for the next 24 hours please? It is unproductive to break off the discussion for a month, even if the patch will be actually commited at the December. Well, I don't know that I have the deciding vote here. I don't have any super-powers to make Tom - or anyone else - review the next version of this patch for this CommitFest, the next CommitFest, or at all. In my ideal world, Tom would review every patch in his area of expertise the day it came in and commit it right away, especially the ones that implement features I happen to like. But unless I offer to pay Tom's salary, and maybe not even then, that isn't going to happen. I'm sorry for this a little bit emotional response. Anyway, what I wanted to say was my wish that I wanted to conclude or make a direction for the issue (what snapshot should be applied.) It is unclear now, so I would like to continue the discussion a bit more. Backing up a little bit, my understanding of the CommitFest process is that it is a time for everyone to stop working on their own patches and help review and commit the patches of others. Because everyone wants to get back to their own work, we try very hard to wrap CommitFests up in one month so that everyone can then have a full month to do their own work before the next CommitFest starts. I don't see any reason to believe that this patch is so important that we should make an exception to that policy on its behalf, and I think it's unfair of you to ask. Tom, I, and others have almost entirely put aside our own development work for the last month to focus on this CommitFest. You haven't offered to help, are now asking us to put aside our work for even longer for your benefit. Furthermore, you haven't
