Re: [HACKERS] [PATCH] Largeobject access controls
KaiGai Kohei wrote: I have to focus on my patches with highest priority in CommitFest, but it may be possible to help reviewing the proposed patches in the off-fest season. It is illegal/undesirable for the process? No, that's absolutely fine. During commitfests patch review is needed the most, but please do help whenever you have the time. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CommitFest 2009-09, two weeks on
Robert Haas wrote: On Wed, Sep 30, 2009 at 12:24 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Do you have any sense of how soon you'll feel confident to commit either patch? I'm bad at estimating. Not this week for sure, and next week I'm traveling and won't be able to spend as much time on it as I am right now. If no new major issues are found, and all the outstanding issues are resolved by me or Simon by then, maybe the week after that. It's now the week after that, so unless you're in the process of typing that 'cvs commit' command, I'm going to move HS + SR out to the next CommitFest so that we can close this one out and stamp alpha2. No objections here. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Skip WAL in ALTER TABLE
On Thu, 2009-10-15 at 13:18 +0900, Itagaki Takahiro wrote: Simon Riggs si...@2ndquadrant.com wrote: Is it possible to use WAL-skipping and BulkInsertState in ATRewriteTable() ? If ok, I'll submit a patch for the next commitfest. Yes Patch attached. This patch skip WAL writes during table rewrites from ALTER TABLE. Looks fine to me, apart from if (!XLogArchivingActive() || newrel-rd_istemp) hi_options |= HEAP_INSERT_SKIP_WAL; I think the second condition is unnecessary, so just if (!XLogArchivingActive()) hi_options |= HEAP_INSERT_SKIP_WAL; which is what COPY does. Temp tables are excluded in heap_insert() -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Wed, Oct 14, 2009 at 11:21 PM, Mark Mielke m...@mark.mielke.cc wrote: On 10/14/2009 05:33 PM, Dave Page wrote: No. Any checks at the client are worthless, as they can be bypassed by 10 minutes worth of simple coding in any of a dozen or more languages. Why care? Because many large (and small for that matter) organisations also have security policies which mandate the enforcement of specific password policies. Just because you think it's worthless to try to prevent someone reusing a password, or using 'password' doesn't mean that everyone else does. Some organisations will use such a feature in a box-ticking exercise when evaluating, and others may actually decide to use the feature, and expect it to work effectively. Beside, we are not in the habit of putting half-arsed features in PostgreSQL. If we do something, we do it properly. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby status
On Thu, 2009-10-15 at 10:33 +0300, Heikki Linnakangas wrote: There's been a lot of churn in hot standby since the beginning of the commitfest, so I thought it would be good to summarize where we are. Attached is the latest and greatest patch against CVS head, taken from the hs-riggs branch in my git repository where I've been working on this. Here's a list of TODOs/issues that've already been mentioned. - clarify default_transaction_read_only and transaction_read_only http://archives.postgresql.org/message-id/4ab75a61.6040...@enterprisedb.com Yes, still outstanding. This is the only unfixed issue I am aware of. It's not that big a deal, hence why its bottom of the pile. Anyway, will fix. The other issues are fixed in one or other of the current dev trees. - allow connections after a shutdown checkpoint - don't clear locks belonging to prepared transactions at startup - rename references to loggable locks to AccessExclusiveLocks in master or similar - race condition in xact_redo_commit/abort (http://archives.postgresql.org/message-id/4abf539f.8050...@enterprisedb.com) Not sure about those ones, as yet, but I do have 9 unapplied patches to move from my tree to the shared one. - connection goes out of sync when an idle-in-transaction transaction is killed (http://archives.postgresql.org/message-id/4acf77a5.1070...@enterprisedb.com) I reported that myself in September and fixed it before your report. So I guess there is some more code to move across as well. Before this is committed, there's some debug code that ought to be removed: - PostAuthDelay in startup process. Or maybe that should be left in, but some doc changes would then be in order. I'm inclined to remove it though. Happy if you'd like to remove it. - the CleanupWaitStats stuff. If we want something like this, I'd like to see it integrated into existing pg_stat views. Would like to leave it in, but only for now. We can always remove it before production. So far I've never seen it block there, so I'm interested in whether its worth the effort to track it more formally through stats. I regard this as in the same area as trace_sort, i.e. developer info. Let me know if I'm missing something. And please feel free to help, by testing, by reviewing and commenting on the patch, or by addressing any of the above issues. I will continue working on this, but this is a big patch so any help is much appreciated. Hopefully this comment isn't at me! I've been down for a few days with laptop problems, slowly re-installing everything (for those laughing, yes, I had a backup). -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Wed, Oct 14, 2009 at 11:44 PM, Stephen Frost sfr...@snowman.net wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: Peter Eisentraut pete...@gmx.net writes: Well, you would lose anyway if the DBA switches the pg_hba.conf setting from md5 to password without telling you. True :-(. Anybody for a zero-knowledge protocol? (Realistically, non-password-based auth methods are the only real solution here, I fear. We should probably be doing more to encourage people to use SSL-cert-based authentication in low-trust situations.) Or GSSAPI.. Helping users understand how they can leverage their existing Kerberos or MS SSPI single-sign-on infrastructures to securely access PG would go a long way to reducing the password-based usage out there, imo. Of course, it'd be nice if we supported GSSAPI encrypted transport too. Separating the encryption into SSL is less than ideal. Such solutions are exactly what I'd expect to actually go into production in most places, but that doesn't mean that people don't pay attention to the basic features offered as part of the core database when they're early in the evaluation phase. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby status
Simon Riggs wrote: On Thu, 2009-10-15 at 10:33 +0300, Heikki Linnakangas wrote: Let me know if I'm missing something. And please feel free to help, by testing, by reviewing and commenting on the patch, or by addressing any of the above issues. I will continue working on this, but this is a big patch so any help is much appreciated. Hopefully this comment isn't at me! Heh, no, I know you're working on it. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] EXPLAIN BUFFERS
Robert Haas robertmh...@gmail.com wrote: In this case, I think that the auto_explain changes out to be part of the same patch as the core EXPLAIN changes Here is a rewritten patch to add EXPLAIN (ANALYZE, BUFFERS) and support for it by contrib/auto_explain. I removed pg_stat_statements support from the patch for now. I modifed heavily in buffer statistics conters; These counters are put all together into struct BufferUsage. The struct is also used in struct Instrumentation. The global buffer usage counters are saved into 'bufusage_start' field at the InstrStartNode(), and accumulated into 'bufusage' field and global counters are reset at InstrStopNode(). EXPLAIN BUFFERS only shows 'hit', 'read' and 'temp read' in text format to fit in display, but xml or json format contains all of them. I removed ShowBufferUsage() because we can retrieve the same information from xml or json explain output, but the patch does not drop log_statement_stats variable families nor ShowUsage() functions. We could also remove all of them if no one use them at all. Regards, --- ITAGAKI Takahiro NTT Open Source Software Center explain_buffers_20091015.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ECPG: store own copy of the prepared statement name
On Wed, Oct 14, 2009 at 06:37:43PM +0200, Boszormenyi Zoltan wrote: the attached patch makes ECPG more robust against applications that free() strings by storing its own copy of the prepared statement name. Please do not call strdup() directly in libecpg. Instead please use ecpg_strdup() which does all the error handling needed. I changed this and committed the patch. Thanks. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org ICQ: 179140304, AIM/Yahoo/Skype: michaelmeskes, Jabber: mes...@jabber.org Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ECPG: store own copy of the prepared statement name
Michael Meskes írta: On Wed, Oct 14, 2009 at 06:37:43PM +0200, Boszormenyi Zoltan wrote: the attached patch makes ECPG more robust against applications that free() strings by storing its own copy of the prepared statement name. Please do not call strdup() directly in libecpg. Instead please use ecpg_strdup() which does all the error handling needed. I changed this and committed the patch. Thanks. Michael Thanks, I will keep it in mind. Best regards, Zoltán Böszörményi -- Bible has answers for everything. Proof: But let your communication be, Yea, yea; Nay, nay: for whatsoever is more than these cometh of evil. (Matthew 5:37) - basics of digital technology. May your kingdom come - superficial description of plate tectonics -- Zoltán Böszörményi Cybertec Schönig Schönig GmbH http://www.postgresql.at/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] visual c++ compile error when included spi.h and trigger.h
hi there,
i have trouble compiling a c style program ( filename with extension cpp)
written for visual C++.
When i added the following #includes,
#include spi.h
#include trigger.h
compilation errors say error C2899: typename cannot be used outside a template
declaration.
I need the above includes for postgres triggers.
If the above #includes were removed, leaving behind
#include postgres.h
#include fmgr.h
then there is no compilation error.
A check on msdn reveals that the error is because in visual c++, the word
typename is a keyword.
Unfortunately, it happens ( coincidence) that postgres also uses the word
typename in its header files, as in below -
typedef struct ColumnDef
{
NodeTag type;
char*colname; /* name of column */
TypeName *typename; /* type of column */
...
}
I have thought of making visual c++ of not recognizing typename as a keyword,
say, by turning some compiler options to forced c mode ( rather than c++).
Is there indeed such an option? If not, is there a way to resolve my problem as
described above?
Would appreciate any help ...
thanks and regards,
baluku
Re: [HACKERS] visual c++ compile error when included spi.h and trigger.h
On Thu, 2009-10-15 at 17:44 +0800, mingsoftt wrote: I have thought of making visual c++ of not recognizing typename as a keyword, say, by turning some compiler options to forced c mode ( rather than c++). Is there indeed such an option? If not, is there a way to resolve my problem as described above? Wait for 8.5. Before that, server header files are not C++ safe. What you discovered is only one of several problems. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Dave Page dp...@pgadmin.org wrote: On Wed, Oct 14, 2009 at 10:51 PM, Kevin Grittner bigger problems, like that slip of paper in their desk drawer with the password written on it. See my previous comment about dates. Check-box items aside, I have absolutely no desire to try to give the illusion of a security feature, when in reality any user could easily bypass it. I think you missed my point -- if you want to try to block the user from compromising their *own* password, you can't. They can tell anybody they want, write it on a slip of paper stuck to their terminal (yes, I've seen that), let it loose any other way they want. Why focus on one (rather unlikely) way that a user could compromise their own password when there are so many other ways, much easier and more likely to actually happen, which are totally out of our control? If a simple client-side strength check would allow the box to be checked, and would protect any user who isn't going out of their way to let their password be abused, I'm not really understanding your objection. Now, if it fails to cover the checkbox because it can't check against the last three passwords used, that's another story, but the server-side plugin can easily cover things like that. And ultimately, if you really care about tight security rather than checking off a box, other posts address how that can actually be done. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Client application name
On Wed, Oct 14, 2009 at 3:42 PM, Tom Lane t...@sss.pgh.pa.us wrote: Sure. I'm envisioning that what the env variable or connection option actually does is cause libpq to include a SET command for a GUC variable in the initial connection request packet. Compare, say, PGCLIENTENCODING - client_encoding. So you can now do any of the following to set the application name: - Set $PGAPPLICATIONNAME on the client, prior to connection. - Include 'application_name=MyCoolApp' in the PQconnectdb connection string. - Use SET application_name Currently though, pg_dump and psql (and presumably their close friends) use PQsetdbLogin to establish their connection. Would it be preferred if I: a) Added PQsetdbLogin2() with an additional option for the application name (my guess is 'no'). b) Updated the apps to use PQconnectdb c) Something else? -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Thu, Oct 15, 2009 at 2:49 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Dave Page dp...@pgadmin.org wrote: On Wed, Oct 14, 2009 at 10:51 PM, Kevin Grittner bigger problems, like that slip of paper in their desk drawer with the password written on it. See my previous comment about dates. Check-box items aside, I have absolutely no desire to try to give the illusion of a security feature, when in reality any user could easily bypass it. I think you missed my point -- if you want to try to block the user from compromising their *own* password, you can't. They can tell anybody they want, write it on a slip of paper stuck to their terminal (yes, I've seen that), let it loose any other way they want. Why focus on one (rather unlikely) way that a user could compromise their own password when there are so many other ways, much easier and more likely to actually happen, which are totally out of our control? It's certainly true that there are other ways for users to compromise their passwords if they want. The fact remains though, that most other DBMSs (and all major operating systems I can think of) offer password policy features as non-client checks which are difficult, if not impossible for the user to bypass. Clearly other people think it's important to do this, and we are compared against their products on a daily basis, so if we want to compete with them on a level playing field we need at least a comparable feature set. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Client application name
Dave Page escreveu: On Wed, Oct 14, 2009 at 3:42 PM, Tom Lane t...@sss.pgh.pa.us wrote: Sure. I'm envisioning that what the env variable or connection option actually does is cause libpq to include a SET command for a GUC variable in the initial connection request packet. Compare, say, PGCLIENTENCODING - client_encoding. So you can now do any of the following to set the application name: - Set $PGAPPLICATIONNAME on the client, prior to connection. - Include 'application_name=MyCoolApp' in the PQconnectdb connection string. - Use SET application_name Works for me. Currently though, pg_dump and psql (and presumably their close friends) use PQsetdbLogin to establish their connection. Would it be preferred if I: a) Added PQsetdbLogin2() with an additional option for the application name (my guess is 'no'). b) Updated the apps to use PQconnectdb c) Something else? My prefered option is (b). But it should be a separate patch. -- Euler Taveira de Oliveira http://www.timbira.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Client application name
Dave Page dp...@pgadmin.org writes: a) Added PQsetdbLogin2() with an additional option for the application name (my guess is 'no'). b) Updated the apps to use PQconnectdb c) Something else? a) is absolutely right out. b) is okay from an overall viewpoint but you would find yourself doing something very much like this: http://archives.postgresql.org/message-id/3073cc9b0909141123s7ec779a0h6524ec90a0a8...@mail.gmail.com So I would suggest d) leave the issue unsolved until some descendant of that patch gets committed, and then use it. There is also e) do nothing, since the environment var and SET options are plenty. Also, I am wondering exactly what you think psql would *do* with the parameter if it had it. If the answer is force the setting to be 'psql', that's the wrong answer. IMO you'd really want the environment variable to take precedence over that, if set. But libpq considers the priority to go the other way. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Client application name
On Thu, Oct 15, 2009 at 3:28 PM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: a) Added PQsetdbLogin2() with an additional option for the application name (my guess is 'no'). b) Updated the apps to use PQconnectdb c) Something else? a) is absolutely right out. b) is okay from an overall viewpoint but you would find yourself doing something very much like this: http://archives.postgresql.org/message-id/3073cc9b0909141123s7ec779a0h6524ec90a0a8...@mail.gmail.com So I would suggest d) leave the issue unsolved until some descendant of that patch gets committed, and then use it. Ooh, I like that patch. I'll wait for that. There is also e) do nothing, since the environment var and SET options are plenty. Also, I am wondering exactly what you think psql would *do* with the parameter if it had it. If the answer is force the setting to be 'psql', that's the wrong answer. IMO you'd really want the environment variable to take precedence over that, if set. But libpq considers the priority to go the other way. Well in the psql case, it could flip that priority itself and only set the value if the environment variable wasn't set - which I agree would seem the right thing to do. On further thought, it would seem reasonable to do the same in the other apps as well, so you could have your backup script do something like PGAPPLICATIONNAME=Nightly backup /usr/bin/pg_dump ... -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Mark Mielke wrote: Does Oracle really do password checks on the base SQL commands used to change an Oracle password? That sounds silly. In Oracle you can write a stored procedure to check passwords; it is invoked whenever a user is created or altered. No matter how you change the password, Oracle can always recover the plaintext and feed it to the password checking function. So, unless you use the Advanced Security option (extra $$) that enables you to encrypt network connections, any eavesdropper with knowledge of Oracle's (secret) encryption algorithms can get your new password when you change it. And the DBA can get your password with ease. Yours, Laurenz Albe -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Trigger with WHEN clause (WIP)
2009/10/15 Tom Lane t...@sss.pgh.pa.us: Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp writes: I think there is a benefit to provide WHEN cluase at least for compatibility with other DBMSs, even through we can move the expressions into the body of trigger functions. This seems to me to be a lot of code to accomplish nothing useful. It will always be the case that any nontrivial logic has to be done inside the trigger. And the compatibility argument is entirely pointless given the lack of compatibility in the trigger function itself. -1 I disagree. When I analysed speed of some operations, I found some unwanted trigger calls should to slow down applications. I am for any method, that could to decrease trigger calls. Regards Pavel Stehule regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Trigger with WHEN clause (WIP)
Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp writes: I think there is a benefit to provide WHEN cluase at least for compatibility with other DBMSs, even through we can move the expressions into the body of trigger functions. This seems to me to be a lot of code to accomplish nothing useful. It will always be the case that any nontrivial logic has to be done inside the trigger. And the compatibility argument is entirely pointless given the lack of compatibility in the trigger function itself. -1 regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] EXPLAIN BUFFERS
On Thu, Oct 15, 2009 at 7:29 AM, Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp wrote: EXPLAIN BUFFERS only shows 'hit', 'read' and 'temp read' in text format to fit in display, but xml or json format contains all of them. I was very careful when I submitted the machine-readable explain patch to make sure that the choice of which information was displayed was independent of the format, and I think that we should stick with that. If you want we could have 'buffers terse' and 'buffers detail' but I don't think we should force JSON/XML on people who want to see that information. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Client application name
Dave Page dp...@pgadmin.org writes: On Thu, Oct 15, 2009 at 3:28 PM, Tom Lane t...@sss.pgh.pa.us wrote: Also, I am wondering exactly what you think psql would *do* with the parameter if it had it. If the answer is force the setting to be 'psql', that's the wrong answer. IMO you'd really want the environment variable to take precedence over that, if set. But libpq considers the priority to go the other way. Well in the psql case, it could flip that priority itself and only set the value if the environment variable wasn't set - which I agree would seem the right thing to do. On further thought, it would seem reasonable to do the same in the other apps as well, so you could have your backup script do something like PGAPPLICATIONNAME=Nightly backup /usr/bin/pg_dump ... Hmm. Maybe this is a generic problem. Should libpq offer some sort of help? Maybe a secondaryappname parameter that doesn't override the env variable. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Trigger with WHEN clause (WIP)
Pavel Stehule pavel.steh...@gmail.com writes: 2009/10/15 Tom Lane t...@sss.pgh.pa.us: This seems to me to be a lot of code to accomplish nothing useful. I disagree. When I analysed speed of some operations, I found some unwanted trigger calls should to slow down applications. I am for any method, that could to decrease trigger calls. That argument is based on a completely evidence-free assumption, namely that this patch would make your case faster. Executing the WHEN tests is hardly going to be zero cost. It's not too hard to postulate cases where implementing a filter this way would be *slower* than doing it inside the trigger. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Largeobject access controls
2009/10/15 Heikki Linnakangas heikki.linnakan...@enterprisedb.com: KaiGai Kohei wrote: I have to focus on my patches with highest priority in CommitFest, but it may be possible to help reviewing the proposed patches in the off-fest season. It is illegal/undesirable for the process? No, that's absolutely fine. During commitfests patch review is needed the most, but please do help whenever you have the time. I agree. I would also mention that reviewing a patch can often be done in a few hours, and there are certainly periods of downtime during a CommitFest when your own patches won't need attention. Of course, a major patch like Hot Standby needs a lot more reviewing, but that's an exceptional case, and there's really no need for you to bite off something that ambitious. Just reviewing 1 or 2 small patches per CommitFest would be much appreciated, if it's something you can manage. Reviewing patches at other times is very helpful too. Many times patch authors complain about not getting feedback between CommitFests, so anything you can pick up and give feedback on helps move the project forward (and also decreases the amount that has to get done during the next CommitFest). Thanks, ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] EXPLAIN BUFFERS
Robert Haas robertmh...@gmail.com writes: On Thu, Oct 15, 2009 at 7:29 AM, Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp wrote: EXPLAIN BUFFERS only shows 'hit', 'read' and 'temp read' in text format to fit in display, but xml or json format contains all of them. I was very careful when I submitted the machine-readable explain patch to make sure that the choice of which information was displayed was independent of the format, and I think that we should stick with that. I thought one of the main purposes of adding the machine-readable formats was to allow inclusion of information that we thought too verbose for the human-readable format. Whether this info falls into that category remains to be seen, but I don't agree with the premise that the information content must always be exactly the same. FWIW, the patch's output as it stood a few days ago (one extra line per node, conditional on a BUFFERS option) did seem perfectly reasonable to me, and I don't see the reason to change that format now. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Client application name
On Thu, Oct 15, 2009 at 3:50 PM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: On Thu, Oct 15, 2009 at 3:28 PM, Tom Lane t...@sss.pgh.pa.us wrote: Also, I am wondering exactly what you think psql would *do* with the parameter if it had it. If the answer is force the setting to be 'psql', that's the wrong answer. IMO you'd really want the environment variable to take precedence over that, if set. But libpq considers the priority to go the other way. Well in the psql case, it could flip that priority itself and only set the value if the environment variable wasn't set - which I agree would seem the right thing to do. On further thought, it would seem reasonable to do the same in the other apps as well, so you could have your backup script do something like PGAPPLICATIONNAME=Nightly backup /usr/bin/pg_dump ... Hmm. Maybe this is a generic problem. Should libpq offer some sort of help? Maybe a secondaryappname parameter that doesn't override the env variable. is it worth uglifying libpq? All we're talking about is something like: if (!getenv(PGAPPLICATIONNAME)) strncat(connstr, application_name=psql, sizeof(connstr) - strlen(connstr) - 1); -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] EXPLAIN BUFFERS
On Thu, Oct 15, 2009 at 11:06 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Thu, Oct 15, 2009 at 7:29 AM, Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp wrote: EXPLAIN BUFFERS only shows 'hit', 'read' and 'temp read' in text format to fit in display, but xml or json format contains all of them. I was very careful when I submitted the machine-readable explain patch to make sure that the choice of which information was displayed was independent of the format, and I think that we should stick with that. I thought one of the main purposes of adding the machine-readable formats was to allow inclusion of information that we thought too verbose for the human-readable format. Whether this info falls into that category remains to be seen, but I don't agree with the premise that the information content must always be exactly the same. Hmm. I thought that the purpose of having a generalized options syntax was that people could have the information they wanted, independently of the format they chose. Even with a lot of extra information, the human readable format is still far shorter and more easily readable than either of the others. If we had gone with the idea of just dumping everything in the world into the XML format, you'd be right: but for various reasons we decided against that, which I'm very happy about. FWIW, the patch's output as it stood a few days ago (one extra line per node, conditional on a BUFFERS option) did seem perfectly reasonable to me, and I don't see the reason to change that format now. Yep, agreed. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Client application name
Dave Page dp...@pgadmin.org writes: On Thu, Oct 15, 2009 at 3:50 PM, Tom Lane t...@sss.pgh.pa.us wrote: Hmm. Maybe this is a generic problem. Should libpq offer some sort of help? Maybe a secondaryappname parameter that doesn't override the env variable. is it worth uglifying libpq? All we're talking about is something like: if (!getenv(PGAPPLICATIONNAME)) strncat(connstr, application_name=psql, sizeof(connstr) - strlen(connstr) - 1); Well, per earlier discussion we're not really interested in doing this with connection strings, it's going to be some sort of parameter array deal. This approach might require a read-write instead of constant parameter array (depending on how the other patch eventually works out). But the main thing that's bugging me about it is the explicit knowledge on the application's part that there is an environment variable that interacts with this setting. Seems ugly and badly-factored. Another possibility that should be mentioned for the record is that we could special-case the appname parameter inside libpq, so that the environment variable takes precedence over the conn setting instead of the other way round. That seems pretty ugly too, but maybe it's the least bad answer, if we decide that most apps need that precedence. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Client application name
On Thu, Oct 15, 2009 at 4:37 PM, Tom Lane t...@sss.pgh.pa.us wrote: Another possibility that should be mentioned for the record is that we could special-case the appname parameter inside libpq, so that the environment variable takes precedence over the conn setting instead of the other way round. That seems pretty ugly too, but maybe it's the least bad answer, if we decide that most apps need that precedence. Looking further, I think this might be quite clean: - Add a precedence flag to PQconninfoOption - In conninfo_parse, in the section that grabs the envvars for empty params, modify the logic to override any existing values if a value is set in the environment and the precedence flag is set for that option. That may be useful in the future for other options, and will certainly be less ugly than special casing one setting. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Client application name
Dave Page dp...@pgadmin.org writes: Looking further, I think this might be quite clean: - Add a precedence flag to PQconninfoOption - In conninfo_parse, in the section that grabs the envvars for empty params, modify the logic to override any existing values if a value is set in the environment and the precedence flag is set for that option. That may be useful in the future for other options, and will certainly be less ugly than special casing one setting. Changing PQconninfoOption would be an ABI break necessitating a soname bump and recompiling all libpq-using applications. If there were a significant probability that we'd have additional options in future that should act this way, it might be worth it. But I don't think there is. The approach with two different conninfo options is really probably the logically cleanest answer short of an ABI break. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On 10/15/2009 03:54 AM, Dave Page wrote: On Wed, Oct 14, 2009 at 11:21 PM, Mark Mielkem...@mark.mielke.cc wrote: On 10/14/2009 05:33 PM, Dave Page wrote: No. Any checks at the client are worthless, as they can be bypassed by 10 minutes worth of simple coding in any of a dozen or more languages. Why care? Because many large (and small for that matter) organisations also have security policies which mandate the enforcement of specific password policies. Just because you think it's worthless to try to prevent someone reusing a password, or using 'password' doesn't mean that everyone else does. Some organisations will use such a feature in a box-ticking exercise when evaluating, and others may actually decide to use the feature, and expect it to work effectively. Beside, we are not in the habit of putting half-arsed features in PostgreSQL. If we do something, we do it properly. You miss my point (and conveniently cut it out). For users who accidentally break policy vs users who purposefully circumvent policy - the approaches must be different, and the risk management decision may be different. It's a lot easier to circumvent policy than most people (management specifically) realize. If your attempt it to absolutely prevent a determined competent individual from circumventing your policy - you need to do a LOT MORE than what you are suggesting. If you just want to prevent accidents - having the client software do the checks is fine. Cheers, mark -- Mark Mielkem...@mielke.cc -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On 10/15/2009 10:08 AM, Dave Page wrote: It's certainly true that there are other ways for users to compromise their passwords if they want. The fact remains though, that most other DBMSs (and all major operating systems I can think of) offer password policy features as non-client checks which are difficult, if not impossible for the user to bypass. Clearly other people think it's important to do this, and we are compared against their products on a daily basis, so if we want to compete with them on a level playing field we need at least a comparable feature set. Not so clear to me. If they're doing strong checks, this means they're sending passwords in the clear or only barely encoded, or using some OTHER method than 'alter role ... password ...' to change the password. Point being - if you think this is absolutely important to do - don't go +5% of the way - go 100% of the way. Then again, I'm not so concerned about what arbitrary criteria some person defines as what makes a good database system. I'm more concerned with what makes the system better for *me*. I don't see how this entire thread helps *me* in any way - and I do understand the need for strong passwords - and my company *does* have policies that require strong passwords. Even if the plugin is provided - I'm not going to activate it. I already have a policy for setting strong passwords that I already follow. Cheers, mark -- Mark Mielkem...@mielke.cc -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On 10/15/2009 10:38 AM, Albe Laurenz wrote: Mark Mielke wrote: Does Oracle really do password checks on the base SQL commands used to change an Oracle password? That sounds silly. In Oracle you can write a stored procedure to check passwords; it is invoked whenever a user is created or altered. No matter how you change the password, Oracle can always recover the plaintext and feed it to the password checking function. So, unless you use the Advanced Security option (extra $$) that enables you to encrypt network connections, any eavesdropper with knowledge of Oracle's (secret) encryption algorithms can get your new password when you change it. And the DBA can get your password with ease. Now I remember. Our secure password server used as single-sign on for most applications in the company, which normally avoids any applications ever having to see or authenticate the login, needs to send the passwords in plain or encoded form (not one-way encrypted) to a few broken systems, which include systems designed around Oracle user management, to allow people to login to these applications using their corporate wide password. I remember thinking one word when I learned this... awesome (sarcastic tone playing in head). Where is the check box that says prevents password recovery? For Dave Page: Understand that in a large company, as you are discussing, with policies that require strong passwords, the usual reason for requiring strong passwords is due to concerns over privilege escalation. Access to one system gives you access to others. In the case of single sign-on, which is a requirement for any business with dozens or more applications, where it is unreasonable for each employee to actually memorize dozens of distinct strong passwords, access to one system gives you access to all systems. Therefore, trust the DBA makes no sense. If the DBA can see my password, then they can login to my employee records and check out what my salary or contact information is, or they can login to one of the secure portals and authorize purchases as me. A *good* system, is not trusted with the password. This is why I say you are focusing on making PostgreSQL what you think is a tiny bit better, but the gain is minor or artificial. If PostgreSQL starts requiring strong passwords - the world is not necessarily a better place in any mind except the person doing the ignorant checkbox evaluation who believes advertising on face value. If you need security - you should know enough to know you need something better than per-application password strength checkers. Cheers, mark -- Mark Mielkem...@mielke.cc -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Thu, Oct 15, 2009 at 5:28 PM, Mark Mielke m...@mark.mielke.cc wrote: Not so clear to me. If they're doing strong checks, this means they're sending passwords in the clear or only barely encoded, or using some OTHER method than 'alter role ... password ...' to change the password. Some are sending them in the clear (though often over SSL connections). Point being - if you think this is absolutely important to do - don't go +5% of the way - go 100% of the way. Exactly - that's why I want to see a check in the server, not the client which should get to 95%. I also happen to agree with Magnus that the only really secure way to do this on outside of SQL, but I can't see us dropping ALTER USER ... WITH PASSWORD in a hurry. Then again, I'm not so concerned about what arbitrary criteria some person defines as what makes a good database system. I'm more concerned with what makes the system better for *me*. I don't see how this entire thread helps *me* in any way - and I do understand the need for strong passwords - and my company *does* have policies that require strong passwords. Even if the plugin is provided - I'm not going to activate it. I already have a policy for setting strong passwords that I already follow. That's an excellent point. It probably doesn't make any difference to you or many of the other people on this list who are concerned with running their own systems and may already use other techniques, such as LDAP, SSPI etc. A not-insignificant percentage of the people here are not concerned with running their own systems though. They are working to help new users adopt PostgreSQL, and make a living selling services or support to those users. Sometimes that can be for huge projects, where it is necessary to justify every difference in check-box items against other products to get past the early eval stages. Like it or not, that is a fact, and this hampers our adoption. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Mark Mielke wrote: On 10/15/2009 10:08 AM, Dave Page wrote: ...other DBMSs (and all major operating systems I can think of) offer password policy features as non-client checks...we are compared ... Not so clear to me. If they're doing strong checks, this means they're sending passwords in the clear or only barely encoded, or using some OTHER method than 'alter role ... password ...' to change the password. This makes it sounds like a documentation problem to me. We need to educate the security-feature-checklist writers. It seems we need to clearly spell out the security risks of sending plain text passwords in the section where we would state the reason why the checks are done in the client - and then hopefully the security checklists writers will include only sends encrypted passwords as a checkbox on the product comparison charts. And if server-side checks are that important, perhaps the wiki needs an example of how to enable server-side check for popular GSSAPI or LDAP or PAM configurations and describe how to make postgres use those. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On 10/15/09 9:41 AM, Dave Page wrote: Sometimes that can be for huge projects, where it is necessary to justify every difference in check-box items against other products to get past the early eval stages. Like it or not, that is a fact, and this hampers our adoption. Precisely, and I think you've gotten away from that in your proposal. Any company who wants to institute a *truly* secure password policy is going to use LDAP, Kerberos, SSPI or GASSPI. Therefore what you're proposing is enabling band-aiding for the companies who don't want to really implement secure password control, but want to *feel* like they have. Why does this sound like a misfeature? Enabling the inclusion of a password checker in the client *would* improve things by preventing stupid users from setting their password the same as their username, or to a 3-letter word, or anything equally stupid which can be checked in a contextless way. This would be an real, incremental improvement *without* breaking anything else. And presumably would help our checkboxyness. I also think that it would be minimally intrusive to allow the PostgreSQL server to refuse connections from clients which didn't send a signal that they had the password-checker enabled. I *don't* think that guarding against users who are skilled enough to fake the password checker signal is worth *anyone's* time. First, a user that skilled presumably knows enough to pick secure passwords in the first place. Second, nothing Postgres can realistcally do in the way of password checking is going to protect us against a hacker with a knowledge of postgres internals. And, again, companies worrying about this are going to be using LDAP or GSSPI. Now, this thread has identified a number of areas where we could realistically improve password security: * Implement GASSPI encryption * Allow superusers to disable ALTER USER SET PASSWORD for normal users. * After the above, create a new createuser which works with ALTER USER disabled and uses a password checklib. * Implement the rest of the above suggestion. But I've seen nothing in Dave's other proposals which would *actually* improve password security as opposed to doing exactly the opposite. Requiring passwords to be sent unhashed over the wire so that they can be checked on the server is like making sure that your front door is always locked by giving keys to everyone you meet. In fact, given Dave's pursuit of a specific set of requirements, I think he has *one* specific client in mind rather than a generalized requirement. For my part, I've not in 10 years had anyone ask me for password checking in Postgres as an evaluation criteron. Encrypted data, yes. --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Thu, Oct 15, 2009 at 12:23 PM, Mark Mielke m...@mark.mielke.cc wrote: You miss my point (and conveniently cut it out). For users who accidentally break policy vs users who purposefully circumvent policy - the approaches must be different, and the risk management decision may be different. It's a lot easier to circumvent policy than most people (management specifically) realize. If your attempt it to absolutely prevent a determined competent individual from circumventing your policy - you need to do a LOT MORE than what you are suggesting. If you just want to prevent accidents - having the client software do the checks is fine. I don't get it. It's easy to circumvent client checks by using a different client. Circumventing server checks is really hard. You have to be able to hack the server. It seems obvious to me that putting the checks on the server is raising the bar by at least one order of magnitude and more likely two or three. Now, it's true that server-side checks on plaintext passwords are a security risk - in paricular, even with SSL, they can be captured by a modified server. So from the point of view of the *employee*, sending plaintext passwords may be less secure, because they don't know where their password is going. But from the point of view of the *server administrator*, they are more secure, because the server administrator believes (likely entirely correctly) that the odds of an employee picking a bad password (perhaps by firing up psql, which is not exactly a difficult-to-obtain utility) are higher than the odds that someone will trojan his server and install a password capture tool. If we were using some kind of real public key system and someone suggested breaking it to add password complexity checking, I would understand the outrage here. But I don't understand why everyone is so worked up about having an *optional* *flag* to force plaintext instead of MD5. I might be wrong here, but can't a determined attacker brute-force an MD5 anyway? The very fact that people are suggesting that password checking might be feasible even on a pre-MD5'd password by using a dictionary suggests that we're not getting a whole lot of real security here. And even if not, dude, it's an *optional* *flag*. Document that using it has certain advantages and certain disadvantages, and let users make their own decision about whether to deploy it. If they make a bad decision and get hacked as a result, tell them it's their own darn fault for setting the flag. I reiterate my previous opposition to the idea that I wouldn't use that feature is a reason not to add it. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Reworks for Access Control facilities (r2363)
KaiGai Kohei kai...@ak.jp.nec.com writes: [ patch r2363 ] I promised I would review this today, but I just can't make myself do it in any detail. This is too large, too ugly, and it is going in a direction that I do not like or want to spend any of my time on. The overwhelming impression after a brief read-through is that the code has been hacked apart with a chainsaw and reassembled into a Frankenstein's monster --- it's alive, but man is it ugly. Code comments that refer to something above or below are still there, but the referent is no longer close enough for that to be a reasonable way of referring to it. It's impossible to follow what's going on or why, either in the shim functions or in the callers --- in the original coding there was context for the aclcheck calls, now there isn't. I don't have any confidence that this is a sane way to proceed forward. The shim layer knows everything about everything --- there may still be a few backend .h files it doesn't include, but that's not for lack of trying. The direction this is heading in is an unmaintainable giant-bowl-of-spaghetti security module, rather than something that can be divided into understandable parts. And I don't think it's really removed any complexity from the callers, nor do I believe that it's going to be a useful basis for imposing a different security policy than the one we have now. Two specific examples of why not: * The skip permissions checks arguments that have been added to various random functions suggest strongly that the factoring still isn't right --- I especially don't believe in that in the context of performDeletion and friends. * There are two special-purpose shims, ac_database_calculate_size and ac_tablespace_calculate_size, that got added for the benefit of utils/adt/dbsize.c. What if that code were still in contrib? How is it different from a lot of the code that is in contrib now, eg dblink or pgrowlocks, to say nothing of third-party modules? Presuming that the shim layer can know explicitly about each individual permission-checking requirement is a dead-end design. Maybe if I weren't burned out after a month of CommitFesting, I could muster a more positive reaction, but right now I just can't summon any enthusiasm for this. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could regexp_matches be immutable?
(I'd bet lunch that the one about add_missing_from is bogus, too, or could easily be made so. mysql isn't forgiving about missing FROM items, so it's hard to believe that they have a lot of such things no matter how little they care about Postgres.) OpenACS does the old-style DELETEs without a subselect, so they rely on add-missing-from for that. I had to debug this for another user. --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Robert Haas robertmh...@gmail.com writes: If we were using some kind of real public key system and someone suggested breaking it to add password complexity checking, I would understand the outrage here. But I don't understand why everyone is so worked up about having an *optional* *flag* to force plaintext instead of MD5. I might be wrong here, but can't a determined attacker brute-force an MD5 anyway? The very fact that people are suggesting that password checking might be feasible even on a pre-MD5'd password by using a dictionary suggests that we're not getting a whole lot of real security here. And even if not, dude, it's an *optional* *flag*. Yes, and it's an optional flag that could perfectly well be implemented in the plugin that I think we do have consensus to add a hook for. The argument is over why do we need to litter the core system with it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Thu, Oct 15, 2009 at 6:17 PM, Josh Berkus j...@agliodbs.com wrote: Enabling the inclusion of a password checker in the client *would* improve things by preventing stupid users from setting their password the same as their username, or to a 3-letter word, or anything equally stupid which can be checked in a contextless way. This would be an real, incremental improvement *without* breaking anything else. And presumably would help our checkboxyness. What client? For the vast majority of users, what you're proposing is 'do it yourself'. Most people don't use pgAdmin or psql. But I've seen nothing in Dave's other proposals which would *actually* Actually, I made just one proposal, to complement a patch that has already been submitted. improve password security as opposed to doing exactly the opposite Requiring passwords to be sent unhashed over the wire so that they can be checked on the server is like making sure that your front door is always locked by giving keys to everyone you meet. As Peter pointed out, it's already game-over if you're worried about the DBA. For other cases, I was clear that SSL should be expected. In fact, given Dave's pursuit of a specific set of requirements, I think My only request was for server-side password policy enforcement, and a way to ensure users/DBAs could use pgAdmin to manage those passwords. he has *one* specific client in mind rather than a generalized requirement. For my part, I've not in 10 years had anyone ask me for password checking in Postgres as an evaluation criteron. Encrypted data, yes. I don't deal with prospective clients, which is where this comes from. I do deal with a team of (pre)sales engineers who complain about this, and maybe half-a-dozen other issues on a very regular basis. They tell me that PostgreSQL loses out in early stages of tech evals because of this issue, and I have no reason to disbelieve them. Sure it's almost certainly not the only reason, but they add up. And yes, data encryption is one of the other checkbox items that they bleat about. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Thu, Oct 15, 2009 at 6:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: If we were using some kind of real public key system and someone suggested breaking it to add password complexity checking, I would understand the outrage here. But I don't understand why everyone is so worked up about having an *optional* *flag* to force plaintext instead of MD5. I might be wrong here, but can't a determined attacker brute-force an MD5 anyway? The very fact that people are suggesting that password checking might be feasible even on a pre-MD5'd password by using a dictionary suggests that we're not getting a whole lot of real security here. And even if not, dude, it's an *optional* *flag*. Yes, and it's an optional flag that could perfectly well be implemented in the plugin that I think we do have consensus to add a hook for. The argument is over why do we need to litter the core system with it. I already said that would suit me. The only other requirement I would have is a way for pgAdmin or other clients to figure out if that flag was set so they could construct queries appropriately (and yes, that could include refusing to send plain text passwords over non-SSL connections). -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Thu, Oct 15, 2009 at 1:47 PM, Dave Page dp...@pgadmin.org wrote: On Thu, Oct 15, 2009 at 6:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: If we were using some kind of real public key system and someone suggested breaking it to add password complexity checking, I would understand the outrage here. But I don't understand why everyone is so worked up about having an *optional* *flag* to force plaintext instead of MD5. I might be wrong here, but can't a determined attacker brute-force an MD5 anyway? The very fact that people are suggesting that password checking might be feasible even on a pre-MD5'd password by using a dictionary suggests that we're not getting a whole lot of real security here. And even if not, dude, it's an *optional* *flag*. Yes, and it's an optional flag that could perfectly well be implemented in the plugin that I think we do have consensus to add a hook for. The argument is over why do we need to litter the core system with it. I already said that would suit me. The only other requirement I would have is a way for pgAdmin or other clients to figure out if that flag was set so they could construct queries appropriately (and yes, that could include refusing to send plain text passwords over non-SSL connections). OK, so we're in violent agreement here? Except for figuring out how an API for checking the flag? Could they just try it with MD5 first and then fall back if that say no MD5? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Thu, Oct 15, 2009 at 6:55 PM, Robert Haas robertmh...@gmail.com wrote: OK, so we're in violent agreement here? From a technical perspective I think we have been for a while. Though clearly some people disagree with my assertion that putting any form of policy enforcement in the client is not actually 'enforcement'. I wonder how many of those folks would implement their website's data sanitisation in the browser only - but I digress... :-) Except for figuring out how an API for checking the flag? Could they just try it with MD5 first and then fall back if that say no MD5? That's what I was trying to avoid, as the architecture of pgAdmin makes that really hard. I know that's not PG's problem, but forcing a retry is quite an ugly solution anyway, so I was hoping we could come up with something better. I suppose in the worst case, I could just have pgAdmin throw the error, and then add a per-server option to disable password hashing in the relevant places, but I'd far rather have that automated so it can't be set unnecessarily. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Could regexp_matches be immutable?
On Thu, Oct 15, 2009 at 10:22:52AM -0700, Josh Berkus wrote: (I'd bet lunch that the one about add_missing_from is bogus, too, or could easily be made so. mysql isn't forgiving about missing FROM items, so it's hard to believe that they have a lot of such things no matter how little they care about Postgres.) OpenACS does the old-style DELETEs without a subselect, so they rely on add-missing-from for that. I had to debug this for another user. Is OpenACS getting enough new deployments to fix this? Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On 10/15/2009 01:44 PM, Dave Page wrote: I don't deal with prospective clients, which is where this comes from. I do deal with a team of (pre)sales engineers who complain about this, and maybe half-a-dozen other issues on a very regular basis. They tell me that PostgreSQL loses out in early stages of tech evals because of this issue, and I have no reason to disbelieve them. Sure it's almost certainly not the only reason, but they add up. A lot of evaluations are designed to fit exactly one product, and it's impossible to win here. In my own company, I recently saw the most ridiculous (to me) evaluations over a suite of products, that effectively listed an exact implementation as requirements. This resulted in a huge split between people who considered the evaluation fair and who went with their choice for exactly that one product, and the rest of the people who called the evaluation a sham and refused to participate, choosing to instead use their own choice of products not caring about the outcome of the evaluation. The evaluation, by the way, included other silly statements, like how a database instance costs $48k in license fees, even though everybody knew we were already using PostgreSQL for $0k or even if we chose to be supported by one of the many PostgreSQL support companies, it would not cost $48k. Where did they get that number? Because they presumed they would go with Oracle. The evaluation was a sham from start to finish. Perhaps you can see how little I value some arbitrary checkbox list on some evaluation? If people want to count PostgreSQL off the list from the start - they will, and there is not much you or I can do about it. Bowing to the pressure of fulfilling these checkboxes, when they'll just change them next time to something else that PostgreSQL doesn't quite do, is a waste of time. We should do what is right to do. We should not be focusing on checkboxes raised by other people who are not competent enough to understand the subject matter or who have already made their choice, and the evaluation is just a rubber stamp to pretend they have done due diligence about justifying their choice compared to alternatives. Cheers, mark -- Mark Mielkem...@mielke.cc -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Dave Page dp...@pgadmin.org writes: On Thu, Oct 15, 2009 at 6:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Yes, and it's an optional flag that could perfectly well be implemented in the plugin that I think we do have consensus to add a hook for. The argument is over why do we need to litter the core system with it. I already said that would suit me. The only other requirement I would have is a way for pgAdmin or other clients to figure out if that flag was set so they could construct queries appropriately (and yes, that could include refusing to send plain text passwords over non-SSL connections). Well, if it's a GUC implemented by a plugin, it's still a GUC. All you need is some side agreement between pgAdmin and potential GUC authors about what the GUC will be called. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Dave Page dp...@pgadmin.org writes: I suppose in the worst case, I could just have pgAdmin throw the error, and then add a per-server option to disable password hashing in the relevant places, but I'd far rather have that automated so it can't be set unnecessarily. As I commented before, I think that automating this is actually a dangerous misfeature. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On 10/15/2009 02:02 PM, Dave Page wrote: On Thu, Oct 15, 2009 at 6:55 PM, Robert Haasrobertmh...@gmail.com wrote: OK, so we're in violent agreement here? From a technical perspective I think we have been for a while. Though clearly some people disagree with my assertion that putting any form of policy enforcement in the client is not actually 'enforcement'. I wonder how many of those folks would implement their website's data sanitisation in the browser only - but I digress... :-) It depends on what your goal is. If your goal is to treat users as monkeys that you do not trust, even with their own password, and the DBA as God, who you absolutely do trust, than you are correct. I don't know about your company - but in my company, the DBAs are in the IT department, and they really have no business knowing my password, which would give them access to my employee records, and my authorization capabilities. For any company that requires security, I do not accept that we can trust the DBA. The database is just one small component in a much larger solution. The DBA is the monkey for a minor backend application, and the designers are the people earning money for the corporation. We have the exact opposite of what you are suggesting. A person can get access to much more data by logging in as the user on their *desktop* than by accessing some database directly. I think you are missing that security is a balance. Your dig at ignorant people who do JS-based browser side checks of input is not applicable. You are exchanging one type of security for another type of security. You think that your proposed type of security is more valid than my proposed type of security. It depends on the application. Sometimes you might be right. Other times, you have arguably made things worse. Any company that truly needs security of this sort - should not be using PostgreSQL based roles with passwords for authentication. The true value of your proposal is pretty limited. I'm not saying don't do it. I am saying that you are not truly achieving any improvement in security for the target audience you are saying that you are representing. I think your proposal might improve things for newbies running PostgreSQL on an open Internet port at home who pick username = password. Frankly, I don't think their data is worth protecting, and their choice to use username = password and make it accessible on an open Internet port confirms that they are either completely ignorant about security, or they also agree that their data is not worth protecting. Cheers, mark -- Mark Mielkem...@mielke.cc -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Thu, Oct 15, 2009 at 7:22 PM, Mark Mielke m...@mark.mielke.cc wrote: It depends on what your goal is. If your goal is to treat users as monkeys that you do not trust, even with their own password, and the DBA as God, who you absolutely do trust, than you are correct. I don't know about your company - but in my company, the DBAs are in the IT department, and they really have no business knowing my password, which would give them access to my employee records, and my authorization capabilities. For any company that requires security, I do not accept that we can trust the DBA. The database is just one small component in a much larger solution. The DBA is the monkey for a minor backend application, and the designers are the people earning money for the corporation. We have the exact opposite of what you are suggesting. A person can get access to much more data by logging in as the user on their *desktop* than by accessing some database directly. You have no choice but to trust the DBA (or sysadmin) if you use PostgreSQL's built in authentication. Just set a pg_hba.conf line to 'password' and start harvesting passwords. I think you are missing that security is a balance. Your dig at ignorant people who do JS-based browser side checks of input is not applicable. You are exchanging one type of security for another type of security. You think that your proposed type of security is more valid than my proposed type of security. It depends on the application. Sometimes you might be right. Other times, you have arguably made things worse. Any company that truly needs security of this sort - should not be using PostgreSQL based roles with passwords for authentication. The true value of your proposal is pretty limited. I never said it wasn't - in fact I said from the outset it was about box-checking, and that anyone doing things properly will use LDAP/SSPI/Kerberos etc. What I did say, was that my proposed GUC doesn't allow the DBA to do anything that he cannot already do in a much easier way, per the comment above. Anyway, as noted in the message you quoted, the current proposal will allow my colleagues to check boxes, and will be implemented in a sensible way on the server side. And it's entirely confined to a plugin, so if you trust all your users, there's no need for you to load it at all. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] inefficient use of relation extension?
I've been spending some time debugging a customer's performance problem, and what I see is that there are a bunch of processes all waiting for the relation extension lock for a particular relation. While looking at this code I notice something that troubles me. Just after extending the relation, we don't insert the new page into the FSM. So if the extending backend does not do any other insertion on the page, it is forgotten as possible insert target until the next vacuum. Moreover, after we acquire the extension lock, we don't recheck the table to see if the lock was just released by someone who had just extended the table. So we extend the table again, even though there is an almost-empty page at the end. Both these things seem to be compounding in our customer's machine. First it was both Slony's sl_log table and its TOAST table. Then we disabled Slony because it was too far behind. Then the behavior appeared again in a regular user table. I regularly (several times a day) see five or six processes all with pg_locks locktype=extend granted=f on the same table, waiting for a long time. I am actually checking this in 8.1, but the code is almost identical in HEAD. So I am wishing for two things: 1. that the new page is registered on the FSM so that other backends can use it before the next VACUUM. 2. that if we need to wait for the lock at all, we release it and restart at the top of RelationGetBufferForTuple, so that if someone else already extended we can reuse the same page. (This would be done only once so that we don't loop forever or an insane number of times.) Thoughts? PS: now that we have the new FSM stuff, perhaps the first part is bogus; but in that case, the comment at the bottom of RelationGetBufferForTuple needs to be updated. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
Dave Page wrote: I never said it wasn't - in fact I said from the outset it was about box-checking, and that anyone doing things properly will use LDAP/SSPI/Kerberos etc. I don't understand why the box-checkers can't already check that box; with the explanation stating Yes - by using LDAP or GSSAPI or PAM configured accordingly. Or do checkbox-lists specifically say can postgres do XYZ with all OS security features disabled. Anyway, as noted in the message you quoted, the current proposal will allow my colleagues to check boxes, and will be implemented in a sensible way on the server side. And it's entirely confined to a plugin, so if you trust all your users, there's no need for you to load it at all. Note that I'm not horribly against the feature (though I wouldn't use it) --- just that ISTM we're checkbox-compliant already by working with the OS, and it's perhaps more a documentation issue than coding issue to get those boxes checked. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] inefficient use of relation extension?
Alvaro Herrera alvhe...@commandprompt.com writes: While looking at this code I notice something that troubles me. Just after extending the relation, we don't insert the new page into the FSM. So if the extending backend does not do any other insertion on the page, it is forgotten as possible insert target until the next vacuum. That is intentional so as not to have write contention on that page. We would rather have several backends concurrently inserting into different pages. Unless you've got a lot of very short-lived backends doing one insertion apiece, it seems like the right tradeoff to me. I regularly (several times a day) see five or six processes all with pg_locks locktype=extend granted=f on the same table, waiting for a long time. I'm not sure what's causing that, but I *seriously* doubt that adding new pages to FSM right away would make it better. What it sounds like is someone is getting hung up while holding the lock. You should try to investigate who's got the lock when this happens, and what they're doing or waiting for. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Trigger with WHEN clause (WIP)
Tom Lane t...@sss.pgh.pa.us writes: That argument is based on a completely evidence-free assumption, namely that this patch would make your case faster. Executing the WHEN tests is hardly going to be zero cost. It's not too hard to postulate cases where implementing a filter this way would be *slower* than doing it inside the trigger. It's pretty often the case (IME) that calling a trigger is the only point in the session where you fire plpgsql, and that's a visible cost. Last time I had to measure it, it was 1ms per call. We were trying to optimize queries running in 3ms to 4ms, called more than 100 times a second (in parallel on multi core architecture, but still). The way I understand it, having the WHEN clause in CREATE TRIGGER would allow to filter out some interpreter initialisations. Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Trigger with WHEN clause (WIP)
Dimitri Fontaine dfonta...@hi-media.com wrote: It's pretty often the case (IME) that calling a trigger is the only point in the session where you fire plpgsql, and that's a visible cost. Wouldn't a connection pool solve this? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rejecting weak passwords
On Thu, Oct 15, 2009 at 10:40 PM, Ron Mayer rm...@cheapcomplexdevices.com wrote: Dave Page wrote: I never said it wasn't - in fact I said from the outset it was about box-checking, and that anyone doing things properly will use LDAP/SSPI/Kerberos etc. I don't understand why the box-checkers can't already check that box; with the explanation stating Yes - by using LDAP or GSSAPI or PAM configured accordingly. Or do checkbox-lists specifically say can postgres do XYZ with all OS security features disabled. Well I guess that's possible, but what I hear our SEs complaining about is having to justify features that require additional software or features that other DBMSs offer natively. Let me offer a couple of examples: Q. Does the product offer password policy enforcement? Oracle: Yes SQL Server: Yes PostgreSQL: Yes (using an external authentication provider with policy enforcement). Q. Does the product support external authentication providers? Oracle: Yes SQL Server: Yes PostgreSQL: Yes Q. Does the product offer transparent data encryption? Oracle: Yes SQL Server: Yes PostgreSQL: Yes (using an encrypted filesystem). Q. Does the product offer audit capabilities? Oracle: Yes SQL Server: Yes PostgreSQL: Yes (must be manually implemented using triggers and functions) Too many of those caveats, and it's easy to see how we can be discounted early in the evaluation phase. It's not helped that often these lists will be drawn up by people used to working with the commercial DBMSs, so we probably wouldn't get extra points for having a dozen procedural languages, or other features that are largely unique to PostgreSQL, no matter how cool and useful they are. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Hot standby, recovery_starts_paused
recovery_starts_paused is useless as it is. It pauses the recovery right after the first WAL record, all right, but before we see a running-xacts record, we won't let any backends in. And if you can't connect, you can't unpause, so it's stuck forever. It should probably behave as pause after reaching WAL record that allows backends to connect. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] inefficient use of relation extension?
Tom Lane wrote: Alvaro Herrera alvhe...@commandprompt.com writes: While looking at this code I notice something that troubles me. Just after extending the relation, we don't insert the new page into the FSM. So if the extending backend does not do any other insertion on the page, it is forgotten as possible insert target until the next vacuum. That is intentional so as not to have write contention on that page. We would rather have several backends concurrently inserting into different pages. Unless you've got a lot of very short-lived backends doing one insertion apiece, it seems like the right tradeoff to me. Yes, I notice it's intentional. I can't really say if this works as intended in this case. For example I think it is possible that rd_targblock is reset when there's a relcache flush. Hmm ... this is something that had not occured to me earlier. There is a connection pool here (JDBCConnectionPool I'm told; hadn't heard about that one) and there are about 100 backends permanently, not all of which are always busy. Perhaps what's going on here is that some of them are idle for long enough that the sinval queue gets full. I regularly (several times a day) see five or six processes all with pg_locks locktype=extend granted=f on the same table, waiting for a long time. I'm not sure what's causing that, but I *seriously* doubt that adding new pages to FSM right away would make it better. What it sounds like is someone is getting hung up while holding the lock. Yeah, I tried to see what was this about; I was fooled by the bit where we fsync the segment, but AFAICS this should not happen on a regular backend. One point of interest is that PGDATA lies on a SAN, so perhaps it's just the protocol being inefficient there. You should try to investigate who's got the lock when this happens, and what they're doing or waiting for. I've been trying to research this but I can't say the tools are great. We'll keep at it. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] inefficient use of relation extension?
Alvaro Herrera alvhe...@commandprompt.com writes: Hmm ... this is something that had not occured to me earlier. There is a connection pool here (JDBCConnectionPool I'm told; hadn't heard about that one) and there are about 100 backends permanently, not all of which are always busy. Perhaps what's going on here is that some of them are idle for long enough that the sinval queue gets full. Hm, that's definitely possible, and 8.1 did not have very good code for coping with sinval overrun. But it's not clear to me why that would affect the rel extension code path in particular. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Trigger with WHEN clause (WIP)
Tom Lane t...@sss.pgh.pa.us wrote: Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp writes: I think there is a benefit to provide WHEN cluase at least for compatibility with other DBMSs, even through we can move the expressions into the body of trigger functions. This seems to me to be a lot of code to accomplish nothing useful. It will always be the case that any nontrivial logic has to be done inside the trigger. And the compatibility argument is entirely pointless given the lack of compatibility in the trigger function itself. I see that WHEN cluase is not always needed, but I think there are several benefits to have it: * WHEN cluase is in SQL standard. * We could recheck trigger conditions when NEW tuple is modified. (not yet implemented in the patch, though) http://archives.postgresql.org/pgsql-hackers/2009-09/msg00286.php * As for compatibility, it is easy for typical users to move trigger bodies into a function, but they don't like to merge the condition and the bodies into a function in my experience. * As for performance, I don't have any evidence. But there was a discussion about benefits of writing partitioning trigger function with C instead of PL/pgSQL. He said pgplsql is slow. http://archives.postgresql.org/pgsql-hackers/2008-12/msg01221.php Also, to separate trigger bodies and conditions are useful when we write triggers in C because we don't have to recomplie the code. Regards, --- ITAGAKI Takahiro NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CommitFest 2009-09, two weeks on
On Thu, Oct 15, 2009 at 2:27 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Robert Haas wrote: On Wed, Sep 30, 2009 at 12:24 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Do you have any sense of how soon you'll feel confident to commit either patch? I'm bad at estimating. Not this week for sure, and next week I'm traveling and won't be able to spend as much time on it as I am right now. If no new major issues are found, and all the outstanding issues are resolved by me or Simon by then, maybe the week after that. It's now the week after that, so unless you're in the process of typing that 'cvs commit' command, I'm going to move HS + SR out to the next CommitFest so that we can close this one out and stamp alpha2. No objections here. OK, done. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Encoding issues in console and eventlog on win32
On Mon, Oct 12, 2009 at 9:13 PM, Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp wrote: Magnus Hagander mag...@hagander.net wrote: One other question - you note that WriteConsoleW() could fail if stderr is redirected. Are you saying that it will always fail when stderr is redirected, or only sometimes? If ony sometimes, do you know under which conditions it happens? It will always fail if redirected. We can test the conditions using: pg_ctl start result.log So, the comment should be: /* WriteConsoleW always fails if stderr is redirected. */ I cleaned up the patch per comments. I hope this will be the final one ;-). * Use in_error_recursion_trouble() instead of own implementation. * Use def_enc2name() macro to avoid adding the codepage field on non-win32 platforms. * Fix a bug of calculation of result length. * Fix a memory leak on error handling path in pgwin32_toUTF16(). If it's always, I assume this just means that the logfile will be in the database encoding and not in UTF16? Is this what we want, or would we like the logfile to also be in UTF16? If we can convert it to UTF16, that would fix the case when you have different databases in different encodings, wouldn't it? (Even if your editor, unlike the console subsystem, can view the individual encoding you need, I bet it can't deal with multiple encodings in the same file) Sure, the logfile will be filled with mixed encoding strings, that could happen in logfile and syslog on non-win32 platforms. I think UTF8 is better than UTF16 for logfile encoding because there are some text editors that do not support wide characters. At any rate, the logfile encoding feature will come from another patch, that might add log_encoding variable and work on any platforms. Magnus has promised me on a stack of instant messages that he will review this soon, but as he hasn't gotten to it yet, I am moving it to the next CommitFest. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Reworks for Access Control facilities (r2363)
Tom Lane wrote: KaiGai Kohei kai...@ak.jp.nec.com writes: [ patch r2363 ] I promised I would review this today, but I just can't make myself do it in any detail. This is too large, too ugly, and it is going in a direction that I do not like or want to spend any of my time on. The overwhelming impression after a brief read-through is that the code has been hacked apart with a chainsaw and reassembled into a Frankenstein's monster --- it's alive, but man is it ugly. Code comments that refer to something above or below are still there, but the referent is no longer close enough for that to be a reasonable way of referring to it. It's impossible to follow what's going on or why, either in the shim functions or in the callers --- in the original coding there was context for the aclcheck calls, now there isn't. Quite frankly, I felt disappointed that we have to repeat these kind of design level issues again. :-( The purpose of this patch is to provide function entrypoints for the upcoming SE-PostgreSQL feature, because I got a few comments that we hesitate to put sepgsql_xxx() hooks on the main routines directly in the first commit fest. In addition, I already tried to put SE-PG hooks within pg_xxx_aclchecks() in this CF, but it was failed due to the differences in the security models. Then, we made a direction to add an abstraction layer for the purpose of access controls which can be available both of DAC and MAC. Apart from this patch, we need to consider the preferable way to host additional security models in PostgreSQL again. I don't have any confidence that this is a sane way to proceed forward. Indeed, ac_xxx_*() routines needs a large scale changes for the core. However, we didn't have any other way, if both of security model have to use common entry points. In the original design, I put sepgsqlCheckXXX() hooks which does not affect anything if disabled on the main routines, and it works well. I would like to consider why reviewers felt these hooks are (possibly) hard to maintain again. One reason was the hooks reflected individual SELinux permissions. e.g) sepgsqlCheckProcedureInstall(Oid procOid); It checks user's privilege to use a certain function as a system internal stuff which is executed on runtime without individual execution permission checks, like an implementation of conversion. However, it may need future contributors to understand the intention why the hooks were deployed here, without enough knowledge about SELinux. In other word, the hooks represented how SELinux makes its decision (method), not what SELinux make its decision on (purpose). Instead of this ac_xxx_*() routines and previous sepgsqlCheckXXX() routines, I would like to propose SE-PG hooks which reflects the purpose of security checks. e.g) sepgsql_relation_create(char *relName, Oid namespace_oid, ...); It internally compute the default security context of the new table and checks permission on the table itself and the namespace to be created on. The series of checks consists of a permission check to create a new table in totally. I think it is not a major issue whether this patch is applied, or not. What is important is to point out the right direction to host SELinux security model correctly. At the PGcon2008 keynote, Bruce talked that our road to the summit is similar to a bendy road. It means our development does not always go into the right direction, but we are certainly getting near to the summit. I've tried several approaches for more than two years, but I cannot feel we are getting near to the summit yet. At least, we need to decide where we should go on the next at the end of this commit fest. The shim layer knows everything about everything --- there may still be a few backend .h files it doesn't include, but that's not for lack of trying. The direction this is heading in is an unmaintainable giant-bowl-of-spaghetti security module, rather than something that can be divided into understandable parts. And I don't think it's really removed any complexity from the callers, nor do I believe that it's going to be a useful basis for imposing a different security policy than the one we have now. Two specific examples of why not: * The skip permissions checks arguments that have been added to various random functions suggest strongly that the factoring still isn't right --- I especially don't believe in that in the context of performDeletion and friends. The reason why we needed to put permission checks on the routines in dependency.c is that we cannot know what objects are dropped due to the cascaded deletion. But some of purely internal stuffs (such as cleaning up temporary objects) uses the routines without any necessity of permission checks. So, the flag to control permission check is necessary. * There are two special-purpose shims, ac_database_calculate_size and ac_tablespace_calculate_size, that got added for the benefit of utils/adt/dbsize.c. What if
Re: [HACKERS] Reworks for Access Control facilities (r2363)
On Thu, Oct 15, 2009 at 1:22 PM, Tom Lane t...@sss.pgh.pa.us wrote: Maybe if I weren't burned out after a month of CommitFesting, I could muster a more positive reaction, but right now I just can't summon any enthusiasm for this. Based on this review, I am marking this patch Rejected. For what it's worth, I took a quick look at this just to see if I had any reason to disagree with your conclusions. I don't. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] CommitFest 2009-09 is closed
Subject line pretty much says it all. Thanks, ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Reworks for Access Control facilities (r2363)
Robert Haas wrote: On Thu, Oct 15, 2009 at 1:22 PM, Tom Lane t...@sss.pgh.pa.us wrote: Maybe if I weren't burned out after a month of CommitFesting, I could muster a more positive reaction, but right now I just can't summon any enthusiasm for this. Based on this review, I am marking this patch Rejected. Basically, I need to agree in spite of Stephen's efforts. For what it's worth, I took a quick look at this just to see if I had any reason to disagree with your conclusions. I don't. Sorry, please make clear the your conclusions? Does it mean that Tom's comment that this reworking does not go into the right direction? Or, my comment on the last message? Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] DELETE not seeing expected rows before COPY on 8.4.1
-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160
I'm trying to figure out why I keep getting an error when trying to COPY
data into a table. The basic process is to get a list of rows via primary
keys, delete from the 'target' table, grab data from the 'source' table
using the same pks, and stuff them in to the target. However, the COPY is
failing, as a row already seems to exist. Here's the logs:
2009-10-15T20:04:29-06:00 mybox postgres[2717]: [26-1] LOG: statement: begin
2009-10-15T20:04:29-06:00 mybox postgres[2717]: [27-1] LOG: duration: 0.195 ms
2009-10-15T20:04:29-06:00 mybox postgres[2717]: [28-1] LOG: statement: SET
TRANSACTION ISOLATION LEVEL SERIALIZABLE READ WRITE
2009-10-15T20:04:29-06:00 mybox postgres[2717]: [29-1] LOG: duration: 0.128 ms
2009-10-15T20:04:29-06:00 mybox postgres[2717]: [30-1] LOG: statement: DELETE
FROM abc.sales WHERE id IN ('RVC0777','SES0443','SES0752','SES0761','SES0805')
2009-10-15T20:04:31-06:00 mybox postgres[2717]: [31-1] LOG: duration: 2155.206
ms
2009-10-15T20:04:31-06:00 mybox postgres[2717]: [32-1] LOG: statement: COPY
abc.sales FROM STDIN
2009-10-15T20:04:31-06:00 mybox postgres[2717]: [33-1] ERROR: duplicate key
value violates unique constraint sales_id
2009-10-15T20:04:31-06:00 mybox postgres[2717]: [33-2] CONTEXT: COPY sales,
line 1: SES0761#011t#011250#011f#011t
2009-10-15T20:04:31-06:00 mybox postgres[2717]: [33-3] STATEMENT: COPY
abc.sales FROM STDIN
Table sales looks like this:
id | character varying(64)
...
Indexes:
sales_id UNIQUE, btree (id)
The source call is:
COPY (SELECT * FROM abc.sales WHERE id IN
('RVC0777','SES0443','SES0752','SES0761','SES0805')) TO STDOUT;
The table on the other side is identical (has the same unique index and
columns, in the same order)
The DELETE returns 0 rows (and thus is the problem) according to the script.
This is through a Perl script using DBD::Pg (and therefore libpq). The DELETE
is run
via $dbh-do, and uses PQexec.
The rows *are* there when I manually check via psql.
A SELECT inside the script just before the DELETE also shows
no matching rows (should be five).
When run from psql, the DELETE works as expected.
I've verified that only one row with SES0761 is being returned from the source.
I created a second unique index on the source and target table with no errors.
Version is 8.4.1
- --
Greg Sabino Mullane g...@turnstep.com
End Point Corporation
PGP Key: 0x14964AC8 200910152337
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-BEGIN PGP SIGNATURE-
iEYEAREDAAYFAkrX68wACgkQvJuQZxSWSsgBGQCg7B4CJX5lZ2WMpNBgWv1Jcyvf
ficAoOXZn1SCkMueqlPDLqh7ErF+p6GA
=dnFa
-END PGP SIGNATURE-
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby, recovery_starts_paused
On Fri, 2009-10-16 at 01:43 +0300, Heikki Linnakangas wrote: recovery_starts_paused is useless as it is. It pauses the recovery right after the first WAL record, all right, but before we see a running-xacts record, we won't let any backends in. And if you can't connect, you can't unpause, so it's stuck forever. Hmmm, strange. That was an earlier bug; I thought I'd fixed that. It should probably behave as pause after reaching WAL record that allows backends to connect. Agreed, thanks. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
