Re: [HACKERS] Reverse-sort indexes and NULLS FIRST/LAST sorting
On Mon, Jan 01, 2007 at 05:53:35PM -0500, Tom Lane wrote: The SQL2003 spec adds optional NULLS FIRST and NULLS LAST modifiers for ORDER BY clauses. Teodor proposed an implementation here: http://archives.postgresql.org/pgsql-patches/2006-12/msg00019.php which I didn't care for at all: http://archives.postgresql.org/pgsql-hackers/2006-12/msg00133.php snip One way we could handle this is to say that reverse-sort indexes are implemented by adding explicit catalog entries for reverse-sort opclasses, with no additions to the underlying btree index mechanisms. So you might make an index using a command like CREATE INDEX fooi ON foo (x, y reverse_int4_ops); Personally I favour this approach. It's also the approach similar to what I did with the COLLATE stuff. It's IMHO the cleanest because it encapsulates the order at the level where it's important. In particular, NULLS FIRST/LAST makes sense for btree, but no other index type, so storing the order seperatly is wasted space for any other index type. But in a sense this doesn't go far enough. In general a column can be ordered four ways, and like you say later, it doesn't allow mixed NULLS FIRST/LAST orderins. The other way that seems like it could win acceptance is to make REVERSE an explicit optional property of an index column; and if we do that we might as well allow NULLS FIRST/LAST to be an optional property as well. Then you could say something like CREATE INDEX fooi ON foo (x, y REVERSE NULLS FIRST); While the syntax is nice, I think this method of implementation is a bad idea. Like I said it's wasted processing for non-btree index types. Issues which you havn't addressed are: - Pathkeys: How is the forward/reverse/nulls first/last going to be encoded in the pathkey? I don't think the current method (using the operator OID) is going to stretch far enough. But that leaves you with deciding whether to keep support for SORT_LT/GTFUNC? - How do you deal with people asking for NULLS FIRST/LAST which is the opposite of how the index is defined. Say you can't use the index? Comments? I've got mixed feelings about which way to jump myself. Somehow neither is quite satisfying. My COLLATE patch solved it by adding an extra layer on top of the operator classes to encode the ordering nulls first/last, but I don't think we really want that. One totally whacked out idea is to allowed the btree code to call the operator to decide nulls first/last, that would allow you to factor that part out at least. Have a nice day, -- Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/ From each according to his ability. To each according to his ability to litigate. signature.asc Description: Digital signature
Re: [HACKERS] Status of Fix Domain Casting TODO
On Mon, Jan 01, 2007 at 06:30:40PM -0600, Andrew Dunstan wrote: Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: FWIW, I'm running into this trying to create a 'raw' domain that would automagically convert hex strings into actual binary data for storage in a bytea. I think you've got 0 chance of implementing that as a domain rather than an independent type. Without or without revisions in the casting rules, a domain has not got its own I/O functions, and never will. This might be less of an issue if we allowed such IO functions to be written in a loadable PL rather than in C. I'm confused... couldn't I just write a cast function? Or is that what's meant by I/O functions? And yes, in this case I should be able to accomplish what I'm looking for just using encode() and decode(). -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Reverse-sort indexes and NULLS FIRST/LAST sorting
I'd like to see this implemented with more general collation support in mind. In general, each index column can be ordered by one collation. A query matching the index collation can use the index directly, a query asking for another collation needs to convert. The trivial way to convert from one collation to another is to sort according to the new collation. For other conversions, there can be cheaper ways. I'd like to have explicit support for collations and converting between them, perhaps by introducing a new ConvertCollation node type. The default implementation would be to sort, but for example to convert from NULLS FIRST to NULLS LAST could be implemented by buffering the null columns to temporary storage and returning all non-null rows first. There's similar tricks that could be used to convert between many Western European collations, without resorting to full sort. The NULLS FIRST/LAST support, as well as ascending and descending orderings would be special cases of the general collation and collation conversion machinery. I don't know how much work that would be to implement, compared to what you're proposing. It would require adding an extra collation concept to all the places that care about sort ordering, but you're proposing to add nulls-first-or-last flag to all those places anyway. Tom Lane wrote: The SQL2003 spec adds optional NULLS FIRST and NULLS LAST modifiers for ORDER BY clauses. Teodor proposed an implementation here: http://archives.postgresql.org/pgsql-patches/2006-12/msg00019.php which I didn't care for at all: http://archives.postgresql.org/pgsql-hackers/2006-12/msg00133.php Doing this right is going to require introducing the nulls-first-or-last concept into all the system's handling of sort ordering. Messy as that sounds, I think it will end up logically cleaner than what we have now, because it will let us fix some issues involving descending-order index opclasses and backwards-sort mergejoins. Neither of those can really work correctly right now, the reason being exactly that we lack a framework for dealing with variable sort positioning of NULLs. I'm hoping to fix this as a consequence of the work I'm doing with operator families for 8.3. What I'd like to come out of it is support for both NULLS FIRST/LAST and reverse-sort index columns. Reverse-sort indexes are already in the TODO list, the application being to create an index whose sort order matches a query like ORDER BY x ASC, y DESC. There are some user-visible decisions to be made first, so this message is to start a discussion about what we want. One way we could handle this is to say that reverse-sort indexes are implemented by adding explicit catalog entries for reverse-sort opclasses, with no additions to the underlying btree index mechanisms. So you might make an index using a command like CREATE INDEX fooi ON foo (x, y reverse_int4_ops); btree indexes would always sort by the given opclass with NULLS LAST. So the two possible orderings that could be derived from this index (using forward or backward scan respectively) are ORDER BY x ASC NULLS LAST, y DESC NULLS LAST ORDER BY x DESC NULLS FIRST, y ASC NULLS FIRST The other way that seems like it could win acceptance is to make REVERSE an explicit optional property of an index column; and if we do that we might as well allow NULLS FIRST/LAST to be an optional property as well. Then you could say something like CREATE INDEX fooi ON foo (x, y REVERSE NULLS FIRST); (Or maybe use DESC instead of REVERSE as the keyword --- not very important at this point.) This index would support scans with these two sort orderings: ORDER BY x ASC NULLS LAST, y DESC NULLS FIRST ORDER BY x DESC NULLS FIRST, y ASC NULLS LAST This second way is more flexible in that it allows indexes to support mixed null orderings; another attraction is that we'd not have to create explicit reverse-sort opclasses, which would be a tedious bit of extra work for every datatype. On the other hand, adding these extra flag bits to indexes seems like a horribly ugly wart, mainly because they're irrelevant to anything except a btree index. (Or at least irrelevant to anything that doesn't support ordered scans, but in practice that's only btree for the foreseeable future.) Also, having to account for these options in the btree code would make it more complex and perhaps slower. Comments? I've got mixed feelings about which way to jump myself. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Loose ends in PG XML patch
Am Sonntag, 24. Dezember 2006 02:44 schrieb Tom Lane: * Isn't mapping XMLSERIALIZE to a cast completely wrong? Aside from the issue already noted in the code that it won't reverse-list correctly, this loses the DOCUMENT-vs-CONTENT flag, which I suppose must be important. It is important, but at the moment it's not so important as to not provide any standards-conforming method to obtain a character string from XML at all. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [PATCHES] xlog directory at initdb time
Am Mittwoch, 27. Dezember 2006 02:56 schrieb Euler Taveira de Oliveira: This simple patch lets someone specifies the xlog directory at initdb time. It uses symlinks to do it, and create and/or set permissions at the directory as appropriate. We already had this functionality in initdb a few versions ago. Did you review why it was removed? -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] effective_cache_size vs units
Am Donnerstag, 28. Dezember 2006 13:25 schrieb Jim C. Nasby: Yes, and I can't think of a single reason why we'd let people specify anything in millibytes, or kilobits. How about a configuration option related to connection throughput, which is typically measured in bits? -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] TODO: Add a GUC to control whether BEGIN inside
Am Donnerstag, 28. Dezember 2006 19:52 schrieb Tom Lane: Not only is it overzealous, but the proposal to have one reflects a failure to learn from history. GUC variables that change transaction-boundary semantics are a bad idea, period: see autocommit. But this option would not, in fact, change the transaction-boundary semantics. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] TODO: Add a GUC to control whether BEGIN inside
Am Donnerstag, 28. Dezember 2006 18:57 schrieb Bruce Momjian: I think you can make the case that this should be an error, or at least that's how it got on the TODO list. I can always remove it if people don't want the item completed. The reason this was added is that modular applications expect that a locally issued BEGIN ... COMMIT executes a transaction, whereas now you don't know what you're getting. I think this change would have merit. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] float8 width_bucket function
Jeremy Drake said: http://momjian.us/mhonarc/patches_hold/msg00162.html There is no patch or anything associated with it, just the suggestion that it be put in when 8.3 devel starts up. Right -- this is on my TODO list for 8.3. I'm traveling at the moment, but I can send a patch for this in a week or two. -Neil ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Reverse-sort indexes and NULLS FIRST/LAST sorting
Martijn van Oosterhout kleptog@svana.org writes: Issues which you havn't addressed are: - Pathkeys: How is the forward/reverse/nulls first/last going to be encoded in the pathkey? I'm envisioning a struct with operator OID and null-ordering flag. If we implement the explicit REVERSE variant then we'd have to be prepared to match the OID against either the LessThan or GreaterThan member of an index opclass depending --- it wouldn't add a third field to pathkeys. But that leaves you with deciding whether to keep support for SORT_LT/GTFUNC? I'm kind of inclined to drop the LTFUNC/GTFUNC business and insist that every operator used as a sort operator must be a btree opclass member. But that decision seems entirely orthogonal to the rest of it. - How do you deal with people asking for NULLS FIRST/LAST which is the opposite of how the index is defined. Say you can't use the index? That's right, you can't. Just like any other ordering incompatibility. One totally whacked out idea is to allowed the btree code to call the operator to decide nulls first/last, that would allow you to factor that part out at least. This doesn't work at all unless you make all the operators non-strict, which I hardly think we want. Also, that path leads to needing FOUR opclasses per datatype to support all the orderings :-( regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Reverse-sort indexes and NULLS FIRST/LAST sorting
Heikki Linnakangas [EMAIL PROTECTED] writes: I'd like to see this implemented with more general collation support in mind. I'm really not prepared to buy into that, simply because it puts ICU or some equivalent large chunk of new code into the critical path to finish what I'm doing. The fact that pathkeys will become structs will probably make it easier to add collation later (adding another field to the struct won't mean a wholesale notational change), but that doesn't mean I have to do it now. The NULLS FIRST/LAST support, as well as ascending and descending orderings would be special cases of the general collation and collation conversion machinery. That seems like a bad idea, because nulls first/last and asc/desc ordering are valid concepts for all btree-indexable datatypes, whereas collation is only meaningful for text. Besides, that approach just moves the bloat over from too-many-opclasses to too-many-collations; do we really want to need four collation objects for each basic collation? regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] TODO: Add a GUC to control whether BEGIN inside
Peter Eisentraut [EMAIL PROTECTED] writes: Am Donnerstag, 28. Dezember 2006 18:57 schrieb Bruce Momjian: I think you can make the case that this should be an error, or at least that's how it got on the TODO list. I can always remove it if people don't want the item completed. The reason this was added is that modular applications expect that a locally issued BEGIN ... COMMIT executes a transaction, whereas now you don't know what you're getting. I think this change would have merit. But how is making BEGIN an error going to improve life for such an application? It'll be just as broken. In fact, if the app depends on getting an error from BEGIN in any critical way, it'll be *more* broken if it's ever run under the default warning-only setting. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Status of Fix Domain Casting TODO
Jim C. Nasby wrote: On Mon, Jan 01, 2007 at 06:30:40PM -0600, Andrew Dunstan wrote: Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: FWIW, I'm running into this trying to create a 'raw' domain that would automagically convert hex strings into actual binary data for storage in a bytea. I think you've got 0 chance of implementing that as a domain rather than an independent type. Without or without revisions in the casting rules, a domain has not got its own I/O functions, and never will. This might be less of an issue if we allowed such IO functions to be written in a loadable PL rather than in C. I'm confused... couldn't I just write a cast function? Or is that what's meant by I/O functions? And yes, in this case I should be able to accomplish what I'm looking for just using encode() and decode(). The I/O functions are set up by the INPUT and OUTPUT params of the CREATE TYPE statement. They convert to and from the type 'cstring'. If you want to change the way a piece of data is read/produced (e.g. automatically encode/decode the value) these are what you would need. A domain is in effect a constrained type. But it inherits the I/O functions of its base type. But constraints are not what you want - you want to deal with representation, which is the property dealt with by I/O functions - their fundamental purpose is to convert between external and internal representation. HTH cheers andrew ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Reverse-sort indexes and NULLS FIRST/LAST sorting
Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: I'd like to see this implemented with more general collation support in mind. I'm really not prepared to buy into that, simply because it puts ICU or some equivalent large chunk of new code into the critical path to finish what I'm doing. ... Yeah, I didn't mean doing that right now. Just to keep it in mind so that what we do now fits in nicely with it in the future. The NULLS FIRST/LAST support, as well as ascending and descending orderings would be special cases of the general collation and collation conversion machinery. That seems like a bad idea, because nulls first/last and asc/desc ordering are valid concepts for all btree-indexable datatypes, whereas collation is only meaningful for text. Besides, that approach just moves the bloat over from too-many-opclasses to too-many-collations; do we really want to need four collation objects for each basic collation? Hmm, I guess we don't. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] [PATCHES] xlog directory at initdb time
Peter Eisentraut [EMAIL PROTECTED] writes: Am Mittwoch, 27. Dezember 2006 02:56 schrieb Euler Taveira de Oliveira: This simple patch lets someone specifies the xlog directory at initdb time. It uses symlinks to do it, and create and/or set permissions at the directory as appropriate. We already had this functionality in initdb a few versions ago. Did you review why it was removed? The discussion thread seems to start here: http://archives.postgresql.org/pgsql-hackers/2002-08/msg00306.php As best I can tell the objections came from the fact that Thomas had implemented it as a postmaster-start-time switch, which made it a foot-gun because you could mistakenly start the postmaster with a different XLOG than you were using before. That would not apply to a symlink-made-by-initdb approach. All this is doing is formalizing something we already suggest people do by hand... regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] effective_cache_size vs units
Peter Eisentraut [EMAIL PROTECTED] writes: Am Donnerstag, 28. Dezember 2006 13:25 schrieb Jim C. Nasby: Yes, and I can't think of a single reason why we'd let people specify anything in millibytes, or kilobits. How about a configuration option related to connection throughput, which is typically measured in bits? But at least as often in bytes. What's more, if the system really were to accept both units, you could reasonably expect that people would get it wrong at least half the time ... regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] TODO: Add a GUC to control whether BEGIN inside
Tom Lane wrote: Peter Eisentraut [EMAIL PROTECTED] writes: Am Donnerstag, 28. Dezember 2006 18:57 schrieb Bruce Momjian: I think you can make the case that this should be an error, or at least that's how it got on the TODO list. I can always remove it if people don't want the item completed. The reason this was added is that modular applications expect that a locally issued BEGIN ... COMMIT executes a transaction, whereas now you don't know what you're getting. I think this change would have merit. But how is making BEGIN an error going to improve life for such an application? It'll be just as broken. In fact, if the app depends on getting an error from BEGIN in any critical way, it'll be *more* broken if it's ever run under the default warning-only setting. While we are on the topic, I have implemented a poor mans nested transaction feature into my database access layer. essentially subsequent calls to begin a transaction after the initial begin simply increase an internal counter and set a savepoint. as you commit the transactions the counter is decreased and the savepoints are released. maybe this could be implemented inside postgresql to make the life of modular programmers easier? regards, Lukas ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] TODO: Add a GUC to control whether BEGIN inside
news.postgresql.org wrote: While we are on the topic, I have implemented a poor mans nested transaction feature into my database access layer. essentially subsequent calls to begin a transaction after the initial begin simply increase an internal counter and set a savepoint. as you commit the transactions the counter is decreased and the savepoints are released. maybe this could be implemented inside postgresql to make the life of modular programmers easier? Yeah, it's called SAVEPOINT foo and RELEASE foo. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] TODO: Add a GUC to control whether BEGIN inside
Alvaro Herrera wrote: news.postgresql.org wrote: While we are on the topic, I have implemented a poor mans nested transaction feature into my database access layer. essentially subsequent calls to begin a transaction after the initial begin simply increase an internal counter and set a savepoint. as you commit the transactions the counter is decreased and the savepoints are released. maybe this could be implemented inside postgresql to make the life of modular programmers easier? Yeah, it's called SAVEPOINT foo and RELEASE foo. Err, I think you misunderstood what I said. My implementation uses SAVEPOINTs already. The point is having some API where you do not have to care of you are already in a transaction or not. Depending on if you are it will either open a new transaction or simply place a savepoint. with the following invented commands: MBEGIN FOO1 // open transaction; set counter to 1 MBEGIN FOO2 // set savepoint FOO2; set counter to 2 MBEGIN FOO3 // set savepoint FOO3; set counter to 3 MROLLBACK FOO3 // rollback to FOO3; set counter to 2 MCOMMIT FOO2 // release FOO2; set counter to 1 MCOMMIT FOO1 // commit regards, Lukas regards, Lukas ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] TODO: Add a GUC to control whether BEGIN inside
On Tue, 2007-01-02 at 11:53 +0100, Peter Eisentraut wrote: Am Donnerstag, 28. Dezember 2006 18:57 schrieb Bruce Momjian: I think you can make the case that this should be an error, or at least that's how it got on the TODO list. I can always remove it if people don't want the item completed. The reason this was added is that modular applications expect that a locally issued BEGIN ... COMMIT executes a transaction, whereas now you don't know what you're getting. I think this change would have merit. Interesting point. My only comment is, Do it one way or the other, don't give me a user or a distribution packager a foot gun. E.g., no GUC parameter. Just change the behavior or don't. Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] TODO: Add a GUC to control whether BEGIN inside
Lukas Kahwe Smith [EMAIL PROTECTED] writes: Err, I think you misunderstood what I said. My implementation uses SAVEPOINTs already. The point is having some API where you do not have to care of you are already in a transaction or not. It's not that hard, is it? if (PQtransactionStatus(conn) == PQTRANS_IDLE) PQexec(conn, BEGIN); else PQexec(conn, SAVEPOINT foo); regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Sync Scan update
On Sat, 2006-12-30 at 13:35 -0600, Jim C. Nasby wrote: My current implementation relies on the scans to stay close together once they start close together. If one falls seriously behind, it will fall outside of the main cache trail and cause the performance to degrade due to disk seeking and lower cache efficiency. That's something else that it would be really good to have data for; in some cases it will be better for the slow case to just fall behind, but in other cases the added seeking will slow everything down enough that it would have been faster to just stay at the speed of the slow scan. The question is where those two thresholds are... Right. I will do more testing for my basic patch soon, but a lot of testing is required to characterize when the scans should move apart and when they should stay together. The problem is that there are a lot of variables. If you have a few scans that uses a moderate amount of CPU, the scans might all stay together (I/0 bound). But as soon as you get more scans, those scans could all become CPU bound (and could be mixed with other types of scans on the same table). If you have some ideas for tests I can run I'll get back to you with the results. However, this kind of test would probably need to be run on a variety of hardware. Regards, Jeff Davis ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] TODO: Add a GUC to control whether BEGIN inside
On 1/2/07, Joshua D. Drake [EMAIL PROTECTED] wrote: E.g., no GUC parameter. Just change the behavior or don't. Please refer the conversation beginning at: http://archives.postgresql.org/pgsql-hackers/2006-05/msg00249.php That is where this TODO item came from. In the conversation, it was understood that such a change would break many applications, hence one of the option was to introduce a GUC var and keep it on by default, and a couple of releases later, remove the GUC and make the behaviour default ( http://archives.postgresql.org/pgsql-hackers/2006-05/msg00273.php). This would help the programmers can realize that they are doing something wrong, and they can make appropriate changes to their code. The usability of the GUC comes in a production environment, where it is not possible to change the application. The DBA can buy some time by turning the GUC var off. I submitted a patch, which was incorrect and incomplete as I was _very_ new to PGSQL. I could not follow up on it as I was switching jobs at the time. Tom objected to the default=ON setting for the GUC. The TODO has been declared misconceived, but I guess there's still interest out here. Would like to finish it once we reach a consensus. Best regards, -- [EMAIL PROTECTED] [EMAIL PROTECTED] gmail | hotmail | yahoo }.com
Re: [HACKERS] TODO: GNU TLS
Stephen Frost wrote: * David Boreham ([EMAIL PROTECTED]) wrote: Fascinating thread for the holidays. I found it interesting that nobody has mentioned NSS (former Netscape SSL library). It has its own bag of problems of course, but for me is potentially more attractive than GNU TLS. e.g. it has FIPS-140 certification and is actively under development by a software company with significant resources. It's also very widely deployed. I'm not saying that OpenSSL is bad (it'd probably be my first choice), just that there is another option besides GNU TLS. Not sure what license that's under, From http://www.mozilla.org/projects/security/pki/nss/: 'NSS is available under the Mozilla Public License, the GNU General Public License, and the GNU Lesser General Public License.' ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] TODO: GNU TLS
* David Boreham ([EMAIL PROTECTED]) wrote: Stephen Frost wrote: Not sure what license that's under, From http://www.mozilla.org/projects/security/pki/nss/: 'NSS is available under the Mozilla Public License, the GNU General Public License, and the GNU Lesser General Public License.' Works for me then, and it's already packaged in Debian. The only downside that I can see is that the work isn't done yet and if we want to support both OpenSSL and NSS then the patch will be at least somewhat invasive/large (since I doubt NSS's API is anything like OpenSSL's, please correct me if I'm wrong). Would a patch to implement dual-support for OpenSSL and NSS be acceptable? Would just replacing OpenSSL support with NSS support be better? Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] Rare corruption of pg_class index
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Tom Lane wrote: Oh! Duh, that's your issue right there, I'll bet. The problem is that relcache-open tries to read the pg_class row under SnapshotNow rules, and if there is another xact concurrently modifying the row, it is entirely possible for none of the row versions to be committed good at the instant they are visited. (The new row version either isn't seen at all or isn't committed good yet when it's visited, and later when the old row version is visited, it has become committed dead.) This results in ScanPgRelation failing (returning NULL) which leads to exactly the could not open relation with OID xxx symptom --- and in fact I see no other code path that yields that failure. Doesn't this violate ACID, or am I misunderstanding something? (FWIW, I'm using a serializable isolation level for the process that changes pg_class) As of 8.2 we have this problem fixed for system-initiated changes to the pg_class row, but you're still going to be at risk if you are doing manual UPDATE pg_class operations. Can you get away from needing to do that? ALTER TABLE DISABLE TRIGGER might help, but we haven't got anything like ALTER TABLE DISABLE RULE. Yeah, triggers alone won't do it. Sounds like a TODO item - Bruce? In any case the important point is that you have to take AccessExclusive lock on a relation whose pg_class row you would like to change, and you need to be on 8.2 because prior releases weren't careful about obtaining lock *before* reading the row. Obtaining an AccessExclusive lock is a last resort, as the tables in question are very busy. That's another reason why DISABLE TRIGGER might not work out either. - -- Greg Sabino Mullane [EMAIL PROTECTED] End Point Corporation PGP Key: 0x14964AC8 200701021325 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iD8DBQFFmqU9vJuQZxSWSsgRAgZ4AJ4wBUI6APz658zaE8bFQ5xmILFiugCgxfsW GI2zgdF6l/tmxWpnO4J9dms= =NtIn -END PGP SIGNATURE- ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] TODO: GNU TLS
David Boreham wrote: Stephen Frost wrote: * David Boreham ([EMAIL PROTECTED]) wrote: Fascinating thread for the holidays. I found it interesting that nobody has mentioned NSS (former Netscape SSL library). It has its own bag of problems of course, but for me is potentially more attractive than GNU TLS. e.g. it has FIPS-140 certification and is actively under development by a software company with significant resources. It's also very widely deployed. I'm not saying that OpenSSL is bad (it'd probably be my first choice), just that there is another option besides GNU TLS. Not sure what license that's under, From http://www.mozilla.org/projects/security/pki/nss/: 'NSS is available under the Mozilla Public License, the GNU General Public License, and the GNU Lesser General Public License.' I suspect most postgres developers and companies would like to keep things as BSDish as possible. Dealing with a multitude of licenses might be fun for some, but many of us find it a pain in the neck. Also, do we really want to import the NSPR into Postgres? I suspect not. Of course, the only thing that people are tripping over license-wise is libpq. But I think we would want to keep that as lean and mean as possible, too. cheers andrew ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] 8.2 Crash on Query
I have been able to crash the database consistently on a Fedora Core 5 machine running postgresql 8.2.0. The attached files are an example database (crash.shema) and the query that is used (crash.sql) as well as the log output from turning on all the debugging (crash.log). I have a couple of other queries that do this as well, but this is the first one that I noticed. This database schema and query works fine version 8.1.4. -- //\\ || D. Hagemandhageman@dracken.com || \\//LOG: 0: database system was shut down at 2007-01-02 12:16:33 CST LOCATION: StartupXLOG, xlog.c:4672 LOG: 0: checkpoint record is at 0/130E42C LOCATION: StartupXLOG, xlog.c:4762 LOG: 0: redo record is at 0/130E42C; undo record is at 0/0; shutdown TRUE LOCATION: StartupXLOG, xlog.c:4789 LOG: 0: next transaction ID: 0/59568; next OID: 32768 LOCATION: StartupXLOG, xlog.c:4793 LOG: 0: next MultiXactId: 1; next MultiXactOffset: 0 LOCATION: StartupXLOG, xlog.c:4796 LOG: 0: database system is ready LOCATION: StartupXLOG, xlog.c:5188 DEBUG: 0: transaction ID wrap limit is 2147484171, limited by database postgres LOCATION: SetTransactionIdLimit, varsup.c:278 DEBUG: 0: proc_exit(0) LOCATION: proc_exit, ipc.c:94 DEBUG: 0: shmem_exit(0) LOCATION: shmem_exit, ipc.c:125 DEBUG: 0: exit(0) LOCATION: proc_exit, ipc.c:112 DEBUG: 0: reaping dead processes LOCATION: reaper, postmaster.c:2010 DEBUG: 0: forked new backend, pid=32688 socket=7 LOCATION: BackendStartup, postmaster.c:2573 DEBUG: 0: Ident protocol identifies remote user as postgres LOCATION: authident, hba.c:1615 DEBUG: 0: postmaster child[32688]: starting with ( LOCATION: BackendRun, postmaster.c:2917 DEBUG: 0: postgres LOCATION: BackendRun, postmaster.c:2920 DEBUG: 0: -v196608 LOCATION: BackendRun, postmaster.c:2920 DEBUG: 0: -y LOCATION: BackendRun, postmaster.c:2920 DEBUG: 0: test LOCATION: BackendRun, postmaster.c:2920 DEBUG: 0: ) LOCATION: BackendRun, postmaster.c:2922 DEBUG: 0: InitPostgres LOCATION: PostgresMain, postgres.c:3137 DEBUG: 0: StartTransaction LOCATION: ShowTransactionState, xact.c:3985 DEBUG: 0: name: unnamed; blockState: DEFAULT; state: INPROGR, xid/subid/cid: 59568/1/0, nestlvl: 1, children: LOCATION: ShowTransactionStateRec, xact.c:4010 DEBUG: 0: CommitTransaction LOCATION: ShowTransactionState, xact.c:3985 DEBUG: 0: name: unnamed; blockState: STARTED; state: INPROGR, xid/subid/cid: 59568/1/0, nestlvl: 1, children: LOCATION: ShowTransactionStateRec, xact.c:4010 DEBUG: 0: StartTransactionCommand LOCATION: start_xact_command, postgres.c:2200 STATEMENT: SELECT f.id, f.category_id, f.status, f.moderated, f.topics, f.posts, f.forum, f.last_post, f.system_name, f.description, p.last_post_date, p.last_post_time, p.topic FROM crash.forum f LEFT JOIN crash.forum_post p ON ( f.last_post = p.id ) WHERE ( f.status = 1 ) ORDER BY f.forum, f.id LIMIT ALL OFFSET 0; DEBUG: 0: StartTransaction LOCATION: ShowTransactionState, xact.c:3985 STATEMENT: SELECT f.id, f.category_id, f.status, f.moderated, f.topics, f.posts, f.forum, f.last_post, f.system_name, f.description, p.last_post_date, p.last_post_time, p.topic FROM crash.forum f LEFT JOIN crash.forum_post p ON ( f.last_post = p.id ) WHERE ( f.status = 1 ) ORDER BY f.forum, f.id LIMIT ALL OFFSET 0; DEBUG: 0: name: unnamed; blockState: DEFAULT; state: INPROGR, xid/subid/cid: 59569/1/0, nestlvl: 1, children: LOCATION: ShowTransactionStateRec, xact.c:4010 STATEMENT: SELECT f.id, f.category_id, f.status, f.moderated, f.topics, f.posts, f.forum, f.last_post, f.system_name, f.description, p.last_post_date, p.last_post_time, p.topic FROM crash.forum f LEFT JOIN crash.forum_post p ON ( f.last_post = p.id ) WHERE ( f.status = 1 ) ORDER BY f.forum, f.id LIMIT ALL OFFSET 0; LOG: 0: statement: SELECT f.id, f.category_id, f.status, f.moderated, f.topics, f.posts, f.forum, f.last_post, f.system_name, f.description, p.last_post_date, p.last_post_time, p.topic FROM crash.forum f LEFT JOIN crash.forum_post p ON ( f.last_post = p.id ) WHERE ( f.status = 1 ) ORDER BY f.forum, f.id LIMIT ALL OFFSET 0; LOCATION: exec_simple_query, postgres.c:811 STATEMENT: SELECT f.id, f.category_id, f.status, f.moderated, f.topics, f.posts, f.forum, f.last_post, f.system_name, f.description, p.last_post_date, p.last_post_time, p.topic FROM crash.forum f LEFT JOIN crash.forum_post p ON ( f.last_post = p.id ) WHERE ( f.status = 1 ) ORDER BY f.forum, f.id LIMIT ALL OFFSET 0; DEBUG: 0: reaping dead processes LOCATION: reaper, postmaster.c:2010 DEBUG: 0: server process (PID 32688) was terminated by signal 11 LOCATION: LogChildExit, postmaster.c:2425 LOG: 0: server process (PID 32688) was terminated by
Re: [HACKERS] TODO: GNU TLS
Stephen Frost wrote: * David Boreham ([EMAIL PROTECTED]) wrote: Stephen Frost wrote: Not sure what license that's under, From http://www.mozilla.org/projects/security/pki/nss/: 'NSS is available under the Mozilla Public License, the GNU General Public License, and the GNU Lesser General Public License.' Works for me then, and it's already packaged in Debian. The only downside that I can see is that the work isn't done yet and if we want to support both OpenSSL and NSS then the patch will be at least somewhat invasive/large (since I doubt NSS's API is anything like OpenSSL's, please correct me if I'm wrong). Unfortunately the NSS and OpenSSL I/O design is quite different. There has been talk over the years (since at least 1996) of adding OpenSSL-like interfaces to NSS, but AFAIK this has never been done. NSS presents a 'layered' I/O model where the application talks to a socket-like API. It also depends on NSPR. For these reasons I would hesitate to recommend it for use in a server vs. OpenSSL. ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] TODO: GNU TLS
Andrew Dunstan [EMAIL PROTECTED] writes: Also, do we really want to import the NSPR into Postgres? I suspect not. Of course, the only thing that people are tripping over license-wise is libpq. But I think we would want to keep that as lean and mean as possible, too. Yeah, requiring NSPR to be imported into all client applications that use libpq is an utter non-starter :-( regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] [PATCHES] xlog directory at initdb time
Tom Lane wrote: Peter Eisentraut [EMAIL PROTECTED] writes: Am Mittwoch, 27. Dezember 2006 02:56 schrieb Euler Taveira de Oliveira: This simple patch lets someone specifies the xlog directory at initdb time. It uses symlinks to do it, and create and/or set permissions at the directory as appropriate. We already had this functionality in initdb a few versions ago. Did you review why it was removed? The discussion thread seems to start here: http://archives.postgresql.org/pgsql-hackers/2002-08/msg00306.php As best I can tell the objections came from the fact that Thomas had implemented it as a postmaster-start-time switch, which made it a foot-gun because you could mistakenly start the postmaster with a different XLOG than you were using before. That would not apply to a symlink-made-by-initdb approach. All this is doing is formalizing something we already suggest people do by hand... Right. Thomas decided he didn't want to adjust the patch based on community input, so the patch was removed. It was one of the few cases where I had to back out someone else's patch against their will. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] TODO: Add a GUC to control whether BEGIN inside
Tom Lane wrote: Lukas Kahwe Smith [EMAIL PROTECTED] writes: Err, I think you misunderstood what I said. My implementation uses SAVEPOINTs already. The point is having some API where you do not have to care of you are already in a transaction or not. It's not that hard, is it? if (PQtransactionStatus(conn) == PQTRANS_IDLE) PQexec(conn, BEGIN); else PQexec(conn, SAVEPOINT foo); Its not exactly convenient either, especially in the case of modular code that may be developed by different people. Anyways, like I said I have a solution in my framework to make life of module developers easier. Obviously proper nested transactions would be the ideal, but so it goes. I was just throwing this out here when I saw Peter's comment. regards, Lukas ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] TODO: GNU TLS
Andrew Dunstan wrote: I suspect most postgres developers and companies would like to keep things as BSDish as possible. Right, hence OpenSSL would be the obvious best choice. In respect of licencing however, NSS is no 'worse' than GNU TLS because it may be distributed under the GPL and LGPL. ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Rare corruption of pg_class index
Greg Sabino Mullane [EMAIL PROTECTED] writes: Tom Lane wrote: Oh! Duh, that's your issue right there, I'll bet. The problem is that relcache-open tries to read the pg_class row under SnapshotNow rules, and if there is another xact concurrently modifying the row, it is entirely possible for none of the row versions to be committed good at the instant they are visited. Doesn't this violate ACID, or am I misunderstanding something? No, it's not a violation of ACID. In this case what you are doing is altering a table's schema without a sufficiently strong lock on the table, and that's a no-no, whether you would like it to be or not. In any case the important point is that you have to take AccessExclusive lock on a relation whose pg_class row you would like to change, and you need to be on 8.2 because prior releases weren't careful about obtaining lock *before* reading the row. Obtaining an AccessExclusive lock is a last resort, as the tables in question are very busy. That's another reason why DISABLE TRIGGER might not work out either. Well, ENABLE/DISABLE TRIGGER can't conceivably operate correctly without locking out writes, because it wouldn't be clear whether any particular write operation should fire the trigger or not. A hypothetical ENABLE/DISABLE RULE would be worse: AFAICS it'd have to lock out reads too, else it wouldn't be clear whether SELECTs should notice an ON SELECT rule. You can't get around those restrictions by trying to implement the enable/disable yourself via UPDATE pg_class; as you've found out, it just doesn't work. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] 8.2 Crash on Query
D. Hageman dhageman@dracken.com writes: I have been able to crash the database consistently on a Fedora Core 5 machine running postgresql 8.2.0. Yeah, LIMIT ALL is known broken in 8.2.0 :-(. It's been fixed for awhile, will be in 8.2.1. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] TODO: GNU TLS
* Andrew Dunstan ([EMAIL PROTECTED]) wrote: I suspect most postgres developers and companies would like to keep things as BSDish as possible. Dealing with a multitude of licenses might be fun for some, but many of us find it a pain in the neck. It'd be great if PostgreSQL could use an SSL library with the same license as PostgreSQL itself has. That'd certainly work for me. Unfortunately, I'm not sure one exists (if anyone knows of one, please mention it...). I don't like having to deal with lots of licenses either but it's pretty much a fact of life in today's OSS world. I hope you don't think I've gotten any enjoyment out of this, it's just a very frustrating quagmire that I have to deal with. Also, do we really want to import the NSPR into Postgres? I suspect not. Of course, the only thing that people are tripping over license-wise is libpq. But I think we would want to keep that as lean and mean as possible, too. erm, I'm not really sure what you're saying here but perhaps I can clarify: I wasn't suggesting to add any serious amount of source code to PostgreSQL - NSS would be used just as OpenSSL is today, and as GNUTLS support was proposed, a seperate library which is distributed independently of PostgreSQL but can be compiled against. I don't know about the memory footprint of NSS, though if we care about that terribly much it's my understanding that GNUTLS has a smaller footprint than OpenSSL... While somehow changing libpq to remove the issue it's unfortunately not the only case. There are GPL'd PostgreSQL server extensions (specifically PostGIS, at least) which are also affected. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] TODO: GNU TLS
* David Boreham ([EMAIL PROTECTED]) wrote: Andrew Dunstan wrote: I suspect most postgres developers and companies would like to keep things as BSDish as possible. Right, hence OpenSSL would be the obvious best choice. In respect of licencing however, NSS is no 'worse' than GNU TLS because it may be distributed under the GPL and LGPL. And the MPL, which at least according to the Mozilla FAQ falls somewhere between the GPL and BSD (though I'm not sure I'd agree...). Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] TODO: GNU TLS
Stephen Frost wrote: Also, do we really want to import the NSPR into Postgres? I suspect not. Of course, the only thing that people are tripping over license-wise is libpq. But I think we would want to keep that as lean and mean as possible, too. erm, I'm not really sure what you're saying here but perhaps I can clarify: I wasn't suggesting to add any serious amount of source code to PostgreSQL - NSS would be used just as OpenSSL is today, and as GNUTLS support was proposed, a seperate library which is distributed independently of PostgreSQL but can be compiled against. I don't know I suspect that Andrew was concerned about the dependency NSS has on NSPR. NSS dates back to the days before universal support for threads and mutexes. NSPR was (is) a library designed to abstract platform differences in those areas, and to provide its own implementations where none was available in the OS (e.g. old MacOS, 16-bit Windows). So for example if you want to open an SSL connection using NSS you get to hand it an NSPR socket handle, not an OS socket (however, there are functions that allow construction of one from the other). For an application that has otherwise no need for cross-platform service abstraction, or that has already solved the problems NSPR solves in a different way, NSPR looks like a big ball of goo that you don't need. NSS can't exist in an application without NSPR. Having said that, except in the case of cooperative threading environments (do those exist today??), there's no requirement on the application to actually use NSPR for anything other than plumbing underneath NSS. Applications that want to handle their own I/O underneath the SSL library (particularly useful in servers) will often not be happy with NSS due to its layering above NSPR I/O. ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] [PATCHES] xlog directory at initdb time
On Jan 2, 2007, at 7:18 AM, Tom Lane wrote: Peter Eisentraut [EMAIL PROTECTED] writes: Am Mittwoch, 27. Dezember 2006 02:56 schrieb Euler Taveira de Oliveira: This simple patch lets someone specifies the xlog directory at initdb time. It uses symlinks to do it, and create and/or set permissions at the directory as appropriate. We already had this functionality in initdb a few versions ago. Did you review why it was removed? The discussion thread seems to start here: http://archives.postgresql.org/pgsql-hackers/2002-08/msg00306.php As best I can tell the objections came from the fact that Thomas had implemented it as a postmaster-start-time switch, which made it a foot-gun because you could mistakenly start the postmaster with a different XLOG than you were using before. That would not apply to a symlink-made-by-initdb approach. All this is doing is formalizing something we already suggest people do by hand... I guess the downside there is that it won't work on platforms that don't support symlinks, whereas the postmaster switch would. Not that I condone using such platforms ;^) -Casey ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] TODO: GNU TLS
* David Boreham ([EMAIL PROTECTED]) wrote: Stephen Frost wrote: erm, I'm not really sure what you're saying here but perhaps I can clarify: I wasn't suggesting to add any serious amount of source code to PostgreSQL - NSS would be used just as OpenSSL is today, and as GNUTLS support was proposed, a seperate library which is distributed independently of PostgreSQL but can be compiled against. I don't know I suspect that Andrew was concerned about the dependency NSS has on NSPR. [...] Ah, this does sound rather ugly and not something we'd want. The particular library doesn't make a whole heck of alot of difference to me provided it has the general functionality necessary and a compatible license (where 'compatible' in this case really means 'Debian feels it is compatible with the GPL'). It'd be wonderful if OpenSSL's license was the same license PostgreSQL has. Honestly, we'd be happy to stop pissing off both those who license their code under the GPL (by asking for exceptions for OpenSSL) and core library maintainers (by asking for GNUTLS support, though in general I like to have options). Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] TODO: GNU TLS
Stephen Frost wrote: -- Start of PGP signed section. * David Boreham ([EMAIL PROTECTED]) wrote: Stephen Frost wrote: erm, I'm not really sure what you're saying here but perhaps I can clarify: I wasn't suggesting to add any serious amount of source code to PostgreSQL - NSS would be used just as OpenSSL is today, and as GNUTLS support was proposed, a seperate library which is distributed independently of PostgreSQL but can be compiled against. I don't know I suspect that Andrew was concerned about the dependency NSS has on NSPR. [...] Ah, this does sound rather ugly and not something we'd want. The particular library doesn't make a whole heck of alot of difference to me provided it has the general functionality necessary and a compatible license (where 'compatible' in this case really means 'Debian feels it is compatible with the GPL'). It'd be wonderful if OpenSSL's license was the same license PostgreSQL has. Honestly, we'd be happy to stop pissing off both those who license their code under the GPL (by asking for exceptions for OpenSSL) and core library maintainers (by asking for GNUTLS support, though in general I like to have options). Keep in mind in most cases OpenSSL is already part of the operating system, unless you are using Win32. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] TODO: GNU TLS
On Tue, Jan 02, 2007 at 01:29:35PM -0500, Stephen Frost wrote: Would a patch to implement dual-support for OpenSSL and NSS be acceptable? Would just replacing OpenSSL support with NSS support be When I was looking into this I looked at NSS, and eventually decided on GnuTLS. Why? Because I read the GnuTLS documentation and I understood it. The basic support for GnuTLS took a whole afternoon, the hard work was leving people with the choice of using OpenSSL. I read the OpenSSL docs too, but I still don't understand how it works properly. IMHO, GnuTLS has the advantage if being designed later which means details like: - Thread safety (GnuTLS is thread-safe by design, no locks needed) - Proper layering (creating your own I/O function is trivial) - Seperate namespace - Non-blocking support from the get-go were taken care of. Since people are citing maintainability as a concern, I think you really have wonder whether NSS is a better choice. Have a nice day, -- Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/ From each according to his ability. To each according to his ability to litigate. signature.asc Description: Digital signature
Re: [HACKERS] TODO: GNU TLS
* Bruce Momjian ([EMAIL PROTECTED]) wrote: Stephen Frost wrote: Ah, this does sound rather ugly and not something we'd want. The particular library doesn't make a whole heck of alot of difference to me provided it has the general functionality necessary and a compatible license (where 'compatible' in this case really means 'Debian feels it is compatible with the GPL'). It'd be wonderful if OpenSSL's license was the same license PostgreSQL has. Honestly, we'd be happy to stop pissing off both those who license their code under the GPL (by asking for exceptions for OpenSSL) and core library maintainers (by asking for GNUTLS support, though in general I like to have options). Keep in mind in most cases OpenSSL is already part of the operating system, unless you are using Win32. Debian, at least, doesn't consider it as such (wouldn't really make much sense since you could claim everything in Debian is part of the OS). That's my understanding anyway, and goes for pretty much everything in Debian. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] TODO: GNU TLS
Bruce Momjian wrote: Keep in mind in most cases OpenSSL is already part of the operating system, unless you are using Win32. My understanding is that the Debian people are saying the exception for libraries shipped with the OS does NOT apply to *other* libraries or programs that are shipped with the OS and linked to that library. (No, it doesn't make much sense to me either) cheers andrew ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] TODO: GNU TLS
* Andrew Dunstan ([EMAIL PROTECTED]) wrote: Bruce Momjian wrote: Keep in mind in most cases OpenSSL is already part of the operating system, unless you are using Win32. My understanding is that the Debian people are saying the exception for libraries shipped with the OS does NOT apply to *other* libraries or programs that are shipped with the OS and linked to that library. This is kind of the flip-side of what I just said but is also correct, in it's own way. In the end Debian doesn't feel the shipped-with-the-OS exception in the GPL can apply. (No, it doesn't make much sense to me either) Well, the GPL does say: --- with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. --- The 'unless that component itself accompanies the executable' bit would be the problem for distributors who ship the whole thing as one OS. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] [PATCHES] xlog directory at initdb time
Casey Duncan [EMAIL PROTECTED] writes: I guess the downside there is that it won't work on platforms that don't support symlinks, whereas the postmaster switch would. Not that I condone using such platforms ;^) Well, we already bit that bullet with respect to tablespaces, and haven't gotten much of any pushback. So I don't see it as a problem here. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] [PATCHES] xlog directory at initdb time
Tom Lane wrote: Casey Duncan [EMAIL PROTECTED] writes: I guess the downside there is that it won't work on platforms that don't support symlinks, whereas the postmaster switch would. Not that I condone using such platforms ;^) Well, we already bit that bullet with respect to tablespaces, and haven't gotten much of any pushback. So I don't see it as a problem here. Note that if Casey is thinking that Win32 does not support symlinks, he is wrong (we do support tablespaces there). -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. ---(end of broadcast)--- TIP 6: explain analyze is your friend
[HACKERS] Upcoming back-branch releases
It seems to be about time to put out 8.2.1, since the flow of new bug reports has ebbed. We're also overdue for updates of the back branches --- for instance, we recently realized that the current 8.0.x and 8.1.x releases don't know about the Canadian DST changes coming into effect in March, and there are various potential-index-corruption and suchlike bug fixes in CVS too. So the core committee has agreed to schedule a set of releases this Monday, Jan 8. We'll be wrapping the tarballs Thursday or Friday to give the RPM and Windows-installer people time to do their thing, so if you've got any last-minute patches, send 'em in ASAP ... regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] TODO: GNU TLS
Martijn van Oosterhout wrote: - Thread safety (GnuTLS is thread-safe by design, no locks needed) - Proper layering (creating your own I/O function is trivial) - Seperate namespace - Non-blocking support from the get-go were taken care of. Since people are citing maintainability as a concern, I think you really have wonder whether NSS is a better choice. Well...IMO NSS has some things that GNU TLS does not (correct me if wrong on this, since my knowledge of GNU TLS is not extensive): 1. Very widely deployed, hence high level of confidence in its interoperability, higher level of trust by the crypto community. 2. Backed by several large commercial organizations, hence has support for new-fangled ciphers (elliptic curve ciphers for example, Suite B, etc) and also hardware crypto accelerators and hard tokens. 3. Used in a popular web browser, hence subject to a reasonably high level of effort to find and fix security bugs. 4. FIPS-140 certified. Used widely by US gubment. 5. Much work done over the years on crypto performance. BTW NSS is also thread-safe, has layering (perhaps not the kind of layering that everyone needs though) and supports non-blocking sockets. NSS and NSPR functions are sensibly prefixed so naming collisions should not occur. Note that I'm not pushing NSS for PG - my choice would be OpenSSL. Just presenting some info for balance, since I happen to know a something about NSS. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] SearchSysCache
My program (indirectly) calls the following function twice, tuple = SearchSysCache(STATRELATT, ObjectIdGetDatum(relid), Int16GetDatum(colnum), 0, 0); The first time it assigns NULL to tuple, while the second time it assigns a valid pointer. Why is it like that? BTW, my program only optimize query plan without executing it. Thanks. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] SearchSysCache
uwcssa [EMAIL PROTECTED] writes: My program (indirectly) calls the following function twice, tuple = SearchSysCache(STATRELATT, ObjectIdGetDatum(relid), Int16GetDatum(colnum), 0, 0); The first time it assigns NULL to tuple, while the second time it assigns a valid pointer. Why is it like that? You did an ANALYZE in between, perhaps? That's the only operation that puts new rows in pg_statistic. regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] WITH support
On 12/30/06, Mark Cave-Ayland [EMAIL PROTECTED] wrote: In short, if people don't mind waiting for my free cycles to come along then I will continue to chip away at it; otherwise if it's considered an essential for 8.3 with an April deadline then I will happily hand over to Jonah. I'd say it's probably essential given my last, shall we say, mishap. So I guess I'll start up discussion on the design again soon. I'm leaving in a couple days to travel up to NJ so expect discussion to start again on Tuesday of next week. As always, I too wouldn't have a problem if someone else is going to see it through for 8.3. -- Jonah H. Harris, Software Architect | phone: 732.331.1324 EnterpriseDB Corporation| fax: 732.331.1301 33 Wood Ave S, 3rd Floor| [EMAIL PROTECTED] Iselin, New Jersey 08830| http://www.enterprisedb.com/ ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] [PATCHES] Patch to log usage of temporary files
Bill Moran [EMAIL PROTECTED] writes:
In response to Alvaro Herrera [EMAIL PROTECTED]:
Please change things to save the stat() syscall when the feature is not
in use.
Do you have a suggestion on how to do that and still have the PG_TRACE1()
work? That was specifically requested by Simon Riggs.
Well, we are NOT paying a stat() call on every single file close,
whether Simon wants it or not. PG_TRACE1 doesn't even do anything
on non-Solaris platforms, for pete's sake.
Perhaps it would be reasonable to define trace_temp_files as the minimum
file size to log; then you could do something like
if (trace_temp_files 0)
{
if (stat(vfdP-fileName, filestats) 0)
elog(LOG, ...);
else
{
if (filestats.st_size / BLCKSZ = trace_temp_files)
ereport(LOG, ...);
PG_TRACE1(temp__file__cleanup, filestats.st_size);
}
}
Note that elog(ERROR) is quite inappropriate here.
regards, tom lane
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Sync Scan update
On Tue, Jan 02, 2007 at 09:48:22AM -0800, Jeff Davis wrote: On Sat, 2006-12-30 at 13:35 -0600, Jim C. Nasby wrote: My current implementation relies on the scans to stay close together once they start close together. If one falls seriously behind, it will fall outside of the main cache trail and cause the performance to degrade due to disk seeking and lower cache efficiency. That's something else that it would be really good to have data for; in some cases it will be better for the slow case to just fall behind, but in other cases the added seeking will slow everything down enough that it would have been faster to just stay at the speed of the slow scan. The question is where those two thresholds are... Right. I will do more testing for my basic patch soon, but a lot of testing is required to characterize when the scans should move apart and when they should stay together. The problem is that there are a lot of variables. If you have a few scans that uses a moderate amount of CPU, the scans might all stay together (I/0 bound). But as soon as you get more scans, those scans could all become CPU bound (and could be mixed with other types of scans on the same table). If you have some ideas for tests I can run I'll get back to you with the results. However, this kind of test would probably need to be run on a variety of hardware. Well, that's the real trick: ideally, syncscan would be designed in such a way that you wouldn't have to manually tune at what point scans should diverge instead of converge; the system should just figure it out. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Loose ends in PG XML patch
* Shouldn't the xml type support binary I/O? Right now it is the only standard datatype that doesn't. I have no idea whether there is an appropriate representation besides text, but if not we could define the binary representation to be the same as text. There is an effort to develop a binary xml format: http://www.w3.org/TR/xbc-characterization/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] contrib regression failures after recent money type changes
Seems that the contrib regression tests, namely the cash and oid tests of the btree_gist contrib module, are failing after the recent commit to widen the money type to 64 bits. Example: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=mongoosedt=2007-01-03%2005:30:01 Also, on a slightly off-topic note, I am subscribed to the -committers list and did not receive an automated commit message for this commit. It is also not in the archives... -- Experience is what causes a person to make new mistakes instead of old ones. ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] contrib regression failures after recent money type
On Tue, 2 Jan 2007, Jeremy Drake wrote: Seems that the contrib regression tests, namely the cash and oid tests of the btree_gist contrib module, are failing after the recent commit to widen the money type to 64 bits. Example: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=mongoosedt=2007-01-03%2005:30:01 Almost forgot, I saw that this commit modified src/include/catalog/pg_type.h, should this have required a catversion change? Also, on a slightly off-topic note, I am subscribed to the -committers list and did not receive an automated commit message for this commit. It is also not in the archives... -- Whom the gods wish to destroy they first call promising. ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] contrib regression failures after recent money type changes
Jeremy Drake [EMAIL PROTECTED] writes: Seems that the contrib regression tests, namely the cash and oid tests of the btree_gist contrib module, are failing after the recent commit to widen the money type to 64 bits. Between D'Arcy and Bruce, there is not *any* buildfarm member passing tonight, but hey it's only early devel cycle .. Also, on a slightly off-topic note, I am subscribed to the -committers list and did not receive an automated commit message for this commit. I saw no commit from D'Arcy in the list either, but the CVS server shows he did. Marc, something wrong with list permissions? regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
