Re: Does anything dump per-database config settings? (was Re: [HACKERS] ALTER DATABASE vs pg_dump)
Tom Lane wrote: Richard Huxton [EMAIL PROTECTED] writes: Tom Lane wrote: So put forward a worked-out proposal for some other behavior. IMHO the time a dump/restore should be issuing ALTER...SET on a database is when it has issued the corresponding CREATE DATABASE. So pg_dump would produce this info when, and only when, you'd used --create? I agree that it seems sensible in that case, I'm just wondering if that will cover all the use-cases. Well, in the -Fc case you'd produce it always and pg_restore would only emit it when you --create. The only time we need to restore per-database settings is if the database has been dropped. If you're not having the dump/restore re-create the database then presumably you've taken charge of the per-database settings. This would mean duplicating some functionality between pg_dump and pg_dumpall ... or maybe we could move all that logic over to pg_dump and have pg_dumpall use --create when invoking pg_dump. -- Richard Huxton Archonet Ltd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Planned obsolescence in identify_system_timezone()
Tom Lane [EMAIL PROTECTED] writes: I just noticed that identify_system_timezone() tries to match our timezones to the system timezone on the basis of probing the date range 1904-2004. While doubtless this seemed like a good idea at the time, it means that we'll be unable to distinguish zones whose histories diverge after 2004. I think there are some already, and certainly there are likely to be some in future. I propose modifying this logic to test 100 years back from the current year, instead. Any objections? Should such a change be back-patched? Seems obvious in retrospect. Or perhaps it should just always go back to about 1900 which is when a lot of places adopted standard timezones. I'm also more than slightly tempted to modify the code so that it will always reject a match to the Factory zone (and fall back to regular GMT instead). The logging behavior illustrated here: http://archives.postgresql.org/pgsql-bugs/2008-06/msg00191.php seems just completely bletcherous to me. It's not our business to nag the user into setting up /etc/localtime, and even if it were, this is obviously an ineffective way to do it ;-) Any objections to that one? If we're going to do it at all it should be once at startup (or config file read or some event like that). But I think I agree that it's just not our place at all and just defaulting to GMT is the right option. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's On-Demand Production Tuning -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bucket and batch
Tom Lane [EMAIL PROTECTED] writes: Ana Carolina Brito de Almeida [EMAIL PROTECTED] writes: So, I would like to know the differences between bucket and batch. A bucket is, well, one bucket of a hash table --- it holds all the tuples that have the same hash code (for as many bits of the hash code as we are choosing to use). We try to size the hash table with enough buckets so there's not more than 10 tuples per bucket on average. A batch is a range of buckets that we process at the same time. Note that we don't currently do batches for hash aggregates, only joins currently. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's On-Demand Production Tuning -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Explain XML patch
I'm reading through the Explain XML patch. Just a couple of quick observations: There are a few whitespace oddities which like: switch (nodeTag(plan)) { case T_IndexScan: + { + StringInfo index; + index = makeStringInfo(); + appendStringInfo(index, name=\%s\, explain_get_index_name(((IndexScan *) plan)-indexid)); I suspect this is actually introduced by the mail archiving software. Something to beware of, if you have locally archived copies of the emails it's probably better to use those to extract patches :( Also, this change is bogus: {without, WITHOUT, UNRESERVED_KEYWORD}, {work, WORK, UNRESERVED_KEYWORD}, {write, WRITE, UNRESERVED_KEYWORD}, ! {xml, XML_P, UNRESERVED_KEYWORD}, {xmlattributes, XMLATTRIBUTES, COL_NAME_KEYWORD}, {xmlconcat, XMLCONCAT, COL_NAME_KEYWORD}, {xmlelement, XMLELEMENT, COL_NAME_KEYWORD}, --- 415,422 {without, WITHOUT, UNRESERVED_KEYWORD}, {work, WORK, UNRESERVED_KEYWORD}, {write, WRITE, UNRESERVED_KEYWORD}, ! {xml, XML, UNRESERVED_KEYWORD}, ! {xmlp, XML_P, UNRESERVED_KEYWORD}, {xmlattributes, XMLATTRIBUTES, COL_NAME_KEYWORD}, {xmlconcat, XMLCONCAT, COL_NAME_KEYWORD}, {xmlelement, XMLELEMENT, COL_NAME_KEYWORD}, Just use the existing XML_P keyword. The _P just stands for parser to disambiguate it (I'm not sure a bison keyword could conflict with anything... but anyways...). -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's 24x7 Postgres support! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP patch: reducing overhead for repeat de-TOASTing
Tom Lane wrote: Attached is a worked-out patch for the approach proposed here: http://archives.postgresql.org/pgsql-hackers/2008-06/msg00777.php namely, that cache management for de-TOASTed datums is handled by TupleTableSlots. To avoid premature detoasting of values that we might never need, the patch introduces a concept of an indirect TOAST pointer, which has the same 0x80 or 0x01 header as an external TOAST pointer, but can be told apart by having a different length byte. Within that we have * pointer to original toasted field within the Slot's tuple * pointer to the owning Slot * pointer to decompressed copy, or NULL if not decompressed yet Some fairly straightforward extensions to the TupleTableSlot code, heaptuple.c, and tuptoaster.c make it all go. My original thoughts had included turning FREE_IF_COPY() into a no-op, but on investigation that seems impractical. One case that still depends on that pfree is where we have palloc'd a 4-byte-header copy of a short-header datum to support code that needs properly aligned datum content. The solution adopted in the patch is to arrange for pfree() applied to a cacheable detoasted object to be a no-op, whereas it still works normally for non-cached detoasted objects. We do this by inserting a dummy chunk header that points to a dummy memory context whose pfree support method does nothing. I think this part of the patch would be required for any toast caching method, not just this one. What I like about this patch is that it's a fairly small-footprint change, it doesn't add much overhead, and it covers caching of decompression for in-line-compressed datums as well as the out-of-line case. One thing I really *don't* like about it is that it requires everyplace that copies Datums to know about indirect pointers: in general, the copy must be a copy of the original toasted Datum, not of the indirect pointer, else we have indirect pointers that can outlive their owning TupleTableSlot (or at least outlive its current tuple cycle). There only seem to be about half a dozen such places in the current code, but still it seems a rather fragile coding rule. After playing with it for a little bit, I'm not convinced that it buys enough performance win to be worth applying --- the restriction of cache lifespan to one tuple cycle of a TupleTableSlot is awfully restrictive. (For example, sorts that involve toasted sort keys continue to suck, because the tuples being sorted aren't in Slots.) It would probably fix the specific case that the PostGIS hackers were complaining of, but I think we need something more. Still, I wanted to get it into the archives because the idea of indirect toast pointers might be useful for something else. Hi Tom, Thanks very much for supplying the WIP patch. In the interest of testing and feedback, I've just downloaded PostgreSQL CVS head and applied your patch, compiled up a slightly modified version of PostGIS (without RECHECKs on the GiST opclass) and loaded in the same dataset. Unfortunately I have to report back that with your WIP patch applied, timings seem to have become several orders of magnitude *worse*: [EMAIL PROTECTED]:~$ psql -d postgis psql (8.4devel) Type help for help. postgis=# explain analyze select count(*) from geography where centroid (select the_geom from geography where id=69495); QUERY PLAN -- Aggregate (cost=7100.28..7100.29 rows=1 width=0) (actual time=18238.932..18238.934 rows=1 loops=1) InitPlan - Seq Scan on geography (cost=0.00..7092.00 rows=1 width=4387) (actual time=27.472..69.223 rows=1 loops=1) Filter: (id = 69495::numeric) - Index Scan using geography_centroid_idx on geography (cost=0.00..8.27 rows=1 width=0) (actual time=118.371..18196.041 rows=32880 loops=1) Index Cond: (centroid $0) Total runtime: 18239.918 ms (7 rows) In fact, even sequential scans seem to have gone up by several orders of magnitude: postgis=# set enable_indexscan = 'f'; SET postgis=# set enable_bitmapscan = 'f'; SET postgis=# explain analyze select count(*) from geography where centroid (select the_geom from geography where id=69495); QUERY PLAN Aggregate (cost=14184.01..14184.01 rows=1 width=0) (actual time=9117.022..9117.024 rows=1 loops=1) InitPlan - Seq Scan on geography (cost=0.00..7092.00 rows=1 width=4387) (actual time=23.780..54.362 rows=1 loops=1) Filter: (id = 69495::numeric) - Seq Scan on geography (cost=0.00..7092.00 rows=1 width=0) (actual time=55.016..9073.084 rows=32880 loops=1) Filter: (centroid $0) Total runtime: 9117.174 ms (7 rows) ATB, Mark. -- Mark
Re: [HACKERS] WIP patch: reducing overhead for repeat de-TOASTing
Mark, On Tue, Jul 1, 2008 at 1:15 PM, Mark Cave-Ayland [EMAIL PROTECTED] wrote: Thanks very much for supplying the WIP patch. In the interest of testing and feedback, I've just downloaded PostgreSQL CVS head and applied your patch, compiled up a slightly modified version of PostGIS (without RECHECKs on the GiST opclass) and loaded in the same dataset. From the discussion we had a few months ago, I don't think the no RECHECK trick works with CVS tip anymore. See my post on the Remove lossy-operator RECHECK flag? thread: http://archives.postgresql.org/pgsql-hackers/2008-04/msg00847.php and follow-ups. That said, perhaps it's not the only problem you have but I thought it was worth mentioning. -- Guillaume -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: Does anything dump per-database config settings? (was Re: [HACKERS] ALTER DATABASE vs pg_dump)
On Tuesday 01 July 2008 03:45:44 Richard Huxton wrote: Tom Lane wrote: Richard Huxton [EMAIL PROTECTED] writes: Tom Lane wrote: So put forward a worked-out proposal for some other behavior. IMHO the time a dump/restore should be issuing ALTER...SET on a database is when it has issued the corresponding CREATE DATABASE. So pg_dump would produce this info when, and only when, you'd used --create? I agree that it seems sensible in that case, I'm just wondering if that will cover all the use-cases. Well, in the -Fc case you'd produce it always and pg_restore would only emit it when you --create. The only time we need to restore per-database settings is if the database has been dropped. If you're not having the dump/restore re-create the database then presumably you've taken charge of the per-database settings. I'm not sure I agree with that entirely. For example, one common usage scenario when upgrading between major versions is to create the database, load contrib modules (whose C functions or similar may have changed), and then load the dump into the database. In those case you still might want the database settings to be dumped, even though you are creating the database manually. (Now, one might argue that you could still dump with --create and ignore the error of the database creation command, but that probably isn't ideal). -- Robert Treat Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TODO assignment
Jiri Dvorak napsal(a): Hello I wish to ask, whether is somebody working on this TODO point: Look on this: http://archives.postgresql.org/pgsql-hackers/2008-04/msg00049.php --- GRANT/REVOKE * Allow column-level privileges * %Allow GRANT/REVOKE permissions to be applied to all schema objects with one command The proposed syntax is: GRANT SELECT ON ALL TABLES IN public TO phpuser; GRANT SELECT ON NEW TABLES IN public TO phpuser; * Allow GRANT/REVOKE permissions to be inherited by objects based on schema permissions * Allow SERIAL sequences to inherit permissions from the base table? --- Or more generally, is there a list of what is who doing? You can found partial information here: http://wiki.postgresql.org/wiki/Development_information Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP patch: reducing overhead for repeat de-TOASTing
Guillaume Smet wrote: From the discussion we had a few months ago, I don't think the no RECHECK trick works with CVS tip anymore. See my post on the Remove lossy-operator RECHECK flag? thread: http://archives.postgresql.org/pgsql-hackers/2008-04/msg00847.php and follow-ups. That said, perhaps it's not the only problem you have but I thought it was worth mentioning. Yeah sorry, that might not have been as clear as I hoped. What I meant was that I removed the explicit RECHECK clause from the GiST operator class definition - since as you correctly mention, CVS tip throws an error if this is present. ATB, Mark. -- Mark Cave-Ayland Sirius Corporation - The Open Source Experts http://www.siriusit.co.uk T: +44 870 608 0063 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Dept of ugly hacks: eliminating padding space in system indexes
Tom Lane napsal(a): Cutting a third off the size of a system index has got to be worth something, but is it worth a hack as ugly as this one? The problem what I see there is how to fit with in-place-upgrade. Catalog should be generate from scratch, but if somebody uses name in regular table it invokes request for reindex. Zdenek -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http://sun.com/postgresql -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP patch: reducing overhead for repeat de-TOASTing
Mark Cave-Ayland [EMAIL PROTECTED] writes: Unfortunately I have to report back that with your WIP patch applied, timings seem to have become several orders of magnitude *worse*: Ugh. Could I pester you to run the case under gprof or oprofile? Or, if you can give me step-by-step directions for setting up the test scenario, I could do that here. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Dept of ugly hacks: eliminating padding space in system indexes
Zdenek Kotala [EMAIL PROTECTED] writes: Tom Lane napsal(a): Cutting a third off the size of a system index has got to be worth something, but is it worth a hack as ugly as this one? The problem what I see there is how to fit with in-place-upgrade. Catalog should be generate from scratch, but if somebody uses name in regular table it invokes request for reindex. Actually, an existing index stored as name would continue to work fine, so I think this could be worked around. But in any case name is deprecated for user use, so anyone who suffers a reindex has only themselves to blame. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusing message in log file
On Mon, Jun 30, 2008 at 7:33 PM, Tom Lane [EMAIL PROTECTED] wrote: Alvaro Herrera [EMAIL PROTECTED] writes: In any case, this seems a case of stuffing too much in the primary message. Yeah, good point. I think it should be something like errmsg(parameter \shared_buffer\ change in configuration file ignored), errdetail(This parameter cannot be changed after server start.) Seems a bit awkwardly phrased. I propose errmsg(attempted change to parameter \shared_buffer\ ignored), errdetail(This parameter cannot be changed after server start.) (or possibly of would read better than to) 'of' sounds better than 'to'. Best regards, -- [EMAIL PROTECTED] [EMAIL PROTECTED] gmail | hotmail | indiatimes | yahoo }.com EnterpriseDB http://www.enterprisedb.com Mail sent from my BlackLaptop device
Re: [HACKERS] PATCH: CITEXT 2.0
David E. Wheeler napsal(a): Howdy, Howdy, Please find attached a patch adding a locale-aware, case-insensitive text type, called citext, as a contrib module. A few notes: What is benefit to have this type when collation per database will be implemented? It seems to me that its overlapped feature? Definition of collation should allow to setup case sensitivity. Only advantage what I see there is that you can combine case sensitive and case insensitive text in one database. However, it will be solved when collation per column will be implemented. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Dept of ugly hacks: eliminating padding space in system indexes
Tom Lane napsal(a): Zdenek Kotala [EMAIL PROTECTED] writes: Tom Lane napsal(a): Cutting a third off the size of a system index has got to be worth something, but is it worth a hack as ugly as this one? The problem what I see there is how to fit with in-place-upgrade. Catalog should be generate from scratch, but if somebody uses name in regular table it invokes request for reindex. Actually, an existing index stored as name would continue to work fine, so I think this could be worked around. But in any case name is deprecated for user use, so anyone who suffers a reindex has only themselves to blame. Yes, it is deprecated by you know a user. Give him a loaded shot-gun and he start play a golf :-). However, reindex is acceptable penalty for user who uses deprecated things. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] A new take on the foot-gun meme
On Monday 30 June 2008 00:57:03 Tom Lane wrote: [ after recovering from choking... ] Tom spot Callaway presents a vivid new image in this line: What you're doing is analogous to using a loaded shotgun as a golf club, and what you're suggesting is that we take the safety off, because it interferes with your golf game. https://www.redhat.com/archives/fedora-devel-list/2008-June/msg01501.html So is that a golf club gun? Careful what you wish for http://www.totallyabsurd.com/12gaugegolfclub.htm -- Robert Treat Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PATCH: CITEXT 2.0
Zdenek Kotala [EMAIL PROTECTED] writes: However, it will be solved when collation per column will be implemented. Well, yeah, but that seems a very long way off. In the meantime a lot of people use the existing pgfoundry citext module. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Latest on CITEXT 2.0
On 6/26/08, Tom Lane [EMAIL PROTECTED] wrote: David E. Wheeler [EMAIL PROTECTED] writes: Datum citext_ne (PG_FUNCTION_ARGS) { // Fast path for different-length inputs. Okay for canonical equivalence? if (VARSIZE(PG_GETARG_TEXT_P(0)) != VARSIZE(PG_GETARG_TEXT_P(1))) PG_RETURN_BOOL( 1 ); PG_RETURN_BOOL( citextcmp( PG_ARGS ) != 0 ); } BTW, I don't think you can use that same-length optimization for citext. There's no reason to think that upper/lowercase pairs will have the same length all the time in multibyte encodings. What about this code in current str_tolower(): /* Output workspace cannot have more codes than input bytes */ workspace = (wchar_t *) palloc((nbytes + 1) * sizeof(wchar_t)); Bug? -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Latest on CITEXT 2.0
Marko Kreen [EMAIL PROTECTED] writes: On 6/26/08, Tom Lane [EMAIL PROTECTED] wrote: BTW, I don't think you can use that same-length optimization for citext. There's no reason to think that upper/lowercase pairs will have the same length all the time in multibyte encodings. What about this code in current str_tolower(): /* Output workspace cannot have more codes than input bytes */ workspace = (wchar_t *) palloc((nbytes + 1) * sizeof(wchar_t)); That's working with wchars, not bytes. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [WIP] patch - Collation at database level
Hello, I'm sending part of the code that I've done and is available for reviewing and I'm asking for your comments and some help because I'm new to PostgreSQL. Proposal: http://archives.postgresql.org/pgsql-hackers/2008-05/msg00857.php The code contains changes of parser, pg_collation catalog, parsenodes.h for CREATE COLLATION... and DROP COLLATION statements. Questions that I would like to know answers: 1) Should I move DROP statement from DropCollationStmt to DropStmt? Some statements are seperated and some are included in DropStmt? Is there any reason for that? 2) What type should all names in CREATE and DROP statement in gram.y have? I've chosen qualified_name but I know it's not the best choice. 3) All collations are created from existing collations. How do I ensure that the collation already exists? Is there any possibility to define it in gram.y? 4) For further functionality development is there anything more needed than adding T_CreateCollationStmt and T_DropCollationStmt to /src/backend/tcop/utility.c, and write functionality into collation.c? 5) Also can you look at the pg_catalog and tell me if anything is wrong with it? Thank you for all your replies in advance. Regards Radek Strnad Index: backend/parser/keywords.c === RCS file: /projects/cvsroot/pgsql/src/backend/parser/keywords.c,v retrieving revision 1.194 diff -c -r1.194 keywords.c *** backend/parser/keywords.c 1 Jan 2008 19:45:50 - 1.194 --- backend/parser/keywords.c 1 Jul 2008 14:29:26 - *** *** 33,38 --- 33,39 /* name, value, category */ {abort, ABORT_P, UNRESERVED_KEYWORD}, {absolute, ABSOLUTE_P, UNRESERVED_KEYWORD}, + {accent, ACCENT, UNRESERVED_KEYWORD}, {access, ACCESS, UNRESERVED_KEYWORD}, {action, ACTION, UNRESERVED_KEYWORD}, {add, ADD_P, UNRESERVED_KEYWORD}, *** *** 81,87 {close, CLOSE, UNRESERVED_KEYWORD}, {cluster, CLUSTER, UNRESERVED_KEYWORD}, {coalesce, COALESCE, COL_NAME_KEYWORD}, ! {collate, COLLATE, RESERVED_KEYWORD}, {column, COLUMN, RESERVED_KEYWORD}, {comment, COMMENT, UNRESERVED_KEYWORD}, {commit, COMMIT, UNRESERVED_KEYWORD}, --- 82,89 {close, CLOSE, UNRESERVED_KEYWORD}, {cluster, CLUSTER, UNRESERVED_KEYWORD}, {coalesce, COALESCE, COL_NAME_KEYWORD}, ! {collate, COLLATE, UNRESERVED_KEYWORD}, ! {collation, COLLATION, UNRESERVED_KEYWORD}, {column, COLUMN, RESERVED_KEYWORD}, {comment, COMMENT, UNRESERVED_KEYWORD}, {commit, COMMIT, UNRESERVED_KEYWORD}, *** *** 206,211 --- 208,215 {language, LANGUAGE, UNRESERVED_KEYWORD}, {large, LARGE_P, UNRESERVED_KEYWORD}, {last, LAST_P, UNRESERVED_KEYWORD}, + {lccollate, LCCOLLATE, UNRESERVED_KEYWORD}, + {lcctype, LCCTYPE, UNRESERVED_KEYWORD}, {leading, LEADING, RESERVED_KEYWORD}, {least, LEAST, COL_NAME_KEYWORD}, {left, LEFT, TYPE_FUNC_NAME_KEYWORD}, *** *** 270,275 --- 274,280 {overlay, OVERLAY, COL_NAME_KEYWORD}, {owned, OWNED, UNRESERVED_KEYWORD}, {owner, OWNER, UNRESERVED_KEYWORD}, + {pad, PAD, UNRESERVED_KEYWORD}, {parser, PARSER, UNRESERVED_KEYWORD}, {partial, PARTIAL, UNRESERVED_KEYWORD}, {password, PASSWORD, UNRESERVED_KEYWORD}, *** *** 317,322 --- 322,328 {second, SECOND_P, UNRESERVED_KEYWORD}, {security, SECURITY, UNRESERVED_KEYWORD}, {select, SELECT, RESERVED_KEYWORD}, + {sensitive, SENSITIVE, UNRESERVED_KEYWORD}, {sequence, SEQUENCE, UNRESERVED_KEYWORD}, {serializable, SERIALIZABLE, UNRESERVED_KEYWORD}, {session, SESSION, UNRESERVED_KEYWORD}, *** *** 329,334 --- 335,341 {simple, SIMPLE, UNRESERVED_KEYWORD}, {smallint, SMALLINT, COL_NAME_KEYWORD}, {some, SOME, RESERVED_KEYWORD}, + {space, SPACE, UNRESERVED_KEYWORD}, {stable, STABLE, UNRESERVED_KEYWORD}, {standalone, STANDALONE_P, UNRESERVED_KEYWORD}, {start, START, UNRESERVED_KEYWORD}, *** *** 337,342 --- 344,350 {stdin, STDIN, UNRESERVED_KEYWORD}, {stdout, STDOUT, UNRESERVED_KEYWORD}, {storage, STORAGE, UNRESERVED_KEYWORD}, + {strcolfn, STRCOLFN, UNRESERVED_KEYWORD}, {strict, STRICT_P, UNRESERVED_KEYWORD}, {strip, STRIP_P, UNRESERVED_KEYWORD}, {substring, SUBSTRING, COL_NAME_KEYWORD}, Index: backend/parser/gram.y === RCS file: /projects/cvsroot/pgsql/src/backend/parser/gram.y,v retrieving revision 2.605 diff -c -r2.605 gram.y *** backend/parser/gram.y 1 Jan 2008 19:45:50 - 2.605 --- backend/parser/gram.y 1 Jul 2008 14:29:25 - *** *** 149,168 } %type node stmt schema_stmt ! AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterGroupStmt ! AlterObjectSchemaStmt AlterOwnerStmt AlterSeqStmt AlterTableStmt ! AlterUserStmt AlterUserSetStmt AlterRoleStmt AlterRoleSetStmt ! AnalyzeStmt ClosePortalStmt ClusterStmt CommentStmt !
Re: [HACKERS] Latest on CITEXT 2.0
On 7/1/08, Tom Lane [EMAIL PROTECTED] wrote: Marko Kreen [EMAIL PROTECTED] writes: On 6/26/08, Tom Lane [EMAIL PROTECTED] wrote: BTW, I don't think you can use that same-length optimization for citext. There's no reason to think that upper/lowercase pairs will have the same length all the time in multibyte encodings. What about this code in current str_tolower(): /* Output workspace cannot have more codes than input bytes */ workspace = (wchar_t *) palloc((nbytes + 1) * sizeof(wchar_t)); That's working with wchars, not bytes. Ah, I missed the point of char2wchar() line. I'm rather unfamiliar with various MB API-s, sorry. There's another thing I'm probably missing: does current code handle multi-wchar codepoints? Or is it guaranteed they don't happen? (Wasn't wchar_t usually 16bit value?) -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: Does anything dump per-database config settings? (was Re: [HACKERS] ALTER DATABASE vs pg_dump)
Robert Treat wrote: On Tuesday 01 July 2008 03:45:44 Richard Huxton wrote: The only time we need to restore per-database settings is if the database has been dropped. If you're not having the dump/restore re-create the database then presumably you've taken charge of the per-database settings. I'm not sure I agree with that entirely. For example, one common usage scenario when upgrading between major versions is to create the database, load contrib modules (whose C functions or similar may have changed), and then load the dump into the database. In those case you still might want the database settings to be dumped, even though you are creating the database manually. (Now, one might argue that you could still dump with --create and ignore the error of the database creation command, but that probably isn't ideal). Well, with -Fc I'd expect it to be dumped all the time and pg_restore would selectively restore it. That should mean it has its own line in the pg_restore --list output which would let you just comment out the database-creation but leave the ALTER...SET in. -- Richard Huxton Archonet Ltd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Latest on CITEXT 2.0
Marko Kreen wrote: On 7/1/08, Tom Lane [EMAIL PROTECTED] wrote: Marko Kreen [EMAIL PROTECTED] writes: On 6/26/08, Tom Lane [EMAIL PROTECTED] wrote: BTW, I don't think you can use that same-length optimization for citext. There's no reason to think that upper/lowercase pairs will have the same length all the time in multibyte encodings. What about this code in current str_tolower(): /* Output workspace cannot have more codes than input bytes */ workspace = (wchar_t *) palloc((nbytes + 1) * sizeof(wchar_t)); That's working with wchars, not bytes. Ah, I missed the point of char2wchar() line. I'm rather unfamiliar with various MB API-s, sorry. There's another thing I'm probably missing: does current code handle multi-wchar codepoints? Or is it guaranteed they don't happen? (Wasn't wchar_t usually 16bit value?) If you want a simple example of wide character use look at oracle_compat.c::upper() which calls str_toupper() in CVS HEAD. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Latest on CITEXT 2.0
On 7/1/08, Bruce Momjian [EMAIL PROTECTED] wrote: If you want a simple example of wide character use look at oracle_compat.c::upper() which calls str_toupper() in CVS HEAD. ATM I'm looking at str_tolower/upper internal implementation. They do: workspace[curr_char] = towlower(workspace[curr_char]); where workspace is wchar_t but towlower() operates on wint_t. Is such inconsistency ok? -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Latest on CITEXT 2.0
Marko Kreen [EMAIL PROTECTED] writes: ATM I'm looking at str_tolower/upper internal implementation. They do: workspace[curr_char] = towlower(workspace[curr_char]); where workspace is wchar_t but towlower() operates on wint_t. IIRC this is exactly comparable to the type situation for the traditional ctype.h macros. The reason is that they are defined to accept EOF in addition to actual char (or wchar) values. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Latest on CITEXT 2.0
Marko Kreen [EMAIL PROTECTED] writes: There's another thing I'm probably missing: does current code handle multi-wchar codepoints? Or is it guaranteed they don't happen? AFAIK we disallow multi-wchar situations (by rejecting the UTF8 combining codes). (Wasn't wchar_t usually 16bit value?) Hmm. It's unsigned int on my ancient HPUX box. I think we could have a problem on any machines whose mbstowcs doesn't support 4-byte UTF8 codes, though ... in particular, what about Windows? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Latest on CITEXT 2.0
On 7/1/08, Tom Lane [EMAIL PROTECTED] wrote: Marko Kreen [EMAIL PROTECTED] writes: ATM I'm looking at str_tolower/upper internal implementation. They do: workspace[curr_char] = towlower(workspace[curr_char]); where workspace is wchar_t but towlower() operates on wint_t. IIRC this is exactly comparable to the type situation for the traditional ctype.h macros. The reason is that they are defined to accept EOF in addition to actual char (or wchar) values. I read SUS v3 and there is no hint on multi-wchar anything, so for unix systems you are right, wint_t == wchar_t. Seems stories how Windows and Java operate have affected me too much. Then I browsed MSDN: http://msdn.microsoft.com/en-us/library/dtxesf6k.aspx and they seem to strongly hint that wchar_t == 16 bits and UTF-16 is used internally. Probably some Windows developer should look into it and decide if there is a #ifdef WIN32 branch needed. -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PATCH: CITEXT 2.0
On Jul 1, 2008, at 07:38, Tom Lane wrote: However, it will be solved when collation per column will be implemented. Well, yeah, but that seems a very long way off. In the meantime a lot of people use the existing pgfoundry citext module. And even more of us have written queries using LOWER(col) = LOWER(?), which is just a PITA. Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Access to localized_str_tolower()
Howdy, In my original implementation of citext, which I'm going to start using for an app I'm developing, I pull in the wstring_lower function from oracle_compat.c by simply declaring it at the top of citext.c, just as if it were in an include file: extern char * wstring_lower (char *str); After I ported it to CVS HEAD, however, and discovered the str_tolower() function in formatting.c, I wanted to try to make use of it in 8.3, as well. But when I tried to include it in citext.c, it simply didn't work. I put this at the top of citext.c: extern char * localized_str_tolower(char *buff); But when I try to use it, like so: lcstr = localized_str_tolower(VARDATA_ANY(left)); rcstr = localized_str_tolower(VARDATA_ANY(right)); I get a compile-time error: Undefined symbols: _localized_str_tolower, referenced from: _citextcmp in citext.o _citextcmp in citext.o ld: symbol(s) not found collect2: ld returned 1 exit status make: *** [citext.so] Error 1 So, stupid C question, I'm sure: How can I use this function from my module? Oh, and on a side note, should I localized_str_tolower() or just str_tolower()? Thanks, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Access to localized_str_tolower()
David E. Wheeler wrote: Howdy, In my original implementation of citext, which I'm going to start using for an app I'm developing, I pull in the wstring_lower function from oracle_compat.c by simply declaring it at the top of citext.c, just as if it were in an include file: extern char * wstring_lower (char *str); After I ported it to CVS HEAD, however, and discovered the str_tolower() function in formatting.c, I wanted to try to make use of it in 8.3, as well. But when I tried to include it in citext.c, it simply didn't work. I put this at the top of citext.c: extern char * localized_str_tolower(char *buff); But when I try to use it, like so: lcstr = localized_str_tolower(VARDATA_ANY(left)); rcstr = localized_str_tolower(VARDATA_ANY(right)); I get a compile-time error: Undefined symbols: _localized_str_tolower, referenced from: _citextcmp in citext.o _citextcmp in citext.o ld: symbol(s) not found collect2: ld returned 1 exit status make: *** [citext.so] Error 1 So, stupid C question, I'm sure: How can I use this function from my module? Oh, and on a side note, should I localized_str_tolower() or just str_tolower()? I am not sure what localized_str_tolower() is but I think you should call str_tolower directly if you want to pass char*, and lower() if you want to pass 'text'. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Access to localized_str_tolower()
On Jul 1, 2008, at 10:38, Bruce Momjian wrote: Oh, and on a side note, should I localized_str_tolower() or just str_tolower()? I am not sure what localized_str_tolower() is but I think you should call str_tolower directly if you want to pass char*, and lower() if you want to pass 'text'. From formatting.c in 8.3.1: #if defined(HAVE_WCSTOMBS) defined(HAVE_TOWLOWER) #define USE_WIDE_UPPER_LOWER /* externs are in oracle_compat.c */ extern char *wstring_upper(char *str); extern char *wstring_lower(char *str); static char *localized_str_toupper(char *buff); static char *localized_str_tolower(char *buff); #else #define localized_str_toupper str_toupper #define localized_str_tolower str_tolower #endif So I assumed it was preferred. FWIW, str_tolower() doesn't work in 8.3, either (note that it is not declared in formatting.h the way it is in CVS HEAD). Best, -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Fairly serious bug induced by latest guc enum changes
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: No, my point was that there are three possible states of sync_bit and your patch only accounted for transitions between two of 'em. Did this every get addressed? I don't see a commit for it. I thought it got fixed here: 2008-05-14 10:02 mha * src/backend/access/transam/xlog.c: Remove the special variable for open_sync_bit used in O_SYNC and O_DSYNC modes, replacing it with a call to a function that derives it from the sync_method variable, now that it has distinct values for these two cases. This means that assign_xlog_sync_method() no longer changes any settings, thus fixing the bug introduced in the change to use a guc enum for wal_sync_method. Hmm ... or at least more or less fixed. Seems like there's no provision to close and reopen the file if enableFsync changes. Not sure if that's worth worrying about. We didn't have that before either, did we? We close it when the sync bit changes, but not just if we change say between fsync() and fdatasync(). Is there any actual reason we'd want to close it? //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCHES] [HACKERS] TRUNCATE TABLE with IDENTITY
Alvaro Herrera wrote: Tom Lane wrote: 2. I had first dismissed Neil's idea of transactional sequence updates as impossible, but on second look it could be done. Suppose RESTART IDENTITY does this for each sequence; * obtain AccessExclusiveLock; * assign a new relfilenode; * insert a sequence row with all parameters copied except last_value copies start_value; * hold AccessExclusiveLock till commit. Hmm, this kills the idea of moving sequence data to a single non-transactional catalog :-( So what I think we should do is leave the patch there, revise the warning per Neil's complaint, and add a TODO item to reimplement RESTART IDENTITY transactionally. I think the TODO item did not make it, but the docs do seem updated. Done: * Fix TRUNCATE ... RESTART IDENTITY so its affect on sequences is rolled back on transaction abort -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Access to localized_str_tolower()
David E. Wheeler wrote: On Jul 1, 2008, at 10:38, Bruce Momjian wrote: Oh, and on a side note, should I localized_str_tolower() or just str_tolower()? I am not sure what localized_str_tolower() is but I think you should call str_tolower directly if you want to pass char*, and lower() if you want to pass 'text'. From formatting.c in 8.3.1: #if defined(HAVE_WCSTOMBS) defined(HAVE_TOWLOWER) #define USE_WIDE_UPPER_LOWER /* externs are in oracle_compat.c */ extern char *wstring_upper(char *str); extern char *wstring_lower(char *str); static char *localized_str_toupper(char *buff); static char *localized_str_tolower(char *buff); #else #define localized_str_toupper str_toupper #define localized_str_tolower str_tolower #endif So I assumed it was preferred. FWIW, str_tolower() doesn't work in 8.3, either (note that it is not declared in formatting.h the way it is in CVS HEAD). That whole use of localized_* is gone in CVS HEAD --- we now have a cleaner API. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Access to localized_str_tolower()
On Jul 1, 2008, at 11:11, Bruce Momjian wrote: That whole use of localized_* is gone in CVS HEAD --- we now have a cleaner API. I know. The patch I sent in uses it. However, I still have a version I want to use on 8.3. So what would be the proper function to use in 8.3.x and how can I get at it? Thanks, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] triggers on prepare, commit, rollback... ?
Andrew Dunstan wrote: Fabien COELHO wrote: Dear Tom, trigger on prepare, commit, rollback, savepoint, Yup, and there won't be. That's a definite answer! This has been suggested and rejected before. See the archives. I'll check into that. This is a sufficiently frequently asked question that I wish someone would add an entry to the FAQ about it, or add it to the TODO list's Features we don't want section. This was already a TODO: * Add database and transaction-level triggers http://archives.postgresql.org/pgsql-hackers/2008-03/msg00451.php http://archives.postgresql.org/pgsql-hackers/2008-05/msg00620.php I have added a URL from the current discussion. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Access to localized_str_tolower()
David E. Wheeler wrote: On Jul 1, 2008, at 11:11, Bruce Momjian wrote: That whole use of localized_* is gone in CVS HEAD --- we now have a cleaner API. I know. The patch I sent in uses it. However, I still have a version I want to use on 8.3. So what would be the proper function to use in 8.3.x and how can I get at it? Uh, good question. I found the upper/lower handling in 8.3 to be pretty convoluted. The big problem is that the macros are used in formatting.c to call the proper functions, and you can't easily reproduce that without copying the macros, which is what you have to do because localized* is not compiled on all platforms. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP patch: reducing overhead for repeat de-TOASTing
[ back on-list ] Mark Cave-Ayland [EMAIL PROTECTED] writes: Thanks very much for supplying the WIP patch. In the interest of testing and feedback, I've just downloaded PostgreSQL CVS head and applied your patch, compiled up a slightly modified version of PostGIS (without RECHECKs on the GiST opclass) and loaded in the same dataset. Unfortunately I have to report back that with your WIP patch applied, timings seem to have become several orders of magnitude *worse*: OK, I've reproduced the test case locally. I believe that when you say worse, you mean worse than 8.3, right? And you did tell me offlist that you were testing with --enable-cassert. CVS HEAD has very substantially greater cassert overhead because of the randomize_memory addition --- oprofile output for this test looks like samples %image name symbol name 1239580 78.7721 postgres randomize_mem 1435449.1218 libc-2.7.so memcpy 48039 3.0528 libc-2.7.so memset 13838 0.8794 postgres LWLockAcquire 12176 0.7738 postgres index_getnext 11697 0.7433 postgres LWLockRelease 10406 0.6613 postgres hash_search_with_hash_value 4739 0.3012 postgres toast_fetch_datum 4099 0.2605 postgres _bt_checkkeys 3905 0.2482 postgres AllocSetAlloc 3751 0.2384 postgres PinBuffer 3545 0.2253 postgres UnpinBuffer I'm inclined to think that we'd better turn that off by default, since it's not looking like it's catching anything new. However, even with cassert turned off, the de-TOAST patch isn't helping your test case; though it does help if I turn the query into a join. Here's what I get for 8.3 HEADHEAD + patch original query [1] 457546694613 original + force_2d [2] 196 195 201 converted to join [3] 46034667 209 original, indexscans off233523912426 join, indexscans off23502373 124 [1] select count(*) from geography where centroid (select the_geom from geography where id=69495); [2] select count(*) from geography where centroid (select force_2d(the_geom) from geography where id=69495); [3] select count(*) from geography g1, geography g2 where g1.centroid g2.the_geom and g2.id=69495; All times in msec, median of 3 trials using psql \timing (times seem to be reproducible within about 1%, if data is already cached). Default parameters all around. The join form of the query produces the results I expected, with g2.the_geom coming from the outer side of a nestloop join and getting detoasted only once. After some digging I found the reason why the original query isn't getting any benefit: it's copying the_geom up from an InitPlan, and nodeSubplan.c does that like this: /* * We need to copy the subplan's tuple into our own context, in case * any of the params are pass-by-ref type --- the pointers stored in * the param structs will point at this copied tuple! node-curTuple * keeps track of the copied tuple for eventual freeing. */ if (node-curTuple) heap_freetuple(node-curTuple); node-curTuple = ExecCopySlotTuple(slot); /* * Now set all the setParam params from the columns of the tuple */ foreach(l, subplan-setParam) { intparamid = lfirst_int(l); ParamExecData *prm = (econtext-ecxt_param_exec_vals[paramid]); prm-execPlan = NULL; prm-value = heap_getattr(node-curTuple, i, tdesc, (prm-isnull)); i++; } So the Param is pointing at a bare toasted Datum, not an indirect pointer in a Slot, and there's no way to avoid detoasting each time the Param is referenced. It would be simple enough to fix nodeSubplan.c to copy the data into an upper-level Slot rather than a bare tuple. But this makes me wonder how many other places are like this. In the past there wasn't that much benefit to pulling data from a Slot instead of a bare tuple, so I'm afraid we might have a number of similar gotchas we'd have to track down. The other thing that worries me is that the force_2d query measurements suggest a runtime penalty of two or three percent for the patch, which is higher than I was hoping for. But that isn't that much more than the noise level in this test. On the whole I'm still feeling pretty discouraged about this patch ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] A new take on the foot-gun meme
Robert Treat wrote: So is that a golf club gun? Careful what you wish for http://www.totallyabsurd.com/12gaugegolfclub.htm I reckon they watched Caddyshack (I think that was the one) and thought they could get the patent before someone actually tried selling them. I reckon it was Rodney Dangerfield that blew the golf ball down the fairway. -- Shane Ambler pgSQL (at) Sheeky (dot) Biz Get Sheeky @ http://Sheeky.Biz -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Location for pgstat.stat
Per this thread: http://archives.postgresql.org/pgsql-general/2007-12/msg00255.php it was pretty much (again, IIRC) concluded that we want some better way to transfer the stats data. But pending that we have that, how about we just move it into it's own subdirectory? AFAICS that would be a simple change of two #defines moving it from global/pgstat.stat to pgstat/pgstat.stat or something like that. Might also need some code to create the directory if it doesn't exist, but that shouldn't be hard. This would make it possible to symlink or mount that directory off to a ramdrive (for example). It's not a perfect solution, but it would at least give a better tool than we have today, no? //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Fairly serious bug induced by latest guc enum changes
Magnus Hagander [EMAIL PROTECTED] writes: Tom Lane wrote: Hmm ... or at least more or less fixed. Seems like there's no provision to close and reopen the file if enableFsync changes. Not sure if that's worth worrying about. We didn't have that before either, did we? No, so I think it's a pre-existing bug. We close it when the sync bit changes, but not just if we change say between fsync() and fdatasync(). Is there any actual reason we'd want to close it? The point is that if you turn the fsync GUC on or off while using a wal sync mode that requires supplying an option flag to open(), then really you ought to close the WAL file and re-open it with the new correct option flags. The fact that we're not doing that implies that the effects of a change in fsync might not fully take effect until the next WAL segment is started. Whether this is worth fixing isn't real clear. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Location for pgstat.stat
Magnus Hagander [EMAIL PROTECTED] writes: But pending that we have that, how about we just move it into it's own subdirectory? This would make it possible to symlink or mount that directory off to a ramdrive (for example). Hmm ... that would almost certainly result in the stats being lost over a system shutdown. How much do we care? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Fairly serious bug induced by latest guc enum changes
Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Tom Lane wrote: Hmm ... or at least more or less fixed. Seems like there's no provision to close and reopen the file if enableFsync changes. Not sure if that's worth worrying about. We didn't have that before either, did we? No, so I think it's a pre-existing bug. Ok, at least I'm reading the code right. We close it when the sync bit changes, but not just if we change say between fsync() and fdatasync(). Is there any actual reason we'd want to close it? The point is that if you turn the fsync GUC on or off while using a wal sync mode that requires supplying an option flag to open(), then really you ought to close the WAL file and re-open it with the new correct option flags. The fact that we're not doing that implies that the effects of a change in fsync might not fully take effect until the next WAL segment is started. Whether this is worth fixing isn't real clear. What scenario does it actually happen in, though? Doesn't the check: if (get_sync_bit(sync_method) != get_sync_bit(new_sync_method)) take care of that? If the sync bit changed, we close the file? Or are you talking about changing the variable fsync? If so, doesn't fsync=off also change the behavior of other parts of the code, so it's not just WAL, which means it'd be pretty unsafe *anyway* unless you actually sync the disks, and not just fsync? //Magnus //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Access to localized_str_tolower()
David E. Wheeler [EMAIL PROTECTED] writes: From formatting.c in 8.3.1: static char *localized_str_toupper(char *buff); static char *localized_str_tolower(char *buff); These are static --- that's why you can't get at them from outside the module. I think your best bet for an 8.3 version of citext is just to copy-and-paste a lot of code from HEAD. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Location for pgstat.stat
Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: But pending that we have that, how about we just move it into it's own subdirectory? This would make it possible to symlink or mount that directory off to a ramdrive (for example). Hmm ... that would almost certainly result in the stats being lost over a system shutdown. How much do we care? Only for those who put it on a ramdrive. The default, unless you move/sync it off, would still be the same as it is today. While not perfect, the performance difference of going to a ramdrive might easily be enough to offset that in some cases, I think. //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Access to localized_str_tolower()
On Jul 1, 2008, at 11:59, Tom Lane wrote: These are static --- that's why you can't get at them from outside the module. I think your best bet for an 8.3 version of citext is just to copy-and-paste a lot of code from HEAD. Well, that's what I'd done already; I was just trying to make it simpler. So, well, now I don't have to do anything. :-) Thanks, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Fairly serious bug induced by latest guc enum changes
Magnus Hagander [EMAIL PROTECTED] writes: Or are you talking about changing the variable fsync? If so, doesn't fsync=off also change the behavior of other parts of the code, so it's not just WAL, which means it'd be pretty unsafe *anyway* unless you actually sync the disks, and not just fsync? No, because the other uses of it are controlling whether to issue fsync() calls dynamically. The use in get_sync_bit is the only one that sets persistent state. In fact md.c goes out of its way to ensure that changing fsync on the fly behaves as expected. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Fairly serious bug induced by latest guc enum changes
Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Or are you talking about changing the variable fsync? If so, doesn't fsync=off also change the behavior of other parts of the code, so it's not just WAL, which means it'd be pretty unsafe *anyway* unless you actually sync the disks, and not just fsync? No, because the other uses of it are controlling whether to issue fsync() calls dynamically. The use in get_sync_bit is the only one that sets persistent state. In fact md.c goes out of its way to ensure that changing fsync on the fly behaves as expected. Not having looked at md.c (I confess...) but don't we have a problem in case we have closed the file without fsyncing it, and then change the fsync parameter? Either way, I see your point, but I doubt it's worth getting upset over. Funning with fsync=off in the first place is bad, and if it takes you one WAL segment to recover, I think that's acceptable... //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Location for pgstat.stat
Magnus Hagander [EMAIL PROTECTED] writes: Tom Lane wrote: Hmm ... that would almost certainly result in the stats being lost over a system shutdown. How much do we care? Only for those who put it on a ramdrive. The default, unless you move/sync it off, would still be the same as it is today. While not perfect, the performance difference of going to a ramdrive might easily be enough to offset that in some cases, I think. Well, what I was wondering about is whether it'd be worth adding logic to copy the file to/from a safer location at startup/shutdown. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Fairly serious bug induced by latest guc enum changes
Magnus Hagander [EMAIL PROTECTED] writes: Not having looked at md.c (I confess...) but don't we have a problem in case we have closed the file without fsyncing it, and then change the fsync parameter? Well, we don't promise to retroactively fsync stuff we didn't before; and I wouldn't expect that to happen if I were changing the setting. What I *would* expect is that the system immediately starts to act according to the new setting, and that's not true as the code stands. As you say, the whole thing is pretty dubious from a data safety standpoint anyway. What I am concerned about here is people trying to compare performance measurements under different settings, and not being aware that the system's behavior doesn't change when they tell it to. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Location for pgstat.stat
Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Tom Lane wrote: Hmm ... that would almost certainly result in the stats being lost over a system shutdown. How much do we care? Only for those who put it on a ramdrive. The default, unless you move/sync it off, would still be the same as it is today. While not perfect, the performance difference of going to a ramdrive might easily be enough to offset that in some cases, I think. Well, what I was wondering about is whether it'd be worth adding logic to copy the file to/from a safer location at startup/shutdown. Oh, I see. I should think more before I answer sometimes :-) Not sure. I guess my own personal concern would be how badly is autovacuum affected by having to start off a blank set of stats? Any other uses I have I think are capable of dealing with reset-to-zero states. //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Location for pgstat.stat
Magnus Hagander wrote: Not sure. I guess my own personal concern would be how badly is autovacuum affected by having to start off a blank set of stats? Any other uses I have I think are capable of dealing with reset-to-zero states. Well, it doesn't :-) No database or table will be processed until stat entries are created, and then I think it will first wait until enough activity gathers to take any actions at all. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] A new take on the foot-gun meme
Shane Ambler [EMAIL PROTECTED] writes: Robert Treat wrote: So is that a golf club gun? Careful what you wish for http://www.totallyabsurd.com/12gaugegolfclub.htm I reckon they watched Caddyshack (I think that was the one) and thought they could get the patent before someone actually tried selling them. Surely a movie counts as published!? -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Get trained by Bruce Momjian - ask me about EnterpriseDB's PostgreSQL training! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Location for pgstat.stat
Alvaro Herrera wrote: Magnus Hagander wrote: Not sure. I guess my own personal concern would be how badly is autovacuum affected by having to start off a blank set of stats? Any other uses I have I think are capable of dealing with reset-to-zero states. Well, it doesn't :-) No database or table will be processed until stat entries are created, and then I think it will first wait until enough activity gathers to take any actions at all. That's not actualliy not affected, but it does seem like it wouldn't be a very big issue. If one table was just about to be vacuumed or analyzed, this would just push it up to twice the threshold, right? //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Location for pgstat.stat
On Tue, 1 Jul 2008, Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Tom Lane wrote: Hmm ... that would almost certainly result in the stats being lost over a system shutdown. How much do we care? Only for those who put it on a ramdrive. The default, unless you move/sync it off, would still be the same as it is today. While not perfect, the performance difference of going to a ramdrive might easily be enough to offset that in some cases, I think. Well, what I was wondering about is whether it'd be worth adding logic to copy the file to/from a safer location at startup/shutdown. Anyone who needs fast stats storage enough that they're going to symlink it to RAM should be perfectly capable of scripting server startup/shutdown to shuffle that to/from a more permanent location. Compared to the admin chores you're likely to encounter before reaching that scale it's a pretty easy job, and it's not like losing that data file is a giant loss in any case. The only thing I could see putting into the server code to help support this situation is rejecting an old stats file and starting from scratch instead if they restored a previous version after a crash that didn't save an updated copy. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Location for pgstat.stat
Magnus Hagander [EMAIL PROTECTED] writes: Alvaro Herrera wrote: Well, it doesn't :-) No database or table will be processed until stat entries are created, and then I think it will first wait until enough activity gathers to take any actions at all. That's not actualliy not affected, but it does seem like it wouldn't be a very big issue. If one table was just about to be vacuumed or analyzed, this would just push it up to twice the threshold, right? Except you could lather, rinse, repeat indefinitely. The stats system started out with the idea that the stats were disposable, but I don't really think that's an acceptable behavior today. We don't even have stats_reset_on_server_start anymore. It doesn't seem to me that it'd be hard to support two locations for the stats file --- it'd just take another parameter to the read and write routines. pgstat.c already knows the difference between a normal write and a shutdown write ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Commitfest status?
Well, it's July 1, and time for another commit fest to begin. Do we have all the submitted patches queued up at http://wiki.postgresql.org/wiki/CommitFest:2008-07 ? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commitfest status?
Tom Lane wrote: Well, it's July 1, and time for another commit fest to begin. Do we have all the submitted patches queued up at http://wiki.postgresql.org/wiki/CommitFest:2008-07 ? I haven't yet committed the dblink patch posted here: http://archives.postgresql.org/pgsql-patches/2008-06/msg00016.php Should I post it on the wiki before committing? Either way I'll commit in the next day or so. Joe -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commitfest status?
Joe Conway wrote: I haven't yet committed the dblink patch posted here: http://archives.postgresql.org/pgsql-patches/2008-06/msg00016.php Should I post it on the wiki before committing? Either way I'll commit in the next day or so. It doesn't matter. Patches are only listed in the wiki so that we don't forget about them, or if you want someone else to review them. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Should enum GUCs be listed as such in config.sgml?
Currently, config.sgml still describes the new enum GUC variables as being of type string --- but pg_settings says they are enum. This is not very consistent, but I wonder whether changing the docs would be more confusing or less so. I note that section 18.1 doesn't mention the enum alternative either. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Commitfest status?
On Tue, 01 Jul 2008 16:19:39 -0400 Tom Lane [EMAIL PROTECTED] wrote: Well, it's July 1, and time for another commit fest to begin. Do we have all the submitted patches queued up at http://wiki.postgresql.org/wiki/CommitFest:2008-07 ? I think Bruce and I have added everything submitted to June 29. I've been offline for 36 hours, though, so I'm scanning hackers and patches now. Help welcomed -- I'm on dial-up and it's slow. Time for people to start volunteering to review stuff! I'll start round-robin after a few days. So put your names on the stuff you know you can review now. Josh Berkus PostgreSQL @ Sun San Francisco 415-752-2500 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TODO assignment
Jiri, I don't know of anyone currently working on the apply permissions to all objects TODO. You're welcome to take a stab at it. --Josh Josh Berkus PostgreSQL @ Sun San Francisco 415-752-2500 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Please claim review items for commit fest!
Hackers, Just in case anyone was unclear, this is how we're trying things for this commitfest: 1) Starting RIGHT NOW, reviewers should claim review items they are interested in or specially qualified to review. 2) This weekend, I will check for all items which don't have one or more reviewers and parcel them out to the Round Robin Reviewers who don't already have patches to review. You do not have to be a committer to be a reviewer. Anyone who knows C code and is familiar with Postgres Josh Berkus PostgreSQL @ Sun San Francisco 415-752-2500 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Please claim review items for commit fest!
Hackers, Just in case anyone was unclear, this is how we're trying things for this commitfest: 1) Starting RIGHT NOW, reviewers should claim review items they are interested in or specially qualified to review. 2) This weekend, I will check for all items which don't have one or more reviewers and parcel them out to the Round Robin Reviewers who don't already have patches to review. You do not have to be a committer to be a reviewer. Anyone who knows C code and is familiar with PostgreSQL can be a reviewer. Heck, even non-C coders can review proposed APIs. Each item can have several reviewers, and probably should. Oh, also reviewers -- please try to use constructive criticism! Some people are submitting their first patch, and we don't want them to leave the project forever. Thanks! Josh Berkus PostgreSQL @ Sun San Francisco 415-752-2500 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
OT: Re: [HACKERS] A new take on the foot-gun meme
Gregory Stark wrote: Shane Ambler [EMAIL PROTECTED] writes: Robert Treat wrote: So is that a golf club gun? Careful what you wish for http://www.totallyabsurd.com/12gaugegolfclub.htm I reckon they watched Caddyshack (I think that was the one) and thought they could get the patent before someone actually tried selling them. Surely a movie counts as published!? No the term is prior art leaving the lawyers to bill $400/hour while they argue over whether or not Caddyshack is art. Though the movie might have inspired this: http://www.rodenator.com/ http://video.google.com/videoplay?docid=2386436112453851581 http://www.youtube.com/watch?v=2umEFHeo6mw Looks fun as long as you don't do this: http://uk.reuters.com/article/oddlyEnoughNews/idUKN2432304520080326 Cheers, Steve -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] A new take on the foot-gun meme
In further OT Gregory Stark wrote: Shane Ambler [EMAIL PROTECTED] writes: Robert Treat wrote: So is that a golf club gun? Careful what you wish for http://www.totallyabsurd.com/12gaugegolfclub.htm I reckon they watched Caddyshack (I think that was the one) and thought they could get the patent before someone actually tried selling them. Surely a movie counts as published!? No the term is prior art leaving the lawyers to bill $400/hour while they argue over whether or not Caddyshack is art. Though the movie might have inspired this: http://www.rodenator.com/ http://video.google.com/videoplay?docid=2386436112453851581 http://www.youtube.com/watch?v=2umEFHeo6mw Looks fun as long as you don't do this: http://uk.reuters.com/article/oddlyEnoughNews/idUKN2432304520080326 In something of the same vein: http://news.bbc.co.uk/2/hi/americas/4593682.stm ... IMHO the pendejo got what was coming to him, though. Back to work, but I really appreciate some of the meanderings here. Greg Williamson Senior DBA DigitalGlobe Confidentiality Notice: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information and must be protected in accordance with those provisions. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. (My corporate masters made me say this.)
[HACKERS] Limits of backwards compatibility for psql's \d commands
I'm fooling around with Guillaume Lelarge's patch to make psql's \d commands work with older server versions. The patch as submitted works with servers back to 7.4 (modulo a small bug or two). I tried to see what it'd take to make it work with 7.3. I count about a dozen trivial diffs and about three nontrivial ones --- nontrivial meaning I didn't see a simple fix right away. This seems a bit more work than is justified for a server version that the community has officially dropped support for, but I wonder if anyone feels hot about it? Pre-7.3 server versions seem entirely out of the realm of reason because they lack schema support, meaning all of those pg_catalog. prefixes break, not to mention the joins to pg_namespace and the schema output columns. So it's just 7.3 that's worth debating, I think. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Limits of backwards compatibility for psql's \d commands
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 I didn't see a simple fix right away. This seems a bit more work than is justified for a server version that the community has officially dropped support for, but I wonder if anyone feels hot about it? Pre-7.3 server versions seem entirely out of the realm of reason because they lack schema support, meaning all of those pg_catalog. prefixes break, not to mention the joins to pg_namespace and the schema output columns. So it's just 7.3 that's worth debating, I think. As a huge long-time supporter of backwards compatibility in psql, I say abandon 7.3 support completely, and good riddance. - -- Greg Sabino Mullane [EMAIL PROTECTED] PGP Key: 0x14964AC8 200807012354 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAkhq/CgACgkQvJuQZxSWSsh9OgCgrmUZKOQbQEdkXddPTDrvNFzu 1f4An0mtoAvt4VGFTjfn8Qoyy1JEIrju =oSyR -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Limits of backwards compatibility for psql's \d commands
On Tue, 2008-07-01 at 22:33 -0400, Tom Lane wrote: Pre-7.3 server versions seem entirely out of the realm of reason because they lack schema support, meaning all of those pg_catalog. prefixes break, not to mention the joins to pg_namespace and the schema output columns. So it's just 7.3 that's worth debating, I think. We already abandoned 7.3 , so IMHO we should not worry about compatibility with it anymore, unless we are strongly forced to. Regards, -- Devrim GÜNDÜZ devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr http://www.gunduz.org -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Limits of backwards compatibility for psql's \d commands
Tom Lane wrote: I'm fooling around with Guillaume Lelarge's patch to make psql's \d commands work with older server versions. The patch as submitted works with servers back to 7.4 (modulo a small bug or two). I tried to see what it'd take to make it work with 7.3. I count about a dozen trivial diffs and about three nontrivial ones --- nontrivial meaning I didn't see a simple fix right away. This seems a bit more work than is justified for a server version that the community has officially dropped support for, but I wonder if anyone feels hot about it? Pre-7.3 server versions seem entirely out of the realm of reason because they lack schema support, meaning all of those pg_catalog. prefixes break, not to mention the joins to pg_namespace and the schema output columns. So it's just 7.3 that's worth debating, I think. EOL is EOL, why is the question even being asked? Joshua D. Drake regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers