Re: [HACKERS] unlogged tables
On Sat, 2010-11-13 at 20:55 -0500, Robert Haas wrote: I think that would be a recipe for bugs. Look at the three new macros I introduced. If you keep relistemp around, then any code which relies on it is likely testing for one of those three things, or maybe even something subtly different from any of them, as in the cases where I needed to add a switch statement. The way I see it, this is ultimately a four-level hierarchy That argument isn't clear enough to avoid me agreeing so far with Tom and Andrew that logged-ness is separate from temp-ness. As you say though, it might be a recipe for bugs, so please explain a little more. -- Simon Riggs http://www.2ndQuadrant.com/books/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables
On Sat, 2010-11-13 at 19:16 -0500, Robert Haas wrote: 3. The third patch (relax-sync-commit-v1) allows asynchronous commit even when synchronous_commit=on if the transaction has not written WAL. Of course, a read-only transaction won't even have an XID and therefore won't need a commit record, so what this is really doing is allowing transactions that have written only to temp - or unlogged - tables to commit asynchronously. I like this, great idea. Avoiding the commit record entirely will break Hot Standby though, since we rely on the assumption that all xids that are assigned are also logged. The xids would be known assigned, yet since they never actually appear they will clog up the machinery (pun unintended). -- Simon Riggs http://www.2ndQuadrant.com/books/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TABLE ... REPLACE WITH
On Wed, 2010-12-15 at 10:54 +0100, Csaba Nagy wrote: On Tue, 2010-12-14 at 14:36 -0500, Robert Haas wrote: Well, you have to do that for DROP TABLE as well, and I don't see any way around doing it for REPLACE WITH. Sure, but in Simon's proposal you can load the data FIRST and then take a lock just long enough to do the swap. That's very different from needing to hold the lock during the whole data load. Except Simon's original proposal has this line in it: * new_table is TRUNCATEd. I guess Simon mixed up new_table and old_table, and the one which should get truncated is the replaced one and not the replacement, otherwise it doesn't make sense to me. What I meant was... REPLACE TABLE target WITH source; * target's old rows are discarded * target's new rows are all of the rows from source. * source is then truncated, so ends up empty Perhaps a more useful definition would be EXCHANGE TABLE target WITH source; which just swaps the heap and indexes of each table. You can then use TRUNCATE if you want to actually destroy data. I will go with that unless we have other objections. BTW, I would have also used such a feature on multiple occasions in the past and expect I would do in the future too. Cheers, Csaba. -- Simon Riggs http://www.2ndQuadrant.com/books/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Segfault related to pg_authid when running initdb from git master
On 15 December 2010 01:35, Robert Haas robertmh...@gmail.com wrote: I am suspicious of the fact that you are invoking initdb as ./initdb. Is it possible you're invoking this from the build tree, and there's an installed copy out there that doesn't match, but is getting used? Like maybe in /usr/local/pgsql/bin? No, I'm not doing that. I'm running initdb from /usr/local/pgsql/bin (nothing pg related can be found in my $PATH), but it's the only copy on my system, which was installed from git master last night. It has debugging symbols, and I've actually re-created this from initdb's point of view within GDB with source level debugging. Can you fire up gdb on this core dump, using gdb /usr/local/pgsql/bin/postgres /path/to/coredump? Or, another possibility is to run initdb with --noclean and then run the command, without routing the output to /dev/null: /usr/local/pgsql/bin/postgres --single -F -O -c search_path=pg_catalog -c exit_on_error=true template1 I cannot find the coredump. Perhaps it's a permissions issue. What do you think? Anyway, I have produced a useful backtrace by debugging postgres directly after running initdb with --noclean as described: [pe...@peter bin]$ /usr/local/pgsql/bin/postgres --single -F -O -c search_path=pg_catalog -c exit_on_error=true template1 Segmentation fault [pe...@peter bin]$ gdb postgres GNU gdb (GDB) Fedora (7.2-26.fc14) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type show copying and show warranty for details. This GDB was configured as x86_64-redhat-linux-gnu. For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/... Reading symbols from /usr/local/pgsql/bin/postgres...done. (gdb) set args --single -F -O -c search_path=pg_catalog -c exit_on_error=true template1 (gdb) start Temporary breakpoint 1 at 0x577360 Starting program: /usr/local/pgsql/bin/postgres --single -F -O -c search_path=pg_catalog -c exit_on_error=true template1 Temporary breakpoint 1, 0x00577360 in main () (gdb) c Continuing. Program received signal SIGSEGV, Segmentation fault. 0x0047615b in _bt_preprocess_keys () (gdb) bt #0 0x0047615b in _bt_preprocess_keys () #1 0x00475382 in _bt_first () #2 0x00473d71 in btgettuple () #3 0x006ba67c in FunctionCall2 () #4 0x0046e08a in index_getnext () #5 0x0046d556 in systable_getnext () #6 0x006a92bf in LookupOpclassInfo () #7 0x006a9a58 in RelationInitIndexAccessInfo () #8 0x006aa9cb in RelationBuildDesc () #9 0x006aabfd in load_critical_index () #10 0x006ac12a in RelationCacheInitializePhase3 () #11 0x006c19ca in InitPostgres () #12 0x0060058f in PostgresMain () #13 0x0057774d in main () For some reason, postgres has limited debugging symbols (no line number information is available). Given that it is available from initdb, that seems very odd: Temporary breakpoint 1 at 0x577360 Starting program: /usr/local/pgsql/bin/postgres --single -F -O -c search_path=pg_catalog -c exit_on_error=true template1 Temporary breakpoint 1, 0x00577360 in main () (gdb) n Single stepping until exit from function main, which has no line number information. Program received signal SIGSEGV, Segmentation fault. 0x0047615b in _bt_preprocess_keys () Hope that helps. -- Regards, Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
On Wed, Dec 15, 2010 at 12:19:53AM +0100, Jan Urbański wrote: Problem: what to do it hstore_plpython gets loaded, but hstore is not yet loaded. hstore_plpython will want to DirectFunctionCall(hstore_in), so loading hstore_plpython without loading hstore will result in an ereport(ERROR, undefined symbol hstore_in) with an errhint of please load hstore first. I could live with that, if no one has a better idea. Correction: you won't get the helpful errhint, because the ERROR will be thrown when some does LOAD. And it still does not solve the problem of knowing whether it's a hstore that's been passed in to you. OK, here's another master plan: 1) hstore_plplython, when loaded, looks for a type called hstore. If you created a hstore type that does not come from hstore.so, and you still load hstore_plpython, you deserve a segfault. If there is no type hstore, it throws an ERROR. If it finds a type with that name, it creates a rendezvous variable with the name OID_plpython_parsers that points to two functions. These functions use the looked up type's I/O funcs and transform things you pass to them from and into Python objects. 2) plpython, when receiving an object with a type with the name X, takes its OID, it the OID happens not to be one of BOOLOID, FLOAT8OID etc, it does one last push of looking for a rendezvous variable OID_plpython_parsers and if it finds one, uses its parsers. If it doesn't find it, it does what it did now (cast to text and pass it to the type's I/O func). That looks almost good to me. It's mildly annoying that you can't load hstore_plpython before hstore, but I could live with that. Observe that this allows you to write a isbn_plpython module that would expose parsers for ISBN for python (or json_plpython), as well as hstore_perl, isbn_tcl and so on. It piggybacks on the rendezvous variables mechanism, and maybe in the future you could get some kind of official support in the backend for this kind of things (ie. a hash table in TopLevelContext keyed on the OIDs of the type and the language). So I'm going to try this approach now. Cheers, Jan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Default mode for shutdown
I'm sure this has been up before, but hey, let's take it another round. Why don't we change the default shutdown mode for pg_ctl from smart to fast? I've never come across a single usecase where smart is what people *want*... Not sure if others have? Yes, I realize it's somewhat of a backwards compatibility thing - but it will at least not change things for most packages, since I believe all those use fast anyway. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] V3: Idle in transaction cancellation
On Wednesday 15 December 2010 02:20:31 Robert Haas wrote: On Sat, Oct 30, 2010 at 4:49 AM, Andres Freund and...@anarazel.de wrote: Here is a proposed patch which enables cancellation of $subject. Disclaimer: This isn't my area of expertise, so take the below with a grain or seven of salt. I don't know whos area of expertise it is except maybe, surprise, surprise, Toms. It sort of looks to me like the LOG_NO_CLIENT error flag and the silent_error_while_idle flag are trying to cooperate to get the effect of throwing an error without actually throwing an error. I'm wondering if it would be at all sensible to do that more directly by making ProcessInterrupts() call AbortCurrentTransaction() in this case. Hm. I think you want the normal server-side error logging continuing to work. Its not really throwing an error without throwing one - its throwing one without confusing the heck out of the client because the protocol is not ready for that. I don't think introducing an half-error state is a good idea because one day the protocol maybe ready to actually transport an error while idle in txn (I would like to get there). I'm not sure if this would work, or if it's better. I'm just throwing it out there, because the current approach looks a little grotty to me. I with you on the grotty aspect... On the other hand the whole code is not exactly nice... Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] V3: Idle in transaction cancellation
On Wed, Dec 15, 2010 at 7:13 AM, Andres Freund and...@anarazel.de wrote: It sort of looks to me like the LOG_NO_CLIENT error flag and the silent_error_while_idle flag are trying to cooperate to get the effect of throwing an error without actually throwing an error. I'm wondering if it would be at all sensible to do that more directly by making ProcessInterrupts() call AbortCurrentTransaction() in this case. Hm. I think you want the normal server-side error logging continuing to work. I was thinking we could get around that by doing elog(LOG), but I guess that doesn't quite work either since we don't know what client_min_messages is. Hrm... I'm not sure if this would work, or if it's better. I'm just throwing it out there, because the current approach looks a little grotty to me. I with you on the grotty aspect... On the other hand the whole code is not exactly nice... Yeah. I'll try to find some time to think about this some more. It would sure be nice if we could find a solution that's a bit conceptually cleaner, even if it basically works the same way as what you've done here. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TABLE ... REPLACE WITH
On Wed, Dec 15, 2010 at 5:39 AM, Simon Riggs si...@2ndquadrant.com wrote: Perhaps a more useful definition would be EXCHANGE TABLE target WITH source; which just swaps the heap and indexes of each table. You can then use TRUNCATE if you want to actually destroy data. I will go with that unless we have other objections. I still don't see how that's going to work with foreign keys. If there's a foreign key referencing the old table, there's no way to be sure that all of those references are still going to be valid with respect to the new table without a full-table check. And that seems to defeat the purpose of the feature. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] V3: Idle in transaction cancellation
On Wednesday 15 December 2010 13:33:30 Robert Haas wrote: On Wed, Dec 15, 2010 at 7:13 AM, Andres Freund and...@anarazel.de wrote: It sort of looks to me like the LOG_NO_CLIENT error flag and the silent_error_while_idle flag are trying to cooperate to get the effect of throwing an error without actually throwing an error. I'm wondering if it would be at all sensible to do that more directly by making ProcessInterrupts() call AbortCurrentTransaction() in this case. Hm. I think you want the normal server-side error logging continuing to work. I was thinking we could get around that by doing elog(LOG), but I guess that doesn't quite work either since we don't know what client_min_messages is. Hrm... I thought about doing that first. Btw, LOG_NO_CLIENT is just a more abstracted way of what COMERROR did before... I'm not sure if this would work, or if it's better. I'm just throwing it out there, because the current approach looks a little grotty to me. I with you on the grotty aspect... On the other hand the whole code is not exactly nice... Yeah. I'll try to find some time to think about this some more. It would sure be nice if we could find a solution that's a bit conceptually cleaner, even if it basically works the same way as what you've done here. I would like that as well. I am not sure you can achieve that in a reasonable amount of work. At least I couldn't. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: Proposed Windows-specific change: Enable crash dumps (like core files)
I've updated this entry in the CommitFest app to note that Craig had some implementation questions attached to his patch submission that I haven't seen anyone address yet, and to include a reference to Tom's latest question--which may make those questions moot, not sure. This pretty clearly need to sit on the stove a little bit longer before it's done regardless. I'm marking this one Returned With Feedback, and hopefully Craig will continue hammering on this to clean up the remaining details and resubmit in the next month. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services and Supportwww.2ndQuadrant.us PostgreSQL 9.0 High Performance: http://www.2ndQuadrant.com/books -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL/MED - file_fdw
Hi hackers, Attached is the revised WIP version of file_fdw patch. This patch should be applied after both of fdw_syntax and fdw_scan patches, which have been posted to another thread SQL/MED - core functionality. In this version, file_fdw consists of two parts, file_fdw core part and copy of COPY FROM codes as they were in last version. The reason of this form is to make it possible to test actual SELECT statement ASAP. I'll revise file_fdw again according to Itagaki-san's export-copy-routines patch. Note that this version of file_fdw doesn't support force_not_null option because column-level generic option is not supported by current fdw_syntax. It will be available if column-level generic option is implemented. And, as possible implementation of FDW-specific EXPLAIN information, EXPLAIN SELECT xxx FROM file shows name and size of the file. It may be better to hide file information if the user was not superuser for security reason. If so, filename option should not appear in output of \det psql command too. Regards, -- Shigeru Hanada file_fdw.patch.gz Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TABLE ... REPLACE WITH
On Dec 15, 2010, at 4:39 AM, Simon Riggs wrote: On Wed, 2010-12-15 at 10:54 +0100, Csaba Nagy wrote: On Tue, 2010-12-14 at 14:36 -0500, Robert Haas wrote: Well, you have to do that for DROP TABLE as well, and I don't see any way around doing it for REPLACE WITH. Sure, but in Simon's proposal you can load the data FIRST and then take a lock just long enough to do the swap. That's very different from needing to hold the lock during the whole data load. Except Simon's original proposal has this line in it: * new_table is TRUNCATEd. I guess Simon mixed up new_table and old_table, and the one which should get truncated is the replaced one and not the replacement, otherwise it doesn't make sense to me. What I meant was... REPLACE TABLE target WITH source; * target's old rows are discarded * target's new rows are all of the rows from source. * source is then truncated, so ends up empty Perhaps a more useful definition would be EXCHANGE TABLE target WITH source; which just swaps the heap and indexes of each table. You can then use TRUNCATE if you want to actually destroy data. Are there any considerations with toast tables and the inline line pointers for toasted tuples? Regards, David -- David Christensen End Point Corporation da...@endpoint.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: Proposed Windows-specific change: Enable crash dumps (like core files)
On 12/15/2010 01:01 AM, Tom Lane wrote: Craig Ringercr...@postnewspapers.com.au writes: I've attached an updated patch that fixes a failure when compiling on gcc/linux. The no-op inline installCrashDumpHandler() for unsupported platforms was not declared static, so it was not being optimized out of objects it wasn't used in and was causing symbol collisions during linkage. Why in the world would you get involved in that portability mess for a function that is called only once? There's no possible performance justification for making it inline. The main concern I heard voiced when first suggesting this was about performance. Given that concern, if I could make it a no-op on unix/linux I thought that worth doing. I'm _much_ happier with a simple, non-ifdef'd extern function declaration and compilation of an empty function body on unsupported platforms. Given how concerned everyone was about *any* effect on backend startup, though, I was concerned that'd be turned down as unnecessary bloat. I've done it a nicer way now, and will post the updated patch once I've had a chance to re-test it on my Windows dev box. I'm also wondering why you have got conflicting declarations in postgres.h and port.h, and why none of these declarations follow ANSI C (write (void) not ()). For postgres.h : that's a good question, as I thought I removed that. I suspect it was reintroduced when reapplying the patch to my working tree to revise it. Whoops. As for the ansi C style - too much time with C++, though long ago now. I think I got the PostgreSQL rules for code formatting right, but missed the void param rule. -- Craig Ringer -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Instrument checkpoint sync calls
Alvaro Herrera wrote: I gave this patch a look and it seems pretty good to me, except that I'm uncomfortable with the idea of mdsync filling in the details for CheckpointStats fields directly. Would it work to pass a struct (say SmgrSyncStats) from CheckPointBuffers to smgrsync and from there to mdsync, have this function fill it, and return it back so that CheckPointBuffers copies the data from this struct into CheckpointStats? That was originally how I planned to write this bit of code. When I realized that the CheckpointStats structure was already visible there and stuffed with details that ultimately go into the same output line at the end, it just didn't seem worth the extra code complexity. The abstraction layer around md.c was not exactly airtight before I poked that extra little hole in there, and I was aiming via the principal of a smaller diff usually being the better patch . If you feel strongly that the result led to a bad abstraction violation, I'll submit a patch to refactor it to pass a structure instead before the next CF. I appreciate your concern, I'm just not sure it's worth spending time on. What I'd really like to do is refactor out major parts of the leaky md/smgr layers altogether instead, but that's obviously a bigger project. Another minor nitpick: inside the block when you call FileSync, why check for log_checkpoints at all? Seems to me that just checking for zero of sync_start should be enough. Alternatively, seems simpler to just have a local var with the value of log_checkpoints at the start of mdsync and use that throughout the function. (Surely if someone turns off log_checkpoints in the middle of a checkpoint, it's not really a problem that we collect and report stats during that checkpoint.) And now you're just getting picky! This is a useful observation though, and I'll try to include that fix along with the next general checkpoint overhaul patch I submit. Doesn't seem worth going through the trouble of committing that minor rework on its own, I'll slip it into the next useful thing that touches this area I do. Thanks for the hint, this would work better than what I did. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services and Supportwww.2ndQuadrant.us PostgreSQL 9.0 High Performance: http://www.2ndQuadrant.com/books -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
Jan =?utf-8?B?VXJiYcWEc2tp?= wulc...@wulczer.org writes: OK, here's another master plan: 1) hstore_plplython, when loaded, looks for a type called hstore. If you created a hstore type that does not come from hstore.so, and you still load hstore_plpython, you deserve a segfault. No, you don't. I said upthread that relying on the name of the type was a nonstarter, and it still is. For one thing, this sketch ignores search path issues. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Default mode for shutdown
Magnus Hagander mag...@hagander.net writes: I'm sure this has been up before, but hey, let's take it another round. Why don't we change the default shutdown mode for pg_ctl from smart to fast? I've never come across a single usecase where smart is what people *want*... Really? Personally I'm quite happy with that default. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] V3: Idle in transaction cancellation
On Wed, Dec 15, 2010 at 7:47 AM, Andres Freund and...@anarazel.de wrote: I thought about doing that first. Btw, LOG_NO_CLIENT is just a more abstracted way of what COMERROR did before... Hmm, but it must not be quite the same, because that didn't require the silent_error_while_idle flag. Yeah. I'll try to find some time to think about this some more. It would sure be nice if we could find a solution that's a bit conceptually cleaner, even if it basically works the same way as what you've done here. I would like that as well. I am not sure you can achieve that in a reasonable amount of work. At least I couldn't. Is there a way that errstart() and/or errfinish() can know enough about the state of the communication with the frontend to decide whether to suppress edata-output_to_client? In other words, instead of explicitly passing in a flag that says whether to inform the client, it would be better for the error-reporting machinery to intrinsically know whether it's right to send_message_to_frontend(). Otherwise, an error thrown from an unexpected location might not have the flag set correctly. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Default mode for shutdown
On Wed, Dec 15, 2010 at 9:39 AM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: I'm sure this has been up before, but hey, let's take it another round. Why don't we change the default shutdown mode for pg_ctl from smart to fast? I've never come across a single usecase where smart is what people *want*... +1. I think we should either have a timeout for smart shutdown mode such that it turns into a fast shutdown after a configurable number of seconds that defaults to, say, 30; or we should just make the default fast shutdown as proposed. Really? Personally I'm quite happy with that default. Why? It seems to me that just leads to, oh, gee, the database isn't shutting down, where's the window where I failed to exit a session? And it's even worse in production, where whatever you're using for connection pooling ensures that shutdown will take, if not forever, at least a very, very long time. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Default mode for shutdown
On Wed, Dec 15, 2010 at 09:39:12AM -0500, Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: I'm sure this has been up before, but hey, let's take it another round. Why don't we change the default shutdown mode for pg_ctl from smart to fast? I've never come across a single usecase where smart is what people *want*... Really? Personally I'm quite happy with that default. regards, tom lane +1 I think the default is perfect. Even if the usecase that is wanted is fast, it should be requested each time to verify that a more destructive shutdown is wanted. If it is really an issue, a script or shell alias can be defined to perform the more aggressive shutdown processes. Regards, Ken -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Default mode for shutdown
Robert Haas robertmh...@gmail.com writes: On Wed, Dec 15, 2010 at 9:39 AM, Tom Lane t...@sss.pgh.pa.us wrote: Really? Personally I'm quite happy with that default. Why? It seems to me that just leads to, oh, gee, the database isn't shutting down, where's the window where I failed to exit a session? Yeah, and more to the point, do I want to finish whatever I was doing in that window? Fast-by-default is a nice hammer to swing, but one day you'll pound your finger. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TABLE ... REPLACE WITH
On Wed, 2010-12-15 at 10:39 +, Simon Riggs wrote: Perhaps a more useful definition would be EXCHANGE TABLE target WITH source; which just swaps the heap and indexes of each table. You can then use TRUNCATE if you want to actually destroy data. Yes please, that's exactly what I would have needed in many occasions. But one problem would be when the replaced table is the _parent_ for a foreign key relationship. I don't think you can have that constraint pre-verified on the replacement table and simply replacing the content could leave the child relations with orphans. Cheers, Csaba. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TABLE ... REPLACE WITH
On Tue, 2010-12-14 at 14:36 -0500, Robert Haas wrote: Well, you have to do that for DROP TABLE as well, and I don't see any way around doing it for REPLACE WITH. Sure, but in Simon's proposal you can load the data FIRST and then take a lock just long enough to do the swap. That's very different from needing to hold the lock during the whole data load. Except Simon's original proposal has this line in it: * new_table is TRUNCATEd. I guess Simon mixed up new_table and old_table, and the one which should get truncated is the replaced one and not the replacement, otherwise it doesn't make sense to me. BTW, I would have also used such a feature on multiple occasions in the past and expect I would do in the future too. Cheers, Csaba. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
On Wed, Dec 15, 2010 at 9:38 AM, Tom Lane t...@sss.pgh.pa.us wrote: Jan =?utf-8?B?VXJiYcWEc2tp?= wulc...@wulczer.org writes: OK, here's another master plan: 1) hstore_plplython, when loaded, looks for a type called hstore. If you created a hstore type that does not come from hstore.so, and you still load hstore_plpython, you deserve a segfault. No, you don't. I said upthread that relying on the name of the type was a nonstarter, and it still is. For one thing, this sketch ignores search path issues. Well then we need a reliable way to identify a type. What would satisfy you? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Default mode for shutdown
On Wed, Dec 15, 2010 at 9:47 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Dec 15, 2010 at 9:39 AM, Tom Lane t...@sss.pgh.pa.us wrote: Really? Personally I'm quite happy with that default. Why? It seems to me that just leads to, oh, gee, the database isn't shutting down, where's the window where I failed to exit a session? Yeah, and more to the point, do I want to finish whatever I was doing in that window? Fast-by-default is a nice hammer to swing, but one day you'll pound your finger. I guess. I've pounded my finger enough time with the current default that I'd be willing to try a different size hammer. The scenario you describe has yet to occur in 10+ years of using the product, but obviously not everyone's experience will match on this point. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
Robert Haas robertmh...@gmail.com writes: Well then we need a reliable way to identify a type. What would satisfy you? Either (1) do nothing (reject this whole proposal) or (2) put hstore in core where it will have a well-known OID. While it would be nice to have some more-workable way to interconnect independent extensions, I feel no need to either design a solution to that on the spot, or to accept half-baked approaches to it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
On 15/12/10 15:38, Tom Lane wrote: Jan =?utf-8?B?VXJiYcWEc2tp?= wulc...@wulczer.org writes: OK, here's another master plan: 1) hstore_plplython, when loaded, looks for a type called hstore. If you created a hstore type that does not come from hstore.so, and you still load hstore_plpython, you deserve a segfault. No, you don't. I said upthread that relying on the name of the type was a nonstarter, and it still is. For one thing, this sketch ignores search path issues. Hm. I wa assuming that if you have a type called hstore that's not the contrib hstore type, but you do install hstore_plpython from contrib, then you can't expect it to work. So how about this: hstore when loaded sets a rendezvous variable that points to its I/O routines, called org.postgresql.types.hstore. hstore_plpython looks for that rendezvous variable instead of looking up the type from the catalogs, and then sets a RVV called org.postgresql.parsers.hstore.plpython. The problem now is how plpython is supposed to know if the object it gets is the same hstore, and not some other type called hstore. What would fix it, is if the hstore module could somehow know what OID did the system assign to it, and would publish its I/O routines *and* its OID as org.postgresql.types.hstore. hstore_plpython would then look for org.postgresql.types.hstore and set up org.postgresql.parsers.OID.plpython and plpython would look for org.postgresql.parsers.plpython.WHATEVER-OID-GOT-PASSED. It almost looks like we need a unique identifier for the extension type that's known beforehand by the type writer (which org.postgresql.types.hstore would be) Cheers, Jan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Crash safe visibility map vs hint bits
Heikki Linnakangas wrote: On 04.12.2010 09:14, jes...@krogh.cc wrote: There has been a lot discussion about index-only scans and how to make the visibillity map crash safe. Then followed by a good discussion about hint bits. What seems to be the main concern is the added wal volume and it makes me wonder if there is a way in-between that looks more like hint bits. How about lazily wal-log the complete visibility map say every X minutes or N amount of tuple updates and make the wal recovery jobs of rechecking visibility of pages touched by the wal stream on recovery. If you WAL-log the visibility map changes after-the-fact, it doesn't solve the race condition we're struggling with: the visibility map change might hit the disk before the PD_ALL_VISIBLE to the heap page. If you crash, you can end up with a situation where the PD_ALL_VISIBLE flag on the heap page is not set, but the bit in the visibility map is. Which causes serious issues later on. Based on hacker emails and a discussion I had with Heikki while we were in Germany, I have updated the index-only scans wiki to document a known solution to making the visibility map crash-safe for use by index-only scan use: http://wiki.postgresql.org/wiki/Index-only_scans#Making_the_Visibility_Map_Crash-Safe Making the Visibility Map Crash-Safe Currently, a heap page that has all-visible tuples is marked by vacuum as PD_ALL_VISIBLE and the visibility map (VM) bit is set. This is currently unlogged, and a crash could require these to be set again. The complexity is that for index-only scans, the VM bit has meaning, and cannot be incorrectly set (though it can be incorrectly cleared because that would just result in additional heap access). If both PD_ALL_VISIBLE and the VM bit were to be set, and a crash resulted the VM bit being written to disk, but not the PD_ALL_VISIBLE bit, a later heap access that wrote a conditionally-visible row would not know to clear the VM bit, causing incorrect results for index-only scans. The solution is to WAL log the VM set bit activity. This will cause full-page writes for the VM page, but this is much less than WAL-logging each heap page because a VM page represents many heap pages. This requires that the VM page not be written to disk until its VM-set WAL record is fsynced to disk. Also, during crash recovering, reading the VM-set WAL record would cause both the VM-set and heap PD_ALL_VISIBLE to be set. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
On Wed, Dec 15, 2010 at 9:53 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Well then we need a reliable way to identify a type. What would satisfy you? Either (1) do nothing (reject this whole proposal) or (2) put hstore in core where it will have a well-known OID. While it would be nice to have some more-workable way to interconnect independent extensions, I feel no need to either design a solution to that on the spot, or to accept half-baked approaches to it. I was asking what would satisfy you as regards a reliable way to identify a type, not what you think we should do about this particular proposal. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
On Dec15, 2010, at 02:14 , James William Pye wrote: On Dec 13, 2010, at 6:16 PM, Tom Lane wrote: how do you identify which type OID is really hstore? How about an identification field on pg_type? CREATE TYPE hstore ..., IDENTIFIER 'org.postgresql.hstore'; -- Where the identifier is an arbitrary string. +1 I've wanted something like this a few times when dealing with custom types within a client. A future protocol version might even transmit these identifiers instead a the type's OID, thereby removing the dependency on OID from clients entirely. best regards, Florian Pflug -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
On Wed, Dec 15, 2010 at 10:00 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: Well then we need a reliable way to identify a type. What would satisfy you? An oid ? Wrong probem. What we need is a way to identify a type without knowing in advance what its OID is. In other words, we need to distinguish between the hstore type that is shipped in contrib, and some stupid DBA who types CREATE DOMAIN hstore as text. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Complier warnings on mingw gcc 4.5.0
Excerpts from Andrew Dunstan's message of mié dic 15 02:08:24 -0300 2010: On 12/14/2010 12:42 PM, Tom Lane wrote: Another line of attack is that we know from the response packet that the failure is being reported at guc.c:4794. It would be really useful to know what the call stack is there. Could you change that elog to an elog(PANIC) and get a stack trace from the ensuing core dump? That didn't work. But git bisect says it's this commit that's to blame: https://github.com/postgres/postgres/commit/e710b65c1c56ca7b91f662c63d37ff2e72862a94 Hmm I wonder if this is reproducible in a non-Windows EXEC_BACKEND scenario. This bug seems closely related to process_postgres_switches. I guess it'd be useful to add some debugging printouts there to figure out what's being passed the second time around. -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables
On Wed, Dec 15, 2010 at 4:20 AM, Simon Riggs si...@2ndquadrant.com wrote: On Sat, 2010-11-13 at 19:16 -0500, Robert Haas wrote: 3. The third patch (relax-sync-commit-v1) allows asynchronous commit even when synchronous_commit=on if the transaction has not written WAL. Of course, a read-only transaction won't even have an XID and therefore won't need a commit record, so what this is really doing is allowing transactions that have written only to temp - or unlogged - tables to commit asynchronously. I like this, great idea. Avoiding the commit record entirely will break Hot Standby though, since we rely on the assumption that all xids that are assigned are also logged. The xids would be known assigned, yet since they never actually appear they will clog up the machinery (pun unintended). Uggh, that's a really, really bad pun. I made the same observation to Tom somewhere-or-other (must have been a different thread because I don't see it on this one), along with the further observation that we actually could suppress the commit record entirely if wal_level hot_standby, but I'm not sure there's enough benefit to doing that to worry about the additional complexity. Changing it from a foreground flush to a background flush already wins so much that I don't really see the point of doing anything further. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
Robert Haas robertmh...@gmail.com writes: I was asking what would satisfy you as regards a reliable way to identify a type, not what you think we should do about this particular proposal. Okay: a preassigned OID is safe. I haven't seen any other safe proposals. Relying on a non-reserved name is transparently unsafe. [ thinks for awhile ... ] You could imagine having the hstore module set up a rendezvous variable containing the OIDs of its type, its I/O functions, and anything else plpython might need to know. Except that the hstore C code doesn't know those OIDs either, at least not when first loaded. There's also the problem that you don't really want plpython's behavior suddenly changing when hstore happens to get loaded or first used. Another possibility is that you make the user tell you the fully-qualified name of the type: plpython.use_hstore = 'public.hstore' Such a GUC would also fix the backwards compatibility issues, since in the absence of a setting you'd continue to use the old behavior. But other than that configurability angle, this seems pretty ugly. Also you'd have to think about protecting yourself against a bad setting, ie the GUC specifies a type that's not hstore. That might not be a big problem though, as long as you aren't directly messing with the type's representation but just calling its I/O functions. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Default mode for shutdown
Excerpts from Robert Haas's message of mié dic 15 12:03:06 -0300 2010: Certainly, if you have an environment where people are mostly logging into the database directly (not through a connection pooler) and they do a few important queries and then disconnect, smart is a better default. But if you have an environment where (for whatever reason) long-lasting connections are common, smart is worse than useless. It occurs to me that we may need a new mode, which disconnects sessions that are not in a transaction (or as soon as they are) but leaves in-progress transactions alone; this could be the new default. Of course, this is much more difficult to implement than the current modes. -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
Robert Haas robertmh...@gmail.com writes:
Wrong probem. What we need is a way to identify a type without
knowing in advance what its OID is. In other words, we need to
distinguish between the hstore type that is shipped in contrib, and
some stupid DBA who types CREATE DOMAIN hstore as text.
Yeah, yeah. Now, what's wrong with the query I sent?
To ease discussion:
=# select objid
from pg_extension_objects('hstore')
where class = 'pg_type'::regclass and objdesc ~ '(^|\.)hstore$';
objid
---
16387
(1 row)
Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Complier warnings on mingw gcc 4.5.0
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Andrew Dunstan's message of mié dic 15 02:08:24 -0300 2010: That didn't work. But git bisect says it's this commit that's to blame: https://github.com/postgres/postgres/commit/e710b65c1c56ca7b91f662c63d37ff2e72862a94 Hmm I wonder if this is reproducible in a non-Windows EXEC_BACKEND scenario. I'm pretty sure I tried the no-flat-files code in that scenario while writing it. But it might be worth trying that again. You'd think though that if EXEC_BACKEND were sufficient to provoke it, all Windows builds would fail. I'm still mystified by what is the difference between Andrew's non-working installation and working mingw builds. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
2010/12/15 Florian Pflug f...@phlo.org On Dec15, 2010, at 02:14 , James William Pye wrote: On Dec 13, 2010, at 6:16 PM, Tom Lane wrote: how do you identify which type OID is really hstore? How about an identification field on pg_type? CREATE TYPE hstore ..., IDENTIFIER 'org.postgresql.hstore'; -- Where the identifier is an arbitrary string. +1 I've wanted something like this a few times when dealing with custom types within a client. A future protocol version might even transmit these identifiers instead a the type's OID, thereby removing the dependency on OID from clients entirely. In some another tread I've proposed CREATE TYPE ... WITH OID... but it was rejected and was proposed to cache OIDs on client side. It is right approach, IMO. But, IMO, comparing strings to determine type for each parameter is not very good idea because it is not so efficient as comparing integers, obviously. best regards, Florian Pflug -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- // Dmitriy.
Re: [HACKERS] hstores in pl/python
On 15/12/10 16:11, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: I was asking what would satisfy you as regards a reliable way to identify a type, not what you think we should do about this particular proposal. Okay: a preassigned OID is safe. I haven't seen any other safe proposals. Relying on a non-reserved name is transparently unsafe. We could preassign OIDs to contrib types, but that gives the not-contrib-nor-core types the cold shoulder. Another possibility is that you make the user tell you the fully-qualified name of the type: plpython.use_hstore = 'public.hstore' Such a GUC would also fix the backwards compatibility issues, since in the absence of a setting you'd continue to use the old behavior. I just had an illumination. The search path problem is the main issue, as (like you noticed), just calling I/O functions of a type should never give you anything worse than an ERROR. But other than that configurability angle, this seems pretty ugly. Also you'd have to think about protecting yourself against a bad setting, ie the GUC specifies a type that's not hstore. That might not be a big problem though, as long as you aren't directly messing with the type's representation but just calling its I/O functions. So how about just adding a text column to pg_type and a IDENTIFIER keywork to CREATE TYPE. It's not guaranteed to be unique, but isn't it pushing the argument to the extreme? Someone can change around bool and text type oids, too... And then hstore_plpython looks up the well-known identifier, sets up a RVV with the OID and everyone's happy. Cheers, Jan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
2010/12/15 Jan Urbański wulc...@wulczer.org On 15/12/10 16:11, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: I was asking what would satisfy you as regards a reliable way to identify a type, not what you think we should do about this particular proposal. Okay: a preassigned OID is safe. I haven't seen any other safe proposals. Relying on a non-reserved name is transparently unsafe. We could preassign OIDs to contrib types, but that gives the not-contrib-nor-core types the cold shoulder. Another possibility is that you make the user tell you the fully-qualified name of the type: plpython.use_hstore = 'public.hstore' Such a GUC would also fix the backwards compatibility issues, since in the absence of a setting you'd continue to use the old behavior. I just had an illumination. The search path problem is the main issue, as (like you noticed), just calling I/O functions of a type should never give you anything worse than an ERROR. But other than that configurability angle, this seems pretty ugly. Also you'd have to think about protecting yourself against a bad setting, ie the GUC specifies a type that's not hstore. That might not be a big problem though, as long as you aren't directly messing with the type's representation but just calling its I/O functions. So how about just adding a text column to pg_type and a IDENTIFIER keywork to CREATE TYPE. It's not guaranteed to be unique, but isn't it pushing the argument to the extreme? Someone can change around bool and text type oids, too... And then hstore_plpython looks up the well-known identifier, sets up a RVV with the OID and everyone's happy. How IDENTIFIER differs from name ? org.postgresql.hstore vs hstore ? Cheers, Jan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- // Dmitriy.
Re: [HACKERS] hstores in pl/python
On 15/12/10 16:25, Dmitriy Igrishin wrote: 2010/12/15 Jan Urbański wulc...@wulczer.org So how about just adding a text column to pg_type and a IDENTIFIER keywork to CREATE TYPE. It's not guaranteed to be unique, but isn't it pushing the argument to the extreme? Someone can change around bool and text type oids, too... And then hstore_plpython looks up the well-known identifier, sets up a RVV with the OID and everyone's happy. How IDENTIFIER differs from name ? org.postgresql.hstore vs hstore ? Hm, now that I think of it, the only real difference is that you don't use search_path to look it up. So public.hstore is just as good an identifier... I could live with plpython_hstore_type = public.hstore, I guess. hstore_plpython would look at that GUC, look up the type, set up a RVV containing the OID and plpython would use it. Cheers, Jan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] unlogged tables
On Wed, Dec 15, 2010 at 4:06 AM, Simon Riggs si...@2ndquadrant.com wrote: On Sat, 2010-11-13 at 20:55 -0500, Robert Haas wrote: I think that would be a recipe for bugs. Look at the three new macros I introduced. If you keep relistemp around, then any code which relies on it is likely testing for one of those three things, or maybe even something subtly different from any of them, as in the cases where I needed to add a switch statement. The way I see it, this is ultimately a four-level hierarchy That argument isn't clear enough to avoid me agreeing so far with Tom and Andrew that logged-ness is separate from temp-ness. As you say though, it might be a recipe for bugs, so please explain a little more. Sure. Most of the existing checks for rd_istemp were actually checking whether the relation required WAL-logging. If there's any third-party code out there that is checking rd_istemp, it likely also needs to be revised to check whether WAL-logging is needed, not whether the relation is temp. The way I've coded it, such code will fail to compile, and can be very easily fixed by substituting a call to RelationNeedsWAL() or RelationUsesLocalBuffers() or RelationUsesTempNamespace(), depending on which property the caller actually cares about. That's better than having the code compile, but then not work as expected. As of today, RelationNeedsWAL() always gives an answer which is directly opposite to the answer given by RelationUsesLocalBuffers() and RelationUsesTempNamespace(). But the main unlogged tables patch changes that. RelationNeedsWAL() will return true for permanent tables and false for unlogged and temp tables, while RelationUsesLocalBuffers() and RelationUsesTempNamespace() will return false for permanent and unlogged tables and true for temp tables. When and if we get global temporary tables, there will be a further split between RelationUsesLocalBuffers() and RelationUsesTempNamespace(). The former will return true for both global and local temporary tables, and the latter only for local temporary tables. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
On Wed, Dec 15, 2010 at 10:15 AM, Dimitri Fontaine
dimi...@2ndquadrant.fr wrote:
Robert Haas robertmh...@gmail.com writes:
Wrong probem. What we need is a way to identify a type without
knowing in advance what its OID is. In other words, we need to
distinguish between the hstore type that is shipped in contrib, and
some stupid DBA who types CREATE DOMAIN hstore as text.
Yeah, yeah. Now, what's wrong with the query I sent?
To ease discussion:
=# select objid
from pg_extension_objects('hstore')
where class = 'pg_type'::regclass and objdesc ~ '(^|\.)hstore$';
objid
---
16387
(1 row)
OK, so I guess your point is that I should read the whole email before
replying. :-)
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Default mode for shutdown
On Wed, Dec 15, 2010 at 10:11 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Robert Haas's message of mié dic 15 12:03:06 -0300 2010: Certainly, if you have an environment where people are mostly logging into the database directly (not through a connection pooler) and they do a few important queries and then disconnect, smart is a better default. But if you have an environment where (for whatever reason) long-lasting connections are common, smart is worse than useless. It occurs to me that we may need a new mode, which disconnects sessions that are not in a transaction (or as soon as they are) but leaves in-progress transactions alone; this could be the new default. Of course, this is much more difficult to implement than the current modes. That would probably be handy, though I think for my use cases fast would still be better, or smart with a 30-second timeout. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CommitFest wrap-up
On Wed, Dec 15, 2010 at 9:20 AM, Florian Pflug f...@phlo.org wrote: On Dec14, 2010, at 15:01 , Robert Haas wrote: On Tue, Dec 14, 2010 at 7:51 AM, Florian Pflug f...@phlo.org wrote: - serializable lock consistency - I am fairly certain this needs rebasing. I don't have time to deal with it right away. That sucks, because I think this is a really important change. I can try to find some time to update the patch if it suffers from bit-rot. Would that help? Yes! I've rebased the patch to the current HEAD, and re-run my FK concurrency test suite, available from https://github.com/fgp/fk_concurrency, to verify that things still work. Thanks, but, EWRONGTHREAD. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TABLE ... REPLACE WITH
On Wed, 2010-12-15 at 07:43 -0600, David Christensen wrote: Are there any considerations with toast tables and the inline line pointers for toasted tuples? Toast tables would be swapped as well. Toast pointers are only applicable within a relfilenode, so we could not do otherwise. -- Simon Riggs http://www.2ndQuadrant.com/books/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TABLE ... REPLACE WITH
On Wed, 2010-12-15 at 12:17 +0100, Csaba Nagy wrote: But one problem would be when the replaced table is the _parent_ for a foreign key relationship. I don't think you can have that constraint pre-verified on the replacement table and simply replacing the content could leave the child relations with orphans. Good point. The only sensible way to handle this is by putting the FK checks into check pending state (as discussed on a different thread). We would probably need to disallow FKs with DELETE or UPDATE CASCADE since it would be difficult to execute those. -- Simon Riggs http://www.2ndQuadrant.com/books/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
On Dec15, 2010, at 16:18 , Dmitriy Igrishin wrote: 2010/12/15 Florian Pflug f...@phlo.org On Dec15, 2010, at 02:14 , James William Pye wrote: On Dec 13, 2010, at 6:16 PM, Tom Lane wrote: how do you identify which type OID is really hstore? How about an identification field on pg_type? CREATE TYPE hstore ..., IDENTIFIER 'org.postgresql.hstore'; -- Where the identifier is an arbitrary string. I've wanted something like this a few times when dealing with custom types within a client. A future protocol version might even transmit these identifiers instead a the type's OID, thereby removing the dependency on OID from clients entirely. In some another tread I've proposed CREATE TYPE ... WITH OID... Yeah, and I believe type identifiers are probably what you were really looking for ;-) but it was rejected and was proposed to cache OIDs on client side. It is right approach, IMO. Yes, but to cache OIDs you first have to find them. As long as their name and schema are known, thats easy, but once they aren't you're pretty much screwed.Since CREATE EXTENSION is going to let you install an extension into any schema you want, not knowing the schema is going to be pretty common, I believe. Type identifiers would solve this, by providing an easy and unambiguous way to find specific types. But, IMO, comparing strings to determine type for each parameter is not very good idea because it is not so efficient as comparing integers, obviously. That's maybe an argument against a possible future protocol version that'd transfer type identifiers instead of OIDS. But not against associating type identifiers with types in the first place, since after your initial lookup you'd still be comparing OIDs. best regards, Florian Pflug -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CommitFest wrap-up
On Dec15, 2010, at 16:45 , Robert Haas wrote: On Wed, Dec 15, 2010 at 9:20 AM, Florian Pflug f...@phlo.org wrote: On Dec14, 2010, at 15:01 , Robert Haas wrote: On Tue, Dec 14, 2010 at 7:51 AM, Florian Pflug f...@phlo.org wrote: - serializable lock consistency - I am fairly certain this needs rebasing. I don't have time to deal with it right away. That sucks, because I think this is a really important change. I can try to find some time to update the patch if it suffers from bit-rot. Would that help? Yes! I've rebased the patch to the current HEAD, and re-run my FK concurrency test suite, available from https://github.com/fgp/fk_concurrency, to verify that things still work. Thanks, but, EWRONGTHREAD. Sorry for that. I wasn't sure whether to post this here or into the original thread, and it seems I ended up on the losing side of that 50-50 chance ;-) Want me to repost there, or just remember to use the correct thread next time? best regards, Florian Pflug -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CommitFest wrap-up
On Wed, Dec 15, 2010 at 10:57 AM, Florian Pflug f...@phlo.org wrote: On Dec15, 2010, at 16:45 , Robert Haas wrote: On Wed, Dec 15, 2010 at 9:20 AM, Florian Pflug f...@phlo.org wrote: On Dec14, 2010, at 15:01 , Robert Haas wrote: On Tue, Dec 14, 2010 at 7:51 AM, Florian Pflug f...@phlo.org wrote: - serializable lock consistency - I am fairly certain this needs rebasing. I don't have time to deal with it right away. That sucks, because I think this is a really important change. I can try to find some time to update the patch if it suffers from bit-rot. Would that help? Yes! I've rebased the patch to the current HEAD, and re-run my FK concurrency test suite, available from https://github.com/fgp/fk_concurrency, to verify that things still work. Thanks, but, EWRONGTHREAD. Sorry for that. I wasn't sure whether to post this here or into the original thread, and it seems I ended up on the losing side of that 50-50 chance ;-) Want me to repost there, or just remember to use the correct thread next time? Nah, don't bother reposting. It'd be helpful if you could add a link to that message on the CF app though. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Complier warnings on mingw gcc 4.5.0
On 12/15/2010 10:17 AM, Tom Lane wrote: Alvaro Herreraalvhe...@commandprompt.com writes: Excerpts from Andrew Dunstan's message of mié dic 15 02:08:24 -0300 2010: That didn't work. But git bisect says it's this commit that's to blame: https://github.com/postgres/postgres/commit/e710b65c1c56ca7b91f662c63d37ff2e72862a94 Hmm I wonder if this is reproducible in a non-Windows EXEC_BACKEND scenario. I'm pretty sure I tried the no-flat-files code in that scenario while writing it. But it might be worth trying that again. You'd think though that if EXEC_BACKEND were sufficient to provoke it, all Windows builds would fail. I'm still mystified by what is the difference between Andrew's non-working installation and working mingw builds. This is a new installation of Mingw. The buildfarm animals were set up years ago, with substantially older versions of Mingw. SO ISTM that either we have tickled a new bug of theirs or their new setup has tickled a bug of ours. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Complier warnings on mingw gcc 4.5.0
Alvaro Herrera alvhe...@commandprompt.com writes: This bug seems closely related to process_postgres_switches. I guess it'd be useful to add some debugging printouts there to figure out what's being passed the second time around. It strikes me that the most obvious source for a platform dependency there would be getopt(), in particular the arrangements to cause getopt to behave sanely when we invoke it on a different argc array the second time around. If that were failing for some reason, you could imagine getopt seeing 'postgres' as the next switch to parse, which could lead to the reported failure. Hence: 1. Is that build using src/port/getopt.c, or a library-supplied getopt? What about getopt_long.c? 2. Is HAVE_INT_OPTRESET getting defined? Should it be? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CommitFest wrap-up
On Dec15, 2010, at 17:17 , Robert Haas wrote: Nah, don't bother reposting. It'd be helpful if you could add a link to that message on the CF app though. Already done. Seems we've hit a race condition there - you must have overlooked the signalling the semaphore on my rooftop did to warn you... best regards, Florian Pflug -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Segfault related to pg_authid when running initdb from git master
On Wed, Dec 15, 2010 at 6:07 AM, Peter Geoghegan peter.geoghega...@gmail.com wrote: On 15 December 2010 01:35, Robert Haas robertmh...@gmail.com wrote: I am suspicious of the fact that you are invoking initdb as ./initdb. Is it possible you're invoking this from the build tree, and there's an installed copy out there that doesn't match, but is getting used? Like maybe in /usr/local/pgsql/bin? No, I'm not doing that. I'm running initdb from /usr/local/pgsql/bin (nothing pg related can be found in my $PATH), but it's the only copy on my system, which was installed from git master last night. It has debugging symbols, and I've actually re-created this from initdb's point of view within GDB with source level debugging. Well, something's clearly funky here because your initdb has debugging symbols but your postgres executable does not. I may be missing something obvious, but I don't see how that can happen without mixing up two different builds. Can you fire up gdb on this core dump, using gdb /usr/local/pgsql/bin/postgres /path/to/coredump? Or, another possibility is to run initdb with --noclean and then run the command, without routing the output to /dev/null: /usr/local/pgsql/bin/postgres --single -F -O -c search_path=pg_catalog -c exit_on_error=true template1 I cannot find the coredump. Perhaps it's a permissions issue. What do you think? It would presumably get dumped into the data directory. So if --noclean isn't used I expect it'll get nuked. Anyway, I have produced a useful backtrace by debugging postgres directly after running initdb with --noclean as described: [pe...@peter bin]$ /usr/local/pgsql/bin/postgres --single -F -O -c search_path=pg_catalog -c exit_on_error=true template1 Segmentation fault [pe...@peter bin]$ gdb postgres GNU gdb (GDB) Fedora (7.2-26.fc14) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type show copying and show warranty for details. This GDB was configured as x86_64-redhat-linux-gnu. For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/... Reading symbols from /usr/local/pgsql/bin/postgres...done. (gdb) set args --single -F -O -c search_path=pg_catalog -c exit_on_error=true template1 (gdb) start Temporary breakpoint 1 at 0x577360 Starting program: /usr/local/pgsql/bin/postgres --single -F -O -c search_path=pg_catalog -c exit_on_error=true template1 Temporary breakpoint 1, 0x00577360 in main () (gdb) c Continuing. Program received signal SIGSEGV, Segmentation fault. 0x0047615b in _bt_preprocess_keys () (gdb) bt #0 0x0047615b in _bt_preprocess_keys () #1 0x00475382 in _bt_first () #2 0x00473d71 in btgettuple () #3 0x006ba67c in FunctionCall2 () #4 0x0046e08a in index_getnext () #5 0x0046d556 in systable_getnext () #6 0x006a92bf in LookupOpclassInfo () #7 0x006a9a58 in RelationInitIndexAccessInfo () #8 0x006aa9cb in RelationBuildDesc () #9 0x006aabfd in load_critical_index () #10 0x006ac12a in RelationCacheInitializePhase3 () #11 0x006c19ca in InitPostgres () #12 0x0060058f in PostgresMain () #13 0x0057774d in main () Ugh. Maybe someone smarter can figure out what that means, but I have no clue. _bt_preprocess_keys() is a pretty good-sized function; there's no obvious way to know which pointer reference is blowing up without line-number information. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Complier warnings on mingw gcc 4.5.0
On 12/15/2010 11:12 AM, Tom Lane wrote: Alvaro Herreraalvhe...@commandprompt.com writes: This bug seems closely related to process_postgres_switches. I guess it'd be useful to add some debugging printouts there to figure out what's being passed the second time around. It strikes me that the most obvious source for a platform dependency there would be getopt(), in particular the arrangements to cause getopt to behave sanely when we invoke it on a different argc array the second time around. If that were failing for some reason, you could imagine getopt seeing 'postgres' as the next switch to parse, which could lead to the reported failure. Hence: 1. Is that build using src/port/getopt.c, or a library-supplied getopt? What about getopt_long.c? 2. Is HAVE_INT_OPTRESET getting defined? Should it be? I had the same thought. I did try forcing use of our getopt and getopt_long, without success, but didn't look at optreset. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TABLE ... REPLACE WITH
On Wed, Dec 15, 2010 at 11:30 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Heikki Linnakangas wrote: On 14.12.2010 20:27, Simon Riggs wrote: 1. Prepare new data into new_table and build indexes 2. Swap old for new BEGIN; DROP TABLE old_table; ALTER TABLE new_table RENAME to old_table; COMMIT; Step (2) works, but any people queuing to access the table will see ERROR: could not open relation with OID x Could we make that work without error? Well, that worked better for us than building up the new contents in a temporary table and doing the sequence Tom suggests, but to eliminate the above error we had to do: BEGIN; ALTER TABLE old_table RENAME TO dead_table; ALTER TABLE new_table RENAME TO old_table; COMMIT; -- Wait for all references to old OID to expire. DROP TABLE dead_table; Been there, done that. Didn't buy the post-card. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Complier warnings on mingw gcc 4.5.0
Andrew Dunstan and...@dunslane.net writes: On 12/15/2010 11:12 AM, Tom Lane wrote: 1. Is that build using src/port/getopt.c, or a library-supplied getopt? What about getopt_long.c? 2. Is HAVE_INT_OPTRESET getting defined? Should it be? I had the same thought. I did try forcing use of our getopt and getopt_long, without success, but didn't look at optreset. Do we use configure at all on a mingw build? If we don't, then HAVE_INT_OPTRESET is surely not getting defined. It looks to me like it might be a good idea to force HAVE_INT_OPTRESET on when we are using our own versions of getopt/getopt_long. If we don't set that, then correct behavior depends on the assumption that the internal variable place is pointing at a null when the second series of getopt calls starts. While I'm prepared to believe that the last call of getopt left it that way, it's not clear that we can safely assume that the underlying argv array hasn't been clobbered meanwhile. You might try adding some debug printouts to src/port/getopt.c to see if you can trace exactly what's happening there. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Complier warnings on mingw gcc 4.5.0
On Wed, Dec 15, 2010 at 17:43, Tom Lane t...@sss.pgh.pa.us wrote: Andrew Dunstan and...@dunslane.net writes: On 12/15/2010 11:12 AM, Tom Lane wrote: 1. Is that build using src/port/getopt.c, or a library-supplied getopt? What about getopt_long.c? 2. Is HAVE_INT_OPTRESET getting defined? Should it be? I had the same thought. I did try forcing use of our getopt and getopt_long, without success, but didn't look at optreset. Do we use configure at all on a mingw build? If we don't, then HAVE_INT_OPTRESET is surely not getting defined. We do use configure on mingw. The output from a regular mingw configure run formed the base for the config file we use for MSVC where we can't run it, but an actual mingw build will re-run configure every time. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Default mode for shutdown
On Dec 15, 2010, at 9:11 AM, Alvaro Herrera wrote: Excerpts from Robert Haas's message of mié dic 15 12:03:06 -0300 2010: Certainly, if you have an environment where people are mostly logging into the database directly (not through a connection pooler) and they do a few important queries and then disconnect, smart is a better default. But if you have an environment where (for whatever reason) long-lasting connections are common, smart is worse than useless. It occurs to me that we may need a new mode, which disconnects sessions that are not in a transaction (or as soon as they are) but leaves in-progress transactions alone; this could be the new default. Of course, this is much more difficult to implement than the current modes. +1; that would certainly be useful for us. -- Jim C. Nasby, Database Architect j...@nasby.net 512.569.9461 (cell) http://jim.nasby.net -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Complier warnings on mingw gcc 4.5.0
Magnus Hagander mag...@hagander.net writes: On Wed, Dec 15, 2010 at 17:43, Tom Lane t...@sss.pgh.pa.us wrote: Do we use configure at all on a mingw build? If we don't, then HAVE_INT_OPTRESET is surely not getting defined. We do use configure on mingw. The output from a regular mingw configure run formed the base for the config file we use for MSVC where we can't run it, but an actual mingw build will re-run configure every time. Hm. It still seems pretty likely to me that the root cause is a change in mingw's getopt library function, but I don't have a theory about the precise mechanism. Is there any convenient place where we can look at the current version of their library sources, as well as the version in use in the working buildfarm members? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Complier warnings on mingw gcc 4.5.0
On 12/15/2010 11:49 AM, Tom Lane wrote: Magnus Hagandermag...@hagander.net writes: On Wed, Dec 15, 2010 at 17:43, Tom Lanet...@sss.pgh.pa.us wrote: Do we use configure at all on a mingw build? If we don't, then HAVE_INT_OPTRESET is surely not getting defined. We do use configure on mingw. The output from a regular mingw configure run formed the base for the config file we use for MSVC where we can't run it, but an actual mingw build will re-run configure every time. Hm. It still seems pretty likely to me that the root cause is a change in mingw's getopt library function, but I don't have a theory about the precise mechanism. Is there any convenient place where we can look at the current version of their library sources, as well as the version in use in the working buildfarm members? I think you're probably right. narwhal reports having optreset, but my Mingw reports not having it, so this looks like a likely culprit. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
2010/12/15 Florian Pflug f...@phlo.org On Dec15, 2010, at 16:18 , Dmitriy Igrishin wrote: 2010/12/15 Florian Pflug f...@phlo.org On Dec15, 2010, at 02:14 , James William Pye wrote: On Dec 13, 2010, at 6:16 PM, Tom Lane wrote: how do you identify which type OID is really hstore? How about an identification field on pg_type? CREATE TYPE hstore ..., IDENTIFIER 'org.postgresql.hstore'; -- Where the identifier is an arbitrary string. I've wanted something like this a few times when dealing with custom types within a client. A future protocol version might even transmit these identifiers instead a the type's OID, thereby removing the dependency on OID from clients entirely. In some another tread I've proposed CREATE TYPE ... WITH OID... Yeah, and I believe type identifiers are probably what you were really looking for ;-) Indeed, but why OID cannot serve as identifier in this case ? Why to encode the code ? :-) but it was rejected and was proposed to cache OIDs on client side. It is right approach, IMO. Yes, but to cache OIDs you first have to find them. As long as their name and schema are known, thats easy, but once they aren't you're pretty much screwed.Since CREATE EXTENSION is going to let you install an extension into any schema you want, not knowing the schema is going to be pretty common, I believe. Agree. Type identifiers would solve this, by providing an easy and unambiguous way to find specific types. Agree with 1st assertion but disagree with 2nd. If I understand correctly, identifier is a second name for type (object), but Java-styled, right ? It probably does solve the problem if there are will be convention that types org.postgresql.* are reserved. But why not reserve name of type hstore and prevent the user to create type with this reserved name ? All this tells me one thing - to avoid conflicts of naming of specific types it is necessary to make them built-in. But, IMO, comparing strings to determine type for each parameter is not very good idea because it is not so efficient as comparing integers, obviously. That's maybe an argument against a possible future protocol version that'd transfer type identifiers instead of OIDS. But not against associating type identifiers with types in the first place, since after your initial lookup you'd still be comparing OIDs. best regards, Florian Pflug -- // Dmitriy.
Re: [HACKERS] Segfault related to pg_authid when running initdb from git master
On 15 December 2010 16:26, Robert Haas robertmh...@gmail.com wrote: On Wed, Dec 15, 2010 at 6:07 AM, Peter Geoghegan peter.geoghega...@gmail.com wrote: On 15 December 2010 01:35, Robert Haas robertmh...@gmail.com wrote: I am suspicious of the fact that you are invoking initdb as ./initdb. Is it possible you're invoking this from the build tree, and there's an installed copy out there that doesn't match, but is getting used? Like maybe in /usr/local/pgsql/bin? No, I'm not doing that. I'm running initdb from /usr/local/pgsql/bin (nothing pg related can be found in my $PATH), but it's the only copy on my system, which was installed from git master last night. It has debugging symbols, and I've actually re-created this from initdb's point of view within GDB with source level debugging. Well, something's clearly funky here because your initdb has debugging symbols but your postgres executable does not. I may be missing something obvious, but I don't see how that can happen without mixing up two different builds. Just to make sure that I'm not going crazy, I did a git pull, rebuilt pg passing --enable-debug and --enable-casssert to configure as before, followed by make make install. Then I tried this: [pe...@peter bin]$ pwd /usr/local/pgsql/bin [pe...@peter bin]$ ls -l total 7720 -rwxr-xr-x. 1 root root 53977 Dec 15 16:47 clusterdb -rwxr-xr-x. 1 root root 55058 Dec 15 16:47 createdb -rwxr-xr-x. 1 root root 58351 Dec 15 16:47 createlang -rwxr-xr-x. 1 root root 58036 Dec 15 16:47 createuser -rwxr-xr-x. 1 root root 53380 Dec 15 16:47 dropdb -rwxr-xr-x. 1 root root 62052 Dec 15 16:47 droplang -rwxr-xr-x. 1 root root 53382 Dec 15 16:47 dropuser -rwxr-xr-x. 1 root root 707190 Dec 15 16:47 ecpg -rwxr-xr-x. 1 root root 123447 Dec 15 16:47 initdb -rwxr-xr-x. 1 root root 26435 Dec 15 16:47 pg_config -rwxr-xr-x. 1 root root 25229 Dec 15 16:47 pg_controldata -rwxr-xr-x. 1 root root 73784 Dec 15 16:47 pg_ctl -rwxr-xr-x. 1 root root 301781 Dec 15 16:47 pg_dump -rwxr-xr-x. 1 root root 75323 Dec 15 16:47 pg_dumpall -rwxr-xr-x. 1 root root 32015 Dec 15 16:47 pg_resetxlog -rwxr-xr-x. 1 root root 131867 Dec 15 16:47 pg_restore -rwxr-xr-x. 1 root root 91006 Dec 6 11:34 pg_upgrade -rwxr-xr-x. 1 root root 5380671 Dec 15 16:47 postgres lrwxrwxrwx. 1 root root 8 Dec 15 16:47 postmaster - postgres -rwxr-xr-x. 1 root root 398677 Dec 15 16:47 psql -rwxr-xr-x. 1 root root 55257 Dec 15 16:47 reindexdb -rwxr-xr-x. 1 root root 32410 Dec 15 16:47 vacuumdb [pe...@peter bin]$ which postgres /usr/bin/which: no postgres in (/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/peter/bin) [pe...@peter bin]$ which initdb /usr/bin/which: no initdb in (/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/peter/bin) Observe that the initdb and postgres timestamps are the same. This laptop is less than 2 weeks old, and has never had any postgres packages installed on it. I can once again reproduce the problem, exactly as before. My postgres executable does have debugging symbols, just less than initdb (I'm not sure what the exact term is, but it just lacks line information while having some debugging symbols). I cannot find the coredump. Perhaps it's a permissions issue. What do you think? It would presumably get dumped into the data directory. So if --noclean isn't used I expect it'll get nuked. It isn't there...it just looks like a virginal PGDATA directory. Ugh. Maybe someone smarter can figure out what that means, but I have no clue. _bt_preprocess_keys() is a pretty good-sized function; there's no obvious way to know which pointer reference is blowing up without line-number information. That's a pity, because I don't have a clue how to get line number information. I could always try printf() debugging, but I really shouldn't have to. -- Regards, Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Instrument checkpoint sync calls
On Wed, Dec 15, 2010 at 9:22 AM, Greg Smith g...@2ndquadrant.com wrote: patch I submit. Doesn't seem worth going through the trouble of committing that minor rework on its own, I'll slip it into the next useful thing that touches this area I do. Thanks for the hint, this would work better than what I did. Well, if I'm the one committing it, I'll pull that part out again and commit it separately. Not sure if that affects your calculus, but I much prefer patches that don't try to do ancillary things along the way. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
On Dec15, 2010, at 18:33 , Dmitriy Igrishin wrote: 2010/12/15 Florian Pflug f...@phlo.org On Dec15, 2010, at 16:18 , Dmitriy Igrishin wrote: 2010/12/15 Florian Pflug f...@phlo.org On Dec15, 2010, at 02:14 , James William Pye wrote: On Dec 13, 2010, at 6:16 PM, Tom Lane wrote: how do you identify which type OID is really hstore? How about an identification field on pg_type? CREATE TYPE hstore ..., IDENTIFIER 'org.postgresql.hstore'; -- Where the identifier is an arbitrary string. I've wanted something like this a few times when dealing with custom types within a client. A future protocol version might even transmit these identifiers instead a the type's OID, thereby removing the dependency on OID from clients entirely. In some another tread I've proposed CREATE TYPE ... WITH OID... Yeah, and I believe type identifiers are probably what you were really looking for ;-) Indeed, but why OID cannot serve as identifier in this case ? Why to encode the code ? :-) Because there are only 2^32 OIDs, so if people start picking them at random, sooner or later there will be collisions. Type identifiers would solve this, by providing an easy and unambiguous way to find specific types. Agree with 1st assertion but disagree with 2nd. If I understand correctly, identifier is a second name for type (object), but Java-styled, right ? It probably does solve the problem if there are will be convention that types org.postgresql.* are reserved. Yeah, that'd be the idea. If everyone uses reversed DNS-style names, and everyone picks a name belonging to a DNS zone under his control, there cannot be any collisions. At least for java packages, this seems to work pretty nicely. But why not reserve name of type hstore and prevent the user to create type with this reserved name ? All this tells me one thing - to avoid conflicts of naming of specific types it is necessary to make them built-in. None of these solutions scale well. best regards, Florian Pflug -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] range intervals in window function frames
Hello, all! I first posted this to pgsql-general, but didn't get a definitive answer to my question concerning if a window function feature is scheduled or being worked on for 9.x. -- I need to do moving averages over time series data and was hoping window functions could solve the problem for me, but it doesn't look like 8.4 or even 9.0 implementations are quite there, yet. Currently, if I have this table: create table sample ( tstimestamp, value integer ); create index sample_ts on sample (ts); and say I want a moving average of value over a fixed interval of five minutes (note that this could mean varying numbers of records in each frame), then I can do this: select *, avg_over_interval(ts, interval '5 min') from sample order by ts; Where avg_over_interval() is defined like this: create or replace function avg_over_interval(timestamp, interval) returns numeric as $$ select avg(value) from sample where (($1-$2) = ts) and (ts = $1); $$ language sql; What I would LIKE to do is this: select *, avg(ts) over(order by ts range (interval '5 min') preceding) from sample order by ts; Which is way cleaner and, I assume, more efficient. Questions: 1) Is there active work on window functions with frames over interval ranges? 2) If not, how can I help with that? 3) Until the functionality is in 9.x, can I make what I'm doing more efficient? Is there a better way to do this without window functions? (I tried an inline subquery instead of the function call, but it was twice as slow as the function.) Thanks all for you help. Dan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Segfault related to pg_authid when running initdb from git master
On Wed, Dec 15, 2010 at 12:39 PM, Peter Geoghegan peter.geoghega...@gmail.com wrote: Observe that the initdb and postgres timestamps are the same. Hrm. I cannot find the coredump. Perhaps it's a permissions issue. What do you think? It would presumably get dumped into the data directory. So if --noclean isn't used I expect it'll get nuked. It isn't there...it just looks like a virginal PGDATA directory. Double hrm. I have no idea how you can be getting line number information for initdb but not postgres. I think what you're getting from postgres is normally what I'd expect to see without --enable-debug. It sounds like you are doing it right, but I have no explanation for the results. What distro are you using? This can't be broken across the board, given the lack of metoos. Can you use git bisect to figure out which commit broke it? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
Robert Haas robertmh...@gmail.com writes:
Well then we need a reliable way to identify a type. What would satisfy you?
An oid ?
=# select objid
from pg_extension_objects('hstore')
where class = 'pg_type'::regclass
and objdesc ~ '(^|\.)hstore$';
objid
---
16387
(1 row)
You have 4 types in there so you want to be somewhat careful here…
=# select * from pg_extension_objects('hstore') where class =
'pg_type'::regclass;
class | classid | objid | objdesc
-+-+---+--
pg_type |1247 | 16387 | type utils.hstore
pg_type |1247 | 16392 | type utils.hstore[]
pg_type |1247 | 16466 | type utils.ghstore
pg_type |1247 | 16469 | type utils.ghstore[]
(4 rows)
Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Default mode for shutdown
On Wed, Dec 15, 2010 at 9:57 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Dec 15, 2010 at 9:47 AM, Tom Lane t...@sss.pgh.pa.us wrote: Yeah, and more to the point, do I want to finish whatever I was doing in that window? Fast-by-default is a nice hammer to swing, but one day you'll pound your finger. I guess. I've pounded my finger enough time with the current default that I'd be willing to try a different size hammer. The scenario you describe has yet to occur in 10+ years of using the product, but obviously not everyone's experience will match on this point. I think the ultimate basis for the way it's set up now is the mantra of be safe by default; which I believe I've heard you repeating in other contexts. Between that principle and the backwards-compatibility hazards, I really don't think there's adequate justification for changing this. Backwards compatibility is, I think, a reasonable argument for maintaining the current default. However, I don't agree that the current behavior is safe by default. What often happens is that the system gets stuck in a state where the existing connections will never terminate (or not for a long time) but new connections aren't accepted either. So you're sitting there waiting for the database to shut down - which it never does - meanwhile, half the people hitting your web site are getting DOS'd. Certainly, if you have an environment where people are mostly logging into the database directly (not through a connection pooler) and they do a few important queries and then disconnect, smart is a better default. But if you have an environment where (for whatever reason) long-lasting connections are common, smart is worse than useless. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] V3: Idle in transaction cancellation
On Wednesday 15 December 2010 15:40:20 Robert Haas wrote: On Wed, Dec 15, 2010 at 7:47 AM, Andres Freund and...@anarazel.de wrote: I thought about doing that first. Btw, LOG_NO_CLIENT is just a more abstracted way of what COMERROR did before... Hmm, but it must not be quite the same, because that didn't require the silent_error_while_idle flag. True. Thats a separate thing. Yeah. I'll try to find some time to think about this some more. It would sure be nice if we could find a solution that's a bit conceptually cleaner, even if it basically works the same way as what you've done here. I would like that as well. I am not sure you can achieve that in a reasonable amount of work. At least I couldn't. Is there a way that errstart() and/or errfinish() can know enough about the state of the communication with the frontend to decide whether to suppress edata-output_to_client? In other words, instead of explicitly passing in a flag that says whether to inform the client, it would be better for the error-reporting machinery to intrinsically know whether it's right to send_message_to_frontend(). Otherwise, an error thrown from an unexpected location might not have the flag set correctly. Currently there are no other locations where we errors could get thrown at that point but I see where youre going. You could use DoingCommandRead to solve that specific use-case, but the COMERROR ones I don't see as being replaced that easily. We could introduce something like NoLogToClientBegin(); NoLogToClientEnd(); int NoLogToClientCntr = 0; but that sounds like overdoing it for me. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Default mode for shutdown
Robert Haas robertmh...@gmail.com writes: On Wed, Dec 15, 2010 at 9:47 AM, Tom Lane t...@sss.pgh.pa.us wrote: Yeah, and more to the point, do I want to finish whatever I was doing in that window? Fast-by-default is a nice hammer to swing, but one day you'll pound your finger. I guess. I've pounded my finger enough time with the current default that I'd be willing to try a different size hammer. The scenario you describe has yet to occur in 10+ years of using the product, but obviously not everyone's experience will match on this point. I think the ultimate basis for the way it's set up now is the mantra of be safe by default; which I believe I've heard you repeating in other contexts. Between that principle and the backwards-compatibility hazards, I really don't think there's adequate justification for changing this. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
2010/12/15 Florian Pflug f...@phlo.org On Dec15, 2010, at 18:33 , Dmitriy Igrishin wrote: 2010/12/15 Florian Pflug f...@phlo.org On Dec15, 2010, at 16:18 , Dmitriy Igrishin wrote: 2010/12/15 Florian Pflug f...@phlo.org On Dec15, 2010, at 02:14 , James William Pye wrote: On Dec 13, 2010, at 6:16 PM, Tom Lane wrote: how do you identify which type OID is really hstore? How about an identification field on pg_type? CREATE TYPE hstore ..., IDENTIFIER 'org.postgresql.hstore'; -- Where the identifier is an arbitrary string. I've wanted something like this a few times when dealing with custom types within a client. A future protocol version might even transmit these identifiers instead a the type's OID, thereby removing the dependency on OID from clients entirely. In some another tread I've proposed CREATE TYPE ... WITH OID... Yeah, and I believe type identifiers are probably what you were really looking for ;-) Indeed, but why OID cannot serve as identifier in this case ? Why to encode the code ? :-) Because there are only 2^32 OIDs, so if people start picking them at random, sooner or later there will be collisions. Yes, but range of PostgreSQL's OIDs can be reserved. One or even ten millions, e.g. can be enough. Type identifiers would solve this, by providing an easy and unambiguous way to find specific types. Agree with 1st assertion but disagree with 2nd. If I understand correctly, identifier is a second name for type (object), but Java-styled, right ? It probably does solve the problem if there are will be convention that types org.postgresql.* are reserved. Yeah, that'd be the idea. If everyone uses reversed DNS-style names, and everyone picks a name belonging to a DNS zone under his control, there cannot be any collisions. At least for java packages, this seems to work pretty nicely. But why not reserve name of type hstore and prevent the user to create type with this reserved name ? All this tells me one thing - to avoid conflicts of naming of specific types it is necessary to make them built-in. None of these solutions scale well. Well, If there are will be identifiers for each type, e.g. org.postgresql.integer, why they need to be built-in ? For historical reasons ? :-) Let them also be in contribs... best regards, Florian Pflug -- // Dmitriy.
[HACKERS] getting composite types info from libpq
Hello,
when a query returns a composite type, the libpq PQftype() function
reports the oid of the record type. In psycopg:
cur.execute(select (1,2))
cur.description
(('row', 2249, None, -1, None, None, None),)
test=# select typname from pg_type where oid = 2249;
typname
-
record
Is there a way to recursively retrieve the types for the record components?
Thanks,
-- Daniele
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Segfault related to pg_authid when running initdb from git master
Robert Haas robertmh...@gmail.com writes: What distro are you using? This can't be broken across the board, given the lack of metoos. Can you use git bisect to figure out which commit broke it? Before that, have you tried the old standby of make distclean and a full rebuild/reinstall? The lack of buildfarm confirmation makes me highly suspicious that there's any real problem. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] getting composite types info from libpq
On Wed, Dec 15, 2010 at 1:25 PM, Daniele Varrazzo
daniele.varra...@gmail.com wrote:
Hello,
when a query returns a composite type, the libpq PQftype() function
reports the oid of the record type. In psycopg:
cur.execute(select (1,2))
cur.description
(('row', 2249, None, -1, None, None, None),)
test=# select typname from pg_type where oid = 2249;
typname
-
record
Is there a way to recursively retrieve the types for the record components?
not without talking to the server, unless you had previously pulled
pg_attribute data.
select * from pg_attribute where attrelid = 2249;
This question is more appropriate for -general, but what are you trying to do?
merlin
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Complier warnings on mingw gcc 4.5.0
On 12/15/2010 12:28 PM, Andrew Dunstan wrote:
I think you're probably right. narwhal reports having optreset, but my
Mingw reports not having it, so this looks like a likely culprit.
And the attached hack allowed make check to succeed.
I think the logic in tcop/postgres.c and postmaster/postmaster.c is
probably wrong. If we are using our getopt/getopt_long, we want to be
setting optreset, whether or not configure found one in the system
libraries.
cheers
andrew
diff --git a/configure b/configure
index 08fd1c8..0cfcb9a 100755
--- a/configure
+++ b/configure
@@ -20758,6 +20758,11 @@ esac
fi
+if test $PORTNAME = win32 -a x$pgac_cv_var_int_optreset != xyes; then
+ LIBOBJS=$LIBOBJS getopt.$ac_objext getopt_long.$ac_objext
+fi
+
+
# Cygwin's erand48() is broken (always returns zero) in some releases,
# so force use of ours.
if test $PORTNAME = cygwin; then
diff --git a/src/backend/postmaster/postmaster.c
b/src/backend/postmaster/postmaster.c
index 90854f4..c2f0436 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -313,9 +313,7 @@ extern char *optarg;
extern int optind,
opterr;
-#ifdef HAVE_INT_OPTRESET
extern int optreset; /* might not be declared by
system headers */
-#endif
#ifdef USE_BONJOUR
static DNSServiceRef bonjour_sdref = NULL;
@@ -751,9 +749,7 @@ PostmasterMain(int argc, char *argv[])
* getopt(3) library so that it will work correctly in subprocesses.
*/
optind = 1;
-#ifdef HAVE_INT_OPTRESET
optreset = 1; /* some systems need this too */
-#endif
/* For debugging: display postmaster environment */
{
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index ff2e9bd..da7db16 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -78,9 +78,7 @@
extern char *optarg;
extern int optind;
-#ifdef HAVE_INT_OPTRESET
extern int optreset; /* might not be declared by
system headers */
-#endif
/*
@@ -3442,9 +3440,7 @@ process_postgres_switches(int argc, char *argv[],
GucContext ctx)
* or when this function is called a second time with another array.
*/
optind = 1;
-#ifdef HAVE_INT_OPTRESET
optreset = 1; /* some systems need this too */
-#endif
return dbname;
}
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Complier warnings on mingw gcc 4.5.0
Andrew Dunstan and...@dunslane.net writes: And the attached hack allowed make check to succeed. I think the logic in tcop/postgres.c and postmaster/postmaster.c is probably wrong. If we are using our getopt/getopt_long, we want to be setting optreset, whether or not configure found one in the system libraries. Yeah, that's what I suggested earlier; but if your build *wasn't* using our versions before, we're still no closer to understanding why it was failing then. Another small problem is that a close inspection of our getopt.c says that it does reset place to point at a constant before returning -1, in every path except the -- case which I doubt is being invoked. So my idea that we were clobbering argv underneath it doesn't seem to hold up. I'm still feeling that we don't understand what's happening. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] V3: Idle in transaction cancellation
On Wed, Dec 15, 2010 at 10:02 AM, Andres Freund and...@anarazel.de wrote: Is there a way that errstart() and/or errfinish() can know enough about the state of the communication with the frontend to decide whether to suppress edata-output_to_client? In other words, instead of explicitly passing in a flag that says whether to inform the client, it would be better for the error-reporting machinery to intrinsically know whether it's right to send_message_to_frontend(). Otherwise, an error thrown from an unexpected location might not have the flag set correctly. You could use DoingCommandRead to solve that specific use-case, but the COMERROR ones I don't see as being replaced that easily. Well, again, I'm not an expert on this, but why would we need to unify the two mechanisms? Asynchronous rollbacks (what we're trying to do here) and protocol violations (which is what COMMERROR looks to be used for) are really sort of different. I'm not really sure we need to handle them in the same way. Let's think about a recovery conflict where ProcessInterrupts() has been called. Right now, if that situation occurs and we are not DoingCommandRead, then we just throw an error. That's either safe, or an already-existing bug. So the question is what to do if we ARE DoingCommandRead. Right now, we throw a fatal error. There's no comment explaining why, but I'm guessing that the reason is the same problem we're trying to fix here: the protocol state gets confused - but if we throw a FATAL then the client goes away and we don't have to worry about it any more. Our goal here, as I understand it, is to handle that case without a FATAL. So let's see... if we're DoingCommandRead at that point, and whereToSendOutput == DestRemote then we set whereToSendOutput = DestNone before throwing the error, and restore it just after we reset DoingCommandRead? stabs blindly at target -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
On Wed, Dec 15, 2010 at 1:22 PM, Dmitriy Igrishin dmit...@gmail.com wrote: Yes, but range of PostgreSQL's OIDs can be reserved. One or even ten millions, e.g. can be enough. No, they can't. PostgreSQL is already deployed without any such restriction. You can reserve those OIDs because they may already be in use on any given system. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
On Wed, Dec 15, 2010 at 2:14 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Dec 15, 2010 at 1:22 PM, Dmitriy Igrishin dmit...@gmail.com wrote: Yes, but range of PostgreSQL's OIDs can be reserved. One or even ten millions, e.g. can be enough. No, they can't. PostgreSQL is already deployed without any such restriction. You can reserve those OIDs because they may already be in use on any given system. Err, you CAN'T reserve these OIDs because blah blah. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
2010/12/15 Pavel Stehule pavel.steh...@gmail.com 2010/12/15 Dmitriy Igrishin dmit...@gmail.com: 2010/12/15 Florian Pflug f...@phlo.org On Dec15, 2010, at 18:33 , Dmitriy Igrishin wrote: 2010/12/15 Florian Pflug f...@phlo.org On Dec15, 2010, at 16:18 , Dmitriy Igrishin wrote: 2010/12/15 Florian Pflug f...@phlo.org On Dec15, 2010, at 02:14 , James William Pye wrote: On Dec 13, 2010, at 6:16 PM, Tom Lane wrote: how do you identify which type OID is really hstore? How about an identification field on pg_type? CREATE TYPE hstore ..., IDENTIFIER 'org.postgresql.hstore'; -- Where the identifier is an arbitrary string. I've wanted something like this a few times when dealing with custom types within a client. A future protocol version might even transmit these identifiers instead a the type's OID, thereby removing the dependency on OID from clients entirely. In some another tread I've proposed CREATE TYPE ... WITH OID... Yeah, and I believe type identifiers are probably what you were really looking for ;-) Indeed, but why OID cannot serve as identifier in this case ? Why to encode the code ? :-) Because there are only 2^32 OIDs, so if people start picking them at random, sooner or later there will be collisions. Yes, but range of PostgreSQL's OIDs can be reserved. One or even ten millions, e.g. can be enough. Type identifiers would solve this, by providing an easy and unambiguous way to find specific types. Agree with 1st assertion but disagree with 2nd. If I understand correctly, identifier is a second name for type (object), but Java-styled, right ? It probably does solve the problem if there are will be convention that types org.postgresql.* are reserved. Yeah, that'd be the idea. If everyone uses reversed DNS-style names, and everyone picks a name belonging to a DNS zone under his control, there cannot be any collisions. At least for java packages, this seems to work pretty nicely. But why not reserve name of type hstore and prevent the user to create type with this reserved name ? All this tells me one thing - to avoid conflicts of naming of specific types it is necessary to make them built-in. None of these solutions scale well. Well, If there are will be identifiers for each type, e.g. org.postgresql.integer, why they need to be built-in ? For historical reasons ? :-) Let them also be in contribs... some types are used in system tables, so without support of these types, then you can't to add a new types. It's a egg-chicken problem So, the formal criterion to make the type built-in is the type is must be primitive ? Pavel best regards, Florian Pflug -- // Dmitriy. -- // Dmitriy.
Re: [HACKERS] hstores in pl/python
2010/12/15 Dmitriy Igrishin dmit...@gmail.com 2010/12/15 Robert Haas robertmh...@gmail.com On Wed, Dec 15, 2010 at 2:14 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Dec 15, 2010 at 1:22 PM, Dmitriy Igrishin dmit...@gmail.com wrote: Yes, but range of PostgreSQL's OIDs can be reserved. One or even ten millions, e.g. can be enough. No, they can't. PostgreSQL is already deployed without any such restriction. You can reserve those OIDs because they may already be in use on any given system. Err, you CAN'T reserve these OIDs because blah blah. Right. Proposed identifiers wins in this case. I mean Java-styled identifiers. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- // Dmitriy. -- // Dmitriy.
Re: [HACKERS] Complier warnings on mingw gcc 4.5.0
On 12/15/2010 02:06 PM, Tom Lane wrote: Andrew Dunstanand...@dunslane.net writes: And the attached hack allowed make check to succeed. I think the logic in tcop/postgres.c and postmaster/postmaster.c is probably wrong. If we are using our getopt/getopt_long, we want to be setting optreset, whether or not configure found one in the system libraries. Yeah, that's what I suggested earlier; but if your build *wasn't* using our versions before, we're still no closer to understanding why it was failing then. Another small problem is that a close inspection of our getopt.c says that it does reset place to point at a constant before returning -1, in every path except the -- case which I doubt is being invoked. So my idea that we were clobbering argv underneath it doesn't seem to hold up. I'm still feeling that we don't understand what's happening. Sure we are closer to understanding it. It seems quite clear to me that Mingw's getopt, which we have been using, has changed between versions, as indicated by the fact that on my mingw optreset is not found, but on narwhal it is found. I haven't looked into our getopt. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ALTER TABLE ... REPLACE WITH
On Wed, Dec 15, 2010 at 10:50 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-12-15 at 12:17 +0100, Csaba Nagy wrote: But one problem would be when the replaced table is the _parent_ for a foreign key relationship. I don't think you can have that constraint pre-verified on the replacement table and simply replacing the content could leave the child relations with orphans. Good point. The only sensible way to handle this is by putting the FK checks into check pending state (as discussed on a different thread). We would probably need to disallow FKs with DELETE or UPDATE CASCADE since it would be difficult to execute those. I'm still wondering if TRUNCATE CONCURRENTLY would be a more elegant solution. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CommitFest wrap-up
Robert Haas robertmh...@gmail.com writes: On Mon, Dec 13, 2010 at 12:37 PM, Robert Haas robertmh...@gmail.com wrote: - fix for seg picksplit function - I don't have confidence this change is for the best and can't take responsibility for it. It needs review by a committer who understands this stuff better than me and can determine whether or not the change is really an improvement. Still outstanding. I will take a look at that one --- it is a bug fix at bottom, so we can't just drop it for lack of reviewers. - Writeable CTEs - I think we need Tom to pick this one up. - Fix snapshot taking inconsistencies - Ready for committer. Can any committer pick this up? Will take a look at these two also. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
2010/12/15 Dmitriy Igrishin dmit...@gmail.com 2010/12/15 Pavel Stehule pavel.steh...@gmail.com 2010/12/15 Dmitriy Igrishin dmit...@gmail.com: 2010/12/15 Florian Pflug f...@phlo.org On Dec15, 2010, at 18:33 , Dmitriy Igrishin wrote: 2010/12/15 Florian Pflug f...@phlo.org On Dec15, 2010, at 16:18 , Dmitriy Igrishin wrote: 2010/12/15 Florian Pflug f...@phlo.org On Dec15, 2010, at 02:14 , James William Pye wrote: On Dec 13, 2010, at 6:16 PM, Tom Lane wrote: how do you identify which type OID is really hstore? How about an identification field on pg_type? CREATE TYPE hstore ..., IDENTIFIER 'org.postgresql.hstore'; -- Where the identifier is an arbitrary string. I've wanted something like this a few times when dealing with custom types within a client. A future protocol version might even transmit these identifiers instead a the type's OID, thereby removing the dependency on OID from clients entirely. In some another tread I've proposed CREATE TYPE ... WITH OID... Yeah, and I believe type identifiers are probably what you were really looking for ;-) Indeed, but why OID cannot serve as identifier in this case ? Why to encode the code ? :-) Because there are only 2^32 OIDs, so if people start picking them at random, sooner or later there will be collisions. Yes, but range of PostgreSQL's OIDs can be reserved. One or even ten millions, e.g. can be enough. Type identifiers would solve this, by providing an easy and unambiguous way to find specific types. Agree with 1st assertion but disagree with 2nd. If I understand correctly, identifier is a second name for type (object), but Java-styled, right ? It probably does solve the problem if there are will be convention that types org.postgresql.* are reserved. Yeah, that'd be the idea. If everyone uses reversed DNS-style names, and everyone picks a name belonging to a DNS zone under his control, there cannot be any collisions. At least for java packages, this seems to work pretty nicely. But why not reserve name of type hstore and prevent the user to create type with this reserved name ? All this tells me one thing - to avoid conflicts of naming of specific types it is necessary to make them built-in. None of these solutions scale well. Well, If there are will be identifiers for each type, e.g. org.postgresql.integer, why they need to be built-in ? For historical reasons ? :-) Let them also be in contribs... some types are used in system tables, so without support of these types, then you can't to add a new types. It's a egg-chicken problem So, the formal criterion to make the type built-in is the type is must be primitive ? I.e. the type for deploying system catalogs. Pavel best regards, Florian Pflug -- // Dmitriy. -- // Dmitriy. -- // Dmitriy.
Re: [HACKERS] ALTER TABLE ... REPLACE WITH
Heikki Linnakangas wrote: On 14.12.2010 20:27, Simon Riggs wrote: 1. Prepare new data into new_table and build indexes 2. Swap old for new BEGIN; DROP TABLE old_table; ALTER TABLE new_table RENAME to old_table; COMMIT; Step (2) works, but any people queuing to access the table will see ERROR: could not open relation with OID x Could we make that work without error? Well, that worked better for us than building up the new contents in a temporary table and doing the sequence Tom suggests, but to eliminate the above error we had to do: BEGIN; ALTER TABLE old_table RENAME TO dead_table; ALTER TABLE new_table RENAME TO old_table; COMMIT; -- Wait for all references to old OID to expire. DROP TABLE dead_table; We don't put foreign keys on the table we do this with; it's rebuilt from the related tables weekly -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Segfault related to pg_authid when running initdb from git master
Before that, have you tried the old standby of make distclean and a full rebuild/reinstall? The lack of buildfarm confirmation makes me highly suspicious that there's any real problem. That's fixed both problems. I should have tried it much sooner. I guess that even though the binaries built were new, they were somehow linked with one or more older, release object files. Thanks. -- Regards, Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Segfault related to pg_authid when running initdb from git master
On Wed, Dec 15, 2010 at 2:40 PM, Peter Geoghegan peter.geoghega...@gmail.com wrote: Before that, have you tried the old standby of make distclean and a full rebuild/reinstall? The lack of buildfarm confirmation makes me highly suspicious that there's any real problem. That's fixed both problems. I should have tried it much sooner. I guess that even though the binaries built were new, they were somehow linked with one or more older, release object files. Thanks. Gah. I assumed you had cleaned out your tree. Oh, well. If you don't use --enable-depend, you can get this kind of issue. Even if you do, it's worth trying a full clean out (I use git clean -dfx) if you get something weird. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] mvcc DML on the same row
Hi, I have an idea about mvcc and different DML of the same row in the same transaction. Normally when a backend do an unpdate on a row ( call it X ) , we done an insert and logical delete on this row (0,1,2..N are the version of the row) : X0 (delete old row) X1 (insert new row) if we continue the transaction and we do for example another update on this row (X) , we again redo the same operation: X0 (deleted old row) X1 (row inserted, NOW deleted) = not needed for rollback X2 (insert new row ) But why we need all these versions of the same row on table, if for rollback we need only the original row X (X0) ? So I think we need it in memory, not on physical space of table (ok there is the cache, but ..) or something similar, or this method is for transaction with isolation level at read uncommited? Kind Regards Matteo Durighetto --- desmodem...@gmail.com m.durighe...@miriade.it -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Segfault related to pg_authid when running initdb from git master
Excerpts from Peter Geoghegan's message of mié dic 15 16:40:41 -0300 2010: Before that, have you tried the old standby of make distclean and a full rebuild/reinstall? The lack of buildfarm confirmation makes me highly suspicious that there's any real problem. That's fixed both problems. I should have tried it much sooner. I guess that even though the binaries built were new, they were somehow linked with one or more older, release object files. Thanks. This is probably caused by failure to use the --enable-depend configure switch. I think we should try to make that the default on platforms that support it. It seems silly not to use it. -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot Standby: too many KnownAssignedXids
On Tue, Dec 7, 2010 at 3:42 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Ok, I've committed this patch now. I can confirm that I could continue replaying the logfiles on the standby host with this patch. Thanks a lot, Joachim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Segfault related to pg_authid when running initdb from git master
On 15 December 2010 19:43, Robert Haas robertmh...@gmail.com wrote: Gah. I assumed you had cleaned out your tree. Oh, well. If you don't use --enable-depend, you can get this kind of issue. Even if you do, it's worth trying a full clean out (I use git clean -dfx) if you get something weird. Thanks for the tip. I guess it simply didn't occur to me to make distclean because I made the rather questionable assumption that it's only necessary when there are weird linking issues. -- Regards, Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
On tis, 2010-12-14 at 11:52 -0500, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On mån, 2010-12-13 at 08:50 +0100, Jan Urbański wrote: It would be cool to be able to transparently use hstores as Python dictionaries and vice versa. It would be easy enough with hstore as a core type, but with hstore as an addon it's not that easy. I have been thinking about this class of problems for a while. I think the proper fix is to have a user-definable mapping between types and languages. It would be another pair of input/output functions, essentially. Interesting thought, but it still leaves you needing to solve the problem of interconnecting two optional addons ... First you create the language and the type (in any order), and then you create an additional SQL-level designation that connects the two. In fact, the SQL standard contains something very similar for connecting user-defined types to host languages. So adapting that syntax a little, it could work like this: CREATE LANGUAGE plpython; CREATE FUNCTION ... ... CREATE TYPE hstore ...; CREATE FUNCTION hstore_to_plpython(hstore) RETURNS internal ... CREATE FUNCTION plpython_to_hstore(internal) RETURNS plpython ... CREATE TRANSFORMS FOR hstore (TO plpython WITH hstore_to_plpython, FROM plpython WITH plpython_to_hstore); A shorter term solution that avoids creating a whole lot of SQL infrastructure might be to write out the same transform specification using a configuration variable, for example plpython.transforms = 'hstore:public.hstore_to_plpython:public.plpython_to_hstore,...' -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Complier warnings on mingw gcc 4.5.0
On 12/15/2010 02:22 PM, Andrew Dunstan wrote: On 12/15/2010 02:06 PM, Tom Lane wrote: Andrew Dunstanand...@dunslane.net writes: And the attached hack allowed make check to succeed. I think the logic in tcop/postgres.c and postmaster/postmaster.c is probably wrong. If we are using our getopt/getopt_long, we want to be setting optreset, whether or not configure found one in the system libraries. Yeah, that's what I suggested earlier; but if your build *wasn't* using our versions before, we're still no closer to understanding why it was failing then. Another small problem is that a close inspection of our getopt.c says that it does reset place to point at a constant before returning -1, in every path except the -- case which I doubt is being invoked. So my idea that we were clobbering argv underneath it doesn't seem to hold up. I'm still feeling that we don't understand what's happening. Sure we are closer to understanding it. It seems quite clear to me that Mingw's getopt, which we have been using, has changed between versions, as indicated by the fact that on my mingw optreset is not found, but on narwhal it is found. And here is where it changed: http://sourceforge.net/project/shownotes.php?release_id=24832 * A replacement implementation for the getopt() family of functions, adding support for the GNU getopt_long_only() function. Users should note that this intentionally *removes* support for the BSD or Mac OS-X specific, and non-standard, `optreset' global variable; to reset the getopt() scanner, use `optind = 0;' instead of relying on this non-standard, non-portable and now-unsupported feature. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstores in pl/python
Florian Pflug f...@phlo.org writes:
Not if CREATE EXTENSION allows you install hstore into an arbitrary schema.
It also allows you to change it after the fact, and to easily track it
down. Here's an updated version of the query to find the hstore type OID
reliably once we have extensions in:
dim=# SELECT t.oid
FROM pg_extension_objects('hstore') o
JOIN pg_type t ON t.oid = o.objid
AND o.classid = 'pg_type'::regclass
WHERE t.typname = 'hstore';
oid
---
16393
(1 row)
For listing all the hstore objects interactively, use \dx hstore.
For pl/python's purposes, requiring the DBA to set plpython_hstore_type
accordingly might work, but clients need to be able to reliably find hstore
too. For them, having to specify the schema of every non-core type your
database adapter might support isn't exactly ideal...
Another reason why you will like the extension's patch :)
If you think you need the schema where the extension's objects are
living, there it is (for interactive use, just issue \dx):
=# SELECT n.nspname, e.extname
FROM pg_catalog.pg_extension e
LEFT JOIN pg_catalog.pg_depend d ON d.objid = e.oid
AND d.refclassid = 'pg_catalog.pg_namespace'::regclass
LEFT JOIN pg_catalog.pg_namespace n ON n.oid = d.refobjid
WHERE extname = 'hstore';
nspname | extname
-+-
utils | hstore
(1 row)
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] BufFreelistLock
On Tue, Dec 14, 2010 at 1:42 PM, Jim Nasby j...@nasby.net wrote: On Dec 14, 2010, at 11:08 AM, Jeff Janes wrote: I wouldn't expect an increase in shared_buffers to make contention on BufFreelistLock worse. If the increased buffers are used to hold heavily-accessed data, then you will find the pages you want in shared_buffers more often, and so need to run the clock-sweep less often. That should make up for longer sweeps. But if the increased buffers are used to hold data that is just read once and thrown away, then the clock sweep shouldn't need to sweep very far before finding a candidate. Well, we're talking about a working set that's between 96 and 192G, but only 8G (or 28G) of shared buffers. So there's going to be a pretty large amount of buffer replacement happening. We also have 210 tables where the ratio of heap buffer hits to heap reads is over 1000, so the stuff that is in shared buffers probably keeps usage_count quite high. Put these two together, and we're probably spending a fairly significant amount of time running the clock sweep. The thing that makes me think the bottleneck is elsewhere is that increasing from 8G to 28G made it worse. If buffer unpins are happening at about the same rate, then my gut feeling is that the clock sweep has to do about the same amount of decrementing before it gets to a free buffer under steady state conditions. Whether it has to decrement 8G in buffers three and a half times each, or 28G of buffers one time each, it would do about the same amount of work. This is all hand waving, of course. Even excluding our admittedly unusual workload, there is still significant overhead in running the clock sweep vs just grabbing something off of the free list (assuming we had separate locks for the two operations). But do we actually know that? Doing a clock sweep is only a lot of overhead if it has to pass over many buffers in order to find a good one, and we don't know the numbers on that. I think you can sweep a lot of buffers for the overhead of a single contended lock. If the sweep and the freelist had separate locks, you still need to lock the freelist to add to it things discovered during the sweep. Does anyone know what the overhead of getting a block from the filesystem cache is? I did tests on this a few days ago. It took on average 20 microseconds per row to select one row via primary key when everything was in shared buffers. When everything was in RAM but not shared buffers, it took 40 microseconds. Of this, about 10 microseconds were the kernel calls to seek and read from OS cache to shared_buffers, and the other 10 microseconds is some kind of PG overhead, I don't know where. The timings are per select, not per page, and one select usually reads two pages, one for the index leaf and one for the table. This was all single-client usage on 2.8GHz AMD Opteron. Not all the components of the timings will scale equally with additional clients on additional CPUs of course. I think the time spent in the kernel calls to do the seek and read will scale better than most other parts. BTW, given our workload I can't see any way of running at debug2 without having a large impact on performance. As long as you are adding #define BGW_DEBUG and recompiling, you might as well promote all the DEBUG2 in src/backend/storage/buffer/bufmgr.c to DEBUG1 or LOG. I think this will only generate a couple log message per bgwriter_delay. That should be tolerable, especially for testing purposes. Cheers, Jeff -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Complier warnings on mingw gcc 4.5.0
Andrew Dunstan and...@dunslane.net writes: And here is where it changed: http://sourceforge.net/project/shownotes.php?release_id=24832 * A replacement implementation for the getopt() family of functions, adding support for the GNU getopt_long_only() function. Users should note that this intentionally *removes* support for the BSD or Mac OS-X specific, and non-standard, `optreset' global variable; to reset the getopt() scanner, use `optind = 0;' instead of relying on this non-standard, non-portable and now-unsupported feature. Great. So instead of a nonstandard but pretty portable API, they decided on a nonstandard interpretation of optind ... which absolutely will not work for our usage, because we need to be able to tell getopt to skip over --single, even if we were willing to figure out whether getopt behaves this way or the more usual way. Dolts. While I don't mind forcing use of our getopt() on mingw, I'm a mite concerned by the idea that this might represent an upstream change we'll soon see elsewhere, rather than just mingw-specific brain damage. Anybody know? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Complier warnings on mingw gcc 4.5.0
On 12/15/2010 03:52 PM, Tom Lane wrote: Andrew Dunstanand...@dunslane.net writes: And here is where it changed: http://sourceforge.net/project/shownotes.php?release_id=24832 * A replacement implementation for the getopt() family of functions, adding support for the GNU getopt_long_only() function. Users should note that this intentionally *removes* support for the BSD or Mac OS-X specific, and non-standard, `optreset' global variable; to reset the getopt() scanner, use `optind = 0;' instead of relying on this non-standard, non-portable and now-unsupported feature. Great. So instead of a nonstandard but pretty portable API, they decided on a nonstandard interpretation of optind ... which absolutely will not work for our usage, because we need to be able to tell getopt to skip over --single, even if we were willing to figure out whether getopt behaves this way or the more usual way. Dolts. While I don't mind forcing use of our getopt() on mingw, I'm a mite concerned by the idea that this might represent an upstream change we'll soon see elsewhere, rather than just mingw-specific brain damage. Anybody know? On my Fedora box, man 3 getopt says this: A program that scans multiple argument vectors, or rescans the same vector more than once, and wants to make use of GNU extensions such as '+' and '-' at the start of optstring, or changes the value of POSIXLY_CORRECT between scans, must reinitialize getopt() by resetting optind to 0, rather than the traditional value of 1. (Resetting to 0 forces the invocation of an internal initialization routine that rechecks POSIXLY_CORRECT and checks for GNU extensions in optstring.) Modulo the --single issue, we don't have to force use of our getopt on Mingw. This patch seems to work, at least to get regression working: diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c index 90854f4..9ae3767 100644 --- a/src/backend/postmaster/postmaster.c +++ b/src/backend/postmaster/postmaster.c @@ -753,6 +753,8 @@ PostmasterMain(int argc, char *argv[]) optind = 1; #ifdef HAVE_INT_OPTRESET optreset = 1;/* some systems need this too */ +#elsif defined (WIN32) !defined(_MSC_VER) +optind = 0; /* modern Mingw needs this instead */ #endif /* For debugging: display postmaster environment */ diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index ff2e9bd..ea4ae79 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -3444,6 +3444,8 @@ process_postgres_switches(int argc, char *argv[], GucContext ctx) optind = 1; #ifdef HAVE_INT_OPTRESET optreset = 1;/* some systems need this too */ +#elsif defined (WIN32) !defined(_MSC_VER) +optind = 0; /* modern Mingw needs this instead */ #endif cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Complier warnings on mingw gcc 4.5.0
Andrew Dunstan and...@dunslane.net writes: On my Fedora box, man 3 getopt says this: A program that scans multiple argument vectors, or rescans the same vector more than once, and wants to make use of GNU extensions such as '+' and '-' at the start of optstring, or changes the value of POSIXLY_CORRECT between scans, must reinitialize getopt() by resetting optind to 0, rather than the traditional value of 1. (Resetting to 0 forces the invocation of an internal initialization routine that rechecks POSIXLY_CORRECT and checks for GNU extensions in optstring.) Hmm, mine says the same, but it's not entirely clear how to parse the AND and OR conditions there. The fact that it works on Fedora suggests to me that the multiple vectors case is somehow ANDed with one of the other conditions. Anyway seems like the next step is to compare the Fedora getopt code with mingw's ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
