Re: [HACKERS] COMMUTATOR doesn't seem to work
On 12-06-23 12:17 AM, Tom Lane wrote: D'Arcy Cain da...@druid.net writes: On 12-06-22 07:09 PM, Robert Haas wrote: I think DirectionFunctionCall2 is what you want. Can you elaborate? I could not find a single hit in Google or the documentation search on the PG site and it does not appear anywhere in the source distribution. He meant DirectFunctionCall2 Still nothing in the documentation. At least the search box doesn't find it. -- D'Arcy J.M. Cain da...@druid.net | Democracy is three wolves http://www.druid.net/darcy/| and a sheep voting on +1 416 425 1212 (DoD#0082)(eNTP) | what's for dinner. IM: da...@vex.net -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] random failing builds on spoonbill - backends not exiting...
On 06/22/2012 11:53 PM, Tom Lane wrote: oh, and just for comparison's sake, what do the postmaster's signal masks look like? # ps -o pid,sig,sigcatch,sigignore,sigmask,command -p 18020 PID PENDING CAUGHT IGNORED BLOCKED COMMAND 180200 74084007 8972b0000 /home/pgbuild/pgbuildfarm/HEAD/pgsql.5709/src/interfaces/ecpg/test/./t... Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] random failing builds on spoonbill - backends not exiting...
On 06/22/2012 11:47 PM, Tom Lane wrote: Stefan Kaltenbrunner ste...@kaltenbrunner.cc writes: PID PENDING CAUGHT IGNORED BLOCKED COMMAND 12480 20004004 34084005 c942b002 fffefeff postgres: writer process 9841 20004004 34084007 c942b000 fffefeff postgres: wal writer process this seems to be SIGUSR1,SIGTERM and SIGQUIT OK, I looked up OpenBSD's signal numbers on the web. It looks to me like these two processes have everything blocked except KILL and STOP (which are unblockable of course). I do not see any place in the PG code that could possibly set such a mask (note that BlockSig should have more holes in it than that). So I'm thinking these must be blocked inside some system function that's installed a restrictive signal mask, or some such function forgot to restore the mask on exit. Could you gdb each of these processes and get a stack trace? background writer (12480): (gdb) bt #0 0x000208eb5928 in poll () from /usr/lib/libc.so.62.0 #1 0x00020a972b88 in _thread_kern_poll (wait_reqd=Variable wait_reqd is not available. ) at /usr/src/lib/libpthread/uthread/uthread_kern.c:784 #2 0x00020a973d04 in _thread_kern_sched (scp=0x0) at /usr/src/lib/libpthread/uthread/uthread_kern.c:384 #3 0x00020a96b35c in poll (fds=0xfffefa80, nfds=Variable nfds is not available. ) at /usr/src/lib/libpthread/uthread/uthread_poll.c:94 #4 0x00395538 in WaitLatchOrSocket (latch=0x212bdc97c, wakeEvents=25, sock=-1, timeout=1) at pg_latch.c:286 #5 0x00399800 in BackgroundWriterMain () at bgwriter.c:325 #6 0x00201850 in AuxiliaryProcessMain (argc=2, argv=0xfffefd98) at bootstrap.c:419 #7 0x003a1534 in StartChildProcess (type=BgWriterProcess) at postmaster.c:4518 #8 0x003a7574 in reaper (postgres_signal_arg=Variable postgres_signal_arg is not available. ) at postmaster.c:2385 #9 0x00020a974528 in _dispatch_signal (sig=20, scp=0x03e0) at /usr/src/lib/libpthread/uthread/uthread_sig.c:408 #10 0x00020a97462c in _dispatch_signals (scp=0x03e0) at /usr/src/lib/libpthread/uthread/uthread_sig.c:437 #11 0x00020a974e28 in _thread_sig_handler (sig=20, info=0x0420, scp=0x03e0) at /usr/src/lib/libpthread/uthread/uthread_sig.c:139 #12 signal handler called #13 _thread_kern_set_timeout (timeout=0x0630) at /usr/src/lib/libpthread/uthread/uthread_kern.c:989 #14 0x00020a96bc8c in select (numfds=9, readfds=0x0730, writefds=0x0, exceptfds=0x0, timeout=Variable timeout is not available. ) at /usr/src/lib/libpthread/uthread/uthread_select.c:85 #15 0x003a2894 in ServerLoop () at postmaster.c:1321 #16 0x003a45ac in PostmasterMain (argc=Variable argc is not available. ) at postmaster.c:1121 #17 0x00326df8 in main (argc=6, argv=0x14f8) at main.c:199 wal writer(9841): #0 0x000208eb5928 in poll () from /usr/lib/libc.so.62.0 #1 0x00020a972b88 in _thread_kern_poll (wait_reqd=Variable wait_reqd is not available. ) at /usr/src/lib/libpthread/uthread/uthread_kern.c:784 #2 0x00020a973d04 in _thread_kern_sched (scp=0x0) at /usr/src/lib/libpthread/uthread/uthread_kern.c:384 #3 0x00020a96b35c in poll (fds=0xfffefa80, nfds=Variable nfds is not available. ) at /usr/src/lib/libpthread/uthread/uthread_poll.c:94 #4 0x00395538 in WaitLatchOrSocket (latch=0x212bdc69c, wakeEvents=25, sock=-1, timeout=5000) at pg_latch.c:286 #5 0x003aa794 in WalWriterMain () at walwriter.c:301 #6 0x00201878 in AuxiliaryProcessMain (argc=2, argv=0xfffefd98) at bootstrap.c:430 #7 0x003a1534 in StartChildProcess (type=WalWriterProcess) at postmaster.c:4518 #8 0x003a7564 in reaper (postgres_signal_arg=Variable postgres_signal_arg is not available. ) at postmaster.c:2387 #9 0x00020a974528 in _dispatch_signal (sig=20, scp=0x03e0) at /usr/src/lib/libpthread/uthread/uthread_sig.c:408 #10 0x00020a97462c in _dispatch_signals (scp=0x03e0) at /usr/src/lib/libpthread/uthread/uthread_sig.c:437 #11 0x00020a974e28 in _thread_sig_handler (sig=20, info=0x0420, scp=0x03e0) at /usr/src/lib/libpthread/uthread/uthread_sig.c:139 #12 signal handler called #13 _thread_kern_set_timeout (timeout=0x0630) at /usr/src/lib/libpthread/uthread/uthread_kern.c:989 #14 0x00020a96bc8c in select (numfds=9, readfds=0x0730, writefds=0x0, exceptfds=0x0, timeout=Variable timeout is not available. ) at /usr/src/lib/libpthread/uthread/uthread_select.c:85 #15 0x003a2894 in ServerLoop () at postmaster.c:1321 #16 0x003a45ac in PostmasterMain (argc=Variable argc is not available. ) at postmaster.c:1121 #17 0x00326df8 in main (argc=6, argv=0x14f8) at main.c:199 Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:
Re: [HACKERS] Allow WAL information to recover corrupted pg_controldata
-Original Message- From: Robert Haas [mailto:robertmh...@gmail.com] Sent: Friday, June 22, 2012 8:59 PM To: Amit Kapila Cc: Tom Lane; Alvaro Herrera; Cédric Villemain; Pg Hackers Subject: Re: [HACKERS] Allow WAL information to recover corrupted pg_controldata On Fri, Jun 22, 2012 at 5:25 AM, Amit Kapila amit.kap...@huawei.com wrote: Based on the discussion and suggestions in this mail chain, following features can be implemented: 1. To compute the value of max LSN in data pages based on user input whether he wants it for an individual file, a particular directory or whole database. 2a. To search the available WAL files for the latest checkpoint record and prints the value. 2b. To search the available WAL files for the latest checkpoint record and recreates a pg_control file pointing at that checkpoint. I have kept both options to address different kind of corruption scenarios. I think I can see all of those things being potentially useful. I shall start working on design and usage(how to provide these options to users) of the features and present it once I am done. There are a couple of pending patches that will revise the WAL format slightly; not sure how much those are likely to interfere with any development you might do on (2) in the meantime. Thanks. I shall look into the patches (WAL Format change by Heikki and any other for Logical Replication) to see which changes can effect the implementation/design. With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] initdb and fsync
On Fri, 2012-06-22 at 10:04 -0400, Robert Haas wrote: This may be a stupid question, by why is it initdb's job to fsync the files the server creates, rather than the server's job? Normally we rely on the server to make its own writes persistent. That was my first reaction as well: http://archives.postgresql.org/message-id/1328468000.15224.32.camel@jdavis However, from the discussion it seems like it will be harder to do it that way (during normal operation, a checkpoint is what syncs the files; but as Tom points out, bootstrap mode does not). Also, I would expect the fsyncs to be in a less-controlled pattern, so it might perform more poorly. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] COMMUTATOR doesn't seem to work
2012/6/23 D'Arcy Cain da...@druid.net: On 12-06-23 12:17 AM, Tom Lane wrote: D'Arcy Cain da...@druid.net writes: On 12-06-22 07:09 PM, Robert Haas wrote: I think DirectionFunctionCall2 is what you want. Can you elaborate? I could not find a single hit in Google or the documentation search on the PG site and it does not appear anywhere in the source distribution. He meant DirectFunctionCall2 Still nothing in the documentation. At least the search box doesn't find it. Try in the search box of postgres doxygen documentation [1].. [1] http://doxygen.postgresql.org/ []s -- Dickson S. Guedes mail/xmpp: gue...@guedesoft.net - skype: guediz http://guedesoft.net - http://www.postgresql.org.br -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] COMMUTATOR doesn't seem to work
On 12-06-23 08:21 AM, Dickson S. Guedes wrote: Still nothing in the documentation. At least the search box doesn't find it. Try in the search box of postgres doxygen documentation [1].. That's source, not documentation. I already found it in the actual source files but that's not the same thing. For one thing, if it isn't documented then it may not be meant to be exposed to the API. For another, without documentation we can't tell what is guaranteed to work and what is an implementation detail. I know that I can fiddle about and try stuff until it works but I don't work for Microsoft. -- D'Arcy J.M. Cain da...@druid.net | Democracy is three wolves http://www.druid.net/darcy/| and a sheep voting on +1 416 425 1212 (DoD#0082)(eNTP) | what's for dinner. IM: da...@vex.net -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] COMMUTATOR doesn't seem to work
D'Arcy Cain da...@druid.net writes: On 12-06-23 08:21 AM, Dickson S. Guedes wrote: Try in the search box of postgres doxygen documentation [1].. That's source, not documentation. I already found it in the actual source files but that's not the same thing. For one thing, if it isn't documented then it may not be meant to be exposed to the API. For another, without documentation we can't tell what is guaranteed to work and what is an implementation detail. By and large, most of what you need to know to write C code for PG is not in the SGML documentation. I haven't seen any volunteers stepping up to write another thousand or two pages of formal docs, so that's how it's going to be for the foreseeable future. You need to be prepared to read the source code and especially to look at existing examples such as the contrib modules. As far as the not guaranteed to work angle is concerned, it's true, we could decide to change that. It's not very likely though: we do try to avoid breaking things that we expect are in use by third-party code, and anything that is in use by half a dozen contrib modules certainly qualifies. If you're thinking of doing something that you don't see lots of precedents for in contrib or the core datatypes, it would be reasonable to ask about it on -hackers first. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Resource Owner reassign Locks
On Thu, Jun 21, 2012 at 5:32 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 18.06.2012 13:59, Heikki Linnakangas wrote: On 10.06.2012 23:39, Jeff Janes wrote: I found the interface between resowner.c and lock.c a bit confusing. resowner.c would sometimes call LockReassignCurrentOwner() to reassign all the locks, and sometimes it would call LockReassignCurrentOwner() on each individual lock, with the net effect that's the same as calling LockReleaseCurrentOwner(). And it doesn't seem right for ResourceOwnerRemember/ForgetLock to have to accept a NULL owner. I rearranged that so that there's just a single LockReassignCurrentOwner() function, like before this patch. But it takes as an optional argument a list of locks held by the current resource owner. If the caller knows it, it can pass them to make the call faster, but if it doesn't it can just pass NULL and the function will traverse the hash table the old-fashioned way. I think that's a better API. Thank you, that does look much cleaner. Please take a look to see if I broke something. I hear no complaints, so committed. Thanks! Thanks. Just for the record, I'd previously promised some long running performance tests with my proposed -P option for pgbench, which are now done and showed a 0.2% degradation with my patch. With enough data collected, that difference is statistically significant, but probably not practically significant. It was with my original version, but I can't imagine your version being different in performance. Also, this test is very pessimistic. Since the primary key look-up in the pl/pgSQL look up runs in a portal each time, it pushes the locks to the parent each time. If the lookup was instead running as the inner side of a nested loop, it would not do the reassign on each loop. Cheers, Jeff -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] initdb and fsync
On mån, 2012-06-18 at 20:57 +0200, Andres Freund wrote: I don't think the difference in initdb cost is relevant when running the regression tests. Should it prove to be we can re-add -N after a week or two in the buildfarm machines. Keep in mind that the regression tests are not only run on the buildfarm, and that pg_regress is not only used for the main regression tests. When you're dealing with things in contrib or src/pl or external modules, then adding 3 seconds to a 5 second test is not so nice. I just remember that there were several OS specific regression when adding the pre-syncing for createdb. Sure, it would be useful to test this, but then we should also turn on fsync in the backend. Make it a separate slow mode (or a separate fast mode). -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CREATE FOREGIN TABLE LACUNA
On 23 March 2012 18:38, David Fetter da...@fetter.org wrote: On Thu, Mar 15, 2012 at 11:23:43AM -0300, Alvaro Herrera wrote: Excerpts from David Fetter's message of jue mar 15 02:28:28 -0300 2012: On Wed, Mar 14, 2012 at 12:06:20PM -0400, Robert Haas wrote: On Wed, Mar 14, 2012 at 10:22 AM, David Fetter da...@fetter.org wrote: I think that instead of inventing new grammar productions and a new node type for this, you should just reuse the existing productions for LIKE clauses and then reject invalid options during parse analysis. OK. Should I first merge CREATE FOREIGN TABLE with CREATE TABLE and submit that as a separate patch? I don't see any reason to do that. I merely meant that you could reuse TableLikeClause or maybe even TableElement in the grammer for CreateForeignTableStmt. Next WIP patch attached implementing this via reusing TableLikeClause and refactoring transformTableLikeClause(). What say? Looks much better to me, but the use of strcmp() doesn't look good. ISTM that stmtType is mostly used for error messages. I think you should add some kind of identifier (such as the original parser Node) into the CreateStmtContext so that you can do a IsA() test instead -- a bit more invasive as a patch, but much cleaner. Also the error messages need more work. How about this one? I spotted a couple of other issues during testing: * You're still allowing INCLUDING DEFAULTS and INCLUDING STORAGE, even though these options are not supported on foreign tables. * If I do INCLUDING ALL, I get an error because of the unsupported options. I think that ALL in this context needs to be made to mean all the options that foreign tables support (just COMMENTS at the moment). Regards, Dean -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] new --maintenance-db options
About the new --maintenance-db options: Why was this option not added to createuser and dropuser? In the original discussion[0] they were mentioned, but it apparently never made it into the code. I find the name to be unfortunate. For example, I think of running vacuum as maintenance. So running vacuumdb --maintenance-db=X would imply that the vacuum maintenance is done on X. In fact, the whole point of this option is to find out where the maintenance is to be run, not to run the maintenance. Maybe something like --initial-db would be better? What is the purpose of these options? The initial discussion was unclear on this. The documentation contains no explanation of why they should be used. If we want to really support the case where both postgres and template1 are removed, an environment variable might be more useful than requiring this to be typed out for every command. [0]: http://archives.postgresql.org/message-id/ca+tgmoacjwsis9nnqjgaaml1vg6c8b6o1ndgqnuca2gm00d...@mail.gmail.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] foreign key locks
Alvaro Herrera wrote: So here's the rebased version. I found a couple problems on `make check-world`. Attached is a patch to fix one of them. The other is on pg_upgrade, pasted below. + pg_upgrade -d /home/kevin/pg/master/contrib/pg_upgrade/tmp_check/data.old -D /home/kevin/pg/master/contrib/pg_upgrade/tmp_check/data -b /home/kevin/pg/master/contrib/pg_upgrade/tmp_check/install//home/ kevin/pg/master/Debug/bin -B /home/kevin/pg/master/contrib/pg_upgrade/tmp_check/install//home/ kevin/pg/master/Debug/bin Performing Consistency Checks - Checking current, bin, and data directories ok Checking cluster versions ok Checking database user is a superuser ok Checking for prepared transactions ok Checking for reg* system OID user data typesok Checking for contrib/isn with bigint-passing mismatch ok Creating catalog dump ok Checking for presence of required libraries ok Checking database user is a superuser ok Checking for prepared transactions ok If pg_upgrade fails after this point, you must re-initdb the new cluster before continuing. Performing Upgrade -- Analyzing all rows in the new cluster ok Freezing all rows on the new clusterok Deleting files from new pg_clog ok Copying old pg_clog to new server ok Setting next transaction ID for new cluster ok Deleting files from new pg_multixact/offsetsok Copying old pg_multixact/offsets to new server ok Deleting files from new pg_multixact/membersok Copying old pg_multixact/members to new server ok Setting next multixact ID and offset for new clustersh: /home/kevin/pg/master/contrib/pg_upgrade/tmp_check/install//home/ kevin/pg/master/Debug/bin: Permission denied *failure* Consult the last few lines of %s/pg_resetxlog -O %u -m %u,%u %s /dev/null for the probable cause of the failure. Failure, exiting make[2]: *** [check] Error 1 make[2]: Leaving directory `/home/kevin/pg/master/contrib/pg_upgrade' make[1]: *** [check-pg_upgrade-recurse] Error 2 make[1]: Leaving directory `/home/kevin/pg/master/contrib' make: *** [check-world-contrib-recurse] Error 2 I'm marking it as Waiting for Author since this seems like a must fix, but I'll keep looking at it, -Kevin fklocks-13a.diff Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] new --maintenance-db options
On Saturday, June 23, 2012, Peter Eisentraut wrote: About the new --maintenance-db options: Why was this option not added to createuser and dropuser? In the original discussion[0] they were mentioned, but it apparently never made it into the code. I find the name to be unfortunate. For example, I think of running vacuum as maintenance. So running vacuumdb --maintenance-db=X would imply that the vacuum maintenance is done on X. In fact, the whole point of this option is to find out where the maintenance is to be run, not to run the maintenance. Maybe something like --initial-db would be better? I'm not saying it's the best name, but I suspect the origin is pgAdmin which has used 'Maintenance DB' for 10+ years. There's likely a certain amount of familiarity with the term amonst pgAdmin users. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Re: [HACKERS] pg_dump and dependencies and --section ... it's a mess
On 06/22/2012 04:43 PM, Tom Lane wrote: Anyway, the attached patch does seem to fix the constraint bug. Looks sane to me. A possible objection to it is that there are now three different ways in which the pg_dump code knows which DO_XXX object types go in which dump section: the new addBoundaryDependencies() function knows this, the SECTION_xxx arguments to ArchiveEntry calls know it, and the sort ordering constants in pg_dump_sort.c have to agree too. My original idea was to add an explicit section field to DumpableObject to reduce the number of places that know this, but that would increase pg_dump's memory consumption still more, and yet still not give us a single point of knowledge. Has anybody got a better idea? Not off hand. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] CREATE FOREGIN TABLE LACUNA
Excerpts from Dean Rasheed's message of sáb jun 23 18:08:31 -0400 2012: I spotted a couple of other issues during testing: David, when you generate a new version of the patch, please also make sure to use RELKIND_RELATION and RELKIND_FOREIGN instead of 'r' and 'f'. * You're still allowing INCLUDING DEFAULTS and INCLUDING STORAGE, even though these options are not supported on foreign tables. Maybe the code should list options allowed instead of the ones disallowed. * If I do INCLUDING ALL, I get an error because of the unsupported options. I think that ALL in this context needs to be made to mean all the options that foreign tables support (just COMMENTS at the moment). I agree. -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
