Re: [HACKERS] Vacuum/visibility is busted
On Fri, Feb 8, 2013 at 10:08 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Alvaro Herrera escribió: Alvaro Herrera escribió: Hm, if the foreign key patch is to blame, this sounds like these tuples had a different set of XMAX hint bits and a different Xmax, and they were clobbered by something like vacuum or page pruning. Hm, I think heap_freeze_tuple is busted, yes. This patch seems to fix the problem for me. Please confirm. I'm trying to reason how this bug explains what we saw. In the test, we'd left with duplicate tuples. If I just take index 219 in the table as an example, that tuple had three duplicates. The tuple with CTID (150, 126) had the index pointer and the rest two were dangling tuples in the heap. I wonder how the index pointers to those tuples got removed: 1. May be HOT prune saw those tuples as DEAD and adjusted the HOT chain by removing those tuples. But then HOT prune would have reclaimed those tuples as well by setting the lp to UNUSED. 2. Index scan saw the HOT chain as DEAD and hence killed the index tuple. That looks unlikely because that would require an intermediate non-HOT update to the tuple. Given that the latest live tuple with the same index value is in the same block, I seriously doubt there was a non-HOT update to those tuples. Also, there are couple of other things to notice. 1. For VACUUM to freeze those tuples as you are suspecting, they should be seen as LIVE when HeapTupleSatisfiesVacuum is run by VACUUM. But for them to be removed from the HOT chain, they must be seen as DEAD to someone else and that must happen before VACUUM is run. 2. Tuple (150, 98) links to (150, 101) and both of them are unwanted duplicates. Can't reason how we end up in this state. Jeff mentioned that this thinks this issue could be reproducible without any crash recovery. Alvaro, I did not try to reproduce the problem using your patch, but can you please check if you see duplicates in similar state that we saw in Jeff's case ? Or can someone explain how we could end up in this state because of heap_tuple_freeze() freezing a potentially DEAD tuple ? Thanks, Pavan -- Pavan Deolasee http://www.linkedin.com/in/pavandeolasee -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Release notes git attribution
Hi, The 9.2.3 release notes say: * Fix pg_upgrade's -O/-o options (Bruce Momjian) You got it the wrong way around, he was the one who introduced the bug! ;) I originally found, debugged and provided the fix: http://www.postgresql.org/message-id/cabrt9rby1urtmy-dfldgbpcp+zqu36m_+bld4wfwugohg-m...@mail.gmail.com No big deal, it was just one space, but I wanted to voice my disagreement with how Postgres committers don't retain original patch authorship information. Regards, Marti -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Release notes git attribution
On 8 February 2013 09:43, Marti Raudsepp ma...@juffo.org wrote: No big deal, it was just one space, but I wanted to voice my disagreement with how Postgres committers don't retain original patch authorship information. I agree its very important to get that right. It's important for the PostgreSQL project to encourage further contribution by acknowledging and celebrating contributions, as well as allowing external observers to see that work was done successfully by particular people to confirm further sponsorship/time allocation etc. That practice has long been followed and encouraged by the core team and committers. Some mistakes are made from time to time but there is a willingness to make corrections to that. I'll look into this. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Considering Gerrit for CFs
On Fri, Feb 8, 2013 at 1:32 AM, Josh Berkus j...@agliodbs.com wrote: Folks, First, thanks for the serious discussion of this. There are obvious tooling gaps (aren't there always?), but I don't really see the model as broken, and I don't think I've been around pgsql-hackers exclusively or extensively enough to have developed Stockholm syndrome. I don't see the model as broken either. Just the tooling, which is why I'm looking at tooling. As in, I'm looking for better tooling in order Yet you are suggesting tooling that requires a change in the model? to solve two problems: 1. maximize the efficiency of existing reviewer time 2. make tooling not be an obstacle to getting new reviewers I think you are missing a fundamental part in this - which is 0. don't negatively affect the efficiency of existing committer time. I'm not saying it necessarily does (though I think it does, but that's not a proven point), but that has to be a pretty high priority. Of these two, (2) is actually the more critical. We have been losing, not gaining, active committers and reviewers for the last couple years. Clearly do more of what we've been doing is a losing strategy. We need to be sucessfully moving people up the contributor chain if we're ever going to get out of this not enough reviewers hole. Agreed. But do you have any actual proof that the problem is in we loose reviewers because we're relying on email? I agree that tooling is a minority of this, but tooling is also the easiest thing to change (compared with project organization), so that's what I'm tackling first. Expect a discussion on the people aspects at the developer meeting. It would probably be a good thing to discuss the tooling there, too. Personally, I find the most annoying thing with the current process being when reviewers post their reviews as a completely separate thread, instead of replying in the original thread. This causes context switches when parsing things, because now you have to jump back and forth between the CF app and your mail reader. But it's still only on the annoyance side, I think the process in general is not broken. (That said, I *have* been on the inside a long time, *and* I live in Stockholm, so I might well have that syndrome) So, look at this from the perspective of a casual reviewer, say at a PUG reviewfest. Instructions to new reviewer: 1. find the feature you want to review on the CF app. 2. Click the link to the mailing list archives. 3. Click all the way through the list thread to make sure there isn't a later version of the patch. Supposedly the latest version should always be listed in the CF app. The fact that this is a manual step is a problem, that could probably be fixed quite easily. 4. Download the patch. Hopefully it's not mangled by the archives (this has gotten much better than it was last year) Yes, several things have been done to make this work better. There shouldn't be any issues at all with it now - and if there are, we are in a much better position to fix them. 5. Apply the patch to HEAD clone. 6. Do actual reviewing/testing. 7. Write an email review. 8. Send it to pgsql-hackers 8.a. this requires you to be subscribed to pgsql-hackers. No, it does not. It will get caught in the moderation queue and get slightly delayed if you're not, but it works perfectly fine. And if we were to use another system, you'd still have to sign up for that one, so it's not really that big a problem. 9. wait for the email to show up in the archives. You do realize this currently counts within seconds or something like that, rather than the 15+ minutes it used to be? And the fact is, you don't actually have to wait for it. 10. create a review comment in the CF app. 10.a. this requires you to be signed up for a community account 10.b. sign up for one now 10.c. wait for it to be approved Huh? There is no approval process for community accounts. There is a verification step, of course, but any system would have that. 11. link the review comment to the messageID 12. change status of the patch This is a few too many steps, and certainly appears completely broken to any newcomer. I agree it's way too many step. Several of those can certainly be made more efficient now that we have a more sane archives, well within the scope of the current system. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Too frequent checkpoints ?
I was looking at the vacuum/visibility bug that Jeff Janes reported and brought up the server with the data directory he has shared. With his configuration, # - Checkpoints - checkpoint_segments = 1 # in logfile segments, min 1, 16MB each checkpoint_timeout = 30s# range 30s-1h # - Archiving - archive_mode = on # allows archiving to be done # (change requires restart) archive_command = 'echo archive_command %p %f `date`' # command to use to archive a logfile segment # placeholders: %p = path of file to archive # %f = file name only # e.g. 'test ! -f /mnt/server/archivedir/%f cp %p /mnt/server/archivedir/%f' archive_timeout = 30# force a logfile segment switch after this # number of seconds; 0 disables I see the following messages coming on the console: 3092 2013-02-08 02:29:01.239 PST:LOG: checkpoints are occurring too frequently (0 seconds apart) 3092 2013-02-08 02:29:01.239 PST:HINT: Consider increasing the configuration parameter checkpoint_segments. 3092 2013-02-08 02:29:01.239 PST:LOG: checkpoint starting: xlog 3092 2013-02-08 02:29:01.246 PST:LOG: checkpoint complete: wrote 0 buffers (0.0%); 0 transaction log file(s) added, 0 removed, 1 recycled; write=0.000 s, sync=0.000 s, total=0.007 s; sync files=0, longest=0.000 s, average=0.000 s archive_command pg_xlog/0001005700F5 0001005700F5 Fri Feb 8 15:59:01 IST 2013 3092 2013-02-08 02:29:31.278 PST:LOG: checkpoints are occurring too frequently (0 seconds apart) 3092 2013-02-08 02:29:31.278 PST:HINT: Consider increasing the configuration parameter checkpoint_segments. 3092 2013-02-08 02:29:31.278 PST:LOG: checkpoint starting: xlog 3092 2013-02-08 02:29:31.285 PST:LOG: checkpoint complete: wrote 0 buffers (0.0%); 0 transaction log file(s) added, 0 removed, 1 recycled; write=0.000 s, sync=0.000 s, total=0.006 s; sync files=0, longest=0.000 s, average=0.000 s archive_command pg_xlog/0001005700F6 0001005700F6 Fri Feb 8 15:59:31 IST 2013 3092 2013-02-08 02:30:01.316 PST:LOG: checkpoints are occurring too frequently (0 seconds apart) 3092 2013-02-08 02:30:01.316 PST:HINT: Consider increasing the configuration parameter checkpoint_segments. 3092 2013-02-08 02:30:01.316 PST:LOG: checkpoint starting: xlog archive_command pg_xlog/0001005700F7 0001005700F7 Fri Feb 8 16:00:01 IST 2013 3092 2013-02-08 02:30:01.323 PST:LOG: checkpoint complete: wrote 0 buffers (0.0%); 0 transaction log file(s) added, 0 removed, 1 recycled; write=0.000 s, sync=0.000 s, total=0.006 s; sync files=0, longest=0.000 s, average=0.000 s 3092 2013-02-08 02:30:31.327 PST:LOG: checkpoints are occurring too frequently (0 seconds apart) 3092 2013-02-08 02:30:31.327 PST:HINT: Consider increasing the configuration parameter checkpoint_segments. 3092 2013-02-08 02:30:31.327 PST:LOG: checkpoint starting: xlog archive_command pg_xlog/0001005700F8 0001005700F8 Fri Feb 8 16:00:31 IST 2013 3092 2013-02-08 02:30:31.334 PST:LOG: checkpoint complete: wrote 0 buffers (0.0%); 0 transaction log file(s) added, 0 removed, 1 recycled; write=0.000 s, sync=0.000 s, total=0.007 s; sync files=0, longest=0.000 s, average=0.000 s I wonder if this is all expected. The database is getting ZERO activity. There are no connections open at this time. The checkpoints are happening at every 30 seconds and new WAL files are being created, AFAIK because the old ones are getting archived. Can't we be smart about not archiving new files if we did not generate any new WAL since the last archive ? Also, the log says checkpoints are occurring too frequently (0 seconds apart). But that looks wrong too. Checkpoints are really happening at 30 seconds apart and not 0 as the log message claims. Thanks, Pavan -- Pavan Deolasee http://www.linkedin.com/in/pavandeolasee -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Comment typo
On Fri, Feb 8, 2013 at 6:49 AM, Etsuro Fujita fujita.ets...@lab.ntt.co.jp wrote: I found a comment typo. Please find attached a patch. Applied, thanks. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Add Makefile dep in bin/scripts for libpgport
On Wed, 2013-01-23 at 12:36 -0500, Phil Sorber wrote: I get the following error when I try to compile just a specific binary in src/bin/scripts: gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard reindexdb.o common.o dumputils.o kwlookup.o keywords.o -L../../../src/port -lpgport -L../../../src/interfaces/libpq -lpq -L../../../src/port -Wl,--as-needed -Wl,-rpath,'/usr/local/pgsql/lib',--enable-new-dtags -lpgport -lz -lreadline -lcrypt -ldl -lm -o reindexdb /usr/bin/ld: cannot find -lpgport /usr/bin/ld: cannot find -lpgport collect2: error: ld returned 1 exit status make: *** [reindexdb] Error 1 It appears it is missing the libpgport dependency. Attached is a patch to correct that. This is not normally a problem because when building the whole tree libpgport is usually compiled already. Committed. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Identity projection
On Friday, February 08, 2013 12:00 AM Tom Lane wrote: Amit Kapila amit.kap...@huawei.com writes: There can be 2 ways to remove result node a. Remove the Result plan node in case it is not required - This is same as currently it does for SubqueryScan. We can check if the result plan is trivial (with logic similar to trivial_subqueryscan()), then remove result node. b. to avoid adding it to Plan node in case it is not required - For this, in grouping_planner() currently it checks if the plan is projection capable (is_projection_capable_plan()), we can enhance this check such that it also check projection is really required depending if the targetlist contains any non Var element. Please suggest which way is more preferable and if one of above 2 seems to be okay, Adding a result node only to remove it again seems a bit expensive. It'd be better not to generate the node in the first place. (There's a technical reason to generate a temporary SubqueryScan, which is to keep Var numbering in the subplan separate from that in the upper plan; but AFAICS that doesn't apply to Result.) An advantage of removing useless Results at setrefs.c time is that we can be sure it will be applied to all Result nodes. However, we might be able to do it the other way with only one point-of-change if we hack make_result itself to check whether the proposed tlist is an identity. So for this, if a,b,c (below mentioned conds.) are true then don't create Result Plan, just return subplan a. subplan is NOT NULL and b. resconstantqual is NULL and c. compare expr of each TargetEntry for proposed tlist with subplan target Note that contains non Var element is the wrong test for this anyway --- the question is does the tlist have the same expressions in the same order as the tlist of the Result's child node. As per my understanding, currently in code wherever Result node can be avoided, it calls function is_projection_capable_plan(), so we can even enhance is_projection_capable_plan() so that it can also verify the expressions of tlists. But for this we need to change at all places from where is_projection_capable_plan() is called. With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alias hstore's ? to ~ so that it works with JDBC
merlin, Yes, you're correct, my phrasing was bad: all I meant was that it was a conflict, not a bug in Postgres or hstore. I personally don't know of any way around the conflict except changing JDBC or hstore, and I don't think JDBC is gonna change. Deciding not to accommodate JDBC on the Postgres side, though, is going to prevent hstore from being used properly with Java or any JVM-based language like JRuby. Please let me know if my assumptions are wrong. Best, Seamus On 2/6/13 10:58 AM, Merlin Moncure wrote: On Tue, Feb 5, 2013 at 11:29 AM, Seamus Abshere sea...@abshere.net wrote: As reported in BUG #7715 [1], hstore's use of ? as an operator conflicts with JDBC's bind variables. I think we could just alias ? to ~ and tell JDBC users to use that instead. This is not a bug with postgres, but with java/JDBC. There are many operators that use '?' besides hstore and JDBC should allow for escaping out of its statement interpretation. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alias hstore's ? to ~ so that it works with JDBC
merlin, I appreciate your perspective and wish that JDBC didn't present this problem. Still, with the rapidly growing adoption of both Postgres and JVM-based languages (thanks Heroku!) - plus the special power of hstore - I think it would be a shame if such a small accommodation could not be made. Other hackers, What is your take? Best, Seamus PS. Switching to the EXIST() or DEFINED() functions is not, in my opinion, a solution because they can't use hstore's GiST or GIN indexes. On 2/6/13 11:34 AM, Merlin Moncure wrote: The point is that Postgres should not introduce language constraints because of broken driver technology. To move forward in your particular case, consider: *) switching to 'hstore defined()' function: *) hacking pg_operator (carefully look up and change oprname for the specific hstore operator) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] sepgsql and materialized views
2013/2/7 Kevin Grittner kgri...@ymail.com: Kohei KaiGai kai...@kaigai.gr.jp wrote: So, I'd like to review two options. 1) we uses db_table object class for materialized-views for a while, until selinux-side become ready. Probably, v9.3 will use db_table class then switched at v9.4. 2) we uses db_materialized_view object class from the begining, but its permission checks are ignored because installed security policy does not support this class yet. My preference is 2), even though we cannot apply label based permission checks until selinux support it, because 1) makes troubles when selinux-side become ready to support new db_materialized_view class. Even though policy support MV class, working v9.3 will ignore the policy. Let me ask selinux folks about this topic also. To make sure I understand, the current patch is consistent with option 1? I believe so, even though I didn't take deep and detailed investigation yet. It sounds like I have code from a prior version of the patch pretty close to what you describe for option 2, so that can be put back in place if you confirm that as the preferred option. As above, I'd like to suggest the option 2. Could you once remove the updates related to contrib/sepgsql? I'll have a discussion about new materialized_view object class on selinux list soon, then I'll submit a patch towards contrib/sepgsql according to the consensus here. From what you describe, it sounds like the only thing it doesn't have is a new hook for REFRESH, but that doesn't sound like it would take that much to implement. I think all we need to give extensions a chance to check permission on REFRESH timing is a hook that informs which materialized-view shall be refreshed. Probably, OAT_MATERIALIZED_VIEW_RERESH event with its oid on object_access_hook is sufficient. Thanks, -- KaiGai Kohei kai...@kaigai.gr.jp -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [JDBC] JPA + enum == Exception
Tom Lane t...@sss.pgh.pa.us wrote: when attempting to use a prepared statement: ps = con.prepareStatement(insert into enumcast values (?)); ps.setString(1, meh); ps.executeUpdate(); we get a org.postgresql.util.PSQLException: ERROR: column current_mood is of type mood but expression is of type character varying Hint: You will need to rewrite or cast the expression. AFAIK this is just business as usual with JDBC: setString() implies that the parameter is of a string type. It'll fall over if the type actually required is anything but a string. (I'm no Java expert, but I seem to recall that using setObject instead is the standard workaround.) Right. It is spelled out pretty specifically in the JDBC spec: http://download.oracle.com/otn-pub/jcp/jdbc-4_1-mrel-spec/jdbc4.1-fr-spec.pdf Table B-2 governs setting PreparedStatement parameters with setString and similar methods, while tables B-4 and B-5 cover the setObject methods. It is clearly not unusual for other vendors to extend the JDBC specification to make life easier for those writing ORMs, etc.; but the behavior of the current PostgreSQL JDBC driver is doing all that is required by the spec. Enums are not suffering any special hardship here, and I'd be against weakening the type system to give them a special pass. This is not entirely unrelated to the discussions about allowing broader use of automatic casting server-side. It seems to me that on one side of the argument is the idea that strict typing reduces bugs and doesn't lead to problems with ambiguity, especially as things change; and on the other side the argument is that where no ambiguity exists we would make life easier for developers of applications or access tools if we relexed things beyond what the related specifications require, and that not doing so discourages adoption. I think that all the same arguments apply here with equal force, on both sides of the issue. The problem with this debate has always been that both sides are completely right. Those are always the toughest to resolve. It comes down to which evils we tolerate to garner which benefits. It seems that in such cases inertia tends to win. I'm not so sure that it should. An ideal solution would find some way to address the concerns of both sides, but so far that has eluded us when it comes to the type system. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Vacuum/visibility is busted
Pavan Deolasee escribió: I'm trying to reason how this bug explains what we saw. In the test, we'd left with duplicate tuples. If I just take index 219 in the table as an example, that tuple had three duplicates. The tuple with CTID (150, 126) had the index pointer and the rest two were dangling tuples in the heap. Hm, the examples I chased had t_infomask 0x2500, that is there were no HOT bits set. It's quite possible that there's another bug in here, but this one is a real one and it explains a very similar problem. Now, how would your answers change if HeapTupleSatisfiesVacuum returned RECENTLY_DEAD instead of DEAD? That's what I saw; and when it happened, vacuum didn't set the tupgone flag, and thus passed the tuple to heap_freeze_tuple; that routine examined the tuple and removed the Xmax and set the HEAP_XMAX_INVALID bit because of the bogus logic. A tuple which was supposed to be dead suddenly turned into visible. I'm not really sure how these bogus conditions make HOT misbehave; I don't fully understand the page pruning stuff. I think if they see a chain and in the middle of it one Xmin doesn't match the next tuple's Xmax, it considers the whole thing to not be a chain at all. So maybe that explains the other effects? -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Considering Gerrit for CFs
On 2/6/13 4:07 PM, Josh Berkus wrote: I think one of them has, now: Gerrit. http://code.google.com/p/gerrit/ I find Gerrit pretty useful, and I would support trying it out. I suggest, build it and they will come, or not. Let people push their patches into Gerrit and attach the reviews to the commit fest items. If reviewers then want to use that, it's their choice. We'll see how it goes. You don't need to replace the commitfest app (if that's what you were getting at), and Gerrit isn't an issue tracking system anyway. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Considering Gerrit for CFs
On 2/8/13 5:23 AM, Magnus Hagander wrote: But do you have any actual proof that the problem is in we loose reviewers because we're relying on email? Here is one: Me. Just yesterday I downloaded a piece of software that was previously unknown to me from GitHub and found a bug. Within 15 minutes or so I had fixed the bug, made a fork, sent a pull request. Today I read, the fix was merged last night, and I'm happy. How would this go with PostgreSQL? You can use the bug form on the web site, but you can't attach any code, so the bug will just linger and ultimately put more burden on a core contributor to deal with the minutiae of developing, testing, and committing a trivial fix and sending feedback to the submitter. Or the user could take the high road and develop and patch and submit it. Just make sure it's in context diff format! Search the wiki if you don't know how to do that! Send it to -hackers, your email will be held for moderation. We won't actually do anything with your patch, but we will tell you to add it to that commitfest app over there. You need to sign up for an account to use that. We will deal with your patch in one or two months. But only if you review another patch. And you should sign up for that other mailing list, to make sure you're doing it right. Chances are, the first review you're going to get is that your patch doesn't apply anymore, but which time you will have lost interest in the patch anyway. So, I don't have any further evidence that we are losing reviewers, but in light of the above and the options out there were interested developers can contribute much more easily, I'm amazed that we are getting any new contributors or reviewers at all. Of course, Gerrit doesn't actually address most of the issues above, but it could be part of a step forward. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Release notes git attribution
Marti Raudsepp ma...@juffo.org writes: The 9.2.3 release notes say: * Fix pg_upgrade's -O/-o options (Bruce Momjian) You got it the wrong way around, he was the one who introduced the bug! ;) Sorry about that. The release notes are made based on commit-log entries. In this case we had Author: Bruce Momjian br...@momjian.us Branch: master [acdb8c225] 2012-12-10 23:03:25 -0500 Branch: REL9_2_STABLE [35fb1434b] 2012-12-10 23:03:28 -0500 Fix pg_upgrade -O/-o options Fix previous commit that added synchronous_commit=off, but broke -O/-o due to missing space in argument passing. Backpatch to 9.2. so it was Bruce's error not to credit you in the commit message. I know that he knows better --- it is project policy to properly credit patch authors. But mistakes do happen, of course. Again, my apologies on behalf of the project. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Considering Gerrit for CFs
On Fri, Feb 8, 2013 at 4:20 PM, Peter Eisentraut pete...@gmx.net wrote: On 2/8/13 5:23 AM, Magnus Hagander wrote: But do you have any actual proof that the problem is in we loose reviewers because we're relying on email? Here is one: Me. Just yesterday I downloaded a piece of software that was previously unknown to me from GitHub and found a bug. Within 15 minutes or so I had fixed the bug, made a fork, sent a pull request. Today I read, the fix was merged last night, and I'm happy. How would this go with PostgreSQL? You can use the bug form on the web site, but you can't attach any code, so the bug will just linger and ultimately put more burden on a core contributor to deal with the minutiae of developing, testing, and committing a trivial fix and sending feedback to the submitter. Or the user could take the high road and develop and patch and submit it. Just make sure it's in context diff format! Search the wiki if you don't know how to do that! Send it to -hackers, your email will be held for moderation. We won't actually do anything with your patch, but we will tell you to add it to that commitfest app over there. You need to sign up for an account to use that. We will deal with your patch in one or two months. But only if you review another patch. And you should sign up for that other mailing list, to make sure you're doing it right. Chances are, the first review you're going to get is that your patch doesn't apply anymore, but which time you will have lost interest in the patch anyway. So, I don't have any further evidence that we are losing reviewers, but in light of the above and the options out there were interested developers can contribute much more easily, I'm amazed that we are getting any new contributors or reviewers at all. Of course, Gerrit doesn't actually address most of the issues above, but it could be part of a step forward. You're outlining an issue for submitters. Berkus was complaining about issues for reviewers. These are clearly different issues. And I don't think gerrit helps at all with the submitters process that you've outlined above - it's a tool to help the reviewing. That doesn't, of course, mean that we shouldn't try to solve both things - but they are completely different. Basically, what you're saying above, is we should start accepting pull requests from github. There's nothing preventing us from doing that (other than the wish to do so), no need to change tooling for review for that. It just means that committers need to use git and add peoples repositories as remotes instead of applying patches. Probably not a huge burden today since most pg developers are used to git by now. However, it's not going to change the requirement to help review other things, that's a pure policy issue. Which I'm pretty sure we don't enforce for 10-minute-trivial-fixup-patches. And it's not going to change the fact that it takes time before someone gets around to your patch, that's a resource issue. And it's not going to change the fact that a patch migt not apply after 2 months, that's a consequence of the second problem. It doesn't change the fact that you have to sign up - it just makes it more likely that you're already signed up, since so many people are on github already, but you *do* have to sign up for a service, wherever it's hosted. But it does change the fact that you don't have to deal with email, and can use web instead. Personally, I find it much easier to just git clone, make changes, git diff, attach to email, than fork on github. git clone, make changes, push to github, create pull request on github. repeatedly check status of said pull request since email notifications aren't usable. But that's me, personally, and I realize many people today prefer web interfaces for as much as possible. There's nothing stopping us from supporting both, of course. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Vacuum/visibility is busted
Tom Lane escribió:
Alvaro Herrera alvhe...@2ndquadrant.com writes:
xid = HeapTupleHeaderGetRawXmax(tuple);
! if (((tuple-t_infomask HEAP_XMAX_IS_MULTI)
!MultiXactIdIsValid(xid)
!MultiXactIdPrecedes(xid, cutoff_multi)) ||
! ((!(tuple-t_infomask HEAP_XMAX_IS_MULTI))
!TransactionIdIsNormal(xid)
!TransactionIdPrecedes(xid, cutoff_xid)))
{
Would this be clearer as a ternary expression? That is,
if ((tuple-t_infomask HEAP_XMAX_IS_MULTI) ?
(MultiXactIdIsValid(xid)
MultiXactIdPrecedes(xid, cutoff_multi)) :
(TransactionIdIsNormal(xid)
TransactionIdPrecedes(xid, cutoff_xid)))
Ah, yes, by far. Thanks, I pushed that way.
--
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Considering Gerrit for CFs
Peter Eisentraut pete...@gmx.net writes: I suggest, build it and they will come, or not. Let people push their patches into Gerrit and attach the reviews to the commit fest items. If reviewers then want to use that, it's their choice. We'll see how it goes. I might be misunderstanding what you're suggesting here, but it sounds like this would imply that reviews could end up off in a Gerrit repo somewhere, never getting posted to the mailing lists at all. That would make me sad. The list archives are this project's community memory, and I have every expectation that they'll still be around and useful when Gerrit is forgotten. I don't object to people using their tools-of-choice to perform reviewing, but we need some way of making sure that the reviews get archived. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Considering Gerrit for CFs
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 The problem with doing it in-house is that the folks who can work on it and maintain it will be taking time away from developing PostgreSQL. Not sure that using Gerrit solves this. Someone will need to install it, maintain it, document, and hack it. Yes, hack it, as it is not a drop-in solution. ... I think one of them has, now: Gerrit. http://code.google.com/p/gerrit/ I use Gerrit in the MediaWiki project, and it ain't pretty. The interface is confusing, the workflow is more complex, and the MediaWiki folks have had to do a lot of work to make things usable, despite their having a non-email-centric workflow already. Maybe we can identify specific issues with our current app instead? - -- Greg Sabino Mullane g...@turnstep.com PGP Key: 0x14964AC8 201302081106 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAlEVIqcACgkQvJuQZxSWSsh04gCfaK80dbuL8NnAVuViGR5sFQXN GzwAoM+2fcI6+zFZPqkslZrWjkZ05AOo =azLj -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Too frequent checkpoints ?
On 02/08/2013 02:37 AM, Pavan Deolasee wrote: I wonder if this is all expected. The database is getting ZERO activity. There are no connections open at this time. The checkpoints are happening at every 30 seconds and new WAL files are being created, AFAIK because the old ones are getting archived. Can't we be smart about not archiving new files if we did not generate any new WAL since the last archive ? It is because you have it set up so that the longest time you can go without checkpoints is 30 seconds. So no matter what, if you have 1000 checkpoint segments you are still going to checkpoint ever 30 seconds. JD -- Command Prompt, Inc. - http://www.commandprompt.com/ PostgreSQL Support, Training, Professional Services and Development High Availability, Oracle Conversion, Postgres-XC @cmdpromptinc - 509-416-6579 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Stamp 9.1.8.
Magnus Hagander mag...@hagander.net wrote: if there is any other committer who [wants to receive emails from the packagers list], let me know and I will add you there as well. Hi Magnus, Please add me. Thanks, -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Considering Gerrit for CFs
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 How would this go with PostgreSQL? You can use the bug form on the web site, but you can't attach any code, so the bug will just linger and ultimately put more burden on a core contributor to deal with the minutiae of developing, testing, and committing a trivial fix and sending feedback to the submitter. Well, they could attach a link to a github patch... Or the user could take the high road and develop and patch and submit it. Just make sure it's in context diff format! Search the wiki if you don't know how to do that! Send it to -hackers, your email will be held for moderation. We won't actually do anything with your patch, but we will tell you to add it to that commitfest app over there. You need to sign up for an account to use that. We will deal with your patch in one or two months. But only if you review another patch. And you should sign up for that other mailing list, to make sure you're doing it right. Chances are, the first review you're going to get is that your patch doesn't apply anymore, but which time you will have lost interest in the patch anyway. +1 to all that. Especially the signing up for the commitfest app. Of course, Gerrit doesn't actually address most of the issues above, but it could be part of a step forward. More of a step sideways. It doesn't address the bigger problems. - -- Greg Sabino Mullane g...@turnstep.com PGP Key: 0x14964AC8 201302081124 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAlEVJrkACgkQvJuQZxSWSshh3gCgz+XHwAbk5rryttYPi68j4EJi 7DcAnjEdxDD4Rm2/oDBaqHbOzQLwR6zR =0lnp -END PGP SIGNATURE- 2~ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alias hstore's ? to ~ so that it works with JDBC
On Wed, 6 Feb 2013, Seamus Abshere wrote: I personally don't know of any way around the conflict except changing JDBC or hstore, and I don't think JDBC is gonna change. I think changing JDBC is the way to go. Currently JDBC supports escape sequences for cross database portability and it seems reasonable to support an escape sequence that allowed passing ? to the backend instead of interpreting it as a parameter. This will be more complicated than you might hope because the escape processing currently happens prior to bind parameter detection so I'm not sure what a good patch would really look like, but given the feedback provided here, it's worth investigating. Kris Jurka -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alias hstore's ? to ~ so that it works with JDBC
On Fri, Feb 8, 2013 at 11:43 AM, Kris Jurka bo...@ejurka.com wrote: On Wed, 6 Feb 2013, Seamus Abshere wrote: I personally don't know of any way around the conflict except changing JDBC or hstore, and I don't think JDBC is gonna change. I think changing JDBC is the way to go. Currently JDBC supports escape sequences for cross database portability and it seems reasonable to support an escape sequence that allowed passing ? to the backend instead of interpreting it as a parameter. This will be more complicated than you might hope because the escape processing currently happens prior to bind parameter detection so I'm not sure what a good patch would really look like, but given the feedback provided here, it's worth investigating. Kris Jurka Would this be an postgresql specific escape sequence ? I don't think the spec allows for this does it ? Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca
Re: [HACKERS] Too frequent checkpoints ?
On Fri, Feb 8, 2013 at 7:37 PM, Pavan Deolasee pavan.deola...@gmail.com wrote: I wonder if this is all expected. The database is getting ZERO activity. There are no connections open at this time. The checkpoints are happening at every 30 seconds and new WAL files are being created, AFAIK because the old ones are getting archived. Can't we be smart about not archiving new files if we did not generate any new WAL since the last archive ? AFAIR this problem has been discussed several times before, but has not been fixed yet. BTW, the cause of the problem is that the following sequences happens. 1. archive_timeout switches WAL file because checkpoint WAL record has has been written since last switch 2. Checkpoint occurs and writes its WAL record because new WAL file was generated since last checkpoint, i.e., checkpoint thinks that there is at least some activity since last checkpoint. 3. Back to #1 after archive_timeout passed through. Regards, -- Fujii Masao -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Too frequent checkpoints ?
On Fri, Feb 8, 2013 at 2:37 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: I was looking at the vacuum/visibility bug that Jeff Janes reported and brought up the server with the data directory he has shared. With his configuration, 3092 2013-02-08 02:30:31.327 PST:LOG: checkpoints are occurring too frequently (0 seconds apart) 3092 2013-02-08 02:30:31.327 PST:HINT: Consider increasing the configuration parameter checkpoint_segments. ... I wonder if this is all expected. The database is getting ZERO activity. There are no connections open at this time. The checkpoints are happening at every 30 seconds and new WAL files are being created, AFAIK because the old ones are getting archived. Can't we be smart about not archiving new files if we did not generate any new WAL since the last archive ? Yeah, this is a known issue. We skip two consecutive checkpoints (forced by checkpoint_timeout) with no intervening activity, and we skip two consecutive log switches (forced by archive_timeout) with no intervening activity, but when you put them both together neither one gets skipped because each one counts as intervening activity from the other ones perspective. I think that at one point this was considered desirable, as the arrival of log files in the archive, even if they had no real content, was viewed as proof of life. You can set up archive_command to send you an email if an archive fails, but the same thing that prevents the archive might prevent the email from arriving. Also, the log says checkpoints are occurring too frequently (0 seconds apart). But that looks wrong too. Checkpoints are really happening at 30 seconds apart and not 0 as the log message claims. I noticed that as well, and I think it might be a regression. I'll run a bisection on it if I get a chance. Cheers, Jeff -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Vacuum/visibility is busted
On Thu, Feb 7, 2013 at 8:38 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Alvaro Herrera escribió: Alvaro Herrera escribió: Hm, if the foreign key patch is to blame, this sounds like these tuples had a different set of XMAX hint bits and a different Xmax, and they were clobbered by something like vacuum or page pruning. Hm, I think heap_freeze_tuple is busted, yes. This patch seems to fix the problem for me. Please confirm. Applied to 0ac5ad5134f276, it has survived 4 wrap-arounds so far without problem. I will let it go for a while longer, but I think we can assume it is fixed. Cheers, Jeff -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] DROP OWNED BY fails to drop privileges granted by non-owners (was Re: [GENERAL] Bug, Feature, or what else?)
Adrian Klaver adrian.kla...@gmail.com writes: On 02/08/2013 08:14 AM, Tom Lane wrote: Of course, postgres has other options besides that, of which DROP OWNED BY ak02 is probably the most appropriate here. Or if you really want to get rid of just that grant, SET ROLE TO akretschmer01 and revoke. The DROP OWNED was tried further up the thread and did not seem to work: Huh. You're right, here is a complete test case: regression=# create schema s1; cCREATE SCHEMA regression=# create user u1; CREATE ROLE regression=# create user u2; CREATE ROLE regression=# grant all on schema s1 to u1 with grant option; GRANT regression=# \c - u1 You are now connected to database regression as user u1. regression= grant all on schema s1 to u2; GRANT regression= \c - postgres You are now connected to database regression as user postgres. regression=# \dn+ s1 List of schemas Name | Owner | Access privileges | Description --+--+--+- s1 | postgres | postgres=UC/postgres+| | | u1=U*C*/postgres+| | | u2=UC/u1 | (1 row) regression=# drop user u2; -- expect failure here ERROR: role u2 cannot be dropped because some objects depend on it DETAIL: privileges for schema s1 regression=# drop owned by u2; DROP OWNED regression=# drop user u2; -- failure here is wrong ERROR: role u2 cannot be dropped because some objects depend on it DETAIL: privileges for schema s1 regression=# \dn+ s1 List of schemas Name | Owner | Access privileges | Description --+--+--+- s1 | postgres | postgres=UC/postgres+| | | u1=U*C*/postgres+| | | u2=UC/u1 | (1 row) I believe the problem is that DROP OWNED for privileges is implemented by calling REVOKE. As noted upthread, when a superuser does REVOKE, it's executed as though the object owner did the REVOKE, so only privileges granted directly by the object owner go away. In this particular example, DROP OWNED BY u1 makes the grant to u1 go away, and then the grant to u2 goes away via cascade ... but DROP OWNED BY u2 fails to accomplish anything at all, because postgres never granted anything directly to u2. We haven't seen this reported before, probably because the use of GRANT OPTIONS isn't very common, but AFAICS it's been wrong since the invention of DROP OWNED. It looks to me like DropOwnedObjects doesn't actually insist on superuserness to do DROP OWNED, only ability to become the role, which means that DROP OWNED BY is completely broken for privileges if executed by a non-superuser; the only privileges it would remove would be those granted by the current user to the target user. I'm not really sure what the desirable behavior would be in such a case though. Ordinary users can't revoke privileges granted *to* them, only privileges granted *by* them. So it's not necessarily the case that a non-superuser should be able to make all privileges granted to a target role go away, even if he's allowed to become the target role and thereby drop objects that it owns. I wonder how sensible it is really to allow DROP OWNED to non-superusers. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Time for an autoconf update
Over in Fedora-land they're trying to institute support for ARM64, which among other things means autoconf 2.69 or later: http://lists.fedoraproject.org/pipermail/devel/2013-February/178273.html We are behind the curve. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Time for an autoconf update
On 02/08/2013 12:21 PM, Tom Lane wrote: Over in Fedora-land they're trying to institute support for ARM64, which among other things means autoconf 2.69 or later: http://lists.fedoraproject.org/pipermail/devel/2013-February/178273.html We are behind the curve. Is there any good reason not to move to whatever the latest and greatest is? 2.69 does seem pretty new - even Fedora 17 only comes with 2.68. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Identity projection
Amit Kapila amit.kap...@huawei.com writes:
On Friday, February 08, 2013 12:00 AM Tom Lane wrote:
As per my understanding, currently in code wherever Result node can be
avoided,
it calls function is_projection_capable_plan(), so we can even enhance
is_projection_capable_plan()
so that it can also verify the expressions of tlists. But for this we need
to change at all places
from where is_projection_capable_plan() is called.
Hm. Really there's a whole dance that typically goes on, which is like
if (!is_projection_capable_plan(result_plan))
{
result_plan = (Plan *) make_result(root,
sub_tlist,
NULL,
result_plan);
}
else
{
/*
* Otherwise, just replace the subplan's flat tlist with
* the desired tlist.
*/
result_plan-targetlist = sub_tlist;
}
Perhaps we could encapsulate this whole sequence into a function called
say assign_targetlist_to_plan(), which would have the responsibility to
decide whether a Result node needs to be inserted.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Considering Gerrit for CFs
On 02/08/2013 07:58 AM, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: I suggest, build it and they will come, or not. Let people push their patches into Gerrit and attach the reviews to the commit fest items. If reviewers then want to use that, it's their choice. We'll see how it goes. I might be misunderstanding what you're suggesting here, but it sounds like this would imply that reviews could end up off in a Gerrit repo somewhere, never getting posted to the mailing lists at all. That would make me sad. It would also be ineffective. I already tried having a secondary review forum for people who wanted to do reviews but not subscribe to -hackers; it didn't work out too well. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] pending triggers infinite list
Hi all I've got suspicious behavior for transaction cooked with deferrable trigger. if trigger has update on row of his target table we get infinite recursion without limitation of stack depth. trigger - update - trigger - update - ... ... -- infinite pending list :) - Misha -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alias hstore's ? to ~ so that it works with JDBC
On Fri, 8 Feb 2013, Dave Cramer wrote: Would this be an postgresql specific escape sequence ? I don't think the spec allows for this does it ? Yes, this would be a postgresql jdbc driver specific escape. The spec doesn't have a concept of private escape sequences, but that doesn't seem like the end of the world. Clearly the user here is writing postgresql specific code to use hstore operators, so there's not a portability loss here. Kris Jurka -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] pg_isready (was: [WIP] pg_ping utility)
On Thu, Feb 7, 2013 at 2:14 AM, Phil Sorber p...@omniti.com wrote: On Wed, Feb 6, 2013 at 11:36 AM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Feb 7, 2013 at 1:15 AM, Phil Sorber p...@omniti.com wrote: On Wed, Feb 6, 2013 at 11:11 AM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Feb 7, 2013 at 12:05 AM, Phil Sorber p...@omniti.com wrote: On Tue, Feb 5, 2013 at 12:44 PM, Phil Sorber p...@omniti.com wrote: On Tue, Feb 5, 2013 at 9:08 AM, Phil Sorber p...@omniti.com wrote: On Tue, Feb 5, 2013 at 9:06 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Phil Sorber escribió: On Tue, Feb 5, 2013 at 6:41 AM, Robert Haas robertmh...@gmail.com wrote: On Sat, Feb 2, 2013 at 9:55 PM, Phil Sorber p...@omniti.com wrote: OK, here is the patch that handles the connection string in dbname. I'll post the other patch under a different posting because I am sure it will get plenty of debate on it's own. I'm sorry, can you remind me what this does for us vs. the existing coding? It's supposed to handle the connection string passed as dbname case to be able to get the right output for host:port. Surely the idea is that you can also give it a postgres:// URI, right? Absolutely. Here is it. I like this approach more than the previous one, but I'd like some feedback. The patch looks complicated to me. I was thinking that we can address the problem just by using PQconninfoParse() and PQconndefaults() like uri-regress.c does. The patch should be very simple. Why do we need so complicated code? Did you like the previous version better? http://www.postgresql.org/message-id/cadakt-hnb3ohcpkr+pcg1c_bjrsb7j__bpv+-jrjs5opjr2...@mail.gmail.com Yes because that version is simpler. But which version is better depends on the reason why you implemented new version. If you have some idea about the merit and demerit of each version, could you elaborate them? I didn't like the way that I had to hard code the options in the first one as you pointed out below. I also was looking through the code for something else and saw that a lot of the apps were starting with defaults then building from there, rather than trying to add the defaults at the end. I think they were still doing it wrong because they were using getenv() on their own rather than asking libpq for the defaults though. So the new version gets the defaults at the beginning and also makes it easy to add new params without changing function definitions. + set_connect_options(connect_options, pgdbname, pghost, pgport, connect_timeout, pguser); This code prevents us from giving options other than the above, for example application_name, in the conninfo. I think that pg_isready should accept all the libpq options. I'm with you there. The new version fixes that as well. When more than one -d options are specified, psql always prefer the last one and ignore the others. OTOH, pg_isready with this patch seems to merge them. I'm not sure if there is specific rule about the priority order of -d option. But it seems better to follow the existing way, i.e., always prefer the last -d option. The problem I am having here is resolving the differences between different -d options and other command line options. For example: -h foo -p 4321 -d host=bar port=1234 -d host=baz I would expect that to be 'baz:1234' but you are saying it should be 'baz:4321'? I look at -d as just a way to pass in multiple options (when you aren't strictly passing in dbname) and should be able to expand the above example to: -h foo -p 4321 -h bar -p 1234 -h baz If we hold off on parsing the value of -d until the end so we are sure we have the last one, then we might lose other parameters that were after the -d option. For example: -h foo -p 4321 -d host=bar port=1234 -d host=baz user=you -U me Should this be 'me@baz:1234' or 'you@baz:4321' or 'me@baz:4321'? So we would have to track the last instance of a parameter as well as the order those final versions came in. Sound to me like there is no clear answer there, but maybe there is a project consensus that has already been reached with regard to this? No maybe. But I think that all the client commands should follow the same rule. Otherwise a user would get confused when specifying options. Regards, -- Fujii Masao -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alias hstore's ? to ~ so that it works with JDBC
On 02/08/2013 12:41 PM, Kris Jurka wrote: On Fri, 8 Feb 2013, Dave Cramer wrote: Would this be an postgresql specific escape sequence ? I don't think the spec allows for this does it ? Yes, this would be a postgresql jdbc driver specific escape. The spec doesn't have a concept of private escape sequences, but that doesn't seem like the end of the world. Clearly the user here is writing postgresql specific code to use hstore operators, so there's not a portability loss here. I assume, though, that you're not talking about something that's hstore-specific, but rather something that will allow the user to put a non-parameter question mark in the query string. As has been noted upthread, the hstore use is far from the only one that causes users to trip on this. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: DROP OWNED BY fails to drop privileges granted by non-owners (was Re: [GENERAL] Bug, Feature, or what else?)
On 02/08/2013 09:09 AM, Tom Lane wrote: Adrian Klaver adrian.kla...@gmail.com writes: On 02/08/2013 08:14 AM, Tom Lane wrote: Of course, postgres has other options besides that, of which DROP OWNED BY ak02 is probably the most appropriate here. Or if you really want to get rid of just that grant, SET ROLE TO akretschmer01 and revoke. The DROP OWNED was tried further up the thread and did not seem to work: Huh. You're right, here is a complete test case: regression=# create schema s1; cCREATE SCHEMA regression=# create user u1; CREATE ROLE regression=# create user u2; CREATE ROLE regression=# grant all on schema s1 to u1 with grant option; GRANT regression=# \c - u1 You are now connected to database regression as user u1. regression= grant all on schema s1 to u2; GRANT regression= \c - postgres You are now connected to database regression as user postgres. regression=# \dn+ s1 List of schemas Name | Owner | Access privileges | Description --+--+--+- s1 | postgres | postgres=UC/postgres+| | | u1=U*C*/postgres+| | | u2=UC/u1 | (1 row) regression=# drop user u2; -- expect failure here ERROR: role u2 cannot be dropped because some objects depend on it DETAIL: privileges for schema s1 regression=# drop owned by u2; DROP OWNED regression=# drop user u2; -- failure here is wrong ERROR: role u2 cannot be dropped because some objects depend on it DETAIL: privileges for schema s1 regression=# \dn+ s1 List of schemas Name | Owner | Access privileges | Description --+--+--+- s1 | postgres | postgres=UC/postgres+| | | u1=U*C*/postgres+| | | u2=UC/u1 | (1 row) I believe the problem is that DROP OWNED for privileges is implemented by calling REVOKE. As noted upthread, when a superuser does REVOKE, it's executed as though the object owner did the REVOKE, so only privileges granted directly by the object owner go away. In this particular example, DROP OWNED BY u1 makes the grant to u1 go away, and then the grant to u2 goes away via cascade ... but DROP OWNED BY u2 fails to accomplish anything at all, because postgres never granted anything directly to u2. We haven't seen this reported before, probably because the use of GRANT OPTIONS isn't very common, but AFAICS it's been wrong since the invention of DROP OWNED. It looks to me like DropOwnedObjects doesn't actually insist on superuserness to do DROP OWNED, only ability to become the role, which means that DROP OWNED BY is completely broken for privileges if executed by a non-superuser; the only privileges it would remove would be those granted by the current user to the target user. I'm not really sure what the desirable behavior would be in such a case though. Ordinary users can't revoke privileges granted *to* them, only privileges granted *by* them. So it's not necessarily the case that a non-superuser should be able to make all privileges granted to a target role go away, even if he's allowed to become the target role and thereby drop objects that it owns. I wonder how sensible it is really to allow DROP OWNED to non-superusers. I am not sure I am following. Are we talking two different cases here? 1) As mentioned in the first paragraph the case where running DROP OWNED as a supersuser does not work. 2) A non-superuser running DROP OWNED and not having the necessary privileges. regards, tom lane -- Adrian Klaver adrian.kla...@gmail.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Stamp 9.1.8.
On Fri, Feb 8, 2013 at 11:09 AM, Kevin Grittner kgri...@ymail.com wrote: Magnus Hagander mag...@hagander.net wrote: if there is any other committer who [wants to receive emails from the packagers list], let me know and I will add you there as well. Hi Magnus, Please add me. Thanks, -Kevin Ditto. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Stamp 9.1.8.
On Fri, Feb 8, 2013 at 6:53 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Feb 8, 2013 at 11:09 AM, Kevin Grittner kgri...@ymail.com wrote: Magnus Hagander mag...@hagander.net wrote: if there is any other committer who [wants to receive emails from the packagers list], let me know and I will add you there as well. Hi Magnus, Please add me. Thanks, -Kevin Ditto. FYI, I've done this for all those who asked on this thread, and you should've received a confirmation email from mj2. I won't ack the requests individually :) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Considering Gerrit for CFs
I don't see the model as broken either. Just the tooling, which is why I'm looking at tooling. As in, I'm looking for better tooling in order Yet you are suggesting tooling that requires a change in the model? Well, my fantasy is a version of Gerrit which accepts email from -hackers and proceeds accordingly. However, I also happen to know intimately how difficult automated processing of email is -- if we wait for a tool which can do this, we're going to be stuck with the existing system forever. I think you are missing a fundamental part in this - which is 0. don't negatively affect the efficiency of existing committer time. I'm not saying it necessarily does (though I think it does, but that's not a proven point), but that has to be a pretty high prity. Realistically, the only way out of the current committer bottleneck is to recruit more reviewers, contributors, and committers. In the 9.3 timeline, we're going to have to look at ways we believe will accomplish recruitment and promotion, even if it means sacrificing committer time (and thus, 9.4 features) in the short run. If we remain focused on maximizing the time of existing major contributors to the exclusion of recruitment, things will never get better. Or to put it another way: as the EU has proven, Austerity Plans are a loser's game. Of these two, (2) is actually the more critical. We have been losing, not gaining, active committers and reviewers for the last couple years. Clearly do more of what we've been doing is a losing strategy. We need to be sucessfully moving people up the contributor chain if we're ever going to get out of this not enough reviewers hole. Agreed. But do you have any actual proof that the problem is in we loose reviewers because we're relying on email? I don't think email is the specific issue. I think the issues are mostly people issues. The only reason I care about email vs. not-email is the technical impossibility of developing a system which can automatically turn patch and review emails into a trackable and transparent view. There are a bunch of other issues I'd like to discuss, but I agree that they should wait until after CF4. It would probably be a good thing to discuss the tooling there, too. Yes. I agree it's way too many step. Several of those can certainly be made more efficient now that we have a more sane archives, well within the scope of the current system. Right. My concern is that the people who have to do that are exactly the people whose time is already the most scarce. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alias hstore's ? to ~ so that it works with JDBC
That would seem to be the implication. JDBC wouldn't really know anything about hstore. Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca On Fri, Feb 8, 2013 at 12:51 PM, Andrew Dunstan and...@dunslane.net wrote: On 02/08/2013 12:41 PM, Kris Jurka wrote: On Fri, 8 Feb 2013, Dave Cramer wrote: Would this be an postgresql specific escape sequence ? I don't think the spec allows for this does it ? Yes, this would be a postgresql jdbc driver specific escape. The spec doesn't have a concept of private escape sequences, but that doesn't seem like the end of the world. Clearly the user here is writing postgresql specific code to use hstore operators, so there's not a portability loss here. I assume, though, that you're not talking about something that's hstore-specific, but rather something that will allow the user to put a non-parameter question mark in the query string. As has been noted upthread, the hstore use is far from the only one that causes users to trip on this. cheers andrew
Re: [HACKERS] function for setting/getting same timestamp during whole transaction
On Wed, Feb 6, 2013 at 7:26 AM, Miroslav Šimulčík
simulcik.m...@gmail.com wrote:
Alternately, you might be able to use a custom GUC from a rather smaller
PL/PgSQL function. At transaction start, issue:
set_config('myapp.trigger_time', '', 't');
This is problem with using custom GUC - clearing variable at transaction
start. Without clearing it's not sufficient solution (see my response to
Pavel's mail).
I might be confused here, but I think the point is that if you pass
true as the third argument to set_config, the setting lasts only for
the duration of the current transaction, like SET LOCAL. Which I
think solves your problem.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] pg_isready (was: [WIP] pg_ping utility)
On Fri, Feb 8, 2013 at 12:46 PM, Fujii Masao masao.fu...@gmail.com wrote: No maybe. But I think that all the client commands should follow the same rule. Otherwise a user would get confused when specifying options. I would consider the rest of the apps using it as a consensus. I will make sure it aligns in behavior. Regards, -- Fujii Masao -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] DROP OWNED BY fails to drop privileges granted by non-owners (was Re: [GENERAL] Bug, Feature, or what else?)
Adrian Klaver adrian.kla...@gmail.com writes: I am not sure I am following. Are we talking two different cases here? What I was pointing out was that the non-superuser case seems to be broken almost completely, whereas the superuser case is only broken if the object owner has given away some grant options and those have been exercised. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: DROP OWNED BY fails to drop privileges granted by non-owners (was Re: [GENERAL] Bug, Feature, or what else?)
On 02/08/2013 10:09 AM, Tom Lane wrote: Adrian Klaver adrian.kla...@gmail.com writes: I am not sure I am following. Are we talking two different cases here? What I was pointing out was that the non-superuser case seems to be broken almost completely, whereas the superuser case is only broken if the object owner has given away some grant options and those have been exercised. Got it, thanks. regards, tom lane -- Adrian Klaver adrian.kla...@gmail.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Time for an autoconf update
Andrew Dunstan wrote: On 02/08/2013 12:21 PM, Tom Lane wrote: Over in Fedora-land they're trying to institute support for ARM64, which among other things means autoconf 2.69 or later: http://lists.fedoraproject.org/pipermail/devel/2013-February/178273.html Is there any good reason not to move to whatever the latest and greatest is? 2.69 does seem pretty new - even Fedora 17 only comes with 2.68. Considering that only a handful of people need the specific required autoconf version, I don't think it's a problem to migrate to the latest and greatest. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Considering Gerrit for CFs
On 2/8/13 10:58 AM, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: I suggest, build it and they will come, or not. Let people push their patches into Gerrit and attach the reviews to the commit fest items. If reviewers then want to use that, it's their choice. We'll see how it goes. I might be misunderstanding what you're suggesting here, but it sounds like this would imply that reviews could end up off in a Gerrit repo somewhere, never getting posted to the mailing lists at all. That would make me sad. The list archives are this project's community memory, and I have every expectation that they'll still be around and useful when Gerrit is forgotten. I don't object to people using their tools-of-choice to perform reviewing, but we need some way of making sure that the reviews get archived. Gerrit sends me an email every times something happens, so I think this is not going to be a problem. What it doesn't support AFAICT is sending emails *in*, but I don't see that as a requirement. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Considering Gerrit for CFs
Peter Eisentraut pete...@gmx.net writes: On 2/8/13 10:58 AM, Tom Lane wrote: ... I don't object to people using their tools-of-choice to perform reviewing, but we need some way of making sure that the reviews get archived. Gerrit sends me an email every times something happens, so I think this is not going to be a problem. As long as it can be persuaded to mail the text of reviews (not just a notification), that'd probably be all right. What it doesn't support AFAICT is sending emails *in*, but I don't see that as a requirement. Meh. Many of the complaints about the current CF application boil down to the fact that it doesn't accept email input, so I'm not really convinced that a different tool that also doesn't accept email input is going to be a big step forward. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Time for an autoconf update
On 2/8/13 12:21 PM, Tom Lane wrote: Over in Fedora-land they're trying to institute support for ARM64, which among other things means autoconf 2.69 or later: http://lists.fedoraproject.org/pipermail/devel/2013-February/178273.html We are behind the curve. What they actually mean is that they need config.guess and config.sub that is shipped with autoconf 2.69. But the ones in the postgresql source tree are already of the required version. The reason I haven't been pushing for autoconf updates in a while is that the release notes of recent versions consist mostly of fix regression in previous release and no actual features that would be of use in PostgreSQL's configure script. This should be revisited from time to time, but it's probably better to do that near the beginning of a development cycle. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] backup.sgml patch that adds information on custom format backups
Hello, I'd like to submit the following patch that extends backup.sgml with a bit of practical but important information. Project: postgresql Patch filename: backup.sgml-cmd-v001.patch The patch extends backup.sgml and adds practical information on custom format backups approach. Basically, we believe that plaintext backup format is suitable for a very limited range of use cases, and that in real world people are usually better off with a custom format backup. This is what we want PostgreSQL users to be aware of and provide some hands-on examples of how to do backups using this approach. It is meant for application, and is against master branch. The patch does pass 'make check' and 'make html' successfully. PS: this is my first submission ever. So, if I'm missing something or not doing it as expected, please, do let me know. Thank you. Ivan backup.sgml-cmd-v001.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Considering Gerrit for CFs
I thought this might be of interest... http://blog.documentfoundation.org/2013/02/07/the-document-foundation-announces-libreoffice-4-0/ [...] Improved code contribution thanks to Gerrit: a web based code review system, facilitating the task for projects using Git version control system (although this is not specific of LibreOffice 4.0, it has entered the production stage just before the 4.0 branch) [...]. Cheers, Gavin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Considering Gerrit for CFs
On Fri, Feb 8, 2013 at 10:20 AM, Peter Eisentraut pete...@gmx.net wrote: On 2/8/13 5:23 AM, Magnus Hagander wrote: But do you have any actual proof that the problem is in we loose reviewers because we're relying on email? Here is one: Me. Just yesterday I downloaded a piece of software that was previously unknown to me from GitHub and found a bug. Within 15 minutes or so I had fixed the bug, made a fork, sent a pull request. Today I read, the fix was merged last night, and I'm happy. How would this go with PostgreSQL? You can use the bug form on the web site, but you can't attach any code, so the bug will just linger and ultimately put more burden on a core contributor to deal with the minutiae of developing, testing, and committing a trivial fix and sending feedback to the submitter. Or the user could take the high road and develop and patch and submit it. Just make sure it's in context diff format! Search the wiki if you don't know how to do that! Send it to -hackers, your email will be held for moderation. We won't actually do anything with your patch, but we will tell you to add it to that commitfest app over there. You need to sign up for an account to use that. We will deal with your patch in one or two months. But only if you review another patch. And you should sign up for that other mailing list, to make sure you're doing it right. Chances are, the first review you're going to get is that your patch doesn't apply anymore, but which time you will have lost interest in the patch anyway. This. This times 1000. So, I don't have any further evidence that we are losing reviewers, but in light of the above and the options out there were interested developers can contribute much more easily, I'm amazed that we are getting any new contributors or reviewers at all. Of course, Gerrit doesn't actually address most of the issues above, but it could be part of a step forward. I'm not sure if Gerrit specifically is the answer, but there are definitely better ways to do code review like this. I really like the way github allows you to post a patch and then have conversation around it, offer comments on specific lines of code, and add updates to the patch all in one interface. Another benefit is that a lot more people are familiar and comfortable with this work flow. There are even some open source work-a-likes that we could use to we don't have to rely on a 3rd party like github. Gerrit seems to do it slightly differently with side by side diff's and patch revisions, but either way would be an improvement. I understand there are other concerns in this thread, like email, etc. I don't have a comprehensive plan that solves all this, but I wanted to add my +1 to the idea of something more sophisticated when it comes to code review. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Considering Gerrit for CFs
On Fri, Feb 8, 2013 at 7:20 AM, Peter Eisentraut pete...@gmx.net wrote: On 2/8/13 5:23 AM, Magnus Hagander wrote: But do you have any actual proof that the problem is in we loose reviewers because we're relying on email? Here is one: Me. Just yesterday I downloaded a piece of software that was previously unknown to me from GitHub and found a bug. Within 15 minutes or so I had fixed the bug, made a fork, sent a pull request. Today I read, the fix was merged last night, and I'm happy. I know quite a bit using git for my own work, but I haven't the foggiest idea how to make a fork (unless that is the same as making a branch?) or to send a pull request, and bet it would take me more than 15 minutes to figure it out and make sure I understood them and did it correctly. Surely using any specific tool would make things easier for that pool of people who are already well versed in that tool. How would this go with PostgreSQL? You can use the bug form on the web site, but you can't attach any code, Should it allow you to attach code? If you have code to attach, should it instead go to hackers? Or send it to bugs by email rather than using the form? (Some parts of the web site sound like the form is preferred over direct email to bugs, while others make it sound that both are equal) so the bug will just linger and ultimately put more burden on a core contributor to deal with the minutiae of developing, testing, and committing a trivial fix and sending feedback to the submitter. Or the user could take the high road and develop and patch and submit it. Just make sure it's in context diff format! Search the wiki if you don't know how to do that! Send it to -hackers, your email will be held for moderation. We won't actually do anything with your patch, That sounds more like a feature submission. My experience with bugs is that I send a patch or just a code snippet, either to hackers or bugs, and then someone, usually Tom, rewrites it to be better and to work for corner-cases, then commits it. Only for complicated bugs that are arguably not really bugs but rather mal-features that need to be redesigned would I be asked to use the commitfest process at all. but we will tell you to add it to that commitfest app over there. You need to sign up for an account to use that. We will deal with your patch in one or two months. But only if you review another patch. And you should sign up for that other mailing list, to make sure you're doing it right. Chances are, the first review you're going to get is that your patch doesn't apply anymore, but which time you will have lost interest in the patch anyway. This too does not sound like how bug reports actually work. Nor does it sound like how very simple enhancements work (i.e. several of my tab-completion enhancements or doc changes), which are usually just summarily committed regardless of the commitfest cycle or whether I review other patches. You are comparing making a drive-by contribution to one project, to being part of the developer community of a different one. Cheers, Jeff -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Considering Gerrit for CFs
On Fri, Feb 8, 2013 at 2:23 AM, Magnus Hagander mag...@hagander.net wrote: On Fri, Feb 8, 2013 at 1:32 AM, Josh Berkus j...@agliodbs.com wrote: 8. Send it to pgsql-hackers 8.a. this requires you to be subscribed to pgsql-hackers. No, it does not. It will get caught in the moderation queue and get slightly delayed if you're not, but it works perfectly fine. http://www.postgresql.org/docs/current/static/bug-reporting.html Note: Due to the unfortunate amount of spam going around, all of the above email addresses are closed mailing lists. That is, you need to be subscribed to a list to be allowed to post on it. Is this wrong, or just strategically over-simplified? Cheers, Jeff -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Considering Gerrit for CFs
Jeff Janes escribió: On Fri, Feb 8, 2013 at 2:23 AM, Magnus Hagander mag...@hagander.net wrote: On Fri, Feb 8, 2013 at 1:32 AM, Josh Berkus j...@agliodbs.com wrote: 8. Send it to pgsql-hackers 8.a. this requires you to be subscribed to pgsql-hackers. No, it does not. It will get caught in the moderation queue and get slightly delayed if you're not, but it works perfectly fine. http://www.postgresql.org/docs/current/static/bug-reporting.html Note: Due to the unfortunate amount of spam going around, all of the above email addresses are closed mailing lists. That is, you need to be subscribed to a list to be allowed to post on it. Is this wrong, or just strategically over-simplified? Well, it is factually wrong, because the moderators will see such requests and approve them. And the parenthical remark following the quoted part is wrong too, in a sense, because even though using the form does not require you to be subscribed, the report will not be posted until a moderator approves it; so it has the same problem. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Considering Gerrit for CFs
On Fri, Feb 8, 2013 at 1:43 PM, Phil Sorber p...@omniti.com wrote: On Fri, Feb 8, 2013 at 10:20 AM, Peter Eisentraut pete...@gmx.net wrote: On 2/8/13 5:23 AM, Magnus Hagander wrote: But do you have any actual proof that the problem is in we loose reviewers because we're relying on email? Here is one: Me. Just yesterday I downloaded a piece of software that was previously unknown to me from GitHub and found a bug. Within 15 minutes or so I had fixed the bug, made a fork, sent a pull request. Today I read, the fix was merged last night, and I'm happy. How would this go with PostgreSQL? You can use the bug form on the web site, but you can't attach any code, so the bug will just linger and ultimately put more burden on a core contributor to deal with the minutiae of developing, testing, and committing a trivial fix and sending feedback to the submitter. Or the user could take the high road and develop and patch and submit it. Just make sure it's in context diff format! Search the wiki if you don't know how to do that! Send it to -hackers, your email will be held for moderation. We won't actually do anything with your patch, but we will tell you to add it to that commitfest app over there. You need to sign up for an account to use that. We will deal with your patch in one or two months. But only if you review another patch. And you should sign up for that other mailing list, to make sure you're doing it right. Chances are, the first review you're going to get is that your patch doesn't apply anymore, but which time you will have lost interest in the patch anyway. This. This times 1000. I, too, could not agree more. I'm not sure if Gerrit specifically is the answer, but there are definitely better ways to do code review like this. I really like the way github allows you to post a patch and then have conversation around it, offer comments on specific lines of code, and add updates to the patch all in one interface. Another benefit is that a lot more people are familiar and comfortable with this work flow. There are even some open source work-a-likes that we could use to we don't have to rely on a 3rd party like github. Gerrit seems to do it slightly differently with side by side diff's and patch revisions, but either way would be an improvement. Please take this for what it's worth - I'm not a code reviewer or committer - just a pretty heavy user, and I lurk on (most?) of the mailing lists. Mostly I find bugs and ask others to fix them, since I lack the necessary intimate knowledge of postgresql internals to produce a meaningful patch. That said, I believe that - from my perspective - having postgresql's interaction with it's *large* community would only be improved by using something like github. I am far more likely to try to introduce a new feature, minor bugfix, code improvement, et cetera when using github than I would be if the interaction starts with a post to a mailing list and at least /looks/ like it might involve rather more than that. -- Jon -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] pgsql: Clean up c.h / postgres.h after Assert() move
commit 381d4b70a9854a7b5b9f12d828a0824f8564f1e7 introduced some compiler warnings: assert.c:26: warning: no previous prototype for 'ExceptionalCondition' elog.c: In function 'pg_re_throw': elog.c:1628: warning: implicit declaration of function 'ExceptionalCondition' elog.c:1630: warning: 'noreturn' function does return -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] sepgsql and materialized views
Kohei KaiGai kai...@kaigai.gr.jp wrote:
I'll adjust contrib/sepgsql portion to fit materialized-view with
matter of existing view.
OK. In case it is of any use to you as a starting point, attached
is what I originally had, which seems to be similar to what you
describe as your preference. I'll revert everything under
contrib/sepgsql/ and wait for a patch from you.
If you have something prior to a commit to the community repo, you
can work against:
https://github.com/kgrittn/postgres/commits/matview
-Kevindiff --git a/contrib/sepgsql/dml.c b/contrib/sepgsql/dml.c
index c3ef2b7..4db5883 100644
--- a/contrib/sepgsql/dml.c
+++ b/contrib/sepgsql/dml.c
@@ -191,6 +191,7 @@ check_relation_privileges(Oid relOid,
switch (relkind)
{
case RELKIND_RELATION:
+ case RELKIND_MATVIEW:
result = sepgsql_avc_check_perms(object,
SEPG_CLASS_DB_TABLE,
required,
@@ -226,7 +227,7 @@ check_relation_privileges(Oid relOid,
/*
* Only columns owned by relations shall be checked
*/
- if (relkind != RELKIND_RELATION)
+ if (relkind != RELKIND_RELATION relkind != RELKIND_MATVIEW)
return true;
/*
diff --git a/contrib/sepgsql/label.c b/contrib/sepgsql/label.c
index a5bdde3..7ebf525 100644
--- a/contrib/sepgsql/label.c
+++ b/contrib/sepgsql/label.c
@@ -764,6 +764,8 @@ exec_object_restorecon(struct selabel_handle * sehnd, Oid catalogId)
objtype = SELABEL_DB_SEQUENCE;
else if (relForm-relkind == RELKIND_VIEW)
objtype = SELABEL_DB_VIEW;
+else if (relForm-relkind == RELKIND_MATVIEW)
+ objtype = SELABEL_DB_MATVIEW;
else
continue; /* no need to assign security label */
@@ -782,7 +784,8 @@ exec_object_restorecon(struct selabel_handle * sehnd, Oid catalogId)
case AttributeRelationId:
attForm = (Form_pg_attribute) GETSTRUCT(tuple);
-if (get_rel_relkind(attForm-attrelid) != RELKIND_RELATION)
+if (get_rel_relkind(attForm-attrelid) != RELKIND_RELATION
+ get_rel_relkind(attForm-attrelid) != RELKIND_MATVIEW)
continue; /* no need to assign security label */
objtype = SELABEL_DB_COLUMN;
diff --git a/contrib/sepgsql/relation.c b/contrib/sepgsql/relation.c
index a277fab..feaecfd 100644
--- a/contrib/sepgsql/relation.c
+++ b/contrib/sepgsql/relation.c
@@ -54,8 +54,8 @@ sepgsql_attribute_post_create(Oid relOid, AttrNumber attnum)
Form_pg_attribute attForm;
/*
- * Only attributes within regular relation have individual security
- * labels.
+ * Only attributes within regular relation or materialized view have
+ * individual security labels.
*/
if (get_rel_relkind(relOid) != RELKIND_RELATION)
return;
@@ -159,7 +159,8 @@ sepgsql_attribute_relabel(Oid relOid, AttrNumber attnum,
ObjectAddress object;
char *audit_name;
- if (get_rel_relkind(relOid) != RELKIND_RELATION)
+ if (get_rel_relkind(relOid) != RELKIND_RELATION
+ get_rel_relkind(relOid) != RELKIND_MATVIEW)
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
errmsg(cannot set security label on non-regular columns)));
@@ -263,6 +264,10 @@ sepgsql_relation_post_create(Oid relOid)
tclass = SEPG_CLASS_DB_VIEW;
tclass_text = view;
break;
+ case RELKIND_MATVIEW:
+ tclass = SEPG_CLASS_DB_MATVIEW;
+ tclass_text = materialized view; /* TODO: matview? */
+ break;
case RELKIND_INDEX:
/* deal with indexes specially; no need for tclass */
sepgsql_index_modify(relOid);
@@ -301,10 +306,11 @@ sepgsql_relation_post_create(Oid relOid)
SetSecurityLabel(object, SEPGSQL_LABEL_TAG, rcontext);
/*
- * We also assigns a default security label on columns of the new regular
- * tables.
+ * We also assign a default security label on columns of new regular
+ * tables and materialized views.
*/
- if (classForm-relkind == RELKIND_RELATION)
+ if (classForm-relkind == RELKIND_RELATION ||
+ classForm-relkind == RELKIND_MATVIEW)
{
Relation arel;
ScanKeyData akey;
@@ -386,6 +392,9 @@ sepgsql_relation_drop(Oid relOid)
case RELKIND_VIEW:
tclass = SEPG_CLASS_DB_VIEW;
break;
+ case RELKIND_MATVIEW:
+ tclass = SEPG_CLASS_DB_MATVIEW;
+ break;
case RELKIND_INDEX:
/* ignore indexes on toast tables */
if (get_rel_namespace(relOid) == PG_TOAST_NAMESPACE)
@@ -420,7 +429,7 @@ sepgsql_relation_drop(Oid relOid)
}
/*
- * check db_table/sequence/view:{drop} permission
+ * check db_table/sequence/view/matview:{drop} permission
*/
object.classId = RelationRelationId;
object.objectId = relOid;
@@ -436,6 +445,8 @@ sepgsql_relation_drop(Oid relOid)
/*
* check db_column:{drop} permission
+ *
+ * TODO: Anything to do here for materialized views?
*/
if (relkind == RELKIND_RELATION)
{
@@ -489,11 +500,13 @@ sepgsql_relation_relabel(Oid relOid, const char *seclabel)
tclass = SEPG_CLASS_DB_SEQUENCE;
else if (relkind == RELKIND_VIEW)
tclass = SEPG_CLASS_DB_VIEW;
+ else if (relkind == RELKIND_MATVIEW)
+ tclass = SEPG_CLASS_DB_MATVIEW;
else
ereport(ERROR,
Re: [HACKERS] pgsql: Clean up c.h / postgres.h after Assert() move
Jeff Janes escribió: commit 381d4b70a9854a7b5b9f12d828a0824f8564f1e7 introduced some compiler warnings: assert.c:26: warning: no previous prototype for 'ExceptionalCondition' elog.c: In function 'pg_re_throw': elog.c:1628: warning: implicit declaration of function 'ExceptionalCondition' elog.c:1630: warning: 'noreturn' function does return Oh, I see the problem -- you don't have asserts enabled. Doh. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Time for an autoconf update
Peter Eisentraut pete...@gmx.net writes: On 2/8/13 12:21 PM, Tom Lane wrote: Over in Fedora-land they're trying to institute support for ARM64, which among other things means autoconf 2.69 or later: http://lists.fedoraproject.org/pipermail/devel/2013-February/178273.html What they actually mean is that they need config.guess and config.sub that is shipped with autoconf 2.69. But the ones in the postgresql source tree are already of the required version. [ looks... ] Ah, you're right, and it's even true in 9.2 so I won't be needing a patch for that. Excellent, thanks. The reason I haven't been pushing for autoconf updates in a while is that the release notes of recent versions consist mostly of fix regression in previous release and no actual features that would be of use in PostgreSQL's configure script. This should be revisited from time to time, but it's probably better to do that near the beginning of a development cycle. Agreed, if there are no features or bugfixes that affect us then there's no particular need to update. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Incorrect behaviour when using a GiST index on points
Alexander Korotkov aekorot...@gmail.com writes: On Sat, Nov 3, 2012 at 4:23 AM, Noah Misch n...@leadboat.com wrote: ... At internal pages, gist_point_consistent() should implement point @ box with an algorithm near-equivalent to box_overlap(). (As an optional deviation, it may use exact comparisons despite box_overlap() using fuzzy comparisons.) Looking at the math again, your latest code does achieve that, too. I was thrown off by your use of a different, albeit mathematically equivalent, algorithm from the one used in box_overlap(). Please don't do that; either use box_overlap()'s algorithm here, or change box_overlap() to use the shorter algorithm you have introduced. Formulating the same calculation differently in related code is a recipe for confusion. (Then again, perhaps the equivalence of the algorithms is obvious to everyone entitled to travel within 1 km of the geometric type implementation.) I've added comment for clarifying this situation. Applied and back-patched with some cosmetic changes (mostly the comments) and a better version of the regression test. As a separate commit, I also simplified box_overlap() to match this logic, since I agree with Noah that it's not good for them to look so different. Besides, it should be at least a bit faster this way. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Identity projection
Friday, February 08, 2013 11:06 PM Tom Lane wrote:
Amit Kapila amit(dot)kapila(at)huawei(dot)com writes:
On Friday, February 08, 2013 12:00 AM Tom Lane wrote:
As per my understanding, currently in code wherever Result node can be
avoided,
it calls function is_projection_capable_plan(), so we can even enhance
is_projection_capable_plan()
so that it can also verify the expressions of tlists. But for this we need
to change at all places
from where is_projection_capable_plan() is called.
Hm. Really there's a whole dance that typically goes on, which is like
if (!is_projection_capable_plan(result_plan))
Perhaps we could encapsulate this whole sequence into a function called
say assign_targetlist_to_plan(), which would have the responsibility to
decide whether a Result node needs to be inserted.
If we want to encapsulate whole of above logic in assign_targetlist_to_plan(),
then the responsibility of new functionwill be much higher, because the code
that
assigns targetlist is not same at all places.
For example
Related code in prepare_sort_from_pathkeys() is as below where it needs to
append junk entry to target list.
if (!adjust_tlist_in_place !is_projection_capable_plan(lefttree))
{ /* copy needed so we don't modify input's tlist
below */
tlist = copyObject(tlist);
lefttree = (Plan *) make_result(root, tlist, NULL,
lefttree);
}
/* Don't bother testing is_projection_capable_plan again */
adjust_tlist_in_place = true;
/*
* Add resjunk entry to input's tlist
*/
tle = makeTargetEntry(sortexpr,
list_length(tlist) + 1,
NULL,
true);
tlist = lappend(tlist, tle);
lefttree-targetlist = tlist; /* just in case
NIL before */
Similar kind of code is there in grouping_planner for the case of
activeWindows.Now we can change the code such that places where any new target
entry has to be added to target list, move that part of code before calling
assign_targetlist_to_plan or pass extra parameters to
assign_targetlist_to_plan, so that it can accomodate all such cases. The story
doesn't ends there, in some places it has to make a copy of targetlist before
assigning it to plan's targetlist.
How about if just enhance the code as below:
if (!is_projection_capable_plan(result_plan) compare_tlist_exprs(sub_tlist,
result_plan-targetlist) )
{
result_plan = (Plan *) make_result(root,
sub_tlist,
NULL,
result_plan);
where the new function will be something as below:
bool
compare_tlist_exprs(List *tlist1, List *tlist2) {
ListCell *lp,*lc;
if (list_length(tlist1) != list_length(tlist2))
return false;/* tlists not same length */
forboth(lp, tlist1, lc, tlist2) {
TargetEntry *ptle = (TargetEntry *) lfirst(lp);
TargetEntry *ctle = (TargetEntry *) lfirst(lc);
if(!equal(ptle-expr,ctle-expr))
return false;
}
return true;
}
With Regards,
Amit Kapila.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Identity projection
Saturday, February 09, 2013 6:56 AM Amit kapila wrote: Friday, February 08, 2013 11:06 PM Tom Lane wrote: Amit Kapila amit(dot)kapila(at)huawei(dot)com writes: On Friday, February 08, 2013 12:00 AM Tom Lane wrote: As per my understanding, currently in code wherever Result node can be avoided, Hm. Really there's a whole dance that typically goes on, which is like if (!is_projection_capable_plan(result_plan)) Perhaps we could encapsulate this whole sequence into a function called say assign_targetlist_to_plan(), which would have the responsibility to decide whether a Result node needs to be inserted. if (!is_projection_capable_plan(result_plan) compare_tlist_exprs(sub_tlist, result_plan-targetlist) ) Sorry, the check I suggested in last mail should be as below: if (!is_projection_capable_plan(result_plan) !compare_tlist_exprs(sub_tlist, result_plan-targetlist) ) With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Identity projection
Amit kapila amit.kap...@huawei.com writes: if (!is_projection_capable_plan(result_plan) compare_tlist_exprs(sub_tlist, result_plan-targetlist) ) Sorry, the check I suggested in last mail should be as below: if (!is_projection_capable_plan(result_plan) !compare_tlist_exprs(sub_tlist, result_plan-targetlist) ) You know, I was thinking that compare_tlist_exprs() was a pretty unhelpfully-chosen name for a function returning boolean, and this thinko pretty much proves the point. It'd be better to call it something like equivalent_tlists(), tlists_are_equivalent(), etc. (I'm not caring for the emphasis on the exprs either, because I think it'll also be necessary to compare resjunk fields for instance.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] missing rename support
Dean Rasheed dean.a.rash...@gmail.com writes: [ alter-rule-rename_complete.v2.patch ] Committed with assorted editorialization. Aside from cosmetic issues, the main changes were: * use RangeVarGetRelidExtended with a callback to perform the lookup and locking of the target relation. This is a new API that the original version of RenameRewriteRule couldn't have known about. I borrowed the code pretty much verbatim from renametrig(), and am now wondering whether there shouldn't be some attempt to unify the callbacks for this. * call CacheInvalidateRelcache to ensure that other sessions notice the rule tuple update. It may be that this isn't necessary because nothing looks at the rule name fields in relcache entries ... but I wouldn't bet on that, and in any case it seems like bad practice to let stale cache entries hang around. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Too frequent checkpoints ?
Also, the log says checkpoints are occurring too frequently (0 seconds apart). But that looks wrong too. Checkpoints are really happening at 30 seconds apart and not 0 as the log message claims. I noticed that as well, and I think it might be a regression. I'll run a bisection on it if I get a chance. Looking further into it, seems there are two checkpoints happening almost immediately. The first happens because of timeout and the second happens immediately because of checkpoint segments (set to 1 in this case) Given that no one has bothered to fix these issues, may be they are non issues after all. Thanks, Pavan Cheers, Jeff -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Considering Gerrit for CFs
On Fri, Feb 8, 2013 at 2:23 AM, Magnus Hagander mag...@hagander.net wrote: On Fri, Feb 8, 2013 at 1:32 AM, Josh Berkus j...@agliodbs.com wrote: This is a few too many steps, and certainly appears completely broken to any newcomer. I agree it's way too many step. Several of those can certainly be made more efficient now that we have a more sane archives, well within the scope of the current system. I have thought it'd be 'nice' if pre-generated binaries and git repos were made for each submitted patch. Is there a nice-and-tidy way to paginate over list traffic from an external program over The Internet? Is there/could there be a nice pagination marker? Is that a reasonable step zero to making other things that consume email? -- fdr -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
