Re: [HACKERS] macports and brew postgresql --universal builds
On 08/10/2012 06:44 PM, Tom Lane wrote: Doug Coleman writes: The MacPorts Project (http://www.macports.org/) supports building universal binaries (32/64bit binaries in one file) for most projects. For PostgreSQL, they apply two patches after the configure script to correct some of the typedefs. Otherwise, the build fails in compiling a switch statement with duplicate cases. The HomeBrew Project (http://mxcl.github.com/homebrew/) is a worthy successor to MacPorts, but they don't yet support a --universal argument for building PostgreSQL. There is currently a pull request on github to add universal support, but it's based on the patches from the MacPorts project. The files you link to don't make much sense to me (they do not look like patch diffs) OMG, I think they are ed scripts, possibly produced using "diff -e". I'm having flashbacks to when I used such stuff twenty or so years ago. cheers andrew - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] -Wformat-zero-length
On Sat, Aug 11, 2012 at 01:48:13AM +0200, Dimitri Fontaine wrote: > > >> Another thing worth considering is to have pg_upgrade init, stop and > >> start clusters as necessary instead of requesting the user to do it. > >> I think this is two less steps. > > > > Then you'd need to expose the entire pg_ctl shutdown mode logic through > > pg_upgrade, which might not make things simpler. > > What about having single user mode talk fe/be protocol, and talk to it via a > UNIX pipe, with pg_upgrade starting the single user backend as a subprocess? You still need to run libpq applications, so would be doing some process juggling to get the libpq clients to talk to a pipe. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] -Wformat-zero-length
>> Another thing worth considering is to have pg_upgrade init, stop and >> start clusters as necessary instead of requesting the user to do it. >> I think this is two less steps. > > Then you'd need to expose the entire pg_ctl shutdown mode logic through > pg_upgrade, which might not make things simpler. What about having single user mode talk fe/be protocol, and talk to it via a UNIX pipe, with pg_upgrade starting the single user backend as a subprocess? -- dim - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug of pg_trgm?
Fujii Masao writes: > On Thu, Aug 9, 2012 at 3:05 AM, Tom Lane wrote: >> Probably a minimal fix for this could be made by backing up "endword" >> one byte before returning it if in_escape is true when the second >> loop exits. That would not scale up to preserving the state of >> in_wildcard_meta, but since the second loop never advances past a >> meta char, that's okay for the moment. > Or what about extending get_wildcard_part() so that it accepts the pointer > to in_escape as an argument? generate_wildcard_trgm() can know the last > value of in_escape and specify it the next call of get_wildcard_part(). Looks > very simple. Yeah, I had considered pushing the state variables out to the caller. If there were any prospect of wanting more state than just in_escape, I'd be for that --- but I don't see any reason to possibly need more, especially in view of your point that in_wildcard_meta isn't really a single flag with an interpretation that remains fixed throughout. I think it's probably better just to take care of the issue inside get_wildcard_part, and not complicate its API. regards, tom lane - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 16-bit page checksums for 9.2
On Sun, 2012-02-19 at 21:49 +, Simon Riggs wrote: > On Thu, Feb 16, 2012 at 11:16 AM, Simon Riggs wrote: > > > v8 attached > > v10 attached. > > This patch covers all the valid concerns discussed and has been > extensively tested. Is there something I can do to help get this ready for the next commitfest? I am willing to rebase it, but I was worried that might cause confusion. I am also willing to review it after the rebase, of course. There are still a couple open issues, including: * Store the checksum in the page version field or the TLI field? * What mechanism to guarantee to the user that all pages are properly protected by checksums (rather than just new pages)? In other words, there are more than the two states represented by the GUC. * What specific data is included in the checksum? I suggested that we include the block number, and maybe the OID. Regards, Jeff Davis - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug of pg_trgm?
Fujii Masao writes: > On Thu, Aug 9, 2012 at 2:28 AM, Tom Lane wrote: >> I'm also inclined to think that we should remove *both* flag resets >> before the second loop. The logic here is that we are reprocessing >> the same character seen in the last iteration of the first loop, >> right? So the flag state ought to remain the same. > No. ISTM that in_wildcard_meta must be reset before the second loop. > Because the meaning of that flag in the first loop is different from that in > the second loop. The former and the latter indicate whether the search > string has *heading* and *tailing* wildcard character, respectively. No? Oh, good point. Maybe it would be clearer to use two separate flag variables? The thought I'd had was that the flag would necessarily get reset during the first iteration of the second loop, which means it all ends up the same anyway. But if we want to think of the flag as meaning two different things for the two loops, I'd be inclined to use two variables. regards, tom lane - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] macports and brew postgresql --universal builds
What it looks like is the first line of each section is pattern matching. If __LP64__ is defined, then it's a 64-bit architecture, and we want to use the top part of the if statement. The #defines they target seem to be all of the ones that are different on 32bit platforms. I agree that you would want to do this in the configure script somehow. Doug On Fri, Aug 10, 2012 at 3:44 PM, Tom Lane wrote: > Doug Coleman writes: >> The MacPorts Project (http://www.macports.org/) supports building >> universal binaries (32/64bit binaries in one file) for most projects. >> For PostgreSQL, they apply two patches after the configure script to >> correct some of the typedefs. Otherwise, the build fails in compiling >> a switch statement with duplicate cases. > >> The HomeBrew Project (http://mxcl.github.com/homebrew/) is a worthy >> successor to MacPorts, but they don't yet support a --universal >> argument for building PostgreSQL. There is currently a pull request on >> github to add universal support, but it's based on the patches from >> the MacPorts project. > > The files you link to don't make much sense to me (they do not look > like patch diffs) but they seem to suggest hard-wiring configure results > into the source code, which does not sound like an acceptable solution > from our standpoint. > > The approach we've suggested to people in the past is running configure > for each architecture and then building against that copy of > pg_config.h, or more likely combining the .h files with arch-specific > #ifdefs. You can find more info in our list archives --- the most > relevant thread I could find easily starts here: > http://archives.postgresql.org/pgsql-hackers/2008-07/msg00884.php > > regards, tom lane - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] macports and brew postgresql --universal builds
On Fri, Aug 10, 2012 at 3:44 PM, Tom Lane wrote: > The files you link to don't make much sense to me (they do not look > like patch diffs) but they seem to suggest hard-wiring configure results > into the source code, which does not sound like an acceptable solution > from our standpoint. I've also never seen this format before, which are...ed expressions. I guess that makes them less sensitive to line churn. Cute, but...wow. -- fdr - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Mailsystem maintenance/migration announcement
Excerpts from Stefan Kaltenbrunner's message of lun ago 06 13:21:49 -0400 2012: > We are currently planning to finalize the ongoing work on the mailsystem > migration we started earlier this year by migrating the > two remaining components of the postgresql.org mailsystem infrastructure > to new systems. > Those parts (listserver and mailbox hosting) will be moved to new > systems in an maintenance window on: > > Friday, 10th of august starting 15:00 GMT ... and finally, after a much longer downtime window than originally expected, we have completed the migration to the new mail system. We're really sorry for the inconvenience and the long downtime. We had unexpected absence of one people of the involved (me) due to an unforeseen unavoidable compromise; and we furthermore had to cope with one other person getting very ill at the last minute. It also turns out that we made a serious mistake in the configuration somewhere and this might have resulted in some undesirable emails being sent out to subscribers of various lists. Sorry about that. Please ignore those; they should not show up again. If they do, please report back. The new system is not completely debugged yet, alas. It will take some time before we really figure out everything so that lists work at least as quickly as they did in the old site; we hope to have that figured that out as soon as possible. In the meantime, things might be a little slow, but emails should get out there. If you see anything not working as it should, be sure to ping us by replying here. Your friendly email admin team, Magnus Hagander, Álvaro Herrera, Stefan Kaltenbrunner -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] macports and brew postgresql --universal builds
Doug Coleman writes: > The MacPorts Project (http://www.macports.org/) supports building > universal binaries (32/64bit binaries in one file) for most projects. > For PostgreSQL, they apply two patches after the configure script to > correct some of the typedefs. Otherwise, the build fails in compiling > a switch statement with duplicate cases. > The HomeBrew Project (http://mxcl.github.com/homebrew/) is a worthy > successor to MacPorts, but they don't yet support a --universal > argument for building PostgreSQL. There is currently a pull request on > github to add universal support, but it's based on the patches from > the MacPorts project. The files you link to don't make much sense to me (they do not look like patch diffs) but they seem to suggest hard-wiring configure results into the source code, which does not sound like an acceptable solution from our standpoint. The approach we've suggested to people in the past is running configure for each architecture and then building against that copy of pg_config.h, or more likely combining the .h files with arch-specific #ifdefs. You can find more info in our list archives --- the most relevant thread I could find easily starts here: http://archives.postgresql.org/pgsql-hackers/2008-07/msg00884.php regards, tom lane - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] macports and brew postgresql --universal builds
The MacPorts Project (http://www.macports.org/) supports building universal binaries (32/64bit binaries in one file) for most projects. For PostgreSQL, they apply two patches after the configure script to correct some of the typedefs. Otherwise, the build fails in compiling a switch statement with duplicate cases. The HomeBrew Project (http://mxcl.github.com/homebrew/) is a worthy successor to MacPorts, but they don't yet support a --universal argument for building PostgreSQL. There is currently a pull request on github to add universal support, but it's based on the patches from the MacPorts project. It would be nice if the PostgreSQL project would make these patches unnecessary. A diff of the HomeBrew pull request to add universal support: https://github.com/mxcl/homebrew/pull/14111/files The patches in question: https://trac.macports.org/export/96361/trunk/dports/databases/postgresql91/files/pg_config.h.ed https://trac.macports.org/export/96361/trunk/dports/databases/postgresql91/files/ecpg_config.h.ed If someone were to fix this, to test if the fix worked you would need a Mac with HomeBrew, the HomeBrew pull request above but without lines 70-74, and this command: brew install postgresql --universal Note that if you previously had postgresql installed with HomeBrew, you would need to uninstall it first. Cheers, Doug - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] -Wformat-zero-length
On 8/8/12 5:29 PM, Alvaro Herrera wrote: I think those 14 is a bit of a made-up number. Several of those steps are about building pg_upgrade, not actually using it. And there are some that are optional anyway. Compare the pg_upgrade instructions http://www.postgresql.org/docs/9.2/static/pgupgrade.html to the old pg_dump-based upgrade instructions that we used to give people: http://www.postgresql.org/docs/8.4/static/install-upgrading.html They are about the same in complexity (the pg_dump approach didn't talk about updating statistics or removing the old cluster, so it has less steps). So I don't think the number of steps is a problem at all. They could be simplified a little bit more, of course. What's more of a problem in my mind is the "unknown unknowns" in pg_upgrade's approach. In the pg_dump/restore approach, you knew that if the dump succeeded and the restore succeeded, your new database was very likely good. The only problem could be that pg_dump forgot to dump something altogether, or that there is a general problem in executing SQL commands, both of which would be obvious problems. With pg_upgrade, however, you never know whether your new instance is good. You could notice problems months later. That's a really tough proposition. Another thing worth considering is to have pg_upgrade init, stop and start clusters as necessary instead of requesting the user to do it. I think this is two less steps. Then you'd need to expose the entire pg_ctl shutdown mode logic through pg_upgrade, which might not make things simpler. I wonder if things would be facilitated by having a config file for pg_upgrade to specify binary and PGDATA paths instead of having awkward command line switches. If you want to do that, why not write a shell script? That's what they are for. - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Ability to listen on two unix sockets
Honza Horak writes: > Alternate-port-number support has been removed from the patch, as per > Tom's e-mail from 07/03/12. It can be add in the future, if we really > need it. I've reviewed and committed this. There were some cosmetic things I adjusted, as well as a couple of fairly large non-cosmetic things: * I did not like rearranging the order in which TCP and Unix sockets get opened. It's possible that this comment in postmaster.c * For the same reason, it's best to grab the TCP socket(s) before the * Unix socket. is no longer relevant, but I'm doubtful of that. The reason you had switched them appeared to be to update the SOCKET_DIR line in postmaster.pid before updating the LISTEN_ADDR line, but the only reason to do that is the implementation restriction in AddToDataDirLockFile that it can't update a non-last line in the pidfile. That's not that hard to get rid of, and it's something we'd probably want someday anyway, so I fixed that function and put the socket opening order back as it had been. * The code in pqcomm.c queued another on_proc_exit function for each socket. There was no purpose in that since the first one would do all the work, but the postmaster's on_proc_exit array isn't very large, and it's not hard to foresee the useless entries causing a failure by overflowing the array. Similarly, miscinit.c queued an on_proc_exit function for each lock file it had to get rid of, which was fine as long as there was an upper bound of 2, but now not so much. I fixed them to use similar logic of keeping a list of file names and queueing the on_proc_exit function when adding the first list entry. * You'd fixed TouchSocketFiles to touch all the sockets, but missed making TouchSocketLockFiles touch all their lock files. That would be problematic if /tmp was a non-first entry in the list. I also simplified the GUC interactions by leaving the GUC variable as a simple string and splitting it at the point of use, so that the code is more parallel to what we do with listen_addresses. regards, tom lane - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Out of office
Dear sender, I'm away from the office till 19th of August. In the meantime, please contact my colleague Alfred Vater at alfred.va...@aegaeon.de - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Out of office
Dear sender, I'm away from the office till 19th of August. In the meantime, please contact my colleague Alfred Vater at alfred.va...@aegaeon.de - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Prevent restored WAL files from being archived again Re: Unnecessary WAL archiving after failover
On Thu, Aug 9, 2012 at 8:08 AM, Simon Riggs wrote: > On 29 July 2012 16:01, Fujii Masao wrote: > >> Attached patch changes the startup process so that it creates .done file >> whenever WAL file is successfully restored, whether archive mode is >> enabled or not. The restored WAL files will not be archived again because >> of .done file. > > The proposed patch works, for archiving only, but I don't like the > code. It's a partial refactoring of existing code. > > I prefer to go for a full re-factoring version for HEAD, and a zero > refactoring version for 9.2 since we're deep into beta. Sounds reasonable. > I've committed the simplified version for 9.2, as well as adding > support for streaming which you seem to have missed out. If the streaming case really must be covered, the fix should be backported to 9.1 and 9.0? Because there is the same "problem" in those versions. But I'm not sure if this fix is right thing to do. I didn't add any code to the streaming case because the current behavior exists since 9.0 and AFAIR we've not received the complaints about the behavior. So ISTM changing that behavior is debatable and looks like 9.3 item rather than 9.2. Regards, -- Fujii Masao - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug of pg_trgm?
On Thu, Aug 9, 2012 at 3:05 AM, Tom Lane wrote: > ... btw, I think there is another problem here, which is that > generate_wildcard_trgm will restart get_wildcard_part at the > same place that the second loop exits, which means it would > do the wrong thing if what it returns is a pointer to the > second char of an escape pair. Consider for instance > > foo\\%bar > > The first call of get_wildcard_part will correctly extract "foo", > but then return a pointer to the second backslash. So the second > call will think that the % is escaped, which it is not, leading to > a wrong decision about whether to pad "bar". Good catch! > Probably a minimal fix for this could be made by backing up "endword" > one byte before returning it if in_escape is true when the second > loop exits. That would not scale up to preserving the state of > in_wildcard_meta, but since the second loop never advances past a > meta char, that's okay for the moment. Or what about extending get_wildcard_part() so that it accepts the pointer to in_escape as an argument? generate_wildcard_trgm() can know the last value of in_escape and specify it the next call of get_wildcard_part(). Looks very simple. Regards, -- Fujii Masao - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP patch for consolidating misplaced-aggregate checks
On Thu, Aug 9, 2012 at 5:40 PM, Tom Lane wrote: > Fair enough. I was not sold on doing that either. I would still like > to know if it's okay to use one string with %s for the cases where > there's not a good reason for the "context" to be more than just a > SQL keyword. Given that the SQL keyword is going to be an English word I can't imagine how this could be a big deal for translators. It might not match gender or case or something but only if the reader is automatically mentally translating the keyword into their language and then applying that language's rules to it. At least to me it makes sense to refer to "VALUES" as a singular noun or "LIMIT" as a generic male noun even though "limitation" would be feminine (I had to look that one up though so perhaps I'm not the best person to judge). -- greg - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] error handling in logging hooks
What is the intended way to handle errors in the new logging hook? For example, errors in establishing a connection to wherever a hook wants to send things. The reference implementation pg_logforward just uses fprintf(stderr) to communicate its own errors, which doesn't seem ideal. Calling elog(ERROR) in the logging hook causes death by recursion. It seems that some of the recursion protection infrastructure of elog.c isn't exposed to the public, so it's not obvious how to use it. Should logging hooks handle this themselves (could be complicated, multiple hooks etc.), or should we try to build this into elog.c where the hooks are called? - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug of pg_trgm?
On Thu, Aug 9, 2012 at 2:28 AM, Tom Lane wrote: > Fujii Masao writes: >> When I used pg_trgm, I encountered the problem that the search result of >> SeqScan was the different from that of BitmapScan even if the search >> keyword was the same. Is this a bug? > > Surely. > >> The cause is ISTM that pg_trgm wrongly ignores the heading wildcard >> character (i.e., %) when an escape (i.e., \\) follows the wildcard character. >> Attached patch fixes this. > > This patch doesn't seem quite right to me, though. I agree that given > '%\x...', we should exit the loop with in_wildcard_meta still true. > However, if we have say '%\+...', the loop will continue, and now we > must reset in_wildcard_meta, no? The next character is not adjacent to > a meta. So I think in the "if (in_escape)" block, *both* assignments > should be moved after the iswordchr check. Is there something I'm > missing? No. You're right. We must reset in_wildcard_meta and must ignore %\+ in '%\+...'. > Also, shouldn't we make a similar change in the second loop? I guess > it's not strictly necessary given that that loop will exit as soon as > it sets in_wildcard_meta, but if you want to depend on that then the > reset in the second "if (in_escape)" block is altogether useless. It > seems better to keep the logic of the two loops as similar as possible. Yes. There is another useless reset of in_wildcard_meta in the second loop. We should also remove that? > I'm also inclined to think that we should remove *both* flag resets > before the second loop. The logic here is that we are reprocessing > the same character seen in the last iteration of the first loop, > right? So the flag state ought to remain the same. No. ISTM that in_wildcard_meta must be reset before the second loop. Because the meaning of that flag in the first loop is different from that in the second loop. The former and the latter indicate whether the search string has *heading* and *tailing* wildcard character, respectively. No? Regards, -- Fujii Masao - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Prevent restored WAL files from being archived again Re: Unnecessary WAL archiving after failover
On Thu, Aug 9, 2012 at 11:24 PM, Simon Riggs wrote: > On 9 August 2012 00:08, Simon Riggs wrote: > >> Will look at the refactored version tomorrow. > > > Rather than implement the minimal patch onto HEAD, I've written a > refactored patch. [attached] > This fixes the bugs discussed here, though allows for multiple archive > statuses, rather than just two, as well as removing similar code. > > We've talked about allowing file based replication to cascade, i.e. > having the archiver run on standby and to push files onwards from the > standby. > > The way I'd like to do this is > > If archiving active > * Allow archiver to run on standby > * When xlog file written on standby it sets archive_status to ".standby" > * When archiver runs it will execute archive_command when it sees a > status of ".ready" or it will execute standby_archive_command when it > sees a status of ".standby". In both cases it will set a status of > ".done" when complete. It doesn't need to know whether promotion has > happened, or even when it happened, because the status of the xlog > files is clearly marked. > > We can't use ".ready" to mean two things because then we'd be confused > how to handle files generated during standby but archived after > promotion has taken place. ".done" always means you can clean up the > xlog files now, in all cases. Sounds good to me. Regards, -- Fujii Masao - Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bugs in superuser_reserved_connections and max_wal_senders vs max_connections
On Thu, Aug 9, 2012 at 4:17 PM, Tom Lane wrote:
> Magnus Hagander writes:
>> The check in PostmasterMain():
>> if (ReservedBackends >= MaxBackends)
>> {
>> write_stderr("%s: superuser_reserved_connections must be less
>> than
>> max_connections\n", progname);
>> ExitPostmaster(1);
>> }
>
>> should probably check against MaxConnections instead of MaxBackends, I think?
>
> Yeah, this code probably dates from before there was a difference.
That was my guess as well.
> In general, I'd bet that this is not the only place where the wrong one
> of those variables is being consulted.
Probably :)
I've fixed this issue. Didn't spot another one with a quick check, but
there's quite possibly something hiding somewhere, yes :)
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
