[HACKERS] pg_upgrade code questions
I read pg_upgrade code glance over, and found 4 issues in it. Are there any issues to be fixed before 9.0 release? 1. NAMEDATASIZE 2. extern PGDLLIMPORT 3. pathSeparator 4. EDB_NATIVE_LANG 1. NAMEDATASIZE pg_upgrade has the following definition, but should it be just NAMEDATALEN? /* Allocate for null byte */ #define NAMEDATASIZE(NAMEDATALEN + 1) Table names should be in NAMEDATELEN - 1 bytes. At least 64th bytes in name data is always '\0'. =# CREATE TABLE 1234567890...(total 70 chars)...1234567890 (i int); NOTICE: identifier 123...890 will be truncated to 123...0123 2. extern PGDLLIMPORT pg_upgrade has own definitions of extern PGDLLIMPORT Oid binary_upgrade_next_xxx in pg_upgrade_sysoids.c. But those variables are not declared as PGDLLIMPORT in the core. Can we access unexported variables here? 3. pathSeparator Path separator for Windows is not only \ but also /. The current code ignores /. Also, it might not work if the path string including multi-byte characters that have \ (0x5c) in the second byte. 4. EDB_NATIVE_LANG Of course it is commented out with #ifdef, but do we have codes for EDB in core? Regards, --- Takahiro Itagaki NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [ANNOUNCE] Bug-fix and new feature of pg_lesslog is released
Thanks a lot for the comment/advice. Yes, full page backup block considerablly shortens the recovery time. As we discussed about two years ago, I have a solution accelerate the recovery even without full page image. I'd like to submit this solution to the community again. When I evaluated this two years ago, recovery speed was as good as those with full page image, depending upon application and tuning, of course. This is a separate tool and can be used in various scenes. Regards; -- Koichi Suzuki 2010/5/13 Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp: Tom Lane t...@sss.pgh.pa.us wrote: Bruce Momjian br...@momjian.us writes: Yes, I would love to get this into /contrib for PG 9.1! How much are people really going to care about pg_lesslog now that we've got streaming replication? There might be some small use-case still left, but it's hard to believe that it would be worth carrying it in contrib. I hope pg_lesslog would work as a WAL filter of streaming replication. It might be hard-coded in WAL sender, or be an addon based on a new common filtering infrastructure of WAL streaming. Also, there is a long-standing issue in pg_lesslog; It slows down recovery because we need to read data pages before write in recovery. We're avoiding reading pages for full-page image in 8.3, but pg_lesslog will disable the optimization. Recovery routine in core also needs to be adjusted to use read-ahead, like posix_fadvise(). There was another idea, full-page image logs separated with WAL logging. In theory, full-page images don't have to be written at commit, but only by writing corresponding data pages, I'm not sure whether it is an actually good idea or not, but if we go the direction, we won't need pg_lesslog. Regards, --- Takahiro Itagaki NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [ANNOUNCE] Bug-fix and new feature of pg_lesslog is released
WAL streming filter is an interesting idea. We can leave full page backup for local recovery and decrease the amount of WAL to transfer. If SR writes WAL in record by record basis, implementation will be simple. If SR writes WAL in block by block basis, WAL records may be divided from transfer to transfer and it is not simple to handle. Regards; -- Koichi Suzuki 2010/5/13 Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp: Tom Lane t...@sss.pgh.pa.us wrote: Bruce Momjian br...@momjian.us writes: Yes, I would love to get this into /contrib for PG 9.1! How much are people really going to care about pg_lesslog now that we've got streaming replication? There might be some small use-case still left, but it's hard to believe that it would be worth carrying it in contrib. I hope pg_lesslog would work as a WAL filter of streaming replication. It might be hard-coded in WAL sender, or be an addon based on a new common filtering infrastructure of WAL streaming. Also, there is a long-standing issue in pg_lesslog; It slows down recovery because we need to read data pages before write in recovery. We're avoiding reading pages for full-page image in 8.3, but pg_lesslog will disable the optimization. Recovery routine in core also needs to be adjusted to use read-ahead, like posix_fadvise(). There was another idea, full-page image logs separated with WAL logging. In theory, full-page images don't have to be written at commit, but only by writing corresponding data pages, I'm not sure whether it is an actually good idea or not, but if we go the direction, we won't need pg_lesslog. Regards, --- Takahiro Itagaki NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_upgrade code questions
On Thu, 2010-05-13 at 15:13 +0900, Takahiro Itagaki wrote: 4. EDB_NATIVE_LANG Of course it is commented out with #ifdef, but do we have codes for EDB in core? I was about to raise similar thing, for the documentation: http://developer.postgresql.org/pgdocs/postgres/pgupgrade.html This includes some references to EDB AS, which should be removed from PostgreSQL official documentation, IMHO. Regards, -- Devrim GÜNDÜZ PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer PostgreSQL RPM Repository: http://yum.pgrpms.org Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr http://www.gunduz.org Twitter: http://twitter.com/devrimgunduz signature.asc Description: This is a digitally signed message part
Re: [HACKERS] How to know killed by pg_terminate_backend
Tatsuo Ishii wrote: If a backend killed by pg_terminate_backend(), the backend returns 57P01 which is identical to the one when it's killed by postmaster. Problem is, pgpool-II needs to trigger failover if postmaster goes down because apparently pgpool-II cannot use the PostgreSQL server anymore. On the otherhand, pg_terminate_backend() just terminates a backend. So triggering failover is overkill. Maybe we could make PostgreSQL a little bit smarter so that it returns a different code than 57P01 when killed by pg_terminate_backend(). Seems reasonable. Does the victim backend currently know why it has been killed? -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Tags missing from GIT mirror?
On ons, 2010-05-12 at 16:11 -0400, Andrew Dunstan wrote: Of course, we might also find some other brokenness if we try to import all the tags. Also, be aware of this (from http://cvs2svn.tigris.org/cvs2git.html): Differences between CVS and git branch/tag models: CVS allows a branch or tag to be created from arbitrary combinations of source revisions from multiple source branches. It even allows file revisions that were never contemporaneous to be added to a single branch/tag. Git, on the other hand, only allows the full source tree, as it existed at some instant in the history, to be branched or tagged as a unit. Moreover, the ancestry of a git revision makes implications about the contents of that revision. This difference means that it is fundamentally impossible to represent an arbitrary CVS history in a git repository 100% faithfully. Right, and omitting tags was in fact one of the features of fromcvs that made us use it, because any tool that tries to convert tags will explode on our CVS tree, for reasons explained in the above paragraph. We have also discussed this in more detail about three times before. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Tags missing from GIT mirror?
Peter Eisentraut wrote: On ons, 2010-05-12 at 16:11 -0400, Andrew Dunstan wrote: Of course, we might also find some other brokenness if we try to import all the tags. Also, be aware of this (from http://cvs2svn.tigris.org/cvs2git.html): Differences between CVS and git branch/tag models: CVS allows a branch or tag to be created from arbitrary combinations of source revisions from multiple source branches. It even allows file revisions that were never contemporaneous to be added to a single branch/tag. Git, on the other hand, only allows the full source tree, as it existed at some instant in the history, to be branched or tagged as a unit. Moreover, the ancestry of a git revision makes implications about the contents of that revision. This difference means that it is fundamentally impossible to represent an arbitrary CVS history in a git repository 100% faithfully. Right, and omitting tags was in fact one of the features of fromcvs that made us use it, because any tool that tries to convert tags will explode on our CVS tree, for reasons explained in the above paragraph. We have also discussed this in more detail about three times before. Well, yes, but I have been wondering if this has to be an all or nothing deal. How many tags can we not tie to a known tree in git? My suspicion is we can probably identify most of them quite well. If we can that would be nice. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Retiring from the PostgreSQL core team
Jan Wieck wrote: To whom it may concern, this is to inform the PostgreSQL community of my retirement from my PostgreSQL core team position. Over the past years I have not been able to dedicate as much time to PostgreSQL as everyone would have liked. The main reason for that was that I was swamped with other work and private matters and simply didn't have time. I did follow the mailing lists but did not participate much. Your good humor and technical brilliance have been sorely missed. Looking at my publicly visible involvement over the last two years or so, there is little that would justify me being on the core team today. I was not involved in the release process, in patch reviewing, organizing and have contributed little. However, in contrast to other previous core team members, I do not plan to disappear. Very much to the contrary. I am right now picking up some things that have long been on my TODO wish list and Afilias is doubling down on the commitment to PostgreSQL and Slony. We can and should talk about that stuff next week at PGCon in Ottawa. I will also stay in close contact with the remaining core team members, many of whom have become very good friends over the past 15 years. The entire core team, me included, hoped that it wouldn't come to this and that I could have returned to active duty earlier. Things in my little sub universe didn't change as fast as we all hoped and we all think it is best now that I focus on getting back to speed and do some serious hacking. We hope for many more good things from you yet! I hope to see many of you in Ottawa. You'll certainly see me! Best wishes and many thanks for your good work over so many years. andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Retiring from the PostgreSQL core team
On Thu, 2010-05-13 at 00:24 -0400, Jan Wieck wrote: Over the past years I have not been able to dedicate as much time to PostgreSQL as everyone would have liked. The main reason for that was that I was swamped with other work and private matters and simply didn't have time. I did follow the mailing lists but did not participate much. Looking at my publicly visible involvement over the last two years or so, there is little that would justify me being on the core team today. I was not involved in the release process, in patch reviewing, organizing and have contributed little. My feeling, which I'm sure would be supported by a great many people, is that you have made huge contributions to PostgreSQL, all very much appreciated. Yes, that may not have been visible to all because much of that was via the Slony project, though I know of your quieter contributions to other major projects. Slony alone has been the backbone of PostgreSQL across many releases and will continue to be important in the future also. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] weird hang while running in HS mode
On Thu, 2010-05-13 at 10:49 +0900, Fujii Masao wrote: On Thu, May 13, 2010 at 3:50 AM, Robert Haas robertmh...@gmail.com wrote: rhaas=# rollback; ROLLBACK So at this point, one would think that there are no locks hanging around anywhere. Back to the standby: rhaas=# select * from pgbench_accounts; really long hang I think that this problem happens because the WAL record of ROLLBACK is not flushed to the disk immediately (i.e., until another transaction flushes the WAL records in wal_buffers) when we execute the ROLLBACK command. The walsender sends only the WAL records in the disk, so the standby server would not be able to replay the ROLLBACK and not release the lock. Rollbacks are always flushed to disk, so this explanation doesn't work. Even if it were it would take no longer than ~1 sec if everything were working correctly on the test system. The weird hang is a lock wait and is perfectly normal in database systems. Robert says he hasn't checked whether it is reproduceable, so there is no evidence to show there is anything other than pilot error, at this point. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] weird hang while running in HS mode
On Thu, May 13, 2010 at 6:47 PM, Simon Riggs si...@2ndquadrant.com wrote: Rollbacks are always flushed to disk, so this explanation doesn't work. Even if it were it would take no longer than ~1 sec if everything were working correctly on the test system. Yeah, rollbacks are always flushed sooner or later, but not *immediately*, since RecordTransactionAbort() calls only XLogInsert() but not XLogFlush(). Until XLogFlush() is executed by another process, the WAL record of rollback would stay in wal_buffers. On the other hand, RecordTransactionCommit() calls XLogFlush(), so commits are always flushed to the disk immediately. The weird hang is a lock wait and is perfectly normal in database systems. Robert says he hasn't checked whether it is reproduceable, so there is no evidence to show there is anything other than pilot error, at this point. I was able to reproduce such a hang by not executing another transaction after rollback. In this case, walsender cannot replicate the rollback since it's not in the disk. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_upgrade code questions
On Thu, May 13, 2010 at 8:22 AM, Devrim GÜNDÜZ dev...@gunduz.org wrote: On Thu, 2010-05-13 at 15:13 +0900, Takahiro Itagaki wrote: 4. EDB_NATIVE_LANG Of course it is commented out with #ifdef, but do we have codes for EDB in core? I was about to raise similar thing, for the documentation: http://developer.postgresql.org/pgdocs/postgres/pgupgrade.html This includes some references to EDB AS, which should be removed from PostgreSQL official documentation, IMHO. +1 on getting rid of those references. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Add PGFILEDESC description to Makefiles for all /contrib
On Thu, May 13, 2010 at 1:47 AM, Bruce Momjian br...@momjian.us wrote: bruce wrote: Tom Lane wrote: Bruce Momjian br...@momjian.us writes: All other binaries had such a designation, and all /contrib binaries were missing them. I assume I was doing cleanup. You want the icon removed from the backend makefile? Yes. I'm prepared to believe that not having the icons set on the contrib executables was an oversight. I'm much less prepared to assume that not marking the postgres executable was an oversight. Again, unless you *know* that this change is needed and appropriate, now is not the time to be making it, and especially not without discussion. OK, done with attached patch, and I added a comment about why it is not labeled. I did some research on PGFILEDESC and it does what I thought it does --- in embeds the 'ico' file into the executable in /pg/tools/msvc/Project.pm, and the image looks like the attached JPEG. The image is of a blue elephant head. So, currently, every binary uses that icon, except for the postmaster. Is that what we want? You could make the argument that a daemon, like the postmaster, shouldn't have one, which I think is Tom's point. It's pretty normal that daemons don't have icons. One could argue the same for binaries that are normally never executed by the user, just internally - but I don't think we have any of those (we're talking EXE not DLL). There is, AFAIK, no rule (hard or of-thumb) for icons for GUI programs only not commandline ones. Commandline tools usually have it. The argument to be made there is that when you go into the bin directory you'll get greeted with a long list of identical elephants. It would probably be better if we could have a *different* icon for tools that the user is likely to execute himself - which is pretty much just psql I think. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] List traffic
On Thu, May 13, 2010 at 2:04 AM, Marc G. Fournier scra...@hub.org wrote: On Wed, 12 May 2010, Greg Stark wrote: I'm thinking I'll move -general (and the useless -novice) to another folder. But I'm left wondering what to do with -admin and -performance. They're a random mix of user content and developer content. I'll probably move them along with -general but that means I won't be likely to see any development discussion on them in the future There shouldn't be any dev discussions on them as it is ... that isn't their mandate ... those are/were meant to be end-user lists, not developer ones ... We know from experience that doesn't work. People just end up crossposting, because they're not sure people are on both lists. And then you want to move a discussion, which just means you have to CC in both lists, leading to even more duplication. If there was a clear distinction between end-user and dev it might make sense. That how commercial software companies tend to work - don't let devs talk to end users. That's not how we work. Forcing people to look in different places just throws hurdles in front of those trying to help out. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] weird hang while running in HS mode
On Thu, 2010-05-13 at 19:08 +0900, Fujii Masao wrote: I was able to reproduce such a hang by not executing another transaction after rollback. In this case, walsender cannot replicate the rollback since it's not in the disk. WALWriter is not active? -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] weird hang while running in HS mode
On Thu, May 13, 2010 at 7:22 PM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, 2010-05-13 at 19:08 +0900, Fujii Masao wrote: I was able to reproduce such a hang by not executing another transaction after rollback. In this case, walsender cannot replicate the rollback since it's not in the disk. WALWriter is not active? WALWriter is active, but unfortunately it doesn't flush all of the WAL records in wal_buffers. Please see XLogBackgroundFlush(). Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] weird hang while running in HS mode
On Thu, 2010-05-13 at 19:32 +0900, Fujii Masao wrote: On Thu, May 13, 2010 at 7:22 PM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, 2010-05-13 at 19:08 +0900, Fujii Masao wrote: I was able to reproduce such a hang by not executing another transaction after rollback. In this case, walsender cannot replicate the rollback since it's not in the disk. WALWriter is not active? WALWriter is active, but unfortunately it doesn't flush all of the WAL records in wal_buffers. Please see XLogBackgroundFlush(). Yes, I wrote it. It flushes after at most 3 cycles, stated in comments. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] weird hang while running in HS mode
On Thu, May 13, 2010 at 8:05 PM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, 2010-05-13 at 19:32 +0900, Fujii Masao wrote: On Thu, May 13, 2010 at 7:22 PM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, 2010-05-13 at 19:08 +0900, Fujii Masao wrote: I was able to reproduce such a hang by not executing another transaction after rollback. In this case, walsender cannot replicate the rollback since it's not in the disk. WALWriter is not active? WALWriter is active, but unfortunately it doesn't flush all of the WAL records in wal_buffers. Please see XLogBackgroundFlush(). Yes, I wrote it. It flushes after at most 3 cycles, stated in comments. Yeah, what is worse is that RecordTransactionAbort() doesn't update the XLogCtl-asyncCommitLSN, so rollback might not be flushed even after at 3 cycles. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] weird hang while running in HS mode
On Thu, 2010-05-13 at 20:13 +0900, Fujii Masao wrote: On Thu, May 13, 2010 at 8:05 PM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, 2010-05-13 at 19:32 +0900, Fujii Masao wrote: On Thu, May 13, 2010 at 7:22 PM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, 2010-05-13 at 19:08 +0900, Fujii Masao wrote: I was able to reproduce such a hang by not executing another transaction after rollback. In this case, walsender cannot replicate the rollback since it's not in the disk. WALWriter is not active? WALWriter is active, but unfortunately it doesn't flush all of the WAL records in wal_buffers. Please see XLogBackgroundFlush(). Yes, I wrote it. It flushes after at most 3 cycles, stated in comments. Yeah, what is worse is that RecordTransactionAbort() doesn't update the XLogCtl-asyncCommitLSN, so rollback might not be flushed even after at 3 cycles. What you mean then is that there is a bug, not that it should work this way. Will look. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] weird hang while running in HS mode
On Thu, 2010-05-13 at 20:13 +0900, Fujii Masao wrote: On Thu, May 13, 2010 at 8:05 PM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, 2010-05-13 at 19:32 +0900, Fujii Masao wrote: On Thu, May 13, 2010 at 7:22 PM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, 2010-05-13 at 19:08 +0900, Fujii Masao wrote: I was able to reproduce such a hang by not executing another transaction after rollback. In this case, walsender cannot replicate the rollback since it's not in the disk. WALWriter is not active? WALWriter is active, but unfortunately it doesn't flush all of the WAL records in wal_buffers. Please see XLogBackgroundFlush(). Yes, I wrote it. It flushes after at most 3 cycles, stated in comments. Yeah, what is worse is that RecordTransactionAbort() doesn't update the XLogCtl-asyncCommitLSN, so rollback might not be flushed even after at 3 cycles. Well spotted. Fix applied, thanks both. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Tags missing from GIT mirror?
On tor, 2010-05-13 at 04:41 -0400, Andrew Dunstan wrote: Right, and omitting tags was in fact one of the features of fromcvs that made us use it, because any tool that tries to convert tags will explode on our CVS tree, for reasons explained in the above paragraph. We have also discussed this in more detail about three times before. Well, yes, but I have been wondering if this has to be an all or nothing deal. How many tags can we not tie to a known tree in git? My suspicion is we can probably identify most of them quite well. If we can that would be nice. http://archives.postgresql.org/pgsql-hackers/2008-04/msg00036.php -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Tags missing from GIT mirror?
Peter Eisentraut wrote: On tor, 2010-05-13 at 04:41 -0400, Andrew Dunstan wrote: Right, and omitting tags was in fact one of the features of fromcvs that made us use it, because any tool that tries to convert tags will explode on our CVS tree, for reasons explained in the above paragraph. We have also discussed this in more detail about three times before. Well, yes, but I have been wondering if this has to be an all or nothing deal. How many tags can we not tie to a known tree in git? My suspicion is we can probably identify most of them quite well. If we can that would be nice. http://archives.postgresql.org/pgsql-hackers/2008-04/msg00036.php Quite so. All the tags apparently causing problems are of no more than historical interest to us. But more recent tags, especially for currently maintained branches, are of interest. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] List traffic
On Thu, 13 May 2010, Magnus Hagander wrote: On Thu, May 13, 2010 at 2:04 AM, Marc G. Fournier scra...@hub.org wrote: On Wed, 12 May 2010, Greg Stark wrote: I'm thinking I'll move -general (and the useless -novice) to another folder. But I'm left wondering what to do with -admin and -performance. They're a random mix of user content and developer content. I'll probably move them along with -general but that means I won't be likely to see any development discussion on them in the future There shouldn't be any dev discussions on them as it is ... that isn't their mandate ... those are/were meant to be end-user lists, not developer ones ... We know from experience that doesn't work. People just end up crossposting, because they're not sure people are on both lists. And then you want to move a discussion, which just means you have to CC in both lists, leading to even more duplication. If there was a clear distinction between end-user and dev it might make sense. That how commercial software companies tend to work - don't let devs talk to end users. That's not how we work. Forcing people to look in different places just throws hurdles in front of those trying to help out. What *are* you talking about? This doesn't seem to have anything related to what I said :) All I was saying was that -performance and -admin are not development discusion lists, not that developers aren't subscribed / talking on them ... that doesn't make them any less end-user lists ... Marc G. FournierHub.Org Hosting Solutions S.A. scra...@hub.org http://www.hub.org Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] List traffic
On Thu, May 13, 2010 at 3:27 PM, Marc G. Fournier scra...@hub.org wrote: On Thu, 13 May 2010, Magnus Hagander wrote: On Thu, May 13, 2010 at 2:04 AM, Marc G. Fournier scra...@hub.org wrote: On Wed, 12 May 2010, Greg Stark wrote: I'm thinking I'll move -general (and the useless -novice) to another folder. But I'm left wondering what to do with -admin and -performance. They're a random mix of user content and developer content. I'll probably move them along with -general but that means I won't be likely to see any development discussion on them in the future There shouldn't be any dev discussions on them as it is ... that isn't their mandate ... those are/were meant to be end-user lists, not developer ones ... We know from experience that doesn't work. People just end up crossposting, because they're not sure people are on both lists. And then you want to move a discussion, which just means you have to CC in both lists, leading to even more duplication. If there was a clear distinction between end-user and dev it might make sense. That how commercial software companies tend to work - don't let devs talk to end users. That's not how we work. Forcing people to look in different places just throws hurdles in front of those trying to help out. What *are* you talking about? This doesn't seem to have anything related to what I said :) All I was saying was that -performance and -admin are not development discusion lists, not that developers aren't subscribed / talking on them ... that doesn't make them any less end-user lists ... Yes, and I'm saying there is no real difference between end-user, development, admin and performance. The amount of crossover is so large the distinction rapidly becomes pointless. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] make install fails due to /bin/mkdir: missing operand
On mån, 2010-05-10 at 20:07 +0900, Kenichiro Tanaka wrote: Reproduce case: #prefix and with-pgport are not important ./configure --enable-nls='UFT_JP' --prefix=/home/p900/posgrehome --with-pgport=1900 make make install make install log--- : : make[3]: Leaving directory `/home/p900/postgresql-9.0beta1/src/timezone' /bin/mkdir: missing operand Try `/bin/mkdir --help' for more information. make[2]: *** [installdirs-po] Error 1 make[2]: Leaving directory `/home/p900/postgresql-9.0beta1/src/backend' make[1]: *** [install] Error 2 make[1]: Leaving directory `/home/p900/postgresql-9.0beta1/src' make: *** [install] Error 2 -- Fixed. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] make install fails due to /bin/mkdir: missing operand
On mån, 2010-05-10 at 11:40 -0400, Alvaro Herrera wrote: Excerpts from Kenichiro Tanaka's message of lun may 10 07:07:27 -0400 2010: Reproduce case: #prefix and with-pgport are not important ./configure --enable-nls='UFT_JP' --prefix=/home/p900/posgrehome --with-pgport=1900 make make install I think this is pilot error, in the sense that it doesn't fail if you don't pass an invalid language name. Maybe the bug is that we allow --enable-nls to pass down junk down to the install Makefile, instead of erroring out right there. The language name in his example is obviously wrong, but in general this case if valid. Even if you use something correct like --enable-nls='de es fr', but one subdirectory doesn't have any of those, the mkdir call would fail. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_upgrade code questions
Magnus Hagander wrote: On Thu, May 13, 2010 at 8:22 AM, Devrim G?ND?Z dev...@gunduz.org wrote: On Thu, 2010-05-13 at 15:13 +0900, Takahiro Itagaki wrote: 4. EDB_NATIVE_LANG Of course it is commented out with #ifdef, but do we have codes for EDB in core? I was about to raise similar thing, for the documentation: http://developer.postgresql.org/pgdocs/postgres/pgupgrade.html This includes some references to EDB AS, which should be removed from PostgreSQL official documentation, IMHO. +1 on getting rid of those references. Agreed. When it was on pgFoundry, I had to mention that because it was unclear who would be using it, but in /contrib we know this is for community Postgres. EnterpriseDB did contribute the code so I would like to keep the code working for EnterpriseDB Advanced Server if that is easy. I have added SGML comments to comment out the text that mentions EDB Advanced Server. Is that enough? Should I remove the text from the SGML? Should I move it to the bottom of the SGML? Should I remove the EnterpriseDB Advanced Server checks from the C code too? I don't remember having to deal with anything like this before, so I am unclear how to proceed. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_upgrade code questions
On Thu, May 13, 2010 at 5:06 PM, Bruce Momjian br...@momjian.us wrote: Magnus Hagander wrote: On Thu, May 13, 2010 at 8:22 AM, Devrim G?ND?Z dev...@gunduz.org wrote: On Thu, 2010-05-13 at 15:13 +0900, Takahiro Itagaki wrote: 4. EDB_NATIVE_LANG Of course it is commented out with #ifdef, but do we have codes for EDB in core? I was about to raise similar thing, for the documentation: http://developer.postgresql.org/pgdocs/postgres/pgupgrade.html This includes some references to EDB AS, which should be removed from PostgreSQL official documentation, IMHO. +1 on getting rid of those references. Agreed. When it was on pgFoundry, I had to mention that because it was unclear who would be using it, but in /contrib we know this is for community Postgres. EnterpriseDB did contribute the code so I would like to keep the code working for EnterpriseDB Advanced Server if that is easy. I have added SGML comments to comment out the text that mentions EDB Advanced Server. Is that enough? Should I remove the text from the SGML? Should I move it to the bottom of the SGML? Should I remove the EnterpriseDB Advanced Server checks from the C code too? I don't remember having to deal with anything like this before, so I am unclear how to proceed. I say remove it. On all accounts. There's a fork of postgres for EDB AS, shouldn't there be a fork of pg_upgrade the same way, if it requires special code? The code in community postgresql certainly shouldn't have any EDB AS code in it. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] wal_mode in postgresql.conf
Why is 'wal_level' listed next to 'fsync' in postgresql.conf? #wal_level = 'hot_standby' # minimal, archive, or hot_standby #fsync = on # turns forced synchronization on or off #synchronous_commit = on# immediate fsync at commit #wal_sync_method = fsync# the default is the first option Seems it should be in the archiving section. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] wal_level in postgresql.conf
Bruce Momjian wrote: Why is 'wal_level' listed next to 'fsync' in postgresql.conf? #wal_level = 'hot_standby' # minimal, archive, or hot_standby #fsync = on # turns forced synchronization on or off #synchronous_commit = on# immediate fsync at commit #wal_sync_method = fsync# the default is the first option Seems it should be in the archiving section. Sorry, subject was wrong. I am asking about 'wal_level'. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_upgrade code questions
Magnus Hagander mag...@hagander.net writes: On Thu, May 13, 2010 at 5:06 PM, Bruce Momjian br...@momjian.us wrote: I have added SGML comments to comment out the text that mentions EDB Advanced Server. Is that enough? Should I remove the text from the SGML? Should I move it to the bottom of the SGML? Should I remove the EnterpriseDB Advanced Server checks from the C code too? I don't remember having to deal with anything like this before, so I am unclear how to proceed. I say remove it. On all accounts. There's a fork of postgres for EDB AS, shouldn't there be a fork of pg_upgrade the same way, if it requires special code? The code in community postgresql certainly shouldn't have any EDB AS code in it. Indeed. Given the (presumably large) delta between EDB's code and ours, having to have some delta in pg_upgrade isn't going to make much difference for them. I think the community code and docs should completely omit any mention of that. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] List traffic
Le 11/05/2010 19:24, Alvaro Herrera a écrit : Excerpts from Marc G. Fournier's message of mar may 11 09:58:34 -0400 2010: If list traffic, especially on -hackers, is getting so large, should we look at maybe splitting it? I could easily enough split things such that I duplicate the subscriber list, so nobody would have to subscribe, but it would make it easier for ppl to filter their incoming ... ? Maybe we could create a separate list where people would send patches, and keep patchless discussion on -hackers? Just a thought ;-) Here's a simple description of how i use and see the -hackers list. I'm what you could call a silent reader, like many other subscribers i don't participate to the discussions but i'm happy to be able to follow them. I'm not an end-user and i'm not a developper. Just a guy that wants to follow the making-of this project. Sure the traffic is huge and sometimes i have thousands of unread messages. But somewhat i managed to follow the threads i'm interested in and leave asides others... If this list is split in two smaller ones, then i guess i'll follow both and it won't help me in any way. I guess it would even make things more difficult to understand. This is my modest experience. Clearly things can be improved, but speaking for myself i don't think that splitting the list is a good idea. -- damien clochard http://www.dalibo.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] multibyte charater set in levenshtein function
On Thu, May 13, 2010 at 6:03 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Well, since it's only used in one place, why are you defining a macro at all? In order to structure code better. My question was about another. Is memcmp function good choice to compare very short sequences of bytes (from 1 to 4 bytes)?
[HACKERS] wal_level and continuous archiving documentation
I was reading through http://www.postgresql.org/docs/9.0/static/continuous-archiving.html and noticed that wal_level isn't mentioned where I'd expect it to be. Specifically, there's a paragraph that starts, To enable WAL archiving, set the archive_mode configuration parameter to on, and specify the shell command to use in the archive_command configuration parameter. There follows a long discussion of archive_command, but no further discussion of other configuration settings for several paragraphs, suggesting that those two configuration changes are the only ones required to end up with a useful archive. However, further on, it discusses wal_level: When wal_level is minimal some SQL commands are optimized to avoid WAL logging, as described in Section 14.4.7. If archiving or streaming replication were turned on during execution of one of these statements, WAL would not contain enough information for archive recovery. ISTM wal_archive should make an appearance where the docs bring up archive_mode and archive_command, to say wal_level must be set to 'archive' or 'hot_standby', so all required configuration changes are mentioned close together. -- Joshua Tolley / eggyknap End Point Corporation http://www.endpoint.com signature.asc Description: Digital signature
Re: [HACKERS] Retiring from the PostgreSQL core team
On Thu, May 13, 2010 at 12:24:47AM -0400, Jan Wieck wrote: To whom it may concern, this is to inform the PostgreSQL community of my retirement from my PostgreSQL core team position. Over the past years I have not been able to dedicate as much time to PostgreSQL as everyone would have liked. The main reason for that was that I was swamped with other work and private matters and simply didn't have time. I did follow the mailing lists but did not participate much. Looking at my publicly visible involvement over the last two years or so, there is little that would justify me being on the core team today. I was not involved in the release process, in patch reviewing, organizing and have contributed little. However, in contrast to other previous core team members, I do not plan to disappear. Very much to the contrary. I am right now picking up some things that have long been on my TODO wish list and Afilias is doubling down on the commitment to PostgreSQL and Slony. We can and should talk about that stuff next week at PGCon in Ottawa. I will also stay in close contact with the remaining core team members, many of whom have become very good friends over the past 15 years. The entire core team, me included, hoped that it wouldn't come to this and that I could have returned to active duty earlier. Things in my little sub universe didn't change as fast as we all hoped and we all think it is best now that I focus on getting back to speed and do some serious hacking. I hope to see many of you in Ottawa. You can run, but you can't hide ;) Thanks for your deep and broad contributions so far, and I'm sure I speak for many when I say we're looking forward to upcoming ones. Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] List traffic
On Wed, May 12, 2010 at 5:24 PM, Robert Haas robertmh...@gmail.com wrote: The difference between discussing a patch and discussing an idea that might lead to a patch is fairly fine. And importantly -- who would be able to subscribe to one and not the other? If you have to subscribe to both to get make any sense of things then there's no point. Fwiw I'm having trouble keeping up these days too. And I'm quite accustomed to very heavy traffic email. I've been throwing all postgres related lists into one folder and skimmed through it looking for important threads. However this has now broken down. There are about 45 new threads every day. I've been travelling for a bit and am now 1,500 threads behind... If we can find a way to split the content sensibly so I could stop reading some of it that would be helpful. But cutting splitting it along subject matter where both sets of subject matter need to be seen by the same people doesn't really help. I'm thinking I'll move -general (and the useless -novice) to another folder. But I'm left wondering what to do with -admin and -performance. They're a random mix of user content and developer content. I'll probably move them along with -general but that means I won't be likely to see any development discussion on them in the future. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] multibyte charater set in levenshtein function
On Wed, May 12, 2010 at 11:04 PM, Alvaro Herrera alvhe...@alvh.no-ip.orgwrote: On a quick look, I didn't like the way you separated the pg_database_encoding_max_length() 1 cases. There seem to be too much common code. Can that be refactored a bit better? I did a little refactoring in order to avoid some similar code. I'm not quite sure about my CHAR_CMP macro. Is it a good idea? fuzzystrmatch-0.2.diff.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Query execution plan from 8.3 - 8.4
Thanks for the advice, will do. Regards, Brendan Hill Chief Information Officer Jims Group Pty Ltd 48 Edinburgh Rd Mooroolbark VIC 3138 www.jims.net For all Jims IT enquiries: infot...@jims.net For emergencies: 1300 130 490 (intl +61 4 3456 5776) -Original Message- From: Kevin Grittner [mailto:kevin.gritt...@wicourts.gov] Sent: Thursday, 13 May 2010 12:29 AM To: Brendan Hill; pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Query execution plan from 8.3 - 8.4 Brendan Hill brend...@jims.net wrote: AND Notes.Person_ID IN (SELECT ISNULL(Personnel.Person_ID, Businesses.Main_Person_ID) You might try switching this to an EXISTS test. If you post on this topic again, really it should be on the -perform list, as Stephen mentioned, and review this page for ideas on other information (like hardware and the postgresql.conf file) which might help people better understand the problem: http://wiki.postgresql.org/wiki/SlowQueryQuestions -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] quoting and recovery.conf
Is there a reason we require single quotes around boolean values in recovery.conf? standby_mode = 'off' This does not work: standby_mode = off I knew there were inconsistencies between quoting in postgresql.conf and recovery.conf, but I didn't realize it extended to boolean quoting. I see this at the top of recovery.conf now: # This file consists of lines of the form: # # name = 'value' # # (The quotes around the value are NOT optional, but the = is.) # and this issue existed in 8.4 as well. Seems I just never noticed it, and it not specifically mentioned in the TODO item we already have. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] max_standby_delay considered harmful
On 5/12/10 8:07 PM, Robert Haas wrote: I think that would be a good thing to check (it'll confirm whether this is the same bug), but I'm not convinced we should actually fix it that way. Prior to 8.4, we handled a smart shutdown during recovery at the conclusion of recovery, just prior to entering normal running. I'm wondering if we shouldn't revert to that behavior in both 8.4 and HEAD. This would be OK as long as we document it well. We patched the shutdown the way we did specifically because Fujii thought it would be an easy fix; if it's complicated, we should revert it and document the issue for DBAs. Oh, and to confirm: the same issue exists, and has always existed, with Warm Standby. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_upgrade code questions
Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: On Thu, May 13, 2010 at 5:06 PM, Bruce Momjian br...@momjian.us wrote: I have added SGML comments to comment out the text that mentions EDB Advanced Server. ?Is that enough? ?Should I remove the text from the SGML? ?Should I move it to the bottom of the SGML? ?Should I remove the EnterpriseDB Advanced Server checks from the C code too? ?I don't remember having to deal with anything like this before, so I am unclear how to proceed. I say remove it. On all accounts. There's a fork of postgres for EDB AS, shouldn't there be a fork of pg_upgrade the same way, if it requires special code? The code in community postgresql certainly shouldn't have any EDB AS code in it. Indeed. Given the (presumably large) delta between EDB's code and ours, having to have some delta in pg_upgrade isn't going to make much difference for them. I think the community code and docs should completely omit any mention of that. I am trying to think of this as a non-EnterpriseDB employee. If suppose Greenplum had given us a utility and they wanted it to work with their version of the database, what accommodation would we make for them? I agree on the documentation, but would we allow #ifdefs that were only used by them if there were only a few of them? Could we treat it as an operating system that none of us use? I don't think Greenplum would require us to keep support for their database, but they would prefer it, and it might encourage more contributions from them. Maybe we would just tell them to keep their own patches, but I figured I would ask specifically so we have a policy for next time. I guess another question is whether we would accept a patch that was useful only for a Greenplum build? And does removing such code use the same criteria? I know pgAdmin supports Greenplum, but that is an external tool so it makes more sense there. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_upgrade code questions
On Thu, 2010-05-13 at 17:19 +0200, Magnus Hagander wrote: I say remove it. On all accounts. There's a fork of postgres for EDB AS, shouldn't there be a fork of pg_upgrade the same way, if it requires special code? The code in community postgresql certainly shouldn't have any EDB AS code in it. If the code would be useful for other projects then keep it. If it is only for a closed source product, dump it. Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_upgrade code questions
Bruce Momjian wrote: Indeed. Given the (presumably large) delta between EDB's code and ours, having to have some delta in pg_upgrade isn't going to make much difference for them. I think the community code and docs should completely omit any mention of that. I am trying to think of this as a non-EnterpriseDB employee. If suppose Greenplum had given us a utility and they wanted it to work with their version of the database, what accommodation would we make for them? I agree on the documentation, but would we allow #ifdefs that were only used by them if there were only a few of them? Could we treat it as an operating system that none of us use? I don't think Greenplum would require us to keep support for their database, but they would prefer it, and it might encourage more contributions from them. Maybe we would just tell them to keep their own patches, but I figured I would ask specifically so we have a policy for next time. I guess another question is whether we would accept a patch that was useful only for a Greenplum build? And does removing such code use the same criteria? I know pgAdmin supports Greenplum, but that is an external tool so it makes more sense there. What if several vendors want the same thing? The code will quickly become spaghetti. AFAIK the Linux kernel expects distros to keep their patchsets separately, and I rather think we should too. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] List traffic
My thought had been a split along the lines of major components of the server ... for instance, a totally seperate list for HS related issues, so that, if nothing else, those 'lurkers' that are only interested in developments on that front could be there but not on the main stream -hackers ... almost like seperate working groups ... Twas just a thought ... On Wed, 12 May 2010, Greg Stark wrote: On Wed, May 12, 2010 at 5:24 PM, Robert Haas robertmh...@gmail.com wrote: The difference between discussing a patch and discussing an idea that might lead to a patch is fairly fine. And importantly -- who would be able to subscribe to one and not the other? If you have to subscribe to both to get make any sense of things then there's no point. Fwiw I'm having trouble keeping up these days too. And I'm quite accustomed to very heavy traffic email. I've been throwing all postgres related lists into one folder and skimmed through it looking for important threads. However this has now broken down. There are about 45 new threads every day. I've been travelling for a bit and am now 1,500 threads behind... If we can find a way to split the content sensibly so I could stop reading some of it that would be helpful. But cutting splitting it along subject matter where both sets of subject matter need to be seen by the same people doesn't really help. I'm thinking I'll move -general (and the useless -novice) to another folder. But I'm left wondering what to do with -admin and -performance. They're a random mix of user content and developer content. I'll probably move them along with -general but that means I won't be likely to see any development discussion on them in the future. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers Marc G. FournierHub.Org Hosting Solutions S.A. scra...@hub.org http://www.hub.org Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_upgrade code questions
On 5/13/10 10:14 AM, Bruce Momjian wrote: I am trying to think of this as a non-EnterpriseDB employee. If suppose Greenplum had given us a utility and they wanted it to work with their version of the database, what accommodation would we make for them? I agree on the documentation, but would we allow #ifdefs that were only used by them if there were only a few of them? Could we treat it as an operating system that none of us use? I don't think Greenplum would require us to keep support for their database, but they would prefer it, and it might encourage more contributions from them. Maybe we would just tell them to keep their own patches, but I figured I would ask specifically so we have a policy for next time. My $0.021746: If something is going to be included in /contrib, it should only include code which relates to standard PostgreSQL. The independant pg_migrator project can be a PG/EDBAS tool; the contrib module needs to be vanilla-postgres only. If the donor of the code wants to keep the specific fork support, then it should remain an independant project. I'm not just referring to EDB here, or even just proprietary forks; even open source forks (like PostgresXC or pgCluster) shouldn't have specific code in /contrib. Within the limits of reasonableness, of course. My argument isn't based on purity, but is rather based on: (a) avoiding confusing the users, and (b) avoiding bulking code with lots of ifdefs if we can avoid it, and (c) fork release cycles are often different from pgsql-core, and EDB's certainly is. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] List traffic
Greg Stark wrote: On Wed, May 12, 2010 at 5:24 PM, Robert Haas robertmh...@gmail.com wrote: The difference between discussing a patch and discussing an idea that might lead to a patch is fairly fine. And importantly -- who would be able to subscribe to one and not the other? If you have to subscribe to both to get make any sense of things then there's no point. Fwiw I'm having trouble keeping up these days too. And I'm quite accustomed to very heavy traffic email. I've been throwing all postgres related lists into one folder and skimmed through it looking for important threads. However this has now broken down. There are about 45 new threads every day. I've been travelling for a bit and am now 1,500 threads behind... I've only been actively reading the pg lists for a few months now, after several previous attempts that failed mainly because the way I set it up did not work nice, mainly because of the volume. I tried digests, didn't like it (how to reply?), also didn't like that the pg mails that were so many completely swamped the 'main' email I use. Now I made a new gmail account, subscribed to all lists with some volume and let it all message per message come into the inbox. Together with thunderbird/imap this works quite nicely. With filters it's possible to tag interesting messages (like does the To: contain my email? - tag it so it becomes green). Now I only need to view unread mails, (by thread or date), read some messages and then ctrl-shift-c - all read. My $0.02 - I like the whole 'don't sort, search' (or how did they call it?) just let the inbox fill up, google is fast enough. What would be really interesting is to have some extra 'tags/headers' added to the emails (document classification with e.g. self organizing map/kohonen), so my local filters could make labels based on that, instead of perhaps badly spelled keywords in subjects or message body. regards, Yeb Havinga -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] List traffic
On Thu, May 13, 2010 at 8:05 PM, Marc G. Fournier scra...@hub.org wrote: My thought had been a split along the lines of major components of the server ... for instance, a totally seperate list for HS related issues, so that, if nothing else, those 'lurkers' that are only interested in developments on that front could be there but not on the main stream -hackers ... almost like seperate working groups ... We tried that with pgsql-hackers-win32 and iirc also pgsql-hackers-pitr, and it was a big failure... -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] List traffic
On Thu, 13 May 2010, Magnus Hagander wrote: On Thu, May 13, 2010 at 8:05 PM, Marc G. Fournier scra...@hub.org wrote: My thought had been a split along the lines of major components of the server ... for instance, a totally seperate list for HS related issues, so that, if nothing else, those 'lurkers' that are only interested in developments on that front could be there but not on the main stream -hackers ... almost like seperate working groups ... We tried that with pgsql-hackers-win32 and iirc also pgsql-hackers-pitr, and it was a big failure... But, we are doing that now with pgsql-cluster-hackers and it looks to be working quite well from what I can see ... guess it depends on if ppl want it to fail in the first place or not *shrug* It also depends if a clear line can be drawn and adhered to ... Marc G. FournierHub.Org Hosting Solutions S.A. scra...@hub.org http://www.hub.org Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] List traffic
Excerpts from Yeb Havinga's message of jue may 13 15:06:53 -0400 2010: Now I made a new gmail account, subscribed to all lists with some volume and let it all message per message come into the inbox. Together with thunderbird/imap this works quite nicely. With filters it's possible to tag interesting messages (like does the To: contain my email? - tag it so it becomes green). Now I only need to view unread mails, (by thread or date), read some messages and then ctrl-shift-c - all read. My $0.02 - I like the whole 'don't sort, search' (or how did they call it?) just let the inbox fill up, google is fast enough. What would be really interesting is to have some extra 'tags/headers' added to the emails (document classification with e.g. self organizing map/kohonen), so my local filters could make labels based on that, instead of perhaps badly spelled keywords in subjects or message body. Yeah, this approach is interesting. A few days ago I started using Sup ( http://sup.rubyforge.org/ ) to manage my email, and after a rather lengthy warm-up process, I find it a lot more comfortable than Mutt (or anything else I've tried earlier, for that matter). I particularly like the multiple buffer approach, avoiding the need for switching between several Mutt instances, one for each mailbox. So it's almost like gmail: you get fast search, labelling, and a thread-based approach rather than message-based. As with gmail, you can mute threads that are not interesting to you, so that if any email arrives later to that thread, you will not see it unless you actively look for it. An old (unmuted) thread receiving a new message jumps back at the top of the list; and you can dismiss stuff as archived with a single keystroke, and it will stop polluting your immediate environment, but you can search for it. And it's pretty *fast* with searches (uses Xapian as backend). It's clearly a programmer's MUA -- if you want automatic labelling, you better be prepared to write some Ruby code. I have already written some simple rules that get me the trivial labels for pgsql lists and such; I have also ported the Perl moderation script I used, and the main advantage is that it's a tad faster (though I spent a lot more time writing that function than I'll ever save actually doing moderation -- but hey, I managed to learn some Ruby in the process). It is rather immature though, so I can't recommend it unless you're prepared to deal with that. -- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] List traffic
On Thu, 13 May 2010, Alvaro Herrera wrote: Excerpts from Yeb Havinga's message of jue may 13 15:06:53 -0400 2010: My $0.02 - I like the whole 'don't sort, search' (or how did they call it?) just let the inbox fill up, google is fast enough. What would be really interesting is to have some extra 'tags/headers' added to the emails (document classification with e.g. self organizing map/kohonen), so my local filters could make labels based on that, instead of perhaps badly spelled keywords in subjects or message body. I missed this when I read it the first time .. all list email does have an X-Mailing-List header added so that you can label based on list itself ... is that what you mean, or are you thinking of something else entirely? Marc G. FournierHub.Org Hosting Solutions S.A. scra...@hub.org http://www.hub.org Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] List traffic
Marc G. Fournier scra...@hub.org writes: On Thu, 13 May 2010, Magnus Hagander wrote: We tried that with pgsql-hackers-win32 and iirc also pgsql-hackers-pitr, and it was a big failure... But, we are doing that now with pgsql-cluster-hackers and it looks to be working quite well from what I can see ... Is it? If they want someplace where the majority of hackers won't see the discussion, maybe, but I am not sure that's not counterproductive. Ideas developed by a small group may or may not survive exposure when they reach this list. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Row-level Locks SERIALIZABLE transactions, postgres vs. Oracle
Hi After the recent discussion about the impossibility of efficiently implementing FK-like constraint triggers in PL/PGSQL that work correctly under SERIALIZABLe transactions, I've compared our behavior to that of Oracle. As it turns out, a slight difference in Oracle's behavior makes those FK constraint triggers which on postgres are only correct in READ COMMITTED mode fully correct in SERIALIZABLE mode also. 1. Summary of the previous discussion The built-in FK constraint trigger looks for rows visible under either the transaction's snapshot *or* a freshly taken MVCC snapshot when checking for child-table rows that'd prevent an UPDATE or DELETE of a row in the parent table. This is necessary even though the parent row is SHARE-locked on INSERTs/UPDATEs to the child table, and would also be necessary if it was UPDATE-locked. The following series of commands illustrates why C1: BEGIN C1: SELECT * FROM t WHERE id = 1 FOR UPDATE C2: BEGIN C2: SET TRANSACTION ISOLATION LEVEL SERIALIZABLE C2: SELECT * FROM t -- Take snapshot before C1 commits C1: COMMIT C2: DELETE FROM t WHERE id = 1 C2: COMMIT Since C1 commits before C2 does DELETE, C2 is entirely unaffected by C1's UPDATE-lock. C2 has no way of detecting possible dependent rows that C1 might have inserted, since C1 is invisible to C2. Note that if you swap the SELECT .. FOR UPDATE and the DELETE commands, the SELECT .. FOR UPDATE will cause a serialization error! 2. The behavior or Oracle Oracle treats a FOR UPDATE lock much like an actual UPDATE when checking for serialization conflicts. This causes the DELETE in the example above to raise a serialization error, and hence prevents the failure case for FK constraint triggers even without a recheck under a current snapshot. One can think of a FOR UPDATE lock as a kind of read barrier on Oracle - it prevents other transactions from messing with the row that don't consider the locking transaction to be visible. 3. Conclusio While it might seem strange at first for a lock to affect other transactions even after the locking transaction has ended, it actually makes sense when viewed as a kind of write barrier. It is very common for locking primitives to use barrier instructions to ensure that one lock holder sees all changes done by the previous owner. Raising a serialization error in the example above is the transactional equivalent of such a barrier instruction in the case of SERIALIZABLE transactions - since updating the transaction's snapshot is obviously not an option, the remaining alternative is to restart the whole transaction under a current snapshot. This is exactly what raising a serialization error accomplishes. Also, while Oracle's behavior has obvious use-cases (e.g. FK-like constraints), I failed to come up with a case where postgres' current behavior is useful. When would you want a (SERIALIZABLE) transaction to wait for a lock, but then continue as if the lock had never existed? What is the point of waiting then in the first place? All in all, I believe that SHARE and UPDATE row-level locks should be changed to cause concurrent UPDATEs to fail with a serialization error. I can come up with a patch that does that, but I wanted to get some feedback on the idea before I put the work in. best regards, Florian Pflug -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] List traffic
Alvaro Herrera alvhe...@alvh.no-ip.org writes: Excerpts from Yeb Havinga's message of jue may 13 15:06:53 -0400 2010: Now I made a new gmail account Yeah, this approach is interesting. A few days ago I started using Sup ( http://sup.rubyforge.org/ ) to manage my email Feature wise, I think gnus offers more than the two approaches combined. Speed wise some people use it with some indexing solution, I'm not finding the need yet. And yes, to handle our lists traffic you must have a MUA made for it. That's the reason why I switched, and it's working great here. Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Row-level Locks SERIALIZABLE transactions, postgres vs. Oracle
Florian Pflug f...@phlo.org writes: All in all, I believe that SHARE and UPDATE row-level locks should be changed to cause concurrent UPDATEs to fail with a serialization error. I don't see an argument for doing that for FOR SHARE locks, and it already happens for FOR UPDATE (at least if the row actually gets updated). AFAICS this proposal mainly breaks things, in pursuit of an unnecessary and probably-impossible-anyway goal of making FK locking work with only user-level snapshots. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Row-level Locks SERIALIZABLE transactions, postgres vs. Oracle
Florian Pflug f...@phlo.org wrote: All in all, I believe that SHARE and UPDATE row-level locks should be changed to cause concurrent UPDATEs to fail with a serialization error. I can come up with a patch that does that, but I wanted to get some feedback on the idea before I put the work in. Before you work on that, you might want to wait until you can review the work that I and Dan Ports (a Ph.D. candidate from MIT) have been doing on support for true serializable transactions. You don't need to use FOR SHARE or FOR UPDATE or any explicit locks as long as the concurrent transactions are SERIALIZABLE. We have it working, but have been holding off on discussion or patch submission at Tom's request -- he felt it would distract from the process of getting the release out. Whenever people are ready, I can submit a WIP patch. All issues discuss on this thread Just Work with the patch applied. There's a Wiki page and a public git repository related to this work, for anyone who is interested and not busy with release work. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] HS/SR Assert server crash
I was able to easily crash the standby server today just by starting it and connecting to it via psql. The master was idle. The failure was: LOG: streaming replication successfully connected to primary TRAP: FailedAssertion(!(((xmax) = ((TransactionId) 3))), File: procarray.c, Line: 1211) LOG: server process (PID 12761) was terminated by signal 6: Abort trap LOG: terminating any other active server processes My master postgresql.conf was: wal_level = hot_standby # minimal, archive, or hot_standby archive_mode = on # allows archiving to be done archive_command = 'cp -i %p /u/pg/archive/%f /dev/null ' # command to use to archive a logfile segment max_wal_senders = 1 # max number of walsender processes My slave postgresql.conf was: port = 5433 # (change requires restart) wal_level = hot_standby # minimal, archive, or hot_standby archive_mode = off # allows archiving to be done archive_command = 'cp -i %p /u/pg/archive/%f /dev/null ' # command to use to archive a logfile segment hot_standby = on# allows queries during recovery max_wal_senders = 1 # max number of walsender processes and my slave recovery.conf was: restore_command = 'cp /u/pg/archive/%f %p' # e.g. 'cp /mnt/server/archivedir/%f %p' standby_mode = 'on' primary_conninfo = 'host=localhost port=5432' # e.g. 'host=localhost port=5432' Let me know what additional information I can supply. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] List traffic
On Thu, 13 May 2010, Tom Lane wrote: Marc G. Fournier scra...@hub.org writes: On Thu, 13 May 2010, Magnus Hagander wrote: We tried that with pgsql-hackers-win32 and iirc also pgsql-hackers-pitr, and it was a big failure... But, we are doing that now with pgsql-cluster-hackers and it looks to be working quite well from what I can see ... Is it? If they want someplace where the majority of hackers won't see the discussion, maybe, but I am not sure that's not counterproductive. Ideas developed by a small group may or may not survive exposure when they reach this list. But that, IMHO, is the point of the smaller list ... it allows the group on that list to hash out their ideas, and, hopefully, deal with both arguments and counter arguments so that when presented to the larger group, they would then have a more cohesive arg for their ideas ... Marc G. FournierHub.Org Hosting Solutions S.A. scra...@hub.org http://www.hub.org Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] nvarchar notation accepted?
Hi, i migrate a ms sql server database to postgres and was trying some queries from the application to find if everything works right... when i was looking to those queries i found some that has a notation for nvarchar (ej: campo = N'sometext') i was expecting those to fail but this actually works, is that fine? i know, we can use E'' strings but N'' ones are no where documented, so can i rely on those or i have to change those strings? create table t1_nvarchar(col1 text); insert into t1_nvarchar values (N'texto'); -- Jaime Casanova www.2ndQuadrant.com Soporte y capacitación de PostgreSQL -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] quoting and recovery.conf
On Thu, May 13, 2010 at 1:00 PM, Bruce Momjian br...@momjian.us wrote: Is there a reason we require single quotes around boolean values in recovery.conf? standby_mode = 'off' This does not work: standby_mode = off I knew there were inconsistencies between quoting in postgresql.conf and recovery.conf, but I didn't realize it extended to boolean quoting. I see this at the top of recovery.conf now: # This file consists of lines of the form: # # name = 'value' # # (The quotes around the value are NOT optional, but the = is.) # and this issue existed in 8.4 as well. Seems I just never noticed it, and it not specifically mentioned in the TODO item we already have. I think we should add a TODO to parse recovery.conf with the same code we use to parse postgresql.conf, or possibly merge the two files. This issue was previously alluded to here: http://archives.postgresql.org/pgsql-hackers/2010-04/msg00211.php -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] List traffic
On Thu, 2010-05-13 at 19:13 -0300, Marc G. Fournier wrote: On Thu, 13 May 2010, Tom Lane wrote: Marc G. Fournier scra...@hub.org writes: On Thu, 13 May 2010, Magnus Hagander wrote: We tried that with pgsql-hackers-win32 and iirc also pgsql-hackers-pitr, and it was a big failure... But, we are doing that now with pgsql-cluster-hackers and it looks to be working quite well from what I can see ... Is it? If they want someplace where the majority of hackers won't see the discussion, maybe, but I am not sure that's not counterproductive. Ideas developed by a small group may or may not survive exposure when they reach this list. But that, IMHO, is the point of the smaller list ... it allows the group on that list to hash out their ideas, and, hopefully, deal with both arguments and counter arguments so that when presented to the larger group, they would then have a more cohesive arg for their ideas ... Yes and no. After being on these lists for years, I have kind of been moving toward the less is more. E.g; for main list traffic I can see the need for two maybe three, that's it: hackers general www There is no reason why advocacy can't happen on general. Theoretically www could be on hackers (although I do see the point of a separate list). A good MUA will deal with any overhead you have. I use Evolution and no its not perfect but I have no problem managing the hordes of email I get from this community. Between labels, filters, watch lists and all the other goodies any MUA will give you, I see no reason to have this all broken out anymore. Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] List traffic
Joshua D. Drake j...@commandprompt.com writes: On Thu, 2010-05-13 at 19:13 -0300, Marc G. Fournier wrote: But that, IMHO, is the point of the smaller list ... it allows the group on that list to hash out their ideas, and, hopefully, deal with both arguments and counter arguments so that when presented to the larger group, they would then have a more cohesive arg for their ideas ... Yes and no. After being on these lists for years, I have kind of been moving toward the less is more. E.g; for main list traffic I can see the need for two maybe three, that's it: hackers general www I can see the need for small tightly-focused special lists. www is a good example, and perhaps pgsql-cluster-hackers is too (though I'm less convinced of that than Marc is). I agree that we've done poorly with lists with wider charters, mainly because there is so little clarity about which topics belong where. I'd keep -bugs and -performance, which seem to be reasonably well focused, but I can definitely see collapsing most of the other user lists into -general. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_upgrade code questions
Josh Berkus wrote:
On 5/13/10 10:14 AM, Bruce Momjian wrote:
I am trying to think of this as a non-EnterpriseDB employee. If suppose
Greenplum had given us a utility and they wanted it to work with their
version of the database, what accommodation would we make for them? I
agree on the documentation, but would we allow #ifdefs that were only
used by them if there were only a few of them? Could we treat it as an
operating system that none of us use? I don't think Greenplum would
require us to keep support for their database, but they would prefer it,
and it might encourage more contributions from them. Maybe we would
just tell them to keep their own patches, but I figured I would ask
specifically so we have a policy for next time.
My $0.021746:
If something is going to be included in /contrib, it should only include
code which relates to standard PostgreSQL. The independant pg_migrator
project can be a PG/EDBAS tool; the contrib module needs to be
vanilla-postgres only. If the donor of the code wants to keep the
specific fork support, then it should remain an independant project.
I'm not just referring to EDB here, or even just proprietary forks; even
open source forks (like PostgresXC or pgCluster) shouldn't have specific
code in /contrib. Within the limits of reasonableness, of course.
My argument isn't based on purity, but is rather based on:
(a) avoiding confusing the users, and
(b) avoiding bulking code with lots of ifdefs if we can avoid it, and
(c) fork release cycles are often different from pgsql-core, and EDB's
certainly is.
I was more interested in understanding our policy rather than how to
handle this specific issue. I have removed all mentions of EnterpriseDB
Advanced Server from pg_upgrade with the attached patch. I will keep
the patch for submission back to EnterpriseDB when they want it, or they
can just pull it from CVS.
--
Bruce Momjian br...@momjian.ushttp://momjian.us
EnterpriseDB http://enterprisedb.com
Index: contrib/pg_upgrade/check.c
===
RCS file: /cvsroot/pgsql/contrib/pg_upgrade/check.c,v
retrieving revision 1.3
diff -c -c -r1.3 check.c
*** contrib/pg_upgrade/check.c 13 May 2010 15:58:15 - 1.3
--- contrib/pg_upgrade/check.c 13 May 2010 22:48:06 -
***
*** 149,158
{
prep_status(ctx, Adjusting sequences);
exec_prog(ctx, true,
! SYSTEMQUOTE \%s/%s\ --set ON_ERROR_STOP=on --port %d
-f \%s\ --dbname template1 \%s\ SYSTEMQUOTE,
! ctx-new.bindir, ctx-new.psql_exe, ctx-new.port,
! sequence_script_file_name, ctx-logfile);
unlink(sequence_script_file_name);
pg_free(sequence_script_file_name);
check_ok(ctx);
--- 149,158
{
prep_status(ctx, Adjusting sequences);
exec_prog(ctx, true,
! SYSTEMQUOTE \%s/psql\ --set ON_ERROR_STOP=on --port %d
-f \%s\ --dbname template1 \%s\ SYSTEMQUOTE,
! ctx-new.bindir, ctx-new.port, sequence_script_file_name,
! ctx-logfile);
unlink(sequence_script_file_name);
pg_free(sequence_script_file_name);
check_ok(ctx);
Index: contrib/pg_upgrade/controldata.c
===
RCS file: /cvsroot/pgsql/contrib/pg_upgrade/controldata.c,v
retrieving revision 1.1
diff -c -c -r1.1 controldata.c
*** contrib/pg_upgrade/controldata.c 12 May 2010 02:19:10 - 1.1
--- contrib/pg_upgrade/controldata.c 13 May 2010 22:48:06 -
***
*** 9,18
#include ctype.h
#include stdlib.h
- #ifdef EDB_NATIVE_LANG
- #include access/tuptoaster.h
- #endif
-
/*
* get_control_data()
--- 9,14
***
*** 88,102
got_float8_pass_by_value = true;
}
- #ifdef EDB_NATIVE_LANG
- /* EDB AS 8.3 is an 8.2 code base */
- if (cluster-is_edb_as GET_MAJOR_VERSION(cluster-major_version) = 803)
- {
- cluster-controldata.toast = TOAST_MAX_CHUNK_SIZE;
- got_toast = true;
- }
- #endif
-
/* we have the result of cmd in output. so parse it line by line now */
while (fgets(bufin, sizeof(bufin), output))
{
--- 84,89
***
*** 140,148
p++;/* removing ':' char */
cluster-controldata.cat_ver = (uint32) atol(p);
}
! else if ((p = strstr(bufin, First log file ID after reset:)) != NULL ||
! (cluster-is_edb_as GET_MAJOR_VERSION(cluster-major_version) = 803
! (p = strstr(bufin, Current log file ID:)) != NULL))
{
p = strchr(p, ':');
--- 127,133
p++;/* removing ':' char */
cluster-controldata.cat_ver = (uint32) atol(p);
}
! else if ((p = strstr(bufin, First log file ID after reset:)) != NULL)
{
p = strchr(p, ':');
***
*** 153,161
cluster-controldata.logid = (uint32) atol(p);
got_log_id = true;
}
! else if ((p = strstr(bufin, First log file segment after reset:)) != NULL
Re: [HACKERS] Row-level Locks SERIALIZABLE transactions, postgres vs. Oracle
On May 13, 2010, at 23:39 , Tom Lane wrote: Florian Pflug f...@phlo.org writes: All in all, I believe that SHARE and UPDATE row-level locks should be changed to cause concurrent UPDATEs to fail with a serialization error. I don't see an argument for doing that for FOR SHARE locks, and it already happens for FOR UPDATE (at least if the row actually gets updated). Yes, actually updating the row is a workaround. A prohibitively expensive one, though. The arguments are as stated a) SHARE or UPDATE locking a concurrently updated row *does* cause as serialization error, making the current behavior asymmetric b) Locking primitives usually ensure that once you obtain the lock you see the most recent version of the data. This is currently true for READ COMMITTED transactions but not for SERIALIZABLE ones, and pretty undesirable a behavior for a locking primitive. c) I fail to see how the current behavior is useful in the presence of SERIALIZABLE transactions. Currently, they could IMHO completely ignore FOR SHARE locks, without making any previously correct algorithm incorrect. plus a weaker one: d) Oracle does it for FOR UPDATE locks, and actually has an example of a FK trigger in PL/SQL in their docs. AFAICS this proposal mainly breaks things, in pursuit of an unnecessary and probably-impossible-anyway goal of making FK locking work with only user-level snapshots. I don't see the breakage this'd cause. For READ COMMITTED transactions nothing changes. For SERIALIZABLE transactions the behavior of FOR UPDATE / FOR SHARE becomes much easier to grasp. In both cases a SHARE lock would then say Only update this row if you have seen the locking transaction's changes. Why do you think that making FK locking work with only user-level snapshots is probably-impossible-anyway? With the proposed changes, simply FOR SHARE locking the parent row on INSERT/UPDATE of the child, plus checking for child rows on UPDATE/DELETE of the parent gives a 100% correct FK trigger. I do not have a formal proof for that last assertion, but I'm not aware of any counter-examples either. Would love to hear of any, though. best regards, Florian Pflug -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Row-level Locks SERIALIZABLE transactions, postgres vs. Oracle
On May 13, 2010, at 23:51 , Kevin Grittner wrote: Florian Pflug f...@phlo.org wrote: All in all, I believe that SHARE and UPDATE row-level locks should be changed to cause concurrent UPDATEs to fail with a serialization error. I can come up with a patch that does that, but I wanted to get some feedback on the idea before I put the work in. Before you work on that, you might want to wait until you can review the work that I and Dan Ports (a Ph.D. candidate from MIT) have been doing on support for true serializable transactions. You don't need to use FOR SHARE or FOR UPDATE or any explicit locks as long as the concurrent transactions are SERIALIZABLE. We have it working, but have been holding off on discussion or patch submission at Tom's request -- he felt it would distract from the process of getting the release out. I'm very exited about the work you're doing there, and believe it'd be a great feature to have. However, I view my proposal as pretty orthogonal to that work. True serializable transaction are much more powerful than what I proposed, but at a much higher price too, due to the necessity of SIREAD locks. My proposal allows for simple FK-like constraints to be implemented at user-level that are correct for all isolation levels. best regards, Florian Pflug -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_upgrade code questions
Takahiro Itagaki wrote: I read pg_upgrade code glance over, and found 4 issues in it. Are there any issues to be fixed before 9.0 release? 1. NAMEDATASIZE 2. extern PGDLLIMPORT 3. pathSeparator 4. EDB_NATIVE_LANG 1. NAMEDATASIZE pg_upgrade has the following definition, but should it be just NAMEDATALEN? /* Allocate for null byte */ #define NAMEDATASIZE (NAMEDATALEN + 1) Table names should be in NAMEDATELEN - 1 bytes. At least 64th bytes in name data is always '\0'. =# CREATE TABLE 1234567890...(total 70 chars)...1234567890 (i int); NOTICE: identifier 123...890 will be truncated to 123...0123 Agreed. I have changed the code to use NAMEDATALEN. 2. extern PGDLLIMPORT pg_upgrade has own definitions of extern PGDLLIMPORT Oid binary_upgrade_next_xxx in pg_upgrade_sysoids.c. But those variables are not declared as PGDLLIMPORT in the core. Can we access unexported variables here? The issue here is that you use PGDLLIMPORT where you are importing the variable, not where it is defined. For example, look at 'seq_page_cost'. You can see PGDLLIMPORT used where it is imported with 'extern', but not where is it defined. 3. pathSeparator Path separator for Windows is not only \ but also /. The current code ignores /. Also, it might not work if the path string including multi-byte characters that have \ (0x5c) in the second byte. Agreed. I have modified the code to use only / and check for / and \. It is used only for checking the last byte so I didn't think it would affect a multi-byte sequence. I am actually unclear on that issue though. Can you review the new code to see if it is OK. 4. EDB_NATIVE_LANG Of course it is commented out with #ifdef, but do we have codes for EDB in core? Yeah, removed. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] How to know killed by pg_terminate_backend
Maybe we could make PostgreSQL a little bit smarter so that it returns
a different code than 57P01 when killed by pg_terminate_backend().
Seems reasonable. Does the victim backend currently know why it has been
killed?
I don't think so.
One idea is postmaster sets a flag in the shared memory area
indicating it rceived SIGTERM before forwarding the signal to
backends.
Backend check the flag and if it's not set, it knows that the signal
has been sent by pg_terminate_backend(), not postmaster.
What about new error code:
#define ERRCODE_BACKEND_STOP_REQUEST
MAKE_SQLSTATE('5','7', 'P','0','4')
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Add PGFILEDESC description to Makefiles for all /contrib
Magnus Hagander wrote: On Thu, May 13, 2010 at 1:47 AM, Bruce Momjian br...@momjian.us wrote: bruce wrote: Tom Lane wrote: Bruce Momjian br...@momjian.us writes: All other binaries had such a designation, and all /contrib binaries were missing them. ?I assume I was doing cleanup. ?You want the icon removed from the backend makefile? Yes. ?I'm prepared to believe that not having the icons set on the contrib executables was an oversight. ?I'm much less prepared to assume that not marking the postgres executable was an oversight. ?Again, unless you *know* that this change is needed and appropriate, now is not the time to be making it, and especially not without discussion. OK, done with attached patch, and I added a comment about why it is not labeled. I did some research on PGFILEDESC and it does what I thought it does --- in embeds the 'ico' file into the executable in /pg/tools/msvc/Project.pm, and the image looks like the attached JPEG. The image is of a blue elephant head. So, currently, every binary uses that icon, except for the postmaster. Is that what we want? ?You could make the argument that a daemon, like the postmaster, shouldn't have one, which I think is Tom's point. It's pretty normal that daemons don't have icons. Yes, that is the logic I was looking for and documented that in the postmaster Makefile. One could argue the same for binaries that are normally never executed by the user, just internally - but I don't think we have any of those (we're talking EXE not DLL). There is, AFAIK, no rule (hard or of-thumb) for icons for GUI programs only not commandline ones. Commandline tools usually have it. The argument to be made there is that when you go into the bin directory you'll get greeted with a long list of identical elephants. It would probably be better if we could have a *different* icon for tools that the user is likely to execute himself - which is pretty much just psql I think. All /bin stuff already had icons --- I just added icons to /contrib binaries. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] quoting and recovery.conf
Robert Haas wrote: On Thu, May 13, 2010 at 1:00 PM, Bruce Momjian br...@momjian.us wrote: Is there a reason we require single quotes around boolean values in recovery.conf? ? ? ? ?standby_mode = 'off' This does not work: ? ? ? ?standby_mode = off I knew there were inconsistencies between quoting in postgresql.conf and recovery.conf, but I didn't realize it extended to boolean quoting. ?I see this at the top of recovery.conf now: ? ? ? ?# This file consists of lines of the form: ? ? ? ?# ? ? ? ?# ? name = 'value' ? ? ? ?# ? ? ? ?# (The quotes around the value are NOT optional, but the = is.) ? ? ? ?# and this issue existed in 8.4 as well. ?Seems I just never noticed it, and it not specifically mentioned in the TODO item we already have. I think we should add a TODO to parse recovery.conf with the same code we use to parse postgresql.conf, or possibly merge the two files. This issue was previously alluded to here: http://archives.postgresql.org/pgsql-hackers/2010-04/msg00211.php We have a TODO already for this: Allow recovery.conf to support the same syntax as postgresql.conf, including quoting * recovery.conf parsing problems I thought the problem was just quotes inside strings, not the requirement of quotes for everything. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Row-level Locks SERIALIZABLE transactions, postgres vs. Oracle
On Thu, May 13, 2010 at 10:25 PM, Florian Pflug f...@phlo.org wrote: C1: BEGIN C1: SELECT * FROM t WHERE id = 1 FOR UPDATE C2: BEGIN C2: SET TRANSACTION ISOLATION LEVEL SERIALIZABLE C2: SELECT * FROM t -- Take snapshot before C1 commits C1: COMMIT C2: DELETE FROM t WHERE id = 1 C2: COMMIT Can you give an actual realistic example -- ie, not doing a select for update and then never updating the row or with an explanation of what the programmer is attempting to accomplish with such an unusual sequence? The rest of the post talks about FKs but I don't see any here... -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_upgrade code questions
Bruce Momjian br...@momjian.us writes: Takahiro Itagaki wrote: 2. extern PGDLLIMPORT pg_upgrade has own definitions of extern PGDLLIMPORT Oid binary_upgrade_next_xxx in pg_upgrade_sysoids.c. But those variables are not declared as PGDLLIMPORT in the core. Can we access unexported variables here? The issue here is that you use PGDLLIMPORT where you are importing the variable, not where it is defined. For example, look at 'seq_page_cost'. You can see PGDLLIMPORT used where it is imported with 'extern', but not where is it defined. Right. Also we are intentionally not exposing those variables in any backend .h file, because they are not meant for general use. So the extern PGDLLIMPORT isn't going to be in the main backend and has to be in pg_upgrade. This was discussed awhile ago when we put in those variables, I believe. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] quoting and recovery.conf
On Thu, May 13, 2010 at 11:33 PM, Robert Haas robertmh...@gmail.com wrote: I think we should add a TODO to parse recovery.conf with the same code we use to parse postgresql.conf, or possibly merge the two files. This issue was previously alluded to here: http://archives.postgresql.org/pgsql-hackers/2010-04/msg00211.php And more than alluded to here: http://archives.postgresql.org/message-id/407d949e1002131017u657e4aefo2647c2cbf24fe...@mail.gmail.com -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_upgrade code questions
Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Takahiro Itagaki wrote: 2. extern PGDLLIMPORT pg_upgrade has own definitions of extern PGDLLIMPORT Oid binary_upgrade_next_xxx in pg_upgrade_sysoids.c. But those variables are not declared as PGDLLIMPORT in the core. Can we access unexported variables here? The issue here is that you use PGDLLIMPORT where you are importing the variable, not where it is defined. For example, look at 'seq_page_cost'. You can see PGDLLIMPORT used where it is imported with 'extern', but not where is it defined. Right. Also we are intentionally not exposing those variables in any backend .h file, because they are not meant for general use. So the extern PGDLLIMPORT isn't going to be in the main backend and has to be in pg_upgrade. This was discussed awhile ago when we put in those variables, I believe. Yes, this was discussed. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] quoting and recovery.conf
Greg Stark gsst...@mit.edu writes: On Thu, May 13, 2010 at 11:33 PM, Robert Haas robertmh...@gmail.com wrote: I think we should add a TODO to parse recovery.conf with the same code we use to parse postgresql.conf, or possibly merge the two files. This issue was previously alluded to here: http://archives.postgresql.org/pgsql-hackers/2010-04/msg00211.php And more than alluded to here: http://archives.postgresql.org/message-id/407d949e1002131017u657e4aefo2647c2cbf24fe...@mail.gmail.com The main reason for having a separate recovery.conf file is that its existence is what drives the setting of InArchiveRecovery. If we were to devise some other trigger for that condition, it'd be possible to fold all those settings in as GUC variables. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] JSON manipulation functions
First off, thank you for allowing me to participate in Google Summer
of Code 2010. I'm sorry I haven't been active for the past few weeks.
Today, I added the wiki page for my project, but the project schedule
is highly tentative:
http://wiki.postgresql.org/wiki/JSON_datatype_GSoC_2010 .
I'd like to discuss how the functions for type checking and conversion
(the majority of my project) should work. Below is my first draft for
the JSON manipulation function specs, along with annotations.
== Type checking ==
The following function returns the type of any JSON value.
json_type as enum ('null', 'string', 'number', 'boolean', 'object', 'array')
json_type(json) returns json_type
Would it be a bad idea to give an enum and a function the same name
(which appears to be allowed by PostgreSQL) ? If so, json_type(json)
could be json_typeof(json) or something instead.
I thought about having predicates like IS STRING and IS NUMBER,
similar to the IS DOCUMENT predicate used for XML. However, a major
problem with that approach is that it could lead to confusion
involving IS NULL. By my understanding, the JSON datatype will just
be a specialization of TEXT (it just validates the input). Like TEXT,
a JSON string can be 'null'. 'null'::JSON is not NULL. Bear in mind
that json_to_*('null') is NULL, though.
I also thought about having a series of json_is_* functions. I don't
think it's a bad idea, but I think json_type is a better solution.
== text/number/boolean conversion ==
These functions each convert a non-compound JSON value to its
respective return type. Run-time type checking is performed; a
conversion will throw an error if the input JSON is not the correct
type. If the JSON value is 'null', then the return value will be
NULL.
json_to_text(json) returns text
json_to_number(json) returns numeric
json_to_bool(json) returns boolean
These functions convert values to JSON. Passing NULL to any of the
functions below will return 'null':
text_to_json(text) returns json
number_to_json(numeric) returns json
bool_to_json(boolean) returns json
There could be generic value_to_json(any), but not a
json_to_value(json) function. See
http://archives.postgresql.org/pgsql-hackers/2010-04/msg00321.php for
more details.
Conversion to/from number or boolean can also be achieved with
casting. Note well that 'string'::JSON::TEXT is 'string', not the
string's actual value. json_to_text is needed for this conversion.
For this reason, casting JSON might seem like something to recommend
against. However, IMHO, casting numbers and booleans to/from JSON is
fine and dandy; the paragraphs below give some weight to this.
I originally considered making json_to_number and number_to_json work
with TEXT instead of NUMERIC. However, as Tom Lane pointed out in the
above link, Forcing people to insert explicit coercions from text
isn't going to be particularly convenient to use.. Nevertheless,
NUMERIC introduces a problem. For instance, if you say:
SELECT '-1e-38'::NUMERIC;
This conversion knocks out the scientific notation and produces a
41-character string. I seriously doubt that all outside applications
will handle 41-character numbers correctly.
Perhaps there should be individual functions for specific data types,
or maybe just a handful for particular cases. There might be
json_to_int, json_to_float, and json_to_numeric. In any case,
converting to/from number types can be achieved quite easily with
casting.
== array/object conversion ==
The json_object function converts a tuple to a JSON object. If there
are duplicate column names, there will be duplicate keys in the
resulting JSON object.
json_object([content [AS name] [, ...]]) returns json
Likewise, the json_array function converts a tuple to a JSON array.
Column names are ignored.
json_array([content [AS name] [, ...]]) returns json
The json_agg function reduces a set of JSON values to a single array
containing those values.
aggregate json_agg(json) returns json
json_object and json_agg can be used together to convert an entire
result set to one JSON array:
SELECT json_agg(json_object(*)) FROM tablename;
json_keys gets the keys of a JSON object as a set.
json_keys(json) returns setof text
json_values gets the values of a JSON object or the iems of a JSON
array as a set.
json_values(json) returns setof json
Note that all JSON slicing and splicing operations retain the original
formatting of JSON content.
== Miscellaneous ==
The features below would be nice, but will probably not be regarded as
required for this Google Summer of Code project to be considered
complete.
json_cleanup accepts a superset of JSON and, if it can, cleans it up
and returns a valid JSON string. This superset of JSON supports the
following extra features:
* Comments:
- Single-line comments with // and #
- C-style comments: /* comment */
* Unquoted object keys: {key: value}
* Single quote strings: 'single quotes; double quotes do not need
to be escaped here'
Re: [HACKERS] pg_upgrade code questions
Bruce Momjian br...@momjian.us wrote:
2. extern PGDLLIMPORT
pg_upgrade has own definitions of
extern PGDLLIMPORT Oid binary_upgrade_next_xxx
The issue here is that you use PGDLLIMPORT where you are importing the
variable, not where it is defined. For example, look at
'seq_page_cost'. You can see PGDLLIMPORT used where it is imported with
'extern', but not where is it defined.
Right. Also we are intentionally not exposing those variables in any
backend .h file, because they are not meant for general use. So the
extern PGDLLIMPORT isn't going to be in the main backend and has to
be in pg_upgrade. This was discussed awhile ago when we put in those
variables, I believe.
Yes, this was discussed.
I wonder some compilers or linkers might hide unexported global variables
from postgres.lib as if they are declared with 'static' specifiers.
I'm especially worried about Windows and MSVC. So, if Windows testers
can see it works, there was nothing to worry about.
Regards,
---
Takahiro Itagaki
NTT Open Source Software Center
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] wal_level and continuous archiving documentation
On Fri, May 14, 2010 at 1:42 AM, Joshua Tolley eggyk...@gmail.com wrote: I was reading through http://www.postgresql.org/docs/9.0/static/continuous-archiving.html and noticed that wal_level isn't mentioned where I'd expect it to be. Specifically, there's a paragraph that starts, To enable WAL archiving, set the archive_mode configuration parameter to on, and specify the shell command to use in the archive_command configuration parameter. There follows a long discussion of archive_command, but no further discussion of other configuration settings for several paragraphs, suggesting that those two configuration changes are the only ones required to end up with a useful archive. However, further on, it discusses wal_level: When wal_level is minimal some SQL commands are optimized to avoid WAL logging, as described in Section 14.4.7. If archiving or streaming replication were turned on during execution of one of these statements, WAL would not contain enough information for archive recovery. ISTM wal_archive should make an appearance where the docs bring up archive_mode and archive_command, to say wal_level must be set to 'archive' or 'hot_standby', so all required configuration changes are mentioned close together. Some descriptions about wal_level were added after beta1 release. http://archives.postgresql.org/pgsql-committers/2010-05/msg00019.php You can read the latest (development) document from http://developer.postgresql.org/pgdocs/postgres/continuous-archiving.html Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] quoting and recovery.conf
On Fri, May 14, 2010 at 9:37 AM, Bruce Momjian br...@momjian.us wrote: I thought the problem was just quotes inside strings, not the requirement of quotes for everything. You can embed a single quote in a parameter value by writing two quotes. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] wal_level in postgresql.conf
On Fri, May 14, 2010 at 12:22 AM, Bruce Momjian br...@momjian.us wrote: Bruce Momjian wrote: Why is 'wal_level' listed next to 'fsync' in postgresql.conf? #wal_level = 'hot_standby' # minimal, archive, or hot_standby #fsync = on # turns forced synchronization on or off #synchronous_commit = on # immediate fsync at commit #wal_sync_method = fsync # the default is the first option Seems it should be in the archiving section. -1 since wal_level affects WAL logging itself, and is to do with not only archiving but also hot standby. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] List traffic
On Thu, 13 May 2010, Joshua D. Drake wrote: Between labels, filters, watch lists and all the other goodies any MUA will give you, I see no reason to have this all broken out anymore. So, if one merges all the lists into one (not arguing for / against that), how do you filter? Based on what? Right now, ppl filter based on the X-Mailing-List header, or just the Participant ... Marc G. FournierHub.Org Hosting Solutions S.A. scra...@hub.org http://www.hub.org Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nvarchar notation accepted?
Jaime Casanova ja...@2ndquadrant.com wrote: i migrate a ms sql server database to postgres and was trying some queries from the application to find if everything works right... when i was looking to those queries i found some that has a notation for nvarchar (ej: campo = N'sometext') Do you have documentation for N'...' literal in SQLServer? Does it mean unicode literal? What is the difference from U literal? http://developer.postgresql.org/pgdocs/postgres/sql-syntax-lexical.html PostgreSQL doesn't have nvarchar types (UTF16 in MSSQL), and only have mutlti-tyte characters. So I think you can remove N and just use SET client_encoding = UTF8 in the cases. Regards, --- Takahiro Itagaki NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_upgrade code questions
Takahiro Itagaki wrote: Bruce Momjian br...@momjian.us wrote: 2. extern PGDLLIMPORT pg_upgrade has own definitions of extern PGDLLIMPORT Oid binary_upgrade_next_xxx The issue here is that you use PGDLLIMPORT where you are importing the variable, not where it is defined. For example, look at 'seq_page_cost'. You can see PGDLLIMPORT used where it is imported with 'extern', but not where is it defined. Right. Also we are intentionally not exposing those variables in any backend .h file, because they are not meant for general use. So the extern PGDLLIMPORT isn't going to be in the main backend and has to be in pg_upgrade. This was discussed awhile ago when we put in those variables, I believe. Yes, this was discussed. I wonder some compilers or linkers might hide unexported global variables from postgres.lib as if they are declared with 'static' specifiers. I'm especially worried about Windows and MSVC. So, if Windows testers can see it works, there was nothing to worry about. Yes, none of the variables pg_upgrade is referencing are 'static', and Magnus tested MSVC and checked MinGW compiles. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] List traffic
Excerpts from Marc G. Fournier's message of jue may 13 23:11:40 -0400 2010: On Thu, 13 May 2010, Joshua D. Drake wrote: Between labels, filters, watch lists and all the other goodies any MUA will give you, I see no reason to have this all broken out anymore. So, if one merges all the lists into one (not arguing for / against that), how do you filter? Based on what? Right now, ppl filter based on the X-Mailing-List header, or just the Participant ... If most of the questions are badly categorized or cross posted to more than one list, how useful a label is the X-Mailing-List header? How useful is to filter on the pgsql-general label? -- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nvarchar notation accepted?
Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp writes: Jaime Casanova ja...@2ndquadrant.com wrote: i migrate a ms sql server database to postgres and was trying some queries from the application to find if everything works right... when i was looking to those queries i found some that has a notation for nvarchar (ej: campo = N'sometext') Do you have documentation for N'...' literal in SQLServer? Does it mean unicode literal? What is the difference from U literal? http://developer.postgresql.org/pgdocs/postgres/sql-syntax-lexical.html PostgreSQL doesn't have nvarchar types (UTF16 in MSSQL), and only have mutlti-tyte characters. So I think you can remove N and just use SET client_encoding = UTF8 in the cases. Actually, the lexer translates N'foo' to NCHAR 'foo' and then the grammar treats that just like CHAR 'foo'. In short, the N doesn't do anything very useful, and it certainly doesn't have any effect on encoding behavior. I think this is something Tom Lockhart put in ten or so years back, and never got as far as making it actually do anything helpful. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nvarchar notation accepted?
On Thu, May 13, 2010 at 10:52 PM, Tom Lane t...@sss.pgh.pa.us wrote: Actually, the lexer translates N'foo' to NCHAR 'foo' and then the grammar treats that just like CHAR 'foo'. In short, the N doesn't do anything very useful, and it certainly doesn't have any effect on encoding behavior. I think this is something Tom Lockhart put in ten or so years back, and never got as far as making it actually do anything helpful. so, the N'' syntax is fine and i don't need to hunt them as a migration step? -- Jaime Casanova www.2ndQuadrant.com Soporte y capacitación de PostgreSQL -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nvarchar notation accepted?
Jaime Casanova ja...@2ndquadrant.com writes: On Thu, May 13, 2010 at 10:52 PM, Tom Lane t...@sss.pgh.pa.us wrote: Actually, the lexer translates N'foo' to NCHAR 'foo' and then the grammar treats that just like CHAR 'foo'. Â In short, the N doesn't do anything very useful, and it certainly doesn't have any effect on encoding behavior. Â I think this is something Tom Lockhart put in ten or so years back, and never got as far as making it actually do anything helpful. so, the N'' syntax is fine and i don't need to hunt them as a migration step? As long as the implied cast to char(n) doesn't cause you problems, it's fine. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nvarchar notation accepted?
On Thu, May 13, 2010 at 10:13 PM, Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp wrote: Jaime Casanova ja...@2ndquadrant.com wrote: i migrate a ms sql server database to postgres and was trying some queries from the application to find if everything works right... when i was looking to those queries i found some that has a notation for nvarchar (ej: campo = N'sometext') Do you have documentation for N'...' literal in SQLServer? Does it mean unicode literal? What is the difference from U literal? http://developer.postgresql.org/pgdocs/postgres/sql-syntax-lexical.html nop, only thing i found is about NVARCHAR: http://msdn.microsoft.com/en-us/library/ms186939.aspx but it has no examples about the N'' notation although you can find examples of it use here: http://msdn.microsoft.com/en-us/library/dd776381.aspx#BasicSyntax PostgreSQL doesn't have nvarchar types (UTF16 in MSSQL), and only have mutlti-tyte characters. So I think you can remove N and just use SET client_encoding = UTF8 in the cases. i don't want to remove it! i'm trying to understand if this is a bug that will be removed if no i can safely tell my client to not look for those queries so it has less work to do for the migration -- Jaime Casanova www.2ndQuadrant.com Soporte y capacitación de PostgreSQL -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Generalized Inverted Generalized Search Tree
We can index multiple scalar values per row with GIN access method, and also can index single vector value per row with GiST AM. Is it worth having a new AM to index multiple vector values per row? It will be an AM for the missing feature in below: | scalar | vector | +++ single per row | btree | gist | multi per row | gin| *HERE* | We can call the new AM gigist. Or, there might be another idea to support expression indexes for SRF functions, like =# CREATE TABLE tbl (c circle[]); =# CREATE INDEX ON tbl USING gist (unnest(c)); Comments and ideas welcome. Regards, --- Takahiro Itagaki NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] nvarchar notation accepted?
On tor, 2010-05-13 at 23:52 -0400, Tom Lane wrote: Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp writes: Jaime Casanova ja...@2ndquadrant.com wrote: i migrate a ms sql server database to postgres and was trying some queries from the application to find if everything works right... when i was looking to those queries i found some that has a notation for nvarchar (ej: campo = N'sometext') Do you have documentation for N'...' literal in SQLServer? Does it mean unicode literal? What is the difference from U literal? http://developer.postgresql.org/pgdocs/postgres/sql-syntax-lexical.html PostgreSQL doesn't have nvarchar types (UTF16 in MSSQL), and only have mutlti-tyte characters. So I think you can remove N and just use SET client_encoding = UTF8 in the cases. Actually, the lexer translates N'foo' to NCHAR 'foo' and then the grammar treats that just like CHAR 'foo'. In short, the N doesn't do anything very useful, and it certainly doesn't have any effect on encoding behavior. I think this is something Tom Lockhart put in ten or so years back, and never got as far as making it actually do anything helpful. This should maybe changed to just ignoring the N and treating N'' like ''. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] List traffic
On Thu, 13 May 2010, Alvaro Herrera wrote: If most of the questions are badly categorized or cross posted to more than one list, how useful a label is the X-Mailing-List header? How useful is to filter on the pgsql-general label? That is a point, but, IMHO, that is one of our key issues ... we *allow* that sort of cross-posting in the first place ... FreeBSD lists allow cross-posting to no more then 2 mailing lists, I believe, but there is definitely a limit ... ... is there a reason why, other the fact that we don't do now, that we can't just put in a restriction against cross posting altogether? ... and, for those that have been here awhile, who should know better, why isn't there any self-management of this sort of stuff in the first place? Marc G. FournierHub.Org Hosting Solutions S.A. scra...@hub.org http://www.hub.org Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] quoting and recovery.conf
On Thu, 2010-05-13 at 21:15 -0400, Tom Lane wrote: Greg Stark gsst...@mit.edu writes: On Thu, May 13, 2010 at 11:33 PM, Robert Haas robertmh...@gmail.com wrote: I think we should add a TODO to parse recovery.conf with the same code we use to parse postgresql.conf, or possibly merge the two files. This issue was previously alluded to here: http://archives.postgresql.org/pgsql-hackers/2010-04/msg00211.php And more than alluded to here: http://archives.postgresql.org/message-id/407d949e1002131017u657e4aefo2647c2cbf24fe...@mail.gmail.com The main reason for having a separate recovery.conf file is that its existence is what drives the setting of InArchiveRecovery. If we were to devise some other trigger for that condition, it'd be possible to fold all those settings in as GUC variables. And the removal of recovery.conf at end of recovery prevents the re-entry into archive recovery if we crash. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Row-level Locks SERIALIZABLE transactions, postgres vs. Oracle
On 05/14/2010 03:37 AM, Greg Stark wrote: On Thu, May 13, 2010 at 10:25 PM, Florian Pflugf...@phlo.org wrote: C1: BEGIN C1: SELECT * FROM t WHERE id = 1 FOR UPDATE C2: BEGIN C2: SET TRANSACTION ISOLATION LEVEL SERIALIZABLE C2: SELECT * FROM t -- Take snapshot before C1 commits C1: COMMIT C2: DELETE FROM t WHERE id = 1 C2: COMMIT Can you give an actual realistic example -- ie, not doing a select for update and then never updating the row or with an explanation of what the programmer is attempting to accomplish with such an unusual sequence? The rest of the post talks about FKs but I don't see any here... Doing a select for update and then never updating the row is a realistic example. I am currently designing a database where this is an issue. The simplified schema to illustrate the problem: create table object ( id integer primary key ); insert into object values(1); create table attribute ( object_id integer not null references object, attr_type integer not null, -- references attr_types value text not null, valid_from timestamp not null, valid_until timestamp ); Now, I want to make sure there are no pairs of (object_id, attr_type) where the valid_from, valid_until times overlap. A problematic sequence for this schema, both transactions in isolation level serializable: C1: begin; C1: select * from object where id = 1 for update; -- check for conflicting attr_type, realistically where condition should have overlapping check, but left out for simplicity... C1: select * from attribute where object_id = 1 and attr_type = 1; -- Ok, nothing overlapping, I am able to insert. C1: insert into attribute values (1, 1, 'Anssi', now(), null); C2: begin; -- This blocks. C2: select * from object where id = 1 for update; C1: commit; -- Check for conflicts. This select won't see the insert C1 did. C2: select * from attribute where object_id = 1 and attr_type = 1; -- C2 doesn't see anything conflicting C2: insert into attribute values (1, 1, 'Matti', now(), null); C2: commit; -- Inconsistency. Now, that same sequence does work for read committed isolation level (C2 sees the insert of C1), and that is my solution for now: require applications to use read committed isolation level. This could also be solved by issuing update object set id = id where id = 1 instead of using select for update. This would result in serialization error. I know that for this particular example the upcoming exclusion constraints would solve the problem. But if I would want to ensure that if attr_value for attr_type 1 is 'Anssi' then attr_value for attr_type 2 is 'Kääriäinen', then exclusion constraints could not be used. -- Anssi Kääriäinen -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
