[HACKERS] ACK from walreceiver to walsender
Hi Heikki, http://git.postgresql.org/gitweb?p=users/heikki/postgres.git;a=commit;h=ebaa89ce8906e0ec45f105d083a0360b1f8bc7ca You dropped all the ACKs from walreceiver to walsender. I have no objection to that, but I think that walsender should handle at least 'X' (which means that the standby is closing down the socket) and EOF (which means unexpected loss of standby connection) messages from walreceiver. Otherwise, walsender might be unable to detect the shutdown of walreceiver for a while. Thought? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
On Jan 7, 2010, at 4:07 PM, Josh Berkus wrote: Building a simple solution which doesn't initially cover all bases but can be steadily improved is a far superior strategy to trying to spec The Perfect Solution before even starting work. And if we want to keep recruiting new contributors, criticism needs to be more constructive. I think we should call it Pecan. And paint it yellow! ;P +1 from me. -- Jim C. Nasby, Database Architect j...@nasby.net 512.569.9461 (cell) http://jim.nasby.net -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
On Fri, Jan 8, 2010 at 05:22, Ron Mayer rm...@cheapcomplexdevices.com wrote: David E. Wheeler wrote: On Jan 7, 2010, at 1:31 PM, Dave Page wrote: No, I'm suggesting the mechanism needs to support source and binary distribution. For most *nix users, source will be fine. For Windows binaries are required. I would love to follow what Strawberry Perl has done to solve this problem. In 2.0. +1. They did a nice job. For those of us who have no idea what Strawberry Perl did (other than not shipping Microsoft compatible libraries, and is thus useless for PostgreSQL), could someone explain it? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Add .gitignore files to CVS?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, Jan 07, 2010 at 04:44:49PM -0700, Alex Hunsaker wrote: On Thu, Jan 7, 2010 at 15:16, Tim Bunce tim.bu...@pobox.com wrote: Is there any reason not to add .gitignore files into the repository? They'll make no difference to those who don't use git, but be very helpful to, and maintained by, those who do. Since it seems we don't want them in CVS, maybe just add it to the git mirror? I don't know that we want the git mirror to have commits/files that CVS does not. *shrug* Thoughts people? ...then you would have to add .gitignore to .cvsignore, lest cvs commit complains about *that* ;-P regards - -- tomás -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFLRvGvBcgs9XrR2kYRAoWbAJwNU6Zt5gfYiBipzNFPuZAnbLsgeACdGWac 0UegxgApq7SHGRwR++tt/sI= =Iatz -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Why doesn't query_tree_walker examine the intoClause field?
Hi, I am kinda puzzled as to why the query_tree_walker() function does not examine the intoClause field? I do see another function raw_expression_tree_walker() which does walk that entry. So what is the exact reason here? Or we just missed it for the earlier function? Regards, Nikhils -- http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ACK from walreceiver to walsender
Fujii Masao wrote: Hi Heikki, http://git.postgresql.org/gitweb?p=users/heikki/postgres.git;a=commit;h=ebaa89ce8906e0ec45f105d083a0360b1f8bc7ca You dropped all the ACKs from walreceiver to walsender. I have no objection to that, but I think that walsender should handle at least 'X' (which means that the standby is closing down the socket) and EOF (which means unexpected loss of standby connection) messages from walreceiver. Otherwise, walsender might be unable to detect the shutdown of walreceiver for a while. Yeah, I just noticed that myself :-(. I guess we'll still have to use select() in the walreceiver loop to detect that then. I don't think we need to treat 'X' differently from EOF. You get an error anyway if the write() fails. That's actually a bit annoying, you get a could not send data to client error in the log every time a standby disconnects for any reason. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
David Fetter da...@fetter.org writes: If we *must* have SR and it's not in by the 15th, let's do another Commitfest rather than jack the people who played by the rules. If we do add another Commitfest what we do is exactly jacking people who played by the rules. Because all those patches that are already part of alpha3 have been worked on by people expecting a 4 CF development cycle, and adjusted their agenda, and want a mid-year release. Now, I'll second Greg Smith and Tom here, in that I think we need to run the last commitfest as usual, knowing that the outcome of the commitfest for any given patch is not it made it but we reviewed it. It's still right for the project to bump a patch on resources ground rather than on technical merit, at the end of the commitfest. Why we can do it this way is because we're not starving on reviewers. We're starving on commiters time. And seeing this: https://commitfest.postgresql.org/action/commitfest_view?id=5 Status Summary. Needs Review: 19, Waiting on Author: 5, Ready for Committer: 2, Committed: 9, Returned with Feedback: 4. Total: 39. I don't see any reason not to consider all the 24 patches requiring our attention. Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Add .gitignore files to CVS?
On Fri, Jan 8, 2010 at 00:44, Alex Hunsaker bada...@gmail.com wrote: On Thu, Jan 7, 2010 at 15:16, Tim Bunce tim.bu...@pobox.com wrote: Is there any reason not to add .gitignore files into the repository? They'll make no difference to those who don't use git, but be very helpful to, and maintained by, those who do. Since it seems we don't want them in CVS, maybe just add it to the git mirror? I don't know that we want the git mirror to have commits/files that CVS does not. *shrug* Thoughts people? Definite -1. We want the git repository to be (as much as possible) identical to the CVS one. You can always create your own branch with just the .gitignore files and merge that into whatever you're working on :) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ACK from walreceiver to walsender
On Fri, Jan 8, 2010 at 5:55 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Fujii Masao wrote: Hi Heikki, http://git.postgresql.org/gitweb?p=users/heikki/postgres.git;a=commit;h=ebaa89ce8906e0ec45f105d083a0360b1f8bc7ca You dropped all the ACKs from walreceiver to walsender. I have no objection to that, but I think that walsender should handle at least 'X' (which means that the standby is closing down the socket) and EOF (which means unexpected loss of standby connection) messages from walreceiver. Otherwise, walsender might be unable to detect the shutdown of walreceiver for a while. Yeah, I just noticed that myself :-(. I guess we'll still have to use select() in the walreceiver loop to detect that then. I don't think we need to treat 'X' differently from EOF. You get an error anyway if the write() fails. That's actually a bit annoying, you get a could not send data to client error in the log every time a standby disconnects for any reason. Yes. And, when walreceiver exits, it sends 'X' message by calling PQfinish(). So I think it's neater for walsender to treat also 'X'. Thought? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
On Fri, Jan 8, 2010 at 10:02, Dimitri Fontaine dfonta...@hi-media.com wrote: David Fetter da...@fetter.org writes: If we *must* have SR and it's not in by the 15th, let's do another Commitfest rather than jack the people who played by the rules. If we do add another Commitfest what we do is exactly jacking people who played by the rules. Because all those patches that are already part of alpha3 have been worked on by people expecting a 4 CF development cycle, and adjusted their agenda, and want a mid-year release. Now, I'll second Greg Smith and Tom here, in that I think we need to run the last commitfest as usual, knowing that the outcome of the commitfest for any given patch is not it made it but we reviewed it. It's still right for the project to bump a patch on resources ground rather than on technical merit, at the end of the commitfest. +1. Why we can do it this way is because we're not starving on reviewers. We're starving on commiters time. And seeing this: Well, we're actually somewhat starving on senior reviewers as well. That can take on things like the index patches, Writable CTE or SR. We're not starving on reviewers for small-to-medium patches. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why doesn't query_tree_walker examine the intoClause field?
Self answer: Because the post-analysis phase does not have anything interesting to peek in it. The only interesting thing could be the RangeVar, but it is not going to be in an RTE form after the transformstmt, so not much point. Sorry for the noise. Regards, Nikhils On Fri, Jan 8, 2010 at 2:22 PM, Nikhil Sontakke nikhil.sonta...@enterprisedb.com wrote: Hi, I am kinda puzzled as to why the query_tree_walker() function does not examine the intoClause field? I do see another function raw_expression_tree_walker() which does walk that entry. So what is the exact reason here? Or we just missed it for the earlier function? Regards, Nikhils -- http://www.enterprisedb.com -- http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
Magnus Hagander mag...@hagander.net writes: Why we can do it this way is because we're not starving on reviewers. We're starving on commiters time. And seeing this: Well, we're actually somewhat starving on senior reviewers as well. That can take on things like the index patches, Writable CTE or SR. We're not starving on reviewers for small-to-medium patches. We've been talking about having specialized reviewers, or multi layered reviewing. There are several things we do in reviewing, and for big enough patches there's no need to have the same reviewer do all of them. [...searching the archives for a proposal I did already send...] http://archives.postgresql.org/pgsql-hackers/2009-08/msg00764.php So this mail proposes we see those separate items to be handled in review: - patch (applies, merge, compiles, pass regression) - code reading (looks like it was already there, no WTF?) [1] - documentation (covers code, targets users, is sufficient) - testing (code behavior is what is documented, works well) - creative testing (tried hard to crash it) - perf testing (profiling, no regression in non optimized cases...) - you name it Now the senior reviewers you're talking about are required the most for code reading. We certainly still can have an army of junior reviewers, or not-wannabe-hackers reviewers checking the other points. That'd push the bottleneck some. Regards, -- dim [1] http://www.osnews.com/images/comics/wtfm.jpg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] new full vacuum doesn't work
Hello
I am testing vacuum changes, and I found some strange behave:
autovacuum off
[pa...@nemesis src]$ /usr/local/pgsql/bin/pgbench -i -F 10 -s 10 test
NOTICE: table pgbench_branches does not exist, skipping
NOTICE: table pgbench_tellers does not exist, skipping
NOTICE: table pgbench_accounts does not exist, skipping
NOTICE: table pgbench_history does not exist, skipping
test=# \dt+
List of relations
Schema | Name | Type | Owner |Size| Description
+--+---+---++-
public | pgbench_accounts | table | pavel | 1302 MB|
public | pgbench_branches | table | pavel | 8192 bytes |
public | pgbench_history | table | pavel | 0 bytes|
public | pgbench_tellers | table | pavel | 48 kB |
(4 rows)
test=# \x
Expanded display is on.
test=# select * from pgstattuple('pgbench_accounts');
-[ RECORD 1 ]--+---
table_len | 1365336064
tuple_count| 100
tuple_len | 12100
tuple_percent | 8.86
dead_tuple_count | 0
dead_tuple_len | 0
dead_tuple_percent | 0
free_space | 1228669388
free_percent | 89.99
test=# vacuum full pgbench_accounts;
VACUUM
test=# \dt+
List of relations
Schema | Name | Type | Owner |Size| Description
+--+---+---++-
public | pgbench_accounts | table | pavel | 1302 MB|
public | pgbench_branches | table | pavel | 8192 bytes |
public | pgbench_history | table | pavel | 0 bytes|
public | pgbench_tellers | table | pavel | 48 kB |
(4 rows)
test=# select * from pgstattuple('pgbench_accounts');
-[ RECORD 1 ]--+---
table_len | 1365336064
tuple_count| 100
tuple_len | 12800
tuple_percent | 9.37
dead_tuple_count | 0
dead_tuple_len | 0
dead_tuple_percent | 0
free_space | 1228669388
free_percent | 89.99
Regards
Pavel
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
On Thu, Jan 7, 2010 at 10:07 PM, Josh Berkus j...@agliodbs.com wrote: Building them is no problem - authors can easily use EC2 for which we have an AMI pre-configured for next to no cost, can build on their own platform, on a community provided system, or get a friend to do it. So any module author, in order to submit any module, would be required to build binaries for 8-12 platforms covering up to 5 PostgreSQL versions (i.e. 30-70 different binaries)? At their own cost? Seems like a good way to not get any contributions at all. No, I'm suggesting that for the module authors that want to support Windows, there are a variety of easy ways to get binaries built. It shouldn't be a requirement of *them*. For that matter, Andrew just pointed out to me (corrected me, actually) on IRC that if a Windows user has MSVC or Mingw installed, it should be no problem supporting them. If it were likely that Windows users would have a suitable compiler installed, I wouldn't be complaining. So what you're asking has nothing to do with Windows users, but is a more general we want support for users who don't have a compiler installed. That's a different problem -- and one which the One-click installer or Stackbuilder should probably solve rather than PGAN. It already does - on Windows, Linux and Mac. I don't need this feature, and nor do the users of my distributions. I just thought I was being helpful by pointing out some problems in David's design which I assumed was the point of the RFC. No. The essence is, 'If you're going to do it in a way that will never work for maybe 50% or more of PostgreSQL installations, then you have fundamental design issues to overcome'. Again, that's the wrong attitude. You're saying If it doesn't work for 100% of Windows users from Day 1, it won't ever work for them. By that logic, we should have held back version 7.4 until the windows port was done, dammit! And we shouldn't have released until it worked with Visual C++. Even if it meant releasing in 2007. And we shouldn't have released PITR until it supported HS. And we shouldn't be releasing 8.5 without Win64 support. Please re-read what I said before churning out ridiculous arguments that in no way relate to what I wrote. I am saying that if the design won't ever work without requiring painful dependency installation that users will likely not want to bother with, then it is fundamentally broken. Better to write one system that can _eventually_ work everywhere, than one that works for some, and then another that works elsewhere. This list *also* has a real tendency to have an incredible negative attitude, which *you* are currently expressing. The constructive way for you to approach this would have been to say I think that the general idea has merit, but that putting off Windows support is a mistake. What about supporting binary distribution at the outset? Again, please read what I wrote. That is almost exactly what I did write. I said it was a good idea, I explained why I thought ignoring Windows was an issue, and I noted that we had previously discussed binary distribution. coding the client in C? Instead, you said this doesn't solve problems A, B, and C, so it's stupid. No - I said it won't work for most windows users, use C instead as that'll work everywhere. At no point did I say or imply it was stupid, just that it wasn't a universal solution. Building a simple solution which doesn't initially cover all bases but can be steadily improved is a far superior strategy to trying to spec The Perfect Solution before even starting work. And if we want to keep recruiting new contributors, criticism needs to be more constructive. For every criticism I made, I gave my reasoning, and offered a solution or an idea that would resolve my concern. In what bloody way is that not constructive? Remember, David posted an RFC. I'm not going to fill this list with grovelling bullshit about how every line he wrote is genius. I commented on the parts I considered a problem, and gave an overall nod to the rest. If all you want on this list is positive comments, then I won't bother any more as that won't help anyone. /D -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ACK from walreceiver to walsender
Fujii Masao wrote: On Fri, Jan 8, 2010 at 5:55 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I don't think we need to treat 'X' differently from EOF. You get an error anyway if the write() fails. That's actually a bit annoying, you get a could not send data to client error in the log every time a standby disconnects for any reason. Yes. And, when walreceiver exits, it sends 'X' message by calling PQfinish(). So I think it's neater for walsender to treat also 'X'. Thought? There's no guarantee walreceiver will read the 'X' before trying to write() to the socket, so we can't rely on that to determine whether to suppress the could not send data to client message. We could try to read() from the socket after the write() has failed, to see if there's an 'X' message pending. Not sure it's worth it. I think we would have to put the socket into non-blocking mode before the read(), to avoid blocking if the write() failed for some other reason. Or select() to see if there's incoming data. I'm inclined to just not bother.. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch: Allow substring/replace() to get/set bit values
What we can do in the back branches is make the code treat any negative value as meaning two-arg form. To throw an error we'd need to refactor the pg_proc representation ... I was going to fix that myself, but I think it has just been done. How can I keep up with who's doing what? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Streaming replication and triggering failover
The trigger file logic feels a bit backwards. As the patch stands, when the standby starts up, it retries connecting to the master server indefinitely, until a connection is successfully established. Then it streams until the connection breaks. If the connection is dropped abruptly, because of a network problem or crash in the master, standby retries indefinitely. If master is shut down cleanly, standby gets out of recovery mode, and starts up. Unless the trigger file is present; if it is, standby waits for it to go away before finishing recovery. So the trigger file is really a holdoff file, like a safety catch on a gun. At the very least it should be renamed, but I don't think that's a very useful behavior anyway. It doesn't seem wise to consider a clean shutdown of the master as a signal to trigger failover. If you're setting up a HA system, that by itself is not robust enough; you also need to trigger failover if the master goes down unexpectedly, or if the standby was disconnected for some reason when the master was shut down. Secondly, what if you want to restart the master server, without initiating failover? You'll have to restart the standby too, to have it reconnect. Let's have a default of no failover, and retry connecting to the master indefinitely. When you *do* want to fail over, create the trigger file. When the standby sees the trigger file, it should stop streaming, finish up replaying what it had streamed up to that point, and start up as new master. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Serializable Isolation without blocking
Hi, Greg Stark wrote: I think we're still talking past the issue. Predicate locks are not row level, nor page level, nor table level. They're locks on predicates. Ie, you have to lock against values which aren't actually currently in the table at all. You need to be able to detect a conflict between a transaction which read rows WHERE i BETWEEN 1 AND 10 and one which tries to insert a row with i=1 even if the first transaction didn't see any rows with i=1 (or indeed even if the first transaction found no rows at all). I absolutely agree to that. That's about predicate locks. I've been talking about SIREAD, which is a different thing (and which I don't consider to be a lock). The SIREAD thingie certainly doesn't help implement predicate locks. And predicate locking isn't necessarily sufficient to implement truly serializable isolation. That's the hard part. How do you represent such a lock in a way that I can efficiently find it and check for the conflict when it comes time for me to do my insert. As it's not really a lock, but rather a mark or a tag, SIREAD may or may not be implemented atop existing or non-existent locking structures, IMO. I've made my points about implementing SIREAD atop table level or row level locking structures. With (non-existent) predicate locking, I'm still unsure. It might help to implement SIREAD atop such a predicate as well. Predicate tagging? Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Streaming replication and triggering failover
On Fri, Jan 8, 2010 at 10:58, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: The trigger file logic feels a bit backwards. As the patch stands, when the standby starts up, it retries connecting to the master server indefinitely, until a connection is successfully established. Then it streams until the connection breaks. If the connection is dropped abruptly, because of a network problem or crash in the master, standby retries indefinitely. If master is shut down cleanly, standby gets out of recovery mode, and starts up. Unless the trigger file is present; if it is, standby waits for it to go away before finishing recovery. So the trigger file is really a holdoff file, like a safety catch on a gun. At the very least it should be renamed, but I don't think that's a very useful behavior anyway. It doesn't seem wise to consider a clean shutdown of the master as a signal to trigger failover. If you're setting up a HA system, that by itself is not robust enough; you also need to trigger failover if the master goes down unexpectedly, or if the standby was disconnected for some reason when the master was shut down. Secondly, what if you want to restart the master server, without initiating failover? You'll have to restart the standby too, to have it reconnect. Let's have a default of no failover, and retry connecting to the master indefinitely. When you *do* want to fail over, create the trigger file. When the standby sees the trigger file, it should stop streaming, finish up replaying what it had streamed up to that point, and start up as new master. +1. The default should be to maintain the replication cluster, if nothing else then by principle of least surprise. It would also agree with a well-established procedure, which is what pg_standby does. Keeping the same basic behavior around something like this can only be a good thing. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Serializable Isolation without blocking
Hi, Kevin Grittner wrote: As I understand it, Greg's line of thinking is that we should use a technique which has never proven practical on a large scale: matching database changes against a list of predicate lock expressions. I find that approach to predicate locking pretty interesting. However, unlike others, it scales with the number of concurrently held locks. And with the current trend towards multi-multi-core platforms, that might get worse and worse (as concurrency must increase to efficiently use these cores). Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
Hi, Josh Berkus wrote: Dave wrote: and frankly, isn't the way this project generally works. Isn't it? We didn't even support Windows for quite a long time. We still have lots more active Unix developers and knowledge that Windows ones. And isn't there some scratch your own itch philosophy in every OSS project? But why is that *this* project's job to solve? It's already the case that contrib modules (or PostgreSQL) are not buildable on Windows in an automated fashion. And that Windows does not ship with developer tools. That's not PGAN's fault. I'm with Josh here, get a PGAN up and running. Make it accessible via web to promote extensibility of Postgres and availability of extensions. Whether the first incarnation of the PGAN client works on Windows or Linux doesn't matter for now. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Streaming replication and triggering failover
Magnus Hagander wrote: On Fri, Jan 8, 2010 at 10:58, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: So the trigger file is really a holdoff file, like a safety catch on a gun. At the very least it should be renamed, but I don't think that's a very useful behavior anyway. It doesn't seem wise to consider a clean shutdown of the master as a signal to trigger failover. If you're setting up a HA system, that by itself is not robust enough; you also need to trigger failover if the master goes down unexpectedly, or if the standby was disconnected for some reason when the master was shut down. Secondly, what if you want to restart the master server, without initiating failover? You'll have to restart the standby too, to have it reconnect. Let's have a default of no failover, and retry connecting to the master indefinitely. When you *do* want to fail over, create the trigger file. When the standby sees the trigger file, it should stop streaming, finish up replaying what it had streamed up to that point, and start up as new master. +1. The default should be to maintain the replication cluster, if nothing else then by principle of least surprise. It would also agree with a well-established procedure, which is what pg_standby does. Keeping the same basic behavior around something like this can only be a good thing. Thinking more clearly, my comment above about the trigger file logic being backwards was bollocks; if the master is shut down, standby waits for the trigger file to appear, not to go away. And creating the trigger file during replication causes it to finish, and failover to happen. Nevertheless, let's make the default no failover if no trigger file location is configured, and remove the notion that normal shutdown of master stops recovery. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
On Fri, Jan 8, 2010 at 10:40 AM, Markus Wanner mar...@bluegap.ch wrote: Hi, Josh Berkus wrote: Dave wrote: and frankly, isn't the way this project generally works. Isn't it? We didn't even support Windows for quite a long time. No, it's quite different for the PostgreSQL not to support a platform at all, than for it to say it's supported on FOO/OS, but you can't use Window Functions or on BAR/OS, you can't use PITR. The only reason we ever offer different functionality on different platforms is when there are external reasons forcing us to - for example, lack of reparse points in NTFS on Windows NT 4.0 prevented us offering table space support, and for some time we had no Win32 port of libuuid, so couldn't offer the UUID contrib module. There are no such problems with this project that I can see - at least that cannot be overcome with a little thought and appropriate tool selection. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] new full vacuum doesn't work
Pavel Stehule pavel.steh...@gmail.com wrote:
I am testing vacuum changes, and I found some strange behave:
Did you need SET (fillfactor=100) before vACUUM FULL?
=# select * from pgstattuple('pgbench_accounts');
-[ RECORD 1 ]--+---
table_len | 1365336064
tuple_count| 100
tuple_len | 12100
tuple_percent | 8.86
dead_tuple_count | 0
dead_tuple_len | 0
dead_tuple_percent | 0
free_space | 1228669388
free_percent | 89.99
=# ALTER TABLE pgbench_accounts SET (fillfactor=100);
ALTER TABLE
=# vacuum full pgbench_accounts;
VACUUM
=# select * from pgstattuple('pgbench_accounts');
-[ RECORD 1 ]--+--
table_len | 134299648
tuple_count| 100
tuple_len | 12800
tuple_percent | 95.31
dead_tuple_count | 0
dead_tuple_len | 0
dead_tuple_percent | 0
free_space | 1840616
free_percent | 1.37
Regards,
---
Takahiro Itagaki
NTT Open Source Software Center
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] new full vacuum doesn't work
2010/1/8 Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp:
Pavel Stehule pavel.steh...@gmail.com wrote:
I am testing vacuum changes, and I found some strange behave:
Did you need SET (fillfactor=100) before vACUUM FULL?
no, I tested it and with FILLFACTOR 100 VACUUM FULL is successful.
Personally I thing, so this behave is bad. Or there is wrong default
fillfactor 0.
Regards
Pavel Stehule
=# select * from pgstattuple('pgbench_accounts');
-[ RECORD 1 ]--+---
table_len | 1365336064
tuple_count | 100
tuple_len | 12100
tuple_percent | 8.86
dead_tuple_count | 0
dead_tuple_len | 0
dead_tuple_percent | 0
free_space | 1228669388
free_percent | 89.99
=# ALTER TABLE pgbench_accounts SET (fillfactor=100);
ALTER TABLE
=# vacuum full pgbench_accounts;
VACUUM
=# select * from pgstattuple('pgbench_accounts');
-[ RECORD 1 ]--+--
table_len | 134299648
tuple_count | 100
tuple_len | 12800
tuple_percent | 95.31
dead_tuple_count | 0
dead_tuple_len | 0
dead_tuple_percent | 0
free_space | 1840616
free_percent | 1.37
Regards,
---
Takahiro Itagaki
NTT Open Source Software Center
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] new full vacuum doesn't work
Pavel Stehule pavel.steh...@gmail.com wrote: Personally I thing, so this behave is bad. Or there is wrong default fillfactor 0. No, you used fillfactor=10 here: [pa...@nemesis src]$ /usr/local/pgsql/bin/pgbench -i -F 10 -s 10 test ~ Pgbench sets the table's fillfactor to 10. Regards, --- Takahiro Itagaki NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] new full vacuum doesn't work
2010/1/8 Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp: Pavel Stehule pavel.steh...@gmail.com wrote: Personally I thing, so this behave is bad. Or there is wrong default fillfactor 0. No, you used fillfactor=10 here: [pa...@nemesis src]$ /usr/local/pgsql/bin/pgbench -i -F 10 -s 10 test ~ Pgbench sets the table's fillfactor to 10. ok, my error thank you Pavel Regards, --- Takahiro Itagaki NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
On Fri, Jan 8, 2010 at 04:46, Alex Hunsaker bada...@gmail.com wrote:
On Thu, Jan 7, 2010 at 20:26, Tom Lane t...@sss.pgh.pa.us wrote:
Alex Hunsaker bada...@gmail.com writes:
We can either drop this in core (with a lot of #ifdef LINUX added)
Any thoughts on doing something like (in fork_process.c)
#ifdef LINUX
void oom_adjust()
{
...
}
#else
void oom_adjust() {}
#endif
So there is only one #ifdef? It still leaves the ugly calls to the
function...
Seems like a much better way, yes. Especially if we in the future want
to do this for more than one platform (if it becomes necessary).
or expect Linux packagers to carry it as a patch. Given that the
packagers would also have to modify their init scripts to go with,
the patch route is not unreasonable. Comments?
Id plus +1 for core. The problem certainly does not look to be going
away soon (if ever).
Yeah, I think core is better. It's not like it's enough code to cause
a huge maintenance problem, I think.
Do we need to make the value configurable? I'd certainly find it
interesting to set backends to say 5 or something like that, that
makes them less likely to be killed than any old oops opened too big
file in an editor-process, but still possible to kill if the system
is *really* running out of memory.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Serializable Isolation without blocking
On Thursday, January 7, 2010, Robert Haas robertmh...@gmail.com wrote: On Thu, Jan 7, 2010 at 2:40 PM, Greg Stark gsst...@mit.edu wrote: On Thu, Jan 7, 2010 at 11:08 AM, Markus Wanner mar...@bluegap.ch wrote: Row level locks are very fine grained, but those are spilled to disk in its current implementation. So those are an even worse fit for the needs of SIREAD. I think we're still talking past the issue. Predicate locks are not row level, nor page level, nor table level. They're not? They're just floating out in space somewhere? There are several possible ways to implement predicate locks, but every possible method I can think of involves attaching them at one of those places. well the one place you *cannot* attach them is on the tuples. because you need to new able to lock hypothetical new tuples which don't exist yet. yes the implementation could work by attach them to tables or indexes but it's representing an abstract concept which is just hanging out in space. And how do you do that without tying the implementation to specific details of how our planner, table scans, and index access methods work? You don't. You add additional methods to the index AMs to support predicate locks, just lilke we do when we want them to support anything else (bitmap index scans, index-only scans, etc.). ...Robert -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5alpha3 hot standby crash report (DatabasePath related?)
On Fri, 2010-01-08 at 01:26 +, Simon Riggs wrote:
I'll test and commit tomorrow, since it's a fairly standalone problem
Fix attached, thanks for testing.
Works for me and I don't expect it to fail on Solaris, since the root
cause of the failure has been addressed and a correctly designed test
executed.
I will wait a day for your confirmation and/or other's comments.
--
Simon Riggs www.2ndQuadrant.com
*** a/src/backend/access/transam/xact.c
--- b/src/backend/access/transam/xact.c
***
*** 4404,4423 xact_redo_commit(xl_xact_commit *xlrec, TransactionId xid, XLogRecPtr lsn)
* maintain the same order of invalidation then release locks
* as occurs in .
*/
! if (xlrec-nmsgs 0)
! {
! /*
! * Relcache init file invalidation requires processing both
! * before and after we send the SI messages. See AtEOXact_Inval()
! */
! if (XactCompletionRelcacheInitFileInval(xlrec))
! RelationCacheInitFileInvalidate(true);
!
! SendSharedInvalidMessages(inval_msgs, xlrec-nmsgs);
!
! if (XactCompletionRelcacheInitFileInval(xlrec))
! RelationCacheInitFileInvalidate(false);
! }
/*
* Release locks, if any. We do this for both two phase and normal
--- 4404,4411
* maintain the same order of invalidation then release locks
* as occurs in .
*/
! ProcessCommittedInvalidationMessages(inval_msgs, xlrec-nmsgs,
! XactCompletionRelcacheInitFileInval(xlrec));
/*
* Release locks, if any. We do this for both two phase and normal
*** a/src/backend/utils/cache/inval.c
--- b/src/backend/utils/cache/inval.c
***
*** 89,94
--- 89,95
#include access/twophase_rmgr.h
#include access/xact.h
#include catalog/catalog.h
+ #include catalog/pg_tablespace.h
#include miscadmin.h
#include storage/sinval.h
#include storage/smgr.h
***
*** 871,876 xactGetCommittedInvalidationMessages(SharedInvalidationMessage **msgs,
--- 872,981
return numSharedInvalidMessagesArray;
}
+ #define RecoveryRelationCacheInitFileInvalidate(dbo, tbo, tf) \
+ { \
+ DatabasePath = GetDatabasePath(dbo, tbo); \
+ elog(trace_recovery(DEBUG4), removing relcache init file in %s, DatabasePath); \
+ RelationCacheInitFileInvalidate(tf); \
+ pfree(DatabasePath); \
+ }
+
+ /*
+ * ProcessCommittedInvalidationMessages is executed by xact_redo_commit()
+ * to process invalidation messages added to commit records.
+ *
+ * If we have to invalidate the relcache init file we need to extract
+ * the database id from each message so we can correctly locate the database
+ * path and so remove that database's init file. We note that the relcache
+ * only contains entries for catalog tables from a single database, or
+ * shared relations. There are smgr invalidations that reference other
+ * databases but they never cause relcache file invalidations.
+ * So we need not scan pg_database to discover tablespace oids, since
+ * we only have either global or default tablespaces. Just lucky, I guess.
+ *
+ * Relcache init file invalidation requires processing both
+ * before and after we send the SI messages. See AtEOXact_Inval()
+ *
+ * We deliberately avoid SetDatabasePath() since it is intended to be used
+ * only once by normal backends, so we just use DatabasePath directly and
+ * immediately pfree after use.
+ */
+ void
+ ProcessCommittedInvalidationMessages(SharedInvalidationMessage *msgs,
+ int nmsgs, bool RelcacheInitFileInval)
+ {
+ Oid dboid = 0;
+ bool invalidate_global = false;
+
+ if (nmsgs 0)
+ elog(trace_recovery(DEBUG4), replaying commit with %d messages%s, nmsgs,
+ (RelcacheInitFileInval ? and relcache file invalidation : ));
+ else
+ return;
+
+ if (RelcacheInitFileInval)
+ {
+ int i;
+
+ /*
+ * Check messages to record dboid
+ */
+ for (i = 0; i nmsgs; i++)
+ {
+ SharedInvalidationMessage *inval_msg = (msgs[i]);
+ Oid loop_dboid = 0;
+
+ /*
+ * Extract the database Oid from the message
+ */
+ if (inval_msg-id = 0)
+ loop_dboid = inval_msg-cc.dbId;
+ else if (inval_msg-id == SHAREDINVALRELCACHE_ID)
+ loop_dboid = inval_msg-rc.dbId;
+ else
+ {
+ /*
+ * Invalidation message is a SHAREDINVALSMGR_ID
+ * which never cause relcache file invalidation,
+ * so we ignore them, whichever database they're for.
+ */
+ }
+
+ if (loop_dboid == 0)
+ invalidate_global = true;
+ else
+ {
+ Assert(dboid == 0 || dboid == loop_dboid);
+ dboid = loop_dboid;
+ }
+ }
+
+ /*
+ * If shared, dboid will be the global tablespace, otherwise it will
+ * be a shared catalog relation in the default tablespace.
+ */
+ if (invalidate_global)
+ RecoveryRelationCacheInitFileInvalidate(0, GLOBALTABLESPACE_OID, true);
+
+ if (dboid != 0)
+ RecoveryRelationCacheInitFileInvalidate(dboid, DEFAULTTABLESPACE_OID, true);
+ }
+
+
Re: [HACKERS] ACK from walreceiver to walsender
On Fri, Jan 8, 2010 at 6:39 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: There's no guarantee walreceiver will read the 'X' before trying to write() to the socket, so we can't rely on that to determine whether to suppress the could not send data to client message. s/walreceiver/walsender? We could try to read() from the socket after the write() has failed, to see if there's an 'X' message pending. Not sure it's worth it. I think we would have to put the socket into non-blocking mode before the read(), to avoid blocking if the write() failed for some other reason. Or select() to see if there's incoming data. I'm inclined to just not bother.. Umm.. if no action is taken, walsender process would remain until it tries to write to the socket in that case. Is this OK? I think that this is more problematic rather than output of the could not send data to client message. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Initial refactoring of plperl.c - updated
On Mon, Jan 04, 2010 at 06:38:03PM -0500, Andrew Dunstan wrote:
Andrew Dunstan wrote:
Yes. I believe the test is highlighting an existing problem: that plperl
function in non-PG_UTF8 databases can't use regular expressions that
require unicode character meta-data.
Either the (GetDatabaseEncoding() == PG_UTF8) test in plperl_safe_init()
should be removed, so the utf8fix function is always called, or the
test should be removed (or hacked to only apply to PG_UTF8 databases).
I tried forcing the test, but it doesn't seem to work, possibly
because in the case that the db is not utf8 we aren't forcing
argument strings to UTF8 :-(
I think we might need to remove the test from the patch.
I have not been able to come up with a fix for this - the whole
thing seems very fragile. I'm going to commit what remains of this
patch, but not add the extra regression test. I'll add a TODO to
allow plperl to do utf8 operations in non-utf8 databases.
I see you've not commited it yet, so to help out I've attached
a new diff, over the current CVS head, with two minor changes:
- Removed the test, as noted above.
- Optimized pg_verifymbstr calls to avoid unneeded strlen()s.
This should apply cleanly to cvs, saving you the need to resolve the
conflicts caused by the recent pg_verifymbstr patch.
I'll add it to the commitfest once it reaches the archives.
Tim.
diff --git a/doc/src/sgml/plperl.sgml b/doc/src/sgml/plperl.sgml
index 7eebfba..37114bd 100644
*** a/doc/src/sgml/plperl.sgml
--- b/doc/src/sgml/plperl.sgml
***
*** 14,20
para
PL/Perl is a loadable procedural language that enables you to write
productnamePostgreSQL/productname functions in the
!ulink url=http://www.perl.com;Perl programming language/ulink.
/para
para
--- 14,20
para
PL/Perl is a loadable procedural language that enables you to write
productnamePostgreSQL/productname functions in the
!ulink url=http://www.perl.org;Perl programming language/ulink.
/para
para
*** SELECT * FROM perl_set();
*** 313,319
use strict;
/programlisting
in the function body. But this only works in applicationPL/PerlU/
!functions, since literaluse/ is not a trusted operation. In
applicationPL/Perl/ functions you can instead do:
programlisting
BEGIN { strict-import(); }
--- 313,320
use strict;
/programlisting
in the function body. But this only works in applicationPL/PerlU/
!functions, since the literaluse/ triggers a literalrequire/
!which is not a trusted operation. In
applicationPL/Perl/ functions you can instead do:
programlisting
BEGIN { strict-import(); }
diff --git a/src/pl/plperl/GNUmakefile b/src/pl/plperl/GNUmakefile
index a3c3495..8989b14 100644
*** a/src/pl/plperl/GNUmakefile
--- b/src/pl/plperl/GNUmakefile
*** PSQLDIR = $(bindir)
*** 45,50
--- 45,55
include $(top_srcdir)/src/Makefile.shlib
+ plperl.o: perlchunks.h
+
+ perlchunks.h: plc_*.pl
+ $(PERL) text2macro.pl --strip='^(\#.*|\s*)$$' plc_*.pl perlchunks.htmp
+ mv perlchunks.htmp perlchunks.h
all: all-lib
*** submake:
*** 65,71
$(MAKE) -C $(top_builddir)/src/test/regress pg_regress$(X)
clean distclean maintainer-clean: clean-lib
! rm -f SPI.c $(OBJS)
rm -rf results
rm -f regression.diffs regression.out
--- 70,76
$(MAKE) -C $(top_builddir)/src/test/regress pg_regress$(X)
clean distclean maintainer-clean: clean-lib
! rm -f SPI.c $(OBJS) perlchunks.htmp perlchunks.h
rm -rf results
rm -f regression.diffs regression.out
diff --git a/src/pl/plperl/plc_perlboot.pl b/src/pl/plperl/plc_perlboot.pl
index ...d2d5518 .
*** a/src/pl/plperl/plc_perlboot.pl
--- b/src/pl/plperl/plc_perlboot.pl
***
*** 0
--- 1,50
+ SPI::bootstrap();
+ use vars qw(%_SHARED);
+
+ sub ::plperl_warn {
+ (my $msg = shift) =~ s/\(eval \d+\) //g;
+ elog(NOTICE, $msg);
+ }
+ $SIG{__WARN__} = \::plperl_warn;
+
+ sub ::plperl_die {
+ (my $msg = shift) =~ s/\(eval \d+\) //g;
+ die $msg;
+ }
+ $SIG{__DIE__} = \::plperl_die;
+
+ sub ::mkunsafefunc {
+ my $ret = eval(qq[ sub { $_[0] $_[1] } ]);
+ $@ =~ s/\(eval \d+\) //g if $@;
+ return $ret;
+ }
+
+ use strict;
+
+ sub ::mk_strict_unsafefunc {
+ my $ret = eval(qq[ sub { use strict; $_[0] $_[1] } ]);
+ $@ =~ s/\(eval \d+\) //g if $@;
+ return $ret;
+ }
+
+ sub ::_plperl_to_pg_array {
+ my $arg = shift;
+ ref $arg eq 'ARRAY' || return $arg;
+ my $res = '';
+ my $first = 1;
+ foreach my $elem (@$arg) {
+ $res .= ', ' unless $first; $first = undef;
+ if (ref $elem) {
+ $res .= _plperl_to_pg_array($elem);
+ }
+ elsif (defined($elem)) {
+ my $str = qq($elem);
+ $str =~ s/([\\\])/\\$1/g;
+ $res .= qq(\$str\);
+ }
+ else {
+ $res .= 'NULL' ;
+ }
+ }
+ return qq({$res});
+ }
diff --git a/src/pl/plperl/plc_safe_bad.pl
Re: [HACKERS] RFC: PostgreSQL Add-On Network
Dave Page wrote: The only reason we ever offer different functionality on different platforms is when there are external reasons forcing us to - for example, lack of reparse points in NTFS on Windows NT 4.0 prevented us offering table space support, and for some time we had no Win32 port of libuuid, so couldn't offer the UUID contrib module. There are no such problems with this project that I can see - at least that cannot be overcome with a little thought and appropriate tool selection. Dave, Windows came late to the buildfarm. According to the CVS log, the buildfarm client was first checked in in Sept 2004, got initial Mingw support in Jan 2005 and MSVC support in March 2007, when we finally got some of the tools sorted out. I have long spoken against making Windows a second class citizen. But I don't think David is going to do that (and I'll hound him if he does). But that doesn't mean it has to be fully supported from day one. Personally, I'd like to see us with a build service for such modules (c.f. OpenSuse's build service). cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Streaming replication and triggering failover
On Fri, Jan 8, 2010 at 7:41 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Thinking more clearly, my comment above about the trigger file logic being backwards was bollocks; if the master is shut down, standby waits for the trigger file to appear, not to go away. And creating the trigger file during replication causes it to finish, and failover to happen. Nevertheless, let's make the default no failover if no trigger file location is configured, and remove the notion that normal shutdown of master stops recovery. You dropped CheckForStandbyTrigger() called at the end of recovery. I think that this would be problem when an invalid record is found before we reaches a streaming recovery state. The standby would be out-of-control of the clusterware, and be brought up. Which might cause a split-brain syndrome. We should need something to prevent such unexpected activation? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Streaming replication and triggering failover
Fujii Masao wrote: You dropped CheckForStandbyTrigger() called at the end of recovery. I think that this would be problem when an invalid record is found before we reaches a streaming recovery state. The standby would be out-of-control of the clusterware, and be brought up. Which might cause a split-brain syndrome. We should need something to prevent such unexpected activation? I modified ReadRecord to PANIC if an invalid record is found during streaming recovery. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch: Allow substring/replace() to get/set bit values
Leonardo F wrote: What we can do in the back branches is make the code treat any negative value as meaning two-arg form. To throw an error we'd need to refactor the pg_proc representation ... I was going to fix that myself, but I think it has just been done. How can I keep up with who's doing what? Read this list and pgsql-committers. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ACK from walreceiver to walsender
Fujii Masao wrote: On Fri, Jan 8, 2010 at 6:39 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: There's no guarantee walreceiver will read the 'X' before trying to write() to the socket, so we can't rely on that to determine whether to suppress the could not send data to client message. s/walreceiver/walsender? Right. We could try to read() from the socket after the write() has failed, to see if there's an 'X' message pending. Not sure it's worth it. I think we would have to put the socket into non-blocking mode before the read(), to avoid blocking if the write() failed for some other reason. Or select() to see if there's incoming data. I'm inclined to just not bother.. Umm.. if no action is taken, walsender process would remain until it tries to write to the socket in that case. Is this OK? Oh, I think we need to fix that, I'm thinking of doing a select() in the loop to check that the socket hasn't been closed yet. I meant we don't need to try reading the 'X' to tell apart e.g a network problem from a standby that's shut down cleanly. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why doesn't query_tree_walker examine the intoClause field?
Nikhil Sontakke nikhil.sonta...@enterprisedb.com writes: I am kinda puzzled as to why the query_tree_walker() function does not examine the intoClause field? Is there any point to it? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Streaming replication and triggering failover
On Fri, Jan 8, 2010 at 10:31 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Fujii Masao wrote: You dropped CheckForStandbyTrigger() called at the end of recovery. I think that this would be problem when an invalid record is found before we reaches a streaming recovery state. The standby would be out-of-control of the clusterware, and be brought up. Which might cause a split-brain syndrome. We should need something to prevent such unexpected activation? I modified ReadRecord to PANIC if an invalid record is found during streaming recovery. Oh, sorry. It was my misunderstanding :( Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] pg.dropped
(continued from -general) W dniu 7 stycznia 2010 22:31 użytkownik Greg Smith g...@2ndquadrant.comnapisał: Filip Rembiałkowski wrote: After dropping a column from table, there is still entry in pg_attribute fi...@la_dev=# select * from pg_attribute where attrelid = (select oid from pg_class where relname='thetable') order by attnum desc limit 1; -[ RECORD 1 ]-+-- attrelid | 4753849 attname | pg.dropped.69 ... attisdropped | t See that last part? That's what happens when you drop a table--attisdropped is set to true. The server can't just delete the pg_attribute entry altogether for various internal reasons, this is what it does instead. When should server delete this row? In my case it looks like it's never deleted (it lasts even server restart). And of course this makes my INSERT not working... There's obviously something wrong here, but the fact that the pg_attribute entry is still there (but marked dropped) is a not a direct cause of your problem. Thanks, I get it. -- Filip Rembiałkowski JID,mailto:filip.rembialkow...@gmail.com http://filip.rembialkowski.net/
Re: [HACKERS] ACK from walreceiver to walsender
On Fri, Jan 8, 2010 at 10:35 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Oh, I think we need to fix that, I'm thinking of doing a select() in the loop to check that the socket hasn't been closed yet. I meant we don't need to try reading the 'X' to tell apart e.g a network problem from a standby that's shut down cleanly. Okey. Sorry for noise. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
Magnus Hagander wrote: On Fri, Jan 8, 2010 at 05:22, Ron Mayer rm...@cheapcomplexdevices.com wrote: David E. Wheeler wrote: On Jan 7, 2010, at 1:31 PM, Dave Page wrote: No, I'm suggesting the mechanism needs to support source and binary distribution. For most *nix users, source will be fine. For Windows binaries are required. I would love to follow what Strawberry Perl has done to solve this problem. In 2.0. +1. They did a nice job. For those of us who have no idea what Strawberry Perl did (other than not shipping Microsoft compatible libraries, and is thus useless for PostgreSQL), could someone explain it? As far as I can tell they shipped the minimal set of tools that they needed to build extensions rather than distribute binaries. http://en.wikipedia.org/wiki/Strawberry_Perl I don't know the details, but it works smoothly for them. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
On Fri, Jan 8, 2010 at 15:14, Ron Mayer rm...@cheapcomplexdevices.com wrote: Magnus Hagander wrote: On Fri, Jan 8, 2010 at 05:22, Ron Mayer rm...@cheapcomplexdevices.com wrote: David E. Wheeler wrote: On Jan 7, 2010, at 1:31 PM, Dave Page wrote: No, I'm suggesting the mechanism needs to support source and binary distribution. For most *nix users, source will be fine. For Windows binaries are required. I would love to follow what Strawberry Perl has done to solve this problem. In 2.0. +1. They did a nice job. For those of us who have no idea what Strawberry Perl did (other than not shipping Microsoft compatible libraries, and is thus useless for PostgreSQL), could someone explain it? As far as I can tell they shipped the minimal set of tools that they needed to build extensions rather than distribute binaries. http://en.wikipedia.org/wiki/Strawberry_Perl I don't know the details, but it works smoothly for them. Right. They are msys compatible, not Microsoft compatible. And msys uses non-Windows-standard format of the import libraries. and the import libraries are needed for people to link to the interpreter, like pl/perl. Also a quick re-check shows that there appears to be no 64-bit version available. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
Magnus Hagander mag...@hagander.net writes: Do we need to make the value configurable? I'd certainly find it interesting to set backends to say 5 or something like that, that makes them less likely to be killed than any old oops opened too big file in an editor-process, but still possible to kill if the system is *really* running out of memory. I don't want to go to the trouble of creating (and documenting) a configure option for this. Much less a GUC ;-) What I suggest is that we do something like #ifdef LINUX_OOM_ADJ ... fprintf(oom, %d\n, LINUX_OOM_ADJ); ... #endif Then, somebody who wants the feature would build with, say, -DLINUX_OOM_ADJ=0 or another value if they want that. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Also update ChangerLog file.
Michael Meskes wrote: Log Message: --- Also update ChangerLog file. Hmm not sure I find that commit message really helpful - but is it actually of any use to maintain a Changelog file specifically for ECPG? Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Serializable implementation
Jeff Davis pg...@j-davis.com wrote:
On Thu, 2010-01-07 at 21:02 -0500, Robert Haas wrote:
Hmm. Why would we use a GUC for this instead of an additional
option to BEGIN TRANSACTION?
I'm with you. I feel pretty strongly that we should not have
behavior-changing GUCs.
OK. I actually thought this might be useful for testing, since we
could run a test with SI and SSI without changing any code in the
test -- change the conf file, reload, and run again. I guess,
though, that it shouldn't be too bad to find some other way to
control that, and adding a GUC for development that we rip out at
the end would be a bit silly.
I guess the question is whether we need the GUC to support people
who are asking for serializable transaction isolation but wouldn't
be prepared to deal with an increase in serialization failures from
actually *getting* serializable isolation.
BEGIN TRANSACTION ISOLATION LEVEL {READ COMMITTED | SNAPSHOT |
SERIALIZABLE}
With our current levels being the first two of those.
Or is that a bad idea?
We already have REPEATABLE READ which already has the same
semantics as SNAPSHOT would have, so I don't think we need to
introduce a new one.
Agreed. Snapshot should clearly continue to map to REPEATABLE READ,
so I see no reason to invent a nonstandard alias for it.
I think the only thing we need to do is suggest that people start
using REPEATABLE READ if they want snapshot isolation. That will
give us more freedom to change SERIALIZABLE to be true
serializability in 8.6.
Now you're being much more optimistic than I. I think that a year
from now we'll be working to get performance to an acceptable range,
and at the point where I feel it's usable for us there will still be
months of work to address concerns which will be raised only when it
seems that an actual commit might be imminent. (I mean, does that
ever *not* happen on a big patch?) That puts this, probably in the
third major release after 8.4, whatever number that gets.
Other than the optimistic release number, however, I think your
point is on target. It could at least help justify making fully
serializable transactions the default for the GUC, and maybe justify
not even having the GUC.
If we need a GUC that aliases SERIALIZABLE to REPEATABLE READ for
backwards compatibility (after we change SERIALIZABLE) for a short
period of time, that sounds reasonable. Even that may not be
necessary though because it shouldn't really break any promises
that we made in the documentation (that I'm aware of).
I agree that it would break no promises; but it could break existing
applications. It would only break applications which depended on
certain assumptions -- namely that the serializable level would
specifically be snapshot isolation and based on that it would be
possible to analyze all combinations of concurrent transactions and
accurately predict which ones might roll back with a serialization
errors, and it is OK to code specifically for such errors only on
the vulnerable transactions.
The only evidence I've seen that such applications exist is the
pushback from some on this list regarding the unpredictability of
which transactions could roll back on serialization failures -- that
it could happen on transactions for which the programmer didn't
handle it. I've assumed that this objection was based on some
knowledge that such coding was in production somewhere. If it is,
we either need the GUC or we rationalize its absence on the basis of
having told people for two years to use REPEATABLE READ if they want
bare snapshot behavior.
Opinions?
-Kevin
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
Alex Hunsaker wrote:
On Thu, Jan 7, 2010 at 20:26, Tom Lane t...@sss.pgh.pa.us wrote:
Alex Hunsaker bada...@gmail.com writes:
We can either drop this in core (with a lot of #ifdef LINUX added)
Any thoughts on doing something like (in fork_process.c)
#ifdef LINUX
void oom_adjust()
{
...
}
#else
void oom_adjust() {}
#endif
So there is only one #ifdef? It still leaves the ugly calls to the
function...
The usual solution for this kind of thing is:
#ifdef LINUX
#define OOM_ADJUST oom_adjust()
#else
#define OOM_ADJUST do {} while (0)
#endif
so there is no call or dummy function and you reference it in the code
as:
OOM_ADJUST;
--
Bruce Momjian br...@momjian.ushttp://momjian.us
EnterpriseDB http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Serializable Isolation without blocking
On Fri, Jan 8, 2010 at 7:34 AM, Greg Stark gsst...@mit.edu wrote: On Thursday, January 7, 2010, Robert Haas robertmh...@gmail.com wrote: On Thu, Jan 7, 2010 at 2:40 PM, Greg Stark gsst...@mit.edu wrote: On Thu, Jan 7, 2010 at 11:08 AM, Markus Wanner mar...@bluegap.ch wrote: Row level locks are very fine grained, but those are spilled to disk in its current implementation. So those are an even worse fit for the needs of SIREAD. I think we're still talking past the issue. Predicate locks are not row level, nor page level, nor table level. They're not? They're just floating out in space somewhere? There are several possible ways to implement predicate locks, but every possible method I can think of involves attaching them at one of those places. well the one place you *cannot* attach them is on the tuples. because you need to new able to lock hypothetical new tuples which don't exist yet. Well, index tuples maybe, for next-key locking... yes the implementation could work by attach them to tables or indexes but it's representing an abstract concept which is just hanging out in space. Yeah, I understand that the concept is abstract, but I'm not sure it's bad to be talking about other kinds of locks because that's how it will be implemented. I think a lot of this is going to end up falling on the index AMs - we could lock have locks on GIST pages, next-key locks on btree tuples. We'll ask each index AM if it can help with the predicate that we're trying to lock against and if they all say no then we lock the whole table... or that's what I'm thinking, anyway. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Serializable implementation
On Fri, Jan 8, 2010 at 9:46 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Opinions? I think anything you decide about how to invoke the different isolation levels will be easy to change later to meet whatever the consensus of the community is at that time. I wouldn't spend any time or energy on it now. For purposes of your prototype patch, using REPEATABLE READ for the current serializable and SERIALIZABLE for the new behavior will be plenty good enough. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Add .gitignore files to CVS?
Magnus Hagander escribió:
On Fri, Jan 8, 2010 at 00:44, Alex Hunsaker bada...@gmail.com wrote:
On Thu, Jan 7, 2010 at 15:16, Tim Bunce tim.bu...@pobox.com wrote:
Is there any reason not to add .gitignore files into the repository?
They'll make no difference to those who don't use git, but be very
helpful to, and maintained by, those who do.
Since it seems we don't want them in CVS, maybe just add it to the git
mirror? I don't know that we want the git mirror to have
commits/files that CVS does not. *shrug* Thoughts people?
Definite -1. We want the git repository to be (as much as possible)
identical to the CVS one.
You can always create your own branch with just the .gitignore files
and merge that into whatever you're working on :)
Do .gitignore files have the same format as .cvsignore? If that's the
case then it's simply a matter of a find /source -name .cvsignore -exec
cp {} .gitignore \; or similar, isn't it? Doesn't sound like something
anybody should sweat over.
--
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Serializable Isolation without blocking
Markus Wanner mar...@bluegap.ch wrote: Greg Stark wrote: That's about predicate locks. I've been talking about SIREAD, which is a different thing (and which I don't consider to be a lock). The SIREAD thingie certainly doesn't help implement predicate locks. And predicate locking isn't necessarily sufficient to implement truly serializable isolation. SIREAD locks need to be acquired according to the exact same rules as normal read locks in predicate locking schemes. We're just using a lock level where the set of locks which conflict with it is empty. ;-) Predicate locking is clearly necessary and clearly not sufficient to implement serializable isolation. Was some part of the Cahill paper unclear on that? That's the hard part. How do you represent such a lock in a way that I can efficiently find it and check for the conflict when it comes time for me to do my insert. As it's not really a lock, but rather a mark or a tag, SIREAD may or may not be implemented atop existing or non-existent locking structures, IMO. Well, the development plan calls for it to start there. Whether a better way is found during optimization is anybody's guess at this point. I've made my points about implementing SIREAD atop table level or row level locking structures. With (non-existent) predicate locking, I'm still unsure. It might help to implement SIREAD atop such a predicate as well. Predicate tagging? It seems both respectful and less confusing to adopt the terminology of those who developed SSI techniques. I could invent new terms for vulnerable edges, dangerous structures, pivots and such which are used in the relevant literature. But I won't. It would just serve to confuse those who have spent the time to read and understand the literature. I'm not quite sure I followed what you were getting at with the (non-existent) predicate locking phrase -- was that just an acknowledgment that it is exactly what is under development and, as such, doesn't yet exist; or were you getting at something else? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Serializable Isolation without blocking
Markus Wanner mar...@bluegap.ch wrote: Kevin Grittner wrote: As I understand it, Greg's line of thinking is that we should use a technique which has never proven practical on a large scale: matching database changes against a list of predicate lock expressions. I find that approach to predicate locking pretty interesting. Sure, I find it interesting, too. Just not practical. unlike others, it scales with the number of concurrently held locks. Times the number of checks for conflicting locks. So, O(N^2). No thanks. Now if there is a breakthrough in that technology which allows it to compete favorably with techniques which model the logical predicates against database structures, I'm all ears. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Serializable Isolation without blocking
Greg Stark gsst...@mit.edu wrote: well the one place you *cannot* attach them is on the tuples. The predicate locking schemes I've been reading about do attach locks to tuples, as *part* of a complete strategy. you need to new able to lock hypothetical new tuples which don't exist yet. That, too. Which is where other granularities and objects come in, as you later noted. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
Hey Andrew On Fri, Jan 8, 2010 at 12:57 PM, Andrew Dunstan and...@dunslane.net wrote: Windows came late to the buildfarm. According to the CVS log, the buildfarm client was first checked in in Sept 2004, got initial Mingw support in Jan 2005 and MSVC support in March 2007, when we finally got some of the tools sorted out. Right - but the buildfarm isn't a feature being offered to end users. I have long spoken against making Windows a second class citizen. But I don't think David is going to do that (and I'll hound him if he does). But that doesn't mean it has to be fully supported from day one. I'm not saying it should be supported from day 1, but I think the initial plan will make it very difficult to add Windows support later without a great deal of rewriting/redesign. It's lack of forward planning I was objecting to. Personally, I'd like to see us with a build service for such modules (c.f. OpenSuse's build service). +1 -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
On Fri, Jan 8, 2010 at 07:53, Bruce Momjian br...@momjian.us wrote:
Alex Hunsaker wrote:
On Thu, Jan 7, 2010 at 20:26, Tom Lane t...@sss.pgh.pa.us wrote:
Alex Hunsaker bada...@gmail.com writes:
The usual solution for this kind of thing is:
#ifdef LINUX
#define OOM_ADJUST oom_adjust()
#else
#define OOM_ADJUST do {} while (0)
#endif
so there is no call or dummy function and you reference it in the code
as:
Surely any compiler worth its salt would turn a call to an empty void
function into a noop? Then again maybe I just hate macros :)
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
On Fri, Jan 8, 2010 at 10:12 AM, Dave Page dp...@pgadmin.org wrote: I have long spoken against making Windows a second class citizen. But I don't think David is going to do that (and I'll hound him if he does). But that doesn't mean it has to be fully supported from day one. I'm not saying it should be supported from day 1, but I think the initial plan will make it very difficult to add Windows support later without a great deal of rewriting/redesign. It's lack of forward planning I was objecting to. I personally suspect that the client is not the most important part of this project. I think the value of CPAN is for searching, more than auto-installing. Personally, I never use the auto-install feature because I always want more control than you get that way. I just use the site to find possible modules and browse the docs, and then if I find something I like I check with I can pull it from the Red Hat repos with rpm, and if not I download it and look it over to see if it DWIW, and then if so I usually make a private SRPM for it and install from that. I'd be happy if we just had a good search-and-download site. That having been said, we should consider our filesystem layout carefully however to make sure that if we want to provide things like Windows installers in the future, we have a clean way to do that. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Serializable Isolation without blocking
I had this flagged as needing a response, but it fell through the cracks yesterday. Apologies for the delayed response. Markus Wanner mar...@bluegap.ch wrote: I'm not clear if Kevin plans to go down to tuple level locking with granularity of the SIREAD thing. Eventually, where possible, subject to escalation. (I don't like calling it a lock, because it actually isn't. Let's better call it a hint or a mark). You have a point, but as I said in another email, I'm reluctant to invent my own terminology which conflicts with the literature. So I'll remain conventional on this. We certainly cannot store all of the SIREAD hint information within the tuple header, as there may be multiple transactions having marked the same tuple SIREAD, but we don't want to spend lots of bits for SSI there (rather no single bit). Absolutely. It's easy to see that the SIREAD hint information doesn't fit in shared memory for large enough tables. The only way I can imagine tuple-level SIREAD locking half-ways working is by using that for transactions reading only a few tuples and fall back to some coarser grained locking strategy after a certain limit (per transaction). Exactly. I'm not sure if I understand this correctly, but I don't think you need to lock tuples that don't exist, at least not with SIREAD. The Cahill paper covers this under Detecting Phantoms and proposes to use plain predicate locking to cover tuple level granularity AFAIUI. The proposed SIREAD hint seems to be an optimization that only works for existing tuples. I don't find it hard to believe that it performs better than a general purpose predicate locking strategy. (Especially for test cases with short transactions, i.e. only few SIREAD locks per txn). When the Cahill paper talks about predicate locking, it *is* talking about what to lock with SIREAD locks, at least as I read it. If you see something to indicate otherwise, please share. (If I'm off base on understanding any of this, I'd sure like to get straightened out as soon as possible.) (It's interesting that with database page level granularity, he states that predicate locking would not be necessary. Instead any page can be locked at any time. For this to work, according to my reasoning, you'd have to know in advance on which page potentially accessible (but not yet visible) new tuples would end up. This is close to impossible for Postgres, however, it seems to work for Berkeley DB). It was the way Sybase worked for ages, too. If you consider that you're locking both the heap and index pages read, it becomes clear that in order to add a row which conflicts with a predicate, you have to modify a page which would have been read to check the predicate. How about storing the SIREAD info in shared memory and using dynamic granularity based on the conflict rate and available memory? *duck* Don't duck. That's the plan. I really wish I was better at communicating these things. :-( See lock.c and the associated README and see if you don't think they would fit the bill. As this seems to be an optimization of predicate locking, don't we need to implement that first? Exactly. That's the plan. Implement it in the simplest form, then optimize. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
On Fri, 2010-01-08 at 15:12 +, Dave Page wrote: Hey Andrew On Fri, Jan 8, 2010 at 12:57 PM, Andrew Dunstan and...@dunslane.net wrote: Windows came late to the buildfarm. According to the CVS log, the buildfarm client was first checked in in Sept 2004, got initial Mingw support in Jan 2005 and MSVC support in March 2007, when we finally got some of the tools sorted out. Right - but the buildfarm isn't a feature being offered to end users. Neither is this. These people are developers or DBAs after all. Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
On Fri, Jan 8, 2010 at 07:27, Tom Lane t...@sss.pgh.pa.us wrote:
Then, somebody who wants the feature would build with, say,
-DLINUX_OOM_ADJ=0
or another value if they want that.
Here is a stab at that.
It sets oom_adj for:
autovacuum workers
archivers (pgarch.c)
regular backends
Also it updates the contrib linux starup script to start under an oom_adj of -17
Comments?
*** a/contrib/start-scripts/linux
--- b/contrib/start-scripts/linux
***
*** 53,58 DAEMON=$prefix/bin/postmaster
--- 53,63
# What to use to shut down the postmaster
PGCTL=$prefix/bin/pg_ctl
+ # Adjust oom_adj on linux to avoid the postmaster from be killed
+ # note you probably want to compile postgres with -DLINUX_OOM_ADJ=0
+ # so that regular backends will be killed on oom
+ OOM_ADJ=-17
+
set -e
# Only start if we can find the postmaster.
***
*** 62,67 test -x $DAEMON || exit 0
--- 67,73
case $1 in
start)
echo -n Starting PostgreSQL:
+ echo $OOM_ADJ /proc/self/oom_adj
su - $PGUSER -c $DAEMON -D '$PGDATA' $PGLOG 21
echo ok
;;
*** a/src/backend/postmaster/autovacuum.c
--- b/src/backend/postmaster/autovacuum.c
***
*** 1403,1408 StartAutoVacWorker(void)
--- 1403,1411
/* Lose the postmaster's on-exit routines */
on_exit_reset();
+ /* allow us to be killed on oom */
+ oom_adjust();
+
AutoVacWorkerMain(0, NULL);
break;
#endif
*** a/src/backend/postmaster/fork_process.c
--- b/src/backend/postmaster/fork_process.c
***
*** 66,68 fork_process(void)
--- 66,97
}
#endif /* ! WIN32 */
+
+ #if defined(__linux__) defined(LINUX_OOM_ADJ)
+ /*
+ * By default linux really likes to kill the postmaster on oom.
+ * Because linux does not take SYSV shared mem into account it
+ * (almost) always will SIGKILL the postmaster on an oom event.
+ *
+ * In the event you started the postmaster under a low (negative)
+ * oom_adj. This will adjust regular backends, autovac and archivers
+ * to LINUX_OOM_ADJ on fork(). Allowing them to be killed in an oom
+ * event.
+ *
+ * Later we might add other OS oom type stuff here.
+ */
+ void
+ oom_adjust(void)
+ {
+ FILE *oom = fopen(/proc/self/oom_adj, w);
+
+ /* ignore errors we dont really care */
+ if (oom)
+ {
+ fprintf(oom, %d\n, LINUX_OOM_ADJ);
+ fclose(oom);
+ }
+ }
+ #else
+ void oom_adjust(void) { }
+ #endif
*** a/src/backend/postmaster/pgarch.c
--- b/src/backend/postmaster/pgarch.c
***
*** 170,175 pgarch_start(void)
--- 170,178
/* Drop our connection to postmaster's shared memory, as well */
PGSharedMemoryDetach();
+ /* allow us to be killed on oom */
+ oom_adjust();
+
PgArchiverMain(0, NULL);
break;
#endif
*** a/src/backend/postmaster/postmaster.c
--- b/src/backend/postmaster/postmaster.c
***
*** 3076,3081 BackendStartup(Port *port)
--- 3076,3084
/* Perform additional initialization and collect startup packet */
BackendInitialize(port);
+ /* allow us to be killed on oom */
+ oom_adjust();
+
/* And run the backend */
proc_exit(BackendRun(port));
}
*** a/src/include/postmaster/fork_process.h
--- b/src/include/postmaster/fork_process.h
***
*** 13,17
--- 13,18
#define FORK_PROCESS_H
extern pid_t fork_process(void);
+ extern void oom_adjust(void);
#endif /* FORK_PROCESS_H */
*** a/contrib/start-scripts/linux
--- b/contrib/start-scripts/linux
***
*** 53,58 DAEMON=$prefix/bin/postmaster
--- 53,63
# What to use to shut down the postmaster
PGCTL=$prefix/bin/pg_ctl
+ # Adjust oom_adj on linux to avoid the postmaster from be killed
+ # note you probably want to compile postgres with -DLINUX_OOM_ADJ=0
+ # so that regular backends will be killed on oom
+ OOM_ADJ=-17
+
set -e
# Only start if we can find the postmaster.
***
*** 62,67 test -x $DAEMON || exit 0
--- 67,73
case $1 in
start)
echo -n Starting PostgreSQL:
+ echo $OOM_ADJ /proc/self/oom_adj
su - $PGUSER -c $DAEMON -D '$PGDATA' $PGLOG 21
echo ok
;;
*** a/src/backend/postmaster/autovacuum.c
--- b/src/backend/postmaster/autovacuum.c
***
*** 1403,1408 StartAutoVacWorker(void)
--- 1403,1411
/* Lose the postmaster's on-exit routines */
on_exit_reset();
+ /* allow us to be killed on oom */
+ oom_adjust();
+
AutoVacWorkerMain(0, NULL);
break;
#endif
*** a/src/backend/postmaster/fork_process.c
--- b/src/backend/postmaster/fork_process.c
***
*** 66,68 fork_process(void)
--- 66,97
}
#endif /* ! WIN32 */
+
+ #if defined(__linux__) defined(LINUX_OOM_ADJ)
+ /*
+ * By default linux really likes to kill the postmaster on oom.
+ * Because linux does not take SYSV shared mem into
Re: [HACKERS] Patch: Allow substring/replace() to get/set bit values
Alvaro Herrera alvhe...@commandprompt.com writes: Leonardo F wrote: How can I keep up with who's doing what? Read this list and pgsql-committers. Or subscribe to the RSS feed from: http://git.postgresql.org/gitweb?p=postgresql.git;a=summary -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Serializable Isolation without blocking
Hi, Greg Stark wrote: well the one place you *cannot* attach them is on the tuples. because you need to new able to lock hypothetical new tuples which don't exist yet. Well, maybe attaching here is meant in a more general or theoretical sense. I think we all agree that adding them to the tuple header on disk is not an option. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] git help (was: Serializable Isolation without blocking)
Magnus Hagander mag...@hagander.net wrote: On Thu, Jan 7, 2010 at 19:08, Kevin Grittner Robert's advice being the last (and only) offered on the topic, I'm taking the silence as agreement and have dropped the request for a serializable repository and added one for /users/kgrittn/postgres instead. ... and i've approved it :-) Being new to git, I'm hoping someone can suggest a reasonable workflow for using this. (I've found no shortage of suggestions for possible workflows on the web, but am not clear what to pick or synthesize as best practice for this particular situation.) A little advice may prevent a lot of flailing and false starts, and would be much appreciated. I assume I want to create a serializable branch in this new repository and clone that on my desktop. I expect I'll be granting write permission on this repository to others, to facilitate teamwork on this, but perhaps there's a better way to organize that with git. What would the normal workflows be to: rebase the postgresql.git clone and the serializable branch on the server? rebase my working copy on my desktop from the serializable branch on the server? integrate work from my desktop to the serializable branch on the server when I've hit a state which seems worth sharing with the team? (i.e., it compiles and isn't going to break something someone else is working on) Or is the above framed in a way which shows that I'm not properly in the git mindset yet? If so, a hint at what I'm missing would be good. Thanks for any help getting git usage figured out. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
On Fri, Jan 8, 2010 at 3:56 PM, Joshua D. Drake j...@commandprompt.com wrote: On Fri, 2010-01-08 at 15:12 +, Dave Page wrote: Hey Andrew On Fri, Jan 8, 2010 at 12:57 PM, Andrew Dunstan and...@dunslane.net wrote: Windows came late to the buildfarm. According to the CVS log, the buildfarm client was first checked in in Sept 2004, got initial Mingw support in Jan 2005 and MSVC support in March 2007, when we finally got some of the tools sorted out. Right - but the buildfarm isn't a feature being offered to end users. Neither is this. These people are developers or DBAs after all. I know I'm a little slower than normal this week as I've had some sort of virus, but say what? I was kinda under the impression that developers/DBAs are end users of PostgreSQL. They make up a significant percentage of people that I know than install, configure and use it. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
On Fri, 2010-01-08 at 16:33 +, Dave Page wrote: On Fri, Jan 8, 2010 at 3:56 PM, Joshua D. Drake j...@commandprompt.com wrote: On Fri, 2010-01-08 at 15:12 +, Dave Page wrote: Hey Andrew On Fri, Jan 8, 2010 at 12:57 PM, Andrew Dunstan and...@dunslane.net wrote: Windows came late to the buildfarm. According to the CVS log, the buildfarm client was first checked in in Sept 2004, got initial Mingw support in Jan 2005 and MSVC support in March 2007, when we finally got some of the tools sorted out. Right - but the buildfarm isn't a feature being offered to end users. Neither is this. These people are developers or DBAs after all. I know I'm a little slower than normal this week as I've had some sort of virus, but say what? I was kinda under the impression that developers/DBAs are end users of PostgreSQL. They make up a significant percentage of people that I know than install, configure and use it. My point is... if they are a developer or DBA, they aren't going to have any trouble dealing with these issues. Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
Alex Hunsaker bada...@gmail.com writes: On Fri, Jan 8, 2010 at 07:27, Tom Lane t...@sss.pgh.pa.us wrote: Then, somebody who wants the feature would build with, say, Â Â Â Â -DLINUX_OOM_ADJ=0 or another value if they want that. Here is a stab at that. Anybody have an objection to this basic approach? I'm in a bit of a hurry to get something like this into the Fedora RPMs, so barring objections I'm going to review this, commit it into HEAD, and then make a back-ported patch I can use with 8.4 in Fedora. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Add .gitignore files to CVS?
On Fri, Jan 8, 2010 at 02:03, Magnus Hagander mag...@hagander.net wrote: You can always create your own branch with just the .gitignore files and merge that into whatever you're working on :) The only thing annoying about that is if you generate diffs ala git diff origin/master.. you get your .gitignore in it. What I do is have a .gitignore that is gitignored. That way its not committed, its on any branch i switch to or make and I don't accidentally commit it. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
On Fri, Jan 8, 2010 at 4:34 PM, Joshua D. Drake j...@commandprompt.com wrote: On Fri, 2010-01-08 at 16:33 +, Dave Page wrote: On Fri, Jan 8, 2010 at 3:56 PM, Joshua D. Drake j...@commandprompt.com wrote: On Fri, 2010-01-08 at 15:12 +, Dave Page wrote: Hey Andrew On Fri, Jan 8, 2010 at 12:57 PM, Andrew Dunstan and...@dunslane.net wrote: Windows came late to the buildfarm. According to the CVS log, the buildfarm client was first checked in in Sept 2004, got initial Mingw support in Jan 2005 and MSVC support in March 2007, when we finally got some of the tools sorted out. Right - but the buildfarm isn't a feature being offered to end users. Neither is this. These people are developers or DBAs after all. I know I'm a little slower than normal this week as I've had some sort of virus, but say what? I was kinda under the impression that developers/DBAs are end users of PostgreSQL. They make up a significant percentage of people that I know than install, configure and use it. My point is... if they are a developer or DBA, they aren't going to have any trouble dealing with these issues. What issues? Lack of support from PGAN, or having to download and install a complex build environment weighing in at over a gigabyte of downloads, just to install an add-on? -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
Robert Haas wrote: On Thu, Jan 7, 2010 at 9:11 PM, Andrew Dunstan and...@dunslane.net wrote: This strikes me as quite premature. Heiki just said he expects to have SR committed next week. Getting it committed is not what I'm worried about. What I'm concerned about is Tom's statement that he believes that HS is not close to being in a state where it is stable enough to go to beta, and the possibility that other large patches committed during this CommitFest might have similar issues. Looking at our existing schedule, I think it is unlikely we will even make a June 2010 release date for 8.5. Let me explain why --- here is the ideal schedule: Jan 15 start commitfest Feb 15 stop commitfest Apr 1 start beta Jun 1 release release candidate (RC) Jun 20 release 8.5 You can't move from commitfest to beta until all _known_ bugs are fixed/addressed, and you can't move from beta to RC using the same criteria. Of course we rarely have an ideal schedule --- we should expect unexpected problems. Now, I know people are going to say I am being over pessimistic, but history will bear out my analysis. I think we should anticipate a July or August release of 8.5. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
Tom Lane wrote: Alex Hunsaker bada...@gmail.com writes: On Fri, Jan 8, 2010 at 07:27, Tom Lane t...@sss.pgh.pa.us wrote: Then, somebody who wants the feature would build with, say, ?? ?? ?? ??-DLINUX_OOM_ADJ=0 or another value if they want that. Here is a stab at that. Anybody have an objection to this basic approach? I'm in a bit of a hurry to get something like this into the Fedora RPMs, so barring objections I'm going to review this, commit it into HEAD, and then make a back-ported patch I can use with 8.4 in Fedora. Go for it, but FYI, we need to udpate the our OOM documentation mention to reflect this change. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Serializable Isolation without blocking
Hi, Kevin Grittner wrote: SIREAD locks need to be acquired according to the exact same rules as normal read locks in predicate locking schemes. Understood. I didn't take that into account at first. Thanks for pointing it out. (Whatever normal read locks are...) We're just using a lock level where the set of locks which conflict with it is empty. ;-) Which kind of defeats the purpose of a lock. Anyway, that's a bike-shed discussion, so I might as well agree to calling it a lock. Well, the development plan calls for it to start there. Whether a better way is found during optimization is anybody's guess at this point. Okay, so you know my guess ;-) It seems both respectful and less confusing to adopt the terminology of those who developed SSI techniques. I could invent new terms for vulnerable edges, dangerous structures, pivots and such which are used in the relevant literature. But I won't. It would just serve to confuse those who have spent the time to read and understand the literature. I agree. My intention was to translate the literature terms to Postgres hacker slang. At least I had trouble to understand that SIREAD is a lock that doesn't actually block anything. I'm not quite sure I followed what you were getting at with the (non-existent) predicate locking phrase -- was that just an acknowledgment that it is exactly what is under development and, as such, doesn't yet exist; or were you getting at something else? SIREAD atop predicate locking serves detecting vulnerable edges (I hope I'm using that term correctly here) between newly inserted tuples and reads, right? I was trying to figure if it would make sense to use predicate locking (instead of table or row level locking) as well for detecting vulnerable edges between updated or deleted tuples and reads. At least you'd be free with its granularity (which cannot be said for row or table level locking, for obvious reasons). However, it certainly depends a lot on the implementation of predicate locking, which we don't have, yet, so that's all speculation... Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
Dave, * Dave Page (dp...@pgadmin.org) wrote: Right - but the buildfarm isn't a feature being offered to end users. And this network isn't a feature of the core code either, nor, do I believe, is it being designed in a way that would require an overhaul down the road to support Windows. To be honest, I really don't feel this compares to features in the core code to begin with- I feel like generally we don't like things which only work on a subset of platforms because it's a small incremental change, and typically just requires good programming practice, to make it work for all supported platforms. That isn't true for this. I'm not saying it should be supported from day 1, but I think the initial plan will make it very difficult to add Windows support later without a great deal of rewriting/redesign. It's lack of forward planning I was objecting to. I disagree with this, but in the end I don't think that it really matters. If all we have resources for to work on this is a Perl expect, then by golly let that be a dependency for the initial work. If there are other resources who are willing to write the initial version in C, or what have you, let them step up and offer. This isn't a zero-sum game here and I'm strongly against telling someone please don't work on this when it's something which will be of benefit to many users. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] damage control mode
Bruce Momjian br...@momjian.us wrote: here is the ideal schedule: Jan 15 start commitfest Feb 15 stop commitfest Apr 1 start beta Jun 1 release release candidate (RC) Jun 20 release 8.5 Of course we rarely have an ideal schedule So for a project which strives for an annual release, we have over six months from initial submission of a *small* patch until the release it goes in, if we meet the ideal schedule? Longer for large patches or if we slip from the ideal schedule? That sounds like there are still some opportunities for improvements in project management, but it's not *so* bad if development for the next release can overlap the tail end of that schedule. I'm not sure how much that is anticipated or encouraged, though. It also seems to call into question the wisdom of annual releases. If we had a two-year cycle which had three times as much in it, would that be an improvement, or not? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
* Magnus Hagander (mag...@hagander.net) wrote: Do we need to make the value configurable? I'd certainly find it interesting to set backends to say 5 or something like that, that makes them less likely to be killed than any old oops opened too big file in an editor-process, but still possible to kill if the system is *really* running out of memory. We do need to make it configurable, in at least the sense of being able to control if it's done or not. There are some environments where you won't be able to set it. Perhaps just handling failure gracefully would work, but I'd be happier if you could just disable it in the config file. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] true serializability and predicate locking
On Thu, Jan 7, 2010 at 9:28 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: All valid points. I could try to make counter-arguments, but in my view the only thing that really matters is how any such attempt performs in a realistic workload. If, when we get to the optimization phase, such a technique shows a performance improvement in benchmarks which we believe realistically model workloads we believe to be reasonable candidates to use serializable transactions, then I'll argue that the burden of proof is on anyone who thinks it's a bad idea in spite of that. If it doesn't show an improvement, I'll be the first to either try to refine it or toss it. Fair enough? My comment was in relation to the idea of representing the costs in the planner. I was a) saying you had to see how the implementation went before you try to come up with how to represent the costs and then b) speculating (hypocritically:) that you might have the direction of adjustment backward. From what I understand your first cut will just take full-table locks anyways so it won't matter what type of plan is used at all. Personally I can't see how that won't generate a serialization failure on basically every query on any moderately concurrent system but at least it would make an interesting test-bed for the SIREAD dependency detection logic. And I agree it's necessary code before we get into more fine-grained siread locks. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
Kevin Grittner kevin.gritt...@wicourts.gov wrote: Bruce Momjian br...@momjian.us wrote: Jan 15 start commitfest Jun 20 release 8.5 over six months OK, so over *five* months. Still -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
* Tom Lane (t...@sss.pgh.pa.us) wrote: I don't want to go to the trouble of creating (and documenting) a configure option for this. Much less a GUC ;-) Requiring a custom build to disable it would be horrible, in my view. Or, at best, just means that the packagers won't enable it, which obviously would be less than ideal. Sorry if it's a pain, but I think it needs to either be configurable or not done. As I said before, it definitely needs to handle failure gracefully, but I worry that even that won't be sufficient in some cases. Just thinking about how we run PG under VServers and Linux Containers and whatnot, we ran into some issues with OpenSSH trying to monkey with proc values and I'd really hate to run into the same issues with PG. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] Setting oom_adj on linux?
On Fri, Jan 8, 2010 at 10:07, Stephen Frost sfr...@snowman.net wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: I don't want to go to the trouble of creating (and documenting) a configure option for this. Much less a GUC ;-) Requiring a custom build to disable it would be horrible, in my view. Or, at best, just means that the packagers won't enable it, which obviously would be less than ideal. FWIW I agree. Sorry if it's a pain, but I think it needs to either be configurable or not done. As I said before, it definitely needs to handle failure gracefully, but I worry that even that won't be sufficient in some cases. Just thinking about how we run PG under VServers and Linux Containers and whatnot, we ran into some issues with OpenSSH trying to monkey with proc values and I'd really hate to run into the same issues with PG. As long as the VM/container you are running under wont kill postmaster for trying to access proc-- the patch I posted should work fine. It just ignores any error (I assumed you might be running in a chroot without proc or some such). -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
* Tom Lane (t...@sss.pgh.pa.us) wrote: Alex Hunsaker bada...@gmail.com writes: On Fri, Jan 8, 2010 at 07:27, Tom Lane t...@sss.pgh.pa.us wrote: Then, somebody who wants the feature would build with, say, -DLINUX_OOM_ADJ=0 or another value if they want that. Here is a stab at that. Anybody have an objection to this basic approach? I'm in a bit of a hurry to get something like this into the Fedora RPMs, so barring objections I'm going to review this, commit it into HEAD, and then make a back-ported patch I can use with 8.4 in Fedora. Whoah, I would caution against doing that without being very confident it won't break when installed under things like VServer, Linux Containers, SELinux configurations, etc, when back-porting it. I don't expect people would be too pleased to discover their nice, simple, no-expected-issues upgrade of a minor point release to all of a sudden mean their database doesn't start anymore.. Sorry I havn't got time right now to run down the issue I had with OpenSSH doing something similar, and it might have just been poor coding in OpenSSH, but I wanted to voice my concern. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] true serializability and predicate locking
Greg Stark gsst...@mit.edu wrote: My comment was in relation to the idea of representing the costs in the planner. I was a) saying you had to see how the implementation went before you try to come up with how to represent the costs and then b) speculating (hypocritically:) that you might have the direction of adjustment backward. I think we may be agreeing violently. I had the thought that costing may need to be adjusted, suggested the easiest hack that seemed like it might show an improvement, and said it needed more thought before we got to trying it out in the optimization phase. I don't think we actually disagree about much there. From what I understand your first cut will just take full-table locks anyways so it won't matter what type of plan is used at all. Right. And it would be totally premature to try to test any optimizations at that phase, which is reflected in the development plan on the wiki. Personally I can't see how that won't generate a serialization failure on basically every query on any moderately concurrent system but at least it would make an interesting test-bed for the SIREAD dependency detection logic. Exactly. And I agree it's necessary code before we get into more fine-grained siread locks. Cool. Overall, it sounds like we've gotten to the same page. :-) -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
On Jan 8, 2010, at 1:35 AM, Dave Page wrote: I am saying that if the design won't ever work without requiring painful dependency installation that users will likely not want to bother with, then it is fundamentally broken. Better to write one system that can _eventually_ work everywhere, than one that works for some, and then another that works elsewhere. This whole bit about Windows is a red herring. Perhaps I should not have phrased it the way I did WRT Windows. So I'm going to change it to: The PGAN client will make no other assumptions about how to build and install extensions, leaving such to the PostgreSQL core. To the extent that PGXS-powered `make` works on a given platform, the client will support it. Discussing it with Andrew, that means it should work if you have mingw, and we might have to tweak it a bit to work with `src/tools/msvc`. So the point is: the PGAN client (which is just one part of this project, after all), will *not* include a build system. It will use whatever build system is supported by the community. Right now that's PGXS. If core switches to something later, or provides binary builds for Windows, the client will be easily adapted to take advantage of it. No sweat. The upshot is this: PGAN does *not* ignore Windows or any other platform. Rather, it relies on others to create the appropriate community-supported installers for all platforms. The issue of build systems and installers is not within its domain. Thus, I've also changed the FAQ to: * '''What about Windows?''' The PGAN client will always follow the lead of the PostgreSQL core on installing extensions. If support for installing extensions on Windows improves such that a compiler is no longer required, the PGAN client will be modified as appropriate to take advantage of it. This applies not specifically to Windows, but to the ability of the core intaller (or any future community-supported installer) to work on ''any'' platform. Please let the Windows thread die now. PGAN doesn't ignore Windows; it ignores installer development. Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
* Kevin Grittner (kevin.gritt...@wicourts.gov) wrote: It also seems to call into question the wisdom of annual releases. If we had a two-year cycle which had three times as much in it, would that be an improvement, or not? At the moment, my vote would be how 'bout we discuss this post-8.5?. Maybe if we postpone the discussion till then, we'll actually be able to chew through everything and release close to on-time, otherwise I wonder if a thread like this won't end up sucking up too much in terms of our resources right at crunch time.. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] damage control mode
On Jan 8, 2010, at 1:02 AM, Dimitri Fontaine wrote: Now, I'll second Greg Smith and Tom here, in that I think we need to run the last commitfest as usual, knowing that the outcome of the commitfest for any given patch is not it made it but we reviewed it. It's still right for the project to bump a patch on resources ground rather than on technical merit, at the end of the commitfest. +1 David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Add .gitignore files to CVS?
On Friday 08 January 2010 17:38:15 Alex Hunsaker wrote: On Fri, Jan 8, 2010 at 02:03, Magnus Hagander mag...@hagander.net wrote: You can always create your own branch with just the .gitignore files and merge that into whatever you're working on :) The only thing annoying about that is if you generate diffs ala git diff origin/master.. you get your .gitignore in it. What I do is have a .gitignore that is gitignored. That way its not committed, its on any branch i switch to or make and I don't accidentally commit it. Thats what .git/info/excludes is for... Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
On Fri, Jan 8, 2010 at 5:13 PM, David E. Wheeler da...@kineticode.com wrote: This whole bit about Windows is a red herring. Perhaps I should not have phrased it the way I did WRT Windows. So I'm going to change it to: The PGAN client will make no other assumptions about how to build and install extensions, leaving such to the PostgreSQL core. To the extent that PGXS-powered `make` works on a given platform, the client will support it. Discussing it with Andrew, that means it should work if you have mingw, and we might have to tweak it a bit to work with `src/tools/msvc`. So the point is: the PGAN client (which is just one part of this project, after all), will *not* include a build system. It will use whatever build system is supported by the community. Right now that's PGXS. If core switches to something later, or provides binary builds for Windows, the client will be easily adapted to take advantage of it. No sweat. The upshot is this: PGAN does *not* ignore Windows or any other platform. Rather, it relies on others to create the appropriate community-supported installers for all platforms. The issue of build systems and installers is not within its domain. Thus, I've also changed the FAQ to: If that is the goal of your project then I withdraw my previous comments, which were written on the belief that you were proposing a generic distribution/build/installation system for PostgreSQL users. Sorry for the noise! -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
Stephen Frost sfr...@snowman.net writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: I don't want to go to the trouble of creating (and documenting) a configure option for this. Much less a GUC ;-) Requiring a custom build to disable it would be horrible, in my view. Or, at best, just means that the packagers won't enable it, which obviously would be less than ideal. I'm a packager, and I think that this approach is perfectly fine. The place where the rubber meets the road is in the init script, which is the packager's responsibility. If the packager is going to provide an init script that sets oom_adj in the first place, he can turn on the compensation code inside the binary. If not, the compensation code has no purpose anyhow. There are no moving parts in this as far as the end user is concerned. Sorry if it's a pain, but I think it needs to either be configurable or not done. As I said before, it definitely needs to handle failure gracefully, We just ignore any error from the attempt to write to /proc. but I worry that even that won't be sufficient in some cases. Just thinking about how we run PG under VServers and Linux Containers and whatnot, I think you are missing the point that the code won't even be compiled except on platforms where the packager has determined that it's sensible to have it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
Alex, * Alex Hunsaker (bada...@gmail.com) wrote: As long as the VM/container you are running under wont kill postmaster for trying to access proc-- the patch I posted should work fine. It just ignores any error (I assumed you might be running in a chroot without proc or some such). As I recall, oom_adj wasn't visible in the container because you explicitly set what proc entries processes can have access to when using VServers, and OpenSSH didn't handle that cleanly. Guess what I'm just saying is don't just assume everything is as it would be on a 'normal' system when dealing with /proc and friends, and, of course, test, test, test when you're talking about back-porting things. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] true serializability and predicate locking
On Fri, Jan 8, 2010 at 5:13 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: From what I understand your first cut will just take full-table locks anyways so it won't matter what type of plan is used at all. Right. And it would be totally premature to try to test any optimizations at that phase, which is reflected in the development plan on the wiki. Personally I can't see how that won't generate a serialization failure on basically every query on any moderately concurrent system but at least it would make an interesting test-bed for the SIREAD dependency detection logic. Exactly. And I agree it's necessary code before we get into more fine-grained siread locks. Cool. Overall, it sounds like we've gotten to the same page. :-) Well we disagree with whether we have any reasonable plan for adding the more fine-grained locks. AFAICT we have either a) add something clean and abstract which doesn't scale at all or b) turn Postgres into a clone of Sybase by adding something grotty with hooks all over the tree which exposes internal details as user-visible behaviour. I'm pretty unhappy with both options. The SIREAD stuff sounds cool but it's all based on these read locks that we have no plausible implementation which doesn't throw away all the wonderful things about Postges like that it does everything at the tuple level and doesn't have arbitrary limits on the size of transactions. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
On Fri, Jan 8, 2010 at 11:44 AM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Thu, Jan 7, 2010 at 9:11 PM, Andrew Dunstan and...@dunslane.net wrote: This strikes me as quite premature. Heiki just said he expects to have SR committed next week. Getting it committed is not what I'm worried about. What I'm concerned about is Tom's statement that he believes that HS is not close to being in a state where it is stable enough to go to beta, and the possibility that other large patches committed during this CommitFest might have similar issues. Looking at our existing schedule, I think it is unlikely we will even make a June 2010 release date for 8.5. Let me explain why --- here is the ideal schedule: Jan 15 start commitfest Feb 15 stop commitfest Apr 1 start beta Jun 1 release release candidate (RC) Jun 20 release 8.5 You can't move from commitfest to beta until all _known_ bugs are fixed/addressed, and you can't move from beta to RC using the same criteria. Hmm. For 8.4, I don't think we actually fixed all known bugs - I think we made a decision about which ones had to be fixed and which ones we were going to push off, and then did so. Of course we rarely have an ideal schedule --- we should expect unexpected problems. Now, I know people are going to say I am being over pessimistic, but history will bear out my analysis. I think we should anticipate a July or August release of 8.5. I think so, too, but I'm actually afraid that if we don't start making some tough decisions soon it's going to be even later than that. I'm dismayed by the number of people who seem to think that the current schedule is not already tight. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Add .gitignore files to CVS?
On Thu, Jan 7, 2010 at 5:45 PM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: On tor, 2010-01-07 at 22:16 +, Tim Bunce wrote: Is there any reason not to add .gitignore files into the repository? I already find the .cvsignore files to be useless and an annoyance to keep up to date (well, I basically ignore them and someone else cleans up after me), When/if we move off CVS, we should drop the .cvsignore files and insert .gitignore (assuming that works the same way). I am not in favor of expecting the core project to maintain support for non-core SCMs though. Will we have .bzrignore and who knows what else in there too? As Peter points out, the only way the things will get maintained is if the complaints they suppress are in-the-face of somebody with commit access to the core repo. I like quiet from my cvs updates, so I'm willing to maintain .cvsignores, but I'm not willing to maintain multiple copies of that information. I would be willing to maintain .gitignore files, under the agreement that if I should fail or cease to do so, and no one else wants to take over, then they all get removed. Would that be acceptable? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
Tom Lane wrote: I don't want to go to the trouble of creating (and documenting) a configure option for this. Much less a GUC ;-) What I suggest is that we do something like #ifdef LINUX_OOM_ADJ ... fprintf(oom, %d\n, LINUX_OOM_ADJ); ... #endif Then, somebody who wants the feature would build with, say, -DLINUX_OOM_ADJ=0 or another value if they want that. +1 for this. Looks like a sound approach. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
On Fri, Jan 8, 2010 at 5:13 PM, David E. Wheeler da...@kineticode.com wrote: Please let the Windows thread die now. PGAN doesn't ignore Windows; it ignores installer development. yeah, I think there are two quite separable projects here. It's quite possible that once the binary installer people have a source package with all the meta data and computer-readable build instructions that PGAN will need that they'll be able to implement support for binary pre-built modules as well. But just tackling the source package problem solves a lot of the problems already. The binary installer people will have their own set of problems to solve as well, they'll have to deal with environment dependencies such as OS versions, external binary versions, privileges, etc. These are all additional problems on top of having a source package already. To make an analogy configure and make are still useful even after you have dpkg -- you still need to be able to build the individual source package first before you get to put it into a binary package. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
On Fri, Jan 8, 2010 at 12:24 PM, Tom Lane t...@sss.pgh.pa.us wrote: Stephen Frost sfr...@snowman.net writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: I don't want to go to the trouble of creating (and documenting) a configure option for this. Much less a GUC ;-) Requiring a custom build to disable it would be horrible, in my view. Or, at best, just means that the packagers won't enable it, which obviously would be less than ideal. I'm a packager, and I think that this approach is perfectly fine. The place where the rubber meets the road is in the init script, which is the packager's responsibility. If the packager is going to provide an init script that sets oom_adj in the first place, he can turn on the compensation code inside the binary. If not, the compensation code has no purpose anyhow. There are no moving parts in this as far as the end user is concerned. There could well be moving parts if the user wants to adjust the value being written to oom_adj, and can't because it's compiled in. I don't see why we can't just add a GUC for this and be done with it. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
On Thu, Jan 7, 2010 at 21:07, David E. Wheeler da...@kineticode.com wrote: Hackers, I've posted a [plan] to implement PGAN][], a CPAN for PostgreSQL extensions. I've tried to closely follow the [CPAN philosophy][] to come up with a plan that requires a minimum-work implementation that builds on the existing PostgreSQL tools and the examples of the [CPAN][] and [JSAN][]. My hope is that it's full of [JFDI][]! I would be very grateful for feedback and suggestions. [plan]: http://wiki.postgresql.org/wiki/PGAN [CPAN philosophy]: http://use.perl.org/article.pl?sid=02/11/12/1616209 [CPAN]: http://cpan.org/ [JSAN]: http://www.openjsan.org [JFDI]: http://acronyms.thefreedictionary.com/JFDI One note on a completely different topic from where this thread appears to have gone.. :-) Is there a particular reason not to use the existing mirroring network to distribute the files? If not, then I suggest using them should be part of the design. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
On Fri, Jan 8, 2010 at 5:38 PM, Magnus Hagander mag...@hagander.net wrote: On Thu, Jan 7, 2010 at 21:07, David E. Wheeler da...@kineticode.com wrote: Hackers, I've posted a [plan] to implement PGAN][], a CPAN for PostgreSQL extensions. I've tried to closely follow the [CPAN philosophy][] to come up with a plan that requires a minimum-work implementation that builds on the existing PostgreSQL tools and the examples of the [CPAN][] and [JSAN][]. My hope is that it's full of [JFDI][]! I would be very grateful for feedback and suggestions. [plan]: http://wiki.postgresql.org/wiki/PGAN [CPAN philosophy]: http://use.perl.org/article.pl?sid=02/11/12/1616209 [CPAN]: http://cpan.org/ [JSAN]: http://www.openjsan.org [JFDI]: http://acronyms.thefreedictionary.com/JFDI One note on a completely different topic from where this thread appears to have gone.. :-) Is there a particular reason not to use the existing mirroring network to distribute the files? If not, then I suggest using them should be part of the design. With StackBuilder, a package can either be listed as a path, relative to the FTP root on the mirror network, or using an alternate URL for something stored elsewhere. The current set of active mirrors can always be found at http://www.postgresql.org/mirrors.xml, so you can build URLs on the mirror network using the protocol, host, port and path from the mirror list, and then the relative path for the file. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Setting oom_adj on linux?
Stephen Frost sfr...@snowman.net writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: I don't want to go to the trouble of creating (and documenting) a configure option for this. Much less a GUC ;-) Requiring a custom build to disable it would be horrible, in my view. BTW, maybe you're confused about the intention here? You'd need a custom build to *enable* it, not to disable it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
On Fri, Jan 8, 2010 at 18:44, Dave Page dp...@pgadmin.org wrote: On Fri, Jan 8, 2010 at 5:38 PM, Magnus Hagander mag...@hagander.net wrote: On Thu, Jan 7, 2010 at 21:07, David E. Wheeler da...@kineticode.com wrote: Hackers, I've posted a [plan] to implement PGAN][], a CPAN for PostgreSQL extensions. I've tried to closely follow the [CPAN philosophy][] to come up with a plan that requires a minimum-work implementation that builds on the existing PostgreSQL tools and the examples of the [CPAN][] and [JSAN][]. My hope is that it's full of [JFDI][]! I would be very grateful for feedback and suggestions. [plan]: http://wiki.postgresql.org/wiki/PGAN [CPAN philosophy]: http://use.perl.org/article.pl?sid=02/11/12/1616209 [CPAN]: http://cpan.org/ [JSAN]: http://www.openjsan.org [JFDI]: http://acronyms.thefreedictionary.com/JFDI One note on a completely different topic from where this thread appears to have gone.. :-) Is there a particular reason not to use the existing mirroring network to distribute the files? If not, then I suggest using them should be part of the design. With StackBuilder, a package can either be listed as a path, relative to the FTP root on the mirror network, or using an alternate URL for something stored elsewhere. The current set of active mirrors can always be found at http://www.postgresql.org/mirrors.xml, so you can build URLs on the mirror network using the protocol, host, port and path from the mirror list, and then the relative path for the file. Or you can hit it off the mirror redirector on the website, which will then give you automatic download tracking. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
2010/1/8 Magnus Hagander mag...@hagander.net: On Fri, Jan 8, 2010 at 18:44, Dave Page dp...@pgadmin.org wrote: The current set of active mirrors can always be found at http://www.postgresql.org/mirrors.xml, so you can build URLs on the mirror network using the protocol, host, port and path from the mirror list, and then the relative path for the file. Or you can hit it off the mirror redirector on the website, which will then give you automatic download tracking. Right, but you need the mirror ID for that iirc. Maybe we should add that to the XML... -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] true serializability and predicate locking
Greg Stark gsst...@mit.edu wrote: Well we disagree with whether we have any reasonable plan for adding the more fine-grained locks. We probably agree on that, too. Perhaps it's that I think we can develop one within a few months and you don't? AFAICT we have either a) add something clean and abstract which doesn't scale at all or b) turn Postgres into a clone of Sybase by adding something grotty with hooks all over the tree which exposes internal details as user-visible behaviour. Well, I sure hope we can avoid falling into either of those pits. It sounds like Robert has some ideas on a clean approach. I haven't looked at that aspect deeply enough to make detailed suggestions. The SIREAD stuff sounds cool but it's all based on these read locks that we have no plausible implementation which doesn't throw away all the wonderful things about Postges like that it does everything at the tuple level and doesn't have arbitrary limits on the size of transactions. I don't plan to throw any of that away; all existing techniques will continue to be used for all transactions, and unless a transaction is serializable it will not use any new techniques. For a serializable transaction the granularity of information used to detect dangerous structures will need to be kept in RAM and will therefore need to support coarser granularity at times to avoid running out of space. Coarser granularities will cause a higher rollback rate for serializable transactions. The optimization phase is mostly about minimizing false positive rollbacks. We probably have different thresholds for how many serialization errors we'd be willing to tolerate to get the benefits of true serializability, but that doesn't seem like a very fundamental difference to me. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
On Jan 8, 2010, at 9:24 AM, Dave Page wrote: If that is the goal of your project then I withdraw my previous comments, which were written on the belief that you were proposing a generic distribution/build/installation system for PostgreSQL users. It is a generic distribution and installation system, but it just uses installer approaches provided by others. This is not unlike CPAN.pm, which doesn't include an installer itself, but things through Module::Build or ExtUtils::MakeMaker as appropriate. Completely separate domain problem, as Greg notes. Sorry for the noise! Glad to have it cleared up! :-) Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: PostgreSQL Add-On Network
On Jan 8, 2010, at 9:38 AM, Magnus Hagander wrote: Is there a particular reason not to use the existing mirroring network to distribute the files? If not, then I suggest using them should be part of the design. No, as long as PAUS can drop uploaded distributions onto the master FTP server, or else the existing mirror system can rsync from PGAN's own master (I'll build all this on my own box to start with). It'll just be rsync, really, it won't where it's mirrored or where the master index lives. Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
