Re: [HACKERS] Postgresql Automatic vacuum
As far as getting into base postgresql distro. I don't mind it rewriting but I have some reservations. 1) As it is postgresql source code is huge. Adding functions to it which directly taps into it's nervous system e.g. cache, would take far more time to perfect in all conditions. It doesn't have to make its way into the postgresql daemon itself -- in fact since some people like tuning the vacuuming, it makes more sense to make this a daemon. No, my suggestion is simple that some sort of auto-vacuumer be compiled as a stand-alone app and included in the standard postgresql tar.gz file, and the install instructions recommend the site adding it as a cron job. On linux, it'd be good if the RPM install it automatically (or else it ran as a mostly-asleep daemon) because so many of the Linux/Postgresql users we see have just no clue about Postgresql, and no intention of reading anything. Just an FYI, a message I received today from the postmaster at a major telco about their postgresql experience: We have experienced some problems but they have generally cleared up after a database vacuum. However, sometimes I have found that the vacuum itself (especially a vacuum analyze) will go into the CPU consumption loop. -john ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] Lyris looking to help fix PostgresSQL crashing problems
Hello -- I'm the lead programmer of Lyris ListManager, an email list server that run on PostgreSQL, Oracle, and MS/SQL. About 20% of our client base of 4000 runs on PostgresSQL -- it's very popular with our clients -- much more than Oracle is (about 3%). Unfortunately we have about a dozen clients who have stability problems with PostgresSQL. This week a major television network cancelled their order with us due to their PostgresSQL stability issues, which is what prompted me to write this email and get involved with the PostgresSQL community. It seems that with larger database sizes (500,000 rows and larger) and high stress, the server daemon has a tendency to core. We've also had cases where a single connection doing a million inserts into a table will cause the daemon to core. We've seen problems with both 7.1 and 7.2.x, with built-on-the-machine and with RPMs. We've also had big stability problems with Solaris 8/Sparc, and don't ship on that platform because of that. What I'd like to do is help solve these problems in the core distribution, so that PostrgesSQL can indeed be able to handle the large databases and high transaction loads that Microsoft SQL can. My company has hired open source people before to help fix bugs or add features to open source projects, most notable from the Tcl community, as we use Tcl quite a bit (we have two programmers from the Tcl Core team working here). This works out well for the Tcl community, as we fund the development of the project, as well as pay someone to work on something they want to work on anyhow. So... what I'm looking for are recommendations on a PostgresSQL guru who could help nail the stability/load issues, and make sure that the fixes make their way back into the PostgresSQL core. What I'd prefer is to get a regular contributor to this list, so that this person could investigate our problems, and then get the community's help in solving them. Thanks! -john ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Lyris looking to help fix PostgresSQL crashing problems
John Buckman [EMAIL PROTECTED] writes: It seems that with larger database sizes (500,000 rows and larger) and high stress, the server daemon has a tendency to core. We'd love to see some stack traces ... Yeah, I just didn't know what form this list prefers to work on things, which is why I'd prefer to hire a regular participant of this list. If gcc 'where' stack traces are what you want, we can do that. I suspect that the problems may be platform-or-build related, because we've often had trouble replicating customer problems on our own sysems. For example, we had many reports of problems with 7.2.x, and saw it crash often on a customer's redhat machine that we had ssh access to, but couldn't make it crash in our own lab. :( That's why we need help. If we could make a simple C test case that crashed pgsql, I'm sure you guys could fix the problem in a jiffy. -john ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Lyris looking to help fix PostgresSQL crashing problems
John Buckman [EMAIL PROTECTED] writes: It seems that with larger database sizes (500,000 rows and larger) and high stress, the server daemon has a tendency to core. We'd love to see some stack traces ... Yeah, I just didn't know what form this list prefers in terms of info to be able to work on things, which is why I'd prefer to hire a regular participant of this list. If gcc 'where' stack traces from core files are what you want, we can do that. I suspect that the problems may be platform-or-build related, because we've often had trouble replicating customer problems on our own sysems. For example, we had many reports of problems with 7.2.x, and saw it crash often on a customer's redhat machine that we had ssh access to, but couldn't make it crash in our own lab. :( That's why we need help. If we could make a simple C test case that crashed pgsql, I'm sure you guys could fix the problem in a jiffym but localizing and recreating a problem is always 80% of it. -john ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Postgresql Automatic vacuum
Just an FYI - this kind of thing would be a *great* feature addition to the generic PostgresSQL release. We at Lyris often hear that postgressql is very slow, and the files are getting larger and then wow! it's so much faster now that we're regularly vacuuming! after we let them know about this need (the RPM install of PostgresSQL is so easy that most people don't read any docs). Automatic maintenance of database tables is a Good Thing (tm) and would make more people we introduce to pgsql favorably disposed toward it. -john I have written a small daemon that can automatically vacuum PostgreSQL database, depending upon activity per table. It sits on top of postgres statistics collector. The postgres installation should have per row statistics collection enabled. Features are, * Vacuuming based on activity on the table * Per table vacuum. So only heavily updated tables are vacuumed. * multiple databases supported * Performs 'vacuum analyze' only, so it will not block the database The project location is http://gborg.postgresql.org/project/pgavd/projdisplay.php Let me know for bugs/improvements and comments.. I am sure real world postgres installations has some sort of scripts doing similar thing. This is an attempt to provide a generic interface to periodic vacuum. Bye Shridhar -- The Abrams' Principle:The shortest distance between two points is off the wall. ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
