Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Bruce Momjian

Yes, it seems we will have to do something for 8.3.  I assume the method
below would reduce frequent updates of the stats_command_string too.

---

Tom Lane wrote:
 Arjen van der Meijden told me that according to the tweakers.net
 benchmark, HEAD is noticeably slower than 8.2.4, and I soon confirmed
 here that for small SELECT queries issued as separate transactions,
 there's a significant difference.  I think much of the difference stems
 from the fact that we now have stats_row_level ON by default, and so
 every transaction sends a stats message that wasn't there by default
 in 8.2.  When you're doing a few thousand transactions per second
 (not hard for small read-only queries) that adds up.
 
 It seems to me that this could be fixed fairly easily by allowing the
 stats to accumulate across multiple small transactions before sending
 a message.  There's surely not much point in kicking stats out quickly
 when the stats collector only reports them to the world every half
 second anyway.
 
 The first design that comes to mind is that at transaction end
 (pgstat_report_tabstat() time) we send a stats message only if at least
 X milliseconds have elapsed since we last sent one, where X is
 PGSTAT_STAT_INTERVAL or closely related to it.  We also make sure to
 flush stats out before process exit.  This approach ensures that in a
 lots-of-short-transactions scenario, we only need to send one stats
 message every X msec, not one per query.  The cost is possible delay of
 stats reports.  I claim that any transaction that makes a really sizable
 change in the stats will run longer than X msec and therefore will send
 its stats immediately.  Cases where a client does a small transaction
 after sleeping for awhile (more than X msec) will also send immediately.
 You might get a delay in reporting the last few transactions of a burst
 of short transactions, but how much does it matter?  So I think that
 complicating the design with, say, a timeout counter to force out the
 stats after a sleep interval is not necessary.  Doing so would add a
 couple of kernel calls to every client interaction so I'd really rather
 avoid that.
 
 Any thoughts, better ideas?
 
   regards, tom lane
 
 ---(end of broadcast)---
 TIP 3: Have you checked our extensive FAQ?
 
http://www.postgresql.org/docs/faq

-- 
  Bruce Momjian  [EMAIL PROTECTED]  http://momjian.us
  EnterpriseDB   http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 Yes, it seems we will have to do something for 8.3.

Yeah, we've kind of ignored any possible overhead of the stats mechanism
for awhile, but I think we've got to face up to this if we're gonna have
it on by default.

 I assume the method
 below would reduce frequent updates of the stats_command_string too.

No, stats_command_string is entirely independent now.

regards, tom lane

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian [EMAIL PROTECTED] writes:
  Yes, it seems we will have to do something for 8.3.
 
 Yeah, we've kind of ignored any possible overhead of the stats mechanism
 for awhile, but I think we've got to face up to this if we're gonna have
 it on by default.
 
  I assume the method
  below would reduce frequent updates of the stats_command_string too.
 
 No, stats_command_string is entirely independent now.

Oh, right, we used shared memory.  No wonder it isn't on the TODO list
anymore.  ;-)

-- 
  Bruce Momjian  [EMAIL PROTECTED]  http://momjian.us
  EnterpriseDB   http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] question for serial types with CHECK conditions

2007-04-29 Thread Martijn van Oosterhout
On Sat, Apr 28, 2007 at 01:34:15PM -0300, Guido Barosio wrote:
   My point was to step on the asumption that the implicit serial
 call for a type represents the fact that the sequence will start
 allways in the same place, unless inmediatelly after your create
 table you plan to modify that, which makes no sense when we go back
 to what the CREATE SEQUENCE represents for the case.

I don't think we need to go second guessing what the user may or may
not have wanted. If the user wanted this, we provide it. If the user
didn't they'll notice soon enough. The most important thing is that it
does exactly what it's documented to do.

Have a nice day,
-- 
Martijn van Oosterhout   [EMAIL PROTECTED]   http://svana.org/kleptog/
 From each according to his ability. To each according to his ability to 
 litigate.


signature.asc
Description: Digital signature


Re: [HACKERS] Feature freeze progress report

2007-04-29 Thread Simon Riggs
On Thu, 2007-04-26 at 23:13 -0400, Bruce Momjian wrote:

 Our community could probably handle a few of these complex patches, but
 the volume for this release is significantly higher than previous
 releases.  The community is doing a good job of giving patch writers
 feedback and new patch versions are getting generated.  However, this
 amount of patch churn is not normal.

I think it is probably going to be normal from now on. We now a
significant number of reasonably prolific developers.

 I think the community has to come up with ideas on how to accomplish this.

My thinking is to move to a two stage release process: Do one
production release annually, and one dev release at the 6 month
mid-point. That way each new release contains a manageable number of new
features and we have a realistic chance of integrating them
successfully. Support companies would then have the option to support
both releases, or just the main production release. Leading edge users,
of which we have many, would then benefit from more frequent additional
features.

I would also suggest that 8.3 be labelled a dev release. We have a
reasonable number of fairly invasive patches, so we need a mechanism to
integrate them with reduced risk.

With those two suggestions, the prod release would freeze on Sep 30 and
the dev release on Mar 31. This would then put us into the same
situation as Linux, where odd-numbered releases are dev and
even-numbered are main releases. Everyone would understand our decision
to take this action, as well as immediately understanding how this
process will work in the future.

By agreeing this now, we can then punt a reasonable number of patches
back to the main release, later this year. The de-selected patches still
have a second chance of making it into a release available in 2007 and
this will diffuse the various tensions and difficulties we now have.

Without this, we face a long wait. 8.2 took 4 months to go through beta
and release, so 8.3 could well take 6 months. If we allow 8.3 to be a
mega-release then it might take much longer than that: increases in
complexity have a non-linear effect on software quality/robustness.
Adding reviewers or committers isn't going to dramatically change that.
IMHO the only sensible thing to do is to make releases more frequent so
that each one is still a manageable size. 

The alternative is to somehow select patches to wait until the next
release, a full year away. That is unlikely to be an easy process and
nobody really wants to volunteer their own or others' patches.

Realistically, people won't speed up the frequency they upgrade their
software and we certainly don't want to increase the number of
production releases in circulation that we must support. This set of
proposals is a realistic way forward from where we are and will be
easily explained to people only briefly in touch with our project.

Whether or not this is accepted, I'm happy to offer assistance wherever
the core team directs to improve the current situation.

-- 
  Simon Riggs 
  EnterpriseDB   http://www.enterprisedb.com



---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate


Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Simon Riggs
On Sun, 2007-04-29 at 00:44 -0400, Tom Lane wrote:

 The first design that comes to mind is that at transaction end
 (pgstat_report_tabstat() time) we send a stats message only if at least
 X milliseconds have elapsed since we last sent one, where X is
 PGSTAT_STAT_INTERVAL or closely related to it.  We also make sure to
 flush stats out before process exit.

Sounds like a good general, long term solution.

-- 
  Simon Riggs 
  EnterpriseDB   http://www.enterprisedb.com



---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [HACKERS] Feature freeze progress report

2007-04-29 Thread Stefan Kaltenbrunner
Simon Riggs wrote:
 On Thu, 2007-04-26 at 23:13 -0400, Bruce Momjian wrote:
 
 Our community could probably handle a few of these complex patches, but
 the volume for this release is significantly higher than previous
 releases.  The community is doing a good job of giving patch writers
 feedback and new patch versions are getting generated.  However, this
 amount of patch churn is not normal.
 
 I think it is probably going to be normal from now on. We now a
 significant number of reasonably prolific developers.
 
 I think the community has to come up with ideas on how to accomplish this.
 
 My thinking is to move to a two stage release process: Do one
 production release annually, and one dev release at the 6 month
 mid-point. That way each new release contains a manageable number of new
 features and we have a realistic chance of integrating them
 successfully. Support companies would then have the option to support
 both releases, or just the main production release. Leading edge users,
 of which we have many, would then benefit from more frequent additional
 features.

I'm not really convinced that this is good idea at all - it would lead
to further fragmentation of developer resources (likely more versions to
support and more frequent releases which to put quite a load on
developers by itself). And a lot of issues only get found in the field
and I'm unsure if a releases labled dev might get the critical mass in
terms of testing (if that would be true we would find most of the bugs
during beta imho).
99% of the people only use what is declared stable and already has a
number of minor releases - and the other 1% is already reading -hackers ...

 
 I would also suggest that 8.3 be labelled a dev release. We have a
 reasonable number of fairly invasive patches, so we need a mechanism to
 integrate them with reduced risk.

I would rather like to see patches we don't are confident enough in to
be dropped from 8.3 and moved to 8.4 - the goal should not be jamming as
much patches into a single release s we can (because they are proposed)
but rather putting those in that meet the quality bar and we trust in.

 
 With those two suggestions, the prod release would freeze on Sep 30 and
 the dev release on Mar 31. This would then put us into the same
 situation as Linux, where odd-numbered releases are dev and
 even-numbered are main releases. Everyone would understand our decision
 to take this action, as well as immediately understanding how this
 process will work in the future.

I don't want to see the current linux model - which is basically that
2.6 is a continous development trunk with snapshots (ie releases) done
every few months that only get supported until the next release or so.
Note that all the distributions spend considerable amount of time
stabilizing those kernels and have to put additional resources into
maintaining those over the years because upstream is not doing that any
more.
Look into the recent discussion about releasing 2.6.21 with a number of
KNOWN regressions for example.

 
 By agreeing this now, we can then punt a reasonable number of patches
 back to the main release, later this year. The de-selected patches still
 have a second chance of making it into a release available in 2007 and
 this will diffuse the various tensions and difficulties we now have.
 
 Without this, we face a long wait. 8.2 took 4 months to go through beta
 and release, so 8.3 could well take 6 months. If we allow 8.3 to be a
 mega-release then it might take much longer than that: increases in
 complexity have a non-linear effect on software quality/robustness.
 Adding reviewers or committers isn't going to dramatically change that.
 IMHO the only sensible thing to do is to make releases more frequent so
 that each one is still a manageable size. 

again - the bottleneck right now seems to be reviewer capacity coupled
with a large number of intrusive patches. So the most logical answer is
to drop those patches we are not fully confident in and try to get them
in during 8.4.


 
 The alternative is to somehow select patches to wait until the next
 release, a full year away. That is unlikely to be an easy process and
 nobody really wants to volunteer their own or others' patches.

see above


Stefan

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Gregory Stark

Tom Lane [EMAIL PROTECTED] writes:

 So I think that complicating the design with, say, a timeout counter to
 force out the stats after a sleep interval is not necessary. Doing so would
 add a couple of kernel calls to every client interaction so I'd really
 rather avoid that.

 Any thoughts, better ideas?

If we want to have an idle_in_statement_timeout then we'll need to introduce a
select loop instead of just directly blocking on recv anyways. Does that mean
we may as well bite the bullet now?

-- 
  Gregory Stark
  EnterpriseDB  http://www.enterprisedb.com


---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] Feature freeze progress report

2007-04-29 Thread Dawid Kuroczko

On 4/28/07, Simon Riggs [EMAIL PROTECTED] wrote:

 I think the community has to come up with ideas on how to accomplish this.
My thinking is to move to a two stage release process: Do one
production release annually, and one dev release at the 6 month
mid-point. That way each new release contains a manageable number of new
features and we have a realistic chance of integrating them
successfully. Support companies would then have the option to support
both releases, or just the main production release. Leading edge users,
of which we have many, would then benefit from more frequent additional
features.


This would mean we would have to have a very well tested upgrade path
for odd releases (8.2 - 8.4).

Also it probably would mean that analytical functions or recursive queries
should be postponed until 8.5 (as they didn't end up inside 8.3, and 8.4
would be stable release).

I think that with introducing stable/devel version we are risking that devel
versions will be less used in production environments (meaning less testing)
and as a result they can lengthen the development cycle.  Currently every
release is stable, therefore we don't accept experimental patches unless
they are really good idea.  Then there is beta sequence, and then a stable
release.  With introducing dev release, we give green light to more
experimental
patches, and then devote dev release as a ripening period for them (equivalent
of current pre-releases, I imagine).  And then we release stable relese (without
experimental patches; experimental patches are postponed until devel release,
and devel release twice the number of experimental patches).

I think we should not go into stable/devel release cycle without carefully
thinking if it will serve us good.  I am afraid this will make many people
stick with stable releases and will make upgrades harder (do you remember
how hard it was to go from Linux 2.2 to 2.4, and from 2.4 to 2.6?).

 Regards,
 Dawid

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
  choose an index scan if your joining column's datatypes do not
  match


Re: [HACKERS] Feature freeze progress report

2007-04-29 Thread Heikki Linnakangas

Simon Riggs wrote:

My thinking is to move to a two stage release process: Do one
production release annually, and one dev release at the 6 month
mid-point. That way each new release contains a manageable number of new
features and we have a realistic chance of integrating them
successfully. Support companies would then have the option to support
both releases, or just the main production release. Leading edge users,
of which we have many, would then benefit from more frequent additional
features.


I like the idea of draining the patch queue mid-way through the release 
cycle. That'll hopefully encourage people to submit patches earlier in 
the release cycle, knowing they will be reviewed. It'll also give people 
working on external projects, drivers and tools, a checkpoint to sync with.


But I don't like the idea of making a release out of it. Who would use 
such a release? No one in production. Making a release comes with a 
cost, even if it's just a dev release.


One could also argue that we don't need the mid-cycle checkpoint, if we 
just keep the patch queue empty all the time. In the end, it comes down 
to how many people we have actively reviewing patches and giving 
feedback (I agree that it's not a linear relationship as you pointed out 
later in your mail, though). I believe a mid-cycle checkpoint would help 
by directing efforts to review, just like the pre-release feature freeze 
does.



I would also suggest that 8.3 be labelled a dev release. We have a
reasonable number of fairly invasive patches, so we need a mechanism to
integrate them with reduced risk.


I have no reason to believe that the next release will have less patches 
in it, so if we went down that path we could never release a stable 
release. If we have reasonable doubts about the stability of a patch, it 
should not be included. That said, all patches come with a risk.



With those two suggestions, the prod release would freeze on Sep 30 and
the dev release on Mar 31. This would then put us into the same
situation as Linux, where odd-numbered releases are dev and
even-numbered are main releases. Everyone would understand our decision
to take this action, as well as immediately understanding how this
process will work in the future.


We're having a short 8.3 cycle because we wanted to shift our release 
schedule from Autumn to Spring. That would get us back to releasing in 
Autumn.


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Lukas Kahwe Smith

Tom Lane wrote:


The first design that comes to mind is that at transaction end
(pgstat_report_tabstat() time) we send a stats message only if at least
X milliseconds have elapsed since we last sent one, where X is
PGSTAT_STAT_INTERVAL or closely related to it.  We also make sure to
flush stats out before process exit.  This approach ensures that in a
lots-of-short-transactions scenario, we only need to send one stats
message every X msec, not one per query.  The cost is possible delay of
stats reports.  I claim that any transaction that makes a really sizable
change in the stats will run longer than X msec and therefore will send
its stats immediately.  Cases where a client does a small transaction
after sleeping for awhile (more than X msec) will also send immediately.
You might get a delay in reporting the last few transactions of a burst
of short transactions, but how much does it matter?  So I think that
complicating the design with, say, a timeout counter to force out the
stats after a sleep interval is not necessary.  Doing so would add a
couple of kernel calls to every client interaction so I'd really rather
avoid that.


Well and if this delaying of updating the stats has an effect on query 
time, then it also increases the likelihood of going past the X msec 
limit of that previously small query. So its sort of self fixing 
with the only risk of one query getting overly long due to lack of stats 
updating.


regards,
Lukas

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] Feature freeze progress report

2007-04-29 Thread Alvaro Herrera
Stefan Kaltenbrunner wrote:
 Simon Riggs wrote:

  I would also suggest that 8.3 be labelled a dev release. We have a
  reasonable number of fairly invasive patches, so we need a mechanism to
  integrate them with reduced risk.
 
 I would rather like to see patches we don't are confident enough in to
 be dropped from 8.3 and moved to 8.4 - the goal should not be jamming as
 much patches into a single release s we can (because they are proposed)
 but rather putting those in that meet the quality bar and we trust in.

Yeah; the agreement we had was that 8.3 would be a short release.  So if
we're going to take too long to review and apply the outstanding patches
we have, we should rather push them to 8.4, get 8.3 released quickly and
then go on with the regular annual release.  The postponed patches can
be reviewed and committed early in 8.4, instead of at the last minute in
8.3.  Sounds like a smarter, safer move.

(The only complication would be the pgindent changes which could cause
merge problems for some patches.  It would be good to have a mechanism
to update a patch over pgindent easily.)

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] Feature freeze progress report

2007-04-29 Thread Gregory Stark

Heikki Linnakangas [EMAIL PROTECTED] writes:

 I like the idea of draining the patch queue mid-way through the release cycle.
 That'll hopefully encourage people to submit patches earlier in the release
 cycle, knowing they will be reviewed. It'll also give people working on
 external projects, drivers and tools, a checkpoint to sync with.

 But I don't like the idea of making a release out of it. Who would use such a
 release? No one in production. Making a release comes with a cost, even if 
 it's
 just a dev release.

On other projects people use these snapshots or dev releases because checking
stuff out from CVS is likely to get you a source tree that won't even build
let alone run cleanly. It's also nice to have everyone using the same
checkouts when report bugs or submitting patches.

But it's not because we're afraid some user will run a CVS checkout that
Postgres CVS is kept clean. Postgres CVS is kept clean because that's just the
way the Postgres developers think it should work. Doing regular snapshot
releases isn't going to cause that to get worse.

 One could also argue that we don't need the mid-cycle checkpoint, if we just
 keep the patch queue empty all the time. In the end, it comes down to how many
 people we have actively reviewing patches and giving feedback

I would argue that. In fact I would argue it would be *easier* to keep the
patch queue empty all the time than to spend months reviewing and merging
six-month-old patches once a year. But I still have hope this is a problem
that will fix itself naturally with time.

-- 
  Gregory Stark
  EnterpriseDB  http://www.enterprisedb.com


---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] Feature freeze progress report

2007-04-29 Thread Gregory Stark

Alvaro Herrera [EMAIL PROTECTED] writes:

 The postponed patches can be reviewed and committed early in 8.4, instead of
 at the last minute in 8.3.

Given that some of those patches have been in the queue since last September
is there any reason to think they'll get reviewed early in this cycle if they
weren't last cycle?

-- 
  Gregory Stark
  EnterpriseDB  http://www.enterprisedb.com


---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate


Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Alvaro Herrera
Tom Lane wrote:

 The first design that comes to mind is that at transaction end
 (pgstat_report_tabstat() time) we send a stats message only if at least
 X milliseconds have elapsed since we last sent one, where X is
 PGSTAT_STAT_INTERVAL or closely related to it.  We also make sure to
 flush stats out before process exit.  This approach ensures that in a
 lots-of-short-transactions scenario, we only need to send one stats
 message every X msec, not one per query.

If you're going to make it depend on the timestamp set by transaction
start, I'm all for it.

 The cost is possible delay of stats reports.  I claim that any
 transaction that makes a really sizable change in the stats will run
 longer than X msec and therefore will send its stats immediately.

I agree with this, particularly if it means we don't get to add another
gettimeofday().


FWIW, am I reading the code wrong or do we send the number of xact
commit and rollback multiple times in pgstat_report_one_tabstat, with
only the first one having non-zero counts?  Maybe we could put these
counters in a separate message to reduce the size of the tabstat
messages themselves.  (It may be that the total impact in bytes is
minimal, and the added overhead of an additional message is greater?)

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes:
 Tom Lane wrote:
 The first design that comes to mind is that at transaction end
 (pgstat_report_tabstat() time) we send a stats message only if at least
 X milliseconds have elapsed since we last sent one, where X is
 PGSTAT_STAT_INTERVAL or closely related to it.  We also make sure to
 flush stats out before process exit.  This approach ensures that in a
 lots-of-short-transactions scenario, we only need to send one stats
 message every X msec, not one per query.

 If you're going to make it depend on the timestamp set by transaction
 start, I'm all for it.

Ah, you're worried about not adding an extra gettimeofday() call.
Actually I was going to make it use the transaction-commit timestamp,
which xact.c already does a kernel call for so it can put a timestamp in
the xlog commit or abort record.  We don't save that aside at the moment
but easily could.

Doing this would probably mean wanting to convert the timestamps
stored in xlog commit/abort records from time_t to timestamptz;
anyone have a problem with that?

 FWIW, am I reading the code wrong or do we send the number of xact
 commit and rollback multiple times in pgstat_report_one_tabstat, with
 only the first one having non-zero counts?  Maybe we could put these
 counters in a separate message to reduce the size of the tabstat
 messages themselves.  (It may be that the total impact in bytes is
 minimal, and the added overhead of an additional message is greater?)

Yeah, that design seems fine to me as-is.  We'd only be sending multiple
messages if we have more than about 1K of tabstat records, so the
overhead is only 16 bytes out of each additional 1K ... not a lot.

regards, tom lane

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] strange buildfarm failures

2007-04-29 Thread Alvaro Herrera
Stefan Kaltenbrunner wrote:

 well - i now have a core file but it does not seem to be much worth
 except to prove that autovacuum seems to be the culprit:
 
 Core was generated by `postgres: autovacuum worker process
  '.
 Program terminated with signal 6, Aborted.
 
 [...]
 
 #0  0x0ed9 in ?? ()
 warning: GDB can't find the start of the function at 0xed9.

Interesting.  Notice how it doesn't have the database name in the ps
display.  This means it must have crashed between the initial
init_ps_display and the set_ps_display call just before starting to
vacuum.  So the bug is probably in the startup code; probably the code
dealing with the PGPROC which is the newest and weirder stuff.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] strange buildfarm failures

2007-04-29 Thread Alvaro Herrera
Alvaro Herrera wrote:
 Stefan Kaltenbrunner wrote:
 
  well - i now have a core file but it does not seem to be much worth
  except to prove that autovacuum seems to be the culprit:
  
  Core was generated by `postgres: autovacuum worker process
   '.
  Program terminated with signal 6, Aborted.
  
  [...]
  
  #0  0x0ed9 in ?? ()
  warning: GDB can't find the start of the function at 0xed9.
 
 Interesting.  Notice how it doesn't have the database name in the ps
 display.  This means it must have crashed between the initial
 init_ps_display and the set_ps_display call just before starting to
 vacuum.  So the bug is probably in the startup code; probably the code
 dealing with the PGPROC which is the newest and weirder stuff.

Oh, another thing that I think may be happening is that the stack is
restored in longjmp, so it is trying to report an error elsewhere but
it crashes because something got overwritten or something; i.e. a
bug in the error recovery code.  I don't know how feasible this is or
even if it makes sense (would longjmp() restore the ps display?), but we
had similar, very hard to debug errors in Mammoth Replicator, which is
why I'm mentioning it in case it rings a bell.

-- 
Alvaro Herrera  Developer, http://www.PostgreSQL.org/
The only difference is that Saddam would kill you on private, where the
Americans will kill you in public (Mohammad Saleh, 39, a building contractor)

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] Feature freeze progress report

2007-04-29 Thread Lukas Kahwe Smith

Alvaro Herrera wrote:


Yeah; the agreement we had was that 8.3 would be a short release.  So if
we're going to take too long to review and apply the outstanding patches
we have, we should rather push them to 8.4, get 8.3 released quickly and
then go on with the regular annual release.  The postponed patches can
be reviewed and committed early in 8.4, instead of at the last minute in
8.3.  Sounds like a smarter, safer move.


Hmm, I do not have an overview on this, but like Alvaro mentions, the 
shorter release cycles for 8.3 was done because we felt that a number of 
patches that were originally slated for 8.2 were almost but not quite 
ready for 8.2. So are all of those patches from back then ready to go 
into 8.3? If not then it would indicate that fast tracking a release 
cycle for patches there are not quite there yet is not paying off?


Otherwise, if all/most of the patches originally planned for 8.2 have 
made it into 8.3, everything is good. If new additions are not yet ready 
then they will just get bumped to 8.4, just like the changes that got 
bumped to 8.3.


regards,
Lukas

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes:
 If we want to have an idle_in_statement_timeout then we'll need to introduce a
 select loop instead of just directly blocking on recv anyways. Does that mean
 we may as well bite the bullet now?

If we wanted such a timeout (which I personally don't) we wouldn't
implement it with select because OpenSSL wouldn't cooperate.  AFAICS
this'd require setting a timer interrupt ... and then unsetting it when
the client response comes back.

regards, tom lane

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] Feature freeze progress report

2007-04-29 Thread Tom Lane
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
 Simon Riggs wrote:
 My thinking is to move to a two stage release process: Do one
 production release annually, and one dev release at the 6 month
 mid-point.

 I'm not really convinced that this is good idea at all - it would lead
 to further fragmentation of developer resources (likely more versions to
 support and more frequent releases which to put quite a load on
 developers by itself).

That's my reaction too.  The overhead of a dev release would be just
as high as a full release, and we don't really have enough manpower
to do two releases a year.  We *definitely* haven't got enough manpower
to double the number of back branches we are trying to keep patched.
So this could only work if dev releases are abandoned from a support
perspective when the next full release comes out, and that will entirely
guarantee that no DBA will use one in production.

regards, tom lane

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] strange buildfarm failures

2007-04-29 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes:
 Oh, another thing that I think may be happening is that the stack is
 restored in longjmp, so it is trying to report an error elsewhere but
 it crashes because something got overwritten or something; i.e. a
 bug in the error recovery code.

Hm, something trying to elog before the setjmp's been executed?
Although I thought it was coded so that elog.c would just proc_exit
if there was noplace to longjmp to.  A mistake here might explain
the lack of any message in the postmaster log: if elog.c thinks it
should longjmp then it doesn't print the message first.

regards, tom lane

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] Feature freeze progress report

2007-04-29 Thread Tom Lane
Lukas Kahwe Smith [EMAIL PROTECTED] writes:
 Hmm, I do not have an overview on this, but like Alvaro mentions, the 
 shorter release cycles for 8.3 was done because we felt that a number of 
 patches that were originally slated for 8.2 were almost but not quite 
 ready for 8.2. So are all of those patches from back then ready to go 
 into 8.3? If not then it would indicate that fast tracking a release 
 cycle for patches there are not quite there yet is not paying off?

In fact several of the major ones are still not ready, so I think that
experience is evidence that we couldn't handle a six-month release cycle
anyway.  We'll still stick to the announced 8.3 schedule though, since
part of the reason for it was to rotate around to a spring release time
instead of a fall release time, on the thought that that might work more
conveniently for many people.

regards, tom lane

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] Feature freeze progress report

2007-04-29 Thread Dave Page

Heikki Linnakangas wrote:
I like the idea of draining the patch queue mid-way through the release 
cycle. That'll hopefully encourage people to submit patches earlier in 
the release cycle, knowing they will be reviewed. It'll also give people 
working on external projects, drivers and tools, a checkpoint to sync with.


This is important for me - the pgAdmin development cycle follows that of 
PostgreSQL's for very obvious reasons, however *after* we enter 'feature 
freeze' we can still end up adding significant amounts of new code. Why? 
Becvause during PostgreSQL's feature freeze, patches are applied which 
add new functionality we need to support. We can't code for the new 
features when patches are submitted because we don't know if they'll go 
in, or how much they'll change when they do.


This means that there is a huge rush of new code in pgAdmin's 
development cycle, right at the time when we should be testing - making 
the release process more and more rushed as each release of PostgreSQL 
gets more efficient and adds more and more new features.


Sooner or later with things working the way they are now, we *will* 
reach a breaking point at which pgAdmin simply won't be ready when 
PostgreSQL is released.


But I don't like the idea of making a release out of it. Who would use 
such a release? No one in production. Making a release comes with a 
cost, even if it's just a dev release.


Agreed. That would have the opposite effect of what should happen.

I like the idea of having a sync point mid cycle, however, what I'd like 
to see even more is an improved system in which we put less pressure on 
the few committers we have, and give them more freedom to commit patches 
they may not understand fully themselves by having an improved community 
review and feedback process to give the patches the approval they need. 
Doing so might allow us to keep the queue of a more or less fixed, short 
length throughout the cycle. There would be a few advantages to this:


- The problem described above practically vanishes.
- Developers see their work accepted more quickly, giving them the 
confidence to produce more.
- Developers are able to build on their earlier work, knowing that it's 
believed to be reasonably sound, unlike now when they may not know for 
months.


I believe we can achieve this with a couple of relatively minor changes:

1) *Slowly* introduce more committers. The are developers gaining 
experience all the time, and as they become trusted by the existing 
committers/core they can be 'promoted' to alleviate some of the pressure 
on the existing guys.


2) Introduce a new patch management system. I suggest a web interface 
through which patches be submitted. This would assign them an ID number, 
and forward them to the patches list. The system would track any 
responses to the initial email, logging the thread automatically and 
making it available through the web interface. Posts from 
trusted/experienced developers might be highlighted so that committers 
can see at a glance if any of the more experienced guys have commented 
on the patch yet. A status flag could easily include a status flag to 
mark them as won't accept, committed, under review or under 
revision. If left at under review for too long, the patch might be 
highlighted, and if at under revision for too long, the patch author 
might be automatically requested to send a status report.


There are potentially a number of benefits to such a system:

- No patches will get lost
- Bruce's time is feed up from the mundane work of tracking patches, to 
allow him to spend time developing, reviewing/committing and all the 
other great jobs he does for the community.

- Status of patches can be seen at a glance.
- *All* discussion of a patch can be logged in one place, allowing the 
committer to put more reliance on the knowledge and experience of his 
peers, rather than being expected to fully understand the minutae of 
every patch - thus allowing him to commit more.


Well, I'll stop there as this is getting long winded - I'm sure others 
will have their own ideas about how we can improve our processes for 
future releases; one thing I'm certain of though, is that we absolutely 
must strive to improve them somehow as whilst they has served us well in 
the past, the current process is starting to show that it just won't 
scale as the project grows.


Regards, Dave

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
  choose an index scan if your joining column's datatypes do not
  match


[HACKERS] SOS, help me please, one problem towards the postgresql developement on windows

2007-04-29 Thread shieldy


my postgresql source code is at c:/mingw/postgresql and instal to
C:/msys/1.0/local/pgsql/
I add a function to src\backend\utils\adt\geo_ops.c as the following:
*Datum
box_add2(PG_FUNCTION_ARGS)
{
 BOX *box = PG_GETARG_BOX_P(0);
 Point*p = PG_GETARG_POINT_P(1);*

* PG_RETURN_BOX_P(box_construct((box-high.x + 2* p-x),
  (box-low.x + 2* p-x),
  (box-high.y +2* p-y),
  (box-low.y + 2* p-y)));
}
*there is another similar one(this is the original one):
*Datum
box_add(PG_FUNCTION_ARGS)
{
 BOX *box = PG_GETARG_BOX_P(0);
 Point*p = PG_GETARG_POINT_P(1);*

* PG_RETURN_BOX_P(box_construct((box-high.x + p-x),
  (box-low.x + p-x),
  (box-high.y + p-y),
  (box-low.y + p-y)));
}*
And i also add the declaration to the src\include\utils\geo_decls.h like
this:

extern Datum box_add2(PG_FUNCTION_ARGS);

and then I did the following like step by step:


$ cd /c/mingw/postgresql
$ ./configure

///as i download the alib, but don't kown where it should be put. so i
ignore this, does it

matter
configure: error: zlib library not found
If you have zlib already installed, see config.log for details on the
failure.  It is possible the compiler isn't looking in the proper
directory.
Use --without-zlib to disable zlib support.
$ make
...
All of PostgreSQL successfully made. Ready to install.
$ make install

PostgreSQL installation complete.
$ initdb -D /usr/local/pgsql/data   //before this i add the
environments

variableslike this:
PGDATA=C:/msys/1.0/local/pgsql/data
PGHOME=C:/msys/1.0/local/pgsql
PGHOST=localhost
PGPORT=5434
PATH= C:/msys/1.0/local/pgsql/bin
.
Success. You can now start the database server using:

C:\msys\1.0\local\pgsql\bin\postgres -D
C:/msys/1.0/local/pgsql/data
or
C:\msys\1.0\local\pgsql\bin\pg_ctl -D C:/msys/1.0/local/pgsql/data
-l logfile start

$ pg_ctl start -l logfile
server starting

$ createdb testdb
CREATE DATABASE

then I use pgadminIII to open the database:
just run the scripts:
*select box_add(box '((0,0),(1,1))',point'(2,2)')*
got:
(3,3),(2,2)

*select box_add2(box '((0,0),(1,1))',point'(2,2)')*
got:
*ERROR: function box_add2(box, point) does not exist
SQL state: 42883
advice:No function matches the given name and argument types. You may need
to add explicit **type casts.
chars:8*

anyone know this??? why this happened? what should I do?
thankyou very much!!!





Re: [HACKERS] Feature freeze progress report

2007-04-29 Thread Stefan Kaltenbrunner
Dave Page wrote:
 Heikki Linnakangas wrote:
 I like the idea of draining the patch queue mid-way through the
 release cycle. That'll hopefully encourage people to submit patches
 earlier in the release cycle, knowing they will be reviewed. It'll
 also give people working on external projects, drivers and tools, a
 checkpoint to sync with.
 
 This is important for me - the pgAdmin development cycle follows that of
 PostgreSQL's for very obvious reasons, however *after* we enter 'feature
 freeze' we can still end up adding significant amounts of new code. Why?
 Becvause during PostgreSQL's feature freeze, patches are applied which
 add new functionality we need to support. We can't code for the new
 features when patches are submitted because we don't know if they'll go
 in, or how much they'll change when they do.
 
 This means that there is a huge rush of new code in pgAdmin's
 development cycle, right at the time when we should be testing - making
 the release process more and more rushed as each release of PostgreSQL
 gets more efficient and adds more and more new features.

this is indeed an issue - but there is nothing that says that pgadminIII
has to support all the new features of a backend the they it get
released. pgadmin is decoupled from the min development cycle anyway so
adding support for a new features a few months later sounds not too much
an issue.

 
 Sooner or later with things working the way they are now, we *will*
 reach a breaking point at which pgAdmin simply won't be ready when
 PostgreSQL is released.

well if it still works but does not yet support $wizzbangnewfeature that
sounds ok too me ?

[...]

 2) Introduce a new patch management system. I suggest a web interface
 through which patches be submitted. This would assign them an ID number,
 and forward them to the patches list. The system would track any
 responses to the initial email, logging the thread automatically and
 making it available through the web interface. Posts from
 trusted/experienced developers might be highlighted so that committers
 can see at a glance if any of the more experienced guys have commented
 on the patch yet. A status flag could easily include a status flag to
 mark them as won't accept, committed, under review or under
 revision. If left at under review for too long, the patch might be
 highlighted, and if at under revision for too long, the patch author
 might be automatically requested to send a status report.

this sounds like trying to reinvent a real bugtracking system with an
email interface ...

[...]

 Well, I'll stop there as this is getting long winded - I'm sure others
 will have their own ideas about how we can improve our processes for
 future releases; one thing I'm certain of though, is that we absolutely
 must strive to improve them somehow as whilst they has served us well in
 the past, the current process is starting to show that it just won't
 scale as the project grows.

not sure I fully agree here - I think we could do a bit better on the
bug tracking front but it is a bit unclear if we cn honestly sy that
the current process does not work or is not going to scale in the
future. Complex patches need time - sometimes much more time than a
release or even two release cycles - it's unclear if trying to get those
in more agressively (by having more commiters or applying them earlier)
might not actually cause more harm due to -HEAD getting less stable and
causing developers to spend time hunting regressions.


Stefan

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


[HACKERS] Programming new commands problems.

2007-04-29 Thread Akmal Akmalhojaev

Need help. I want to create new system catalog and extend existing commands
- CREATE ROLE and another commands connected with roles. I can't find how to
add new features into parser. Can you help me?

Thanks.

Akmal.


Re: [HACKERS] SOS, help me please, one problem towards the postgresql developement on windows

2007-04-29 Thread Martijn van Oosterhout
On Mon, Apr 30, 2007 at 01:36:36AM +0800, shieldy wrote:
 
 my postgresql source code is at c:/mingw/postgresql and instal to
 C:/msys/1.0/local/pgsql/
 I add a function to src\backend\utils\adt\geo_ops.c as the following:

A few things:
- You never did a CREATE FUNCTION so ofcourse the DB doesn't know about
it
- Why are you adding it to the backend? Place it in a module.
- Read the manual on how to create new functions
- look at the examples in the contrib directory.

Have a nice day,
-- 
Martijn van Oosterhout   [EMAIL PROTECTED]   http://svana.org/kleptog/
 From each according to his ability. To each according to his ability to 
 litigate.


signature.asc
Description: Digital signature


Re: [HACKERS] Programming new commands problems.

2007-04-29 Thread Martijn van Oosterhout
On Sun, Apr 29, 2007 at 07:37:09PM +, Akmal Akmalhojaev wrote:
 Need help. I want to create new system catalog and extend existing commands
 - CREATE ROLE and another commands connected with roles. I can't find how to
 add new features into parser. Can you help me?

Check out gram.y in the parser directory.

Have a nice day,
-- 
Martijn van Oosterhout   [EMAIL PROTECTED]   http://svana.org/kleptog/
 From each according to his ability. To each according to his ability to 
 litigate.


signature.asc
Description: Digital signature


Re: [HACKERS] Feature freeze progress report

2007-04-29 Thread Dave Page
Stefan Kaltenbrunner wrote:
 This means that there is a huge rush of new code in pgAdmin's
 development cycle, right at the time when we should be testing - making
 the release process more and more rushed as each release of PostgreSQL
 gets more efficient and adds more and more new features.
 
 this is indeed an issue - but there is nothing that says that pgadminIII
 has to support all the new features of a backend the they it get
 released. pgadmin is decoupled from the min development cycle anyway so
 adding support for a new features a few months later sounds not too much
 an issue.

No it's not decoupled; it runs almost exactly in sync. Our most popular
platform ships with pgAdmin as standard because thats what the users of
that platform expect. Not shipping with pgAdmin (or a functionally
complete one) would be like shipping SQL Server without Enterprise Manager.

 Sooner or later with things working the way they are now, we *will*
 reach a breaking point at which pgAdmin simply won't be ready when
 PostgreSQL is released.
 
 well if it still works but does not yet support $wizzbangnewfeature that
 sounds ok too me ?

Who's to say it will? Changes to pg_database have required a new release
in the past.

 this sounds like trying to reinvent a real bugtracking system with an
 email interface ...

More or less - but one that's simple by design, and specifically for
tracking what happens with our patches with the absolute minimum amount
of change required to our existing communication methods.

 not sure I fully agree here - I think we could do a bit better on the
 bug tracking front but it is a bit unclear if we cn honestly sy that
 the current process does not work or is not going to scale in the
 future. Complex patches need time - sometimes much more time than a
 release or even two release cycles - 

I'm not specifically talking about complex patches (nor am I talking at
all about bug tracking) - there are a variety of patches in the queue,
of varying complexity. Some have been there for months, and worse, some
of them recieved little or no feedback when submitted leaving the
authors completely in the dark about whether their work will be
included, whether further changes are required, or whether they should
continue with additional enhancements.

it's unclear if trying to get those
 in more agressively (by having more commiters or applying them earlier)
 might not actually cause more harm due to -HEAD getting less stable and
 causing developers to spend time hunting regressions.

I'm not advocating committing patches that might destabilize the code,
I'm suggesting making it easier for individual committers to make use of
the knowledge and experience of everyone else in the community, whilst
at the same time reducing the reliance on their own experience. Even now
we occasionally see patches getting committed that (for example) Tom has
rejected months earlier. At the very least a tracker should help prevent
that happening, at best it will help committers work faster and more
effectively because they have all the relevant discussion in front of them.

Regards, Dave.

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] Feature freeze progress report

2007-04-29 Thread Marc G. Fournier
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



- --On Sunday, April 29, 2007 21:30:36 +0200 Stefan Kaltenbrunner 
[EMAIL PROTECTED] wrote:

 not sure I fully agree here - I think we could do a bit better on the
 bug tracking front but it is a bit unclear if we cn honestly sy that
 the current process does not work or is not going to scale in the
 future. Complex patches need time - sometimes much more time than a
 release or even two release cycles - it's unclear if trying to get those
 in more agressively (by having more commiters or applying them earlier)
 might not actually cause more harm due to -HEAD getting less stable and
 causing developers to spend time hunting regressions.

re: -HEAD getting less stable ... we wouldn't be adding just anyone as a 
committer, only those that are trusted to know what they are doing ... even 
'slapping in a patch', I would expect it to have some sort of preliminary 
review by *someone* with a bit of knowledge in that aspect of the code ...

- 
Marc G. Fournier   Hub.Org Networking Services (http://www.hub.org)
Email . [EMAIL PROTECTED]  MSN . [EMAIL PROTECTED]
Yahoo . yscrappy   Skype: hub.orgICQ . 7615664
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (FreeBSD)

iD8DBQFGNQrf4QvfyHIvDvMRAoUqAKDKZY8rThsQi1vTKeQgq/c4HPIyzQCfe9o2
2fHy763eMw/hxGrMnDwXnLM=
=g6E8
-END PGP SIGNATURE-


---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] Feature freeze progress report

2007-04-29 Thread Lukas Kahwe Smith

Stefan Kaltenbrunner wrote:


2) Introduce a new patch management system. I suggest a web interface
through which patches be submitted. This would assign them an ID number,
and forward them to the patches list. The system would track any
responses to the initial email, logging the thread automatically and
making it available through the web interface. Posts from
trusted/experienced developers might be highlighted so that committers
can see at a glance if any of the more experienced guys have commented
on the patch yet. A status flag could easily include a status flag to
mark them as won't accept, committed, under review or under
revision. If left at under review for too long, the patch might be
highlighted, and if at under revision for too long, the patch author
might be automatically requested to send a status report.


this sounds like trying to reinvent a real bugtracking system with an
email interface ...


I think the angle of this suggestion is to approach things from the 
perspective of trying to automate more according to how people are 
currently working instead of shoehorning an existing solution. It seems 
that most folks on -hackers prefer email based systems. The proposals 
sounds like it would not change anything much for people who choose to 
ignore the web interface and as such it has some appeal to it.


regards,
Lukas

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
  choose an index scan if your joining column's datatypes do not
  match


Re: [HACKERS] Feature freeze progress report

2007-04-29 Thread Andrew Dunstan



Dave Page wrote:


But I don't like the idea of making a release out of it. Who would 
use such a release? No one in production. Making a release comes with 
a cost, even if it's just a dev release.


Agreed. That would have the opposite effect of what should happen.

I like the idea of having a sync point mid cycle, however, what I'd 
like to see even more is an improved system in which we put less 
pressure on the few committers we have, and give them more freedom to 
commit patches they may not understand fully themselves by having an 
improved community review and feedback process to give the patches the 
approval they need. Doing so might allow us to keep the queue of a 
more or less fixed, short length throughout the cycle. There would be 
a few advantages to this:





I don't think we need a sync point. I think we need to get better at 
setting expectations and at managing the patch queue so that it gets 
drained better all the time. Nothing can be more frustrating for patch 
authors than to have patches in the queue for a very long time. They 
bitrot, and we sometime end up throwing curly questions at authors long 
after the issues are hot in their minds. I'd like to see us set 
ourselves some targets for handling patches. Something like: patches 
held over from feature freeze from the previous release will be reviewed 
within two months of the tree re-opening, and all other patches will be 
reviewed within one month of being submitted. That implies that one 
month after feature freeze the tree will only be open for bug fixes. Any 
patches unapplied at that time would be held over. Maybe that would give 
pgAdmin and friends enough head room to catch up.


cheers

andrew

---(end of broadcast)---
TIP 4: Have you searched our list archives?

  http://archives.postgresql.org


Re: [HACKERS] too much WAL volume

2007-04-29 Thread Greg Smith

On Fri, 27 Apr 2007, Jim Nasby wrote:

Yes, but how many data drives would you need to have to bottleneck on WAL? 
Even if the entire database is memory resident you'd still have to write all 
the pages out at some point, and it seems to me that you'd need a fair amount 
of disk capacity the data directory before you got pegged by WAL.


Depends on the type of transactions.  If you're doing something with lots 
of INSERT and perhaps some UPDATE volume that doesn't need to read heavily 
from the database to complete, because most of the important stuff is 
already in memory, you might run into the WAL limit without too much on 
the database disk side.  I did say it was difficult...


When I did some DBT2 testing a bit over a year ago I had a 20 drive RAID10 
for data and a mirror for WAL and was nowhere close to pegged on WAL (this 
was on a Sun V40 connected to one of their storage arrays).


No doubt, the main reason I haven't used DBT2 more is because the WAL 
volume produced before you run into database limited bottlenecks isn't 
large, and certainly not in the same ratio as some of the apps I'm 
modeling.  Mine lean more toward straightforward transaction logging in 
parts.


I'm running on similar hardware (V40 is very close, I think the EMC array 
I test against is a bit better than the most of the Sun models) and I've 
seen some scenarios that produce 40MB/s average - 60MB/s peak of WAL 
volume.  Sure seems like I'm rate limited by the RAID-1 WAL disk.  As you 
say, eventually all the data has to make it to disk, but since it's not 
too expensive nowadays to have gigabytes worth of memory and disk array 
cache you can put off database writes for a surprisingly long period of 
time with the right system design.  It's harder to buffer those pesky 
O_DIRECT WAL writes when they blow right though at least one level of 
cache.


--
* Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
  choose an index scan if your joining column's datatypes do not
  match


Re: [HACKERS] Feature freeze progress report

2007-04-29 Thread Marc G. Fournier
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



- --On Sunday, April 29, 2007 19:38:21 -0400 Andrew Dunstan
[EMAIL PROTECTED] 
wrote:

 patches held over from feature freeze from the previous
 release will be reviewed within two months of the tree re-opening

a. why 2 months?  shouldn't they be given priority, period?
b. what happens after 2 months?  we delete them?


- 
Marc G. Fournier   Hub.Org Networking Services (http://www.hub.org)
Email . [EMAIL PROTECTED]  MSN . [EMAIL PROTECTED]
Yahoo . yscrappy   Skype: hub.orgICQ . 7615664
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (FreeBSD)

iD8DBQFGNVQA4QvfyHIvDvMRAordAJ9SDrMHFumGNynXVjpdjlR5WoMfAgCgt1Xe
+7jMjrOUODmEK+glmGyrHYA=
=SgPr
-END PGP SIGNATURE-


---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] Feature freeze progress report

2007-04-29 Thread Tom Lane
Dave Page [EMAIL PROTECTED] writes:
 I like the idea of having a sync point mid cycle, however, what I'd like 
 to see even more is an improved system in which we put less pressure on 
 the few committers we have, and give them more freedom to commit patches 
 they may not understand fully themselves

That is a recipe for disaster :-(.  The real problem I see with the big
patches that are currently in the queue is that I'm not sure even the
authors understand the patches (or more accurately, all their potential
consequences) completely.  Telling committers they should apply such
patches without having understood them either is just going to lead to
an unfixably broken system.

[ thinks for a bit... ]  What we need to expand is not so much the pool
of committers as the pool of reviewers.  If a patch had been signed off
on by X number of reasonably-qualified people then it'd be fair to
consider that it could be committed.  The tracking system you suggest
could make that sort of approach manageable.

regards, tom lane

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] SOS, help me please, one problem towards the postgresql developement on windows

2007-04-29 Thread FAST PostgreSQL
I assume you are trying to create a built-in function, right?

If thats the case you may want to create an entry in pg_proc (That seems
to be missing). Search for box_add in the source code and you will get
an idea what code needs to be added where.

Rgds,
Arul Shaji


shieldy wrote:
 my postgresql source code is at c:/mingw/postgresql and instal to
 C:/msys/1.0/local/pgsql/
 I add a function to src\backend\utils\adt\geo_ops.c as the following:
 /Datum
 box_add2(PG_FUNCTION_ARGS)
 {
  BOX *box = PG_GETARG_BOX_P(0);
  Point*p = PG_GETARG_POINT_P(1);/
 
 / PG_RETURN_BOX_P(box_construct((box-high.x + 2* p-x),
   (box-low.x + 2* p-x),
   (box-high.y +2* p-y),
   (box-low.y + 2* p-y)));
 }
 /there is another similar one(this is the original one):
 /Datum
 box_add(PG_FUNCTION_ARGS)
 {
  BOX *box = PG_GETARG_BOX_P(0);
  Point*p = PG_GETARG_POINT_P(1);/
 
 / PG_RETURN_BOX_P(box_construct((box-high.x + p-x),
   (box-low.x + p-x),
   (box-high.y + p-y),
   (box-low.y + p-y)));
 }/
 And i also add the declaration to the src\include\utils\geo_decls.h
 like this:
 
 extern Datum box_add2(PG_FUNCTION_ARGS);
 
 and then I did the following like step by step:
 
 
 $ cd /c/mingw/postgresql
 $ ./configure
 
 ///as i download the alib, but don't kown where it should be put. so
 i ignore this, does it
 
 matter
 configure: error: zlib library not found
 If you have zlib already installed, see config.log for details on the
 failure.  It is possible the compiler isn't looking in the proper
 directory.
 Use --without-zlib to disable zlib support.
 $ make
 ...
 All of PostgreSQL successfully made. Ready to install.
 $ make install
 
 PostgreSQL installation complete.
 $ initdb -D /usr/local/pgsql/data   //before this i add the
 environments
 
 variableslike this:
 PGDATA=C:/msys/1.0/local/pgsql/data
 PGHOME=C:/msys/1.0/local/pgsql
 PGHOST=localhost
 PGPORT=5434
 PATH= C:/msys/1.0/local/pgsql/bin
 .
 Success. You can now start the database server using:
 
 C:\msys\1.0\local\pgsql\bin\postgres -D
 C:/msys/1.0/local/pgsql/data
 or
 C:\msys\1.0\local\pgsql\bin\pg_ctl -D
 C:/msys/1.0/local/pgsql/data -l logfile start
 
 $ pg_ctl start -l logfile
 server starting
 
 $ createdb testdb
 CREATE DATABASE
 
 then I use pgadminIII to open the database:
 just run the scripts:
 /select box_add(box '((0,0),(1,1))',point'(2,2)')/
 got:
 (3,3),(2,2)
 
 /select box_add2(box '((0,0),(1,1))',point'(2,2)')/
 got:
 *ERROR: function box_add2(box, point) does not exist
 SQL state: 42883
 advice:No function matches the given name and argument types. You
 may need to add explicit **type casts.
 chars:8*
 
 anyone know this??? why this happened? what should I do?
 thankyou very much!!!
 
  
 
 


---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org