Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-06-04 Thread Marko Karppinen
Frank Wiles wrote:
  shared_buffers = 1 ( shared_buffers in pages )
  shared_buffers = 100M  ( 100 MBs of shared_buffers )
  shared_buffers = 2048K ( 2MBs of shared_buffers )
I don't know if this is pedantic or just obsessive-compulsive,
but I think it should be MB and KB (or more properly, kB)
instead of just M and K, to distinguish from kilopages
or megapages.
mk
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
 subscribe-nomail command to [EMAIL PROTECTED] so that your
 message can get through to the mailing list cleanly


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-06-04 Thread Frank Wiles
On Fri, 4 Jun 2004 13:10:02 +0300
Marko Karppinen [EMAIL PROTECTED] wrote:

 Frank Wiles wrote:
shared_buffers = 1 ( shared_buffers in pages )
shared_buffers = 100M  ( 100 MBs of shared_buffers )
shared_buffers = 2048K ( 2MBs of shared_buffers )
 
 I don't know if this is pedantic or just obsessive-compulsive,
 but I think it should be MB and KB (or more properly, kB)
 instead of just M and K, to distinguish from kilopages
 or megapages.

  I can't see why anyone would have a problem with that. That extra
  keystroke isn't going to give anyone carpal tunnel! :) 

 -
   Frank Wiles [EMAIL PROTECTED]
   http://frank.wiles.org
 -


---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-06-04 Thread Greg Stark

Marko Karppinen [EMAIL PROTECTED] writes:

 Frank Wiles wrote:
shared_buffers = 1 ( shared_buffers in pages )
shared_buffers = 100M  ( 100 MBs of shared_buffers )
shared_buffers = 2048K ( 2MBs of shared_buffers )
 
 I don't know if this is pedantic or just obsessive-compulsive,
 but I think it should be MB and KB (or more properly, kB)
 instead of just M and K, to distinguish from kilopages
 or megapages.

Well, that's not how other tools handle it. I would suggest being consistent
with the rest of the more user friendly unix toolset that does this. GNU dd
for example uses these suffixes:

 BLOCKS and BYTES may be followed by the following multiplicative suffixes:
 xM M, c 1, w 2, b 512, kB 1000, K 1024, MB 1000*1000, M 1024*1024,
 GB 1000*1000*1000, G 1024*1024*1024, and so on for T, P, E, Z, Y.

(Though in this case the base unit is unambiguous because of the parameter.
But other examples are the output of ls -lh, the -c and -n parameters of head,
etc)

-- 
greg


---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-06-02 Thread Shridhar Daithankar
Hi,

Any updates/opinions? Should we convert assign hooks to perform actual 
assignment and custom validation instead of just custom validation? It is 
clear from README that it is for validation purposes only..

Or Shall i look for some place else to perform conversion?

Shridhar

On Tuesday 01 June 2004 18:01, Shridhar Daithankar wrote:
 On Tuesday 01 June 2004 14:12, Shridhar Daithankar wrote:
  Actually I need to find out few more things about it. It is not as simple
  as adding a assign_hook. When I tried to initdb with changes, it demanded
  64MB of shared buffers which I (now) think that somewhere NBuffers are
  used before postgresql.conf is parsed. So 8192*8000=64MB. But this is
  just guesswork. Haven't looked in it there.

 Found  it. Following is the code that is causing problem.

 guc.c:2998
 ---
   if (conf-assign_hook)
   if (!(*conf-assign_hook) (newval, changeVal, 
 source))
   {
   ereport(elevel,
   
 (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg(invalid value 
 for parameter \%s\: %d,
   name, 
 newval)));
   return false;
   }

   if (changeVal || makeDefault)
   {
   if (changeVal)
   {
   *conf-variable = newval;
   conf-gen.source = source;
   }
 ---

 So even if assign_hook is executed, the value of variable is overwritten in
 next step which nullifies any factoring/change in value done in assign
 hook.

 I find this as a convention at many other place at guc.c. Call assign_hook
 and the overwrite the value. So is assign_hook called only to validate the
 value? How do I modify the value of the variable without getting specific?

 I tried

 if (changeVal  !(conf-assign_hook))

---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-06-02 Thread Tom Lane
Shridhar Daithankar [EMAIL PROTECTED] writes:
 Any updates/opinions? Should we convert assign hooks to perform actual 
 assignment and custom validation instead of just custom validation? It is 
 clear from README that it is for validation purposes only..

As it should be.  Assign hooks have no business altering the
user-supplied value.

We do have provisions for letting string assign hooks do that, but the
intended use of this was just for trivial display adjustments like
case-normalizing time zone names.

I remain unalterably opposed to the notion of measuring shared_buffers
in KB, but if you think you can get such a thing in over my objections,
the way to do it is to decouple the GUC parameter from NBuffers.  The
GUC setting is whatever it is; you can reject the value if it's too
far out of range, but you do not editorialize upon it.  What you do is
compute the derived value for NBuffers and assign that in the assign
hook.

regards, tom lane

---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-06-02 Thread Tom Lane
Shridhar Daithankar [EMAIL PROTECTED] writes:
 I remain unalterably opposed to the notion of measuring shared_buffers
 in KB, but if you think you can get such a thing in over my objections,

 Are you OK with MBs? I am fine with anything.

No, I'm not.  shared_buffers should be measured in buffers (ie, pages).
Anything else is obscurantism.  Not to mention highly likely to confuse
people who are used to how it's been set in the past.

regards, tom lane

---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-06-02 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 Also, I it seems postgres --describe-config isn't working.  It outputs
 nothing here.

Yeah, same here.  I'll take a look --- I may have side-swiped that during
recent hacking in main.c.

regards, tom lane

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-06-02 Thread Shridhar Daithankar
On Wednesday 02 June 2004 20:16, Tom Lane wrote:
 Shridhar Daithankar [EMAIL PROTECTED] writes:
  Any updates/opinions? Should we convert assign hooks to perform actual
  assignment and custom validation instead of just custom validation? It is
  clear from README that it is for validation purposes only..

 As it should be.  Assign hooks have no business altering the
 user-supplied value.

OK

 I remain unalterably opposed to the notion of measuring shared_buffers
 in KB, but if you think you can get such a thing in over my objections,

Are you OK with MBs? I am fine with anything.

 the way to do it is to decouple the GUC parameter from NBuffers.  The
 GUC setting is whatever it is; you can reject the value if it's too
 far out of range, but you do not editorialize upon it.  What you do is
 compute the derived value for NBuffers and assign that in the assign
 hook.

That means removing NBuffers from declaration for config structure and 
substituting a dummy variable for it?

If you think this is good and acceptable enough, I will proceed making changes 
that way. Shall I take this as guideline for other parameters as well?

 Shridhar

---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-06-02 Thread Shridhar Daithankar
On Wednesday 02 June 2004 20:59, Tom Lane wrote:
 Frank Wiles [EMAIL PROTECTED] writes:
This may be an unreasonable suggestion, but how about allowing both?
I've seen several configuration systems do the following:
 
shared_buffers = 1 ( shared_buffers in pages )
shared_buffers = 100M  ( 100 MBs of shared_buffers )
shared_buffers = 2048K ( 2MBs of shared_buffers )

 I could live with that.  I'm not sure how painful it would be to wedge
 into GUC though; and I have a feeling that it is exactly *not* what
 Shridhar would think is simpler than the present behavior ;-)

Usability POV, I could live with that. That would mean converting the GUC from 
int to string though.

 Shridhar

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-06-02 Thread Tom Lane
Frank Wiles [EMAIL PROTECTED] writes:
   This may be an unreasonable suggestion, but how about allowing both? 
   I've seen several configuration systems do the following: 

   shared_buffers = 1 ( shared_buffers in pages ) 
   shared_buffers = 100M  ( 100 MBs of shared_buffers )
   shared_buffers = 2048K ( 2MBs of shared_buffers ) 

I could live with that.  I'm not sure how painful it would be to wedge
into GUC though; and I have a feeling that it is exactly *not* what
Shridhar would think is simpler than the present behavior ;-)

regards, tom lane

---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-06-02 Thread Frank Wiles
On Wed, 02 Jun 2004 11:05:43 -0400
Tom Lane [EMAIL PROTECTED] wrote:

 Shridhar Daithankar [EMAIL PROTECTED] writes:
  I remain unalterably opposed to the notion of measuring
 shared_buffers in KB, but if you think you can get such a thing in
 over my objections,
 
  Are you OK with MBs? I am fine with anything.
 
 No, I'm not.  shared_buffers should be measured in buffers (ie,
 pages). Anything else is obscurantism.  Not to mention highly likely
 to confuse people who are used to how it's been set in the past.

  This may be an unreasonable suggestion, but how about allowing both? 
  I've seen several configuration systems do the following: 

  shared_buffers = 1 ( shared_buffers in pages ) 
  shared_buffers = 100M  ( 100 MBs of shared_buffers )
  shared_buffers = 2048K ( 2MBs of shared_buffers ) 

  Using something like this would leave the old functionality in tact,
  allow users to use what they like, and shouldn't introduce that much
  complexity into the code. 

 -
   Frank Wiles [EMAIL PROTECTED]
   http://frank.wiles.org
 -


---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-06-02 Thread Harald Fuchs
In article [EMAIL PROTECTED],
Tom Lane [EMAIL PROTECTED] writes:

 Shridhar Daithankar [EMAIL PROTECTED] writes:
 I was toying around with idea of converting all the memory related
 parameters in postgresql.conf to kilobytes for simplicity and
 uniformity.

 Why is that a good idea?

Two reasons:
1. Some values are in KB, some in 8 KB
2. I find it easier to calculate in KB

I'd like to see the following:
* If the value is purely numeric, treat it as before (to ensure
  compatibility with older versions)
* If the value is numeric with a prefix of [KMG], interpret it as KB,
  MB, or GB, respectively


---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-06-01 Thread Shridhar Daithankar
On Monday 31 May 2004 22:00, Tom Lane wrote:
 Shridhar Daithankar [EMAIL PROTECTED] writes:
  Right now following are measured in pages
  wal_buffers
  shared_buffers
  effective_cachesize
  while rest of the memory parameters are in kb. I thought being uniform
  would be good. Besides it will make it independent of page size as well.

 It would probably be reasonable to change effective_cache_size, since we
 really do not know what the kernel's unit of buffering is (except on
 Linux, where we *do* know that it ain't 8K ;-)).  Personally I'd opt for
 measuring it in MB not KB, though; that would be a much more convenient
 unit on modern machines.  We could easily make it a float for anyone who
 thinks they know the cache size to sub-MB accuracy.

I have no problems with MB. Only thing I want to see is a bit more user 
friendly and consistent configuration.

Initially I thought of bytes as oracle does but said 'let's be little more 
practical' and put KB..:-) MB is perfectly OK. It would just change the 
multiplier.

And I don't think specifying a float is such a good idea. It is just so 
counter-intuitive. I mean how many people would care for fraction of an MB 
wasted?

 As for the others, I'll side with Emerson: a foolish consistency is the
 hobgoblin of little minds.  We know very well what the unit of
 allocation of those is, and it's pages.  There's no advantage to using
 KB except making it harder to work out what's really happening.  We
 could measure max_connections in KB too if we had a mind to: there's
 a very definite shared-mem cost per connection slot.  Or the FSM
 parameters, or checkpoint_segments, or max_locks_per_transaction.
 The fact that they have quantifiable space costs doesn't mean that space
 is the most useful way to measure them.

Agreed. This is not to change things left and right. It is only to put some 
consistency in place.

 BTW, were you intending to convert KB to NBuffers by charging exactly 8K
 per buffer, or were you intending to allow for the additional shmem
 costs such as buffer headers, per-buffer LWLocks, etc?  If not the
 latter, then what are you really measuring?  For sure it's not shared
 memory size --- charging an artificial number isn't going to help anyone
 who's trying to pick shared_buffers to arrive at a particular actual
 shmem size.  But if it is the latter then it'll become even more
 impossible to tell what's really happening, and we'll be forced to
 invent some way of reading out how many buffers really got allocated.

Well, for the purpose, this is beyond what I am trying to do. As of now there 
is a int value sitting in postgresql.conf which is in page blocks. Now there 
will be a conversion before it is used anytime so that it is bit more user 
friendly. The change should be skin deep so as to be low impact.

As far putting a prefix such as K or M, I don't know much work that would be. 
Does that mean we need to convert shared_buffers to a string parameter and 
parse it? [EMAIL PROTECTED](I would gladly write a real name but alas) 
said yesterday that there is a patch pending with Bruce for such a framework. 
I don't know what and how it does.

For simplicity, I would convert all memory parameters to either KB or MB and 
state so in postgresql.conf. No floats no suffixes. This is my opinion of 
course. Any suggestions are always welcome..

Actually I need to find out few more things about it. It is not as simple as 
adding a assign_hook. When I tried to initdb with changes, it demanded 64MB 
of shared buffers which I (now) think that somewhere NBuffers are used before 
postgresql.conf is parsed. So 8192*8000=64MB. But this is just guesswork. 
Haven't looked in it there.

If this seems reasonably OK, then I would spend some more time on it. We would 
need quite some documentation update then.

 So I disagree with the premise.  Measuring these things in KB is not an
 improvement.

As I said, KBs or MBs is not the issue. Not having it in terms of pagesize is.

 Shridhar


---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-06-01 Thread Shridhar Daithankar
On Tuesday 01 June 2004 14:12, Shridhar Daithankar wrote:
 Actually I need to find out few more things about it. It is not as simple
 as adding a assign_hook. When I tried to initdb with changes, it demanded
 64MB of shared buffers which I (now) think that somewhere NBuffers are used
 before postgresql.conf is parsed. So 8192*8000=64MB. But this is just
 guesswork. Haven't looked in it there.

Found  it. Following is the code that is causing problem.

guc.c:2998
---
if (conf-assign_hook)
if (!(*conf-assign_hook) (newval, changeVal, 
source))
{
ereport(elevel,

(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
 errmsg(invalid value 
for parameter \%s\: %d,
name, 
newval)));
return false;
}

if (changeVal || makeDefault)
{
if (changeVal)
{
*conf-variable = newval;
conf-gen.source = source;
}
---

So even if assign_hook is executed, the value of variable is overwritten in 
next step which nullifies any factoring/change in value done in assign hook.

I find this as a convention at many other place at guc.c. Call assign_hook and 
the overwrite the value. So is assign_hook called only to validate the value?  
How do I modify the value of the variable without getting specific?

I tried 

if (changeVal  !(conf-assign_hook))

and it worked. However that is just for int variables. I am not sure if that 
is a design decision. What should I do?

Regards,
 Shridhar

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-06-01 Thread Josh Berkus
Shridhar, Tom,

As long as we're messing around with PostgreSQL.conf, I propose that we 
comment the file much more thouroughly -- in the style of Apache's httpd.conf 
and our own pg_hba.conf (though maybe not quite as long as hba).   Someone 
proposed this for 7.4 and we ran out of time, and as I've thought about the 
idea over the last 6 months I've come to like it.

I'm happy to do the work for this ... it should be relatively easy if 
everyone's Doc patches are up to date.I can't see any drawback to it; is 
there something I'm missing?

-- 
-Josh Berkus
 Aglio Database Solutions
 San Francisco


---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-06-01 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes:
 As long as we're messing around with PostgreSQL.conf, I propose that we 
 comment the file much more thouroughly -- in the style of Apache's httpd.conf
 and our own pg_hba.conf (though maybe not quite as long as hba).

ISTM we had decided that putting vast amounts of documentation into the
file comments was exactly the thing *not* to do.  It duplicates the SGML
documentation, thereby doubling the maintenance effort, to very little
purpose.  pg_hba's comments in particular had gotten quite out of hand
at one point, and have been pruned back severely.  Let's not follow the
same wheel of fate for postgresql.conf.

regards, tom lane

---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-06-01 Thread Neil Conway
Tom Lane wrote:
ISTM we had decided that putting vast amounts of documentation into the
file comments was exactly the thing *not* to do.  It duplicates the SGML
documentation, thereby doubling the maintenance effort, to very little
purpose.
I agree. If people really think that adding more comments to 
pg_hba.conf is a good idea, I think the best way to do that is to 
automatically extract that information from the main SGML docs.

-Neil
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-06-01 Thread Bruce Momjian
Neil Conway wrote:
 Tom Lane wrote:
  ISTM we had decided that putting vast amounts of documentation into the
  file comments was exactly the thing *not* to do.  It duplicates the SGML
  documentation, thereby doubling the maintenance effort, to very little
  purpose.
 
 I agree. If people really think that adding more comments to 
 pg_hba.conf is a good idea, I think the best way to do that is to 
 automatically extract that information from the main SGML docs.

And if folks want more info in postgresql.conf, we can perhaps pull them
from the docs too, or at least the descriptions we added.

Also, I it seems postgres --describe-config isn't working.  It outputs
nothing here.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-05-31 Thread Shridhar Daithankar
On Sunday 30 May 2004 21:33, Tom Lane wrote:
 Shridhar Daithankar [EMAIL PROTECTED] writes:
  I was toying around with idea of converting all the memory related
  parameters in postgresql.conf to kilobytes for simplicity and
  uniformity.

 Why is that a good idea?

Right now following are measured in pages
wal_buffers
shared_buffers 
effective_cachesize 

Following are in kbs
work_mem 
maintenance_work_mem 
max_stack_depth 

while rest of the memory parameters are in kb. I thought being uniform would 
be good. Besides it will make it independent of page size as well.

 Shridhar

---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-05-31 Thread Bruce Momjian
Shridhar Daithankar wrote:
 On Sunday 30 May 2004 21:33, Tom Lane wrote:
  Shridhar Daithankar [EMAIL PROTECTED] writes:
   I was toying around with idea of converting all the memory related
   parameters in postgresql.conf to kilobytes for simplicity and
   uniformity.
 
  Why is that a good idea?
 
 Right now following are measured in pages
 wal_buffers
 shared_buffers 
 effective_cachesize 
 
 Following are in kbs
 work_mem 
 maintenance_work_mem 
 max_stack_depth 
 
 while rest of the memory parameters are in kb. I thought being uniform would 
 be good. Besides it will make it independent of page size as well.

Sounds good to me.  How are you handling cases where the value has to be
a multiple of page size --- rounding or throwing an error?

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-05-31 Thread Shridhar Daithankar
On Monday 31 May 2004 18:41, Bruce Momjian wrote:
 Shridhar Daithankar wrote:
  On Sunday 30 May 2004 21:33, Tom Lane wrote:
   Shridhar Daithankar [EMAIL PROTECTED] writes:
I was toying around with idea of converting all the memory related
parameters in postgresql.conf to kilobytes for simplicity and
uniformity.
  
   Why is that a good idea?
 
  Right now following are measured in pages
  wal_buffers
  shared_buffers
  effective_cachesize
 
  Following are in kbs
  work_mem
  maintenance_work_mem
  max_stack_depth
 
  while rest of the memory parameters are in kb. I thought being uniform
  would be good. Besides it will make it independent of page size as well.

 Sounds good to me.  How are you handling cases where the value has to be
 a multiple of page size --- rounding or throwing an error?

The patch rounds it down owing to assignment of possible real number to 
integer.  but I didn't imagine of this requirement, to be honest. 

This seems to be better behavior than throwing an error.

 Shridhar

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-05-31 Thread Tom Lane
Shridhar Daithankar [EMAIL PROTECTED] writes:
 Right now following are measured in pages
 wal_buffers
 shared_buffers 
 effective_cachesize 
 while rest of the memory parameters are in kb. I thought being uniform would 
 be good. Besides it will make it independent of page size as well.

It would probably be reasonable to change effective_cache_size, since we
really do not know what the kernel's unit of buffering is (except on
Linux, where we *do* know that it ain't 8K ;-)).  Personally I'd opt for
measuring it in MB not KB, though; that would be a much more convenient
unit on modern machines.  We could easily make it a float for anyone who
thinks they know the cache size to sub-MB accuracy.

As for the others, I'll side with Emerson: a foolish consistency is the
hobgoblin of little minds.  We know very well what the unit of
allocation of those is, and it's pages.  There's no advantage to using
KB except making it harder to work out what's really happening.  We
could measure max_connections in KB too if we had a mind to: there's
a very definite shared-mem cost per connection slot.  Or the FSM
parameters, or checkpoint_segments, or max_locks_per_transaction.
The fact that they have quantifiable space costs doesn't mean that space
is the most useful way to measure them.

BTW, were you intending to convert KB to NBuffers by charging exactly 8K
per buffer, or were you intending to allow for the additional shmem
costs such as buffer headers, per-buffer LWLocks, etc?  If not the
latter, then what are you really measuring?  For sure it's not shared
memory size --- charging an artificial number isn't going to help anyone
who's trying to pick shared_buffers to arrive at a particular actual
shmem size.  But if it is the latter then it'll become even more
impossible to tell what's really happening, and we'll be forced to
invent some way of reading out how many buffers really got allocated.

So I disagree with the premise.  Measuring these things in KB is not an
improvement.

regards, tom lane

---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-05-31 Thread Bruce Momjian
Tom Lane wrote:
 Shridhar Daithankar [EMAIL PROTECTED] writes:
  Right now following are measured in pages
  wal_buffers
  shared_buffers 
  effective_cachesize 
  while rest of the memory parameters are in kb. I thought being uniform would 
  be good. Besides it will make it independent of page size as well.
 
 It would probably be reasonable to change effective_cache_size, since we
 really do not know what the kernel's unit of buffering is (except on
 Linux, where we *do* know that it ain't 8K ;-)).  Personally I'd opt for
 measuring it in MB not KB, though; that would be a much more convenient
 unit on modern machines.  We could easily make it a float for anyone who
 thinks they know the cache size to sub-MB accuracy.

I thought the idea was that you could put 'm', 'k', or maybe 'p' after
the value to specify the units.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-05-31 Thread Neil Conway
Tom Lane wrote:
So I disagree with the premise.  Measuring these things in KB is not an
improvement.
I agree, although I think changing effective_cache_size to be 
measured in KB/MB is worth doing.

-Neil
---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
 joining column's datatypes do not match


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-05-31 Thread Greg Stark
Neil Conway [EMAIL PROTECTED] writes:

 Tom Lane wrote:
  So I disagree with the premise.  Measuring these things in KB is not an
  improvement.
 
 I agree, although I think changing effective_cache_size to be measured in KB/MB
 is worth doing.

I have to say as a user the parameters that are measured in postgres blocks
are really annoying and confusing. Really really annoying and confusing.

If someone's playing with this I would suggest they should work something like
dd parameters and take a unit. So you could specify effective_cache=500k or
effective_cache=1M or effective_cache=1G or whatever unit you wanted. 

And this should be true for _all_ parameters that measure space. Consistency
makes it much much easier for people to learn a new system.

-- 
greg


---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-05-30 Thread Tom Lane
Shridhar Daithankar [EMAIL PROTECTED] writes:
 I was toying around with idea of converting all the memory related
 parameters in postgresql.conf to kilobytes for simplicity and
 uniformity.

Why is that a good idea?

regards, tom lane

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [HACKERS] Converting postgresql.conf parameters to kilobytes

2004-05-30 Thread pgsql
 Hi,

 I was toying around with idea of converting all the memory related
 parameters
 in postgresql.conf to kilobytes for simplicity and uniformity.

 Attached is a proof of concept patch that converts shared_buffers to
 kilobytes
 using assign_hook.

 It compiled all-right but I experienced a strange behavior. At the time of
 initdb, it demanded 69MB of shared memory. I had to bump up SHMMAX from
 32MB
 to 128MB to get initdb through. Certainly I did something wrong somewhere
 but
 I don't know what and where. This is linux 2.6.4.

 The postgresql.conf is properly altered and shows 8000(Though the
 description
 around is no longer in sync.)

 I also don't know where to put the assign_hook. I have put in guc.c for
 the
 time being. Only other int hook I found was assign_max_stack_depth which
 is
 in postgres.c

 Any comments/pointers?

 Regards,
  Shridhar

Changing the format of numbers is tricky. I wouldn't change it, I'd allow
for a suffix, i.e. 1024K or 1M

Additionally, the configuration patch I submited and I think Bruce has
already or will merge, allows for a configuration function. A specific
keyword can be handled by a handler function. This is how I got include
to work, but I would modify *all* the number handlers in GUC to accept the
suffix rather than explicitly change anything that currently works. It is
more consistent, more readble i.e. 128 vs 128M.



---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster