Re: [HACKERS] GUC with units, details
Tom Lane wrote: If that actually worked, it'd be one thing, but it doesn't work and isn't going to do so in 8.2. So I think people will indeed be trying to use setting || unit for display purposes. In any case 8kB isn't a valid unit. I thought we set SHOW ALL aside for display purposes and pg_settings for processing purposes? -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] GUC with units, details
Ühel kenal päeval, N, 2006-07-27 kell 01:03, kirjutas Tom Lane: Michael Glaesemann [EMAIL PROTECTED] writes: I've seen this mentioned a couple of times. I'm not nearly as familiar with these settings as I should be, but it seems to me that if the memory size *does* need to be a integral multiple of page size, e.g., n * page_size = memory_size, why isn't that memory configured as the integer n rather than memory_size? It is. For instance shared_buffers is configured as the number of buffers. What we're talking about here is ways to specify the intended usage with other units (eg I want N megabytes of shared buffers) but that's not going to magically let you allocate half a shared buffer. What are the plans for SHOW command ? Will it show actual number of buffers allocated, original number requested or actual amount allocated in units requested ? Or some combination of above ? Peter's not said exactly how he plans to deal with this, but I suppose it'll round off one way or the other ... regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings -- Hannu Krosing Database Architect Skype Technologies OÜ Akadeemia tee 21 F, Tallinn, 12618, Estonia Skype me: callto:hkrosing Get Skype for free: http://www.skype.com ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] GUC with units, details
Peter Eisentraut [EMAIL PROTECTED] writes: Tom Lane wrote: If that actually worked, it'd be one thing, but it doesn't work and isn't going to do so in 8.2. So I think people will indeed be trying to use setting || unit for display purposes. In any case 8kB isn't a valid unit. I thought we set SHOW ALL aside for display purposes and pg_settings for processing purposes? Right, but it's difficult to get at the result of SHOW from SQL. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] GUC with units, details
On Fri, Jul 28, 2006 at 01:03:00AM +0200, Peter Eisentraut wrote: Accepting page (or block?) as a unit might be a reasonable You hit on something that's always irked me a bit... we tend to toss out 'page' and 'block' (and sometimes even 'buffer') randomly when referring to different things that are keyed to BLCKSZ; perhaps we should pick one as the standard? I know all of us know what we're talking about, but I suspect this could be confusing to users. -- Jim C. Nasby, Sr. Engineering Consultant [EMAIL PROTECTED] Pervasive Software http://pervasive.comwork: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461 ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] GUC with units, details
On Jul 27, 2006, at 14:03 , Tom Lane wrote: What we're talking about here is ways to specify the intended usage with other units (eg I want N megabytes of shared buffers) but that's not going to magically let you allocate half a shared buffer. Peter's not said exactly how he plans to deal with this, but I suppose it'll round off one way or the other ... Thanks, Tom. That make sense to me, and helps me better understand the motivation for the patch. I was a bit thrown off by comments such as this one by Peter [1]: I'd imagine that one of the first things someone will want to try is something like SET work_mem TO '10MB', which will fail or misbehave because 1000 bytes do not divide up into chunks of 1024 bytes. Who wants to explain to users that they have to write '10MiB'? Granted, the K=1024/1000 issue will affect how 10MB is interpreted, but if it's rounded to the nearest whole page or buffer, it shouldn't fail or misbehave, I'd think. Michael Glaesemann grzm seespotcode net [1](http://archives.postgresql.org/pgsql-hackers/2006-07/msg01249.php) ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] GUC with units, details
Tom Lane wrote: It is. For instance shared_buffers is configured as the number of buffers. What we're talking about here is ways to specify the intended usage with other units (eg I want N megabytes of shared buffers) but that's not going to magically let you allocate half a shared buffer. Peter's not said exactly how he plans to deal with this, but I suppose it'll round off one way or the other ... It'll get truncated by integer division. I wouldn't mind if someone proposed a patch to create a warning or error in this case, but I wanted to keep the initial version simple. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] GUC with units, details
Bort, Paul wrote: I still think it would be a good idea to use the standard, and that this is an opportunity to do so. I have committed it using the 1024 multiplier, but if you want to propose changing all uses of kB, MB, and GB in PostgreSQL to the other system, now would be the time to do it. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] GUC with units, details
Peter Eisentraut [EMAIL PROTECTED] writes: Tom Lane wrote: Peter's not said exactly how he plans to deal with this, but I suppose it'll round off one way or the other ... It'll get truncated by integer division. I wouldn't mind if someone proposed a patch to create a warning or error in this case, but I wanted to keep the initial version simple. I'd recommend against that. Apple recently changed OS X so that it rejects SHMMAX settings that aren't an exact multiple of something-or-other, and I've found that to be a *serious* PITA. Of course part of the problem is that there's no helpful message, but it's still a big loss from a usability standpoint, and quite unnecessary (every other Unix seems willing to round off...) One thought is that maybe we should round up not down? I'm having a hard time making a specific case either way, though. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] GUC with units, details
Peter Eisentraut wrote: I have committed it using the 1024 multiplier, but if you want to propose changing all uses of kB, MB, and GB in PostgreSQL to the other system, now would be the time to do it. I think it would be a good idea. I know I don't have time to do it for 8.2. I get the feeling that there isn't enough interest on -hackers that anyone else is likely to do this, so I guess it waits until later. ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] GUC with units, details
Bort, Paul wrote: Peter Eisentraut wrote: I have committed it using the 1024 multiplier, but if you want to propose changing all uses of kB, MB, and GB in PostgreSQL to the other system, now would be the time to do it. I think it would be a good idea. I know I don't have time to do it for 8.2. I get the feeling that there isn't enough interest on -hackers that anyone else is likely to do this, so I guess it waits until later. Which means it'll never be done, because as soon as we have one release out with the current behavior, it's unlikely that we'll want to change it afterwards. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] GUC with units, details
On Jul 27, 2006, at 9:16 AM, Bort, Paul wrote: Peter Eisentraut wrote: I have committed it using the 1024 multiplier, but if you want to propose changing all uses of kB, MB, and GB in PostgreSQL to the other system, now would be the time to do it. I think it would be a good idea. I know I don't have time to do it for 8.2. I get the feeling that there isn't enough interest on -hackers that anyone else is likely to do this, so I guess it waits until later. First, when it comes to page values, perhaps we should allow pages/ blocks as a valid unit. That would allow people who want to to specify things in pages and still be explicit about what they mean. Second, lack of interest or no, I feel forcing specific casing/SI units is a flat-out bad idea. The truth is, virtually no one, even highly technical people, ever picks nits between kB vs KiB vs KB. People will sometimes use b for bits and B for bytes, but in the case of postgresql.conf it's very clear that everything is talking about bytes and not bits. Forcing people to use a specific casing scheme is just going to lead to confusion and user frustration. If there's not a very solid *functional* argument for it, we shouldn't do it. Wanting to enforce a convention that people rarely use isn't a good reason. -- Jim C. Nasby, Sr. Engineering Consultant [EMAIL PROTECTED] Pervasive Software http://pervasive.comwork: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461 ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] GUC with units, details
[snip] Forcing people to use a specific casing scheme is just going to lead to confusion and user frustration. If there's not a very solid I guess nobody will force people to use the units at all. *functional* argument for it, we shouldn't do it. Wanting to enforce a convention that people rarely use isn't a good reason. But if you implement a new feature, history shows that it will stay like that forever. So if in 5 years everybody will use the ISO stuff, and postgres will want to do the same, then the users you don't want to confuse now will be forced to change their config files or be completely confused. Or it will be as with everything else, an early arbitrary decision sets everything in stone. And I do find confusing all these ambiguous meanings of K,G etc., and I think ISO is the right way to clear out the confusion at the cost of some inconvenience until the users get used to it. For postgres that would mean no user resistance anyway, as the possibility of specifying the unit is new, so who knows about it must have read the docs first, and the docs must specify the units you can use. Just my 2c. Cheers, Csaba. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] GUC with units, details
Tom Lane wrote: Peter Eisentraut [EMAIL PROTECTED] writes: Tom Lane wrote: Peter's not said exactly how he plans to deal with this, but I suppose it'll round off one way or the other ... It'll get truncated by integer division. I wouldn't mind if someone proposed a patch to create a warning or error in this case, but I wanted to keep the initial version simple. I'd recommend against that. Apple recently changed OS X so that it rejects SHMMAX settings that aren't an exact multiple of something-or-other, and I've found that to be a *serious* PITA. Of course part of the problem is that there's no helpful message, but it's still a big loss from a usability standpoint, and quite unnecessary (every other Unix seems willing to round off...) One thought is that maybe we should round up not down? I'm having a hard time making a specific case either way, though. Rounding up would have the advantage that you could just specify 0 in the config file, and have postgres use the smallest value possible. greetings, Florian Pflug ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] GUC with units, details
Jim Nasby wrote: The truth is, virtually no one, even highly technical people, ever picks nits between kB vs KiB vs KB. The question isn't so much whether to allow KiB and such -- that would obviously be trivial. The question is whether we want to have kB mean 1000 bytes instead of 1024 bytes. In my mind, that goes against current practice. The only argument raised in favor was that international standards require such use. I'm as much a fan of measurement standards as anyone, but I'm also a practitioner of current practice. This consideration would become much more interesting if *any* software product actually made use of this newer proposed convention, but so far I haven't seen one yet. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] GUC with units, details
Florian G. Pflug wrote: Rounding up would have the advantage that you could just specify 0 in the config file, and have postgres use the smallest value possible. In most algebras, dividing zero by something is still zero, so there'd be no need to round anything. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] GUC with units, details
Peter Eisentraut wrote: Florian G. Pflug wrote: Rounding up would have the advantage that you could just specify 0 in the config file, and have postgres use the smallest value possible. In most algebras, dividing zero by something is still zero, so there'd be no need to round anything. I guess a clicked the send button a little too fast. You're right, of course. greetings, Florian Pflug ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] GUC with units, details
On Thu, 2006-07-27 at 17:57, Peter Eisentraut wrote: Florian G. Pflug wrote: Rounding up would have the advantage that you could just specify 0 in the config file, and have postgres use the smallest value possible. In most algebras, dividing zero by something is still zero, so there'd be no need to round anything. I think he was refering to silently apply the minimum allowed if the value is less than that... a lot of the settings have a minimum allowable value. The question is if this can qualify as rounding :-) Cheers, Csaba. ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] GUC with units, details
Peter Eisentraut wrote: This consideration would become much more interesting if *any* software product actually made use of this newer proposed convention, but so far I haven't seen one yet. So we'll look at it when Oracle does it? I think we should be leading this charge, rather than following. Regards, Paul Bort ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] GUC with units, details
Bort, Paul wrote: So we'll look at it when Oracle does it? I didn't say Oracle, I said anyone. It could be Microsoft or Samba or Red Hat or NetBSD or my VoIP phone. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] GUC with units, details
Peter Eisentraut wrote: I didn't say Oracle, I said anyone. It could be Microsoft or Samba or Red Hat or NetBSD or my VoIP phone. OK, I did some further digging, and (http://members.optus.net/alexey/prefBin.xhtml) has a list at the end of the page of software that the author claims use the standard. From that list: Azureus - Has this (http://www.azureuswiki.com/index.php/Data_units) entry in the FAQ. All of the screen shots show 'kB/s'. but since they're referring to bandwidth, it would be reasonable to conclude that they intend decimal units. Lynx - The documentation (http://lynx.isc.org/current/lynx2-8-6/lynx_help/Lynx_users_guide.html) consistently uses KiB instead of KB. FreeDOS-32 - Their standards page (http://freedos-32.sourceforge.net/showdoc.php?page=standards) states that they comply with this standard. Regards, Paul Bort ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] GUC with units, details
Peter Eisentraut wrote: Jim Nasby wrote: The truth is, virtually no one, even highly technical people, ever picks nits between kB vs KiB vs KB. The question isn't so much whether to allow KiB and such -- that would obviously be trivial. The question is whether we want to have kB mean 1000 bytes instead of 1024 bytes. Would it satisfy everyone if the documentation states that specifying a value of N kB means that *at least* N 1000 bytes are allocated; and perhaps even documenting that in the current implementation it happens to be 24 extra bytes. In my mind, that goes against current practice. The only argument raised in favor was that international standards require such use. I'm as much a fan of measurement standards as anyone, but I'm also a practitioner of current practice. With the spec reading at least N KB, even the most pedantic spec reader can't complain, because it is true. ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] GUC with units, details
On Thu, Jul 27, 2006 at 05:56:15PM +0200, Peter Eisentraut wrote: Jim Nasby wrote: The truth is, virtually no one, even highly technical people, ever picks nits between kB vs KiB vs KB. The question isn't so much whether to allow KiB and such -- that would obviously be trivial. The question is whether we want to have kB mean 1000 bytes instead of 1024 bytes. The things I wonder about are that memory usage programs use K=1024, so will we be getting questions like: I wrote 128KB in the config file yet it's only using 126KB according to program Y, why? Secondly, if someone wants exactly 1,000,000 bytes, that's easy to type, but if someone wants exactly 1024*1024 bytes, they need to pull out a calculator. Ofcourse we could use KiB, MiB, etc.. My main problem with the kibi, mibi, etc is that they're basically unknown. I polled some random (non-computer) user and they'd never heard of it. It's not taught in schools which pretty much means it's never going to happen. When manufacturers like Dell start using KiB in their glossy magazines, maybe it's time to look into it. Have a nice day, -- Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/ From each according to his ability. To each according to his ability to litigate. signature.asc Description: Digital signature
Re: [HACKERS] GUC with units, details
Jim Nasby [EMAIL PROTECTED] writes: First, when it comes to page values, perhaps we should allow pages/ blocks as a valid unit. That would allow people who want to to specify things in pages and still be explicit about what they mean. Second, lack of interest or no, I feel forcing specific casing/SI units is a flat-out bad idea. +1 on both of those. I think that pg_settings should actually show pages as the native unit for shared_buffers et al. The current 8kb display isn't a valid unit --- consider what happens if a program does select setting || unit from pg_settings ... You'll get a completely misleading result. And if you want to know how big a page is, there's already a way to find that out, ie, SHOW block_size, so displaying the unit this way isn't adding any functionality. Lastly, shouldn't unit read out as null where it's not applicable? It looks like it's null in some of the rows but empty string elsewhere. That's just weird. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] GUC with units, details
Peter Eisentraut [EMAIL PROTECTED] writes: ... The question is whether we want to have kB mean 1000 bytes instead of 1024 bytes. In my mind, that goes against current practice. I concur. Most of the places where we are using these units, they are for specifying memory sizes, and *everyone* does memory sizes in binary. The argument from disk drive makers' obviously marketing-driven decision to use decimal doesn't impress me. And the argument from standards that are just about universally ignored doesn't impress me either ... regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] GUC with units, details
Martijn van Oosterhout kleptog@svana.org writes: My main problem with the kibi, mibi, etc is that they're basically unknown. Frankly the main problem is that they're idiotic. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] GUC with units, details
Tom Lane wrote: +1 on both of those. I think that pg_settings should actually show pages as the native unit for shared_buffers et al. The current 8kb display isn't a valid unit --- consider what happens if a program does select setting || unit from pg_settings ... Physicists know that the proper way to do that calculation is setting * unit not setting || unit I was toying for a moment with putting '1' into the column when the quantity is dimensionless. That would have been a fun debate. I realize that it's not all that consistent or easy to process now. Accepting page (or block?) as a unit might be a reasonable addition. Then again, I'm not sure why anyone would want to do setting || unit in the first place, because setting by itself will give you the right quantity to feed back into the system. Lastly, shouldn't unit read out as null where it's not applicable? It looks like it's null in some of the rows but empty string elsewhere. That's just weird. It's null when units do not apply (for booleans, strings for conceptual reasons, for reals for implementation reasons), and it's empty when the unit is known to be nothing. I don't contest that that's weird but I think it's the proper way to express the information (short of putting in '1' again). (I'm thinking something like a units type that is occasionally discussed would be a good addition, so you can do calculations like setting * unit.) -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] GUC with units, details
Peter Eisentraut [EMAIL PROTECTED] writes: Tom Lane wrote: +1 on both of those. I think that pg_settings should actually show pages as the native unit for shared_buffers et al. The current 8kb display isn't a valid unit --- consider what happens if a program does select setting || unit from pg_settings ... Physicists know that the proper way to do that calculation is setting * unit not setting || unit If that actually worked, it'd be one thing, but it doesn't work and isn't going to do so in 8.2. So I think people will indeed be trying to use setting || unit for display purposes. In any case 8kB isn't a valid unit. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] GUC with units, details
Neil Conway wrote: On Tue, 2006-07-25 at 19:00 -0400, Tom Lane wrote: Maybe I'm missing something, but I thought it was fairly common to use k for 1000, K for 1024, etc (mnemonic: upper case for the larger multiplier). Well, that only works for K vs. k: the SI prefix for mega is M (meaning 10^6), not m. Similarly for G. Indeed. The k vs K idea is an excuse for not wanting to side with either camp, but it does not scale. Why it is impractical to use the IEC prefixes? I'd imagine that one of the first things someone will want to try is something like SET work_mem TO '10MB', which will fail or misbehave because 1000 bytes do not divide up into chunks of 1024 bytes. Who wants to explain to users that they have to write '10MiB'? Since about forever, PostgreSQL has used kB, MB, GB to describe memory allocation. If we want to change that, we ought to do it across the board. But that's a big board. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] GUC with units, details
On Wed, 2006-07-26 at 08:12 +0200, Peter Eisentraut wrote: Neil Conway wrote: On Tue, 2006-07-25 at 19:00 -0400, Tom Lane wrote: Maybe I'm missing something, but I thought it was fairly common to use k for 1000, K for 1024, etc (mnemonic: upper case for the larger multiplier). Well, that only works for K vs. k: the SI prefix for mega is M (meaning 10^6), not m. Similarly for G. Indeed. The k vs K idea is an excuse for not wanting to side with either camp, but it does not scale. Why it is impractical to use the IEC prefixes? I'd imagine that one of the first things someone will want to try is something like SET work_mem TO '10MB', which will fail or misbehave because 1000 bytes do not divide up into chunks of 1024 bytes. Who wants to explain to users that they have to write '10MiB'? Since about forever, PostgreSQL has used kB, MB, GB to describe memory allocation. If we want to change that, we ought to do it across the board. But that's a big board. Neil is right: K, M, G are the correct SI terms, however, I don't see any value in using that here. Nobody is suggesting we encourage or even allow people to write max_fsm_pages = 10M rather than 1000, so we don't ever need to say that K = 1000, AFAICS. I think we are safe to assume that kB = KB = kb = Kb = 1024 bytes mB = MB = mb = Mb = 1024 * 1024 bytes gB = GB = gb = Gb = 1024 * 1024 * 1024 bytes There's no value in forcing the use of specific case and it will be just confusing for people. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] GUC with units, details
Simon Riggs wrote:
don't ever need to say that K = 1000, AFAICS. I think we are safe to
assume that
kB = KB = kb = Kb = 1024 bytes
mB = MB = mb = Mb = 1024 * 1024 bytes
gB = GB = gb = Gb = 1024 * 1024 * 1024 bytes
There's no value in forcing the use of specific case and it will be just
confusing for people.
It's fairly common to use 'b' for 'bits' and 'B' for 'bytes'. My suggestion would be to be
much more restrictive and avoid small caps:
KB = 1024 bytes
MB = 1024 KB
GB = 1024 KB
TB = 1024 GB
Although I don't expect to see bit-rates or fractions ('m' == 'milli') in GUC, it might be
good to use consistent units everywhere.
Regards,
Thomas Hallgren
---(end of broadcast)---
TIP 6: explain analyze is your friend
Re: [HACKERS] GUC with units, details
Simon Riggs wrote: There's no value in forcing the use of specific case and it will be just confusing for people. The issue was not the case of the units, but people were suggesting that we should enforce the use of kiB, MiB, and GiB. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] GUC with units, details
Tom Lane wrote: That seems OK for SHOW, which is mainly intended for human consumption, but what will you do with pg_settings? For programmatic use I think we want more predictable behavior. I'd think that a program would not care. Or do you want a units-free display that can be parsed as integer? Do we want to introduce a difference between pg_settings and SHOW ALL? (Is there one already?) -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] GUC with units, details
Peter Eisentraut [EMAIL PROTECTED] writes: Tom Lane wrote: That seems OK for SHOW, which is mainly intended for human consumption, but what will you do with pg_settings? For programmatic use I think we want more predictable behavior. I'd think that a program would not care. Or do you want a units-free display that can be parsed as integer? Yeah. If the value might be shown as either 99kB or 9MB then a program *must* have a pretty complete understanding of the units system to make sense of it at all. Furthermore this is not backwards compatible --- it'll break any existing code that inspects pg_settings values. I suggest that the values column should continue to display exactly as it does today (ie, the integer value in the var's native units) and we just add a column saying what the native units are. Do we want to introduce a difference between pg_settings and SHOW ALL? Yup, I think that's the lesser of the evils. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] GUC with units, details
Peter Eisentraut wrote: I'd imagine that one of the first things someone will want to try is something like SET work_mem TO '10MB', which will fail or misbehave because 1000 bytes do not divide up into chunks of 1024 bytes. Who wants to explain to users that they have to write '10MiB'? How about this: INFO: Your setting was converted to IEC standard binary units. Use KiB, MiB, and GiB to avoid this warning. Since about forever, PostgreSQL has used kB, MB, GB to describe memory allocation. If we want to change that, we ought to do it across the board. But that's a big board. The standard hasn't been around forever; some incarnation of PostgreSQL certainly pre-dates it. But it was created to reduce confusion between binary and decimal units. The Linux kernel changed to the standard years ago. And that's just a few more lines of code than PostgreSQL. ( http://kerneltrap.org/node/340 and others ) Regards, Paul Bort ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] GUC with units, details
Bort, Paul wrote: The Linux kernel changed to the standard years ago. And that's just a few more lines of code than PostgreSQL. ( http://kerneltrap.org/node/340 and others ) For your entertainment, here are the usage numbers from the linux-2.6.17 kernel: kilobyte (-i) 82 kibibyte (-i) 2 megabyte (-i) 98 mebibyte (-i) 0 gigabyte (-i) 32 gibibyte (-i) 0 KB 1151 kB 407 KiB 181 MB 3830 MiB 298 GB 815 GiB 17 So I remain unconvinced. Of course, your general point is a good one. If there are actually systems using this, it might be worth considering. But if not, then we're just going to confuse people. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] GUC with units, details
Peter Eisentraut politely corrected: For your entertainment, here are the usage numbers from the linux-2.6.17 kernel: kilobyte (-i) 82 kibibyte (-i) 2 megabyte (-i) 98 mebibyte (-i) 0 gigabyte (-i) 32 gibibyte (-i) 0 KB1151 kB407 KiB 181 MB3830 MiB 298 GB815 GiB 17 Thanks for the info. I had seen several articles on it, and shot my mouth off without double-checking. My apologies. I still think it would be a good idea to use the standard, and that this is an opportunity to do so. Regards, Paul Bort ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] GUC with units, details
Peter Eisentraut wrote: Bort, Paul wrote: The Linux kernel changed to the standard years ago. And that's just a few more lines of code than PostgreSQL. ( http://kerneltrap.org/node/340 and others ) For your entertainment, here are the usage numbers from the linux-2.6.17 kernel: kilobyte (-i) 82 kibibyte (-i) 2 megabyte (-i) 98 mebibyte (-i) 0 gigabyte (-i) 32 gibibyte (-i) 0 KB 1151 kB 407 KiB 181 MB 3830 MiB 298 GB 815 GiB 17 So I remain unconvinced. Of course, your general point is a good one. If there are actually systems using this, it might be worth considering. But if not, then we're just going to confuse people. Is it worth bothering about the small deviation, if 1 was meant, but 10k gives 10240 buffers? Isn't it quite common that systems round config values to the next sensible value anyway? Regards, Andreas ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] GUC with units, details
On Wed, Jul 26, 2006 at 12:17:00PM -0400, Bort, Paul wrote: Peter Eisentraut wrote: I'd imagine that one of the first things someone will want to try is something like SET work_mem TO '10MB', which will fail or misbehave because 1000 bytes do not divide up into chunks of 1024 bytes. Who wants to explain to users that they have to write '10MiB'? How about this: INFO: Your setting was converted to IEC standard binary units. Use KiB, MiB, and GiB to avoid this warning. That's silly. If you're going to treat KB as 1024 bytes anyway, complaining about it is just being pedantic. The thing is, most memory sizes in postgres need to be some multiple of a page size. You can't have a shared buffers of exactly 10 bytes, while 102400 bytes is possible. When someone has a GB of memory, they really mean a GiB, but no-one bothers to correct them. Is there anywhere in postgres where using K=1000 would be significantly clearer than K=1024? Have a nice day, -- Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/ From each according to his ability. To each according to his ability to litigate. signature.asc Description: Digital signature
Re: [HACKERS] GUC with units, details
Martijn van Oosterhout wrote: How about this: INFO: Your setting was converted to IEC standard binary units. Use KiB, MiB, and GiB to avoid this warning. That's silly. If you're going to treat KB as 1024 bytes anyway, complaining about it is just being pedantic. But after a version or two with warnings, we have grounds to make it an error. I'd rather just go with the standard from day 1 and reject decimal units where they don't make sense, but that seems unlikely. The thing is, most memory sizes in postgres need to be some multiple of a page size. You can't have a shared buffers of exactly 10 bytes, while 102400 bytes is possible. When someone has a GB of memory, they really mean a GiB, but no-one bothers to correct them. And hard drives are just the opposite: a 250GB drive does not have 268,435,456,000 bytes of unformatted space. Is there anywhere in postgres where using K=1000 would be significantly clearer than K=1024? If the unit for a setting is pages, then a value of '1K' could cause some confusion as to whether that's 1,000 or 1,024 pages. Have a nice day, -- Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/ From each according to his ability. To each according to his ability to litigate. ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] GUC with units, details
On Jul 27, 2006, at 6:10 , Martijn van Oosterhout wrote: The thing is, most memory sizes in postgres need to be some multiple of a page size. You can't have a shared buffers of exactly 10 bytes, while 102400 bytes is possible. I've seen this mentioned a couple of times. I'm not nearly as familiar with these settings as I should be, but it seems to me that if the memory size *does* need to be a integral multiple of page size, e.g., n * page_size = memory_size, why isn't that memory configured as the integer n rather than memory_size? Wouldn't this get around the issue altogether? Granted, this is a larger change than allowing units for the values, which I think is a good thing. But it is perhaps shows more clearly the relationship between the different values in postgresql.conf and prevents setting memory sizes that *aren't* multiples of page size. Michael Glaesemann grzm seespotcode net ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] GUC with units, details
Michael Glaesemann [EMAIL PROTECTED] writes: I've seen this mentioned a couple of times. I'm not nearly as familiar with these settings as I should be, but it seems to me that if the memory size *does* need to be a integral multiple of page size, e.g., n * page_size = memory_size, why isn't that memory configured as the integer n rather than memory_size? It is. For instance shared_buffers is configured as the number of buffers. What we're talking about here is ways to specify the intended usage with other units (eg I want N megabytes of shared buffers) but that's not going to magically let you allocate half a shared buffer. Peter's not said exactly how he plans to deal with this, but I suppose it'll round off one way or the other ... regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[HACKERS] GUC with units, details
It seems everyone likes the units, so here are some details of the implementation I have prepared. Memory units are kB, MB, GB. The factor is 1024. Time units are ms, s, min, h, d. I intentionally did not support microseconds because that would make the coding extremely overflow-risky, and the only candidate commit_delay isn't used much. This can be added once int64 support is required. For similar reasons, the unit byte is not supported. The full list of candidates then is: post_auth_delay s deadlock_timeoutms vacuum_cost_delay ms autovacuum_vacuum_cost_delayms statement_timeout ms authentication_timeout s pre_auth_delay s checkpoint_timeout s log_min_duration_statement ms bgwriter_delay ms log_rotation_agemin autovacuum_naptime s tcp_keepalives_idle s tcp_keepalives_interval s shared_buffers 8kB temp_buffers8kB work_memkB maintenance_work_memkB log_rotation_size kB effective_cache_sizekB (pending switch to int) Units can be specified with or without space after the number. In the configuration file, writing a space after the number would require quoting the entire the value, without a space not. With SET of course you have to quote anyway. If you don't specify any unit, you get the behavior from before. Output from SHOW uses the largest unit that fits as long as the number is an integer. (I guess you could change that later to some more complex scheme, but I feel that this is better than what we have.) If the value is zero or negative, no unit is used. (-1 sometimes means off.) The error messages complaining about range violations and similar things should perhaps also be changed to use units. I'm a bit afraid of removing all references to the true internal units of these parameters, because before long someone will see a message I don't like value 123 and they won't know what unit it is. We'll have to deal with those as we go along I guess. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] GUC with units, details
Peter Eisentraut [EMAIL PROTECTED] writes: Output from SHOW uses the largest unit that fits as long as the number is an integer. That seems OK for SHOW, which is mainly intended for human consumption, but what will you do with pg_settings? For programmatic use I think we want more predictable behavior. I'm inclined to suggest adding a column native units to pg_settings, which shows what the underlying units are (ie, the existing interpretations) and then always show the value of a given variable in its native unit. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] GUC with units, details
On Jul 26, 2006, at 6:56 , Peter Eisentraut wrote: Memory units are kB, MB, GB. The factor is 1024. Time units are ms, s, min, h, d. Are units case-sensitive? I've noticed you've been consistent in your capitalization in these posts, so I'm wondering if you're enforcing the same case in postgresql.conf. Michael Glaesemann grzm seespotcode net ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] GUC with units, details
On Jul 26, 2006, at 7:12 , Michael Glaesemann wrote: On Jul 26, 2006, at 6:56 , Peter Eisentraut wrote: Memory units are kB, MB, GB. The factor is 1024. Time units are ms, s, min, h, d. Are units case-sensitive? I've noticed you've been consistent in your capitalization in these posts, so I'm wondering if you're enforcing the same case in postgresql.conf. Sorry for the noise. Didn't see that this was answered previously. Michael Glaesemann grzm seespotcode net ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] GUC with units, details
Peter Eisentraut wrote: Memory units are kB, MB, GB. The factor is 1024. Then shouldn't the factor be 1000? If the factor is 1024, then the units should be KiB, MiB, GiB per IEEE 1541 (http://en.wikipedia.org/wiki/IEEE_1541) and others. I'm not trying to be pedantic, but the general approach with -hackers seems to be towards compliance where practical. Regards, Paul Bort ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] GUC with units, details
Bort, Paul wrote: I'm not trying to be pedantic, but the general approach with -hackers seems to be towards compliance where practical. But in this case it's not practical. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] GUC with units, details
Peter Eisentraut [EMAIL PROTECTED] writes: Bort, Paul wrote: [ 1000 vs 1024 ] But in this case it's not practical. Maybe I'm missing something, but I thought it was fairly common to use k for 1000, K for 1024, etc (mnemonic: upper case for the larger multiplier). So I'd vote for accepting KB rather than kB ... regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] GUC with units, details
On Tue, 2006-07-25 at 19:00 -0400, Tom Lane wrote: Maybe I'm missing something, but I thought it was fairly common to use k for 1000, K for 1024, etc (mnemonic: upper case for the larger multiplier). Well, that only works for K vs. k: the SI prefix for mega is M (meaning 10^6), not m. Similarly for G. Why it is impractical to use the IEC prefixes? -Neil ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
