Re: [HACKERS] Visibility map and freezing
ITAGAKI Takahiro wrote: Gregory Stark st...@enterprisedb.com wrote: I don't think we can perfectly capture the meaning of these GUCs in the name. I think our goal should be to avoid confusion between them. I was thinking it would be clearer if the options which control *when* autovacuum fires off a worker consistently had some action word in them like trigger or start or launch. I think we need more explanations about those variables, not only how to work but also how to tune them. I feel they are un-tunable parameters. Our documentation says: | Larger values of these settings | preserve transactional information longer, while smaller values increase | the number of transactions that can elapse before the table must be | vacuumed again. i.e, we are explaining the variables only as Larger is better, but is it really true? Yes, that is explicitly explained in the docs: The sole disadvantage of increasing varnamevacuum_freeze_table_age/ and varnameautovacuum_freeze_max_age/ is that the filenamepg_clog/ subdirectory of the database cluster will take more space, because it must store the commit status for all transactions back to the varnameautovacuum_freeze_max_age/ horizon. ... - What relation are there between autovacuum_freeze_max_age, vacuum_freeze_min_age and vacuum_freeze_table_age? If we increase one of them, should we also increase the others? Yeah, that's a fair question. I'll try to work a doc patch to explain that better. - Is it ok to increase the variables to maximum values? Are there any trade-off? Disk space. - Are there some conditions where whole-table-scanning vacuum is more effective than vacuums using visibility map? If so, we should switch to full-scan *automatically*, without relying on user configurations. Hmm, the only downside I can see is that skipping a page here and there could defeat the OS read-ahead. Perhaps we should call posix_fadvise(SEQUENTIAL) to compensate. Or, we could modify the logic to only skip pages when there's at least N consecutive pages that can be skipped. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] MemoryContextSwitchTo (Re: [GENERAL] Autovacuum daemon terminated by signal 11)
Tom Lane wrote: Alvaro Herrera alvhe...@commandprompt.com writes: But freed memory is clobbered, so if we were to have an assert that checks the node tag, it should show up. In fact, we do have such an assert, but only for compilers other than GCC, because the inline version of palloc() cannot have it for lack of infrastructure. Well, but production installations don't have either memory clobbering or Asserts, so fooling with that wouldn't have helped anyway. I suspect what really happened here is that the bug was created by some late change during 8.1 development, and nobody ever exercised the anti-wraparound code path after that in an assert-enabled build :-( In a non-assert build there's a fairly good chance that it'd still work because the context header would still be there undamaged. Well, my builds are all assert-enabled, and I still wasn't able to make it crash in any way (the new context being allocated in the same position as the old one is the only explanation I have, but I did not investigate whether that's what happening). Maybe Greg Stark's idea of offsetting pointers returned by palloc could have helped to find the problem from the outset. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] is 8.4 array_agg() supposed to work with array values?
On Tue, Jan 20, 2009 at 5:09 AM, Peter Eisentraut pete...@gmx.net wrote: This would work much better if integer[][] where array of integer[], in the same way as integer[] is array of integer, in the way C deals with arrays. Well the situation in C is substantially more complicated than you make out. I think the analogous situation in C is actually having an array of integer* pointing to separately allocated arrays. There are a number of places where Postgres has to special-case arrays and array references (think of update foo set bar[n] = baz for example) so I don't think it would work to just create the array type for integer[] either. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq WSACleanup is not needed
Bruce Momjian wrote: Andrew Chernow wrote: Bruce Momjian wrote: We could have gone with a more elegant init/uninit solution but there is a history of slow upstream adoption of libpq API changes. If that's the case, adding a connectdb option seems like a good alternative. Orignally suggested here: http://archives.postgresql.org/pgsql-hackers/2009-01/msg01358.php Right, well the big question is how many people are going to use the connection option vs. doing it for everyone automatically. One possible approach might be to do it automatically, and allow a connection option to disable the WSACleanup() call. I think that was the suggestion. Have an option that would disable *both* the startup and the cleanup call, leaving the responsibility to the app. You can do this for SSL today by calling PQinitSSL(). Actually, right now, if you have two libpq connections, and close one, does WSACleanup() get called, and does it affect the existing connection? WSACleanup() gets called, but it has an internal reference count so it does not have any effect on existing connections. //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FWD: Re: Updated backslash consistency patch
Robert Haas wrote: Here is what I hope is a consensus patch. It adds 'A' to show all objects, including system ones. It turns out that this is how 'S' works now in CVS, but 'S' is unclear because it suggests just system objects; 'A' for show 'all' objects seems clearer. I think it's probably fine for S to mean include system objects rather than show only system objects. Everyone should be relatively used to S by now; I think it's less confusing to keep the same letter even if the behavior has been adjusted somewhat. Though others may disagree? OK, so you are saying you like the behavior in current CVS, which uses 'S' as you described. In thinking last night, I am now wondering if a letter is really the right symbol for this. We already have letter flags which control object type selection, but the system table addition is kind of independent of those flags, like '+' now. I am thinking maybe '' is the right symbol rather than 'A' or 'S'. \df and \df '' would stand for and system objects. We could use '*' but that might be confused with a wildcard symbol. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FWD: Re: Updated backslash consistency patch
Bruce Momjian escribió: In thinking last night, I am now wondering if a letter is really the right symbol for this. We already have letter flags which control object type selection, but the system table addition is kind of independent of those flags, like '+' now. I am thinking maybe '' is the right symbol rather than 'A' or 'S'. is probably going to be useful as a modifier when we have concurrent psql, so I suggest you do not use it here. Or maybe not, but in any case see http://archives.postgresql.org/message-id/8204.1207689056%40sss.pgh.pa.us before you use . -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FWD: Re: Updated backslash consistency patch
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 In thinking last night, I am now wondering if a letter is really the right symbol for this. We already have letter flags which control object type selection, but the system table addition is kind of independent of those flags, like '+' now. I am thinking maybe '' is the right symbol rather than 'A' or 'S'. What problem are we trying to solve here that is not already solved by using 'S' for system tables, which has been in use in psql for over a decade now? - -- Greg Sabino Mullane g...@turnstep.com End Point Corporation PGP Key: 0x14964AC8 200901201026 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAkl17psACgkQvJuQZxSWSsjdRACgoAJPOBsWYPf420KLl/7tuPTa By0AoORZBVnszD5SmGUpKvBHR7YFES7r =AILx -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FWD: Re: Updated backslash consistency patch
Le mardi 20 janvier 2009, Bruce Momjian a écrit : Robert Haas wrote: Here is what I hope is a consensus patch. It adds 'A' to show all objects, including system ones. It turns out that this is how 'S' works now in CVS, but 'S' is unclear because it suggests just system objects; 'A' for show 'all' objects seems clearer. I think it's probably fine for S to mean include system objects rather than show only system objects. Everyone should be relatively used to S by now; I think it's less confusing to keep the same letter even if the behavior has been adjusted somewhat. Though others may disagree? I still think that given a pattern, psql commands should simply mimic whatever is the server way of using search_path. I'd really like \df foo and \d foo to follow the same rules as my production queries wrt to how to find objects when I'm too lazy to schema qualify their name. Now, it's been advocated for the sake of simplicity to have with-pattern and without-pattern options behave roughly the same way. I can't find it difficult to explain the two behaviours here, all the more when looking at current \d and \dt differences. What I'd like to propose is for \df without pattern to behave exactly like \df with pattern, *including* wrt to ordering the output. Functions listed in search_path order, pg_catalog implicitly part of it, but as its *last* element. Or whatever server object lookup code sayth. '' would stand for and system objects. We could use '*' but that might be confused with a wildcard symbol. If you insist on treating system object differently than server itself, I'd prefer for the client not to have me use well known non-alpha keys, which a lot of shells associate to e.g. jobs processing (bash, csh, lftp, some more I'm sure). Regards, -- dim signature.asc Description: This is a digitally signed message part.
Re: [HACKERS] FWD: Re: Updated backslash consistency patch
Bruce, * Bruce Momjian (br...@momjian.us) wrote: In thinking last night, I am now wondering if a letter is really the right symbol for this. We already have letter flags which control object type selection, but the system table addition is kind of independent of those flags, like '+' now. I am thinking maybe '' is the right symbol rather than 'A' or 'S'. This really feels like over-engineering at this point. I don't particularly like '', and 'S' has been in use like this for quite some time. Stephen signature.asc Description: Digital signature
Re: [HACKERS] libpq WSACleanup is not needed
Magnus Hagander wrote: Bruce Momjian wrote: Andrew Chernow wrote: Bruce Momjian wrote: We could have gone with a more elegant init/uninit solution but there is a history of slow upstream adoption of libpq API changes. If that's the case, adding a connectdb option seems like a good alternative. Orignally suggested here: http://archives.postgresql.org/pgsql-hackers/2009-01/msg01358.php Right, well the big question is how many people are going to use the connection option vs. doing it for everyone automatically. One possible approach might be to do it automatically, and allow a connection option to disable the WSACleanup() call. I think that was the suggestion. Have an option that would disable *both* the startup and the cleanup call, leaving the responsibility to the app. You can do this for SSL today by calling PQinitSSL(). Right. Actually, right now, if you have two libpq connections, and close one, does WSACleanup() get called, and does it affect the existing connection? WSACleanup() gets called, but it has an internal reference count so it does not have any effect on existing connections. Ah, OK, so it does its own cleanup on last close, great. I agree a connection option for this would be good. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FWD: Re: Updated backslash consistency patch
Alvaro Herrera wrote: Bruce Momjian escribi?: In thinking last night, I am now wondering if a letter is really the right symbol for this. We already have letter flags which control object type selection, but the system table addition is kind of independent of those flags, like '+' now. I am thinking maybe '' is the right symbol rather than 'A' or 'S'. is probably going to be useful as a modifier when we have concurrent psql, so I suggest you do not use it here. Or maybe not, but in any case see http://archives.postgresql.org/message-id/8204.1207689056%40sss.pgh.pa.us before you use . Yea, I thought '' might be coming for that someday. I figured it wouldn't be used for backslash commands, but why confuse things. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FWD: Re: Updated backslash consistency patch
Greg Sabino Mullane wrote: [ There is text before PGP section. ] -BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 In thinking last night, I am now wondering if a letter is really the right symbol for this. We already have letter flags which control object type selection, but the system table addition is kind of independent of those flags, like '+' now. I am thinking maybe '' is the right symbol rather than 'A' or 'S'. What problem are we trying to solve here that is not already solved by using 'S' for system tables, which has been in use in psql for over a decade now? Well, the 'S' behavior used to be only system, now it is _add_ system. Anyway, 'S' is still probably the best choice. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Fixes for compiler warnings
Peter Eisentraut wrote: -Wformat-security warns about printf(var); but not about printf(var, a); I don't understand that; the crash or exploit potential is pretty much the same in both cases. Not sure this is the reason, but in the first case any risk is trivially avoided by using puts() or printf(%s, var) instead. So printf(var) is almost certainly not what you mean. I think that's a reasonable warning to have enabled, whereas the other one is more of a try it sometime, you might find something kind of warning. Jeroen -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] is 8.4 array_agg() supposed to work with array values?
On 1/20/09, Peter Eisentraut pete...@gmx.net wrote: integer[] is array of integer, in the way C deals with arrays. This is also a main reason why composite types and arrays don't mix orthogonally; there is no way to represent that in the system catalogs. What do you mean by this exactly? We've had arrays of composite types since 8.2. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq WSACleanup is not needed
Merlin Moncure wrote: I think init/uninit is the answer. While writing libpqtypes, we noted that libpq is just plain awkward in a few different ways and probably deserves a rewrite at some point. not today though Would there be any serious harm in: 1. doing the WSAStartup() when the first connection is opened, but 2. cleaning up from either 2a. some kind of on-unload version of DllMain() if possible, or 2b. an explicit new cleanup function ? (I know it says you can't set the thing up from DllMain, but does it say something similar about shutdown?) Jeroen -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FWD: Re: Updated backslash consistency patch
Dimitri Fontaine wrote: -- Start of PGP signed section. Le mardi 20 janvier 2009, Bruce Momjian a ?crit?: Robert Haas wrote: Here is what I hope is a consensus patch. It adds 'A' to show all objects, including system ones. It turns out that this is how 'S' works now in CVS, but 'S' is unclear because it suggests just system objects; 'A' for show 'all' objects seems clearer. I think it's probably fine for S to mean include system objects rather than show only system objects. Everyone should be relatively used to S by now; I think it's less confusing to keep the same letter even if the behavior has been adjusted somewhat. Though others may disagree? I still think that given a pattern, psql commands should simply mimic whatever is the server way of using search_path. I'd really like \df foo and \d foo to follow the same rules as my production queries wrt to how to find objects when I'm too lazy to schema qualify their name. Now, it's been advocated for the sake of simplicity to have with-pattern and without-pattern options behave roughly the same way. I can't find it difficult to explain the two behaviours here, all the more when looking at current \d and \dt differences. The \d and \dt differences are fixed/gone in current CVS. What I'd like to propose is for \df without pattern to behave exactly like \df with pattern, *including* wrt to ordering the output. Functions listed in search_path order, pg_catalog implicitly part of it, but as its *last* element. Or whatever server object lookup code sayth. I personally liked the idea of searching pg_catalog for a pattern, but what turned me against it was this behavior: \d long list of user tables and then the user wants to see just the tables that begin with 'p': \d p* list of system and user tables that start with 'p' All of a sudden they see many system tables. It is hard to call that behavior logical or expected. One unusual approach would be to search pg_catalog only when a _non-wildcard_ pattern was supplied, so: \d p* would show user tables beginning with 'p', but: \d pg_class would find the 'pg_class' table that is the search path, typically from pg_catalog. It might be a little awkward to document, but might be the most acceptable solution. The very good argument _against_ this solution is that: \d pg_class* would show no rows while: \d pg_class would show the pg_catalog entry. This is also odd and unexpected, which led me to just having people use 'S' when they want pg_catalog involved. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq WSACleanup is not needed
Jeroen Vermeulen wrote: Would there be any serious harm in: 1. doing the WSAStartup() when the first connection is opened, but The only problem is how to detect the first connection. In a threaded environment you'd have to perform locking in connectdb, which is probably not going to fly. but does it say something similar about shutdown? From the WSACleanup docs: The WSACleanup function typically leads to protocol-specific helper DLLs being unloaded. As a result, the WSACleanup function should not be called from the DllMain function in a application DLL. This can potentially cause deadlocks -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq WSACleanup is not needed
Bruce Momjian wrote: Ah, OK, so it does its own cleanup on last close, great. I agree a connection option for this would be good. What would the option be? wsainit = [enable | disable]? Maybe it should allow setting the version to load: wsa_version = 2.0. Maybe the two should be combined: wsa_version = [default | disable | 2.0]. -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq WSACleanup is not needed
Andrew Chernow wrote: Bruce Momjian wrote: Ah, OK, so it does its own cleanup on last close, great. I agree a connection option for this would be good. What would the option be? wsainit = [enable | disable]? Maybe it should allow setting the version to load: wsa_version = 2.0. Maybe the two should be combined: wsa_version = [default | disable | 2.0]. I will say, the cleanest solution is still an optional init()/uninit() for libpq. Has this been ruled out? IMHO, the next best solution is a connection option. -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq WSACleanup is not needed
Andrew Chernow wrote: Bruce Momjian wrote: Ah, OK, so it does its own cleanup on last close, great. I agree a connection option for this would be good. What would the option be? wsainit = [enable | disable]? Maybe it should allow setting the version to load: wsa_version = 2.0. Maybe the two should be combined: wsa_version = [default | disable | 2.0]. I assumed it would be like SSL, which is a libpq function call, not a connection option, e.g. PQinitSSL(), and I think true/false is probably enough. PQinitSSL info: If you are using acronymSSL/ inside your application (in addition to inside applicationlibpq/application), you can use functionPQinitSSL(int)/ to tell applicationlibpq/application that the acronymSSL/ library has already been initialized by your application. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq WSACleanup is not needed
Andrew Chernow wrote: Andrew Chernow wrote: Bruce Momjian wrote: Ah, OK, so it does its own cleanup on last close, great. I agree a connection option for this would be good. What would the option be? wsainit = [enable | disable]? Maybe it should allow setting the version to load: wsa_version = 2.0. Maybe the two should be combined: wsa_version = [default | disable | 2.0]. I will say, the cleanest solution is still an optional init()/uninit() for libpq. Has this been ruled out? IMHO, the next best solution is a connection option. What happened to the idea of counting connections? That seemed a relatively clean way to go, I thought, although I haven't followed the discussion very closely. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq WSACleanup is not needed
Bruce Momjian wrote: Andrew Chernow wrote: Bruce Momjian wrote: Ah, OK, so it does its own cleanup on last close, great. I agree a connection option for this would be good. What would the option be? wsainit = [enable | disable]? Maybe it should allow setting the version to load: wsa_version = 2.0. Maybe the two should be combined: wsa_version = [default | disable | 2.0]. I assumed it would be like SSL, which is a libpq function call, not a connection option, e.g. PQinitSSL(), and I think true/false is probably enough. PQinitSSL info: If you are using acronymSSL/ inside your application (in addition to inside applicationlibpq/application), you can use functionPQinitSSL(int)/ to tell applicationlibpq/application that the acronymSSL/ library has already been initialized by your application. That smells dirty to me. How many PQinitXXX() functions are needed before we drop the XXX and run with PQinit(...)? -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq WSACleanup is not needed
Andrew Dunstan wrote: Andrew Chernow wrote: Andrew Chernow wrote: Bruce Momjian wrote: Ah, OK, so it does its own cleanup on last close, great. I agree a connection option for this would be good. What would the option be? wsainit = [enable | disable]? Maybe it should allow setting the version to load: wsa_version = 2.0. Maybe the two should be combined: wsa_version = [default | disable | 2.0]. I will say, the cleanest solution is still an optional init()/uninit() for libpq. Has this been ruled out? IMHO, the next best solution is a connection option. What happened to the idea of counting connections? That seemed a relatively clean way to go, I thought, although I haven't followed the discussion very closely. I was told WSACleanup does connection counting internally (only the final close has a performance impact) so there is no need to do the counting like we do for SSL callbacks. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq WSACleanup is not needed
Andrew Chernow wrote: Bruce Momjian wrote: Andrew Chernow wrote: Bruce Momjian wrote: Ah, OK, so it does its own cleanup on last close, great. I agree a connection option for this would be good. What would the option be? wsainit = [enable | disable]? Maybe it should allow setting the version to load: wsa_version = 2.0. Maybe the two should be combined: wsa_version = [default | disable | 2.0]. I assumed it would be like SSL, which is a libpq function call, not a connection option, e.g. PQinitSSL(), and I think true/false is probably enough. PQinitSSL info: If you are using acronymSSL/ inside your application (in addition to inside applicationlibpq/application), you can use functionPQinitSSL(int)/ to tell applicationlibpq/application that the acronymSSL/ library has already been initialized by your application. That smells dirty to me. How many PQinitXXX() functions are needed before we drop the XXX and run with PQinit(...)? Odds are you would still need per-library control over initialization so I am not sure that helps, i.e. the library initialized WSA already but needs SSL. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] visibility maps and heap_prune
Pavan Deolasee wrote: On Mon, Dec 8, 2008 at 11:33 AM, Pavan Deolasee pavan.deola...@gmail.comwrote: On Sat, Dec 6, 2008 at 8:08 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: If you see a straightforward way, please submit a patch! Will do that. Here is a patch which implements this. The PD_ALL_VISIBLE flag is set if all tuples in the page are visible to all transactions and there are no DEAD line pointers in the page. The second check is required so that VACUUM takes up the page. We could slightly distinguish the two cases (one where the page requires vacuuming only because of DEAD line pointers and the other where the page-tuples do not require any visibility checks), but I thought its not worth the complexity. Is this patch for 8.4? -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] visibility maps
Tom Lane wrote: Pavan Deolasee pavan.deola...@gmail.com writes: OTOH I think we can still set PD_ALL_VISIBLE page header flag even when there are DEAD line pointers. That would mean the header flag means something different than the map bit does, which would mean extra tests need to be made before propagating the flag bit to the map. This is doable, no doubt, but it seems pretty error-prone ... Are there any open items from this discussion? -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq WSACleanup is not needed
Bruce Momjian wrote: Andrew Chernow wrote: Bruce Momjian wrote: Andrew Chernow wrote: Bruce Momjian wrote: Ah, OK, so it does its own cleanup on last close, great. I agree a connection option for this would be good. What would the option be? wsainit = [enable | disable]? Maybe it should allow setting the version to load: wsa_version = 2.0. Maybe the two should be combined: wsa_version = [default | disable | 2.0]. I assumed it would be like SSL, which is a libpq function call, not a connection option, e.g. PQinitSSL(), and I think true/false is probably enough. PQinitSSL info: If you are using acronymSSL/ inside your application (in addition to inside applicationlibpq/application), you can use functionPQinitSSL(int)/ to tell applicationlibpq/application that the acronymSSL/ library has already been initialized by your application. That smells dirty to me. How many PQinitXXX() functions are needed before we drop the XXX and run with PQinit(...)? Odds are you would still need per-library control over initialization so I am not sure that helps, i.e. the library initialized WSA already but needs SSL. That's fine. I solved that issue here: http://archives.postgresql.org/pgsql-hackers/2009-01/msg01349.php One of arguments is an options bit mask. PG_OPT_LOADSSL, PG_OPT_LOADWSA, etc... I also suggested a int inittype, void *initdata arguments that can be used now or for future expansion; that way PQinit is not limited to a single int argument. This could be used right away with the PG_OPT_LOADWSA idea, to pass the wsa version you want. -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq WSACleanup is not needed
Andrew Chernow wrote: Bruce Momjian wrote: Andrew Chernow wrote: Bruce Momjian wrote: Andrew Chernow wrote: Bruce Momjian wrote: Ah, OK, so it does its own cleanup on last close, great. I agree a connection option for this would be good. What would the option be? wsainit = [enable | disable]? Maybe it should allow setting the version to load: wsa_version = 2.0. Maybe the two should be combined: wsa_version = [default | disable | 2.0]. I assumed it would be like SSL, which is a libpq function call, not a connection option, e.g. PQinitSSL(), and I think true/false is probably enough. PQinitSSL info: If you are using acronymSSL/ inside your application (in addition to inside applicationlibpq/application), you can use functionPQinitSSL(int)/ to tell applicationlibpq/application that the acronymSSL/ library has already been initialized by your application. That smells dirty to me. How many PQinitXXX() functions are needed before we drop the XXX and run with PQinit(...)? Odds are you would still need per-library control over initialization so I am not sure that helps, i.e. the library initialized WSA already but needs SSL. That's fine. I solved that issue here: http://archives.postgresql.org/pgsql-hackers/2009-01/msg01349.php One of arguments is an options bit mask. PG_OPT_LOADSSL, PG_OPT_LOADWSA, etc... I also suggested a int inittype, void *initdata arguments that can be used now or for future expansion; that way PQinit is not limited to a single int argument. This could be used right away with the PG_OPT_LOADWSA idea, to pass the wsa version you want. That seems overly complex to support just two init functions (we only had one for SSL for years). -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] visibility maps and heap_prune
Bruce Momjian wrote: Pavan Deolasee wrote: On Mon, Dec 8, 2008 at 11:33 AM, Pavan Deolasee pavan.deola...@gmail.comwrote: On Sat, Dec 6, 2008 at 8:08 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: If you see a straightforward way, please submit a patch! Will do that. Here is a patch which implements this. The PD_ALL_VISIBLE flag is set if all tuples in the page are visible to all transactions and there are no DEAD line pointers in the page. The second check is required so that VACUUM takes up the page. We could slightly distinguish the two cases (one where the page requires vacuuming only because of DEAD line pointers and the other where the page-tuples do not require any visibility checks), but I thought its not worth the complexity. Is this patch for 8.4? We already went through this: http://archives.postgresql.org/message-id/496f6a8e.8020...@enterprisedb.com I guess I'll follow Robert's advice and add this to the first 8.5 commit fest page. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Polymorphic types vs. domains
Where are we on this? I tested CVS and the problem still seems to exist. --- Tom Lane wrote: The proximate cause of this complaint: http://archives.postgresql.org/pgsql-general/2008-12/msg00283.php seems to be that the polymorphic-type code doesn't consider a domain over an enum type to match an ANYENUM function argument. ISTM this is probably wrong: we need such a domain to act like its base type for matching purposes. There is an analogous problem with a domain over an array type failing to match ANYARRAY; conversely, such a domain is considered to match ANYNONARRAY which it likely should not. Comments? If this is agreed to be a bug, should we consider back-patching it? (I'd vote not, I think, because the behavioral change could conceivably break some apps that work now.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] visibility maps
Bruce Momjian wrote: Tom Lane wrote: Pavan Deolasee pavan.deola...@gmail.com writes: OTOH I think we can still set PD_ALL_VISIBLE page header flag even when there are DEAD line pointers. That would mean the header flag means something different than the map bit does, which would mean extra tests need to be made before propagating the flag bit to the map. This is doable, no doubt, but it seems pretty error-prone ... Are there any open items from this discussion? No. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] new vacuum is slower for small tables
Tom Lane wrote: Pavel Stehule pavel.steh...@gmail.com writes: It's strange, when I repeat tests, I get usually times about 10 ms, but cca cca every 5 test it is about 2ms Hmm. The theory I'd developed for what I see here is that the slow timings correspond to when the pgstat code decides it needs a new stats file (and so it has to signal the stats collector and wait for the file to show up). The fast timings occur if the existing stats file is considered fresh enough to re-use. Hence, it's fast if you re-execute the VACUUM within half a second of the previous one, else slow. I can't tell if that's the same thing you see or not. Now that we have the flexibility to allow different levels of stats stale-ness for different callers, I wonder whether it wouldn't be okay to let pgstat_vacuum_stat work with quite stale files, eg up to a minute or so. Are we doing anything on this? -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] is 8.4 array_agg() supposed to work with array values?
On Tuesday 20 January 2009 18:09:51 Merlin Moncure wrote: On 1/20/09, Peter Eisentraut pete...@gmx.net wrote: integer[] is array of integer, in the way C deals with arrays. This is also a main reason why composite types and arrays don't mix orthogonally; there is no way to represent that in the system catalogs. What do you mean by this exactly? We've had arrays of composite types since 8.2. Yeah, the simple cases work. I meant it more in a general sense of structured and arbitrarily composed types. I'm also interested in multiset support, for example. Under the current arrangement, you couldn't really have array of multiset of array of something. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Column-Level Privileges
Stephen Frost sfr...@snowman.net writes: Attached is an updated patch for column-level privileges. I'm working on getting this committed. I've run into a major stumbling block in the parse-time marking of columns for SELECT privileges: the do-it-as-the-Vars-get-transformed approach basically doesn't work as-is. The problem comes from whole-row Vars referencing JOIN relations. We already talked about the need to mark column 0 as referenced for a whole-row Var, but if the Var is referencing a join then the correct thing is to recursively mark column 0 of the two input relations. (The existing patch simply crashes in this case ...) The problem is that there is no reliable way to identify the two input relations given only the join RTE. The normal thing we do is to dig in the query jointree for the JoinExpr, but during parse analysis the jointree is still being built and we don't have access to it :-(. The failure case would be where an upper JOIN/ON clause contains a whole-row reference to a contained JOIN relation. I considered a couple of alternatives: * Modify the recursion in transformFromClauseItem so that we can somehow get at the partially-built jointree for the current join item. Ugly and probably fragile. * Depend on the join's joinaliasvars list to contain references to both sides of the join. This fails in various corner cases, for instance LEFT NATURAL JOIN where every column of the righthand rel is a common column. * Add the left and right input RT indexes to join RTEs, so that we can get hold of them without needing to search the jointree. Kind of annoying to do this just for the one usage, mainly because there's a lot of infrastructure needed to update such entries during rewriter/planner manipulations. (Which would all be 100% useless anyway if the fields are only examined by the parser, but I dislike the idea of having RTE fields that aren't valid after parse time ...) On the whole I think we have to go back to the original plan of recursively searching the query's expressions after we've finished all the transformations (and have a completed jointree to refer to). This is slightly annoying on the grounds of adding parsing overhead that's completely useless unless per-column privileges are in use. On the other hand, none of the workable alternatives are exactly overhead-free either. Comments? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCHES] GIN improvements
On Mon, 2009-01-19 at 19:53 +0300, Teodor Sigaev wrote: I see only two guaranteed solution of the problem: - after limit is reached, force normal index inserts. One of the motivation of patch was frequent question from users: why update of whole table with GIN index is so slow? So this way will not resolve this question. - after limit is reached, force cleanup of pending list by calling gininsertcleanup. Not very good, because users sometimes will see a huge execution time of simple insert. Although users who runs a huge update should be satisfied. I have difficulties in a choice of way. Seems to me, the better will be second way: if user gets very long time of insertion then (auto)vacuum of his installation should tweaked. I agree that the second solution sounds better to me. With the new Visibility Map, it's more reasonable to run VACUUM more often, so those that are inserting single tuples at a time should not encounter the long insert time. I'm still looking at the rest of the patch. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] BUG #4186: set lc_messages does not work
Magnus Hagander wrote: Hiroshi Inoue wrote: Magnus Hagander wrote: There still needs to be some error checking added in IsoLocaleName(), but this is a start. Can someone please test this? :-) OK I would check it tonight. Thanks. OK seems to works here. The attached is a test case using lc_messages C, fr, de, es and sv. regards, Hiroshi Inoue SET lc_messages - c (1 row) ERROR: syntax error at or near 1 LINE 1: 1; ^ SET lc_messages - fr (1 row) ERREUR: erreur de syntaxe sur ou près de « 1 » LINE 1: 1; ^ SET lc_messages - de (1 row) FEHLER: Fehler »Syntaxfehler« bei »1« LINE 1: 1; ^ SET lc_messages - es (1 row) ERROR: error de sintaxis en o cerca de «1» LINE 1: 1; ^ SET lc_messages - sv (1 row) FEL: syntaxfel vid eller nära 1 LINE 1: 1; ^ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Visibility map and freezing
On Tue, 2009-01-20 at 11:03 +0200, Heikki Linnakangas wrote: - Are there some conditions where whole-table-scanning vacuum is more effective than vacuums using visibility map? If so, we should switch to full-scan *automatically*, without relying on user configurations. Hmm, the only downside I can see is that skipping a page here and there could defeat the OS read-ahead. Perhaps we should call posix_fadvise(SEQUENTIAL) to compensate. Or, we could modify the logic to only skip pages when there's at least N consecutive pages that can be skipped. I would rather we didn't skip any pages at all unless the gains are significant. Skipping the odd page makes no difference from a performance perspective but may have a robustness impact. Significant gains should take into account the size of both heap and indexes, and recognise that we still scan whole indexes in either case. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Explicitly bind gettext() to the UTF8 locale when in use.
Hiroshi Inoue wrote:
Magnus Hagander wrote:
Hiroshi Inoue wrote:
Hiroshi Inoue wrote:
Bruce Momjian wrote:
Hiroshi, is this patch still needed?
Yes though it should be slightly changed now.
In what way should it be changed?
One is already committed by you.
[COMMITTERS] pgsql: Use the new text domain names
Another is to bind the codeset EUC-JP for
PG_EUC_JIS_2004 server encoding.
The attached is an updated patch.
regards,
Hiroshi Inoue
Index: mbutils.c
===
RCS file: /projects/cvsroot/pgsql/src/backend/utils/mb/mbutils.c,v
retrieving revision 1.77
diff -c -c -r1.77 mbutils.c
*** mbutils.c 19 Jan 2009 15:34:23 - 1.77
--- mbutils.c 20 Jan 2009 12:54:33 -
***
*** 837,842
--- 837,881
return clen;
}
+ #ifdefWIN32
+ static const struct codeset_map {
+ int encoding;
+ const char *codeset;
+ } codeset_map_array[] = {
+ {PG_UTF8, UTF-8},
+ {PG_LATIN1, LATIN1},
+ {PG_LATIN2, LATIN2},
+ {PG_LATIN3, LATIN3},
+ {PG_LATIN4, LATIN4},
+ {PG_ISO_8859_5, ISO-8859-5},
+ {PG_ISO_8859_6, ISO_8859-6},
+ {PG_ISO_8859_7, ISO-8859-7},
+ {PG_ISO_8859_8, ISO-8859-8},
+ {PG_LATIN5, LATIN5},
+ {PG_LATIN6, LATIN6},
+ {PG_LATIN7, LATIN7},
+ {PG_LATIN8, LATIN8},
+ {PG_LATIN9, LATIN-9},
+ {PG_LATIN10, LATIN10},
+ {PG_KOI8R, KOI8-R},
+ {PG_WIN1250, CP1250},
+ {PG_WIN1251, CP1251},
+ {PG_WIN1252, CP1252},
+ {PG_WIN1253, CP1253},
+ {PG_WIN1254, CP1254},
+ {PG_WIN1255, CP1255},
+ {PG_WIN1256, CP1256},
+ {PG_WIN1257, CP1257},
+ {PG_WIN1258, CP1258},
+ {PG_WIN866, CP866},
+ {PG_WIN874, CP874},
+ {PG_EUC_CN, EUC-CN},
+ {PG_EUC_JP, EUC-JP},
+ {PG_EUC_KR, EUC-KR},
+ {PG_EUC_TW, EUC-TW},
+ {PG_EUC_JIS_2004, EUC-JP}};
+ #endif /* WIN32 */
+
/* mbcliplen for any single-byte encoding */
static int
cliplen(const char *str, int len, int limit)
***
*** 852,857
--- 891,898
void
SetDatabaseEncoding(int encoding)
{
+ const char *target_codeset = NULL;
+
if (!PG_VALID_BE_ENCODING(encoding))
elog(ERROR, invalid database encoding: %d, encoding);
***
*** 873,879
*/
#ifdef ENABLE_NLS
if (encoding == PG_UTF8)
! if (bind_textdomain_codeset(textdomain(NULL), UTF-8) == NULL)
elog(LOG, bind_textdomain_codeset failed);
#endif
}
--- 914,935
*/
#ifdef ENABLE_NLS
if (encoding == PG_UTF8)
! target_codeset = UTF-8;
! #ifdefWIN32
! else
! {
! int i;
!
! for (i = 0; i sizeof(codeset_map_array) / sizeof(struct
codeset_map); i++)
! if (codeset_map_array[i].encoding == encoding)
! {
! target_codeset = codeset_map_array[i].codeset;
! break;
! }
! }
! #endif /* WIN32 */
! if (target_codeset != NULL)
! if (bind_textdomain_codeset(textdomain(NULL), target_codeset)
== NULL)
elog(LOG, bind_textdomain_codeset failed);
#endif
}
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Column-Level Privileges
... btw, what is the reasoning behind the special cases for SELECT FOR UPDATE in execMain.c? /* Check if this is SELECT-FOR-UPDATE and handle * accordingly. */ if(remainingPerms ACL_UPDATE pg_attribute_aclcheck_all(relOid, userid, ACL_UPDATE, ACLMASK_ALL) != ACLCHECK_OK) aclcheck_error(ACLCHECK_NO_PRIV, ACL_KIND_CLASS, get_rel_name(relOid)); If there actually is a need to treat SELECT FOR UPDATE specially, then this code is quite wrong because it will also fire on a plain UPDATE (assuming the UPDATE reads any existing column values, which it usually would). Offhand though I don't see why we can't just use code that is symmetric with the SELECT case: if requiredPerms includes UPDATE but there are no columns called out for UPDATE, then allow it if we have UPDATE on any column. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Column-Level Privileges
* Tom Lane (t...@sss.pgh.pa.us) wrote: On the whole I think we have to go back to the original plan of recursively searching the query's expressions after we've finished all the transformations (and have a completed jointree to refer to). This is slightly annoying on the grounds of adding parsing overhead that's completely useless unless per-column privileges are in use. On the other hand, none of the workable alternatives are exactly overhead-free either. Comments? Honestly, I like this approach. There is some additional overhead during parsing, but it seems cleaner and more robust. Also, hopefully in most cases where people are concerned about parse time they're preparing their queries. If it's warrented, we could try doing benchmarks to see how bad the impact is and if we need to do something different. It doesn't strike me as likely to be a significant amount of overhead though. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] Hot standby, RestoreBkpBlocks and cleanup locks
Simon Riggs wrote: It would be useful to nibble away at the patch, committing all the necessary refactorings to make the patch work. That will reduce size of eventual commit. I committed this part now; one less thing to review. Please adjust the patch accordingly. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Column-Level Privileges
* Tom Lane (t...@sss.pgh.pa.us) wrote: ... btw, what is the reasoning behind the special cases for SELECT FOR UPDATE in execMain.c? Basically, because the original logic allowed SELECT-FOR-UPDATE if you only had SELECT rights, which wasn't right. If there actually is a need to treat SELECT FOR UPDATE specially, then this code is quite wrong because it will also fire on a plain UPDATE (assuming the UPDATE reads any existing column values, which it usually would). Offhand though I don't see why we can't just use code that is symmetric with the SELECT case: if requiredPerms includes UPDATE but there are no columns called out for UPDATE, then allow it if we have UPDATE on any column. I agree, this makes alot more sense to me. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] Hot standby, RestoreBkpBlocks and cleanup locks
On Tue, 2009-01-20 at 21:01 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: It would be useful to nibble away at the patch, committing all the necessary refactorings to make the patch work. That will reduce size of eventual commit. I committed this part now; one less thing to review. Please adjust the patch accordingly. OK, thanks. I did want you to commit those changes, but that was 8 days ago and much has changed since then. Now, I'm slightly worried that this may be a retrograde step. The last 3 days I've been refactoring the code to account for other requirements, as I have been discussing, and some of these things have now changed. Though the general pattern of your suggested refactoring remains the same. I'll check in more detail, but please could we talk before you commit parts, otherwise we'll just trip over each other. I'll post the new version (v8f) (which is pre-commit) this evening, so we can discuss. Can we plan a move to GIT? We can both work on things at the same time and my changes can be more visible. There will be some initial pain... -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Visibility map and freezing
Simon Riggs wrote:
On Tue, 2009-01-20 at 11:03 +0200, Heikki Linnakangas wrote:
- Are there some conditions where whole-table-scanning vacuum is
more
effective than vacuums using visibility map? If so, we should
switch
to full-scan *automatically*, without relying on user
configurations.
Hmm, the only downside I can see is that skipping a page here and
there could defeat the OS read-ahead. Perhaps we should call
posix_fadvise(SEQUENTIAL) to compensate. Or, we could modify the logic
to only skip pages when there's at least N consecutive pages that can
be skipped.
I would rather we didn't skip any pages at all unless the gains are
significant. Skipping the odd page makes no difference from a
performance perspective but may have a robustness impact.
Significant gains should take into account the size of both heap and
indexes, and recognise that we still scan whole indexes in either case.
That sounds pretty complex, approaching what the planner does. I'd
rather keep it simple.
Attached is a simple patch to only start skipping pages after 20
consecutive pages marked as visible in the visibility map. This doesn't
do any look-ahead, so it will always scan the first 20 pages of a
table before it starts to skip pages, and whenever there's even one page
that needs vacuuming, the next 19 pages will also be vacuumed.
We could adjust that figure 20 according to table size. Or by
seq_page_cost/random_page_cost. But I'm leaning towards a simple
hard-coded value for now.
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
*** src/backend/commands/vacuumlazy.c
--- src/backend/commands/vacuumlazy.c
***
*** 271,276 lazy_scan_heap(Relation onerel, LVRelStats *vacrelstats,
--- 271,277
int i;
PGRUsage ru0;
Buffer vmbuffer = InvalidBuffer;
+ BlockNumber all_visible_streak;
pg_rusage_init(ru0);
***
*** 292,297 lazy_scan_heap(Relation onerel, LVRelStats *vacrelstats,
--- 293,299
lazy_space_alloc(vacrelstats, nblocks);
+ all_visible_streak = 0;
for (blkno = 0; blkno nblocks; blkno++)
{
Buffer buf;
***
*** 309,315 lazy_scan_heap(Relation onerel, LVRelStats *vacrelstats,
/*
* Skip pages that don't require vacuuming according to the
! * visibility map.
*/
if (!scan_all)
{
--- 311,324
/*
* Skip pages that don't require vacuuming according to the
! * visibility map. But only if we've seen a streak of at least
! * 20 pages marked as clean. Since we're reading sequentially,
! * the OS should be doing readahead for us and there's no gain
! * in skipping a page now and then. You need a longer run of
! * consecutive skipped pages before it's worthwhile. Also,
! * skipping even a single page means that we can't update
! * relfrozenxid or reltuples, so we only want to do it if
! * there's a good chance to skip a goodly number of pages.
*/
if (!scan_all)
{
***
*** 317,325 lazy_scan_heap(Relation onerel, LVRelStats *vacrelstats,
visibilitymap_test(onerel, blkno, vmbuffer);
if (all_visible_according_to_vm)
{
! vacrelstats-scanned_all = false;
! continue;
}
}
vacuum_delay_point();
--- 326,340
visibilitymap_test(onerel, blkno, vmbuffer);
if (all_visible_according_to_vm)
{
! all_visible_streak++;
! if (all_visible_streak = 20)
! {
! vacrelstats-scanned_all = false;
! continue;
! }
}
+ else
+ all_visible_streak = 0;
}
vacuum_delay_point();
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby, RestoreBkpBlocks and cleanup locks
Simon Riggs wrote: I did want you to commit those changes, but that was 8 days ago and much has changed since then. Now, I'm slightly worried that this may be a retrograde step. The last 3 days I've been refactoring the code to account for other requirements, as I have been discussing, and some of these things have now changed. Though the general pattern of your suggested refactoring remains the same. I figured there's possibly some further changes, but the general idea to move the responsibility of restoring backup blocks to the redo function should remain the same. Can we plan a move to GIT? We can both work on things at the same time and my changes can be more visible. There will be some initial pain... Sure, just go ahead and publish a repository. Or would you like me to apply the patches and publish a repository you can clone from? Perhaps that would be easier since I'm already familiar with GIT. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Visibility map and freezing
On Tue, 2009-01-20 at 21:51 +0200, Heikki Linnakangas wrote: Attached is a simple patch to only start skipping pages after 20 consecutive pages marked as visible in the visibility map. This doesn't do any look-ahead, so it will always scan the first 20 pages of a table before it starts to skip pages, and whenever there's even one page that needs vacuuming, the next 19 pages will also be vacuumed. We could adjust that figure 20 according to table size. Or by seq_page_cost/random_page_cost. But I'm leaning towards a simple hard-coded value for now. Yes, sounds good. Can we stick to multiples of 2 as the OS readahead does IIRC? So either 16 or 32. I'd go 32. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] New pg_dump patch, --no-stats flag, disables sending to statistics collector
This patch adds another flag to pg_dump, this time to disable statistics
collection. This is useful if your don't want pg_dump activity to show
(or clutter) your normal statistics. This may be appropriate for an
organization that regularly dumps a database for backup purposes, but
wants to analyze only the application's database use.
This is patched against CVS HEAD from today, though the code is quite
version independent. This patch is unsolicited, and as far as I know
has not been discussed on the list prior to now.
Comments?
Index: pg_dump.c
===
RCS file: /projects/cvsroot/pgsql/src/bin/pg_dump/pg_dump.c,v
retrieving revision 1.514
diff -c -2 -r1.514 pg_dump.c
*** pg_dump.c 18 Jan 2009 20:44:45 - 1.514
--- pg_dump.c 20 Jan 2009 20:47:25 -
***
*** 236,239
--- 236,240
static int outputNoTablespaces = 0;
static int use_setsessauth = 0;
+ static int noStatsCollection = 0;
static struct option long_options[] = {
***
*** 278,281
--- 279,283
{role, required_argument, NULL, 3},
{use-set-session-authorization, no_argument, use_setsessauth, 1},
+ {no-stats, no_argument, noStatsCollection, 1},
{NULL, 0, NULL, 0}
***
*** 430,433
--- 432,437
else if (strcmp(optarg, no-tablespaces) == 0)
outputNoTablespaces = 1;
+ else if (strcmp(optarg, no-stats) == 0)
+ noStatsCollection = 1;
else if (strcmp(optarg, use-set-session-authorization) == 0)
use_setsessauth = 1;
***
*** 613,616
--- 617,629
do_sql_command(g_conn, SET statement_timeout = 0);
+ /*
+ * Disable collection of statistics. pg_dump's activity may be very different
+ * from what you are trying to analyze in the stats tables.
+ */
+ if( noStatsCollection ) {
+ do_sql_command(g_conn, SET stats_block_level = false);
+ do_sql_command(g_conn, SET stats_row_level = false);
+ }
+
/*
* Start serializable transaction to dump consistent data.
***
*** 833,836
--- 846,850
printf(_( -U, --username=NAME connect as specified database user\n));
printf(_( -W, --password force password prompt (should happen automatically)\n));
+ printf(_( --no-stats disable statistics collection (superuser only)\n));
printf(_(\nIf no database name is supplied, then the PGDATABASE environment\n
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New pg_dump patch, --no-stats flag, disables sending to statistics collector
On Tue, Jan 20, 2009 at 4:04 PM, Bryce Nesbitt bry...@obviously.com wrote: This patch adds another flag to pg_dump, this time to disable statistics collection. This is useful if your don't want pg_dump activity to show (or clutter) your normal statistics. This may be appropriate for an organization that regularly dumps a database for backup purposes, but wants to analyze only the application's database use. i haven't looked at the patch nor it's functional use... but from the top of my head jumps a question: is there a reason to not make this the default behaviour? -- Atentamente, Jaime Casanova Soporte y capacitación de PostgreSQL Asesoría y desarrollo de sistemas Guayaquil - Ecuador Cel. +59387171157 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] about truncate
Peter Eisentraut pete...@gmx.net writes: The SQL standard uses a recursive-by-default language. For example, the rules for the DELETE command state: Actually, I'm not convinced. Take a look at the SELECT WITH HIERARCHY OPTION stuff in SQL99 and later, in particular this from SQL99 12.2 grant privilege statement: 7) Let SWH be the set of privilege descriptors in CPD whose action is SELECT WITH HIERARCHY OPTION, and let ST be the set of subtables of O, then for every grantee G in SWH and for every table T in ST, the following grant statement is effectively executed without further Access Rule checking: GRANT SELECT ON T TO G GRANTED BY A It's difficult to read that any other way than that privileges are *not* auto-recursive, and they have chosen to spell * in GRANT as WITH HIERARCHY OPTION (gackk). On the other hand, it's hard to square that reading with the lack of any UPDATE or DELETE WITH HIERARCHY OPTION syntax. What am I missing here? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Visibility map and freezing
Simon Riggs escreveu: On Tue, 2009-01-20 at 21:51 +0200, Heikki Linnakangas wrote: Attached is a simple patch to only start skipping pages after 20 consecutive pages marked as visible in the visibility map. This doesn't do any look-ahead, so it will always scan the first 20 pages of a table before it starts to skip pages, and whenever there's even one page that needs vacuuming, the next 19 pages will also be vacuumed. We could adjust that figure 20 according to table size. Or by seq_page_cost/random_page_cost. But I'm leaning towards a simple hard-coded value for now. Yes, sounds good. Can we stick to multiples of 2 as the OS readahead does IIRC? So either 16 or 32. I'd go 32. Agreed. And do it a constant (ALL_VISIBLE_VM_THRESHOLD?). -- Euler Taveira de Oliveira http://www.timbira.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New pg_dump patch, --no-stats flag, disables sending to statistics collector
Jaime Casanova wrote: On Tue, Jan 20, 2009 at 4:04 PM, Bryce Nesbitt bry...@obviously.com wrote: This patch adds another flag to pg_dump, this time to disable statistics collection. This is useful if your don't want pg_dump activity to show (or clutter) your normal statistics. This may be appropriate for an organization that regularly dumps a database for backup purposes, but wants to analyze only the application's database use. i haven't looked at the patch nor it's functional use... but from the top of my head jumps a question: is there a reason to not make this the default behaviour? If this is a generally desired feature (and I question that), I think we need a more general solution. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] about truncate
Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: The SQL standard uses a recursive-by-default language. For example, the rules for the DELETE command state: Actually, I'm not convinced. Take a look at the SELECT WITH HIERARCHY OPTION stuff in SQL99 and later, in particular this from SQL99 12.2 grant privilege statement: 7) Let SWH be the set of privilege descriptors in CPD whose action is SELECT WITH HIERARCHY OPTION, and let ST be the set of subtables of O, then for every grantee G in SWH and for every table T in ST, the following grant statement is effectively executed without further Access Rule checking: GRANT SELECT ON T TO G GRANTED BY A It's difficult to read that any other way than that privileges are *not* auto-recursive, and they have chosen to spell * in GRANT as WITH HIERARCHY OPTION (gackk). On the other hand, it's hard to square that reading with the lack of any UPDATE or DELETE WITH HIERARCHY OPTION syntax. What am I missing here? It's just occurred to me that if TRUNCATE no longer means TRUNCATE ONLY, parallel restore will need to detect which server version is being used so that for version 8.3 it issues TRUNCATE ONLY. Otherwise there would be a danger of a collision between a table and its children. The only alternative would be to create a dependency between the data of a table and the data of its children, which would be undesirable as well as more complicated - in general the data should only depend on the table creation (at most). cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New pg_dump patch, --no-stats flag, disables sending to statistics collector
On Tue, 20 Jan 2009, Bryce Nesbitt wrote: This patch adds another flag to pg_dump, this time to disable statistics collection. You can pass session parameters to anything that uses the standard libpq library using PGOPTIONS. See http://www.postgresql.org/docs/8.3/static/config-setting.html for a sample. I suspect that something like: PGOPTIONS='-c stats_block_level=false -c stats_row_level=false' pg_dump would do the same thing as your patch without having to touch the code. That's a pretty obscure bit of information though, and it would be worthwhile to update the documentation suggesting such a syntax because I think this would be handy for a lot of people. I was already planning to do that for another use case (pgbench) once the 8.4 work here shifts from development to testing and I have some more time for writing. -- * Greg Smith gsm...@gregsmith.com http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New pg_dump patch, --no-stats flag, disables sending to statistics collector
Bruce Momjian wrote: Jaime Casanova wrote: i haven't looked at the patch nor it's functional use... but from the top of my head jumps a question: is there a reason to not make this the default behaviour? If this is a generally desired feature (and I question that), I think we need a more general solution. I'm not a big fan of flags, preferring good defaults. But I was not bold enough to suggest this as a new default, as someone would probably want the opposite flag. If you're measuring total server load (rather than analyzing an application), you may want to see pg_dump activity. As for a general solution: one could add the ability to inject arbitrary sql just prior to a dump run. That would let someone roll their own by injecting SET stats_block_level = false, or make any other arbitrary settings changes. Or one might slice the statistics collector by role or user (so your 'backup' role would keep a separate tally). On the other hand, the flag's advantage is simplicity and directness. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq WSACleanup is not needed
Andrew Chernow wrote: The only problem is how to detect the first connection. In a threaded environment you'd have to perform locking in connectdb, which is probably not going to fly. Well, if you do an atomic test for a flag being zero, and if so then enter a critsec, do the init iff you're the first in, and then set the flag on the way out, then: - most times, you'll just have the atomic test - other times, you have a short critsec I can't see that being a big deal considering you're about to resolve the server hostname and then do a TCP/IP connect. My understanding is that if you do WSAStartup and WSACleanup scoped to each connection then: - the internal counting means that only the 0 - 1 and 1 - 0 transitions are expensive - libpq will only incur the cost if the application didn't do it already So it seems that the cost is incurred by an application that: - makes no other use of winsock (or also does startup/cleanup often) - does not retain a connection (or pool) but creates and closes a single connection often How many applications are there that match this pattern? Isn't it enough just to tell the user to do WSAStartup and WSACleanup in main() if they find they have a performance problem? Surely most Windows programs effectively do that anyway, often as a side effect of using a framework. James -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New pg_dump patch, --no-stats flag, disables sending to statistics collector
Bruce, If this is a generally desired feature (and I question that), I think we need a more general solution. I'd argue that it is generally desired (or some convenient workaround) but not urgently so. I'd also argue that if we're going to have a --no-stats flag, it should exist for the other client ultilites as well; if I don't want pg_dump showing up, I probably don't want Vacuum showing up, or various other already-debugged maintenance routines. I'd suggest putting this into the first patch review for 8.5. --Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status Report on Hot Standby
On Friday 16 January 2009 19:16:42 Simon Riggs wrote: Bruce asked for 2 more weeks to get patches into shape for commit. Current patch v8e is attached here. Ready for commit? Up to you. My overall opinion is that it's in very good shape. Worth the community including it in this release and spending further time on it. I'm happy to stand by this going forwards. +1 The patch could benefit further from input from other various hackers, what couldn't? It's time to put this in a shared repository (of some kind) and make further changes to it in a controlled manner. Yep. I've now got this running on Linux and Solaris and testing so far has looked good. I've also spoken to a couple other people who have built it and run it, and everyone has been pretty happy. It'd certainly be nice to see this get into the main source tree to make it easier for future testing. (For example, one hurdle on Solaris, I had to get a different version of patch to handle Simon's diff... ugh!) -- Robert Treat Conjecture: http://www.xzilla.net Consulting: http://www.omniti.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New pg_dump patch, --no-stats flag, disables sending to statistics collector
Josh Berkus j...@agliodbs.com writes: Bruce, If this is a generally desired feature (and I question that), I think we need a more general solution. I'd argue that it is generally desired (or some convenient workaround) but not urgently so. One person asking for it does not make it generally desired. I think that the use-case is more than adequately served already by using PGOPTIONS, or by running pg_dump under a user id that has the appropriate GUC settings applied via ALTER USER. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Status Report on Hot Standby
On Tue, 2009-01-20 at 18:08 -0500, Robert Treat wrote: The patch could benefit further from input from other various hackers, what couldn't? It's time to put this in a shared repository (of some kind) and make further changes to it in a controlled manner. Yep. I've now got this running on Linux and Solaris and testing so far has looked good. I've also spoken to a couple other people who have built it and run it, and everyone has been pretty happy. It'd certainly be nice to see this get into the main source tree to make it easier for future testing. (For example, one hurdle on Solaris, I had to get a different version of patch to handle Simon's diff... ugh!) There'll be a new version out soon, hopefully tomorrow, with some fancy new deferred conflict resolution. Timing depends on passage through testing. I think it will then be fermented in GIT for a while. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New pg_dump patch, --no-stats flag, disables sending to statistics collector
On Tue, 2009-01-20 at 18:37 -0500, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: Bruce, If this is a generally desired feature (and I question that), I think we need a more general solution. I'd argue that it is generally desired (or some convenient workaround) but not urgently so. One person asking for it does not make it generally desired. I think that the use-case is more than adequately served already by using PGOPTIONS, or by running pg_dump under a user id that has the appropriate GUC settings applied via ALTER USER. +1 Sincerely, Joshua D. Drake regards, tom lane -- PostgreSQL - XMPP: jdr...@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New pg_dump patch, --no-stats flag, disables sending to statistics collector
Josh Berkus wrote: I'd argue that it is generally desired (or some convenient workaround) but not urgently so. I'd also argue that if we're going to have a --no-stats flag, it should exist for the other client ultilites as well; if I don't want pg_dump showing up, I probably don't want Vacuum showing up, or various other already-debugged maintenance routines. I'd suggest putting this into the first patch review for 8.5. --Josh As pg_dumpall calls pg_dump, I think this is covered or at least coverable. For vaccum, I've never seen that activity in stats? Can you supply a more specific scenario where routine maintenance is harmfully cluttering the stats table? A specific utility that needs attention? For this feature I'm not so sure about generally desired -- I'll bet most people don't even think about this. The question is among those who DO think about it, what's the best behavior? Can it be argued that excluding pg_dump is generally desirable, for the average use case? If there is not enough demand for a dedicated flag, I may submit a man page patch documenting the Do-It-Yourself solution proposed by Greg Smith, or the one proposed by Tom Lane. G'day -Bryce PS: Note that no respondent on the psql user's lists thought excluding pg_dump was even possible -- so that at least argues for desirability of instructional material :-). -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] New pg_dump patch -- document statistics collector exception
This is a proposed patch to document disabling the statistics collector pg_dump activity, and give a bit more visibility to the PGOPTIONS environment variable supported by libpq. It is an alternative to the prior patch, which supplied a --no-stats flag. This is a documentation only patch, not tied to a recent code change. ? HTML.index ? test.man Index: ref/pg_dump.sgml === RCS file: /projects/cvsroot/pgsql/doc/src/sgml/ref/pg_dump.sgml,v retrieving revision 1.106 diff -c -2 -r1.106 pg_dump.sgml *** ref/pg_dump.sgml 5 Jan 2009 16:54:36 - 1.106 --- ref/pg_dump.sgml 21 Jan 2009 01:15:41 - *** *** 727,733 termenvarPGDATABASE/envar/term termenvarPGHOST/envar/term termenvarPGPORT/envar/term - termenvarPGUSER/envar/term listitem para --- 727,734 termenvarPGDATABASE/envar/term termenvarPGHOST/envar/term + termenvarPGOPTIONS/envar/term termenvarPGPORT/envar/term + termenvarPGUSER/envar/term listitem para *** *** 735,738 --- 736,740 /para /listitem + /varlistentry /variablelist *** *** 758,761 --- 760,770 library will apply. /para + para +The database activity of applicationpg_dump/application is normally collected by the statistics collector. +If this is undesirable, you can set parameters literalstats_block_level/literal and literalstats_row_level/literal to false +via the applicationlibpq/ envarPGOPTIONS/envar environment varriable, or via literalALTER USER/literal. + /para + + /refsect1 Index: ref/pg_dumpall.sgml === RCS file: /projects/cvsroot/pgsql/doc/src/sgml/ref/pg_dumpall.sgml,v retrieving revision 1.74 diff -c -2 -r1.74 pg_dumpall.sgml *** ref/pg_dumpall.sgml 5 Jan 2009 16:54:36 - 1.74 --- ref/pg_dumpall.sgml 21 Jan 2009 01:15:41 - *** *** 446,449 --- 446,450 varlistentry termenvarPGHOST/envar/term + termenvarPGOPTIONS/envar/term termenvarPGPORT/envar/term termenvarPGUSER/envar/term Index: ref/pg_restore.sgml === RCS file: /projects/cvsroot/pgsql/doc/src/sgml/ref/pg_restore.sgml,v retrieving revision 1.77 diff -c -2 -r1.77 pg_restore.sgml *** ref/pg_restore.sgml 5 Jan 2009 16:54:36 - 1.77 --- ref/pg_restore.sgml 21 Jan 2009 01:15:41 - *** *** 538,541 --- 538,542 varlistentry termenvarPGHOST/envar/term + termenvarPGOPTIONS/envar/term termenvarPGPORT/envar/term termenvarPGUSER/envar/term -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New pg_dump patch -- document statistics collector exception
On Tuesday 20 January 2009 20:22:10 Bryce Nesbitt wrote: This is a proposed patch to document disabling the statistics collector pg_dump activity, and give a bit more visibility to the PGOPTIONS environment variable supported by libpq. It is an alternative to the prior patch, which supplied a --no-stats flag. This is a documentation only patch, not tied to a recent code change. s/varriable/variable/g also, I forget which way is proper, but you're inconsistent with your closing tags for application in that paragraph (using both /application and /) -- Robert Treat Conjecture: http://www.xzilla.net Consulting: http://www.omniti.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New pg_dump patch -- document statistics collector exception
Robert Treat wrote: also, I forget which way is proper, but you're inconsistent with your closing tags for application in that paragraph (using both /application and /) They're both acceptable. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Column-Level Privileges
Stephen Frost wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: On the whole I think we have to go back to the original plan of recursively searching the query's expressions after we've finished all the transformations (and have a completed jointree to refer to). This is slightly annoying on the grounds of adding parsing overhead that's completely useless unless per-column privileges are in use. On the other hand, none of the workable alternatives are exactly overhead-free either. Comments? Honestly, I like this approach. There is some additional overhead during parsing, but it seems cleaner and more robust. Also, hopefully in most cases where people are concerned about parse time they're preparing their queries. If it's warrented, we could try doing benchmarks to see how bad the impact is and if we need to do something different. It doesn't strike me as likely to be a significant amount of overhead though. I agree with Stephen's opinion. Indeed, the walker approach requires additional steps during query parsing, but the code obviousness is a significant factor from the point of view of security. Thanks, -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Column-Level Privileges
KaiGai Kohei kai...@ak.jp.nec.com writes: Stephen Frost wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: On the whole I think we have to go back to the original plan of recursively searching the query's expressions after we've finished all the transformations (and have a completed jointree to refer to). Honestly, I like this approach. I agree with Stephen's opinion. Indeed, the walker approach requires additional steps during query parsing, but the code obviousness is a significant factor from the point of view of security. On looking closer, though, it's *still* messy and unobvious :-(. There is no single place in the parser where we have the complete multi-level query tree available in a convenient form for this sort of postprocessing. I've thought of a less painful variant of my third option: instead of making a permanent addition to RangeTblEntry, we can have a transient data structure attached to ParseState that lets us find the JoinExpr nodes for already-parsed joins. I'm going to try that next. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Column-Level Privileges
* Tom Lane (t...@sss.pgh.pa.us) wrote: On looking closer, though, it's *still* messy and unobvious :-(. There is no single place in the parser where we have the complete multi-level query tree available in a convenient form for this sort of postprocessing. That's unfortunate. :/ I've thought of a less painful variant of my third option: instead of making a permanent addition to RangeTblEntry, we can have a transient data structure attached to ParseState that lets us find the JoinExpr nodes for already-parsed joins. I'm going to try that next. Sounds reasonable. I'd be happy to help if there's anything useful I can do at this point. Thanks, Stephen signature.asc Description: Digital signature
