Re: [HACKERS] VirtualXactLockTableInsert
On Fri, 2008-06-27 at 18:00 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: In cases where we know we will assign a real xid, can we just skip the assignment of the virtual xid completely? Even if we could do this I doubt it would be a good idea. It'd destroy the invariant that all transactions have a vxid, which at the very least would create naming problems. Ahh, no, I meant go straight to assigning a real xid, to avoid the wasted effort in inserting a vxid *and* a real xid. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] VirtualXactLockTableInsert
On Fri, 2008-06-27 at 17:44 +0200, Florian G. Pflug wrote: Simon Riggs wrote: When we move from having a virtual xid to having a real xid I don't see any attempt to re-arrange the lock queues. Surely if there are people waiting on the virtual xid, they must be moved across to wait on the actual xid? Otherwise the locking queue will not be respected because we have two things on which people might queue. Anybody explain that? Locks on real xids serve a different purpose than locks on virtual xids. Locks on real xids are used to wait for transaction who touched a certain tuple (in which case they certainly must have acquired a real xid) to end. Locks on vxids on the other hand are used to wait for the ending of transactions which either hold a certain lock or use a snapshot with a xmin earlier than some point in time. indexcmds.c is the only place where VirtualXactLockTableWait() is used - the concurrent index creation needs to wait for all transactions to end which either might not know about the index (after phase 1 and 2), or who might still see tuples not included in the index (before marking the index valid). Thanks, So there is no attempt to migrate the vxid lock queue onto the xid lock queue because it doesn't matter now/yet. That seems fragile, but as long as we know about it we're OK. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Join Removal/ Vertical Partitioning
On Fri, 2008-06-27 at 17:50 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: On Thu, 2008-06-26 at 13:42 -0400, Tom Lane wrote: It might be possible to treat ignore the RHS as a join strategy and try to apply it while forming join relations, which would be late enough to have all the needed info available. Oh, actually have a join node that is a no-op, with a path cost of zero? Not even that: just return the best path(s) for the LHS as the paths for the joinrel. Much neater. Cool. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TODO item: Allow data to be pulled directly from indexes
Karl Schnaitter wrote: The main difference between (1) (4) is that (1) will sometimes require heap lookups and (4) never will. Moreover, the heap lookups in (1) will be difficult for the optimizer to estimate, unless some special statistics can be maintained for this purpose. Yeah, we certainly should maintain a statistic for it. (2) (3) can work for any index, and they are quite elegant in the way that the overhead does not change with the number of indexes. The TODO also notes the benefit of (2) for efficient vacuuming. Thus, I think that (2) is a great idea in general, but it does not serve the intended purpose of this TODO item. Once a page gets marked as requiring visibility checks, it cannot be unmarked until the next VACUUM. The whole point of this feature is that we are willing to be more proactive during updates in order to make index access more efficient. In some cases we can mark a page earlier, as soon as we see that the condition is true. Most importantly, when new tuples are inserted, we can mark the page as soon as the inserting transaction is visible to all. Also, the visibility map ought to make vacuums cheaper, as you only need to scan the parts of the table that have beem modified since last vacuum. You still need to scan all indexes, though. But assuming that you somehow solve the correctness issues in the add visibility fields to index tuples approach, we can use the same solution to perform retail vacuums, which would bring vacuuming and the visibility map approach on par with that approach anyway. Please let me know if you agree/disagree with anything here. I need to get this feature implemented for my research, but I would also love to contribute it to the community so your opinions matter a lot. Well, I think the visibility map is a much better approach. This has been discussed many times before, so I don't really have anything new to add. I've been working adding support for so-called relation forks (http://archives.postgresql.org/message-id/[EMAIL PROTECTED]), to allow attaching metadata to relations, like the visibility map. I'm going to use the facility for a new FSM implementation, which I'm working on at the moment, but after that's done I'm going to start working on the visibility map. And after that's done and working for VACUUM, I'm going to work on using it for index-only-scans. I'm not sure I have enough time to get all that done for 8.4, it's looking bad at the moment, so help would be much appreciated. If you don't agree with taking the visibility map approach, I would suggest working on the indexam API changes first, to allow returning index tuples from an index. I believe that part is the same regardless of how we check the visibility. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: Does anything dump per-database config settings? (was Re: [HACKERS] ALTER DATABASE vs pg_dump)
Tom Lane wrote: So put forward a worked-out proposal for some other behavior. OK My first thought is that the -c and -C options create a lot of the issues in this area. -c in particular is evidently meant for merging a dump into a database that already contains unrelated objects. (In fact you could argue that the *default* behavior is meant for this, -c just changes the result for conflicts.) It seems unlikely that having pg_dump issue ALTER DATABASE SET commands is a good idea in all of these scenarios. Can't comment on --clean since I don't use it. I've always assumed it's for the case where you don't have a user with permissions to drop/recreate a database (e.g. web hosting). IMHO the time a dump/restore should be issuing ALTER...SET on a database is when it has issued the corresponding CREATE DATABASE. If you want to tweak this sort of thing, just manually create the database with whatever options you want and don't use --create. I'm also wondering why it'd be bright to treat ALTER ... SET properties different from, say, database owner and encoding properties. Not sure what you mean here. -- Richard Huxton Archonet Ltd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Postgresql 8.3 issue
Hi, I am trying to build a new installer application. I am in the process of upgrading postgresql 8.1 to 8.3.3 but i am having a issue which i can't seemed to resolve. Error output from rpmbuilder + su -c - user'$RPM_BUILD_ROOT/usr/local/app/pgsql/bin/postmaster -D $RPM_BUILD_ROOT/usr/local/app/pgsql/data -S ' /var/tmp/app-root/usr/local/app/pgsql/bin/postmaster: option requires an argument -- S Try postmaster --help for more information. error: Bad exit status from /var/tmp/rpm-tmp.12297 (%install) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TODO item: Allow data to be pulled directly from indexes
Heikki Linnakangas [EMAIL PROTECTED] writes: I'm not sure I have enough time to get all that done for 8.4, it's looking bad at the moment, so help would be much appreciated. If you don't agree with taking the visibility map approach, I would suggest working on the indexam API changes first, to allow returning index tuples from an index. I believe that part is the same regardless of how we check the visibility. That part is the elephant in the room in all these discussions. I wonder if we want to have a new plan node for the heap accesses so they can be postponed up above a join or other quals. Even for tuples which are of questionable visibility we ought to be able to check cheap quals first before checking visibility (though we might need a new function property to indicate it's safe to call extra times on data which doesn't really exist -- immutable doesn't mean it might not throw errors, for example). -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's PostGIS support! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Processing database query-results piecemeal
I'm looking at the most efficient and lean way to interface with the
DB in a least-overhead scenario to process large(r) amounts of binary
data.
For simplicity, I want to avoid using the Large-Object facility.
It seems that the most efficient way to communicate with the DB would
be through PQexecParams(), which avoids the whole bytea-encoding issues.
However, two questions spring to mind:
- The docs say that you can use $1, $2, etc. to reference parameters.
What happens if you have more than 9 parameters?
Does it become $10 or ${10} or $(10) or is it simply not possible
te reference more than nine parameters this way?
- Say that the SELECT returns 1000 rows of 100MB each, is there a way
to avoid PQexecParams() from wanting to allocate 1000*100MB = 100GB
at once, and somehow extract the rows in smaller chunks?
(Incidentally, MySQL has such a facility).
I.e. we call libpq several times, and get a few rows at a time, which
are read from the DB-stream when needed.
--
Sincerely,
Stephen R. van den Berg.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Processing database query-results piecemeal
At 2008-06-30 13:17:42 +0200, [EMAIL PROTECTED] wrote:
It seems that the most efficient way to communicate with the
DB would be through PQexecParams(), which avoids the whole
bytea-encoding issues.
Yes.
Does it become $10 or ${10} or $(10) or is it simply not possible
te reference more than nine parameters this way?
$10 etc.
- Say that the SELECT returns 1000 rows of 100MB each, is there a way
to avoid PQexecParams() from wanting to allocate 1000*100MB = 100GB
Use a cursor and keep executing FETCH.
-- ams
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Git Repository for WITH RECURSIVE and others
David Fetter wrote: On Tue, Jun 24, 2008 at 10:27:28PM +0200, Magnus Hagander wrote: Now, if you can give us a step-by-step on how to set it up, that would certainly help ;-) Gitosis does not, as far as I can tell, have that delegation capability, but I've come up with a way to do this: 1. Use git-shell. Yes, this does involve creating one shell account for each project, but git-shell is, by design, very short on exploitable capability. 2. Make the .ssh directory a git repository. 3. Edit .ssh/authorized_keys and push via git. I was looking into being able to do it using gitosis, with an interface on top of it's existing GIT repository for being able to delegate this. I discussed this with gitosis's author, and he wants to keep gitosis from becoming a sourceforge reimplementation. He did, however, commit to stamping it 1.0 and putting up a TODO list. I'd like to package it up for FreeBSD and Fedora, those being two common platforms. That would be good. It *would* be good, if the author seemed even vaguely interested in packaging up so much as a tarball, but he is not. His attitude is (paraphrasing from conversations with him the past few days), it's good enough as a git repository, and everybody who's using it is a git administrator, so they should know how to wrangle git repositories. While he may someday outgrow this, we really should not put him and his attitude in critical paths for our project. Let's go with git-shell, which is supported and packaged software on just about every platform, and stop waiting for Godot^Wgitosis. I'm not sure I agree that this is a big problem, but sure, we should at least consider git-shell. Is there any product out there that makes it possible to admin a git-shell based system without having all the admins being root on the server? Because that's simply not an option if you want anything remotely scalable. What do you think of this idea? It's complicated :( Wouldn't it be easier to have a gitosis admin team with the needed access? Yes, that'd probably be easier, and it's what I'd start the implementation out at. Here's an even simpler implementation: git-ssh and public keys. Yes, it involves work by administrators, which I'd be delighted to do. Are you referring to git-shell, or is this a different product? If so, reference to said product, please? I certainly don't mind having the work pushed off to an admin team. But it has to be automated enough that there is no risk that different people se tit up differently. And it must not require root. Show me such a solution, and I'll be happy to consider it :-) //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Auto-explain patch
Hi, This is a small patch I posted a few months back, and then kinda forgot about / got distracted with other things. Is there any interest in this? If so I am willing to put more work into it, if people like it or have suggested improvements. Otherwise I'll let it drop. Here's what is does: As it stands, basically it's just another debug parameter, called debug_explain_plan, similar to debug_print_plan, except that the output is in the form of EXPLAIN ANALYSE. The main advantage it offers over a standard EXPLAIN ANALYSE is that it explains *all* SQL executed, including any from within stored prodecures and triggers, so it is quite useful for debugging these. With a suitable logging level, it can also be used to produce very verbose logfile output to help spot any inefficient database access by other applications. Example usage: test=# SET debug_explain_plan=on; SET test=# SET client_min_messages=debug1; SET test=# CREATE TABLE foo(a int primary key); NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index foo_pkey for table foo CREATE TABLE test=# INSERT INTO foo VALUES (1), (2), (3), (4), (5); DEBUG: --- query plan --- DETAIL: Values Scan on *VALUES* (cost=0.00..0.06 rows=5 width=4) (actual time=0.001..0.006 rows=5 loops=1) Query runtime: 0.329 ms INSERT 0 5 test=# CREATE FUNCTION foo() RETURNS int as 'select max(a) from foo;' LANGUAGE SQL STABLE; CREATE FUNCTION test=# SELECT * FROM foo WHERE a=foo(); DEBUG: --- query plan --- DETAIL: Result (cost=0.04..0.05 rows=1 width=0) (actual time=0.044..0.044 rows=1 loops=1) InitPlan - Limit (cost=0.00..0.04 rows=1 width=4) (actual time=0.032..0.034 rows=1 loops=1) - Index Scan Backward using foo_pkey on foo (cost=0.00..84.25 rows=2400 width=4) (actual time=0.025..0.025 rows=1 loops=1) Filter: (a IS NOT NULL) Query runtime: 0.050 ms CONTEXT: SQL function foo statement 1 DEBUG: --- query plan --- DETAIL: Result (cost=0.04..0.05 rows=1 width=0) (actual time=0.037..0.037 rows=1 loops=1) InitPlan - Limit (cost=0.00..0.04 rows=1 width=4) (actual time=0.027..0.029 rows=1 loops=1) - Index Scan Backward using foo_pkey on foo (cost=0.00..84.25 rows=2400 width=4) (actual time=0.021..0.021 rows=1 loops=1) Filter: (a IS NOT NULL) Query runtime: 0.044 ms CONTEXT: SQL function foo statement 1 DEBUG: --- query plan --- DETAIL: Index Scan using foo_pkey on foo (cost=0.25..8.52 rows=1 width=4) (actual time=1.638..1.642 rows=1 loops=1) Index Cond: (a = foo()) Query runtime: 1.686 ms a --- 5 (1 row) test=# EXPLAIN SELECT * FROM foo WHERE a=foo(); DEBUG: --- query plan --- DETAIL: Result (cost=0.04..0.05 rows=1 width=0) (actual time=0.012..0.012 rows=1 loops=1) InitPlan - Limit (cost=0.00..0.04 rows=1 width=4) (actual time=0.011..0.011 rows=1 loops=1) - Index Scan Backward using foo_pkey on foo (cost=0.00..84.25 rows=2400 width=4) (actual time=0.010..0.010 rows=1 loops=1) Filter: (a IS NOT NULL) Query runtime: 0.014 ms CONTEXT: SQL function foo statement 1 QUERY PLAN Index Scan using foo_pkey on foo (cost=0.25..8.52 rows=1 width=4) Index Cond: (a = foo()) (2 rows) (The last example shows foo() being called during the planning of this query, which explains why it is called twice during the previous execution) Simon Riggs reviewed this last time and said that what this patch currently does is probably not exactly what is wanted for PostgreSQL. Possible improvements might be to integrate this with the EXPLAIN command (eg. EXPLAIN TRACE query) and have a separate parameter (log_explain) for logging purposes. Comments? Regards, Dean _ Live Search Charades - guess correctly and find hidden videos http://www.searchcharades.com/*** ./doc/src/sgml/config.sgml.orig Sun Jan 27 19:12:28 2008 --- ./doc/src/sgml/config.sgml Sun Jan 27 21:58:32 2008 *** *** 2790,2795 --- 2790,2796 termvarnamedebug_print_rewritten/varname (typeboolean/type)/term termvarnamedebug_print_plan/varname (typeboolean/type)/term termvarnamedebug_pretty_print/varname (typeboolean/type)/term + termvarnamedebug_explain_plan/varname (typeboolean/type)/term indexterm primaryvarnamedebug_print_parse/ configuration parameter/primary /indexterm *** *** 2802,2807 --- 2803,2811 indexterm primaryvarnamedebug_pretty_print/ configuration parameter/primary /indexterm + indexterm +primaryvarnamedebug_explain_plan/ configuration parameter/primary + /indexterm listitem para These
[HACKERS] TODO assignment
Hello I wish to ask, whether is somebody working on this TODO point: --- GRANT/REVOKE * Allow column-level privileges * %Allow GRANT/REVOKE permissions to be applied to all schema objects with one command The proposed syntax is: GRANT SELECT ON ALL TABLES IN public TO phpuser; GRANT SELECT ON NEW TABLES IN public TO phpuser; * Allow GRANT/REVOKE permissions to be inherited by objects based on schema permissions * Allow SERIAL sequences to inherit permissions from the base table? --- Or more generally, is there a list of what is who doing? Thanks Jiri Dvorak -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Auto-explain patch
On 6/30/08, Dean Rasheed [EMAIL PROTECTED] wrote: This is a small patch I posted a few months back, and then kinda forgot about / got distracted with other things. Is there any interest in this? If so I am willing to put more work into it, if people like it or have suggested improvements. Otherwise I'll let it drop. +1 for including it in whatever form. It is useful. We actually already use in live settings (8.2 / 8,3). (The last example shows foo() being called during the planning of this query, which explains why it is called twice during the previous execution) Simon Riggs reviewed this last time and said that what this patch currently does is probably not exactly what is wanted for PostgreSQL. Possible improvements might be to integrate this with the EXPLAIN command (eg. EXPLAIN TRACE query) and have a separate parameter (log_explain) for logging purposes. I don't have strong opinion either way, It seems its more question on style than any technical details. Just that plain EXPLAN MORE is not enough, it would bo good have a way to turn it on in global/session level for all queries. -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Git Repository for WITH RECURSIVE and others
On Mon, Jun 30, 2008 at 01:50:26PM +0200, Magnus Hagander wrote: David Fetter wrote: [gitosis] It *would* be good, if the author seemed even vaguely interested in packaging up so much as a tarball, but he is not. His attitude is (paraphrasing from conversations with him the past few days), it's good enough as a git repository, and everybody who's using it is a git administrator, so they should know how to wrangle git repositories. While he may someday outgrow this, we really should not put him and his attitude in critical paths for our project. Let's go with git-shell, which is supported and packaged software on just about every platform, and stop waiting for Godot^Wgitosis. I'm not sure I agree that this is a big problem, but sure, we should at least consider git-shell. Please explain your reasoning here. The project has taken nasty hits on its infrastructure already (pgfoundry) because the author of the software had a go-it-alone, I-know-best attitude that sooner than later forced us to fork. As a direct consequence, pgfoundry now needs a redo that will take a pgfoundry administrator many of work in their ample spare time. Let's not cause more pinch points here. Is there any product out there that makes it possible to admin a git-shell based system without having all the admins being root on the server? Because that's simply not an option if you want anything remotely scalable. I don't know what you mean by remotely scalable, but it's clearly not the same definition I have. A sudo wrapper which only allows creation, editing and deletion of accounts restricted to git-shell will scale just fine. Here's an even simpler implementation: git-ssh and public keys. Yes, it involves work by administrators, which I'd be delighted to do. Are you referring to git-shell, or is this a different product? If so, reference to said product, please? Same. I certainly don't mind having the work pushed off to an admin team. But it has to be automated enough that there is no risk that different people set it up differently. OK And it must not require root. This is what sudo is built to do :) Show me such a solution, and I'll be happy to consider it :-) 1. Create a (set of) program(s) which does exactly the following things: * Create an account with git-ssh as its shell. * Manipulate the contact information, ssh keys and groups of said account. * Delete the account. 2. Create a unix group and corresponding sudo role that accesses the above. 3. Create shell accounts as needed with the above group. Yes, that's a root-only task, but it's a short one. I believe that the above takes care of 90% or more of tasks. If it turns out that we need to automate more, we can add that (semi)automation to the capabilities above :) Cheers, David. -- David Fetter [EMAIL PROTECTED] http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: [EMAIL PROTECTED] Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] A new take on the foot-gun meme
On Mon, Jun 30, 2008 at 12:57:03AM -0400, Tom Lane wrote: So is that a golf club gun? Maybe a gun club? A footclub? A foot-gun that can create 18 holes (36 for professionals)? Anyway, with recent USian legal rulings, I anticipate we'll have to stop talking about feet-guns. People will complain we are interfering with their right to bare feet. Or something. (Apologies, everyone. I guess I better go have more coffee.) A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] VirtualXactLockTableInsert
Simon Riggs [EMAIL PROTECTED] writes: So there is no attempt to migrate the vxid lock queue onto the xid lock queue because it doesn't matter now/yet. That seems fragile, but as long as we know about it we're OK. It never will matter, because vxids and xids are unrelated. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Git Repository for WITH RECURSIVE and others
David Fetter [EMAIL PROTECTED] writes: Please explain your reasoning here. The project has taken nasty hits on its infrastructure already (pgfoundry) because the author of the software had a go-it-alone, I-know-best attitude that sooner than later forced us to fork. As a direct consequence, pgfoundry now needs a redo that will take a pgfoundry administrator many of work in their ample spare time. Let's not cause more pinch points here. Well sure, but I'm not sure the software used to distribute the program makes the main difference there. I don't know much about the two programs, what makes you think one is more of a go-it-alone style of development than the other? -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Get trained by Bruce Momjian - ask me about EnterpriseDB's PostgreSQL training! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Git Repository for WITH RECURSIVE and others
David Fetter wrote: On Mon, Jun 30, 2008 at 01:50:26PM +0200, Magnus Hagander wrote: David Fetter wrote: [gitosis] It *would* be good, if the author seemed even vaguely interested in packaging up so much as a tarball, but he is not. His attitude is (paraphrasing from conversations with him the past few days), it's good enough as a git repository, and everybody who's using it is a git administrator, so they should know how to wrangle git repositories. While he may someday outgrow this, we really should not put him and his attitude in critical paths for our project. Let's go with git-shell, which is supported and packaged software on just about every platform, and stop waiting for Godot^Wgitosis. I'm not sure I agree that this is a big problem, but sure, we should at least consider git-shell. Please explain your reasoning here. The project has taken nasty hits on its infrastructure already (pgfoundry) because the author of the software had a go-it-alone, I-know-best attitude that sooner than later forced us to fork. As a direct consequence, pgfoundry now needs a redo that will take a pgfoundry administrator many of work in their ample spare time. If the reason for this is that the software isn't usable, that's one thing. If it's just the author that considers a git snapshot is my release packaging, not a tarball, I don't see how that in itself has any effect on the quality of the software. If that's the only thing it's saying, I don't think that in itself is enough to disqualify gitosis. Is there any product out there that makes it possible to admin a git-shell based system without having all the admins being root on the server? Because that's simply not an option if you want anything remotely scalable. I don't know what you mean by remotely scalable, but it's clearly not the same definition I have. A sudo wrapper which only allows creation, editing and deletion of accounts restricted to git-shell will scale just fine. A properly working sudo wrapper that will let you do *everything needed* is good enough for me. Here's an even simpler implementation: git-ssh and public keys. Yes, it involves work by administrators, which I'd be delighted to do. Are you referring to git-shell, or is this a different product? If so, reference to said product, please? Same. Ok, good. And it must not require root. This is what sudo is built to do :) Yes. Show me such a solution, and I'll be happy to consider it :-) 1. Create a (set of) program(s) which does exactly the following things: * Create an account with git-ssh as its shell. * Manipulate the contact information, ssh keys and groups of said account. * Delete the account. Rght. Is there a product out there already that lets us do this, or is it something we need to write ourselves? You'll also need scripts to create and modify the GIT responsitories themselves, no? Since it's sudo, it has to be secure after all, so it's not necessarily a 2 minute hack. 2. Create a unix group and corresponding sudo role that accesses the above. 3. Create shell accounts as needed with the above group. Yes, that's a root-only task, but it's a short one. Um, not following that step. What account are you talking about here? Creating the accounts for the admins? That's not an issue, since I assume that's not something that would be done very often :-) I believe that the above takes care of 90% or more of tasks. If it turns out that we need to automate more, we can add that (semi)automation to the capabilities above :) As long as it allows it. For example, having a webserver do sudo is not something that makes me feel very safe (and yes, I've seen solutions that do that claiming to be secure. And sure, you *can* build them secure, it's just a lot harder than most people who choose to do it are aware of) //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Auto-explain patch
On Mon, Jun 30, 2008 at 6:34 AM, Dean Rasheed [EMAIL PROTECTED] wrote: Hi, This is a small patch I posted a few months back, and then kinda forgot about / got distracted with other things. Is there any interest in this? If so I am willing to put more work into it, if people like it or have suggested improvements. Otherwise I'll let it drop. Here's what is does: As it stands, basically it's just another debug parameter, called debug_explain_plan, similar to debug_print_plan, except that the output is in the form of EXPLAIN ANALYSE. snip Simon Riggs reviewed this last time and said that what this patch currently does is probably not exactly what is wanted for PostgreSQL. Possible improvements might be to integrate this with the EXPLAIN command (eg. EXPLAIN TRACE query) and have a separate parameter (log_explain) for logging purposes. Comments? Its certainly not useful to *me* in its current form. It would produce way to much (usless) output. However if it were tied to log_min_duration_statement so I get auto explains for long running queries... That would be very useful indeed. Even if it has to explain everything just to toss out the explain if it did not meet log_min_duration_statement. Unless I missed something and thats exactly what it does? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP patch: reducing overhead for repeat de-TOASTing
Tom Lane [EMAIL PROTECTED] writes: After playing with it for a little bit, I'm not convinced that it buys enough performance win to be worth applying --- the restriction of cache lifespan to one tuple cycle of a TupleTableSlot is awfully restrictive. (For example, sorts that involve toasted sort keys continue to suck, because the tuples being sorted aren't in Slots.) It would probably fix the specific case that the PostGIS hackers were complaining of, but I think we need something more. Still, I wanted to get it into the archives because the idea of indirect toast pointers might be useful for something else. I do like that it handles even inline-compressed cases. What I didn't like about the managed cache was that it couldn't handle such cases. I could easily imagine the PostGIS case arising for inline compressed data structures. I wonder if it isn't worthwhile just for that case even if there's a further cache behind it for repeated fetches of out-of-line data. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's RemoteDBA services! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] Fragments in tsearch2 headline
1. Respects ShortWord and MinWords
2. Uses hlCover instead of Cover
3. Does not store norm (or lexeme) for headline marking
4. Removes ts_rank.h
5. Earlier it was counting even NONWORDTOKEN in the headline. Now it
only counts the actual words and excludes spaces etc.
I have also changed NumFragments option to MaxFragments as there may not
be enough covers to display NumFragments.
Nice. But it will be good to resolve following issues:
1) Patch contains mistakes, I didn't investigate or carefully read it. Get
http://www.sai.msu.su/~megera/postgres/fts/apod.dump.gz and load in db.
Queries
# select ts_headline(body, plainto_tsquery('black hole'), 'MaxFragments=1') from
apod where to_tsvector(body) @@ plainto_tsquery('black hole');
and
# select ts_headline(body, plainto_tsquery('black hole'), 'MaxFragments=1') from
apod;
crash postgresql :(
2) pls, include in your patch documentation and regression tests.
Another change that I was thinking:
Right now if cover size max_words then I just cut the trailing words.
Instead I was thinking that we should split the cover into more
fragments such that each fragment contains a few query words. Then each
fragment will not contain all query words but will show more occurrences
of query words in the headline. I would like to know what your opinion
on this is.
Agreed.
--
Teodor Sigaev E-mail: [EMAIL PROTECTED]
WWW: http://www.sigaev.ru/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Bucket and batch
Hi! I saw that functions which use hash join, like as ExecHashGetBucketAndBatch and ExecScanHashBucket, have bucket and batch concepts. So, I would like to know the differences between bucket and batch. I dont understand tables partitioning mechanism in hash join. Thanks, Ana Carolina
Re: [HACKERS] psql wrapped format default for backslash-d commands
Bryce Nesbitt wrote: Brendan Jurd wrote: I really like the idea of wrapping, but after playing with the format a bit myself, I have to agree with Tom that breaking in the middle of words produces some very nasty output. If the format could be improved to only wrap on word boudaries, that would increase its appeal dramatically. Anybody got a rough idea how difficult it would be to add word-awareness int o the wrapping code? I chose not to, when writing the patch, because it makes the result flow over many more lines. And regardless, it pretty much has to cut long words, of which there are many in typical SQL output. And, I hardly ever read actual large blocks of English at the SQL prompt, so: designing for that use case did not float my boat at the time. And, word wrapping messes with the cell contents more than I was comfortable doing. With the current wrapping all whitespace is preserved and apparent. But, if your boat is floated, then by all means go for it. Added to TODO: o Add option to wrap column values at whitespace boundaries, rather than chopping them at a fixed width. Currently, wrapped format chops values into fixed widths. Perhaps the word wrapping could use the same algorithm documented in the W3C specification. http://archives.postgresql.org/pgsql-hackers/2008-05/msg00404.php http://www.w3.org/TR/CSS21/tables.html#auto-table-layout -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql wrapped format default for backslash-d commands
Bruce Momjian wrote: Also, about the format 'auto' idea that uses expanded display, I am thinking because expanded is a separate setting and not a format, I should just add a possible 'auto' value to the expanded format that could be triggered by either 'aligned' or 'wrapped' formats when the output doesn't fit on the screen. (In hindsight, I think we should have had pager be on/auto/off rather than always/on/off; the later is kind of confusing and would be inconsistent if I add 'auto' to expanded.) Added to psql TODO: o Add auto expanded mode that outputs in expanded format if wrapped mode can't wrap the output to the screen width -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove redundant extra_desc info for enum GUC variables?
Tom Lane wrote:
Magnus Hagander [EMAIL PROTECTED] writes:
Tom Lane wrote:
Yeah: LOG level sorts differently in the two cases; it's fairly high
priority for server log output and much lower for client output.
Ok, easy fix if we break them apart. Should we continue to accept
values that we're not going to care about, or should I change that at
the same time? (for example, client_min_messages doesn't use INFO,
but we do accept that in = 8.3 anyway)
I'd be inclined to keep the actual behavior the same as it was.
We didn't document INFO for this variable, perhaps, but it's accepted
and has a well-defined behavior.
Sorry for not getting back to this one sooner, it ended up in the wrong
end of the queue.
Does this patch look like what you meant? It should split them apart,
and it also hides the undocumented levels, but still accept it (now that
we have the ability to hide GUC vars)
//Magnus
Index: src/backend/utils/misc/guc.c
===
RCS file: /cvsroot/pgsql/src/backend/utils/misc/guc.c,v
retrieving revision 1.457
diff -c -r1.457 guc.c
*** src/backend/utils/misc/guc.c 30 Jun 2008 10:58:47 - 1.457
--- src/backend/utils/misc/guc.c 30 Jun 2008 19:50:24 -
***
*** 172,178
/*
* Options for enum values defined in this module.
*/
! static const struct config_enum_entry message_level_options[] = {
{debug, DEBUG2, false},
{debug5, DEBUG5, false},
{debug4, DEBUG4, false},
--- 172,183
/*
* Options for enum values defined in this module.
*/
!
! /*
! * We have different sets for client and server message level options because
! * they sort slightly different (see log level)
! */
! static const struct config_enum_entry client_message_level_options[] = {
{debug, DEBUG2, false},
{debug5, DEBUG5, false},
{debug4, DEBUG4, false},
***
*** 180,189
--- 185,211
{debug2, DEBUG2, false},
{debug1, DEBUG1, false},
{log, LOG, false},
+ {info, INFO, true},
+ {notice, NOTICE, false},
+ {warning, WARNING, false},
+ {error, ERROR, false},
+ {fatal, FATAL, true},
+ {panic, PANIC, true},
+ {NULL, 0, false}
+ };
+
+ static const struct config_enum_entry server_message_level_options[] = {
+ {debug, DEBUG2, false},
+ {debug5, DEBUG5, false},
+ {debug4, DEBUG4, false},
+ {debug3, DEBUG3, false},
+ {debug2, DEBUG2, false},
+ {debug1, DEBUG1, false},
{info, INFO, false},
{notice, NOTICE, false},
{warning, WARNING, false},
{error, ERROR, false},
+ {log, LOG, false},
{fatal, FATAL, false},
{panic, PANIC, false},
{NULL, 0, false}
***
*** 2449,2461
{
{client_min_messages, PGC_USERSET, LOGGING_WHEN,
gettext_noop(Sets the message levels that are sent to the client.),
! gettext_noop(Valid values are DEBUG5, DEBUG4, DEBUG3, DEBUG2,
! DEBUG1, LOG, NOTICE, WARNING, and ERROR. Each level includes all the
! levels that follow it. The later the level, the fewer messages are
! sent.)
},
client_min_messages,
! NOTICE, message_level_options,NULL, NULL
},
{
--- 2471,2481
{
{client_min_messages, PGC_USERSET, LOGGING_WHEN,
gettext_noop(Sets the message levels that are sent to the client.),
! gettext_noop(Each level includes all the levels that follow it. The later
! the level, the fewer messages are sent.)
},
client_min_messages,
! NOTICE, client_message_level_options,NULL, NULL
},
{
***
*** 2480,2491
{
{log_min_messages, PGC_SUSET, LOGGING_WHEN,
gettext_noop(Sets the message levels that are logged.),
! gettext_noop(Valid values are DEBUG5, DEBUG4, DEBUG3, DEBUG2, DEBUG1,
! INFO, NOTICE, WARNING, ERROR, LOG, FATAL, and PANIC. Each level
! includes all the levels that follow it.)
},
log_min_messages,
! WARNING, message_level_options, NULL, NULL
},
{
--- 2500,2509
{
{log_min_messages, PGC_SUSET, LOGGING_WHEN,
gettext_noop(Sets the message levels that are logged.),
! gettext_noop(Each level includes all levels that follow it.)
},
log_min_messages,
! WARNING, server_message_level_options, NULL, NULL
},
{
***
*** 2495,2501
specified level or a higher level are logged.)
},
log_min_error_statement,
! ERROR, message_level_options, NULL, NULL
},
{
--- 2513,2519
specified level or a higher level are logged.)
},
log_min_error_statement,
! ERROR, server_message_level_options, NULL, NULL
},
{
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove redundant extra_desc info for enum GUC variables?
Magnus Hagander [EMAIL PROTECTED] writes: Does this patch look like what you meant? It should split them apart, and it also hides the undocumented levels, but still accept it (now that we have the ability to hide GUC vars) Seems reasonable, although I'm still dissatisfied with the handling of the debug alias for debug2. I think if it's not hidden then it has to be placed in correct sort position. Since it's not documented in config.sgml, I think marking it hidden would be fine. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove redundant extra_desc info for enum GUC variables?
Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Does this patch look like what you meant? It should split them apart, and it also hides the undocumented levels, but still accept it (now that we have the ability to hide GUC vars) Seems reasonable, although I'm still dissatisfied with the handling of the debug alias for debug2. I think if it's not hidden then it has to be placed in correct sort position. Since it's not documented in config.sgml, I think marking it hidden would be fine. Good point, and thanks for the quick review. Will fix and apply. //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Fairly serious bug induced by latest guc enum changes
Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Right, but I still need the other part of the check, right? This one still fails the same check as my patch, no? Because I assume the hole you found there was that get_sync_bit() will return 0 for two different sync methods as long as none of them are O_SYNC or O_DSYNC... No, my point was that there are three possible states of sync_bit and your patch only accounted for transitions between two of 'em. For instance, if sync_bit goes to 0 we must close and reopen the file, else we'll be doing both O_SYNC flush and whatever flush method is supposed to be getting used. Did this every get addressed? I don't see a commit for it. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Fairly serious bug induced by latest guc enum changes
Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: No, my point was that there are three possible states of sync_bit and your patch only accounted for transitions between two of 'em. Did this every get addressed? I don't see a commit for it. I thought it got fixed here: 2008-05-14 10:02 mha * src/backend/access/transam/xlog.c: Remove the special variable for open_sync_bit used in O_SYNC and O_DSYNC modes, replacing it with a call to a function that derives it from the sync_method variable, now that it has distinct values for these two cases. This means that assign_xlog_sync_method() no longer changes any settings, thus fixing the bug introduced in the change to use a guc enum for wal_sync_method. Hmm ... or at least more or less fixed. Seems like there's no provision to close and reopen the file if enableFsync changes. Not sure if that's worth worrying about. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] odd output in restore mode
Andrew Dunstan wrote:
I have just been working on setting up a continuous recovery failover
system, and noticed some odd log lines, shown below. (Using 8.3).
First note that our parsing of recovery.conf in xlog.c is pretty bad,
and at least we need to document the quirks if it's not going to be
fixed. log_restartpoints is said to be boolean, but when I set it to an
unquoted true I got a fatal error, while a quoted 'on' sets it to false,
as seen. Ick. What is more, I apparently managed to get the recovery
I have fixed the boolean problem with the attached, applied patch. It
exposes guc.c::parse_bool() for use in xlog.c.
I assume all the other problems you reported have been corrected.
--
Bruce Momjian [EMAIL PROTECTED]http://momjian.us
EnterpriseDB http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
Index: src/backend/access/transam/xlog.c
===
RCS file: /cvsroot/pgsql/src/backend/access/transam/xlog.c,v
retrieving revision 1.314
diff -c -c -r1.314 xlog.c
*** src/backend/access/transam/xlog.c 12 Jun 2008 09:12:30 - 1.314
--- src/backend/access/transam/xlog.c 30 Jun 2008 22:10:07 -
***
*** 4523,4535
/*
* does nothing if a recovery_target is not also set
*/
! if (strcmp(tok2, true) == 0)
! recoveryTargetInclusive = true;
! else
! {
! recoveryTargetInclusive = false;
! tok2 = false;
! }
ereport(LOG,
(errmsg(recovery_target_inclusive = %s, tok2)));
}
--- 4523,4532
/*
* does nothing if a recovery_target is not also set
*/
! if (!parse_bool(tok2, recoveryTargetInclusive))
! ereport(ERROR,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg(parameter \recovery_target_inclusive\ requires a Boolean value)));
ereport(LOG,
(errmsg(recovery_target_inclusive = %s, tok2)));
}
***
*** 4538,4550
/*
* does nothing if a recovery_target is not also set
*/
! if (strcmp(tok2, true) == 0)
! recoveryLogRestartpoints = true;
! else
! {
! recoveryLogRestartpoints = false;
! tok2 = false;
! }
ereport(LOG,
(errmsg(log_restartpoints = %s, tok2)));
}
--- 4535,4544
/*
* does nothing if a recovery_target is not also set
*/
! if (!parse_bool(tok2, recoveryLogRestartpoints))
! ereport(ERROR,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg(parameter \log_restartpoints\ requires a Boolean value)));
ereport(LOG,
(errmsg(log_restartpoints = %s, tok2)));
}
Index: src/backend/utils/misc/guc.c
===
RCS file: /cvsroot/pgsql/src/backend/utils/misc/guc.c,v
retrieving revision 1.457
diff -c -c -r1.457 guc.c
*** src/backend/utils/misc/guc.c 30 Jun 2008 10:58:47 - 1.457
--- src/backend/utils/misc/guc.c 30 Jun 2008 22:10:07 -
***
*** 3991,3997
* If the string parses okay, return true, else false.
* If okay and result is not NULL, return the value in *result.
*/
! static bool
parse_bool(const char *value, bool *result)
{
size_t len = strlen(value);
--- 3991,3997
* If the string parses okay, return true, else false.
* If okay and result is not NULL, return the value in *result.
*/
! bool
parse_bool(const char *value, bool *result)
{
size_t len = strlen(value);
Index: src/include/utils/guc.h
===
RCS file: /cvsroot/pgsql/src/include/utils/guc.h,v
retrieving revision 1.96
diff -c -c -r1.96 guc.h
*** src/include/utils/guc.h 28 May 2008 09:04:06 - 1.96
--- src/include/utils/guc.h 30 Jun 2008 22:10:07 -
***
*** 223,228
--- 223,229
extern void AtEOXact_GUC(bool isCommit, int nestLevel);
extern void BeginReportingGUCOptions(void);
extern void ParseLongOption(const char *string, char **name, char **value);
+ extern bool parse_bool(const char *value, bool *result);
extern bool set_config_option(const char *name, const char *value,
GucContext context, GucSource source,
GucAction action, bool changeVal);
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] odd output in restore mode
Dave Page wrote: On Sun, May 18, 2008 at 1:38 PM, Andrew Dunstan [EMAIL PROTECTED] wrote: I don't know how you tested, Copy a large file across a relatively slow network, and check the size on the destination drive before it finishes. but could you please repeat the test with GnuWin32's cp.exe? If it doesn't preallocate the space then I think our way forward is reasonably clear: It does not pre-allocate. . we recommend its use for Windows archive_command settings . we provide the delay kluge as switchable behaviour on Windows instead of having it always on. Sounds reasonable to me. Are there any changes we need to make here? -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] Surprising syntax error
Marc Munro wrote: -- Start of PGP signed section. The statement: revoke all on view internal.all_objects from public; yields a syntax error. The docs show that the word view is not acceptable in this statement which is fine but the surprising thing is that: ? revoke all on table internal.all_objects from public; works fine even though all_objects is a view and not a table. Now that I know about it, this doesn't bother me but it was a surprise and I wonder whether the the parser/planner/whatever should be a bit smarter about allowing the word table to apply to non-tables, and whether the word view ought to be allowed. Yes, I can confirm this behavior on CVS HEAD, and it is odd: test= CREATE SCHEMA internal; CREATE SCHEMA test= CREATE VIEW internal.all_objects AS SELECT 1; CREATE VIEW test= REVOKE ALL ON VIEW internal.all_objects FROM PUBLIC; ERROR: syntax ERROR AT OR near internal LINE 1: REVOKE ALL ON VIEW internal.all_objects FROM PUBLIC; ^ test= REVOKE ALL ON TABLE internal.all_objects FROM PUBLIC; REVOKE Is there a downside to adding VIEW in parser/gram.y:privilege_target? -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusing message in log file
Gurjeet Singh wrote: Hi All, I changed the postgresql.conf file (of an 8.2.4 server), and issued relaod using pg_reload_config(). Following are the messages I see in the log files: May 14 21:38:40 sfphotodb001 postgres[29658]: [19-1] 2008-05-14 21:38:40 PDTLOG: received SIGHUP, reloading configuration files May 14 21:38:40 sfphotodb001 postgres[29658]: [20-1] 2008-05-14 21:38:40 PDTLOG: parameter shared_buffers cannot be changed after server start; configuration file change ignored May 14 21:39:03 sfphotodb001 postgres[22928]: [21-1] 2008-05-14 21:39:03 PDTLOG: archived transaction log file 00010E2300C8 What's confusing about this is that the second message says 'configuration file change ignored', so I expect the changed (newly enabled) archive_command to not take effect. But in fact, it does take effect. The message probably should be rephrased to say that this setting (shared_buffers) will not be changed. Actually, no one else has been confused by this wording before, and I can't think of better wording that doesn't sound redundant. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding variables for segment_size, wal_segment_size and block sizes
Bernd Helmle wrote: Now that we have customizable segment sizes for heap and WAL at compilation time i would like to have some runtime variables to query that information (besides pg_controldata). I can imagine to have the following names: segment_size: Reports heap segment size wal_segment_size: Reports wal segment size block_size: Available yet wal_block_size: wal block size I'd like to implement them if we agree on them Bernd, have you made any progress on this? -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusing message in log file
Bruce Momjian [EMAIL PROTECTED] writes: Gurjeet Singh wrote: May 14 21:38:40 sfphotodb001 postgres[29658]: [20-1] 2008-05-14 21:38:40 PDTLOG: parameter shared_buffers cannot be changed after server start; configuration file change ignored The message probably should be rephrased to say that this setting (shared_buffers) will not be changed. Actually, no one else has been confused by this wording before, and I can't think of better wording that doesn't sound redundant. How about ... after server start; change ignored ? Or attempted change ignored ? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusing message in log file
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Gurjeet Singh wrote: May 14 21:38:40 sfphotodb001 postgres[29658]: [20-1] 2008-05-14 21:38:40 PDTLOG: parameter shared_buffers cannot be changed after server start; configuration file change ignored The message probably should be rephrased to say that this setting (shared_buffers) will not be changed. Actually, no one else has been confused by this wording before, and I can't think of better wording that doesn't sound redundant. How about ... after server start; change ignored ? Or attempted change ignored ? Yea, I like change ignored rather than mentioning the configuration file. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusing message in log file
Bruce Momjian escribió: Gurjeet Singh wrote: May 14 21:38:40 sfphotodb001 postgres[29658]: [19-1] 2008-05-14 21:38:40 PDTLOG: received SIGHUP, reloading configuration files May 14 21:38:40 sfphotodb001 postgres[29658]: [20-1] 2008-05-14 21:38:40 PDTLOG: parameter shared_buffers cannot be changed after server start; configuration file change ignored What's confusing about this is that the second message says 'configuration file change ignored', so I expect the changed (newly enabled) archive_command to not take effect. But in fact, it does take effect. The message probably should be rephrased to say that this setting (shared_buffers) will not be changed. Actually, no one else has been confused by this wording before, and I can't think of better wording that doesn't sound redundant. Perhaps this is because not enough people have seen it. I agree that the message should specify that only this setting has been ignored. In any case, this seems a case of stuffing too much in the primary message. I think it should be something like errmsg(parameter \shared_buffer\ change in configuration file ignored), errdetail(This parameter cannot be changed after server start.) -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Planned obsolescence in identify_system_timezone()
I just noticed that identify_system_timezone() tries to match our timezones to the system timezone on the basis of probing the date range 1904-2004. While doubtless this seemed like a good idea at the time, it means that we'll be unable to distinguish zones whose histories diverge after 2004. I think there are some already, and certainly there are likely to be some in future. I propose modifying this logic to test 100 years back from the current year, instead. Any objections? Should such a change be back-patched? I'm also more than slightly tempted to modify the code so that it will always reject a match to the Factory zone (and fall back to regular GMT instead). The logging behavior illustrated here: http://archives.postgresql.org/pgsql-bugs/2008-06/msg00191.php seems just completely bletcherous to me. It's not our business to nag the user into setting up /etc/localtime, and even if it were, this is obviously an ineffective way to do it ;-) Any objections to that one? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] odd output in restore mode
Bruce Momjian wrote: Dave Page wrote: On Sun, May 18, 2008 at 1:38 PM, Andrew Dunstan [EMAIL PROTECTED] wrote: I don't know how you tested, Copy a large file across a relatively slow network, and check the size on the destination drive before it finishes. but could you please repeat the test with GnuWin32's cp.exe? If it doesn't preallocate the space then I think our way forward is reasonably clear: It does not pre-allocate. . we recommend its use for Windows archive_command settings . we provide the delay kluge as switchable behaviour on Windows instead of having it always on. Sounds reasonable to me. Are there any changes we need to make here? Yes. Simon has promised a patch to do the above. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Confusing message in log file
Alvaro Herrera [EMAIL PROTECTED] writes: In any case, this seems a case of stuffing too much in the primary message. Yeah, good point. I think it should be something like errmsg(parameter \shared_buffer\ change in configuration file ignored), errdetail(This parameter cannot be changed after server start.) Seems a bit awkwardly phrased. I propose errmsg(attempted change to parameter \shared_buffer\ ignored), errdetail(This parameter cannot be changed after server start.) (or possibly of would read better than to) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: Does anything dump per-database config settings? (was Re: [HACKERS] ALTER DATABASE vs pg_dump)
Richard Huxton [EMAIL PROTECTED] writes: Tom Lane wrote: So put forward a worked-out proposal for some other behavior. IMHO the time a dump/restore should be issuing ALTER...SET on a database is when it has issued the corresponding CREATE DATABASE. So pg_dump would produce this info when, and only when, you'd used --create? I agree that it seems sensible in that case, I'm just wondering if that will cover all the use-cases. This would mean duplicating some functionality between pg_dump and pg_dumpall ... or maybe we could move all that logic over to pg_dump and have pg_dumpall use --create when invoking pg_dump. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bucket and batch
Ana Carolina Brito de Almeida [EMAIL PROTECTED] writes: So, I would like to know the differences between bucket and batch. A bucket is, well, one bucket of a hash table --- it holds all the tuples that have the same hash code (for as many bits of the hash code as we are choosing to use). We try to size the hash table with enough buckets so there's not more than 10 tuples per bucket on average. A batch is a range of buckets that we process at the same time. Tuples (from either side of the join) whose hash codes show they fall into batches other than the first one get dumped into temporary holding files, and then (after finishing joining the first batch) we pull each successive batch back into memory and join that portion of the tuples. The batch size is chosen to make the amount of memory needed be approximately work_mem. IOW, there are really nbuckets * nbatches virtual buckets in the hash table, but only nbuckets worth of them are kept in memory at any one time. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Vacuuming leaked temp tables (once again)
Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: We might have to rearrange the logic a bit to make that happen (I'm not sure what order things get tested in), but a log message does seem like a good idea. I'd go for logging anytime an orphaned table is seen, and dropping once it's past the anti-wraparound horizon. I don't think this requires much of a rearrangement -- see autovacuum.c 1921ff. Hmm, maybe I'm missing something but I see no good way to do it without refactoring relation_check_autovac. Since that function is only called in one place, I'm thinking of just inlining it; do you see a reason not to? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Vacuuming leaked temp tables (once again)
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: We might have to rearrange the logic a bit to make that happen (I'm not sure what order things get tested in), but a log message does seem like a good idea. I'd go for logging anytime an orphaned table is seen, and dropping once it's past the anti-wraparound horizon. I don't think this requires much of a rearrangement -- see autovacuum.c 1921ff. Hmm, maybe I'm missing something but I see no good way to do it without refactoring relation_check_autovac. Hmm, oops :-) Since that function is only called in one place, I'm thinking of just inlining it; do you see a reason not to? Nope, go ahead. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the un-vacuumable table
On Fri, Jun 27, 2008 at 8:14 PM, Tom Lane [EMAIL PROTECTED] wrote: Andrew Hammond [EMAIL PROTECTED] writes: (I thought this line was interesting) Jun 27 15:54:31 qadb2 postgres[92519]: [44-1] PANIC: could not open relation 1663/16386/679439393: No such file or directory I googled to find out what the numbers 1663/16386/679439393 from the PANIC message mean, but no luck. tablespaceOID/databaseOID/relfilenode. Looks like just some random user table. Not clear why this would be a crash, *especially* since WAL recovery is generally willing to create nonexistent files. Is this reproducible? Yes, both when I just tried to restart the recovery: Jun 30 16:04:43 qadb2 postgres[20797]: [27-1] DEBUG: invoking IpcMemoryCreate(size=92938240) Jun 30 16:04:43 qadb2 postgres[20797]: [28-1] DEBUG: max_safe_fds = 983, usable_fds = 1000, already_open = 7 Jun 30 16:04:43 qadb2 postgres[20798]: [29-1] LOG: database system was interrupted while in recovery at 2008-06-27 15:54:31 PDT Jun 30 16:04:43 qadb2 postgres[20798]: [29-2] HINT: This probably means that some data is corrupted and you will have to use the last backup for recovery. Jun 30 16:04:43 qadb2 postgres[20798]: [30-1] LOG: starting archive recovery Jun 30 16:04:43 qadb2 postgres[20798]: [31-1] LOG: restore_command = cp -p /usr/tmp/2008-06-25_wals/%f %p Jun 30 16:04:43 qadb2 postgres[20798]: [32-1] DEBUG: executing restore command cp -p /usr/tmp/2008-06-25_wals/0001.history pg_xlog/RECOVERYHISTORY Jun 30 16:04:43 qadb2 postgres[20798]: [33-1] DEBUG: could not restore file 0001.history from archive: return code 256 Jun 30 16:04:43 qadb2 postgres[20798]: [34-1] DEBUG: executing restore command cp -p /usr/tmp/2008-06-25_wals/000101D60078.0055F0B8.backup Jun 30 16:04:43 qadb2 postgres[20798]: [34-2] pg_xlog/RECOVERYHISTORY Jun 30 16:04:43 qadb2 postgres[20798]: [35-1] LOG: restored log file 000101D60078.0055F0B8.backup from archive Jun 30 16:04:43 qadb2 postgres[20798]: [36-1] DEBUG: executing restore command cp -p /usr/tmp/2008-06-25_wals/000101D60078 pg_xlog/RECOVERYXLOG Jun 30 16:04:44 qadb2 postgres[20797]: [29-1] DEBUG: forked new backend, pid=20805 socket=8 Jun 30 16:04:44 qadb2 postgres[20805]: [29-1] LOG: connection received: host=[local] Jun 30 16:04:44 qadb2 postgres[20805]: [30-1] FATAL: the database system is starting up Jun 30 16:04:44 qadb2 postgres[20805]: [31-1] DEBUG: proc_exit(0) Jun 30 16:04:44 qadb2 postgres[20805]: [32-1] DEBUG: shmem_exit(0) Jun 30 16:04:44 qadb2 postgres[20805]: [33-1] DEBUG: exit(0) Jun 30 16:04:44 qadb2 postgres[20797]: [30-1] DEBUG: reaping dead processes Jun 30 16:04:44 qadb2 postgres[20797]: [31-1] DEBUG: server process (PID 20805) exited with exit code 0 Jun 30 16:04:44 qadb2 postgres[20798]: [37-1] LOG: restored log file 000101D60078 from archive Jun 30 16:04:44 qadb2 postgres[20798]: [38-1] LOG: checkpoint record is at 1D6/7855F0B8 Jun 30 16:04:44 qadb2 postgres[20798]: [39-1] LOG: redo record is at 1D6/7855F0B8; undo record is at 0/0; shutdown FALSE Jun 30 16:04:44 qadb2 postgres[20798]: [40-1] LOG: next transaction ID: 397171279; next OID: 679516596 Jun 30 16:04:44 qadb2 postgres[20798]: [41-1] LOG: next MultiXactId: 857318; next MultiXactOffset: 1718141 Jun 30 16:04:44 qadb2 postgres[20798]: [42-1] LOG: automatic recovery in progress Jun 30 16:04:44 qadb2 postgres[20798]: [43-1] LOG: redo starts at 1D6/7855F108 Jun 30 16:04:45 qadb2 postgres[20798]: [44-1] PANIC: could not open relation 1663/16386/679439393: No such file or directory Jun 30 16:04:45 qadb2 postgres[20797]: [32-1] DEBUG: reaping dead processes Jun 30 16:04:45 qadb2 postgres[20797]: [33-1] LOG: startup process (PID 20798) was terminated by signal 6 Jun 30 16:04:45 qadb2 postgres[20797]: [34-1] LOG: aborting startup due to startup process failure Jun 30 16:04:45 qadb2 postgres[20797]: [35-1] DEBUG: proc_exit(1) Jun 30 16:04:45 qadb2 postgres[20797]: [36-1] DEBUG: shmem_exit(1) Jun 30 16:04:45 qadb2 postgres[20797]: [37-1] DEBUG: exit(1) And also when I tried to wipe the slate clean and recover it freshly. Jun 30 19:11:59 qadb2 postgres[23091]: [1-1] DEBUG: postmaster: PostmasterMain: initial environ dump: Jun 30 19:11:59 qadb2 postgres[23091]: [2-1] DEBUG: - Jun 30 19:11:59 qadb2 postgres[23091]: [3-1] DEBUG: USER=pgsql Jun 30 19:11:59 qadb2 postgres[23091]: [4-1] DEBUG: MAIL=/var/mail/pgsql Jun 30 19:11:59 qadb2 postgres[23091]: [5-1] DEBUG: LD_LIBRARY_PATH=:/usr/local/adecn/lib Jun 30 19:11:59 qadb2 postgres[23091]: [6-1] DEBUG: HOME=/usr/local/pgsql Jun 30 19:11:59 qadb2 postgres[23091]: [7-1] DEBUG: PGLIB=/usr/local/lib Jun 30 19:11:59 qadb2 postgres[23091]: [8-1] DEBUG: PS1=[QA2] [EMAIL PROTECTED]:\w\$ Jun 30 19:11:59 qadb2 postgres[23091]: [9-1] DEBUG: BLOCKSIZE=K Jun 30 19:11:59 qadb2 postgres[23091]: [10-1] DEBUG:TERM=xterm Jun 30 19:11:59 qadb2 postgres[23091]: [11-1]
