[HACKERS] Help extending pg_class
Hi all, I added a new variable-length field to the pg_class catalog, but
I did something wrong, and I can't tell what else I'd need to change. (I
know about how extending pg_class is bad and all, but it seems to be the
simplest solution to my problem right now, and I'd just like to get it
working.) The code I'm working with is based on Postgres 7.3.2 (avail.
at http://telegraph.cs.berkeley.edu). I've added a char field to it
before, but now I'm trying to add a text field, which pg_type also has.
I modified pg_class.h and pg_attribute.h (which I've pasted to
http://rafb.net/paste/results/c1CiIo27.html), but when I run initdb, I'm
getting a seg fault in the first postgres being run under the section
'CREATE VIEWS and other things'. gdb tells me that it's happening for
the line:
backend REVOKE ALL on pg_shadow FROM public;
QUERY: REVOKE ALL on pg_shadow FROM public;
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1076389696 (LWP 7710)]
0x0807a6e8 in DataFill (data=0x83d4710 '\177' repeats 200 times...,
tupleDesc=0x83971e4, value=0x83d4400,
nulls=0x83d45d8 ' ' repeats 24 times,
n~\177\177\177\177\177\177$F=\b , infomask=0x83d4688, bit=0x83d468b
ÿÿÿ\002) at heaptuple.c:120
120 if (VARATT_IS_EXTERNAL(value[i]))
(gdb) l
115 }
116 else if (att[i]-attlen == -1)
117 {
118 /* varlena */
119 *infomask |= HEAP_HASVARWIDTH;
120 if (VARATT_IS_EXTERNAL(value[i]))
121 *infomask |= HEAP_HASEXTERNAL;
122 if (VARATT_IS_COMPRESSED(value[i]))
123 *infomask |= HEAP_HASCOMPRESSED;
124 data_length =
VARATT_SIZE(DatumGetPointer(value[i]));
(gdb) p i
$1 = 25
(gdb) bt
#0 0x0807a6e8 in DataFill (data=0x83d4710 '\177' repeats 200 times...,
tupleDesc=0x83971e4, value=0x83d4400,
nulls=0x83d45d8 ' ' repeats 24 times,
n~\177\177\177\177\177\177$F=\b , infomask=0x83d4688, bit=0x83d468b
ÿÿÿ\002) at heaptuple.c:120
#1 0x0807bbc7 in heap_formtuple (tupleDescriptor=0x83971e4,
value=0x83d4400,
nulls=0x83d45d8 ' ' repeats 24 times,
n~\177\177\177\177\177\177$F=\b )
at heaptuple.c:622
#2 0x0807c0fe in heap_modifytuple (tuple=0x403b6a60, relation=0x83970cc,
replValue=0xb5b0,
replNull=0xb590 ' ' repeats 26 times, ÿ¿1\t\b,
repl=0xb570 ' ' repeats 25 times, rÿ¿1\t\b, ' ' repeats
26 times, ÿ¿1\t\b) at heaptuple.c:697
#3 0x080ce977 in ExecuteGrantStmt_Relation (stmt=0x83c1954) at aclchk.c:236
#4 0x080ce4e8 in ExecuteGrantStmt (stmt=0x83c1954) at aclchk.c:138
#5 0x081ec38d in ProcessUtility (parsetree=0x83c1954, dest=Debug,
completionTag=0xb790 ) at utility.c:690
#6 0x081e7fdc in pg_exec_query_string (query_string=0x83c1664, dest=Debug,
parse_context=0x83954ec) at postgres.c:1035
#7 0x081ea453 in PostgresMain (argc=7, argv=0x837e5c8,
username=0x837f988 z) at postgres.c:2774
#8 0x08177692 in main (argc=7, argv=0xb9f4) at main.c:236
The REVOKE command invokes ExecuteGrantStmt_Relation() to modify the
relacl attribute of pg_class, which is the last attribute and also
var-length. My new field is interfering with this operation somehow. For
some reason, in frame 2, the new 'value' array is allocated with length
numberOfAttributes = RelationGetForm(relation)-relnatts = 25, instead
of 26. This looks like a problem (not sure if it's *the* problem), but I
have no idea where/how this was set. I've tried setting the
Natts_pg_class_fixed to both 24 and 25, to no avail.
Please let me know if more details are needed; thanks very much in
advance for any help!
---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Shared row locking
On Sun, 2004-12-19 at 04:04, Bruce Momjian wrote: BTom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: You mean all empty/zero rows can be removed? Can we guarantee that on commit we can clean up the bitmap? If not the idea doesn't work. For whatever data structure we use, we may reset the structure to empty during backend-crash recovery. So your objection boils down to what if a backend exits normally but forgets to clean up its locks? Assuming that doesn't happen isn't any worse than assuming a backend will clean up its shared memory state on non-crash exit, so I don't think it's a serious concern. That brings another thought: really what this is all about is working around the fact that the standard lock manager can only cope with a finite number of coexisting locks, because it's working in a fixed-size shared memory arena. Maybe we should instead think about ways to allow the existing lock table to spill to disk when it gets too big. That would eliminate max_locks_per_transaction as a source of hard failures, which would be a nice benefit. Agreed. Once concern I have about allowing the lock table to spill to disk is that a large number of FOR UPDATE locks could push out lock entries used by other backends, causing very poor performance. In similar circumstances, DB2 uses these techniques: - when locktable X % full, then escalate locks to full table locks: both locktable memory and threshold% are instance parameters - use a lock mode called Cursor Stability that locks only those rows currently being examined by a cursor, those maintaining the lock usage of a cursor at a constant level as the cursor moves. The lock mode of Repeatable Read *does* lock all rows read (these are not actually mutually exclusive) The first one is a real pain, but the idea might be of use somewhere. The second is a usable, practical alternative that should be considered and might avoid the need to write the spill-to-disk code and then discover it performs very badly. -- Best Regards, Simon Riggs ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
[HACKERS] postgres protocol dissector plugin for ethereal
This weekend, I decided to teach Ethereal to decode the FE/BE protocol properly: until now, it could only extract likely-looking strings from the conversation, which I found woefully inadequate for debugging. I'm hoping the result will be useful to other people too: http://www.oryx.com/ams/packet-pgsql.c Copy it to epan/dissectors/ within an Ethereal source tree, and change the reference to packet-postgresql.c in Makefile.common to -pgsql.c, then configure and build. (Thanks to Kris Jurka for some testing. I've asked the Ethereal people if they want to distribute this with Ethereal.) Questions, comments, and suggestions are welcome. -- ams ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] buildfarm improvements
Andrew Dunstan schrieb: Reini Urban wrote: What I also miss is the successful output of the make test step. Something like the Log in Details, just behind an additional request. Config = Log Link to Details Without those details one doesn't trust the presented result. He might think that only the build was successful, and not the make test step also. People I redirect to this page from other projects, not reading the status pages everyday. That would actually be a substantial change in the way it works. Basically, it sends the log of the step that failed. That preserves bandwidth and doesn't clog the database with success cases. These logs are not inconsiderable - I just checked on the canonical system and for the last successful run they were 640Kb. I was originally given this (virtual) server on the basis of my assurance that the bandwidth and database requirements would be very modest, so I'm inclined to keep to that. Sure. Convinced. Regarding your last sentence - the intended prime users are the postgresql hackers. If it had a vastly more general audience I would have produced something a good less spartan in style. I'm not quite sure why you're redirecting people to the status pages from other projects. This is not the official list of supported platforms, and is not intended as a substitute for it. Perhaps we could put a statement at the top of the details page saying what steps have succeeded (which we could infer from the result). Of course, if people don't want to believe it then they won't - having logs should not make believing it any easier - faking the logs would be quite trivial. FYI here's what happens during a run - a status of OK means that *all* of this has run successfully: [EMAIL PROTECTED] buildfarm]$ ./run_build.pl --verbose checking out source ... checking if build run needed ... copying source to pgsql.3034 ... running configure ... running make ... running make check ... running make contrib ... running make install ... setting up db cluster ... starting db ... running make installcheck ... restarting db ... running make contrib install ... running make contrib installcheck ... stopping db ... OK Printing this output would be enough for me, and other manager types. All the buildfarm members are known, by the way, and every status report is signed with a SHA1 signature. We don't just accept anonymous reports. In many cases I know these people from previous electronic interaction, via email and/or IRC. That, more than the presence of success logs, should give you some confidence in the results, I hope. -- Reini Urban ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Help extending pg_class
On Sun, Dec 19, 2004 at 01:56:02AM -0800, overbored wrote: Hi all, I added a new variable-length field to the pg_class catalog, but I did something wrong, and I can't tell what else I'd need to change. (I know about how extending pg_class is bad and all, but it seems to be the simplest solution to my problem right now, and I'd just like to get it working.) The code I'm working with is based on Postgres 7.3.2 (avail. at http://telegraph.cs.berkeley.edu). I've added a char field to it before, but now I'm trying to add a text field, which pg_type also has. I modified pg_class.h and pg_attribute.h (which I've pasted to http://rafb.net/paste/results/c1CiIo27.html), You need to fix CLASS_TUPLE_SIZE. Not sure how far that would take you, because adding a varlena field to pg_class does not strike me as a trivial exercise. For a modification that's gonna stay academic, I would have opted for a different catalog. -- Alvaro Herrera ([EMAIL PROTECTED]) Ellos andaban todos desnudos como su madre los parió, y también las mujeres, aunque no vi más que una, harto moza, y todos los que yo vi eran todos mancebos, que ninguno vi de edad de más de XXX años (Cristóbal Colón) ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] Bitmapset data type???
Hi all,
I'm tring to understand a part of code of postgres and I saw a line like
this:
bms_is_subset(restrictinfo-right_relids,outerrelids) (createplan.c,
function get_switched_clauses() at ~/src/backend/optimizer/plan).
I don't understand the data types of the the arguments
of that function. The datatype of those arguments is Bitmapset,
i.e.
typedef struct Bitmapset
{
int nwords;
bitmapword words[1];
}Bitmapset; (bitmapset.h at ~/src/include/nodes)
and bitmapword is:
typedef unit32 bitmapword; (bitmapset.h at ~/src/include/nodes)
and the prototype of the function is:
extern bool bms_is_subset(const Bitmapset *a, const Bitmapset *b);
(bitmapset.h at ~/src/include/nodes)
The following is at the file bitmapset.h at dir ~/src/include/nodes)
A bitmap set can represent any set of nonnegative integers, although it
is mainly intented for sets where the maximum value is not large, say
at most a few hundred.By convention, a NULL pointer is always accepted
by all operations to represent the empty set.(But beware that this is
not the only representation of the empty set. Use bms_is_empty() in
preference to testing for NULL.)
Can someone explain to me please the datatype Bitmapset?
I'm confused.
Thanks in advance!!!
- martha
---(end of broadcast)---
TIP 8: explain analyze is your friend
[HACKERS] Stable functions problem
Hi all,
after several weeks away I downloaded today the 8.0rc1 and I
tested it with our application.
I'm having a bounce of errors because IMMUTABLE and STABLE
attributes for some of my functions. Let me explain with an example,
what I do is ( plpgsql )
my_id_user = sp_id_user( a_login );
IF my_id_user 0 THEN
RETURN -5; -- error code for existing user
END IF;
INSERT INTO users ( login ) VALUES ( a_login );
my_id_user = sp_id_user( a_login );
now here I can continue my function using the my_id_user, unfortunatelly
that sp_id_user is declared as IMMUTABLE this mean that at the second call
of sp_id_user my_id_user will not contain the user id.
This was working untill 7.4, is there a way to force sp_id_user to
be reevaluated ? That function is declared immutable because is ofthen
used in expresssion like this:
select * from user_data where id_user = sp_id_user('login');
I believe is a good idea write in the release notes, with a bigger font,
this
optimization about stable and immutable functions.
Regards
Gaetano Mendola
---(end of broadcast)---
TIP 8: explain analyze is your friend
Re: [HACKERS] Bitmapset data type???
On Sun, Dec 19, 2004 at 05:39:43PM +0200, Martha Chronopoulou wrote: Hi, A bitmap set can represent any set of nonnegative integers, although it is mainly intented for sets where the maximum value is not large, say at most a few hundred. Clearly this can only come from the contorted mind of a JPEG hacker. (And indeed it does, as you can see from the CVS logs.) I'm tring to understand a part of code of postgres and I saw a line like this: bms_is_subset(restrictinfo-right_relids,outerrelids) (createplan.c, function get_switched_clauses() at ~/src/backend/optimizer/plan). What that is doing is check whether restrictingo-right_relids is a subset of outerrelids. I don't understand the data types of the the arguments of that function. The only thing you need to know is that it represents a set of integers. Anything beyond that is going to drive you crazy (and is irrelevant anyway.) -- Alvaro Herrera ([EMAIL PROTECTED]) I can't go to a restaurant and order food because I keep looking at the fonts on the menu. Five minutes later I realize that it's also talking about food (Donald Knuth) ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Shared row locking
On Sun, Dec 19, 2004 at 09:52:01AM +, Simon Riggs wrote: Simon, In similar circumstances, DB2 uses these techniques: - when locktable X % full, then escalate locks to full table locks: both locktable memory and threshold% are instance parameters This is not useful at all, because the objective of this exercise is to downgrade locks, from exclusive row locking (SELECT ... FOR UPDATE) to shared row locking. Keep in mind that this is for foreign key locking, which is one area where deadlocks are frequently encountered because we use too strong a lock. - use a lock mode called Cursor Stability that locks only those rows currently being examined by a cursor, those maintaining the lock usage of a cursor at a constant level as the cursor moves. The lock mode of Repeatable Read *does* lock all rows read We can't release the locks until the end of the transaction. The second is a usable, practical alternative that should be considered and might avoid the need to write the spill-to-disk code and then discover it performs very badly. I don't think any of them serves the objective I'm trying to accomplish :-( Thanks for your ideas anyway. And keep having them! -- Alvaro Herrera ([EMAIL PROTECTED]) Uno puede defenderse de los ataques; contra los elogios se esta indefenso ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Stable functions problem
Gaetano Mendola [EMAIL PROTECTED] writes: I'm having a bounce of errors because IMMUTABLE and STABLE attributes for some of my functions. Let me explain with an example, Hmm. This particular example is a bug in exec_eval_simple_expr() ... if we're going to bypass SPI then we'd better do the things SPI does that are needed to maintain the correct execution environment, and as of 8.0 one of those things is to advance ActiveSnapshot. I've applied a patch for this. (Memo to self: I'm beginning to wonder if exec_eval_simple_expr is worth the trouble at all, compared to just using SPI. The amount of overhead it saves seems to get less with each new release.) now here I can continue my function using the my_id_user, unfortunatelly that sp_id_user is declared as IMMUTABLE this mean that at the second call of sp_id_user my_id_user will not contain the user id. That actually doesn't have anything to do with it --- the same failure would have occurred if you'd (correctly) declared sp_id_user as STABLE. So it's a good bug report. But I trust you do realize you are playing with fire. While I have been heard to suggest mislabeling functions as immutable if they're only going to be used in interactive queries, I don't think I have ever failed to mention that you *will* get burnt if you call such functions from other functions. When this coding someday does break for you, I won't have any sympathy at all... regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Bitmapset data type???
Alvaro Herrera [EMAIL PROTECTED] writes: On Sun, Dec 19, 2004 at 05:39:43PM +0200, Martha Chronopoulou wrote: I'm tring to understand a part of code of postgres and I saw a line like this: bms_is_subset(restrictinfo-right_relids,outerrelids) (createplan.c, function get_switched_clauses() at ~/src/backend/optimizer/plan). What that is doing is check whether restrictingo-right_relids is a subset of outerrelids. In particular, it's looking for the case where we are using a WHERE clause like a.f1 = b.f2 as a merge or hash join clause, but we decided that b ought to be the outer (left-side) relation in the join. The executor expects the outer relation's variable to be on the left in the constructed plan's join clause, so we've got to commute the clause in such cases. The reason it's a subset test is that in three-or-more-relation queries, we might be doing this in an upper join level, where the selected joining sequence is (b join c) join a. So the relations mentioned on the right side of the clause might be just a subset of those we have so far collected into a single joined relation. In recent releases (perhaps only 7.4 and later, not sure offhand) we can handle expressions as arguments of merge/hash join clauses, so the expression might even be like a.f1 = (b.f2 + c.f3) This is why all the relation-membership fields have to be sets and not just single relations. We know by construction that there is no overlap between the sets of relations being joined (ie we'd never do b join b). We also know there's no overlap between the relation membership of the sets of variables on the two sides of the clause (else it wouldn't have been selected as a usable merge or hash join clause). So this subset test is sufficient to distinguish the two cases we have to distinguish. There are several other ways it could have been written, of course; eg we could have tested left_relids for not being a subset of outerrelids, or we could have passed in innerrelids and made the tests against that. If what's bugging you is just the physical representation: when we have to reason about sets of relations in the planner, we represent them as sets of integers where the individual integers are rangetable indexes for the relations. Bitmapset is just an abstract data type that happens to provide operations that are convenient for the reasoning we need to do. (Of course that's not by chance ;-) ... but the internal representation of Bitmapset is not at all the planner's concern. The planner just sees it as a set of integers.) regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Shared row locking
Alvaro Herrera [EMAIL PROTECTED] writes: On Sun, Dec 19, 2004 at 09:52:01AM +, Simon Riggs wrote: - use a lock mode called Cursor Stability that locks only those rows currently being examined by a cursor, those maintaining the lock usage of a cursor at a constant level as the cursor moves. The lock mode of Repeatable Read *does* lock all rows read We can't release the locks until the end of the transaction. Well, what Simon is essentially proposing is that we work around a potential performance issue by exposing a less-clean semantic model to users. I'd prefer to adopt such a thing as a last resort, not a first one. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Shared row locking
On Sun, 19 Dec 2004, Alvaro Herrera wrote: On Sun, Dec 19, 2004 at 09:52:01AM +, Simon Riggs wrote: Simon, In similar circumstances, DB2 uses these techniques: - when locktable X % full, then escalate locks to full table locks: both locktable memory and threshold% are instance parameters This is not useful at all, because the objective of this exercise is to downgrade locks, from exclusive row locking (SELECT ... FOR UPDATE) to shared row locking. Keep in mind that this is for foreign key locking, which is one area where deadlocks are frequently encountered because we use too strong a lock. Actually it might help in some scenarios. Remember, we're not talking about upgrading shared locks to exclusive locks. We're only talking about locking more rows than necessary (all rows). I believe DB2 does the escalation also for perfomance. Getting a full table lock is cheaper than individually locking every row. - Heikki ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Shared row locking
Heikki Linnakangas [EMAIL PROTECTED] writes: On Sun, 19 Dec 2004, Alvaro Herrera wrote: This is not useful at all, because the objective of this exercise is to downgrade locks, from exclusive row locking (SELECT ... FOR UPDATE) to shared row locking. Actually it might help in some scenarios. Remember, we're not talking about upgrading shared locks to exclusive locks. We're only talking about locking more rows than necessary (all rows). Nonetheless, it would mean that locks would be taken depending on implementation-dependent, not-visible-to-the-user considerations. Shared locks can still cause deadlocks, and so you would have an unreliable application, which would only be unreliable under load. As I said in connection with the other proposal, weird user-visible semantics should be the last resort not the first. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Shared row locking
On Sun, 19 Dec 2004, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: On Sun, 19 Dec 2004, Alvaro Herrera wrote: This is not useful at all, because the objective of this exercise is to downgrade locks, from exclusive row locking (SELECT ... FOR UPDATE) to shared row locking. Actually it might help in some scenarios. Remember, we're not talking about upgrading shared locks to exclusive locks. We're only talking about locking more rows than necessary (all rows). Nonetheless, it would mean that locks would be taken depending on implementation-dependent, not-visible-to-the-user considerations. Shared locks can still cause deadlocks, and so you would have an unreliable application, which would only be unreliable under load. As I said in connection with the other proposal, weird user-visible semantics should be the last resort not the first. I agree that lock escalation is not a good solution, we run into problems with DB2 lock escalation at work all the time. - Heikki ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Kerberos includes (was Re: [HACKERS] Port report: Fedora Core 3 x86_64)
I wrote: [ concerning a discussion about Kerberos' com_err.h being in /usr/include/et/ on some systems ] Actually, I'm wondering why we directly include com_err.h at all. At least in the version of krb5.h I have here, that file is included by krb5.h; so both backend/libpq/auth.c and interfaces/libpq/fe-auth.c compile just fine with #include com_err.h diked out. After some digging in dusty old tarballs, I have learned that Kerberos 5 releases 1.0.* did indeed require a separate #include of com_err.h, but in releases 1.1 and later krb5.h itself includes com_err.h and so there's no need for a separate #include. Kerberos 5 1.0.* includes serious known, never-patched vulnerabilities. I can't believe that anyone is going to build PG 8.0 with krb5 1.0, or that we need to be complicit in their trying to do so. Accordingly, I think we should just avoid the whole problem of exactly where com_err.h lives by removing the #includes for it as well as the configure test for it. regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Shared row locking
On Sun, 2004-12-19 at 18:31, Alvaro Herrera wrote: Thanks for your ideas anyway. And keep having them! No problem. Just giving some info on what works and doesn't work in other implementations. -- Best Regards, Simon Riggs ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: Kerberos includes (was Re: [HACKERS] Port report: Fedora Core 3 x86_64)
Tom Lane said: I wrote: [ concerning a discussion about Kerberos' com_err.h being in /usr/include/et/ on some systems ] Actually, I'm wondering why we directly include com_err.h at all. At least in the version of krb5.h I have here, that file is included by krb5.h; so both backend/libpq/auth.c and interfaces/libpq/fe-auth.c compile just fine with #include com_err.h diked out. After some digging in dusty old tarballs, I have learned that Kerberos 5 releases 1.0.* did indeed require a separate #include of com_err.h, but in releases 1.1 and later krb5.h itself includes com_err.h and so there's no need for a separate #include. Kerberos 5 1.0.* includes serious known, never-patched vulnerabilities. I can't believe that anyone is going to build PG 8.0 with krb5 1.0, or that we need to be complicit in their trying to do so. Accordingly, I think we should just avoid the whole problem of exactly where com_err.h lives by removing the #includes for it as well as the configure test for it. Works for me. I'm not sure why the reasoning only applies to 8.0 - is it a case of the 'only fix serious bugs in stable releases' rule? cheers andrew ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: Kerberos includes (was Re: [HACKERS] Port report: Fedora Core 3 x86_64)
Andrew Dunstan [EMAIL PROTECTED] writes: Tom Lane said: Accordingly, I think we should just avoid the whole problem of exactly where com_err.h lives by removing the #includes for it as well as the configure test for it. Works for me. I'm not sure why the reasoning only applies to 8.0 - is it a case of the 'only fix serious bugs in stable releases' rule? Pretty much. All this is really saving is the need to specify a --with-includes option for Kerberos builds on some platforms, so I'd classify it as a minor improvement and not a bug fix. regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] bgwriter changes
Mark Kirkwood wrote: It occurs to me that cranking up the number of transactions (say 1000-10) and seeing if said regression persists would be interesting. This would give the smoothing effect of the bgwriter (plus the ARC) a better chance to shine. I ran a few of these over the weekend - since it rained here :-) , and the results are quite interesting: [2xPIII, 2G, 2xATA RAID 0, FreeBSD 5.3 with the same non default Pg parameters as before] clients = 4 transactions = 10 (/client), each test run twice Version tps 7.4.6 49 8.0.0.0RC1 50 8.0.0.0RC1 + rem49 8.0.0.0RC1 + bg250 Needless to way, all well within measurement error of each other (the variability was about 1). I suspect that my previous tests had too few transactions to trigger many (or any) checkpoints. With them now occurring in the test, they look to be the most significant factor (contrast with 70-80 tps for 4 clients with 1000 transactions). Also with a small number of transactions, the fsyn'ed blocks may have all fitted in the ATA disk caches (2x2M). In hindsight I should have disabled this! (might run the smaller no. transactions again with hw.ata.wc=0 and see if this is enlightening) regards Mark ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Shared row locking
Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: On Sun, 19 Dec 2004, Alvaro Herrera wrote: This is not useful at all, because the objective of this exercise is to downgrade locks, from exclusive row locking (SELECT ... FOR UPDATE) to shared row locking. Actually it might help in some scenarios. Remember, we're not talking about upgrading shared locks to exclusive locks. We're only talking about locking more rows than necessary (all rows). Nonetheless, it would mean that locks would be taken depending on implementation-dependent, not-visible-to-the-user considerations. Shared locks can still cause deadlocks, and so you would have an unreliable application, which would only be unreliable under load. As I said in connection with the other proposal, weird user-visible semantics should be the last resort not the first. One idea is to stamp the same shared lock counter on multiple rows and just prevent another application from also sharing that lock if too many rows are locked. It would wait for the lock to be released. This prevents the problem of having to store thousands of shared lock counters. There would have to be some flag so say which shared counters are only for a single backend. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Shared row locking
On Sun, Dec 19, 2004 at 11:35:02PM +0200, Heikki Linnakangas wrote: On Sun, 19 Dec 2004, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: On Sun, 19 Dec 2004, Alvaro Herrera wrote: This is not useful at all, because the objective of this exercise is to downgrade locks, from exclusive row locking (SELECT ... FOR UPDATE) to shared row locking. Actually it might help in some scenarios. Remember, we're not talking about upgrading shared locks to exclusive locks. We're only talking about locking more rows than necessary (all rows). Nonetheless, it would mean that locks would be taken depending on implementation-dependent, not-visible-to-the-user considerations. Shared locks can still cause deadlocks, and so you would have an unreliable application, which would only be unreliable under load. As I said in connection with the other proposal, weird user-visible semantics should be the last resort not the first. I agree that lock escalation is not a good solution, we run into problems with DB2 lock escalation at work all the time. Does anyone know how Oracle deals with this? They use MVCC like PostgreSQL, so they'd be a better source for inspiration. -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 Windows: Where do you want to go today? Linux: Where do you want to go tomorrow? FreeBSD: Are you guys coming, or what? ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] yyin's value of postgresql parser
hello what is the value of yyin variable for postgresql parser. It might be the default(stdout) when postgresql is in interactive backend mode...but what happens when clients from different workstations sends their queries? __ Do you Yahoo!? The all-new My Yahoo! - What will yours do? http://my.yahoo.com ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Shared row locking
On Sat, 18 Dec 2004, Bruce Momjian wrote: BTom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: You mean all empty/zero rows can be removed? Can we guarantee that on commit we can clean up the bitmap? If not the idea doesn't work. For whatever data structure we use, we may reset the structure to empty during backend-crash recovery. So your objection boils down to what if a backend exits normally but forgets to clean up its locks? Assuming that doesn't happen isn't any worse than assuming a backend will clean up its shared memory state on non-crash exit, so I don't think it's a serious concern. That brings another thought: really what this is all about is working around the fact that the standard lock manager can only cope with a finite number of coexisting locks, because it's working in a fixed-size shared memory arena. Maybe we should instead think about ways to allow the existing lock table to spill to disk when it gets too big. That would eliminate max_locks_per_transaction as a source of hard failures, which would be a nice benefit. Agreed. Once concern I have about allowing the lock table to spill to disk is that a large number of FOR UPDATE locks could push out lock entries used by other backends, causing very poor performance. I think if we allow the lock manager to spill to disk (and I think we do need to allow it) then we should also be able to control the amount of shared memory allocated. There's little point spilling the lock table to disk if we have huge amounts of memory. Gavin ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
