RE: [HACKERS] CORBA and PG
Quoting Franck Martin [EMAIL PROTECTED]: I guess these stubs are for accessing PG as a corba server... I'm trying to look to see if I can store CORBA objects inside PG, any ideas... Although I've not tried it (yet) it should be possible to access Java EJB's from corba. If so, then using an EJB server (JBoss www.jboss.org) you could then store them as Entity beans. Each one would then have its own table in the database. Peter Franck Martin Network and Database Development Officer SOPAC South Pacific Applied Geoscience Commission Fiji E-mail: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] Web site: http://www.sopac.org/ http://www.sopac.org/ Support FMaps: http://fmaps.sourceforge.net/ http://fmaps.sourceforge.net/ This e-mail is intended for its addresses only. Do not forward this e-mail without approval. The views expressed in this e-mail may not be necessarily the views of SOPAC. -Original Message- From: Peter T Mount [mailto:[EMAIL PROTECTED]] Sent: Tuesday, 6 March 2001 3:52 To: Franck Martin Cc: PostgreSQL List Subject: Re: [HACKERS] CORBA and PG Quoting Franck Martin [EMAIL PROTECTED]: Does anyone has pointers on CORBA and PostgreSQL? What is the story ? There's some old stubs for one of the orbs somewhere in the source (C/C++) Also the old JDBC/Corba example is still there (src/interfaces/jdbc/example/corba) Peter -- Peter Mount [EMAIL PROTECTED] PostgreSQL JDBC Driver: http://www.retep.org.uk/postgres/ RetepPDF PDF library for Java: http://www.retep.org.uk/pdf/ -- Peter Mount [EMAIL PROTECTED] PostgreSQL JDBC Driver: http://www.retep.org.uk/postgres/ RetepPDF PDF library for Java: http://www.retep.org.uk/pdf/ ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: AW: [HACKERS] WAL-based allocation of XIDs is insecure
Zeugswetter Andreas SB wrote: 1. A new transaction inserts a tuple. The tuple is entered into its heap file with the new transaction's XID, and an associated WAL log entry is made. Neither one of these are on disk yet --- the heap tuple is in a shmem disk buffer, and the WAL entry is in the shmem WAL buffer. 2. Now do a lot of read-only operations, in the same or another backend. The WAL log stays where it is, but eventually the shmem disk buffer will get flushed to disk so that the buffer can be re-used for some other disk page. 3. Assume we now crash. Now, we have a heap tuple on disk with an XID that does not correspond to any XID visible in the on-disk WAL log. 4. Upon restart, WAL will initialize the XID counter to the first XID not seen in the WAL log. Guess which one that is. 5. We will now run a new transaction with the same XID that was in use before the crash. If that transaction commits, then we have a tuple on disk that will be considered valid --- and should not be. I do not think this is true. Before any modification to a page the original page will be written to the log (aka physical log). Yes there must be XLogFlush() before writing buffers. BTW how do we get the next XID if WAL files are corrupted ? Regards, Hiroshi Inoue ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
[HACKERS] Banner links not working (fwd)
This just came to the webmaster mailbox: --- Most of the top banner links on http://jdbc.postgresql.org (like Documentation, Tutorials, Resources, Development) throw up 404s if followed. Thought you ought to know. Still trying to find the correct driverClass/connectString for the Postgres JDBC driver... --- Who maintains this site? It's certainly not me. From looking at the page I'm guessing Peter Mount, can we get some kind of prominent contact info on it? I've had a few emails on it so far. Vince. -- == Vince Vielhaber -- KA8CSHemail: [EMAIL PROTECTED]http://www.pop4.net 56K Nationwide Dialup from $16.00/mo at Pop4 Networking Online Campground Directoryhttp://www.camping-usa.com Online Giftshop Superstorehttp://www.cloudninegifts.com == ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
AW: AW: [HACKERS] WAL-based allocation of XIDs is insecure
5. We will now run a new transaction with the same XID that was in use before the crash. If that transaction commits, then we have a tuple on disk that will be considered valid --- and should not be. I do not think this is true. Before any modification to a page the original page will be written to the log (aka physical log). Yes there must be XLogFlush() before writing buffers. BTW how do we get the next XID if WAL files are corrupted ? Normally: 1. pg_control checkpoint info 2. checkpoint record in WAL ? 3. then rollforward of WAL If WAL is corrupt the only way to get a consistent state is to bring the db into a state as it was during last good checkpoint. But this is only possible if you can at least read all "physical log" records from WAL. Failing that, the only way would probably be to scan all heap files for XID's that are greater than the XID from checkpoint. I think the utility Tom has in mind, that resets WAL, will allow you to dump the db so you can initdb and reload. I don't think it is intended that you can immediately resume operation, (unless of course for the mentioned case of an upgrade with a good checkpoint as last WAL record (== proper shutdown)). Andreas ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
I especially don't think that we should second-guess what the admin wants us to do by auto-killing backends that are still serving clients. Sure. But it would be nice anyway if pg_ctl could do this with a specific command line switch. -- Tout n'y est pas parfait, mais on y honore certainement les jardiniers Dominique Quatravaux [EMAIL PROTECTED] ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
RE: [HACKERS] Proposed WAL changes
-Original Message- From: Tom Lane I have just sent to the pgsql-patches list a rather large set of proposed diffs for the WAL code. These changes: * Store two past checkpoint locations, not just one, in pg_control. On startup, we fall back to the older checkpoint if the newer one is unreadable. Also, a physical copy of the newest checkpoint record is kept in pg_control for possible use in disaster recovery (ie, complete loss of pg_xlog). Also add a version number for pg_control itself. Remove archdir from pg_control; it ought to be a GUC parameter, not a special case (not that it's implemented yet anyway). Is archdir really a GUC parameter ? Regards, Hiroshi Inoue ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
[HACKERS] Re: CORBA and PG
I'm trying to look to see if I can store CORBA objects inside PG, any ideas... CORBA has several mechanisms for finding CORBA objects, including the naming service and the implementation repository. The naming service provides a directory for objects, returning IORs to allow a client to contact a server. A database could be used to provide a persistant store for this information. One could use a database to store rules for an implementation repository, as well as IOR info. A CORBA object itself is an executable. So it could be stored as a binary object, but I'm not sure what the benefits of storage in a database would be. Some time ago I saw an article on using PostgreSQL to implment a versioned file system, which might have some aspects similar to what you are asking about. Do you have a use case to help us out? - Thomas ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] There is error at the examples in PL/pgSQL
Hello,all
I use your a example of PL/pgSQL, but I found some errors when I execute
these codes. The details are followings,
First, I create a exam.sql that includes these codes as followings,
CREATE TABLE emp (
empname text,
salary int4,
last_date datetime,
last_user name);
CREATE FUNCTION emp_stamp () RETURNS OPAQUE AS'
BEGIN
-- Check that empname and salary are given
IF NEW.empname ISNULL THEN
RAISE EXCEPTION ''empname cannot be NULL value'';
END IF;
IF NEW.salary ISNULL THEN
RAISE EXCEPTION ''% cannot have NULL salary'', NEW.empname;
END IF;
-- Who works for us when she must pay for?
IF NEW.salary 0 THEN
RAISE EXCEPTION ''% cannot have a negative salary'', NEW.empname;
END IF;
-- Remember who changed the payroll when
NEW.last_date := ''now'';
NEW.last_user := getpgusername();
RETURN NEW;
END;
' LANGUAGE 'plpgsql';
CREATE TRIGGER emp_stamp BEFORE INSERT OR UPDATE ON emp
FOR EACH ROW EXECUTE PROCEDURE emp_stamp();
Secondly, I execute exam.sql and the postgress can create the table emp,
the function emp_stamp() and the trigger emp_stamp seccessfully.But when I
insert one record to table emp, there are some errors on the screen.
the insert statement is followings,
INSERT INTO emp Values('','','20001220','raymond');
the error of screen is:
NOTICE: plpgsql: ERROR during compile of emp_stamp near line 1
"RROR: parse error at or near "
Why? and what wrong is it? Please give me reply as possible as you can.
Thanks!
_
Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com.
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: AW: [HACKERS] WAL-based allocation of XIDs is insecure
Zeugswetter Andreas SB [EMAIL PROTECTED] writes: 5. We will now run a new transaction with the same XID that was in use before the crash. If that transaction commits, then we have a tuple on disk that will be considered valid --- and should not be. I do not think this is true. Before any modification to a page the original page will be written to the log (aka physical log). Hmm. Actually, what is written to the log is the *modified* page not its original contents. However, on studying the buffer manager I see that it tries to fsync the log entry describing the last mod to a data page before it writes out the page itself. So perhaps that can be relied on to ensure all XIDs known in the heap are known in the log. However, I'd just as soon have the NEXTXID log records too to be doubly sure. I do now agree that we needn't fsync the NEXTXID records, however. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] Query Planning time increased 3 times on 7.1 compared to 7.0.3
Hello,
We noticed that after upgrading to 7.1beta[245] the execution time for
some often used queries went up by a factor of 2 or more. Considering
the early beta state I was not alarmed. But since I noticed that
yesterday's snapshot still has the problem, I'd really like to tell you
about it.
Here is one of the queries, it takes about half a second on our computer
(PII 233 with 256MB RAM) to execute and returns typically 1-4 rows via
two index scans with high selectivity. So it looks to me that planning
time outwages execution time by far. 7.0 took about 0.15 seconds (which
is still much).
Here is the query:
explain verbose select gaenge , s . artikelid , text from
schaertabelle s , extartbez e where maschine = int2(109) and
schaerdatum = '2001-01-13' and s . artikelid = e . artikelid and
extartbezid = 1 and bezkomptype = 0 order by textlimit 10;
And the plan for 7.0 and 7.1 (attached).
The data and schema is accessible via
http://home.wtal.de/petig/pg_test.sql.gz
If you omit 'int2(' the index scan collapses into a sequential scan.
(Well known problem with int2 indices)
Christof
Oh, I'll attach the schema, too. So if you just want to take a look at
the table definition you don't have to download the data.
NOTICE: QUERY DUMP:
{ LIMIT :startup_cost 11.70 :total_cost 11.70 :rows 1 :width 22 :qptargetlist ({
TARGETENTRY :resdom { RESDOM :resno 1 :restype 21 :restypmod -1 :resname gaenge
:reskey 0 :reskeyop 0 :ressortgroupref 0 :resjunk false } :expr { VAR :varno 1
:varattno 4 :vartype 21 :vartypmod -1 :varlevelsup 0 :varnoold 1 :varoattno 4}} {
TARGETENTRY :resdom { RESDOM :resno 2 :restype 23 :restypmod -1 :resname artikelid
:reskey 0 :reskeyop 0 :ressortgroupref 0 :resjunk false } :expr { VAR :varno 1
:varattno 8 :vartype 23 :vartypmod -1 :varlevelsup 0 :varnoold 1 :varoattno 8}} {
TARGETENTRY :resdom { RESDOM :resno 3 :restype 1043 :restypmod 54 :resname text
:reskey 0 :reskeyop 0 :ressortgroupref 1 :resjunk false } :expr { VAR :varno 2
:varattno 3 :vartype 1043 :vartypmod 54 :varlevelsup 0 :varnoold 2 :varoattno 3}})
:qpqual :lefttree { SORT :startup_cost 11.70 :total_cost 11.70 :rows 1 :width 22
:qptargetlist ({ TARGETENTRY :resdom { RESDOM :resno 1 :restype 21 :restypmod -1
:resname gaenge :reskey 0 :reskeyop 0 :ressortgroupref 0 :resjunk false } :expr { VAR
:varno 1 :varattno 4 :vartype 21 :vartypmod -1 :varlevelsup 0 :varnoold 1 :varoattno
4}} { TARGETENTRY :resdom { RESDOM :resno 2 :restype 23 :restypmod -1 :resname
artikelid :reskey 0 :reskeyop 0 :ressortgroupref 0 :resjunk false } :expr { VAR :varno
1 :varattno 8 :vartype 23 :vartypmod -1 :varlevelsup 0 :varnoold 1 :varoattno 8}} {
TARGETENTRY :resdom { RESDOM :resno 3 :restype 1043 :restypmod 54 :resname text
:reskey 1 :reskeyop 1071 :ressortgroupref 1 :resjunk false } :expr { VAR :varno 2
:varattno 3 :vartype 1043 :vartypmod 54 :varlevelsup 0 :varnoold 2 :varoattno 3}})
:qpqual :lefttree { NESTLOOP :startup_cost 0.00 :total_cost 11.69 :rows 1 :width 22
:qptargetlist ({ TARGETENTRY :resdom { RESDOM :resno 1 :restype 21 :restypmod -1
:resname gaenge :reskey 0 :reskeyop 0 :ressortgroupref 0 :resjunk false } :expr { VAR
:varno 65001 :varattno 1 :vartype 21 :vartypmod -1 :varlevelsup 0 :varnoold 1
:varoattno 4}} { TARGETENTRY :resdom { RESDOM :resno 2 :restype 23 :restypmod -1
:resname artikelid :reskey 0 :reskeyop 0 :ressortgroupref 0 :resjunk false } :expr {
VAR :varno 65001 :varattno 2 :vartype 23 :vartypmod -1 :varlevelsup 0 :varnoold 1
:varoattno 8}} { TARGETENTRY :resdom { RESDOM :resno 3 :restype 1043 :restypmod 54
:resname text :reskey 0 :reskeyop 0 :ressortgroupref 1 :resjunk false } :expr { VAR
:varno 65000 :varattno 1 :vartype 1043 :vartypmod 54 :varlevelsup 0 :varnoold 2
:varoattno 3}}) :qpqual :lefttree { INDEXSCAN :startup_cost 0.00 :total_cost 2.02
:rows 1 :width 6 :qptargetlist ({ TARGETENTRY :resdom { RESDOM :resno 1 :restype 21
:restypmod -1 :resname :reskey 0 :reskeyop 0 :ressortgroupref 0 :resjunk false }
:expr { VAR :varno 1 :varattno 4 :vartype 21 :vartypmod -1 :varlevelsup 0 :varnoold 1
:varoattno 4}} { TARGETENTRY :resdom { RESDOM :resno 2 :restype 23 :restypmod -1
:resname :reskey 0 :reskeyop 0 :ressortgroupref 0 :resjunk false } :expr { VAR
:varno 1 :varattno 8 :vartype 23 :vartypmod -1 :varlevelsup 0 :varnoold 1 :varoattno
8}}) :qpqual :lefttree :righttree :extprm () :locprm () :initplan :nprm 0
:scanrelid 1 :indxid ( 2424224) :indxqual (({ EXPR :typeOid 16 :opType op :oper {
OPER :opno 94 :opid 63 :opresulttype 16 } :args ({ VAR :varno 1 :varattno 1 :vartype
21 :vartypmod -1 :varlevelsup 0 :varnoold 1 :varoattno 2} { CONST :consttype 21
:constlen 2 :constbyval true :constisnull false :constvalue 2 [ 109 0 0 0 ] })} {
EXPR :typeOid 16 :opType op :oper { OPER :opno 1093 :opid 1086 :opresulttype 16 }
:args ({ VAR :varno 1 :varattno 2 :vartype 1082 :vartypmod -1 :varlevelsup 0
:varnoold 1 :varoattno 3} { CONST :consttype
Re: AW: [HACKERS] WAL-based allocation of XIDs is insecure
Hiroshi Inoue [EMAIL PROTECTED] writes: Yes there must be XLogFlush() before writing buffers. BTW how do we get the next XID if WAL files are corrupted ? My not-yet-committed changes include storing the latest CheckPoint record in pg_control (as well as in the WAL files). Recovery from XLOG disaster will consist of generating a new XLOG that's empty except for a CheckPoint record based on the one cached in pg_control. In particular we can extract the nextOid and nextXid fields. It might be that writing NEXTXID or NEXTOID log records should update pg_control too with new nextXid/nextOid values --- what do you think? Otherwise there's a possibility that the stored checkpoint is too far back to cover all the values used since then. OTOH, we are not going to be able to guarantee absolute consistency in this disaster recovery scenario anyway; duplicate XIDs may be the least of one's worries. Of course, if you lose both XLOG and pg_control, you're still in big trouble. So it seems we should minimize the number of writes to pg_control, which is an argument not to update it more than we must. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] There is error at the examples in PL/pgSQL
"Lu Raymond" [EMAIL PROTECTED] writes: NOTICE: plpgsql: ERROR during compile of emp_stamp near line 1 "RROR: parse error at or near " Save your script with Unix-style newlines, not DOS-style (LF not CR/LF). regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] Re: pg_dump writes SEQUENCEs twice with -a
At 16:07 6/03/01 +0100, kovacsz wrote: The problem hasn't disappeared yet. In 7.1beta4... As per an earlier message today, the problem is fixed in CVS Philip Warner| __---_ Albatross Consulting Pty. Ltd. |/ - \ (A.B.N. 75 008 659 498) | /(@) __---_ Tel: (+61) 0500 83 82 81 | _ \ Fax: (+61) 0500 83 82 82 | ___ | Http://www.rhyme.com.au |/ \| |---- PGP key available upon request, | / and from pgp5.ai.mit.edu:11371 |/ ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: AW: AW: [HACKERS] WAL-based allocation of XIDs is insecure
Zeugswetter Andreas SB [EMAIL PROTECTED] writes: Hmm. Actually, what is written to the log is the *modified* page not its original contents. I thus really doubt above statement. Read the code. Each page about to be modified should be written to the txlog once, and only once before the first modification after each checkpoint. Yes, there's only one page dump per page per checkpoint. But the sequence is (1) make the modification in shmem buffers then (2) make the XLOG entry. I believe this is OK since the XLOG entry is flushed before any of the pages it affects are written out from shmem. Since we have not changed the storage management policy, it's OK if heap pages contain changes from uncommitted transactions --- all we must avoid is inconsistencies (eg not all three pages of a btree split written out), and redo of the XLOG entry will ensure that for us. However, I'd just as soon have the NEXTXID log records too to be doubly sure. I do now agree that we needn't fsync the NEXTXID records, however. I do not really see an additional benefit. If the WAL is busted those records are likely busted too. The point is to make the allocation of XIDs and OIDs work the same way. In particular, if we are forced to reset the XLOG using what's stored in pg_control, it would be good if what's stored in pg_control is a value beyond the last-used XID/OID, not a value less than the last-used ones. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
AW: AW: AW: [HACKERS] WAL-based allocation of XIDs is insecure
Hmm. Actually, what is written to the log is the *modified* page not its original contents. Well, that sure is not what was discussed on the list for implementation !! I thus really doubt above statement. Read the code. Ok, sad. Each page about to be modified should be written to the txlog once, and only once before the first modification after each checkpoint. Yes, there's only one page dump per page per checkpoint. But the sequence is (1) make the modification in shmem buffers then (2) make the XLOG entry. I believe this is OK since the XLOG entry is flushed before any of the pages it affects are written out from shmem. Since we have not changed the storage management policy, it's OK if heap pages contain changes from uncommitted transactions Sure, but the other way would be a lot less complex. --- all we must avoid is inconsistencies (eg not all three pages of a btree split written out), and redo of the XLOG entry will ensure that for us. Is it so hard to swap ? First write page to log then modify in shmem. Then those pages would have additional value, because then utilities could do all sorts of things with those pages. 1. Create a consistent state of the db by only applying "physical log" pages after checkpoint (in case a complete WAL rollforward bails out) 2. Create a consistent online backup snapshot, by first doing something like an ordinary tar, and after that save all "physical log" pages. Andreas ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
Lamar Owen writes: I missed something somehwere: wasn't the consensus a few weeks ago that pg_ctl shouldn't be used for a system initscript? The consensus(?) was that there was some work to do in pg_ctl before it was robust enough to be used (for anything). That work has been done. An example Linux init.d script is at contrib/start-scripts/linux. The only fault in that script that I can see is that it has no recipe for the case when the postmaster does not come down after 60 seconds. But this is really no problem for the issue at hand because if you do a normal runlevel switch then the postmaster will simply keep running, and during a system shutdown all the backends are going to die anyway. -- Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/ ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Query Planning time increased 3 times on 7.1 compared to 7.0.3
Justin Clift wrote: Hi Christof, I'm not aware of the problem with int2 indexes collapsing. Can you give me some more info, and I'll put it on the techdocs.postgresql.org website. Oh, I'm sorry for my strange wording. I said that the index search collapses to a sequential scan if you do not cast the number to int2. Because an int2 index is not used to look up an int4. And untyped numbers are int4 or numeric the int2 index is never used unless explicitely specified (by a type cast). Yes this is a known bug in PostgreSQL 7.1 and below. Hopefully this will get addressed in 7.2? Why don't I code it? I'm busy working on ecpg (dyn. SQL) at the moment. Christof ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: AW: AW: AW: [HACKERS] WAL-based allocation of XIDs is insecure
Zeugswetter Andreas SB [EMAIL PROTECTED] writes: Is it so hard to swap ? First write page to log then modify in shmem. Then those pages would have additional value, because then utilities could do all sorts of things with those pages. After thinking about this a little, I believe I see why Vadim did it the way he did. Suppose we tried to make the code sequence be obtain write lock on buffer; XLogOriginalPage(buffer); // copy page to xlog if first since ckpt modify buffer; XLogInsert(xlog entry for modification); mark buffer dirty and release write lock; so that the saving of the original page is a separate xlog entry from the modification data. Looks easy, and it'd sure simplify XLogInsert a lot. The only problem is it's wrong. What if a checkpoint occurs between the two XLOG records? The decision whether to log the whole buffer has to be atomic with the actual entry of the xlog record. Unless we want to hold the xlog insert lock for the entire time that we're (eg) splitting a btree page, that means we log the buffer after the modification work is done, not before. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
Bruce Momjian writes: This will try a pg_ctl shutdown for 60 seconds, then kill pg_ctl. You would then need a kill of you own. pg_ctl automatically times out after 60 seconds. Oh, yea, that's right, I saw that in the documenation. Forget my script. Just run pg_ctl first, then kill the postmaster if it is still there. Much safer than doing kill and checking because pg_ctl knows when the system cleanly shuts down and exits. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] mailing list messages
Bruce Momjian writes: I wonder if the new Tips at the bottom of email messages can be enabled for users during their first 30 days of mailing list subscription, then not appear? I once again refer people to RFC 2369 http://www.faqs.org/rfcs/rfc2369.html about how to embed email list management information into messages. Secondly, I would also tolerate the "monthly reminders" that some list managers send out (e.g., GNU, Great Bridge). At the very least, though, the tips should be preceded by a 'dashdashspacenewline' sequence so that some mail readers can strip them off in replies. I filter out the tips anyway, so I really don't care a lot. # for procmail users :0 Bbf | sed -n -e '/^---(end of |broadcast)---$/q' -e 'p' -- Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/ ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
AW: AW: [HACKERS] WAL-based allocation of XIDs is insecure
5. We will now run a new transaction with the same XID that was in use before the crash. If that transaction commits, then we have a tuple on disk that will be considered valid --- and should not be. I do not think this is true. Before any modification to a page the original page will be written to the log (aka physical log). Hmm. Actually, what is written to the log is the *modified* page not its original contents. Well, that sure is not what was discussed on the list for implementation !! The physical log page should be the page as it was during the last checkpoint. Anything else would also not have the benefit of fixing the index page problem this solution was intended to fix in the first place. I thus really doubt above statement. However, on studying the buffer manager I see that it tries to fsync the log entry describing the last mod to a data page before it writes out the page itself. So perhaps that can be relied on to ensure all XIDs known in the heap are known in the log. Each page about to be modified should be written to the txlog once, and only once before the first modification after each checkpoint. During rollforward the pages are written back to the heap, thus no open XIDs can be in heap pages. However, I'd just as soon have the NEXTXID log records too to be doubly sure. I do now agree that we needn't fsync the NEXTXID records, however. I do not really see an additional benefit. If the WAL is busted those records are likely busted too. Andreas ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] mailing list messages
Tom Lane [EMAIL PROTECTED] writes: Bruce Momjian [EMAIL PROTECTED] writes: I wonder if the new Tips at the bottom of email messages can be enabled for users during their first 30 days of mailing list subscription, then not appear? I'm pretty durn tired of 'em, and it's not been 30 days yet ;-) I think the tips would be greatly enhanced if there was a 25% chance that they included the output of the fortune program. Ian ---(end of broadcast)--- TIP 57: I have discovered the art of deceiving diplomats. I tell them the truth and they never believe me. -- Camillo Di Cavour ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: AW: AW: AW: [HACKERS] WAL-based allocation of XIDs is insecure
I wrote: The decision whether to log the whole buffer has to be atomic with the actual entry of the xlog record. Unless we want to hold the xlog insert lock for the entire time that we're (eg) splitting a btree page, that means we log the buffer after the modification work is done, not before. On third thought --- we could still log the original page contents and the modification log record atomically, if what were logged in the xlog record were (essentially) the parameters to the operation being logged, not its results. That is, make the log entry before you start doing the mod work, not after. This might also simplify redo, since redo would be no different from the normal case. I'm not sure why Vadim didn't choose to do it that way; maybe there's some other fine point I'm missing. In any case, it'd be a big code change and not something I'd want to undertake at this point in the release cycle ... maybe we can revisit this issue for 7.2. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
AW: AW: AW: AW: [HACKERS] WAL-based allocation of XIDs is insecure
After thinking about this a little, I believe I see why Vadim did it the way he did. Suppose we tried to make the code sequence be obtain write lock on buffer; XLogOriginalPage(buffer); // copy page to xlog if first since ckpt modify buffer; XLogInsert(xlog entry for modification); mark buffer dirty and release write lock; so that the saving of the original page is a separate xlog entry from the modification data. Looks easy, and it'd sure simplify XLogInsert a lot. The only problem is it's wrong. What if a checkpoint occurs between the two XLOG records? The decision whether to log the whole buffer has to be atomic with the actual entry of the xlog record. Unless we want to hold the xlog insert lock for the entire time that we're (eg) splitting a btree page, that means we log the buffer after the modification work is done, not before. Yes, I see. Can't currently come up with a workaround eighter. Hmm .. Duplicating the buffer is probably not a workable solution. I do not however see how the current solution fixes the original problem, that we don't have a rollback for index modifications. The index would potentially point to an empty heaptuple slot. When this slot, because marked empty is reused after startup, the index points to the wrong record. Unless of course startup rollforward visits all heap pages pointed at by index xlog records and inserts a tuple into heap marked deleted. Additionally I do not see how this all works for userland index types. In short I do not think that the current implementation of "physical log" does what it was intended to do :-( Andreas ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
Peter Eisentraut wrote: Lamar Owen writes: I missed something somehwere: wasn't the consensus a few weeks ago that pg_ctl shouldn't be used for a system initscript? The consensus(?) was that there was some work to do in pg_ctl before it was robust enough to be used (for anything). That work has been done. That was the detail I missed. case when the postmaster does not come down after 60 seconds. But this is really no problem for the issue at hand because if you do a normal runlevel switch then the postmaster will simply keep running, and during a system shutdown all the backends are going to die anyway. Only if each and every shutdown script succeeds in its task. And I have to make sure that the RPM's shipping script successfully pulls down the system in an orderly fashion -- of course, I don't have to worry about the case where a postmaster is going to be started back up if we are in system shutdown -- but, as Tom also stated, I can't assume I'm in the system's death throes when called with the stop parameter. And it _is_ possible for an admin to set up the runlevels such that a level is set aside where even networking isn't running (actually, that level already exists, and is called 'single user mode') -- or a run level for website maintenance where networking is still up, but the webserver and postgresql (and other associated) processes are to be shut down. I personally use this -- I have set up runlevel 4 as a 'remote single user mode' of sorts where I still have sshd running (and the networking stack, obviously), but AOLserver, postgresql, and RealServer are shut down. I then switch runlevels back to 3 to return to normal. Much easier than manually stopping and restarting (in the correct order, as AOLserver is not a happy camper if postmaster drops out from underneath it) all the necessary pieces. So I can't assume anything. The default RPM installation used to automatically configure runlevels 3, 4, and 5 (not any more), but my script can't assume that the system is actually in that state by any means. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] mailing list messages
oh, let me add that as a TIP :)
On Tue, 6 Mar 2001, Peter Eisentraut wrote:
Bruce Momjian writes:
I wonder if the new Tips at the bottom of email messages can be enabled
for users during their first 30 days of mailing list subscription, then
not appear?
I once again refer people to RFC 2369
http://www.faqs.org/rfcs/rfc2369.html about how to embed email list
management information into messages. Secondly, I would also tolerate the
"monthly reminders" that some list managers send out (e.g., GNU, Great
Bridge). At the very least, though, the tips should be preceded by a
'dashdashspacenewline' sequence so that some mail readers can
strip them off in replies.
I filter out the tips anyway, so I really don't care a lot.
# for procmail users
:0 Bbf
| sed -n -e '/^---(end of
broadcast)---$/q' -e 'p'
--
Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: [EMAIL PROTECTED] secondary: scrappy@{freebsd|postgresql}.org
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
Alfred Perlstein [EMAIL PROTECTED] writes: I'm sure some sort of encoding of the PGDATA directory along with the pids stored in the shm segment... I thought about this too, but it strikes me as not very trustworthy. The problem is that there's no guarantee that the new postmaster will even notice the old shmem segment: it might select a different shmem key. (The 7.1 coding of shmem key selection makes this more likely than it used to be, but even under 7.0, it will certainly fail to work if I choose to start the new postmaster using a different port number than the old one had. The shmem key is driven primarily by port number not data directory ...) The interlock has to be tightly tied to the PGDATA directory, because what we're trying to protect is the files in and under that directory. It seems that something based on file(s) in that directory is the way to go. The best idea I've seen so far is Hiroshi's idea of having all the backends hold fcntl locks on the same file (probably postmaster.pid would do fine). Then the new postmaster can test whether any backends are still alive by trying to lock the old postmaster.pid file. Unfortunately, I read in the fcntl man page: Locks are not inherited by a child process in a fork(2) system call. This makes the idea much less attractive than I originally thought: a new backend would not automatically inherit a lock on the postmaster.pid file from the postmaster, but would have to open/lock it for itself. That means there's a window where the new backend exists but would be invisible to a hypothetical new postmaster. We could work around this with the following, very ugly protocol: 1. Postmaster normally maintains fcntl read lock on its postmaster.pid file. Each spawned backend immediately opens and read-locks postmaster.pid, too, and holds that file open until it dies. (Thus wasting a kernel FD per backend, which is one of the less attractive things about this.) If the backend is unable to obtain read lock on postmaster.pid, then it complains and dies. We must use read locks here so that all these processes can hold them separately. 2. If a newly started postmaster sees a pre-existing postmaster.pid file, it tries to obtain a *write* lock on that file. If it fails, conclude that an old postmaster or backend is still alive; complain and quit. If it succeeds, sit for say 1 second before deleting the file and creating a new one. (The delay here is to allow any just-started old backends to fail to acquire read lock and quit. A possible objection is that we have no way to guarantee 1 second is enough, though it ought to be plenty if the lock acquisition is just after the fork.) One thing that worries me a little bit is that this means an fcntl read-lock request will exist inside the kernel for each active backend. Does anyone know of any performance problems or hard kernel limits we might run into with large numbers of backends (lots and lots of fcntl locks)? At least the locks are on a file that we don't actually touch in the normal course of business. A small savings is that the backends don't actually need to open new FDs for the postmaster.pid file; they can use the one they inherit from the postmaster, even though they do need to lock it again. I'm not sure how much that saves inside the kernel, but at least something. There are also the usual set of concerns about portability of flock, though this time we're locking a plain file and not a socket, so it shouldn't be as much trouble as it was before. Comments? Does anyone see a better way to do it? regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
Lamar Owen writes: case when the postmaster does not come down after 60 seconds. But this is really no problem for the issue at hand because if you do a normal runlevel switch then the postmaster will simply keep running, and during a system shutdown all the backends are going to die anyway. Only if each and every shutdown script succeeds in its task. And I have to make sure that the RPM's shipping script successfully pulls down the system in an orderly fashion -- of course, I don't have to worry about the case where a postmaster is going to be started back up if we are in system shutdown -- but, as Tom also stated, I can't assume I'm in the system's death throes when called with the stop parameter. Well, if you have something clever you want to do if the postmaster doesn't come down after an orderly shutdown then please share it. The current alternatives are 'leave running' or 'kill -9'. I know I'd prefer the former. -- Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/ ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: AW: AW: AW: AW: [HACKERS] WAL-based allocation of XIDs is insecur e
Zeugswetter Andreas SB [EMAIL PROTECTED] writes: I do not however see how the current solution fixes the original problem, that we don't have a rollback for index modifications. The index would potentially point to an empty heaptuple slot. How? There will be an XLOG entry inserting the heap tuple before the XLOG entry that updates the index. Rollforward will redo both. The heap tuple might not get committed, but it'll be there. Additionally I do not see how this all works for userland index types. None of it works for index types that don't do XLOG entries (which I think may currently be true for everything except btree :-( ...). I don't see how that changes if we alter the way this bit is done. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
* Tom Lane [EMAIL PROTECTED] [010306 10:10] wrote: Alfred Perlstein [EMAIL PROTECTED] writes: I'm sure some sort of encoding of the PGDATA directory along with the pids stored in the shm segment... I thought about this too, but it strikes me as not very trustworthy. The problem is that there's no guarantee that the new postmaster will even notice the old shmem segment: it might select a different shmem key. (The 7.1 coding of shmem key selection makes this more likely than it used to be, but even under 7.0, it will certainly fail to work if I choose to start the new postmaster using a different port number than the old one had. The shmem key is driven primarily by port number not data directory ...) This seems like a mistake. I'm suprised you guys aren't just using some form of the FreeBSD ftok() algorithm for this: FTOK(3)FreeBSD Library Functions ManualFTOK(3) ... The ftok() function attempts to create a unique key suitable for use with the msgget(3), semget(2) and shmget(2) functions given the path of an ex- isting file and a user-selectable id. The specified path must specify an existing file that is accessible to the calling process or the call will fail. Also, note that links to files will return the same key, given the same id. BUGS The returned key is computed based on the device minor number and inode of the specified path in combination with the lower 8 bits of the given id. Thus it is quite possible for the routine to return duplicate keys. The "BUGS" seems to be exactly what you guys are looking for, a somewhat reliable method of obtaining a system id. If that sounds evil, read below for an alternate suggestion. The interlock has to be tightly tied to the PGDATA directory, because what we're trying to protect is the files in and under that directory. It seems that something based on file(s) in that directory is the way to go. The best idea I've seen so far is Hiroshi's idea of having all the backends hold fcntl locks on the same file (probably postmaster.pid would do fine). Then the new postmaster can test whether any backends are still alive by trying to lock the old postmaster.pid file. Unfortunately, I read in the fcntl man page: Locks are not inherited by a child process in a fork(2) system call. This makes the idea much less attractive than I originally thought: a new backend would not automatically inherit a lock on the postmaster.pid file from the postmaster, but would have to open/lock it for itself. That means there's a window where the new backend exists but would be invisible to a hypothetical new postmaster. We could work around this with the following, very ugly protocol: 1. Postmaster normally maintains fcntl read lock on its postmaster.pid file. Each spawned backend immediately opens and read-locks postmaster.pid, too, and holds that file open until it dies. (Thus wasting a kernel FD per backend, which is one of the less attractive things about this.) If the backend is unable to obtain read lock on postmaster.pid, then it complains and dies. We must use read locks here so that all these processes can hold them separately. 2. If a newly started postmaster sees a pre-existing postmaster.pid file, it tries to obtain a *write* lock on that file. If it fails, conclude that an old postmaster or backend is still alive; complain and quit. If it succeeds, sit for say 1 second before deleting the file and creating a new one. (The delay here is to allow any just-started old backends to fail to acquire read lock and quit. A possible objection is that we have no way to guarantee 1 second is enough, though it ought to be plenty if the lock acquisition is just after the fork.) One thing that worries me a little bit is that this means an fcntl read-lock request will exist inside the kernel for each active backend. Does anyone know of any performance problems or hard kernel limits we might run into with large numbers of backends (lots and lots of fcntl locks)? At least the locks are on a file that we don't actually touch in the normal course of business. A small savings is that the backends don't actually need to open new FDs for the postmaster.pid file; they can use the one they inherit from the postmaster, even though they do need to lock it again. I'm not sure how much that saves inside the kernel, but at least something. There are also the usual set of concerns about portability of flock, though this time we're locking a plain file and not a socket, so it shouldn't be as much trouble as it was before. Comments? Does anyone see a better way to do it? Possibly... What about encoding the shm id in the pidfile? Then one can just ask how many processes are attached to that segment? (if it doesn't exist, one can assume all backends have exited) you want the field 'shm_nattch' The shmid_ds struct is
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
Alfred Perlstein [EMAIL PROTECTED] writes: * Tom Lane [EMAIL PROTECTED] [010306 10:10] wrote: The shmem key is driven primarily by port number not data directory ...) This seems like a mistake. I'm suprised you guys aren't just using some form of the FreeBSD ftok() algorithm for this: This has been discussed before --- see the archives. The conclusion was that since ftok doesn't guarantee uniqueness, it adds nothing except lack of predictability to the shmem key selection process. We'd still need logic to cope with key collisions, and given that, we might as well select keys that have some obvious relationship to user-visible parameters, viz the port number. As is, you can fairly easily tell which shmem segment belongs to which postmaster from the shmem key; with ftok-derived keys, you couldn't tell a thing. Comments? Does anyone see a better way to do it? What about encoding the shm id in the pidfile? Then one can just ask how many processes are attached to that segment? (if it doesn't exist, one can assume all backends have exited) Hmm ... that might actually be a pretty good idea. A small problem is that the shm key isn't yet selected at the time we initially create the lockfile, but I can't think of any reason that we could not go back and append the key to the lockfile afterwards. you want the field 'shm_nattch' Are there any portability problems with relying on shm_nattch to be available? If not, I like this a lot... regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
* Tom Lane [EMAIL PROTECTED] [010306 10:35] wrote: Alfred Perlstein [EMAIL PROTECTED] writes: What about encoding the shm id in the pidfile? Then one can just ask how many processes are attached to that segment? (if it doesn't exist, one can assume all backends have exited) Hmm ... that might actually be a pretty good idea. A small problem is that the shm key isn't yet selected at the time we initially create the lockfile, but I can't think of any reason that we could not go back and append the key to the lockfile afterwards. you want the field 'shm_nattch' Are there any portability problems with relying on shm_nattch to be available? If not, I like this a lot... Well it's available on FreeBSD and Solaris, I'm sure Redhat has some deamon that resets the value to 0 periodically just for kicks so it might not be viable... :) Seriously, there's some dispute on the type that 'shm_nattch' is, under Solaris it's "shmatt_t" (unsigned long afaik), under FreeBSD it's 'short' (i should fix this. :)). But since you're really only testing for 0'ness then it shouldn't really be a problem. -- -Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] mailing list messages
Bruce Momjian wrote: I wonder if the new Tips at the bottom of email messages can be enabled for users during their first 30 days of mailing list subscription, then not appear? What about having some basic _PostgreSQL_ tips in there? This would be especially cute for -novice, I think. We must all be able to come up with 100 or so little one or two liners about PostgreSQL can't we? Just a thought, Andrew. -- _ Andrew McMillan, e-mail: [EMAIL PROTECTED] Catalyst IT Ltd, PO Box 10-225, Level 22, 105 The Terrace, Wellington Me: +64 (21) 635 694, Fax: +64 (4) 499 5596, Office: +64 (4) 499 2267 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] mailing list messages
On Wed, 7 Mar 2001, Andrew McMillan wrote: Bruce Momjian wrote: I wonder if the new Tips at the bottom of email messages can be enabled for users during their first 30 days of mailing list subscription, then not appear? What about having some basic _PostgreSQL_ tips in there? This would be especially cute for -novice, I think. We must all be able to come up with 100 or so little one or two liners about PostgreSQL can't we? Since Peter has shown how easy it is to get rid of the TIPs for those that dont' like it, I think that's a cool idea :) ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
Alfred Perlstein [EMAIL PROTECTED] writes: Are there any portability problems with relying on shm_nattch to be available? If not, I like this a lot... Well it's available on FreeBSD and Solaris, I'm sure Redhat has some deamon that resets the value to 0 periodically just for kicks so it might not be viable... :) I notice that our BeOS and QNX emulations of shmctl() don't support IPC_STAT, but that could be dealt with, at least to the extent of stubbing it out. This does raise the question of what to do if shmctl(IPC_STAT) fails for a reason other than EINVAL. I think the conservative thing to do is refuse to start up. On EPERM, for example, it's possible that there is a postmaster running in your PGDATA but with a different userid. Seriously, there's some dispute on the type that 'shm_nattch' is, under Solaris it's "shmatt_t" (unsigned long afaik), under FreeBSD it's 'short' (i should fix this. :)). But since you're really only testing for 0'ness then it shouldn't really be a problem. We need not copy the value anywhere, so as long as the struct is correctly declared in the system header files I don't think it matters what the field type is ... regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
Peter Eisentraut wrote: Well, if you have something clever you want to do if the postmaster doesn't come down after an orderly shutdown then please share it. The current alternatives are 'leave running' or 'kill -9'. I know I'd prefer the former. Well, my preferences aren't really relevant here. I have a job to do as an initscript in the RPMish environment -- and I really have to meet my obligations (using the first personal pronoun there to anthropomorph the initscript to a person, allowing us to have a little sympathy for the poor shell script's plight :-)). My preference is to let it float in limbo -- if it's in limbo and won't come out, then we have bigger issues. However, I could do something really sneaky in the RedHat environment and let init do the dirty work for me -- but, again, I am not at all guaranteed that things will come down orderly -- if it is at all possible for me to bring about an orderly (if slow) shutdown that does terminate as the rest of the system needs it to do, then I'll attempt to do so. But, the immediate issue is preventing chaotic stops within the initscript, so I'm going to experiment with things and see if I can make the initscript hang -- if I can't, then I'll likely put in the 'killproc postmaster -INT' with escalation to -TERM if it doesn't come down within sixty seconds (and, no, I am not going to sleep 60 then check things -- I am going to sleep 1 and loop sixty times) -- no need to unnecessarily delay system shutdown (and potential restart). And I won't put in the -KILL unless I can find a safe and thorough way to do so. Or I may go ahead and pg_ctl-ize things and let pg_ctl do the dirty work, as that IS what pg_ctl is supposed to accomplish. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
Alfred Perlstein writes:
Seriously, there's some dispute on the type that 'shm_nattch' is,
under Solaris it's "shmatt_t" (unsigned long afaik), under FreeBSD
it's 'short' (i should fix this. :)).
What I don't like is that my /usr/include/sys/shm.h (through other
headers) has:
typedef unsigned long int shmatt_t;
/* Data structure describing a set of semaphores. */
struct shmid_ds
{
struct ipc_perm shm_perm; /* operation permission struct */
size_t shm_segsz; /* size of segment in bytes */
__time_t shm_atime; /* time of last shmat() */
unsigned long int __unused1;
__time_t shm_dtime; /* time of last shmdt() */
unsigned long int __unused2;
__time_t shm_ctime; /* time of last change by shmctl() */
unsigned long int __unused3;
__pid_t shm_cpid; /* pid of creator */
__pid_t shm_lpid; /* pid of last shmop */
shmatt_t shm_nattch;/* number of current attaches */
unsigned long int __unused4;
unsigned long int __unused5;
};
whereas /usr/src/linux/include/shm.h has:
struct shmid_ds {
struct ipc_perm shm_perm; /* operation perms */
int shm_segsz; /* size of segment (bytes) */
__kernel_time_t shm_atime; /* last attach time */
__kernel_time_t shm_dtime; /* last detach time */
__kernel_time_t shm_ctime; /* last change time */
__kernel_ipc_pid_t shm_cpid; /* pid of creator */
__kernel_ipc_pid_t shm_lpid; /* pid of last operator */
unsigned short shm_nattch; /* no. of current attaches */
unsigned short shm_unused; /* compatibility */
void*shm_unused2; /* ditto - used by DIPC */
void*shm_unused3; /* unused */
};
Not only note the shm_nattch type, but also shm_segsz, and the "unused"
fields in between. I don't know a thing about the Linux kernel sources,
but this doesn't seem right.
--
Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
* Tom Lane [EMAIL PROTECTED] [010306 11:03] wrote: Alfred Perlstein [EMAIL PROTECTED] writes: Are there any portability problems with relying on shm_nattch to be available? If not, I like this a lot... Well it's available on FreeBSD and Solaris, I'm sure Redhat has some deamon that resets the value to 0 periodically just for kicks so it might not be viable... :) I notice that our BeOS and QNX emulations of shmctl() don't support IPC_STAT, but that could be dealt with, at least to the extent of stubbing it out. Well since we already have spinlocks, I can't see why we can't keep the refcount and spinlock in a special place in the shm for all cases? This does raise the question of what to do if shmctl(IPC_STAT) fails for a reason other than EINVAL. I think the conservative thing to do is refuse to start up. On EPERM, for example, it's possible that there is a postmaster running in your PGDATA but with a different userid. Yes, if possible a more meaningfull error message and pointer to some docco would be nice or even a nice "i don't care, i killed all the backends, just start darnit" flag, it's really no fun at all to have to attempt to decypher some cryptic error message at 3am when the database/system is acting up. :) Seriously, there's some dispute on the type that 'shm_nattch' is, under Solaris it's "shmatt_t" (unsigned long afaik), under FreeBSD it's 'short' (i should fix this. :)). But since you're really only testing for 0'ness then it shouldn't really be a problem. We need not copy the value anywhere, so as long as the struct is correctly declared in the system header files I don't think it matters what the field type is ... Yup, my point exactly. -- -Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
Alfred Perlstein [EMAIL PROTECTED] writes: * Tom Lane [EMAIL PROTECTED] [010306 11:03] wrote: I notice that our BeOS and QNX emulations of shmctl() don't support IPC_STAT, but that could be dealt with, at least to the extent of stubbing it out. Well since we already have spinlocks, I can't see why we can't keep the refcount and spinlock in a special place in the shm for all cases? No, we mustn't go there. If the kernel isn't keeping the refcount then it's worse than useless: as soon as some process crashes without decrementing its refcount, you have a condition that you can't recover from without reboot. What I'm currently imagining is that the stub implementations will just return a failure code for IPC_STAT, and the outer code will in turn fail with a message along the lines of "It looks like there's a pre-existing shmem block (id XXX) still in use. If you're sure there are no old backends still running, remove the shmem block with ipcrm(1), or just delete $PGDATA/postmaster.pid." I dunno what shmem management tools exist on BeOS/QNX, but deleting the lockfile will definitely suppress the startup interlock ;-). Yes, if possible a more meaningfull error message and pointer to some docco would be nice Is the above good enough? regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
Peter Eisentraut wrote: Not only note the shm_nattch type, but also shm_segsz, and the "unused" fields in between. I don't know a thing about the Linux kernel sources, but this doesn't seem right. Red Hat 7, right? My RedHat 7 system isn't running RH 7 right now (it's this notebook that I'm running Win95 on right now), but see which RPM's own the two headers. You may be in for a shock. IIRC, the first system include is from the 2.4 kernel, and the second in the kernel source tree is from the 2.2 kernel. Odd, but not really broken. Should be fixed in the latest public beta of RedHat, that actually has the 2.4 kernel. I can't really say any more about that, however. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
Peter Eisentraut [EMAIL PROTECTED] writes: What I don't like is that my /usr/include/sys/shm.h (through other headers) has [foo] whereas /usr/src/linux/include/shm.h has [bar] Are those declarations perhaps bit-compatible? Looks a tad endian- dependent, though ... regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
RE: [HACKERS] mailing list messages
I like this idea too. How about TIP #1: Don't 'kill -9' the postmaster ;-) Mike Mascari [EMAIL PROTECTED] -Original Message- From: The Hermit Hacker [SMTP:[EMAIL PROTECTED]] Sent: Tuesday, March 06, 2001 1:57 PM To: Andrew McMillan Cc: PostgreSQL-development Subject:Re: [HACKERS] mailing list messages On Wed, 7 Mar 2001, Andrew McMillan wrote: Bruce Momjian wrote: I wonder if the new Tips at the bottom of email messages can be enabled for users during their first 30 days of mailing list subscription, then not appear? What about having some basic _PostgreSQL_ tips in there? This would be especially cute for -novice, I think. We must all be able to come up with 100 or so little one or two liners about PostgreSQL can't we? Since Peter has shown how easy it is to get rid of the TIPs for those that dont' like it, I think that's a cool idea :) ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
* Tom Lane [EMAIL PROTECTED] [010306 11:30] wrote: Alfred Perlstein [EMAIL PROTECTED] writes: * Tom Lane [EMAIL PROTECTED] [010306 11:03] wrote: I notice that our BeOS and QNX emulations of shmctl() don't support IPC_STAT, but that could be dealt with, at least to the extent of stubbing it out. Well since we already have spinlocks, I can't see why we can't keep the refcount and spinlock in a special place in the shm for all cases? No, we mustn't go there. If the kernel isn't keeping the refcount then it's worse than useless: as soon as some process crashes without decrementing its refcount, you have a condition that you can't recover from without reboot. Not if the postmaster outputs the following: What I'm currently imagining is that the stub implementations will just return a failure code for IPC_STAT, and the outer code will in turn fail with a message along the lines of "It looks like there's a pre-existing shmem block (id XXX) still in use. If you're sure there are no old backends still running, remove the shmem block with ipcrm(1), or just delete $PGDATA/postmaster.pid." I dunno what shmem management tools exist on BeOS/QNX, but deleting the lockfile will definitely suppress the startup interlock ;-). Yes, if possible a more meaningfull error message and pointer to some docco would be nice Is the above good enough? Sure. :) -- -Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] ERROR: cannot open relation center_out_analog_proc
I was trying to add a column to a table and fill it but ran into a big error. Apparently now Postgres can't open this table to vacuum or to select although it does show up when I ask psql to describe the table (i.e. db01=# /d center_out_analog_proc). I'm using Postgres 7.0.3 on a PII/400 MHz with RedHat Linux (kernel 2.2.14-5). The command that started the problem was from the script: -- Re-arranges the columns in a table -- -- Tony Reina -- Created: 6 March 2001 -- The BEGIN and COMMIT statements ensure that either all statements are done or none are done BEGIN WORK; -- ADD THE NEW COLUMN TO THE TABLE ALTER TABLE center_out_analog_proc ADD COLUMN name text; -- SELECT the columns from the table in whatever new format you wish. Place into a temporary table. SELECT subject, arm, target, rep, channel, name, cut_off_freq, quality, analog_data INTO temp_table FROM center_out_analog_proc; -- DROP THE OLD TABLE DROP TABLE center_out_analog_proc; -- MAKE THE NEW TABLE INTO THE OLD TABLE ALTER TABLE temp_table RENAME TO center_out_analog_proc; -- FILL THE NEW COLUMN WITH THE CORRECT DATA UPDATE center_out_analog_proc SET name = (SELECT name FROM center_out_analog AS a WHERE a.subject = center_out_analog_proc.subject AND a.arm = center_out_analog_proc.arm AND a.target = center_out_analog_proc.target AND a.rep = center_out_analog_proc.rep AND a.channel = center_out_analog_proc.channel); -- VACUUM THE TABLE VACUUM VERBOSE ANALYZE center_out_analog_proc; COMMIT WORK; --- When I ran this, I had an error in the UPDATE command (so the entire transaction aborted). I assumed that becuase the transaction aborted that nothing would have changed in the db. However, after this happened, I corrected the UPDATE command but ran into this error when I re-ran the script: db01=# \i alter_table_format.sql BEGIN psql:alter_table_format.sql:14: NOTICE: mdopen: couldn't open center_out_analog_proc: No such file or directory psql:alter_table_format.sql:14: NOTICE: mdopen: couldn't open center_out_analog_proc: No such file or directory psql:alter_table_format.sql:14: NOTICE: mdopen: couldn't open center_out_analog_proc: No such file or directory psql:alter_table_format.sql:14: NOTICE: mdopen: couldn't open center_out_analog_proc: No such file or directory psql:alter_table_format.sql:14: ERROR: cannot open relation center_out_analog_proc psql:alter_table_format.sql:17: NOTICE: current transaction is aborted, queries ignored until end of transaction block *ABORT STATE* psql:alter_table_format.sql:20: NOTICE: current transaction is aborted, queries ignored until end of transaction block *ABORT STATE* psql:alter_table_format.sql:26: NOTICE: mdopen: couldn't open center_out_analog_proc: No such file or directory psql:alter_table_format.sql:26: NOTICE: mdopen: couldn't open center_out_analog_proc: No such file or directory psql:alter_table_format.sql:26: NOTICE: mdopen: couldn't open center_out_analog_proc: No such file or directory psql:alter_table_format.sql:26: NOTICE: mdopen: couldn't open center_out_analog_proc: No such file or directory psql:alter_table_format.sql:26: NOTICE: current transaction is aborted, queries ignored until end of transaction block *ABORT STATE* psql:alter_table_format.sql:29: NOTICE: current transaction is aborted, queries ignored until end of transaction block *ABORT STATE* COMMIT When I try to vacuum the table or the database I get: NOTICE: Pages 190: Changed 0, reaped 0, Empty 0, New 0; Tup 9280: Vac 0, Keep/VTL 0/0, Crash 0, UnUsed 0, MinLen 652, MaxLen 652; Re-using: Free/Avail. Space 0/0; EndEmpty/Avail. Pages 0/0. CPU 0.07s/0.14u sec. NOTICE: --Relation circles_analog_proc -- NOTICE: Pages 187: Changed 0, reaped 0, Empty 0, New 0; Tup 9140: Vac 0, Keep/VTL 0/0, Crash 0, UnUsed 0, MinLen 652, MaxLen 652; Re-using: Free/Avail. Space 0/0; EndEmpty/Avail. Pages 0/0. CPU 0.08s/0.13u sec. NOTICE: mdopen: couldn't open center_out_analog_proc: No such file or directory NOTICE: RelationIdBuildRelation: smgropen(center_out_analog_proc): No such file or directory NOTICE: --Relation center_out_analog_proc -- NOTICE: mdopen: couldn't open center_out_analog_proc: No such file or directory ERROR: cannot open relation center_out_analog_proc db01=# select distinct monkey from center_out_analog_proc; NOTICE: mdopen: couldn't open center_out_analog_proc: No such file or directory NOTICE: mdopen: couldn't open center_out_analog_proc: No such file or directory NOTICE: mdopen: couldn't open center_out_analog_proc: No such file or directory NOTICE: mdopen: couldn't open center_out_analog_proc: No such file or directory ERROR: cannot open relation center_out_analog_proc Likewise, a select gives me: db01=# select distinct arm from center_out_analog_proc; NOTICE: mdopen: couldn't open center_out_analog_proc: No such file or directory NOTICE: mdopen: couldn't open center_out_analog_proc:
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
* Lamar Owen [EMAIL PROTECTED] [010306 11:39] wrote: Peter Eisentraut wrote: Not only note the shm_nattch type, but also shm_segsz, and the "unused" fields in between. I don't know a thing about the Linux kernel sources, but this doesn't seem right. Red Hat 7, right? My RedHat 7 system isn't running RH 7 right now (it's this notebook that I'm running Win95 on right now), but see which RPM's own the two headers. You may be in for a shock. IIRC, the first system include is from the 2.4 kernel, and the second in the kernel source tree is from the 2.2 kernel. Odd, but not really broken. Should be fixed in the latest public beta of RedHat, that actually has the 2.4 kernel. I can't really say any more about that, however. Y'know, I was only kidding about Linux going out of its way to defeat the 'shm_nattch' trick... *sigh* As a FreeBSD developer I'm wondering if Linux keeps compatibility calls around for old binaries or not. Any idea? -- -Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] epoch
Trying example from:
http://www.postgresql.org/devel-corner/docs/user/functions-datetime.html
patrimoine=# SELECT EXTRACT(EPOCH FROM TIMESTAMP '2001-02-16 20:38:40');
ERROR: parser: parse error at or near "epoch"
patrimoine=# select version();
version
---
PostgreSQL 7.1beta3 on i386-unknown-netbsdelf1.5Q, compiled by GCC egcs-1.1.2
(1 row)
patrimoine=# select date_part('epoch','2001-02-16 20:38:40'::timestamp);
date_part
---
982355920
(1 row)
Is my version already too old?
Cheers,
Patrick
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [HACKERS] WAL-based allocation of XIDs is insecure
Ian Lance Taylor [EMAIL PROTECTED] writes: I described myself unclearly. I was suggesting an addition to what you are suggesting. The worst case can not be worse. Then I didn't (and still don't) understand your suggestion. Want to try again? If you are going to allocate a few thousand XIDs each time, then I agree that my suggested addition is not worth it. But how do you deal with XID wraparound on an unstable system? About the same as we do now: not very well. But if your system is that unstable, XID wrap is the least of your worries, I think. Up through 7.0, Postgres allocated XIDs a thousand at a time, and not only did the not-yet-used XIDs get lost in a crash, they'd get lost in a normal shutdown too. What I propose will waste XIDs in a crash but not in a normal shutdown, so it's still an improvement over prior versions as far as XID consumption goes. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
RE: [HACKERS] mailing list messages
there, added ... and even gave an attribute to you :)
On Tue, 6 Mar 2001, Mike Mascari wrote:
I like this idea too. How about TIP #1: Don't 'kill -9' the postmaster ;-)
Mike Mascari
[EMAIL PROTECTED]
-Original Message-
From: The Hermit Hacker [SMTP:[EMAIL PROTECTED]]
Sent: Tuesday, March 06, 2001 1:57 PM
To: Andrew McMillan
Cc: PostgreSQL-development
Subject: Re: [HACKERS] mailing list messages
On Wed, 7 Mar 2001, Andrew McMillan wrote:
Bruce Momjian wrote:
I wonder if the new Tips at the bottom of email messages can be enabled
for users during their first 30 days of mailing list subscription, then
not appear?
What about having some basic _PostgreSQL_ tips in there? This would be
especially cute for -novice, I think.
We must all be able to come up with 100 or so little one or two liners
about PostgreSQL can't we?
Since Peter has shown how easy it is to get rid of the TIPs for those that
dont' like it, I think that's a cool idea :)
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: [EMAIL PROTECTED] secondary: scrappy@{freebsd|postgresql}.org
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
* Tom Lane [EMAIL PROTECTED] [010306 11:49] wrote: Peter Eisentraut [EMAIL PROTECTED] writes: What I don't like is that my /usr/include/sys/shm.h (through other headers) has [foo] whereas /usr/src/linux/include/shm.h has [bar] Are those declarations perhaps bit-compatible? Looks a tad endian- dependent, though ... Of course not, the size of the struct changed (short-unsigned long, basically int16_t - uint32_t), because the kernel and userland in Linux are hardly in sync you have the fun of guessing if you get: old struct - old syscall (ok) new struct - old syscall (boom) old struct - new syscall (boom) new struct - new syscall (ok) Honestly I think this problem should be left to the vendor to fix properly (if it needs fixing), the sysV API was published at least 6 years ago, they ought to have it mostly correct by now. -- -Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://www.postgresql.org/search.mpl
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
Alfred Perlstein [EMAIL PROTECTED] writes: Of course not, the size of the struct changed (short-unsigned long, basically int16_t - uint32_t), because the kernel and userland in Linux are hardly in sync you have the fun of guessing if you get: old struct - old syscall (ok) new struct - old syscall (boom) old struct - new syscall (boom) new struct - new syscall (ok) Ugh. However, it looks like it might be fairly fail-soft: if we have the wrong declaration then we pick up some other field of the struct, and probably end up complaining because nattch appears nonzero. Recovery method (clean up the shm seg or delete lockfile) is the same. I'm still inclined to go with this; it beats corrupting the WAL log, and the fcntl(SETLK) alternative has its own set of portability booby-traps. regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] epoch
Patrick Welche writes: http://www.postgresql.org/devel-corner/docs/user/functions-datetime.html patrimoine=# SELECT EXTRACT(EPOCH FROM TIMESTAMP '2001-02-16 20:38:40'); ERROR: parser: parse error at or near "epoch" patrimoine=# select version(); version --- PostgreSQL 7.1beta3 on i386-unknown-netbsdelf1.5Q, compiled by GCC egcs-1.1.2 (1 row) Is my version already too old? Yes. -- Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/ ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
Bruce Momjian writes: This will try a pg_ctl shutdown for 60 seconds, then kill pg_ctl. You would then need a kill of you own. pg_ctl automatically times out after 60 seconds. -- Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/ ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
Nathan Myers wrote: That is why there is no problem with version skew in the syscall argument structures on a correctly-configured Linux system. (On a Red Hat system it is very easy to get them out of sync, but RH fans are used to problems.) Is RedHat bashing really necessary here? At least they are payrolling Second Chair on the Linux kernel hierarchy. And they are very supportive of PostgreSQL (by shipping us with their distribution). -- Lamar Owen WGCR Internet Radio 1 Peter 4:11 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] ERROR: cannot open relation center_out_analog_proc
"G. Anthony Reina" [EMAIL PROTECTED] writes: BEGIN WORK; ... DROP TABLE center_out_analog_proc; ... [fail transaction] psql:alter_table_format.sql:14: NOTICE: mdopen: couldn't open center_out_analog_proc: No such file or directory You can't roll back a DROP TABLE under pre-7.1 releases (and 7.0 has a big fat warning notice to tell you so!). The physical table file is deleted immediately by the DROP, so rolling back the system catalog changes doesn't get you back to a working table. The only way to clean up at this point is to drop the table for real. regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] ERROR: cannot open relation center_out_analog_proc
Tom Lane wrote: You can't roll back a DROP TABLE under pre-7.1 releases (and 7.0 has a big fat warning notice to tell you so!). The physical table file is deleted immediately by the DROP, so rolling back the system catalog changes doesn't get you back to a working table. The only way to clean up at this point is to drop the table for real. Okay, so then you are saying that even though the DROP TABLE and ALTER TABLE RENAME went through correctly, the line after that bombed out, tried to rollback the transaction, and gave me the error? I definitely missed that warning. Are there any big warnings for things that don't work so well within a transaction (BEGIN WORK; COMMIT WORK)? Thanks. Off to rebuild a table. -Tony ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] ERROR: cannot open relation center_out_analog_proc
"G. Anthony Reina" [EMAIL PROTECTED] writes: Tom Lane wrote: You can't roll back a DROP TABLE under pre-7.1 releases (and 7.0 has a big fat warning notice to tell you so!). The physical table file is deleted immediately by the DROP, so rolling back the system catalog changes doesn't get you back to a working table. Okay, so then you are saying that even though the DROP TABLE and ALTER TABLE RENAME went through correctly, the line after that bombed out, tried to rollback the transaction, and gave me the error? Right. The system catalogs roll back just fine, but the Unix filesystem doesn't know from rollbacks. I definitely missed that warning. Are there any big warnings for things that don't work so well within a transaction (BEGIN WORK; COMMIT WORK)? ALTER TABLE RENAME is another one... This is all fixed in 7.1 btw. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://www.postgresql.org/search.mpl
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
On Tue, Mar 06, 2001 at 12:46:24PM -0800, Nathan Myers wrote: On Linux, /usr/src/linux/include is meaningless for anything in userland; it's meant only for building the kernel and kernel modules. That Red Hat tends to expose it to user-level builds is a long-standing bug in Red Hat's distribution, in violation of the File Hierarchy Standard as well as explicit instructions from Linus crew and from the maintainer of the C library. Red Hat's Fisher Beta has split the 2 includes, which caused an error trying to compile a (I guess badly configured) kernel module. The header files in /usr/include now give an error if you try to build a kernel module that gets header files from there. So whether they were wrong in the past or not, they are now doing things the way you say is proper. ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
* Lamar Owen [EMAIL PROTECTED] [010306 13:27] wrote: Nathan Myers wrote: That is why there is no problem with version skew in the syscall argument structures on a correctly-configured Linux system. (On a Red Hat system it is very easy to get them out of sync, but RH fans are used to problems.) Is RedHat bashing really necessary here? At least they are payrolling Second Chair on the Linux kernel hierarchy. And they are very supportive of PostgreSQL (by shipping us with their distribution). Just because they do some really nice things and have some really nice stuff doesn't mean they should really get cut any slack for doing things like shipping out of sync kernel/system headers, kill -9'ing databases and having programs like 'tmpwatch' running on the boxes. It really shows a lack of understanding of how Unix is supposed to run. What they really need to do is hire some grey beards (old school Unix folks) to QA the releases and keep stuff like this from happening/shipping. -- -Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
[HACKERS] Contributions?
Greetings, I wrote a few simple programs to log Apache access_log entries to pg. If this is something anyone would be interested in or if there is someplace I can submit these to, please let me know. Thanks, Matthew ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] Re: CORBA and PG
I'm trying to look to see if I can store CORBA objects inside PG, any ideas... CORBA has mechanisms for locating and executing remote objects. Some services, like the naming service, could use a database as a persistant store. Other services, like the implementation repository, could use a database to hold rules for *how* to start a service, as well as holding persistant info. CORBA IORs are glue holding clients and servers together; storing those in a database would make them persistant (as mentioned above for the naming service). An actual CORBA object typically is an executable, which would need to be stored as a binary object. Not sure what storing that in a database would do for you; perhaps you could give us a use case? - Thomas ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
[HACKERS] Re: CORBA and PG
I'm trying to look to see if I can store CORBA objects inside PG, any ideas... CORBA has mechanisms for locating and executing remote objects. Some services, like the naming service, could use a database as a persistant store. Other services, like the implementation repository, could use a database to hold rules for *how* to start a service, as well as holding persistant info. CORBA IORs are glue holding clients and servers together; storing those in a database would make them persistant (as mentioned above for the naming service). An actual CORBA object typically is an executable, which would need to be stored as a binary object. Not sure what storing that in a database would do for you; perhaps you could give us a use case? - Thomas ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
BeOS haven't this stat (I have a bunch of others but not this one). If I unsterstand correctly, you want to check if there is some backend still attached to shared mem segment of a given key ? In this case, I have an easy solution to fake the stat, because all segment have an encoded name containing this key, so I can count them. cyril Alfred Perlstein [EMAIL PROTECTED] writes: Are there any portability problems with relying on shm_nattch to be available? If not, I like this a lot... Well it's available on FreeBSD and Solaris, I'm sure Redhat has some deamon that resets the value to 0 periodically just for kicks so it might not be viable... :) I notice that our BeOS and QNX emulations of shmctl() don't support IPC_STAT, but that could be dealt with, at least to the extent of stubbing it out. This does raise the question of what to do if shmctl(IPC_STAT) fails for a reason other than EINVAL. I think the conservative thing to do is refuse to start up. On EPERM, for example, it's possible that there is a postmaster running in your PGDATA but with a different userid. Seriously, there's some dispute on the type that 'shm_nattch' is, under Solaris it's "shmatt_t" (unsigned long afaik), under FreeBSD it's 'short' (i should fix this. :)). But since you're really only testing for 0'ness then it shouldn't really be a problem. We need not copy the value anywhere, so as long as the struct is correctly declared in the system header files I don't think it matters what the field type is ... regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED] ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://www.postgresql.org/search.mpl
[HACKERS] psql missing feature
What about (optionally) printing the type of the column data? i.e: io | tu | tipo |data int | int | int2 |date +---+--+ 102242 | 26404 | 1203 | 2000-11-22 (1 row) ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
Alfred Perlstein [EMAIL PROTECTED] writes: Are there any portability problems with relying on shm_nattch to be available? If not, I like this a lot... Well it's available on FreeBSD and Solaris, I'm sure Redhat has some deamon that resets the value to 0 periodically just for kicks so it might not be viable... :) I notice that our BeOS and QNX emulations of shmctl() don't support IPC_STAT, but that could be dealt with, at least to the extent of stubbing it out. * Cyril VELTER [EMAIL PROTECTED] [010306 16:15] wrote: BeOS haven't this stat (I have a bunch of others but not this one). If I unsterstand correctly, you want to check if there is some backend still attached to shared mem segment of a given key ? In this case, I have an easy solution to fake the stat, because all segment have an encoded name containing this key, so I can count them. We need to be able to take a single shared memory segment and determine if any other process is using it. -- -Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://www.postgresql.org/search.mpl
Re: [HACKERS] Red Hat bashing
On Tue, Mar 06, 2001 at 04:20:13PM -0500, Lamar Owen wrote: Nathan Myers wrote: That is why there is no problem with version skew in the syscall argument structures on a correctly-configured Linux system. (On a Red Hat system it is very easy to get them out of sync, but RH fans are used to problems.) Is RedHat bashing really necessary here? I recognize that my last seven words above contributed nothing. In the future I will only post strictly factual statements about Red Hat and similarly charged topics, and keep the opinions to myself. I value the collegiality of this list too much to risk it further. I offer my apologies for violating it. By the way... do they call Red Hat "RedHat" at Red Hat? Nathan Myers [EMAIL PROTECTED] ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
Alfred Perlstein wrote: What they really need to do is hire some grey beards (old school Unix folks) to QA the releases and keep stuff like this from happening/shipping. Like the 250-strong RedHat Beta Team, of which I am a member? :-) I can't disclose the discussions on that list, but, suffice to say the traffic there is at least as great as the traffic on this one. Of course, 7.1 hasn't shipped with a RedHat release yet -- and it's my job to make sure the postmaster gets killed properly in my initscript inside the package for 7.1 -- there will be no kill -9 unless it is an emergency to do so for postmaster. I've seen the advisories and the bug lists -- RedHat is not alone with bugs -- not even unusual with bugs. And every OS I know of (and you too) has had a brown paper bag release before. Even PostgreSQL, given its high release quality standards, has had a brown paper bag release -- we all still make mistakes (I know -- I've made more than my share of them). Anyway, that's more than what the rest of the list wanted to read. Replies to private e-mail, please. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Red Hat bashing
Nathan Myers wrote: it further. I offer my apologies for violating it. Well, an apology is not really necessary -- but I do get a little tired at the treatment a good open source company gets at the hands of open source advocates. Yes, they make mistakes. Everyone does. By the way... do they call Red Hat "RedHat" at Red Hat? No, they don't. I don't know how I got into the habit of leaving out the space, but the space is supposed to be there -- unless you are on the Red Hat CD, where you will find a directory called 'RedHat'. Oh well. Totally off topic. If the from header had your personal address in it (Reply-All only lets me reply to the list for that message) I wouldn't grieve the list further with it. My last words on that subject. Let's go on making PostgreSQL better. And preventing the kill -9 will make PostgreSQL better, even if it is masking a certain amount of shortsightedness on a certain initscripts author's part. :-) -- Lamar Owen WGCR Internet Radio 1 Peter 4:11 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
[EMAIL PROTECTED] (Nathan Myers) writes: On Linux, /usr/src/linux/include is meaningless for anything in userland; it's meant only for building the kernel and kernel modules. That Red Hat tends to expose it to user-level builds is a long-standing bug in Red Hat's distribution 1) it isn't this way anyore 2) this was so for most distributions for a loong time, not a "Red Hat" bug. in violation of the File Hierarchy Standard as well as explicit instructions from Linus crew and from the maintainer of the C library. Which obviously hasn't always been the case - the FHS isn't exactly old. Things have changed since then, we have followed. -- Trond Eivind Glomsrd Red Hat, Inc. ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
RE: [HACKERS] How to shoot yourself in the foot: kill -9 postmaster
-Original Message- From: Tom Lane [mailto:[EMAIL PROTECTED]] The interlock has to be tightly tied to the PGDATA directory, because what we're trying to protect is the files in and under that directory. It seems that something based on file(s) in that directory is the way to go. The best idea I've seen so far is Hiroshi's idea of having all the backends hold fcntl locks on the same file (probably postmaster.pid would do fine). Then the new postmaster can test whether any backends are still alive by trying to lock the old postmaster.pid file. Unfortunately, I read in the fcntl man page: Locks are not inherited by a child process in a fork(2) system call. Yes flock() works well here but fcntl() doesn't. This makes the idea much less attractive than I originally thought: a new backend would not automatically inherit a lock on the postmaster.pid file from the postmaster, but would have to open/lock it for itself. That means there's a window where the new backend exists but would be invisible to a hypothetical new postmaster. We could work around this with the following, very ugly protocol: 1. Postmaster normally maintains fcntl read lock on its postmaster.pid file. Each spawned backend immediately opens and read-locks postmaster.pid, too, and holds that file open until it dies. (Thus wasting a kernel FD per backend, which is one of the less attractive things about this.) If the backend is unable to obtain read lock on postmaster.pid, then it complains and dies. We must use read locks here so that all these processes can hold them separately. 2. If a newly started postmaster sees a pre-existing postmaster.pid file, it tries to obtain a *write* lock on that file. If it fails, conclude that an old postmaster or backend is still alive; complain and quit. If it succeeds, sit for say 1 second before deleting the file and creating a new one. (The delay here is to allow any just-started old backends to fail to acquire read lock and quit. A possible objection is that we have no way to guarantee 1 second is enough, though it ought to be plenty if the lock acquisition is just after the fork.) I have another idea. My main point is to not remove the existent pidfile. For example 1) A newly started postmaster tries to obtain a write lock on the first byte of the pidfile. If it fails the postmaster quit. 2) The postmaster tries to obtain a write lock on the second byte of the pidfile. If it fails the postmaster quit. 3) The postmaster releases the lock of 2). 4) Each backend obtains a read-lock on the second byte of the pidfile. Regards, Hiroshi Inoue ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
RE: [HACKERS] Proposed WAL changes
-Original Message- From: Zeugswetter Andreas SB Remove archdir from pg_control; it ought to be a GUC parameter, not a special case (not that it's implemented yet anyway). Is archdir really a GUC parameter ? Why shouldn't it be? I see nothing wrong with changing it on-the-fly. Yes, I think this is a good change, like all others except XID assignment :-) Could GUC parameters be changed permanently e.g. by SET command ? For example, 1) start postmaster 2) set archdir to '' 3) shutdown postmaster Does PostgreSQL remember the archdir parameter ? Regards, Hiroshi Inoue ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] Proposed WAL changes
"Hiroshi Inoue" [EMAIL PROTECTED] writes: Could GUC parameters be changed permanently e.g. by SET command ? That's what postgresql.conf is for ... regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
[HACKERS] I'm back
Hi! Snow in New York - I'm arrived only today. Reading mail... Vadim ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] Re: Uh, this is *not* a 64-bit CRC ...
This isn't a 64-bit CRC. It's two independent 32-bit CRCs, one done on just the odd-numbered bytes and one on just the even-numbered bytes of the datastream. That's hardly any stronger than a single 32-bit CRC; I believe that the longer data the more chance to get same CRC/hash for different data sets (if data length CRC/hash length). Or am I wrong? Having no crc64 implementation (see below) I decided to use 2 CRC32 instead of one - it looked better, without any additional cost (record header is 8 byte aligned anyway, on, mmm, most platform). it's certainly not what I thought we had agreed to implement. I've asked if anyone can send crc64 impl to me and got only one from Nathan Myers. Unfortunately, SWISS-PROT impl assumes that long long is 8 bytes - is it portable? Vadim ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Proposed WAL changes
Tom Lane wrote: "Hiroshi Inoue" [EMAIL PROTECTED] writes: Could GUC parameters be changed permanently e.g. by SET command ? That's what postgresql.conf is for ... Do I have to send SIGHUP after changing postgresql.conf ? Regards, Hiroshi Inoue ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
AW: [HACKERS] Proposed WAL changes
Could GUC parameters be changed permanently e.g. by SET command ? For example, 1) start postmaster 2) set archdir to '' 3) shutdown postmaster I thought the intended way to change a GUC parameter permanently was to edit data/postgresql.conf . No ? Andreas ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: AW: [HACKERS] Proposed WAL changes
Zeugswetter Andreas SB wrote: Could GUC parameters be changed permanently e.g. by SET command ? For example, 1) start postmaster 2) set archdir to '' 3) shutdown postmaster I thought the intended way to change a GUC parameter permanently was to edit data/postgresql.conf . No ? What I've thought is to implement a new command to change archdir under WAL's control. If it's different from Vadim's plan I don't object. Regards, Hiroshi Inoue ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://www.postgresql.org/search.mpl
