[HACKERS] Two-phase commit

2004-02-07 Thread Heikki Linnakangas
I've been very slowly continuing my work on two-phase commits for a couple months now, and I now have my original patch updated so that it applies to the current CVS tip, with some improvements. The patch introduces three new commands, PREPCOMMIT, COMMITPREPARED and ABORTPREPARED. To start a 2PC

Re: [HACKERS] Two-phase commit

2004-02-15 Thread Heikki Linnakangas
On Sun, 8 Feb 2004, Jeroen T. Vermeulen wrote: On Wed, Feb 04, 2004 at 10:22:16PM +0200, Heikki Linnakangas wrote: There is a system view pg_prepared_xacts that gives you all transactions that are in prepared state waiting for COMMITPREPARED or ABORTPREPARED. Great to hear that you've

Re: [HACKERS] Two-phase commit

2004-02-28 Thread Heikki Linnakangas
On Sat, 7 Feb 2004, Bruce Momjian wrote: Please have a look and comment, the patches can be found here: http://www.iki.fi/hlinnaka/pgsql/ What is the schedule for 7.5? Any chance of getting this in? 7.5 is certainly possible. We are months away from beta on 7.5 and I would like ot see

Re: [HACKERS] Two-phase commit

2004-03-23 Thread Heikki Linnakangas
I have again updated my two-phase commit patches. Only minor modifications. I haven't received any comments and there hasn't been any discussion on the implementation, I suppose that nobody has given it a try. :( There is still some rough edges, but I think it's good enough as a first cut. I

Re: [HACKERS] Nested transaction proposal - take N (N 2)

2004-03-25 Thread Heikki Linnakangas
I haven't seen your patch yet, but the proposal looks good to me. On Tue, 23 Mar 2004, Alvaro Herrera wrote: Let the currently unused fourth state in pg_clog indicate a committed subtransaction. In pg_clog there are two bits per transaction, commit and abort, with the following

Re: [HACKERS] PITR, nested transactions, tablespaces, 2-phase commit: Status

2004-04-23 Thread Heikki Linnakangas
On Sat, 17 Apr 2004, Bruce Momjian wrote: Would folks report on the current status of these projects: o nested transactions (Alvaro Herrera) o tablespaces (Gavin Sherry) o PITR (Simon Riggs) o 2-phase commit (Heikki Linnakangas) I've been very busy with at work

Re: [HACKERS] Call for 7.5 feature completion

2004-05-18 Thread Heikki Linnakangas
On Mon, 17 May 2004, Alvaro Herrera Munoz wrote: On Mon, May 17, 2004 at 04:55:50PM +0200, Zeugswetter Andreas SB SD wrote: Can we try to do the 2PC patch now instead of waiting for subtransactions ? The last post from Heikki I read said that he discovered some serious problems with his

[HACKERS] XactIsoLevel handling

2004-05-18 Thread Heikki Linnakangas
In tcop/utility.c, the isolation level is set with a call like: SetPGVariable(transaction_isolation, makeList(item-arg), false) when a BEGIN SERIALIZABLE etc. call is made. Why is the isLocal-parameter false? Couldn't it be true as well? It works as it is, since the XactIsoLevel variable is

Re: [HACKERS] Delaying the planning of unnamed statements until Bind

2004-05-30 Thread Heikki Linnakangas
On Sat, 22 May 2004, Greg Stark wrote: Tom Lane [EMAIL PROTECTED] writes: I think it's wishful thinking to assume that picking an indexscan is the right thing when we don't know any better. If we don't know any better then any solution is going to be wishful thinking. It's kind of like a

Re: [HACKERS] Win32, PITR, nested transactions, tablespaces

2004-06-01 Thread Heikki Linnakangas
On Thu, 27 May 2004, Christopher Browne wrote: [EMAIL PROTECTED] (Bruce Momjian) wrote: Win32 has 98% of its code in CVS, so it will make it Tablespaces - Christopher says it is ready, and has run tests PITR - some functionality might be in 7.5, but we aren't sure Nested

[HACKERS] File leak?

2004-06-12 Thread Heikki Linnakangas
template1=# BEGIN; BEGIN template1=# CREATE TABLE foobar (foo char(10)); CREATE TABLE template1=# select relname, relfilenode from pg_class where relname='foobar'; relname | relfilenode -+- foobar | 66372 (1 row) killall -9 postmaster ll data/base/1/ -rw--- 1

Re: [HACKERS] File leak?

2004-06-13 Thread Heikki Linnakangas
On Sat, 12 Jun 2004, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: I wonder if we could clean up those lost files on database recovery or vacuum. There is a TODO for this, but it seems exceedingly low priority to me. Are you sure? I read through the TODO list but couldn't

Re: [HACKERS] File leak?

2004-06-14 Thread Heikki Linnakangas
On Sun, 13 Jun 2004, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: (viz, log at the instant of file creation, and the replayer would have to keep track of whether it sees the creating transaction commit and delete the file if not). I don't see how we could WAL log it because

Re: [HACKERS] Contrib -- PostgreSQL shared variables

2004-08-28 Thread Heikki Linnakangas
On Sat, 28 Aug 2004 [EMAIL PROTECTED] wrote: I don't see how this is different from CREATE TABLE shared_variables (name VARCHAR PRIMARY KEY, value VARCHAR) and inserting/updating/deleting/selecting from that. Perhaps these are per-session shared variables? IN which case, what is the utility if

Re: [HACKERS] Indexed views?

2004-09-12 Thread Heikki Linnakangas
On Sat, 11 Sep 2004, Tiago Wright wrote: IMHO, it is worth duplicating the mvcc data to all index entries. To summarize what I understand from this discussion, with the current method: a1 - Index seeks must return invisible tuples because mvcc data is not found in the index. These tuples are

Re: [HACKERS] CVS fixed ...

2004-10-11 Thread Heikki Linnakangas
On Sun, 10 Oct 2004, Marc G. Fournier wrote: pgsql-server renamed to pgsql contrib/earthdistance moved back into pgsql/contrib/earthdistance modules file updated for changes checkout seems to work fine for me ... let me know if there are any problems cvs update stopped working for me, I guess I'll

Re: [HACKERS] CVS fixed ...

2004-10-12 Thread Heikki Linnakangas
On Mon, 11 Oct 2004, Marc G. Fournier wrote: Also, the link to cvsweb interface at http://developer.postgresql.org/ broke. How so? I just wwent to the link and everything appears fine to me ... There's a link to PostgreSQL Server CVS web interface in the bottom of that page, which still points

Re: [HACKERS] Two-phase commit

2004-10-12 Thread Heikki Linnakangas
On Thu, 7 Oct 2004, Zeugswetter Andreas DAZ SD wrote: Note that this is only really necessary because of Heikki's choice to make the API work in terms of a user-assigned GID. This was not an arbitrary choice, but is required by most/all TX managers :-( I agree that it would be cleaner for the tx

Re: [HACKERS] Two-phase commit

2004-10-12 Thread Heikki Linnakangas
On Thu, 7 Oct 2004, Oliver Jowett wrote: Probably the next question is, do we want a database-side timeout on how long prepared txns can stay alive before being summarily rolled back? That sounds very dangerous to me. You could end up breaking global atomicity if some other resource in the

[HACKERS] Two-phase commit patch updated

2004-10-12 Thread Heikki Linnakangas
I brought the 2PC patch up to date: http://www.hut.fi/~hlinnaka/pgsql/ There's no new functionality, I just fixed all the bit rot so that it applies to the current CVS tip. - Heikki ---(end of broadcast)--- TIP 2: you can get off all lists at once

Re: [HACKERS] Two-phase commit

2004-10-12 Thread Heikki Linnakangas
On Wed, 6 Oct 2004, Tom Lane wrote: Quite some time ago, Heikki Linnakangas [EMAIL PROTECTED] wrote: I haven't received any comments and there hasn't been any discussion on the implementation, I suppose that nobody has given it a try. :( I finally got around to taking a close look

[HACKERS] Two-phase commit security restrictions

2004-10-13 Thread Heikki Linnakangas
What kind of security restrictions do we want for prepared transactions? Who has the right to finish a transaction that was started by user A? At least the original user, I suppose, but who else? Under what account is the transaction manager typically going to run? A separate TM account

Re: [HACKERS] Two-phase commit security restrictions

2004-10-13 Thread Heikki Linnakangas
On Wed, 13 Oct 2004, Peter Eisentraut wrote: Heikki Linnakangas wrote: What kind of security restrictions do we want for prepared transactions? Who has the right to finish a transaction that was started by user A? At least the original user, I suppose, but who else? Do we not require transaction

Re: [HACKERS] cvs problem

2004-10-22 Thread Heikki Linnakangas
On Fri, 22 Oct 2004, Dennis Bjorklund wrote: I'm having problems with cvs: cvs diff: failed to create lock directory for `/cvsroot/pgsql-server' (/cvsroot/pgsql-server/#cvs.lock): No such file or directory cvs diff: failed to obtain dir lock in repository `/cvsroot/pgsql-server' cvs [diff

[HACKERS] fsync, ext2 on Linux

2004-10-31 Thread Heikki Linnakangas
The Linux fsync man page says: It does not necessarily ensure that the entry in the directory containing the file has also reached disk. For that an explicit fsync on the file descriptor of the directory is also needed. AFAIK, we don't care about it at the moment. The actual behaviour depends

Re: [HACKERS] fsync, ext2 on Linux

2004-10-31 Thread Heikki Linnakangas
On Mon, 1 Nov 2004, Oliver Jowett wrote: Heikki Linnakangas wrote: The Linux fsync man page says: It does not necessarily ensure that the entry in the directory containing the file has also reached disk. For that an explicit fsync on the file descriptor of the directory is also needed. AFAIK

Re: [HACKERS] fsync, ext2 on Linux

2004-10-31 Thread Heikki Linnakangas
On Sun, 31 Oct 2004, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: The Linux [ext2] fsync man page says: It does not necessarily ensure that the entry in the directory containing the file has also reached disk. For that an explicit fsync on the file descriptor of the directory

Re: [HACKERS] [PATCHES] CVS should die

2004-11-05 Thread Heikki Linnakangas
On Fri, 5 Nov 2004, Gaetano Mendola wrote: Peter Eisentraut wrote: I'm certainly open to considering subversion, although I have a certain traumatic experience with it that may or may not be related to the BDB backend that it uses. I think for a start it would be nice if pgfoundry could

Re: [HACKERS] [PATCHES] CVS should die

2004-11-05 Thread Heikki Linnakangas
On Fri, 5 Nov 2004, Gaetano Mendola wrote: Heikki Linnakangas wrote: FWIW, I think Peter's idea of offering Subversion as an alternative in pgfoundry is very good. Mmm, do you mean createing periodically snapshot? Yes this could be a good idea. No, I mean that each project could choose to use

Re: [HACKERS] [PATCHES] CVS should die

2004-11-05 Thread Heikki Linnakangas
On Fri, 5 Nov 2004, Travis P wrote: Heikki Linnakangas wrote: Interestingly, the subversion repository is 585MB, and the CVS repository is only 260MB, BDB or FSFS back-end? FSFS seems to require less space. (The BDB backend tends to pre-allocate space though, so maybe there was a big jump

Re: [HACKERS] HOOKS for Synchronous Replication

2005-06-20 Thread Heikki Linnakangas
On Mon, 20 Jun 2005, Alvaro Herrera wrote: On Mon, Jun 20, 2005 at 11:03:45AM +0100, Alfranio Correia Junior wrote: Alfranio, The implementation of a set of hooks for efficient synchronous replication without extensive patching of Postgresql source is now available at:

Re: [HACKERS] [PATCHES] Cleaning up unreferenced table files

2005-06-27 Thread Heikki Linnakangas
No, not for now. Maybe for 8.2. And maybe as a contrib tool at first after all. - Heikki On Mon, 27 Jun 2005, Bruce Momjian wrote: Heikki, do you have any interest in completing your file checking patch for inclusion in 8.1 by adding tablespace information and other fixes as requested by

Re: [HACKERS] 2PC transaction id

2005-07-01 Thread Heikki Linnakangas
On Fri, 1 Jul 2005, Oliver Jowett wrote: Ok, so how do we get XA working when a single global transaction involves two databases on the same cluster? The scenario is: - there are two independent resource managers participating in a single global transaction - each resource manager has a

Re: [HACKERS] 2PC transaction id

2005-07-01 Thread Heikki Linnakangas
On Fri, 1 Jul 2005, Oliver Jowett wrote: Heikki Linnakangas wrote: branch id: Branch Identifier. Every RM involved in the global transaction is given a *different* branch id. Hm, I am confused then -- the XA spec definitely talks about enlisting multiple RMs in a single transaction branch

Re: [HACKERS] 2PC transaction id

2005-07-01 Thread Heikki Linnakangas
On Fri, 1 Jul 2005, Oliver Jowett wrote: What I'm confused about is, for example, 3.3.1 in the DTP:XA spec: 3.3.1 Registration of Resource Managers Normally, a TM involves all associated RMs in a transaction branch. (The TMs set of RM switches, described in Section 4.3 on page 21 tells the

Re: [HACKERS] 2PC transaction id

2005-07-01 Thread Heikki Linnakangas
On Fri, 1 Jul 2005, Oliver Jowett wrote: PS: noticed in passing: psql's help doesn't seem to know about the 2PC command syntax yet. True. Should we add support for it? 2PC is not something you normally do interactively... - Heikki ---(end of

Re: [HACKERS] 2PC transaction id

2005-07-03 Thread Heikki Linnakangas
On Sat, 2 Jul 2005, Oliver Jowett wrote: Sorry to keep beating on this, but I still don't see where the spec says that you must have only one RM per transaction branch. Sure, it's important to get this right. 2.2.6 says: 2.2.6 Transaction Branches A global transaction has one or more

Re: [HACKERS] Checkpoint cost, looks like it is WAL/CRC

2005-07-08 Thread Heikki Linnakangas
On Thu, 7 Jul 2005, Tom Lane wrote: We still don't know enough about the situation to know what a solution might look like. Is the slowdown Josh is seeing due to the extra CPU cost of the CRCs, or the extra I/O cost, or excessive locking of the WAL-related data structures while we do this

Re: [HACKERS] TODO questions

2005-08-25 Thread Heikki Linnakangas
On Thu, 25 Aug 2005, Tom Lane wrote: Ron Mayer [EMAIL PROTECTED] writes: The most unambiguous behavior would be to not have commented out values in the config file at all. Yeah, Robert Treat suggested that upthread, and I think it's been pushed by others too. The only argument I can see

Re: [HACKERS] Call for 7.5 feature completion

2005-08-26 Thread Heikki Linnakangas
On Thu, 25 Aug 2005, Alvaro Herrera wrote: Or, slightly different, what are people's most wanted features? Since you asked: * concurrent, partial vacuum that would for example only scan pages that happen to be in memory * index-only scans * database assertions * lightwight PITR that

Re: [HACKERS] 8.1beta, SunOS and shmget

2005-08-29 Thread Heikki Linnakangas
On Mon, 29 Aug 2005, Tom Lane wrote: Sergey E. Koposov [EMAIL PROTECTED] writes: So, are the shared memory requirements increased for 8.1 ? Yes; mostly from 2PC support I think. Try reducing max_prepared_transactions. (We might want to debate whether the default setting should be smaller

Re: [HACKERS] 8.1beta, SunOS and shmget

2005-08-29 Thread Heikki Linnakangas
On Mon, 29 Aug 2005, Tom Lane wrote: Sergey E. Koposov [EMAIL PROTECTED] writes: Yes, the decreasing of max_prepared_transaction helped (after some testing, I've found that the max_prepared_transactions=3 max_connections=10 shared_buffers=20 was well enough to fit 1mb of shared memory) 20

Re: [HACKERS] fixing LISTEN/NOTIFY

2005-10-06 Thread Heikki Linnakangas
First of all, I'd like to give a name to the thing that a backend listens to. The code talks about listening to a relation, but as the comments there point out, the name doesn't actually identify a relation. I'm going to call it a topic from now on. I'd like to point out that we don't really

Materialized views (Was Re: [HACKERS] Improving count(*))

2005-11-19 Thread Heikki Linnakangas
On Fri, 18 Nov 2005, Josh Berkus wrote: Alvaro, I guess there must be a query-rewriting mechanism for implementing materialized views.  With that in place we may be able to implement this other thing ...  Is anybody working on materialized views? I have a bundle of academic code designed to

Re: Materialized views (Was Re: [HACKERS] Improving count(*))

2005-11-20 Thread Heikki Linnakangas
On Sat, 19 Nov 2005, Josh Berkus wrote: Could you post it to the list? I'd be interested to take a look, though I'm afraid don't have the time to work on it. Yeah, I should put it up on pgFoundry. I'm not sure exactly where, though -- I don't want to launch a new project if it's not going to

Re: Materialized views (Was Re: [HACKERS] Improving count(*))

2005-11-20 Thread Heikki Linnakangas
On Sat, 19 Nov 2005, Nicolas Barbier wrote: You might want to take a look at the pages that I set up to track the progress on my master's thesis: url:http://www.nicolas.barbier.easynet.be/itsme/thesis/ especially the literature page:

[HACKERS] Data directory on read-only media

2005-11-20 Thread Heikki Linnakangas
Hi, I've been thinking about running postgres from read-only media. It's handy for creating demo CDs etc. I hacked together a patch that allows you to run Postgres without write permissions to the data directory. Some changes are needed: 1. Force all transactions to be read-only, using the

Re: [HACKERS] Data directory on read-only media

2005-11-21 Thread Heikki Linnakangas
On Mon, 21 Nov 2005, Peter Eisentraut wrote: Heikki Linnakangas wrote: I've been thinking about running postgres from read-only media. It's handy for creating demo CDs etc. I think that a read-only installation of PostgreSQL would be a very poor demonstration of its capabilities. Better put

Re: [HACKERS] Data directory on read-only media

2005-11-21 Thread Heikki Linnakangas
On Sun, 20 Nov 2005, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: 5. Don't try to write buffers with commit hint modifications. Just discard them. The performance costs of that alone are astonishing (ie, repeated verifications of commit status). You'd vacuum first to avoid

Re: [HACKERS] [pgsql-hackers] Daily digest v1.5568 (24 messages)

2005-11-21 Thread Heikki Linnakangas
On Mon, 21 Nov 2005, Marc Munro wrote: I wonder if this idea might be taken a little further, to allow read-only tablespaces? This would allow old partitions in very large databases to be kept on read-only media, and would allow normal backups to ignore this unchanging set of data. I guess

Re: [HACKERS] Shared row locking

2004-12-19 Thread Heikki Linnakangas
On Sun, 19 Dec 2004, Alvaro Herrera wrote: On Sun, Dec 19, 2004 at 09:52:01AM +, Simon Riggs wrote: Simon, In similar circumstances, DB2 uses these techniques: - when locktable X % full, then escalate locks to full table locks: both locktable memory and threshold% are instance parameters This

Re: [HACKERS] Shared row locking

2004-12-19 Thread Heikki Linnakangas
On Sun, 19 Dec 2004, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: On Sun, 19 Dec 2004, Alvaro Herrera wrote: This is not useful at all, because the objective of this exercise is to downgrade locks, from exclusive row locking (SELECT ... FOR UPDATE) to shared row locking. Actually

Re: [HACKERS] WAL logging of heap_mark4update

2005-01-16 Thread Heikki Linnakangas
On Sat, 15 Jan 2005, Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Hackers, In access/heap/heapam.c, in heap_mark4update(), there's a comment that states /* * XLOG stuff: no logging is required as long as we have no * savepoints. For savepoints private log

[HACKERS] Two-phase commit for 8.1

2005-01-19 Thread Heikki Linnakangas
Hi, Now that we got 8.0 out of the door, I'm submitting my two-phase commit patch again for discussion. http://www.hut.fi/~hlinnaka/pgsql/ Do we want it in 8.1, if we want a short development cycle? It needs a new pg_twophase subdirectory, and it introduces a new system view, so I guess it

Re: [HACKERS] Two-phase commit for 8.1

2005-01-21 Thread Heikki Linnakangas
On Wed, 19 Jan 2005, Tom Lane wrote: Marc G. Fournier [EMAIL PROTECTED] writes: If the patch is ready to be committed early in the cycle, I'd say most definitely ... just depends on how late in the cycle its ready ... My recollection is that it's quite far from being complete. I had hoped to

Re: [HACKERS] Two-phase commit for 8.1

2005-01-23 Thread Heikki Linnakangas
On Sun, 23 Jan 2005, Hans-Jürgen Schönig wrote: Heikki, What is still missing to complete the 2PC patch?. Here's my TODO on things that need to be done: * large objects * guc variables * notify/listen Large objects and notify/listen should be quite straightforward. GUC variables need

Re: [HACKERS] Two-phase commit for 8.1

2005-01-23 Thread Heikki Linnakangas
On Sun, 23 Jan 2005, Alvaro Herrera wrote: On Sun, Jan 23, 2005 at 01:37:30PM +0200, Heikki Linnakangas wrote: As the patch gets more attention, I'm sure more issues will come up. I see the changes to the lock manager are huge. Can you explain what's the idea behind those? Do you release

Re: [HACKERS] Cost of XLogInsert CRC calculations

2005-03-06 Thread Heikki Linnakangas
On Sun, 6 Mar 2005, Tom Lane wrote: I suppose that the bulk of the CPU cycles being attributed to XLogInsert are going into the inlined CRC calculations. Maybe we need to think twice about the cost/benefit ratio of using 64-bit CRCs to protect xlog records that are often only a few dozen bytes.

Re: [HACKERS] Recording vacuum/analyze/dump times

2005-03-07 Thread Heikki Linnakangas
On Mon, 7 Mar 2005, Jim Buttafuoco wrote: Its there a reason postgresql doesn't record vacuum/analyze and dump times in pg_class (or another table). This seems like it would be a very helpful feature. for pg_dump I would add an option --record=YES|NO for vacuum and analyze I would add a

Re: [HACKERS] We are not following the spec for HAVING without GROUP

2005-03-11 Thread Heikki Linnakangas
On Thu, 10 Mar 2005, Tom Lane wrote: Would those of you with access to other DBMSes try this: create table tab (col integer); select 1 from tab having 1=0; select 1 from tab having 1=1; insert into tab values(1); insert into tab values(2); select 1 from tab having 1=0; select 1 from tab having

Re: [HACKERS] Two phase commit

2005-04-06 Thread Heikki Linnakangas
On Tue, 5 Apr 2005, Alvaro Herrera wrote: What happenned to your two phase commit patch? Are you still working on it? Have you got something done from the last time we heard from you? I've kept it up-to-date by doing cvs update every now and then and fixing possible conflicts. It would be nice

Re: [HACKERS] Urgent

2005-04-18 Thread Heikki Linnakangas
On Mon, 18 Apr 2005, ElayaRaja S wrote: My postgresql was running successfully in my Redhat LInux 9. After 1 day when i try to login to the folder /usr/local/pgsql/data to update the ip address i am getting as [EMAIL PROTECTED] data]# ls Segmentation fault I am unable to see the files under

Re: [HACKERS] Feature freeze date for 8.1

2005-05-02 Thread Heikki Linnakangas
On Mon, 2 May 2005, Hannu Krosing wrote: Well, I've had problems with clients which resolve DB timeouts by closing the current connection and establish a new one. If it is actual DB timeout, then it all is ok, the server soon notices that the client connection is closed and kills itself. Problems

Re: [HACKERS] Feature freeze date for 8.1

2005-05-03 Thread Heikki Linnakangas
On Tue, 3 May 2005, Tom Lane wrote: I am a tad worried about the possibility that if the client does nothing for long enough, the TCP output buffer will fill causing the backend to block at send(). A permanently blocked backend is bad news from a performance point of view (it degrades the sinval

[HACKERS] Bogus assertion in multixact.c?

2005-05-03 Thread Heikki Linnakangas
There's an assertion in multixact.c, MultiXactIdExpand function, line 273: Assert(!TransactionIdEquals(multi, xid)); where multi is a MultiXactId and xid is a TransactionId. Isn't this bogus? If I understand the code correctly, multixactids and regular xids live in completely separate id

Re: [HACKERS] Bogus assertion in multixact.c?

2005-05-03 Thread Heikki Linnakangas
Never mind. multi is in effect a TransactionId in that code path, and thus the assertion makes sense. Sorry for the noise. On Tue, 3 May 2005, Heikki Linnakangas wrote: There's an assertion in multixact.c, MultiXactIdExpand function, line 273: Assert(!TransactionIdEquals(multi, xid

Re: [HACKERS] [PATCHES] Cleaning up unreferenced table files

2005-05-07 Thread Heikki Linnakangas
Maybe we should take a different approach to the problem: 1. Create new file with an extension to mark that it's not yet committed (eg. 1234.notcommitted) 2. ... 3. Take CheckpointStartLock 4. Write commit record to WAL, with list of created files. 5. rename created file (1234.notcommitted -

Re: [HACKERS] [PATCHES] Cleaning up unreferenced table files

2005-05-08 Thread Heikki Linnakangas
On Sat, 7 May 2005, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Maybe we should take a different approach to the problem: 1. Create new file with an extension to mark that it's not yet committed (eg. 1234.notcommitted) This is pushing the problem into the wrong place, viz

Re: [HACKERS] Please clarify

2005-05-10 Thread Heikki Linnakangas
On Tue, 10 May 2005 [EMAIL PROTECTED] wrote: I need to check for the existence of a user defined view named 'audit_vw' and if exists, then i need to delete the same. Please help me to solve the issue. If you don't need to do anything else in the transaction, you could just issue DROP VIEW

Re: [HACKERS] [PATCHES] Cleaning up unreferenced table files

2005-05-10 Thread Heikki Linnakangas
On Sun, 8 May 2005, Tom Lane wrote: While your original patch is buggy, it's at least fixable and has localized, limited impact. I don't think these schemes are safe at all --- they put a great deal more weight on the semantics of the filesystem than I care to do. I'm going to try this some more,

Re: [HACKERS] [PATCHES] Cleaning up unreferenced table files

2005-05-10 Thread Heikki Linnakangas
On Tue, 10 May 2005, Bruce Momjian wrote: The current code is nice and localized and doesn't add any burden on our existing code, which is already complicated enough. I think we either fix checkfiles.c, or we remove it and decide it isn't worth checking for unrefrenced files. Let's pull the patch

Re: [HACKERS] Two-phase commit issues

2005-05-19 Thread Heikki Linnakangas
On Wed, 18 May 2005, Tom Lane wrote: * The major missing issue that I've come across so far is that subtransaction and multixact state isn't preserved across a crash. Assuming that we want to store only top-level XIDs in the shared-memory list of prepared XIDs (which I think is important), it is

Re: [HACKERS] Two-phase commit issues

2005-05-21 Thread Heikki Linnakangas
On Thu, 19 May 2005, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: * I'm inclined to think that the gid identifiers for prepared transactions ought to be SQL identifiers (names), not string literals. Was there a particular reason for making them strings? Sure. No Reason

Re: [HACKERS] Quick-and-dirty compression for WAL backup blocks

2005-06-06 Thread Heikki Linnakangas
On Mon, 6 Jun 2005, Tom Lane wrote: Junji TERAMOTO [EMAIL PROTECTED] writes: In whole buffer log, there is a page header that includes offset of hole (lower and upper). If we use that information, we don't need any overhead, do we? No, because the WAL code cannot assume that all pages

Re: [HACKERS] Quick-and-dirty compression for WAL backup blocks

2005-06-06 Thread Heikki Linnakangas
On Mon, 6 Jun 2005, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Vacuum doesn't zero out the free space between lower and upper, It does now ;-) Oh :). Does it affect vacuum performance? How about adding a flag to XLogRecData to indicate if the space between pd_lower

Re: [HACKERS] Two-phase commit issues

2005-06-11 Thread Heikki Linnakangas
On Tue, 7 Jun 2005, Alvaro Herrera wrote: On Sat, May 21, 2005 at 06:57:24PM +0300, Heikki Linnakangas wrote: Heikki, I took a closer look at the JTA spec and saw that the Xid, which is translated to a gid in the jdbc driver, consists of a format identifier (32-bit int), a branch qualifier

Re: [HACKERS] Two-phase commit issues

2005-06-11 Thread Heikki Linnakangas
On Sat, 11 Jun 2005, Jochem van Dieten wrote: The OSI CCR format, which appears to refer to ISO/IEC 9805-1. ISO/IEC 9805-1:1998 15-12-1998 Information technology - Open Systems Interconnection - Protocol for the Commitment, Concurrency and Recovery service element: Protocol specification This

Re: [DOCS] [HACKERS] Online backup vs Continuous backup

2005-12-26 Thread Heikki Linnakangas
On Mon, 26 Dec 2005, Bruce Momjian wrote: Joshua D. Drake wrote: I have never heard the term Continuous backup. Although I have heard online backup. The problem is that when I hear the term online backup I think Hot backup which is what we do with pg_dump. Yes, that is my problem too. I

Re: [HACKERS] Commands per transaction

2006-01-21 Thread Heikki Linnakangas
On Sat, 21 Jan 2006, Alvaro Herrera wrote: Rod Taylor wrote: Is there any way of bumping this limit or am I stuck breaking up the transaction? Wow, I never heard of anyone reaching the limit :-( Sorry, you are stuck (short of changing CommandId to 64 bits, which would bloat your tables

Re: [HACKERS] Locks

2006-02-11 Thread Heikki Linnakangas
On Sat, 11 Feb 2006, Alfranio Correia Junior wrote: Are there some sort of functions that I could use to know which process has an exclusive lock on relations, pages, tuples, transactions, etc... ? SELECT * FROM pg_locks; - Heikki ---(end of

[HACKERS] Dead Space Map

2006-02-27 Thread Heikki Linnakangas
Hi, The idea of using a so called dead space map to speed up vacuum has come up multiple times in this list in the last couple of years. I wrote an initial implementation of it to measure the performance impact it has on updates and on vacuum. Potential uses for a dead space map are: *

Re: [HACKERS] Dead Space Map

2006-02-27 Thread Heikki Linnakangas
On Mon, 27 Feb 2006, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Vacuum will need to be modified to use index lookups to find index tuples corresponding the dead heap tuples. Otherwise you have to scan through all the indexes anyway. This strikes me as a fairly bad idea

Re: [HACKERS] Dead Space Map

2006-02-28 Thread Heikki Linnakangas
On Mon, 27 Feb 2006, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: On Mon, 27 Feb 2006, Tom Lane wrote: This strikes me as a fairly bad idea, because it makes VACUUM dependent on correct functioning of user-written code --- consider a functional index involving a user-written

Re: [HACKERS] Dead Space Map

2006-02-28 Thread Heikki Linnakangas
On Tue, 28 Feb 2006, Bruce Momjian wrote: Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: Tom Lane wrote: MVCC goes out the window, eh? Not to mention transaction rollback ability? If the old row is not visible to any transactions, why would it not work? The old row is

[HACKERS] Index vacuum improvements

2006-03-29 Thread Heikki Linnakangas
Hi, As we know, index vacuum could be sped up significantly if it didn't have to lock every page in left to right direction because of the integrity issue described in nbtree/README. We could then scan the index in physical order, and AFAICS combine the btbulkdelete and btvacuumcleanup logic

Re: [HACKERS] Index vacuum improvements

2006-03-30 Thread Heikki Linnakangas
On Wed, 29 Mar 2006, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: 1. Instead of stopping on the first matching tuple, scan the whole index page for all matching entries at once. That loses the ability to reflect tuple deadness back into LP_DELETE flags, no? Which

Re: [HACKERS] Index vacuum improvements

2006-03-30 Thread Heikki Linnakangas
On Wed, 29 Mar 2006, Simon Riggs wrote: First off, we need some good timings that show this effect. I believe it, but we need some publicly discussable performance test cases to show the effect and then show how much we've improved upon it, repeatably. Yeah, a good vacuum benchmark would be

Re: [HACKERS] Constraint Exclusion + Joins?

2006-04-30 Thread Heikki Linnakangas
On Fri, 28 Apr 2006, Brandon Black wrote: I dug around in CVS to have a look for this, and I did eventually find it (well, I found the corresponding docs patch that removed the note about not working for joins). I see it's in MAIN but not in 8_1_STABLE. Does that mean it's headed for 8.2.x

Re: [HACKERS] LIKE, leading percent, bind parameters and indexes

2006-05-28 Thread Heikki Linnakangas
On Sat, 27 May 2006, Martijn van Oosterhout wrote: On Sat, May 27, 2006 at 10:57:05AM -0400, Tom Lane wrote: * Up to now, the only functions directly invoked by an index AM were members of index opclasses; and since opclasses can only be defined by superusers, there was at least some basis for

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Heikki Linnakangas
On Tue, 6 Jun 2006, Mark Woodward wrote: OK, here's my problem, I have a nature study where we have about 10 video cameras taking 15 frames per second. For each frame we make a few transactions on a PostgreSQL database. I would suggest doing all the inserts of one frame in one transaction.

Re: [HACKERS] 2-phase commit

2003-10-14 Thread Heikki Linnakangas
On Thu, 9 Oct 2003, Bruce Momjian wrote: Agreed. Let's get it into 7.5 and see it in action. If we need to adjust it, we can, but right now, we need something for distributed transactions, and this seems like the logical direction. I've started working on two-phase commits last week, and

Re: [HACKERS] 2-phase commit

2003-10-24 Thread Heikki Linnakangas
On Fri, 10 Oct 2003, Heikki Linnakangas wrote: On Thu, 9 Oct 2003, Bruce Momjian wrote: Agreed. Let's get it into 7.5 and see it in action. If we need to adjust it, we can, but right now, we need something for distributed transactions, and this seems like the logical direction. I've

Re: [HACKERS] AGREGATE FUNCTIONS

2006-06-06 Thread Heikki Linnakangas
On Tue, 6 Jun 2006, Roberto Rezende de Assis wrote: Hello, I would like to know where in the source-code of postgres is located the code of the aggregate functions min, max, avg. They're in src/backend/utils/adt/numeric.c I wish to develop more statistical aggregate functions, and I prefer

Re: [HACKERS] MultiXacts WAL

2006-06-17 Thread Heikki Linnakangas
On Sat, 17 Jun 2006, paolo romano wrote: May be this is needed to support savepoints/subtransactions? Or is it something else that i am missing? It's for two-phase commit. A prepared transaction can hold locks that need to be recovered. - Heikki ---(end of

Re: [HACKERS] MultiXacts WAL

2006-06-17 Thread Heikki Linnakangas
On Sat, 17 Jun 2006, paolo romano wrote: When a transaction enters (successfully) the prepared state it only retains its exclusive locks and releases any shared locks (i.e. multixacts)... or, at least, that's how it should be in principle according to serializiaton theory, i haven't yet

Re: [HACKERS] MultiXacts WAL

2006-06-17 Thread Heikki Linnakangas
On Sat, 17 Jun 2006, paolo romano wrote: * Reduced I/O Activity: during transaction processing: current workloads are typically dominated by reads (rather than updates)... and reads give rise to multixacts (if there are at least two transactions reading the same page or if an explicit lock

Re: [HACKERS] MultiXacts WAL

2006-06-17 Thread Heikki Linnakangas
On Sat, 17 Jun 2006, paolo romano wrote: The original point I was moving is if there were any concrete reason (which still I can't see) to require Multixacts recoverability (by means of logging). Concerning the prepare state of two phase commit, as I was pointing out in my previous post,

Re: [HACKERS] MultiXacts WAL

2006-06-18 Thread Heikki Linnakangas
On Sun, 18 Jun 2006, paolo romano wrote: Anyway, again in theory, if one wanted to minimize logging overhead for shared locks, one might adopt a different treatment for (i) regular shared locks (i.e. locks due to plain reads not requiring durability in case of 2PC) and (ii) shared locks held

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-22 Thread Heikki Linnakangas
On Thu, 22 Jun 2006, Jonah H. Harris wrote: On 6/22/06, Rod Taylor [EMAIL PROTECTED] wrote: If you INSERT into multiple partitions (by time -- say one per minute) and TRUNCATE periodically (30 minute old partitions for 30 minute expiry) it works much better. Expiring the session is quite fast

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-22 Thread Heikki Linnakangas
On Thu, 22 Jun 2006, Jim Nasby wrote: DB2 switched to MVCC in version 8. ... Um, no it didn't. - Heikki ---(end of broadcast)--- TIP 6: explain analyze is your friend

  1   2   3   4   5   6   7   8   9   10   >