Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-11 Thread Florian G. Pflug
On 11.04.10 20:47 , Robert Haas wrote: On Sun, Apr 11, 2010 at 10:26 AM, Heikki Linnakangas wrote: Robert Haas wrote: 2010/4/10 Andrew Dunstan: Heikki Linnakangas wrote: 1. Keep the materialized view up-to-date when the base tables change. This can be further divided into many steps, you ca

Re: [HACKERS] Project suggestion: benchmark utility for PostgreSQL

2007-03-17 Thread Florian G. Pflug
Mickael DELOISON wrote: For a programming language, as it would be for GSoC, it has to be realized in three month and I believe the utility has to be cross-platform (anyway I want it to be). So I think Java would be good. I am very used to Java and Swing programming. What do you think about that

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Florian G. Pflug
Gregory Stark wrote: "Andrew Dunstan" <[EMAIL PROTECTED]> writes: Yes, more or less. That's what made me think of it. OTOH, before we rush out and do it someone needs to show that it's a net win. I agree with Tom that making tuplestore faster would probably be a much better investment of time.

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Florian G. Pflug
Andrew Dunstan wrote: Florian G. Pflug wrote: Just a thought - I believe that there are portable user-space thread implementations that contain little or no machine-specific code. What if postgres used one of those to switch from the PL into the executor and back after, say, 1000 rows were

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Florian G. Pflug
Gregory Stark wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: Since context switching would occur only at two well-defined places (Some return_next_row function that PLs call when a SFR returns a row, and in the executor if no more previously returned rows from that S

Re: [HACKERS] modifying the tbale function

2007-03-20 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: Here is a paper about portable userspace threads that I just googled. http://www.gnu.org/software/pth/rse-pmt.ps I'm not impressed --- the list of unsolved problems is a bit too long. One that seems a showsto

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-21 Thread Florian G. Pflug
Pavan Deolasee wrote: On 3/21/07, Merlin Moncure <[EMAIL PROTECTED]> wrote: On 3/21/07, Pavan Deolasee <[EMAIL PROTECTED]> wrote: > It seems much simpler to me do something like this. But important > question is whether the restriction that CREATE INDEX can not > be run in a transaction block i

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-23 Thread Florian G. Pflug
Pavan Deolasee wrote: There is a slight hole in that SERIALIZABLE transactions won't be able to use any indexes they build during their transaction, since they may need to be able to see prior data, but I don't think anybody is going to complain about that restriction. Anyone? Oh, I did not se

Re: [HACKERS] tsearch_core for inclusion

2007-03-23 Thread Florian G. Pflug
Teodor Sigaev wrote: For given schema and server's locale, it's possible to have several FTS configurations, but the only one (with special flag enabled) could be used as default. Current (active) FTS configuration contains in GUC variable tsearch_conf_name. If it's not defined, then FTS config

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-23 Thread Florian G. Pflug
Pavan Deolasee wrote: On 3/23/07, Florian G. Pflug <[EMAIL PROTECTED]> wrote: Why exactly can't a SERIALIZABLE transaction use the index it created itself? If you add a pointer to the root of all HOT update chains where either the HEAD is alive, or some tuple is visible to the

Re: [HACKERS] tsearch_core for inclusion

2007-03-26 Thread Florian G. Pflug
Oleg Bartunov wrote: On Fri, 23 Mar 2007, Florian G. Pflug wrote: Teodor Sigaev wrote: For given schema and server's locale, it's possible to have several FTS configurations, but the only one (with special flag enabled) could be used as default. Current (active) FTS configuration c

Re: [HACKERS] tsearch_core for inclusion

2007-03-26 Thread Florian G. Pflug
Teodor Sigaev wrote: For given schema and server's locale, it's possible to have several FTS configurations, but the only one (with special flag enabled) could be used as default. Current (active) FTS configuration contains in GUC variable tsearch_conf_name. If it's not defined, then FTS config

Re: [HACKERS] Reduction in WAL for UPDATEs

2007-03-28 Thread Florian G. Pflug
Kenneth Marshall wrote: We use DSPAM as one of our anti-spam options. Its UPDATE pattern is to increment a spam counter or a not-spam counter while keeping the user and token information the same. This would benefit from this optimization. Currently we are forced to use MySQL with MyISM tables to

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-28 Thread Florian G. Pflug
Simon Riggs wrote: On Wed, 2007-03-28 at 22:24 +0530, Pavan Deolasee wrote: Just when I thought we have nailed down CREATE INDEX, I realized that there something more to worry. The problem is with the HOT-chains created by our own transaction which is creating the index. We thought it will be

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-28 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: Couldn't you store the creating transaction's xid in pg_index, and let other transaction check that against their snapshot like they would for any tuple's xmin or xmax? What snapshot? I keep havi

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-28 Thread Florian G. Pflug
Pavan Deolasee wrote: On 3/28/07, Tom Lane <[EMAIL PROTECTED]> wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: > Couldn't you store the creating transaction's xid in pg_index, and > let other transaction check that against their snapshot like the

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-29 Thread Florian G. Pflug
Pavan Deolasee wrote: In this specific context, this particular case is easy to handle because we are only concerned about the serializable transactions started before CREATE INDEX commits. If PREPARE can see the new index, it implies that the CI transaction is committed. So the transaction start

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-29 Thread Florian G. Pflug
Pavan Deolasee wrote: On 3/29/07, Florian G. Pflug <[EMAIL PROTECTED]> wrote: Yes, but the non-index plan PREPARE generated will be used until the end of the session, nut only until the end of the transaction. Frankly I don't know this works, but are you sure that the plan will be

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-29 Thread Florian G. Pflug
Tom Lane wrote: "Simon Riggs" <[EMAIL PROTECTED]> writes: ISTM that the run-another-transaction-afterwards idea is the only one that does everything I think we need. I really do wish we could put in a wait, like CIC, but I just think it will break existing programs. Actually, there's a showsto

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-29 Thread Florian G. Pflug
Simon Riggs wrote: On Thu, 2007-03-29 at 17:27 -0400, Tom Lane wrote: "Simon Riggs" <[EMAIL PROTECTED]> writes: ISTM that the run-another-transaction-afterwards idea is the only one that does everything I think we need. I really do wish we could put in a wait, like CIC, but I just think it will

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Florian G. Pflug
Tom Lane wrote: "Pavan Deolasee" <[EMAIL PROTECTED]> writes: How about storing the snapshot which we used during planning in CachedPlanSource, if at least one index was seen unusable because its CREATE INDEX transaction was seen as in-progress ? I'm getting tired of repeating this, but: the pl

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Florian G. Pflug
Tom Lane wrote: "Pavan Deolasee" <[EMAIL PROTECTED]> writes: What I am suggesting is to use ActiveSnapshot (actually Florian's idea) to decide whether the transaction that created index was still running when we started. Isn't it the case that some snapshot will be "active" when we plan ? I do

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: Tom Lane wrote: I do not think you can assume that the plan won't be used later with some older snapshot. So maybe we'd need to use the SerializableSnapshot created at the start of each transaction f

Re: [HACKERS] CREATE INDEX and HOT - revised design

2007-03-30 Thread Florian G. Pflug
Pavan Deolasee wrote: On 3/30/07, Florian G. Pflug <[EMAIL PROTECTED]> wrote: My idea was to store a list of xid's together with the cached plan that are assumed to be uncommitted accoring to the IndexSnapshot. The query is replanned if upon execution the IndexSnapshot assumes

Re: [HACKERS] Minor changes to Recovery related code

2007-03-30 Thread Florian G. Pflug
Simon Riggs wrote: On Fri, 2007-03-30 at 16:34 -0400, Tom Lane wrote: "Simon Riggs" <[EMAIL PROTECTED]> writes: 2. pg_stop_backup() should wait until all archive files are safely archived before returning Not sure I agree with that one. If it fails, you can't tell whether the action is done a

[HACKERS] Fate of pgsnmpd

2007-04-06 Thread Florian G. Pflug
Hi Does anyone know if pgsnmpd is still actively developed? The last version (0.1b1) is about 15 months old. greetings, Florian Pflug ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [HACKERS] "select ('{}'::text[])[1]" returns NULL -- is it correct?

2007-04-10 Thread Florian G. Pflug
Nikolay Samokhvalov wrote: On 4/10/07, Tom Lane <[EMAIL PROTECTED]> wrote: "Nikolay Samokhvalov" <[EMAIL PROTECTED]> writes: > I remember several cases when people (e.g. me :-) ) were spending some > time trying to find an error in some pl/pgsql function and the reason > lied in incorrect work w

Re: [HACKERS] [DOCS] uuid type not documented

2007-04-10 Thread Florian G. Pflug
Neil Conway wrote: On Tue, 2007-04-10 at 18:28 +0200, Peter Eisentraut wrote: The problem is that most of the standard methods are platform dependent, as they require MAC addresses or a "good" random source, for instance. http://archives.postgresql.org/pgsql-patches/2007-01/msg00392.php ISTM

[HACKERS] What tools do people use to hack on PostgreSQL?

2007-04-12 Thread Florian G. Pflug
Hi I'm very excited that my project for implementing read-only queries on PITR slaves was accepted for GSoC, and I'm now trying to work out what tools I'll use for that job. I'd like to be able to create some sort of branches and tags for my own work (only inside my local repository of course).

Re: [HACKERS] What tools do people use to hack on PostgreSQL?

2007-04-12 Thread Florian G. Pflug
Joshua D. Drake wrote: Alexey Klyukin wrote: Alvaro Herrera wrote: But if you have a checked out tree, does it work to do an update after the tree has been regenerated? As far as I know, the repo is generated completely every few hours, so it wouldn't surprise me that the checked out copy is n

[HACKERS] Build-Problem with pgc.c on OSX 10.4

2007-04-15 Thread Florian G. Pflug
Hi When I try to build CVS HEAD on OSX 10.4, compiling src/interfaces/ecpg/preproc/preproc.c fails with: In file included from preproc.y:6951: pgc.l:3:20: error: config.h: No such file or directory In file included from pgc.l:28, from preproc.y:6951: preproc.h:996: error: conflic

Re: [HACKERS] Build-Problem with pgc.c on OSX 10.4

2007-04-15 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: When I try to build CVS HEAD on OSX 10.4, compiling src/interfaces/ecpg/preproc/preproc.c fails with: ... If I delete pgc.c, it is rebuilt automatically, and then preproc.c compiles just fine. ... I'm using gcc

Re: [HACKERS] Build-Problem with pgc.c on OSX 10.4

2007-04-16 Thread Florian G. Pflug
Alvaro Herrera wrote: Ah, it seems the SVN repo just got its first user ;-) Congratulations. Ask Joshua to send you a Command Prompt tee shirt, maybe he is excited enough. I hope the fact that I use the SVN repo just to get the changes into git doesn't reduce my chances of getting that t-shirt

Re: [HACKERS] Build-Problem with pgc.c on OSX 10.4

2007-04-16 Thread Florian G. Pflug
Alvaro Herrera wrote: Florian G. Pflug wrote: Alvaro Herrera wrote: Ah, it seems the SVN repo just got its first user ;-) Congratulations. Ask Joshua to send you a Command Prompt tee shirt, maybe he is excited enough. I hope the fact that I use the SVN repo just to get the changes into git

Re: [HACKERS] What tools do people use to hack on PostgreSQL?

2007-04-16 Thread Florian G. Pflug
Joshua D. Drake wrote: http://projects.commandprompt.com/public/pgsql/browser or do the anonymous checkout with: svn co http://projects.commandprompt.com/public/pgsql/repo/ But if you have a checked out tree, does it work to do an update after the tree has been regenerated? As far as I know,

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-16 Thread Florian G. Pflug
Martin Langhoff wrote: Hi Florian, I am right now running an rsync of the Pg CVS repo to my work machine to get a git import underway. I'm rather keen on seeing your cool PITR Pg project go well and I have some git+cvs fu I can apply here (being one of the git-cvsimport maintainers) ;-) Cool -

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-16 Thread Florian G. Pflug
Aidan Van Dyk wrote: Martin Langhoff wrote: Well, now that more than one of us are working with git on PostgreSQL... I've had a repo conversion running for a while... I've only got it to what I consider "stable" last week: http://repo.or.cz/w/PostgreSQL.git git://repo.or.cz/PostgreSQL.g

Re: [HACKERS] Re: IDENTITY/GENERATED v36 Re: [PATCHES] Final version of IDENTITY/GENERATED patch

2007-04-16 Thread Florian G. Pflug
Zoltan Boszormenyi wrote: Tom Lane írta: Zoltan Boszormenyi <[EMAIL PROTECTED]> writes: Also, the current grammar is made to give a syntax error if you say "colname type GENERATED BY DEFAULT AS ( expr )". But it makes the grammar unbalanced, and gives me: bison -y -d gram.y conflicts: 2 shift/

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-16 Thread Florian G. Pflug
Tom Lane wrote: Alvaro Herrera <[EMAIL PROTECTED]> writes: These files are generated (from gram.y, pgc.l and preproc.y respectievly) and are not present in the CVS repo, though I think they have been at some point. It's strange that other generated files (that have also been in the repo in th

Re: [HACKERS] RESET command seems pretty disjointed now

2007-04-16 Thread Florian G. Pflug
Tom Lane wrote: Mark Kirkwood <[EMAIL PROTECTED]> writes: Tom Lane wrote: The current documentation for RESET exhibits a certain lack of, um, intellectual cohesiveness: Synopsis RESET configuration_parameter RESET ALL RESET { PLANS | SESSION | TEMP | TEMPORARY } Maybe DISCARD for the plans

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-17 Thread Florian G. Pflug
Martin Langhoff wrote: Aidan Van Dyk wrote: And remember the warning I gave that my conversion is *not* a direct CVS import - I intentionally *unexpand* all Keywords before stuffing them into GIT so that merging and branching can ignore all the Keyword conflicts... My import is unexpanding th

[HACKERS] [PATCH] A crash and subsequent recovery of the master can cause the slave to get out-of-sync

2007-04-19 Thread Florian G. Pflug
Hi I believe I have discovered the following problem in pgsql 8.2 and HEAD, concerning warm-standbys using WAL log shipping. The problem is that after a crash, the master might complete incomplete actions via rm_cleanup() - but since it won't wal-log those changes, the slave won't know about thi

Re: [HACKERS] [PATCH] A crash and subsequent recovery of themaster can cause the slave to get out-of-sync

2007-04-23 Thread Florian G. Pflug
Simon Riggs wrote: On Thu, 2007-04-19 at 22:37 +0200, Florian G. Pflug wrote: The problem is that after a crash, the master might complete incomplete actions via rm_cleanup() - but since it won't wal-log those changes, the slave won't know about this. This will at least prevent the c

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-30 Thread Florian G. Pflug
Martin Langhoff wrote: So - if you are committed to providing your gateway long term to Florian, I'm happy to drop my gateway in favour of yours. (Florian, before basing your code on either you should get a checkout of Aidan's and mine and check that the tips of the branches you are working on

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-30 Thread Florian G. Pflug
Aidan Van Dyk wrote: * Florian G. Pflug <[EMAIL PROTECTED]> [070430 08:58]: It seems as if git pulls all revisions of all files during the pull - which it shouldn't do as far as I understand things - it should only pull those objects referenced by some head, no? Git pulls full h

Re: [HACKERS] Heap page diagnostic functions

2007-05-01 Thread Florian G. Pflug
Zdenek Kotala wrote: I did not find "forensics" in translator and It mentions in Oxford vocabulary but explanation is not clear for me. I agree with Bruce It is not good name. What about short form of diagnostic "diag"? Doesn't forensics basically mean to find the cause of something *after* it

Re: [HACKERS] Concurrently updating an updatable view

2007-05-14 Thread Florian G. Pflug
Richard Huxton wrote: Richard Huxton wrote: Heikki Linnakangas wrote: The problem is that the new tuple version is checked only against the condition in the update rule, id=OLD.id, but not the condition in the original update-claus, dt='a'. Yeah, that's confusing :(. Bit more than just nor

Re: [HACKERS] Concurrently updating an updatable view

2007-05-15 Thread Florian G. Pflug
Richard Huxton wrote: Hiroshi Inoue wrote: Florian G. Pflug wrote: I think there should be a big, fat warning that self-referential updates have highly non-obvious behaviour in read-committed mode, and should be avoided. It seems pretty difficult for PostgreSQL rule system to avoid such

Re: [HACKERS] Not ready for 8.3

2007-05-19 Thread Florian G. Pflug
Andrew Dunstan wrote: What would making a branch actually do for you? The only advantage I can see is that it will give you a way of checkpointing your files. As I remarked upthread, I occasionally use RCS for that. But mostly I don't actually bother. I don't see how you can do it reasonably of

Re: Do we need a TODO? (was Re: [HACKERS] Concurrently updating anupdatable view)

2007-06-01 Thread Florian G. Pflug
Simon Riggs wrote: On Mon, 2007-05-28 at 19:56 -0400, Bruce Momjian wrote: Added to TODO: * Fix self-referential UPDATEs seeing inconsistent row versions in read-committed mode http://archives.postgresql.org/pgsql-hackers/2007-05/msg00507.php I'm sorry guys but I don't agree this is a T

[HACKERS] recovery_target_xid & crashes on the master

2007-06-04 Thread Florian G. Pflug
Hi I'm currently working on splitting StartupXLog into smaller parts, because I need to reuse some of the parts for concurrent wal recovery (for my GSoC project) The function recoveryStopsHere in xlog.c checks if we should stop recovery due to the values of recovery_target_xid and recovery_targe

[HACKERS] [RFC] GSoC Work on readonly queries done so far

2007-06-06 Thread Florian G. Pflug
Hi This is a description of the GSoC work I've so for on readonly support for PITR slaves. I'm looking for any kind of comments on this - I want to make sure that I work in a direction that the community approves. Work done so far: - .) Added a new GUC operational_mode, which can

Re: [HACKERS] [RFC] GSoC Work on readonly queries done so far

2007-06-06 Thread Florian G. Pflug
Matthew T. O'Connor wrote: Florian G. Pflug wrote: Work done so far: - .) Don't start autovacuum and bgwriter. Do table stats used by the planner get replicated on a PITR slave? I assume so, but if not, you would need autovac to do analyzes. Yes - everything th

Re: [HACKERS] [RFC] GSoC Work on readonly queries done so far

2007-06-06 Thread Florian G. Pflug
Jeff Davis wrote: On Wed, 2007-06-06 at 16:11 +0200, Florian G. Pflug wrote: .) Since the slaves needs to track an Snapshot in shared memory, it cannot resize that snapshot to accomodate however many concurrent transactions might have been running on the master. My current plan

Re: [HACKERS] [RFC] GSoC Work on readonly queries done so far

2007-06-07 Thread Florian G. Pflug
Simon Riggs wrote: On Wed, 2007-06-06 at 16:11 +0200, Florian G. Pflug wrote: .) Added a new GUC operational_mode, which can be set to either readwrite or readonly. If it is set to readwrite (the default), postgres behaves as usual. All the following changes are only in effect if

Re: [HACKERS] [RFC] GSoC Work on readonly queries done so far

2007-06-07 Thread Florian G. Pflug
Heikki Linnakangas wrote: Florian G. Pflug wrote: Jeff Davis wrote: Are you referring to the size of the xip array being a problem? Would it help to tie the size of the xip array to max_connections? I understand that max_connections might be greater on the master, but maybe something similar

Re: [HACKERS] [RFC] GSoC Work on readonly queries done so far

2007-06-07 Thread Florian G. Pflug
Jeff Davis wrote: On Wed, 2007-06-06 at 22:36 +0100, Simon Riggs wrote: .) Transactions are assigned a dummy xid ReadOnlyTransactionId, that is considered to be "later" than any other xid. So you are bumping FirstNormalTransactionId up by one for this? You're assuming then that we will

Re: [HACKERS] Is this a feature?

2007-06-11 Thread Florian G. Pflug
Joshua D. Drake wrote: Take the following: INFO: analyzing "pg_catalog.pg_authid" INFO: "pg_authid": scanned 1 of 1 pages, containing 5 live rows and 0 dead rows; 5 rows in sample, 5 estimated total rows The above is completely redundant. Why not just say: INFO: "pg_authid": scanned 1 of

Re: [HACKERS] Selecting a constant question

2007-06-12 Thread Florian G. Pflug
Dann Corbit wrote: -Original Message- From: Hannu Krosing [mailto:[EMAIL PROTECTED] Since libpq function PQfsize returns -2 for all constant character strings in SQL statements ... What is the proper procedure to determine the length of a constant character column after query execution b

Re: [HACKERS] EXPLAIN omits schema?

2007-06-13 Thread Florian G. Pflug
Gregory Stark wrote: "Joshua D. Drake" <[EMAIL PROTECTED]> writes: I agree. XML seems like a fairly natural fit for this. Just as people should not try to shoehorn everything into XML, neither should they try to shoehorn everything into a relational format either. Now all we need is an XML sche

Re: [HACKERS] Controlling Load Distributed Checkpoints

2007-06-13 Thread Florian G. Pflug
Heikki Linnakangas wrote: Jim C. Nasby wrote: On Thu, Jun 07, 2007 at 10:16:25AM -0400, Tom Lane wrote: Heikki Linnakangas <[EMAIL PROTECTED]> writes: Thinking about this whole idea a bit more, it occured to me that the current approach to write all, then fsync all is really a historical arti

Re: [HACKERS] Worries about delayed-commit semantics

2007-06-22 Thread Florian G. Pflug
PFC wrote: On Fri, 22 Jun 2007 16:43:00 +0200, Bruce Momjian <[EMAIL PROTECTED]> wrote: Simon Riggs wrote: On Fri, 2007-06-22 at 14:29 +0100, Gregory Stark wrote: > "Joshua D. Drake" <[EMAIL PROTECTED]> writes: > > Tom Lane wrote: > >> untrustworthy disk hardware, for instance. I'd much rather

Re: [HACKERS] Worries about delayed-commit semantics

2007-06-22 Thread Florian G. Pflug
Richard Huxton wrote: Bruce Momjian wrote: Tom Lane wrote: What's wrong with synchronous_commit? It's accurate and simple. That is fine too. My concern would be that it can be read two ways: 1. When you commit, sync (something or other - unspecified) 2. Synchronise commits (to each other?

Re: [HACKERS] Still recommending daily vacuum...

2007-07-05 Thread Florian G. Pflug
Michael Paesold wrote: Alvaro Herrera wrote: So what you are proposing above amounts to setting scale factor = 0.05. The threshold is unimportant -- in the case of a big table it matters not if it's 0 or 1000, it will be almost irrelevant in calculations. In the case of small tables, then the t

Re: [HACKERS] [PATCH] A crash and subsequent recovery of themaster can cause the slave to get out-of-sync

2007-07-05 Thread Florian G. Pflug
Tom Lane wrote: [ back to dealing with this patch, finally ] "Florian G. Pflug" <[EMAIL PROTECTED]> writes: While creating the patch, I've been thinking if it might be worthwile to note that we just did recovery in the ShutdownCheckpoint (or create a new checkpoint ty

Re: [HACKERS] [PATCH] A crash and subsequent recovery of themaster can cause the slave to get out-of-sync

2007-07-05 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: Tom Lane wrote: Conclusion: we should apply Florian's patch as-is in 8.2, do something morally equivalent in 8.1 and before, and invent a CrashRecoveryCheckpoint record type in HEAD. Sounds good. Actu

[HACKERS] ReadRecord, EndRecPtr and XLOG_SWITCH

2007-07-09 Thread Florian G. Pflug
Hi When ReadRecord encounters an XLOG_SWITCH record, it does EndRecPtr.xrecoff += XLogSegSize - 1; EndRecPtr.xrecoff -= EndRecPtr.xrecoff % XLogSegSize; which seems to set xrecoff to either 0 (if it was 0) or to XLogSegSize (if it was > 0). Note that xrecoff == XLogSegSize is kind of "denormaliz

Re: [HACKERS] ReadRecord, EndRecPtr and XLOG_SWITCH

2007-07-09 Thread Florian G. Pflug
Florian G. Pflug wrote: Please disregard - I was confusing xlogid with xlog segments, so most of my mail was nonsense. I've fixed my problem by storing not the EndRecPtr, but rather the ReadRecPtr, in shmem and rereading the last already applied record in my bgreplay process. Then I can

[HACKERS] "Running readonly queried on PITR slaves" status update

2007-07-09 Thread Florian G. Pflug
Hi After struggling with understanding xlog.c and friends far enough to be able to refactor StartupXLOG to suit the needs of concurrent recovery, I think I've finally reached a workable (but still a bit hacky) solution. My design is centered around the idea of a bgreplay process that takes over

Re: [HACKERS] 2PC-induced lockup

2007-07-11 Thread Florian G. Pflug
Tom Lane wrote: "Simon Riggs" <[EMAIL PROTECTED]> writes: I'd be much more comfortable if LOCK TABLE caused a message to the log if it is executed on any system table. Enabled by "set training_wheels = on", perhaps? This is really pretty silly to be getting worked up about. The command in qu

[HACKERS] xlog switch & last record before the switch

2007-07-12 Thread Florian G. Pflug
Hi To test my PITR-slave readonly-query patch, I continously do insert into test ... pg_switch_xlog() sleep 1 on the master, and let the slave process the generated xlogs The log output on the slave looks the following (unnecessary lines remove) LOG: restored log file "0001016E

Re: [HACKERS] xlog switch & last record before the switch

2007-07-12 Thread Florian G. Pflug
Simon Riggs wrote: On Thu, 2007-07-12 at 16:17 +0200, Florian G. Pflug wrote: To test my PITR-slave readonly-query patch, I continously do insert into test ... pg_switch_xlog() sleep 1 on the master, and let the slave process the generated xlogs The log output on the slave looks the following

[HACKERS] Full page images in WAL & Cache Invalidation

2007-07-22 Thread Florian G. Pflug
Hi I'm currently working on correctly flushing the catalog/relation/sgmr caches on a readonly PITR slave during recovery. These are the things that currently cause me headache. 1) It seems that the btree code sends out relcache invalidation msgs during normal operation (No DDL statements a

Re: [HACKERS] Full page images in WAL & Cache Invalidation

2007-07-22 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: I'm currently working on correctly flushing the catalog/relation/sgmr caches on a readonly PITR slave during recovery. I don't believe there is any workable solution to that short of logging cache-fl

Re: [HACKERS] Full page images in WAL & Cache Invalidation

2007-07-22 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: Leaving aside the btree issues, are you worried about performance problems, or can aggressive cache flushing hurt correctness? It shouldn't hurt correctness, but I don't think you entirely grasp the ma

Re: [HACKERS] Full page images in WAL & Cache Invalidation

2007-07-22 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: My basic assumption is that DDL is something quite uncommon on a production system. I'm not sure I believe that, because of temp tables. There's also the problem that plain VACUUM (or ANALYZE) causes

Re: [HACKERS] Full page images in WAL & Cache Invalidation

2007-07-23 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: Anyway, if you believe that DDL is infrequent, why are you resistant to the idea of WAL-logging cache flushes? First, cache invalidations are not the only problem caused by replaying system-table updates. The w

Re: [HACKERS] Full page images in WAL & Cache Invalidation

2007-07-23 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: I came up with the following plan for both inval events and locks .) Store two flags in the commit record of a transaction, for "transaction generated inval events" and "transaction held an

Re: [HACKERS] Full page images in WAL & Cache Invalidation

2007-07-23 Thread Florian G. Pflug
Simon Riggs wrote: On Sun, 2007-07-22 at 19:58 +0200, Florian G. Pflug wrote: Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: I'm currently working on correctly flushing the catalog/relation/sgmr caches on a readonly PITR slave during recovery. I don&#

[HACKERS] Tracking a snapshot on PITR slaves

2007-08-02 Thread Florian G. Pflug
Hi Since my attempts to find a simple solution for the read-only query locking problems (Once that doesn't need full wal logging of lock requests) haven't been successfully yet, I've decided to turn to the problems of tracking a snapshot on the slaves for now. (Because first such a snapshot is ne

Re: [HACKERS] XID wraparound and busy databases

2007-08-16 Thread Florian G. Pflug
Tom Lane wrote: Bruce Momjian <[EMAIL PROTECTED]> writes: Is enlarging the xid field something we should consider for 8.4? No. We just got the tuple header down to 24 bytes, we are not going to give that back and then some. If you are processing 6K transactions per second, you can afford to

Re: [HACKERS] SQL feature requests

2007-08-22 Thread Florian G. Pflug
Ben Tilly wrote: On 8/22/07, Gregory Stark <[EMAIL PROTECTED]> wrote: "Ben Tilly" <[EMAIL PROTECTED]> writes: 2. Why is 'non-integer constant in GROUP BY' an error? Hm... I was a bit surprised by this warning myself. IIRC there was an implementation convenience issue. If your implementation

Re: [HACKERS] SQL feature requests

2007-08-23 Thread Florian G. Pflug
Ben Tilly wrote: On 8/22/07, Michael Glaesemann <[EMAIL PROTECTED]> wrote: On Aug 22, 2007, at 20:49 , Ben Tilly wrote: If your implementation accepts: group by case when true then 'foo' end What would that mean? Regardless of whether or not it's accepted, it should have *some* meaning.

Re: [HACKERS] SQL feature requests

2007-08-23 Thread Florian G. Pflug
Kevin Grittner wrote: On Thu, Aug 23, 2007 at 3:01 PM, in message <[EMAIL PROTECTED]>, Tom Lane <[EMAIL PROTECTED]> wrote: The only argument I've heard that carries much weight with me is that it eases porting from other DBMS's that allow this. Are there any others besides Oracle? select *

Re: [HACKERS] simple replication

2007-08-24 Thread Florian G. Pflug
Edoardo wrote: On Aug 24, 4:27 pm, [EMAIL PROTECTED] ("Heikki Linnakangas") wrote: Edoardo wrote: - Google Summer of Code effort by Florian Pflug I would like to talk about the last one: I couldn't find any website / explanation or further post. The google linkhttp://code.google.com/soc/postgr

[HACKERS] [WIP PATCH] Lazily assign xids for toplevel Transactions

2007-08-25 Thread Florian G. Pflug
I'm resending this without that actual patch attached, since my first mail didn't get through. The patch can be found here: http://soc.phlo.org/lazyxidassign.patch Hi Lately, there was some interest in assigning XIDs for toplevel Transactions only when the transaction actually needs one (that is

Re: [HACKERS] [WIP PATCH] Lazily assign xids for toplevel Transactions

2007-08-25 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: I've spent the last few days factoring out that work, and turning it into a general solution. The result is this patch, which basically does the following .) It defines a special TemporaryTransactionId that

Re: [HACKERS] [WIP PATCH] Lazily assign xids for toplevel Transactions

2007-08-27 Thread Florian G. Pflug
Tom Lane wrote: I wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: Yeah - I do not really like that dual-locking thing either. But it makes prepared transaction handling much easier - if we were to only lock the RID, we'd have to store the rid<->xid mappin

Re: [HACKERS] [WIP PATCH] Lazily assign xids for toplevel Transactions

2007-08-27 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: Sounds good, if we decide to go with the transient XID idea. So below for an alternative that I just came up with. This proposal appears to require taking and releasing a brand-new lock type every time a sna

Re: [HACKERS] [WIP PATCH] Lazily assign xids for toplevel Transactions

2007-08-27 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: What do you think about solving the requirements of the *first* waiting phase (Where we wait for current ShareLock holders) inside the lock manager? The only real downside I can see is that I feel uneasy about messin

[HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-29 Thread Florian G. Pflug
Hi When we lazily assign XIDs, we gain another flag beside the existing MyXactMadeTempRelUpdates, MyXactMadeXLogEntry, MyLastRecPtr and smgr's pendingDeletes to tell what kind of actions a transaction performed. Adding TransactionIsIsValid(GetCurrentTransactionIdIfAny()) on top of that makes thin

[HACKERS] Representation of ResourceOwnerIds (transient XIDs) in system views (lazy xid assignment)

2007-08-29 Thread Florian G. Pflug
Hi Since generating transient XIDs (named ResourceOwnerIDs in my patch, since their lifetime is coupled to the lifetime of a transaction's toplevel resource owner) seems to be to way to go for lazx xid assignment, I need to find a way to represent them in the pg_locks view. ResourceOwnerIds are

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-29 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: I propose to do the following in my lazy XID assignment patch - can anyone see a hole in that? > One comment is that at the time we make an entry into smgr's pending-deletes list, I think we might not

Re: [HACKERS] Representation of ResourceOwnerIds (transient XIDs) in system views (lazy xid assignment)

2007-08-29 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: Since generating transient XIDs (named ResourceOwnerIDs in my patch, since their lifetime is coupled to the lifetime of a transaction's toplevel resource owner) seems to be to way to go for lazx xid assignment,

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-29 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: Tom Lane wrote: One thought here is that it's not clear that we really need a concept of transaction-controlled vs not-transaction-controlled xlog records anymore. I've thinking about keeping XLOG_NO_TR

[HACKERS] int8 & INT64_IS_BUSTED

2007-08-29 Thread Florian G. Pflug
Hi I'm confused about whether int8s work on a machine on which INT64_IS_BUSTED. My reading of the code suggests that int8 will be available, but be, well, busted in such a machine. For example, int8mul seems as if I'd just return the wrong answer on such a machine. Or are platforms with INT64_I

Re: [HACKERS] Representation of ResourceOwnerIds (transient XIDs) in system views (lazy xid assignment)

2007-08-29 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: What about the following. .) Remove the right-hand side XID from pg_locks (The one holder or waiter of the lock). It seems to make more sense to store a RID here, Yeah, we have to do that since there might

Re: [HACKERS] int8 & INT64_IS_BUSTED

2007-08-29 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: I'm confused about whether int8s work on a machine on which INT64_IS_BUSTED. My reading of the code suggests that int8 will be available, but be, well, busted in such a machine. The datatype exists, but i

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-29 Thread Florian G. Pflug
Tom Lane wrote: One comment is that at the time we make an entry into smgr's pending-deletes list, I think we might not have acquired an XID yet --- if I understand your patch correctly, a CREATE TABLE would acquire an XID when it makes its first catalog insertion, and that happens after creating

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-29 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: One comment is that at the time we make an entry into smgr's pending-deletes list, I think we might not have acquired an XID yet Hm.. I was just going to implement this, but I'm now wondering if thats re

  1   2   3   4   >