Forgot to include the group...
-- Forwarded message --
From: Gokulakannan Somasundaram <[EMAIL PROTECTED]>
Date: Nov 5, 2007 3:04 PM
Subject: Re: Clarification about HOT
To: Gregory Stark <[EMAIL PROTECTED]>
On 11/2/07, Gregory Stark <[EMAIL PROTECTED]> wrote:
>
>
> "Gokulakan
Though 8.3 isn't out of the oven just yet, I've been thinking about the
dead space map a lot, and decided I have to start writing down those
thoughts.
Reducing VACUUM time is important, but the real big promise is the
ability to do index-only-scans. Because that's the main focus of this
exerc
Gokulakannan Somasundaram wrote:
Thanks for all the inputs. My question would be if we decide to update the
top of the HOT chain in the Index itself. Right now we are carrying a list
of tuple-ids to be vacuumed, when we vacuum the index. Say we carry another
list (or through some better mechanism
On 11/5/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote:
>
> Gokulakannan Somasundaram wrote:
> > Thanks for all the inputs. My question would be if we decide to update
> the
> > top of the HOT chain in the Index itself. Right now we are carrying a
> list
> > of tuple-ids to be vacuumed, when we v
Gokulakannan Somasundaram wrote:
I think pointer swinging is still about maintaining the HOT chain within a
page. Actually i am thinking about continuing the HOT chain across pages.
AFAICS, pointer-swinging would be exactly the same on cross-page HOT
chains as same-page chains.
--
Heikki L
On 11/5/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote:
>
> Gokulakannan Somasundaram wrote:
> > I think pointer swinging is still about maintaining the HOT chain within
> a
> > page. Actually i am thinking about continuing the HOT chain across
> pages.
>
> AFAICS, pointer-swinging would be exact
Gokulakannan Somasundaram wrote:
On 11/5/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote:
AFAICS, pointer-swinging would be exactly the same on cross-page HOT
chains as same-page chains.
When i read pointer-swinging, it talks a lot about in-page updates, pointing
to the latest tuple instead o
>
> > b) Whenever we Vacuum the index, we take a list of tids and check
> whether
> > there are any index tuples pointing to it. If the Vacuumed tuple is a
> start
> > of the HOT chain, then we will carry the next in-line HOT tuple when we
> goto
> > Vacuum the index. If the next in-line also satis
Hi Tom
On Sun, 4 Nov 2007, Tom Lane wrote:
> Date: Sun, 04 Nov 2007 19:47:04 -0500
> From: Tom Lane <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Cc: pgsql-hackers@postgresql.org
> Subject: Re: [HACKERS] should I worry?
>
> I wrote:
> > Hmm, this is messier than I thought. What evidently has happ
Gokulakannan Somasundaram wrote:
b) Whenever we Vacuum the index, we take a list of tids and check
whether
there are any index tuples pointing to it. If the Vacuumed tuple is a
start
of the HOT chain, then we will carry the next in-line HOT tuple when we
goto
Vacuum the index. If the next i
Again Forgot to include the group...
-- Forwarded message --
From: Gokulakannan Somasundaram <[EMAIL PROTECTED]>
Date: Nov 5, 2007 5:09 PM
Subject: Re: [HACKERS] Fwd: Clarification about HOT
To: Heikki Linnakangas <[EMAIL PROTECTED]>
On 11/5/07, Heikki Linnakangas <[EMAIL PROTEC
I am trying to run regression tests on both windows and RH.
It looks like that the tests on windows run slower than linux
using two machines with same hardware config.
Is this known?
Gevik Babakhani
PostgreSQL NL http://www.postgresql.nl
[EMAIL PROTECTED] writes:
> On Sun, 4 Nov 2007, Tom Lane wrote:
>> So you have a *bunch* of partially broken FK constraints in that source
>> database.
> I just talk to my customer and he/we'll make a big batch deleting and
> recreating all foreign keys on 8.2.5.
> The question is, how do we get r
Gevik Babakhani wrote:
I am trying to run regression tests on both windows and RH.
It looks like that the tests on windows run slower than linux
using two machines with same hardware config.
Is this known?
We need far more information than this before we can say much sensibly,
I thin
"Gokulakannan Somasundaram" <[EMAIL PROTECTED]> writes:
> May be i am missing something in the big picture. Please clarify me on that.
Locking. Your proposal involves lots of multi-page operations, which
are best avoided.
Moreover, you have offered no data to suggest that there would be any
real
This is my feedback regarding the Visibility map.
I just want to disagree on the fact that the DSM/Visibility map would not
focus on Vacuum. I think Index only scans should be thought of as a fringe
benefit of DSMs/ Visibility Maps. There are some basic assumptions in the
design, which says that
a
Tom,
Let me try to understand your statement.
What extra multi-page operations are we doing?
Currently, during Vacuum, we goto the Index and mark it as dead and
reclaim the space. For doing this, we are acquiring a Super-Exclusive lock.
After this implementation, we would update the inde
On Nov 5, 2007 7:37 PM, Gokulakannan Somasundaram <[EMAIL PROTECTED]>
wrote:
> Tom,
>Let me try to understand your statement.
>
> What extra multi-page operations are we doing?
> Currently, during Vacuum, we goto the Index and mark it as dead and
> reclaim the space. For doing this, we
Thanks for the feedback. Let me try to put what is there in my mind for
this. Please clarify whether my assumptions are valid
On 11/5/07, Pavan Deolasee <[EMAIL PROTECTED]> wrote:
>
>
>
> On Nov 5, 2007 7:37 PM, Gokulakannan Somasundaram <[EMAIL PROTECTED]>
> wrote:
>
> > Tom,
> >Let me tr
"Gokulakannan Somasundaram" <[EMAIL PROTECTED]> writes:
> Currently, during Vacuum, we goto the Index and mark it as dead and
> reclaim the space. For doing this, we are acquiring a Super-Exclusive lock.
> After this implementation, we would update the index tuple instead of
> marking it for cle
On Nov 5, 2007 8:04 PM, Gokulakannan Somasundaram <[EMAIL PROTECTED]>
wrote:
>
>
> There may not be anything called chain-pruning. Instead the tuples, which
> are to be vacuumed, will get vacuumed, after redirecting their index tuple
> peers, during the Vacuum process.
>
>
This won't help us chec
Gokulakannan Somasundaram wrote:
a) The inserts won't increase the size of the table. If it increases, it has
to lock one full page of Visibility map and this is not suitable for tables,
which are short-lived like partitioned tables
The overhead of locking a page is very small.
Actually, exten
Tom Lane wrote:
Andrew Dunstan <[EMAIL PROTECTED]> writes:
Tom Lane wrote:
A BEFORE UPDATE trigger would be better, and probably hardly more
expensive than a wired-in facility (especially if you were willing to
write it in C).
Yes. I also prefer the trigger idea to a rule
On 11/5/07, Tom Lane <[EMAIL PROTECTED]> wrote:
>
> "Gokulakannan Somasundaram" <[EMAIL PROTECTED]> writes:
> > Currently, during Vacuum, we goto the Index and mark it as dead and
> > reclaim the space. For doing this, we are acquiring a Super-Exclusive
> lock.
> > After this implementation, we
Gokulakannan Somasundaram wrote:
There's also the little problem that a redirect line pointer doesn't
have room for a cross-page link,
i don't know, what is a re-direct line pointer.
Then you clearly don't understand at all how HOT works. Please go read
src/backend/access/heap/README.HOT.
On 11/5/07, Pavan Deolasee <[EMAIL PROTECTED]> wrote:
>
>
>
> On Nov 5, 2007 8:04 PM, Gokulakannan Somasundaram <[EMAIL PROTECTED]>
> wrote:
>
> >
> >
> > There may not be anything called chain-pruning. Instead the tuples,
> > which are to be vacuumed, will get vacuumed, after redirecting their ind
On Mon, Nov 05, 2007 at 08:43:01AM -0500, Andrew Dunstan wrote:
>
>
> Gevik Babakhani wrote:
> >I am trying to run regression tests on both windows and RH.
> >It looks like that the tests on windows run slower than linux
> >using two machines with same hardware config.
> >
> >Is this known?
> >
Thanks Heikki. To clarify, there won't be any redirect-line pointers in this
implementation. That space is saved. We will have the index tuple point to
the latest live tuple in the update chain. So no need for redirect-line
pointers.
Thanks,
Gokul.
On 11/5/07, Heikki Linnakangas <[EMAIL PROTECTE
I was somewhat bemused just now to find that this function stopped
working:
regression=# create function estimate_rows(query text) returns float8 as $$
declare r text;
begin
for r in execute 'explain ' || query loop
if substring(r from 'rows=[0-9]') is not null then
return substring (r
Andrew Dunstan <[EMAIL PROTECTED]> writes:
> Tom Lane wrote:
>> Well, you could write the trigger in C and it'd work for any table.
>> I think it could be as simple as a memcmp of the tuples' data areas,
>> since we now require padding bytes to be 0 ...
> Something like this fragment?
> newtupl
Gokulakannan Somasundaram wrote:
Thanks Heikki. To clarify, there won't be any redirect-line pointers in this
implementation. That space is saved. We will have the index tuple point to
the latest live tuple in the update chain. So no need for redirect-line
pointers.
The redirected line pointer
"Tom Lane" <[EMAIL PROTECTED]> writes:
> I was somewhat bemused just now to find that this function stopped
> working:
>
> regression=# create function estimate_rows(query text) returns float8 as $$
...
> This works fine in 8.2. The reason it no longer works is that "query"
> is now a special to
Magnus Hagander wrote:
From what I can see MinGW regression is somewhat slower than MSVC (and
even than Cygwin) on my buildfarm VM that runs all three, even though
the latter two are rate limited by MAX_CONNECTIONS.
Uh, you're saying MSVC and Cygwin somehow differ from MingW?
I
On 11/5/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote:
>
> Gokulakannan Somasundaram wrote:
> > Thanks Heikki. To clarify, there won't be any redirect-line pointers in
> this
> > implementation. That space is saved. We will have the index tuple point
> to
> > the latest live tuple in the update
On Mon, Nov 05, 2007 at 11:01:26AM -0500, Andrew Dunstan wrote:
>
>
> Magnus Hagander wrote:
> >>From what I can see MinGW regression is somewhat slower than MSVC (and
> >>even than Cygwin) on my buildfarm VM that runs all three, even though
> >>the latter two are rate limited by MAX_CONNECTION
This has been saved for the 8.4 release:
http://momjian.postgresql.org/cgi-bin/pgpatches_hold
---
Pavel Stehule wrote:
> Hello,
>
> this proposal change older unaccepted proposal
> http://archives.postgresql.org/pg
Gregory Stark <[EMAIL PROTECTED]> writes:
> FWIW I found another issue with this variable when I was experimenting with
> small block sizes. If you set the target <= the tuple header the toaster
> breaks. This is because it's doing unsigned arithmetic (Size is unsigned). I
> think the right solutio
Magnus Hagander wrote:
In fact, on the last run of each, MinGW was by far the slowest and
Cygwin the fastest in the "make check" step. But there could be any
number of reasons for that, including extraneous activity on the VM host
that could have slowed the whole VM down.
Yeah. It does
On 11/5/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote:
>
> Gokulakannan Somasundaram wrote:
> > a) The inserts won't increase the size of the table. If it increases, it
> has
> > to lock one full page of Visibility map and this is not suitable for
> tables,
> > which are short-lived like partiti
"Tom Lane" <[EMAIL PROTECTED]> writes:
> Gregory Stark <[EMAIL PROTECTED]> writes:
>> FWIW I found another issue with this variable when I was experimenting with
>> small block sizes. If you set the target <= the tuple header the toaster
>> breaks. This is because it's doing unsigned arithmetic (S
On Mon, Nov 05, 2007 at 09:32:08PM +0530, Gokulakannan Somasundaram wrote:
> Say, if we have a table with 4 indexes and updates occur in such intervals,
> we may not be able to find space in the same page for the update. Currently
> we are incurring the overhead of updating all the indexes in this
On 11/5/07, Martijn van Oosterhout <[EMAIL PROTECTED]> wrote:
>
> On Mon, Nov 05, 2007 at 09:32:08PM +0530, Gokulakannan Somasundaram wrote:
> > Say, if we have a table with 4 indexes and updates occur in such
> intervals,
> > we may not be able to find space in the same page for the update.
> Curr
Tom Lane wrote:
> Alvaro Herrera <[EMAIL PROTECTED]> writes:
> > I think the mythical pg_ping utility should be written. It seems the
> > easiest way out of the problem.
>
> If pg_ctl were still a shell script there would be some point in that,
> but since it's a C program it can certainly do any
Alvaro Herrera <[EMAIL PROTECTED]> writes:
> Tom Lane wrote:
>> Alvaro Herrera <[EMAIL PROTECTED]> writes:
>>> I think the mythical pg_ping utility should be written. It seems the
>>> easiest way out of the problem.
>>
>> If pg_ctl were still a shell script there would be some point in that,
>> b
This has been saved for the 8.4 release:
http://momjian.postgresql.org/cgi-bin/pgpatches_hold
---
Decibel! wrote:
> Is it intentional that dblink's unnamed connections don't get re-used?
>
> stats=# select datname,
I understand your suggestions but it seems there would be too many
individual items to be readable. Can you suggest a full list so we can
get an idea of how long it would be?
---
Brendan Jurd wrote:
> On 11/1/07, Bruce Momj
In an attempt to move us toward 8.3 RC1 I have put all open item emails
into the patches queue:
http://momjian.postgresql.org/cgi-bin/pgpatches
--
Bruce Momjian <[EMAIL PROTECTED]>http://momjian.us
EnterpriseDB http://postgres.enterprisedb.com
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On Mon, 5 Nov 2007 12:47:10 -0500 (EST)
Bruce Momjian <[EMAIL PROTECTED]> wrote:
> In an attempt to move us toward 8.3 RC1 I have put all open item
> emails into the patches queue:
>
> http://momjian.postgresql.org/cgi-bin/pgpatches
>
It woul
Gregory Stark <[EMAIL PROTECTED]> writes:
> "Tom Lane" <[EMAIL PROTECTED]> writes:
>> This works fine in 8.2. The reason it no longer works is that "query"
>> is now a special token in the plpgsql lexer, and that means that it will
>> never be substituted for by read_sql_construct(). So it's effe
John DeSoi <[EMAIL PROTECTED]> writes:
> Is there any feasibility to the idea of allowing pl/pgsql variables
> and parameters to be prefixed with a special character like '$'?
I find this (a) really ugly, (b) incompatible with Oracle, which
you'll recall is one of the main driving ideas for plpg
I'm trying fix independence of pg_dump.c on postgres.h. And I found
following construct in dumpSequence function:
09391 snprintf(bufm, sizeof(bufm), INT64_FORMAT, SEQ_MINVALUE);
09392 snprintf(bufx, sizeof(bufx), INT64_FORMAT, SEQ_MAXVALUE);
09393
09394 appendPQExpBuffer(query,
093
Joshua D. Drake wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On Mon, 5 Nov 2007 12:47:10 -0500 (EST)
> Bruce Momjian <[EMAIL PROTECTED]> wrote:
>
> > In an attempt to move us toward 8.3 RC1 I have put all open item
> > emails into the patches queue:
> >
> > http://momjian.pos
"Joshua D. Drake" <[EMAIL PROTECTED]> writes:
> On Mon, 5 Nov 2007 12:47:10 -0500 (EST)
> Bruce Momjian <[EMAIL PROTECTED]> wrote:
>
>> In an attempt to move us toward 8.3 RC1 I have put all open item
>> emails into the patches queue:
>>
>> http://momjian.postgresql.org/cgi-bin/pgpatches
>>
Hi All,
we are trying to modify the source code of postgresql to support a new
command pivot
we are doing the following steps:
1. After parsing the input, we are able to get the values entered by the
user for column A , column B and tablename.
2. Then, from postgresmain, we call exec_simple_que
Guillaume Smet wrote:
> I did the test again with the reference database I used a month ago.
>
> My previous figures with 8.3devel of October 1st were:
> - autovacuum off: 14m39
> - autovacuum on, delay 20: 51m37
>
> With 8.3devel of today, I have:
> - autovacuum on, delay 20: 15m26
Yay! Thank
Peter Eisentraut <[EMAIL PROTECTED]> writes:
> Heikki Linnakangas wrote:
>> It still feels unsafe to call ExecEvalExpr while holding on to xml
>> structs. It means that it's not safe for external modules to use
>> libxml2 and call xmlMemSetup or xmlSetGenericErrorFunc themselves.
> Well yeah, they
Zdenek Kotala <[EMAIL PROTECTED]> writes:
> This construct is used to determine if max_value/min_value is used and
> after that pg_dump add NO MAXVALUE to the output instead of the value.
> If I compare it with manual documentation NO MAXVALUE uses default value
> and I do not see any reason to
"Rose Catherine K" <[EMAIL PROTECTED]> writes:
> we are trying to modify the source code of postgresql to support a new
> command pivot
> we are doing the following steps:
> 1. After parsing the input, we are able to get the values entered by the
> user for column A , column B and tablename.
> 2
On Mon, 2007-11-05 at 09:52 +, Heikki Linnakangas wrote:
> Reducing VACUUM time is important, but the real big promise is the
> ability to do index-only-scans. Because that's the main focus of this
> exercise, I'm calling it the the Visibility Map from now on, because
> it's not about tracki
Tom Lane wrote:
A better solution might be to move the declarations of
SEQ_MINVALUE/SEQ_MAXVALUE someplace else.
Hmm. It seems better, but it is also hard to find correct place. :( I'm
thinking put it into c.h.
Another question why sequence does not have separate flag which
determines if
Zdenek Kotala <[EMAIL PROTECTED]> writes:
> Tom Lane wrote:
>> A better solution might be to move the declarations of
>> SEQ_MINVALUE/SEQ_MAXVALUE someplace else.
> Hmm. It seems better, but it is also hard to find correct place. :( I'm
> thinking put it into c.h.
The idea that was in the back o
Doesn't DROP TRIGGER require the name of the trigger? He says they
are unnamed. How then does he drop them?
On Nov 5, 2007, at 6:31 AM, Tom Lane wrote:
[EMAIL PROTECTED] writes:
On Sun, 4 Nov 2007, Tom Lane wrote:
So you have a *bunch* of partially broken FK constraints in that
source
Rick Gigger wrote:
Doesn't DROP TRIGGER require the name of the trigger? He says they are
unnamed. How then does he drop them?
They're not really unnamed. pg_dump just replaces the real name with
"".
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
Heikki Linnakangas <[EMAIL PROTECTED]> writes:
> Though 8.3 isn't out of the oven just yet, I've been thinking about the
> dead space map a lot, and decided I have to start writing down those
> thoughts.
I think we should do this at the same time as pushing the FSM out of
shared memory, and desi
Heikki Linnakangas <[EMAIL PROTECTED]> writes:
> Rick Gigger wrote:
>> Doesn't DROP TRIGGER require the name of the trigger? He says they are
>> unnamed. How then does he drop them?
> They're not really unnamed. pg_dump just replaces the real name with
> "".
And \d will show the real names of
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On Mon, 05 Nov 2007 18:57:39 +
Gregory Stark <[EMAIL PROTECTED]> wrote:
> "Joshua D. Drake" <[EMAIL PROTECTED]> writes:
>
> > On Mon, 5 Nov 2007 12:47:10 -0500 (EST)
> > Bruce Momjian <[EMAIL PROTECTED]> wrote:
> >
> >> In an attempt to move us t
Bruce Momjian wrote:
> Marko Kreen wrote:
> > On 8/24/07, Manuel Sugawara <[EMAIL PROTECTED]> wrote:
> > > If something is not going a work (or is going a work in a different
> > > way) in some version after loading a shot form a previous one I think
> > > it should be documented or some kind of b
Alvaro Herrera wrote:
> > > > If something is not going a work (or is going a work in a different
> > > > way) in some version after loading a shot form a previous one I think
> > > > it should be documented or some kind of backwards compatibility
> > > > mechanism should be provided.
> > >
> > >
On 11/4/07, Simon Riggs <[EMAIL PROTECTED]> wrote:
> Mark,
>
> Why don't you post a TODO list for TPC-E somewhere, so people can bite
> small pieces off of the list. I'm sure there's lots of people can help
> if we do it that way.
This should be a good start:
http://osdldbt.sourceforge.net/dbt5/t
Jeff Davis wrote:
On Mon, 2007-11-05 at 09:52 +, Heikki Linnakangas wrote:
It's not useful for VACUUM FREEZE, unless we're willing to freeze much
more aggressively, and change the meaning of a set bit to "all tuples on
heap page are frozen".
This means that a regular VACUUM will no longer
Ah, yes it was the quotes. I guess that makes me a newbie. :)
On Nov 5, 2007, at 1:53 PM, Tom Lane wrote:
Heikki Linnakangas <[EMAIL PROTECTED]> writes:
Rick Gigger wrote:
Doesn't DROP TRIGGER require the name of the trigger? He says
they are
unnamed. How then does he drop them?
The
Tom Lane wrote:
Heikki Linnakangas <[EMAIL PROTECTED]> writes:
Though 8.3 isn't out of the oven just yet, I've been thinking about the
dead space map a lot, and decided I have to start writing down those
thoughts.
I think we should do this at the same time as pushing the FSM out of
shared mem
On Nov 5, 2007, at 10:20 AM, Tom Lane wrote:
I don't see any fix for this that's reasonable to try to shoehorn
into 8.3, but I think we really need to revisit the whole area of
plpgsql variable substitution during 8.4. We could make this problem
go away if variable substitution happened throug
On 11/6/07, Bruce Momjian <[EMAIL PROTECTED]> wrote:
>
> I understand your suggestions but it seems there would be too many
> individual items to be readable. Can you suggest a full list so we can
> get an idea of how long it would be?
If the body of material on writing good Postgres code becomes
On Mon, 5 Nov 2007, Gregory Stark wrote:
> How many developers have even jumped through the hoops to get wiki accounts?
According to
http://developer.postgresql.org/index.php?title=Special:Listusers&group=pgdev&limit=500
there are currently 51 members of the group "pgdev" on the wiki.
--
"Spa
> > How many developers have even jumped through the hoops to get wiki accounts?
>
> According to
> http://developer.postgresql.org/index.php?title=Special:Listusers&group=pgdev&limit=500
>
> there are currently 51 members of the group "pgdev" on the wiki.
>
Well, a lot of those people aren't a
"Note that the bottom line for the problems with hash indexes is that the
current implementation doesn't offer any advantages over btree indexes. Hash
indexes need to be not only as good of a choice as btree indexes but
significantly better a significantly better choice at least for some specific
c
"Heikki Linnakangas" <[EMAIL PROTECTED]> writes:
> One problem is that you have to atomically update the visibility map when
> you update the heap. That means that you have to lock the visibility map
> page and the heap page at the same time. If the visibility map is in the
> heap, you need to tak
78 matches
Mail list logo
