This is now in the TODO list:
* Merge xmin/xmax/cmin/cmax back into three header fields
Before subtransactions, there used to be only three fields needed to
store these four values. This was possible because only the current
transaction looks at the cmin/cmax values. If the current
On Wed, Sep 07, 2005 at 12:31:01AM -0500, Jim C. Nasby wrote:
On Tue, Sep 06, 2005 at 07:02:20PM -0400, Tom Lane wrote:
Jim C. Nasby [EMAIL PROTECTED] writes:
If the 4 header fields in question were just normalized out, wouldn't
all the semantics continue to work the same? All I'm
On Wed, Sep 07, 2005 at 01:05:52PM -0400, Alvaro Herrera wrote:
Anyway you are just moving the storage somewhere else -- instead of
having 4 fields in the tuple itself, you have one field which points
the same 4 fields elsewhere. I don't see how is that a win; it's
actually worse because you
On Wed, Sep 07, 2005 at 01:20:27PM -0400, Tom Lane wrote:
Anyway the fundamental insight has been completely lost here. The
original point was that cmin and cmax are only interesting within the
originating transaction, and not to anyone else, and thus perhaps don't
need to be kept in
Alvaro Herrera [EMAIL PROTECTED] writes:
I'm curious to know how can you store the cmin/cmax pair completely out
of the tuple. It's easy to see how to store a single identifier in each
tuple that would be an index to a structure in local memory. However,
to eliminate both you'd have to keep
Alvaro Herrera [EMAIL PROTECTED] writes:
I think this could be done with our SLRU mechanism, just like pg_clog,
pg_subtrans and pg_multixact do. Whether it's really a good idea or
not, it's another story. The problem is that you would be creating new
ones all the time, it would become a huge
On Fri, Sep 02, 2005 at 03:51:15PM -0400, Bruce Momjian wrote:
One possible solution is to create a phantom cid which represents a
cmin/cmax pair and is stored in local memory.
If we're going to look at doing that I think it would also be good to
consider including xmin and xmax
On Tue, Sep 06, 2005 at 03:58:28PM -0500, Jim C. Nasby wrote:
On Fri, Sep 02, 2005 at 03:51:15PM -0400, Bruce Momjian wrote:
One possible solution is to create a phantom cid which represents a
cmin/cmax pair and is stored in local memory.
If we're going to look at doing that I
On Tue, Sep 06, 2005 at 05:37:06PM -0400, Alvaro Herrera wrote:
On Tue, Sep 06, 2005 at 03:58:28PM -0500, Jim C. Nasby wrote:
On Fri, Sep 02, 2005 at 03:51:15PM -0400, Bruce Momjian wrote:
One possible solution is to create a phantom cid which represents a
cmin/cmax pair and is
Jim C. Nasby [EMAIL PROTECTED] writes:
If we're going to look at doing that I think it would also be good to
consider including xmin and xmax as well.
If you do that, you'll never be able to delete or update the tuple.
My idea was to use an int to represent combinations of (c|x)(min|max),
On Tue, Sep 06, 2005 at 06:02:27PM -0400, Tom Lane wrote:
Jim C. Nasby [EMAIL PROTECTED] writes:
If we're going to look at doing that I think it would also be good to
consider including xmin and xmax as well.
If you do that, you'll never be able to delete or update the tuple.
My idea
Jim C. Nasby [EMAIL PROTECTED] writes:
If the 4 header fields in question were just normalized out, wouldn't
all the semantics continue to work the same? All I'm envisioning is
replacing them in each tuple with a pointer (vis_id) to another
datastore that would be roughly equivalent to:
On Tue, Sep 06, 2005 at 07:02:20PM -0400, Tom Lane wrote:
Jim C. Nasby [EMAIL PROTECTED] writes:
If the 4 header fields in question were just normalized out, wouldn't
all the semantics continue to work the same? All I'm envisioning is
replacing them in each tuple with a pointer (vis_id) to
On Fri, 2 Sep 2005 20:41:48 -0400 (EDT), Bruce Momjian
pgman@candle.pha.pa.us wrote:
Once I had a patch based on 7.4 that stored cmin and cmax in
backend-local memory.
Interesting idea, but how would you record the cmin/xmin values without
requiring unlimited memory?
That's exactly the reason
On Sat, Sep 03, 2005 at 10:59:31AM +0200, Manfred Koizar wrote:
On Fri, 2 Sep 2005 20:41:48 -0400 (EDT), Bruce Momjian
pgman@candle.pha.pa.us wrote:
Once I had a patch based on 7.4 that stored cmin and cmax in
backend-local memory.
Interesting idea, but how would you record the cmin/xmin
Manfred Koizar wrote:
On Fri, 2 Sep 2005 20:41:48 -0400 (EDT), Bruce Momjian
pgman@candle.pha.pa.us wrote:
Once I had a patch based on 7.4 that stored cmin and cmax in
backend-local memory.
Interesting idea, but how would you record the cmin/xmin values without
requiring unlimited
[skip]
happening in the bgwriter's inner loop. Even more to the point, you
can't do such changes without getting a superexclusive lock on the
page
(not only locked, but no one else has it pinned), which is a real
nonstarter for the bgwriter, both for performance and possible
deadlock
Tom Lane wrote:
Of course, it's fair to ask whether such a program would be any faster
than binary-mode COPY by the time you got done ... or enough faster to
justify your effort, anyway.
THe only fundamental disadvantage that COPY labors under is having to
write WAL records. It might be
Tom Lane wrote:
ITAGAKI Takahiro [EMAIL PROTECTED] writes:
I think it would be a waste to retain xmin and cmin for frozen tuples
because their values represent only 'visible for all transactions'.
True, but the hard part is getting rid of the storage for them.
I wrote a makeshift patch
Bruce Momjian pgman@candle.pha.pa.us writes:
Tom Lane wrote:
THe only fundamental disadvantage that COPY labors under is having to
write WAL records. It might be interesting to do something similar to
the recent hacks for CREATE TABLE AS, so that a COPY into a table just
created in the
Bruce Momjian pgman@candle.pha.pa.us writes:
Considering the cost/benefits, rather than doing some optimization for
long-lived tuples, I would like to see us merge the existing
xmin/xmax/cmin/cmax values back into three storage fields like we had in
7.4 and had to expand to a full four in 8.0
On Fri, Sep 02, 2005 at 04:02:08PM -0400, Tom Lane wrote:
Bruce Momjian pgman@candle.pha.pa.us writes:
Tom Lane wrote:
THe only fundamental disadvantage that COPY labors under is having to
write WAL records. It might be interesting to do something similar to
the recent hacks for CREATE
Tom, Alvaro,
It has to be a *new* table, not an *empty* table. If it's already
visible to other xacts then somebody else could insert into it in
parallel with you, because COPY doesn't take an exclusive lock.
There's still major gains to be had, for ETL, in being able to disable
logging
People:
Bruce Momjian pgman@candle.pha.pa.us writes:
Considering the cost/benefits, rather than doing some optimization for
long-lived tuples, I would like to see us merge the existing
xmin/xmax/cmin/cmax values back into three storage fields like we had
in 7.4 and had to expand to a
Alvaro Herrera [EMAIL PROTECTED] writes:
On Fri, Sep 02, 2005 at 04:02:08PM -0400, Tom Lane wrote:
It has to be a *new* table, not an *empty* table. If it's already
visible to other xacts then somebody else could insert into it in
parallel with you, because COPY doesn't take an exclusive
On Fri, Sep 02, 2005 at 04:27:59PM -0400, Tom Lane wrote:
Alvaro Herrera [EMAIL PROTECTED] writes:
Contrariwise, it doesn't really matter (I think) if there are WAL-logged
records already in the table and COPY is adding more that aren't logged.
Only if the page is locked in a fashion
On Fri, Sep 02, 2005 at 01:30:58PM -0700, Josh Berkus wrote:
Contrariwise, it doesn't really matter (I think) if there are WAL-logged
records already in the table and COPY is adding more that aren't logged.
(You might have to force COPY to start adding the rows on freshly added
pages ...
On Fri, Sep 02, 2005 at 01:35:42PM -0700, Josh Berkus wrote:
Bruce Momjian pgman@candle.pha.pa.us writes:
Considering the cost/benefits, rather than doing some optimization for
long-lived tuples, I would like to see us merge the existing
xmin/xmax/cmin/cmax values back into three
Tom Lane wrote:
Alvaro Herrera [EMAIL PROTECTED] writes:
On Fri, Sep 02, 2005 at 04:02:08PM -0400, Tom Lane wrote:
It has to be a *new* table, not an *empty* table. If it's already
visible to other xacts then somebody else could insert into it in
parallel with you, because COPY doesn't
Alvaro Herrera [EMAIL PROTECTED] writes:
On Fri, Sep 02, 2005 at 01:35:42PM -0700, Josh Berkus wrote:
Where I do see value is in enabling index-only access for frozen tables.
That would be a *huge* gain, especially with bitmaps. I think we've
discussed this before, though.
That's a
Bruce Momjian pgman@candle.pha.pa.us writes:
Updated TODO:
o Allow COPY on a newly-created table to skip WAL logging
On crash recovery, the table involved in the COPY would
have its heap and index files truncated. One issue is
that no other backend
Tom Lane wrote:
Bruce Momjian pgman@candle.pha.pa.us writes:
Updated TODO:
o Allow COPY on a newly-created table to skip WAL logging
On crash recovery, the table involved in the COPY would
have its heap and index files truncated. One issue is
On Fri, Sep 02, 2005 at 05:18:09PM -0400, Tom Lane wrote:
Bruce Momjian pgman@candle.pha.pa.us writes:
Updated TODO:
o Allow COPY on a newly-created table to skip WAL logging
On crash recovery, the table involved in the COPY would
have its heap and index
Alvaro Herrera [EMAIL PROTECTED] writes:
I remember somebody mentioned some time ago that if you create a table
and then crash before ending the transaction, the tuple in pg_class is
no longer valid, but the file remains.
Right --- it will be removed on transaction rollback, but not if the
On Fri, 2 Sep 2005 15:51:15 -0400 (EDT), Bruce Momjian
pgman@candle.pha.pa.us wrote:
* Merge xmin/xmax/cmin/cmax back into three header fields
And don't forget xvac, please.
Before subtransactions, there used to be only three fields needed to
store these four
Manfred Koizar wrote:
On Fri, 2 Sep 2005 15:51:15 -0400 (EDT), Bruce Momjian
pgman@candle.pha.pa.us wrote:
* Merge xmin/xmax/cmin/cmax back into three header fields
And don't forget xvac, please.
Before subtransactions, there used to be only three fields needed to
ITAGAKI Takahiro [EMAIL PROTECTED] writes:
I agree. I think an good position of freezer is on bgwriter.
My idea is:
1. Just before bgwriter writes an dirty page in LRU order,
2. Freeze tuples in the page and repair fragmentation.
3. (Replace the fsm page that has least freespace.)
4.
On Wed, Aug 31, 2005 at 09:14:42PM -0700, Josh Berkus wrote:
One thing that comes to mind is that this makes somewhat easier to build
a tool to write pre-built tables, for bulk-loading purposes. You just
construct the binary file with the HEAP_FROZEN bit set, and then attach
the file to
Alvaro Herrera [EMAIL PROTECTED] writes:
What I'm saying is that you can write a heap file, on which the tuples
would all have xmin=FrozenTransactionId, xmax=Invalid, and the
corresponding bits set in the infomask. This ensures that no matter the
state of the server, you can plug the file in
On Wed, 2005-08-31 at 22:25 -0400, Alvaro Herrera wrote:
On Thu, Sep 01, 2005 at 10:45:44AM +0900, ITAGAKI Takahiro wrote:
Hi,
I think it would be a waste to retain xmin and cmin for frozen tuples
because their values represent only 'visible for all transactions'.
Additionally, most
On Thu, Sep 01, 2005 at 11:08:36AM -0400, Tom Lane wrote:
Alvaro Herrera [EMAIL PROTECTED] writes:
What I'm saying is that you can write a heap file, on which the tuples
would all have xmin=FrozenTransactionId, xmax=Invalid, and the
corresponding bits set in the infomask. This ensures that
Alvaro,
What issues did he raise on this?
On having no Xmin.
What I'm saying is that you can write a heap file, on which the tuples
would all have xmin=FrozenTransactionId, xmax=Invalid, and the
corresponding bits set in the infomask. This ensures that no matter the
state of the server,
Tom,
THe only fundamental disadvantage that COPY labors under is having to
write WAL records. It might be interesting to do something similar to
the recent hacks for CREATE TABLE AS, so that a COPY into a table just
created in the current transaction would skip writing WAL and instead
fsync
On Thu, Sep 01, 2005 at 09:20:48AM -0700, Josh Berkus wrote:
What I'm saying is that you can write a heap file, on which the tuples
would all have xmin=FrozenTransactionId, xmax=Invalid, and the
corresponding bits set in the infomask. This ensures that no matter the
state of the server,
On Thu, Sep 01, 2005 at 04:34:09PM +0100, Simon Riggs wrote:
On Wed, 2005-08-31 at 22:25 -0400, Alvaro Herrera wrote:
I was thinking that when the tuple
needs to be obsoleted it would need to grow to accomodate the Xmax, but
you are not actually proposing to remove that, so it seems
Josh Berkus josh@agliodbs.com writes:
THe only fundamental disadvantage that COPY labors under is having to
write WAL records. It might be interesting to do something similar to
the recent hacks for CREATE TABLE AS, so that a COPY into a table just
created in the current transaction would
Alvaro Herrera [EMAIL PROTECTED] writes:
On Thu, Sep 01, 2005 at 04:34:09PM +0100, Simon Riggs wrote:
On Wed, 2005-08-31 at 22:25 -0400, Alvaro Herrera wrote:
Now, one thing of note is that you need to compress the page in order
to actually be able to use the just-freed space. VACUUM could do
Jim C. Nasby [EMAIL PROTECTED] writes:
On Thu, Sep 01, 2005 at 09:22:35AM -0400, Tom Lane wrote:
This is a bad idea. The bgwriter isn't the place to be doing freezing,
So is this something that another daemon could handle?
Possibly, but I'd be inclined to think of it as autovacuum's problem.
Hi Hackers,
I think it would be a waste to retain xmin and cmin for frozen tuples
because their values represent only 'visible for all transactions'.
Additionally, most tuples in database can be frozen potentially.
I wrote a makeshift patch to compress xmin and cmin (8bytes) to
1-bit flag, using
On Thu, Sep 01, 2005 at 10:45:44AM +0900, ITAGAKI Takahiro wrote:
Hi,
I think it would be a waste to retain xmin and cmin for frozen tuples
because their values represent only 'visible for all transactions'.
Additionally, most tuples in database can be frozen potentially.
I think this is an
ITAGAKI Takahiro [EMAIL PROTECTED] writes:
I think it would be a waste to retain xmin and cmin for frozen tuples
because their values represent only 'visible for all transactions'.
True, but the hard part is getting rid of the storage for them.
I wrote a makeshift patch to compress xmin and
Alvaro,
One thing that comes to mind is that this makes somewhat easier to build
a tool to write pre-built tables, for bulk-loading purposes. You just
construct the binary file with the HEAP_FROZEN bit set, and then attach
the file to a dummy table. (Then again, you can do it today, using a
Alvaro Herrera [EMAIL PROTECTED] wrote:
Now, one thing of note is that you need to compress the page in order
to actually be able to use the just-freed space. VACUUM could do that,
but maybe it would be better to do it on-line -- the freezing process is
going to have to write the page
Tom Lane [EMAIL PROTECTED] wrote:
I think this is incredibly ugly :-(.
Yes, I think so, too :-(My patch is product of the thought that
I don't want to modify codes widely. So if we want to do it more cool way,
lots of changes are needed as you said.
I'm also less than enthused about
54 matches
Mail list logo
