I'm on the way to open a ticket for hash indexes (adding WAL support) anyway:
May I open a ticket for adding GiST support to unlogged tables ?
Stefan
2011/9/14 Stefan Keller sfkel...@gmail.com:
Robert,
2011/9/6 Alexander Korotkov aekorot...@gmail.com:
GiST use serial numbers of operations
On Tue, Sep 13, 2011 at 5:00 PM, Stefan Keller sfkel...@gmail.com wrote:
2011/9/6 Alexander Korotkov aekorot...@gmail.com:
GiST use serial numbers of operations for concurrency. In current
implementation xlog record ids are used in capacity of that numbers. In
unlogged table no xlog records
Robert,
2011/9/6 Alexander Korotkov aekorot...@gmail.com:
GiST use serial numbers of operations for concurrency. In current
implementation xlog record ids are used in capacity of that numbers. In
unlogged table no xlog records are produced. So, we haven't serial numbers
of operations. AFAIK,
On 06.09.2011 01:18, Alexander Korotkov wrote:
Small bugfix: in gistBufferingFindCorrectParent check that downlinkoffnum
doesn't exceed maximal offset number.
I've committed the patch now, including that fix. Thanks for a great
GSoC project!
--
Heikki Linnakangas
EnterpriseDB
On Thu, Sep 8, 2011 at 10:59 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 06.09.2011 01:18, Alexander Korotkov wrote:
Small bugfix: in gistBufferingFindCorrectParent check that downlinkoffnum
doesn't exceed maximal offset number.
I've committed the patch now,
My congratulations too, Alexander ! Hope to work on SP-GiST together !
Oleg
On Thu, 8 Sep 2011, Heikki Linnakangas wrote:
On 06.09.2011 01:18, Alexander Korotkov wrote:
Small bugfix: in gistBufferingFindCorrectParent check that downlinkoffnum
doesn't exceed maximal offset number.
I've
Thanks for congratulations!
Tnanks to Heikki for mentoring and his work on patch!
--
With best regards,
Alexander Korotkov.
Hi,
Unlogged tables seems to me to follow a similar goal. Obviously GiST
indexes are not supported there.
Do you know the technical reason?
Do you see some synergy in your work on fast GiST index building and
unlogged tables?
Yours, Stefan
2011/9/6 Alexander Korotkov aekorot...@gmail.com:
Hi!
Unlogged tables seems to me to follow a similar goal. Obviously GiST
indexes are not supported there.
Do you know the technical reason?
GiST using serial numbers of operations for concurrency. In current
implementation xlog record ids are used in capacity of that numbers. In
unlogged
On 05.09.2011 14:10, Heikki Linnakangas wrote:
On 01.09.2011 12:23, Alexander Korotkov wrote:
On Thu, Sep 1, 2011 at 12:59 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
So I changed the test script to generate the table as:
CREATE TABLE points AS SELECT random() as x,
Small bugfix: in gistBufferingFindCorrectParent check that downlinkoffnum
doesn't exceed maximal offset number.
--
With best regards,
Alexander Korotkov.
gist_fast_build-0.14.3.patch.gz
Description: GNU Zip compressed data
--
Sent via pgsql-hackers mailing list
On 30.08.2011 13:38, Alexander Korotkov wrote:
On Tue, Aug 30, 2011 at 1:08 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Thanks. Meanwhile, I hacked together my own set of test scripts, and let
them run over the weekend. I'm still running tests with ordered data, but
On Thu, Sep 1, 2011 at 12:59 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
So I changed the test script to generate the table as:
CREATE TABLE points AS SELECT random() as x, random() as y FROM
generate_series(1, $NROWS);
The unordered results are in:
On 01.09.2011 12:23, Alexander Korotkov wrote:
On Thu, Sep 1, 2011 at 12:59 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
So I changed the test script to generate the table as:
CREATE TABLE points AS SELECT random() as x, random() as y FROM
generate_series(1, $NROWS);
On 26.08.2011 17:18, Alexander Korotkov wrote:
On Thu, Aug 25, 2011 at 11:08 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Could you share the test scripts, patches and data sets etc. needed to
reproduce the tests you've been running? I'd like to try them out on a test
On 30.08.2011 12:08, Heikki Linnakangas wrote:
What's going on here? This data set was large enough to not fit in RAM,
the table was about 8.5 GB in size (and I think the index is even larger
than that), and the box has 4GB of RAM. Does the buffering only help
with even larger indexes that
On Tue, Aug 30, 2011 at 1:13 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
So, over 50% of the CPU time is spent in choosing a block from the
temporary files. That should be pretty easy to improve..
Yes, probably we can just remove free blocks sorting.
--
With best
On Tue, Aug 30, 2011 at 1:08 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Thanks. Meanwhile, I hacked together my own set of test scripts, and let
them run over the weekend. I'm still running tests with ordered data, but
here are some preliminary results:
On 30.08.2011 13:29, Alexander Korotkov wrote:
On Tue, Aug 30, 2011 at 1:13 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
So, over 50% of the CPU time is spent in choosing a block from the
temporary files. That should be pretty easy to improve..
Yes, probably we can just
On 30.08.2011 13:38, Alexander Korotkov wrote:
On Tue, Aug 30, 2011 at 1:08 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Thanks. Meanwhile, I hacked together my own set of test scripts, and let
them run over the weekend. I'm still running tests with ordered data, but
On 30.08.2011 13:29, Alexander Korotkov wrote:
On Tue, Aug 30, 2011 at 1:13 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
So, over 50% of the CPU time is spent in choosing a block from the
temporary files. That should be pretty easy to improve..
Yes, probably we can just
On Tue, Aug 30, 2011 at 9:29 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 30.08.2011 13:29, Alexander Korotkov wrote:
On Tue, Aug 30, 2011 at 1:13 PM, Heikki Linnakangas
heikki.linnakangas@**enterprisedb.comheikki.linnakan...@enterprisedb.com
wrote:
So, over 50%
On 26.08.2011 17:18, Alexander Korotkov wrote:
On Thu, Aug 25, 2011 at 11:08 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Could you share the test scripts, patches and data sets etc. needed to
reproduce the tests you've been running? I'd like to try them out on a test
On Thu, Aug 25, 2011 at 11:08 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Could you share the test scripts, patches and data sets etc. needed to
reproduce the tests you've been running? I'd like to try them out on a test
server.
1) I've updated links to the datasets on
On Thu, Aug 25, 2011 at 10:53 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
In the tests on the first version of patch I found index quality of
regular
build much better than it of buffering build (without neighborrelocation).
Now it's similar, though it's because index
On 24.08.2011 16:57, Alexander Korotkov wrote:
I've added some testing results to the wiki page:
http://wiki.postgresql.org/wiki/Fast_GiST_index_build_GSoC_2011
There are not all the results I planned for the first chunk because it takes
more time than I expect.
Some notes about it.
Now I see
I've added some testing results to the wiki page:
http://wiki.postgresql.org/wiki/Fast_GiST_index_build_GSoC_2011
There are not all the results I planned for the first chunk because it takes
more time than I expect.
Some notes about it.
Now I see two causes which accelerate regular build of GiST
On Wed, Aug 17, 2011 at 11:11 AM, Alexander Korotkov
aekorot...@gmail.comwrote:
On Tue, Aug 16, 2011 at 11:15 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 16.08.2011 22:10, Heikki Linnakangas wrote:
Here's an version of the patch with a bunch of minor changes:
On Tue, Aug 16, 2011 at 11:15 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 16.08.2011 22:10, Heikki Linnakangas wrote:
Here's an version of the patch with a bunch of minor changes:
And here it really is, this time with an attachment...
Thanks a lot. I'm going to
I found that I forgot to remove levelstep and buffersize from reloptions.c.
Updated patch is attached.
--
With best regards,
Alexander Korotkov.
gist_fast_build-0.14.1.patch.gz
Description: GNU Zip compressed data
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To
Looking at the calculation of levelStep:
+ /*
+* Calculate levelStep by available amount of memory. We should be able
to
+* load into main memory one page of each underlying node buffer (which
+* are in levelStep below). That give constraint over
+*
On Tue, Aug 16, 2011 at 4:04 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
I can see that that's equal to the formula given in the paper, log_B(M/4B),
but I couldn't see any explanation for that formula in the paper. Your
explanation makes sense, but where did it come
Why is there ever a buffer on the root node? It seems like a waste of
time to load N tuples from the heap into the root buffer, only to empty
the buffer after it fills up. You might as well pull tuples directly
from the heap.
--
Heikki Linnakangas
EnterpriseDB
On Tue, Aug 16, 2011 at 9:43 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Why is there ever a buffer on the root node? It seems like a waste of time
to load N tuples from the heap into the root buffer, only to empty the
buffer after it fills up. You might as well pull
On 16.08.2011 21:46, Alexander Korotkov wrote:
On Tue, Aug 16, 2011 at 9:43 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Why is there ever a buffer on the root node? It seems like a waste of time
to load N tuples from the heap into the root buffer, only to empty the
Hi!
Thank you for your notes.
On Fri, Aug 12, 2011 at 7:04 PM, Robert Haas robertmh...@gmail.com wrote:
On Thu, Aug 11, 2011 at 6:21 AM, Alexander Korotkov
aekorot...@gmail.com wrote:
[ new patch ]
Some random comments:
- It appears that the noFollowFight flag is really supposed to be
On 11.08.2011 23:30, Alexander Korotkov wrote:
On Thu, Aug 11, 2011 at 2:28 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 10.08.2011 22:44, Alexander Korotkov wrote:
Manual and readme updates.
Thanks, I'm reviewing these now.
Do we want to expose the level-step
On Fri, Aug 12, 2011 at 12:23 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
I think it would also be fairly simple to decrease levelstep and/or adjust
buffersize on-the-fly. The trick would be in figuring out the heuristics on
when to do that.
I would be simple to
On Thu, Aug 11, 2011 at 6:21 AM, Alexander Korotkov
aekorot...@gmail.com wrote:
[ new patch ]
Some random comments:
- It appears that the noFollowFight flag is really supposed to be
called noFollowRight.
- In gist_private.h you've written halt-filled where you really mean
half-filled.
- It
Split of an internal node works like this:
1. Gather all the existing tuples on the page, plus the new tuple being
inserted.
2. Call picksplit on the tuples, to divide them into pages
3. Go through all tuples on the buffer associated with the page, and
divide them into buffers on the new
On Thu, Aug 11, 2011 at 10:21 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Split of an internal node works like this:
1. Gather all the existing tuples on the page, plus the new tuple being
inserted.
2. Call picksplit on the tuples, to divide them into pages
3. Go
On Wed, Aug 10, 2011 at 11:45 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
unloadNodeBuffers() is now dead code.
processEmptyingStack calls it.
LEAF_PAGES_STATS_* are unused now.
Removed.
Should avoid calling smgrnblocks() on every tuple, the overhead of that
could
On 10.08.2011 22:44, Alexander Korotkov wrote:
Manual and readme updates.
Thanks, I'm reviewing these now.
Do we want to expose the level-step and buffersize parameters to users?
They've been useful during testing, but I'm thinking we should be able
to guess good enough values for them
On 10.08.2011 22:44, Alexander Korotkov wrote:
Manual and readme updates.
I went through these, and did some editing and rewording. Attached is an
updated README, and an updated patch of the doc changes. Let me know if
I screwed up something.
--
Heikki Linnakangas
EnterpriseDB
On Thu, Aug 11, 2011 at 2:28 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 10.08.2011 22:44, Alexander Korotkov wrote:
Manual and readme updates.
Thanks, I'm reviewing these now.
Do we want to expose the level-step and buffersize parameters to users?
They've been
Hi!
Here is last verion of the patch.
List of changes:
1) Neighbor relocation and prefetch were removed. They will be supplied as
separate patches.
2) Final emptying now using standart lists of all buffers by levels.
3) Automatic switching again use simple comparison of index size and
Manual and readme updates.
--
With best regards,
Alexander Korotkov.
gist_fast_build-0.12.0.patch.gz
Description: GNU Zip compressed data
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
On 10.08.2011 13:19, Alexander Korotkov wrote:
Hi!
Here is last verion of the patch.
List of changes:
1) Neighbor relocation and prefetch were removed. They will be supplied as
separate patches.
unloadNodeBuffers() is now dead code.
2) Final emptying now using standart lists of all buffers
On 07.08.2011 22:28, Alexander Korotkov wrote:
There is last version of patch. There is the list of most significant
changes in comparison with your version of patch:
1) Reference counting of path items was added. It should helps against
possible accumulation of path items.
Ok.
2) Neighbor
On Mon, Aug 8, 2011 at 1:23 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
2) Neighbor relocation was added.
Ok. I think we're going to need some sort of a heuristic on when to enable
neighbor relocation. If I remember the performance tests correctly, it
improves the
Hi!
There is last version of patch. There is the list of most significant
changes in comparison with your version of patch:
1) Reference counting of path items was added. It should helps against
possible accumulation of path items.
2) Neighbor relocation was added.
3) Subtree prefetching was
Uhh, my bad, really stupid bug. Many thanks.
--
With best regards,
Alexander Korotkov.
On Wed, Aug 3, 2011 at 8:31 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 03.08.2011 11:18, Alexander Korotkov wrote:
I found that in previous version of patch I missed
I found that in previous version of patch I missed PageSetLSN
and PageSetTLI, but huge amount of WAL is still here. Also I found that huge
amount of WAL appears only with -O2. With -O0 amount of WAL is ok, but
messages FATAL: xlog flush request BFF11148/809A600 is not satisfied ---
flushed only
On Wed, Aug 3, 2011 at 4:18 AM, Alexander Korotkov aekorot...@gmail.com wrote:
I found that in previous version of patch I missed PageSetLSN
and PageSetTLI, but huge amount of WAL is still here. Also I found that huge
amount of WAL appears only with -O2. With -O0 amount of WAL is ok, but
On 03.08.2011 11:18, Alexander Korotkov wrote:
I found that in previous version of patch I missed PageSetLSN
and PageSetTLI, but huge amount of WAL is still here. Also I found that huge
amount of WAL appears only with -O2. With -O0 amount of WAL is ok, but
messages FATAL: xlog flush request
On 01.08.2011 13:44, Heikki Linnakangas wrote:
On 01.08.2011 13:13, Simon Riggs wrote:
Did you want me to write the patch for 9.0?
I'm looking at it now.
So, in 9.0, we currently leave the rightlink and NSN invalid when
replaying a page split. To set them correctly, we'd need the old
On Tue, Aug 2, 2011 at 12:03 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 01.08.2011 13:44, Heikki Linnakangas wrote:
On 01.08.2011 13:13, Simon Riggs wrote:
Did you want me to write the patch for 9.0?
I'm looking at it now.
So, in 9.0, we currently leave the
On 02.08.2011 14:36, Simon Riggs wrote:
On Tue, Aug 2, 2011 at 12:03 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
If we change the WAL record, we have to make it so that the new version can
still read the old format, which complicates the implementation a bit.
Neverthelss,
On Tue, Aug 2, 2011 at 12:43 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 02.08.2011 14:36, Simon Riggs wrote:
On Tue, Aug 2, 2011 at 12:03 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
If we change the WAL record, we have to make it so that the
Hi!
I'm now working on adding features to your version of patch. Current version
is attached. Somehow this version produce huge amount of WAL and that makes
it slow. Though count and avg. length of WAL records is similar to that of
non-buffering build.
test=# create table points as (select
On 02.08.2011 15:18, Simon Riggs wrote:
On Tue, Aug 2, 2011 at 12:43 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 02.08.2011 14:36, Simon Riggs wrote:
Actually I think we can append the new information to the end of the page
split record, so that an old version server
On 02.08.2011 20:06, Alvaro Herrera wrote:
Excerpts from Heikki Linnakangas's message of mar ago 02 11:59:24 -0400 2011:
On 02.08.2011 15:18, Simon Riggs wrote:
On Tue, Aug 2, 2011 at 12:43 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 02.08.2011 14:36, Simon Riggs
On 27.07.2011 17:43, Alexander Korotkov wrote:
OK, thanks. I also found behaviour of GiST(without patch) with streaming
replication that seems strange for me. On master there are only few
rightlinks are InvalidBlockNumber while on slave there are a lot of them. I
hack gevel for getting index
On Mon, Aug 1, 2011 at 10:38 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 27.07.2011 17:43, Alexander Korotkov wrote:
OK, thanks. I also found behaviour of GiST(without patch) with streaming
replication that seems strange for me. On master there are only few
rightlinks
On 01.08.2011 13:13, Simon Riggs wrote:
On Mon, Aug 1, 2011 at 10:38 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Attached is a patch for that for 9.1/master. The 9.0 GiST replay code was
quite different, it will require a separate patch.
Hmm, I was assured no changes
On 1 August 2011 11:44, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 01.08.2011 13:13, Simon Riggs wrote:
And what does NSN stand for? :-)
Hmm, I don't know actually know what NSN is an acronym for :-).
Node Sequence Number.
--
Thom Brown
Twitter: @darkixion
IRC
On Mon, Aug 1, 2011 at 11:47 AM, Thom Brown t...@linux.com wrote:
On 1 August 2011 11:44, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 01.08.2011 13:13, Simon Riggs wrote:
And what does NSN stand for? :-)
Hmm, I don't know actually know what NSN is an acronym for :-).
On 1 August 2011 12:25, Simon Riggs si...@2ndquadrant.com wrote:
On Mon, Aug 1, 2011 at 11:47 AM, Thom Brown t...@linux.com wrote:
On 1 August 2011 11:44, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 01.08.2011 13:13, Simon Riggs wrote:
And what does NSN stand for? :-)
On Mon, Aug 1, 2011 at 3:25 PM, Simon Riggs si...@2ndquadrant.com wrote:
On Mon, Aug 1, 2011 at 11:47 AM, Thom Brown t...@linux.com wrote:
On 1 August 2011 11:44, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 01.08.2011 13:13, Simon Riggs wrote:
And what does NSN stand
On Mon, Aug 1, 2011 at 11:44 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Does the order of locking of the buffers matter? I'm sure it does.
Yep.
Do you mean that the BlockNumbers are already in correct sequence, or
that you will be adding this code to redo?
--
Simon
On 01.08.2011 14:35, Simon Riggs wrote:
On Mon, Aug 1, 2011 at 11:44 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Does the order of locking of the buffers matter? I'm sure it does.
Yep.
Do you mean that the BlockNumbers are already in correct sequence, or
that you
On Mon, Aug 1, 2011 at 2:29 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 01.08.2011 14:35, Simon Riggs wrote:
On Mon, Aug 1, 2011 at 11:44 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Does the order of locking of the buffers matter? I'm sure it
On 01.08.2011 17:26, Simon Riggs wrote:
On Mon, Aug 1, 2011 at 2:29 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
I believe we code acquire the locks in right order already, and the patch I
posted fixes the premature release of locks at page split.
Your patch is good, but
On Mon, Aug 1, 2011 at 3:34 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 01.08.2011 17:26, Simon Riggs wrote:
On Mon, Aug 1, 2011 at 2:29 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
I believe we code acquire the locks in right order already, and
On Tue, Jul 26, 2011 at 9:24 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Let me know if you have questions on the approach taken.
I realized that approach which comes as replace to loaded-subtrees keeping
is unclear to me. We save paths between node buffers. But those
On 28.07.2011 23:57, Alexander Korotkov wrote:
On Tue, Jul 26, 2011 at 9:24 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Let me know if you have questions on the approach taken.
I realized that approach which comes as replace to loaded-subtrees keeping
is unclear to
On Fri, Jul 29, 2011 at 1:10 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
gistFindCorrectParent() should take care of that.
OK, now it seems that I understood. I need to verify amount memory needed
for paths because it seems that they tends to accumulate. Also I need to
I found a problem in WAL with this patch. I use simplified insert algorithm
in my patch which don't insert downlink one by one but insert them at once.
Thus FollowRight flag is leaving uncleared when redoing from WAL, because
only one flag can be cleared by one WAL record. Do you think
On 27.07.2011 15:29, Alexander Korotkov wrote:
I found a problem in WAL with this patch. I use simplified insert algorithm
in my patch which don't insert downlink one by one but insert them at once.
Thus FollowRight flag is leaving uncleared when redoing from WAL, because
only one flag can be
On Wed, Jul 27, 2011 at 6:05 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Dunno, both approaches seem reasonable to me. There's no rule against
changing WAL record structure across major releases, if that's what you were
worried about.
OK, thanks. I also found behaviour
On 27.07.2011 17:43, Alexander Korotkov wrote:
1(l:1) blk: 324 numTuple: 129 free: 2472b(69.71%) rightlink:4294967295
(InvalidBlockNumber)
1(l:2) blk: 242 numTuple: 164 free: 932b(88.58%)
rightlink:4294967295 (InvalidBlockNumber)
2(l:2) blk: 525 numTuple: 121 free:
On Tue, Jul 26, 2011 at 9:24 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
That was a quite off-the-cuff remark, so I took the patch and culled out
loaded-tree bookkeeping. A lot of other changes fell off from that, so it
took me quite some time to get it working again,
On 22.07.2011 12:38, Alexander Korotkov wrote:
Patch with my try to detect ordered datasets is attached. The implemented
idea is desribed below.
Index tuples are divided by chunks of 128. On each chunk we measure how much
leaf pages where index tuples was inserted don't match those of previous
Hi!
Patch with my try to detect ordered datasets is attached. The implemented
idea is desribed below.
Index tuples are divided by chunks of 128. On each chunk we measure how much
leaf pages where index tuples was inserted don't match those of previous
chunk. Based on statistics of several chunks
Hi!
New version of patch is attached. There are following changes.
1) Since proposed tchnique is not always a fast build, it was renamed
everywhere in the patch to buffering build.
2) Parameter buffering now has 3 possible values yes, no and auto.
auto means automatic switching from regular index
Fri, Jul 15, 2011 at 12:53 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 14.07.2011 23:41, Alexander Korotkov wrote:
Do you think using rightlink as pointer to parent page is possible
during
index build? It would allow to simplify code significantly, because of no
On Thu, Jul 14, 2011 at 12:42 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Pinning a buffer that's already in the shared buffer cache is cheap, I
doubt you're gaining much by keeping the private hash table in front of the
buffer cache.
Yes, I see. Pinning a lot of
Do you think using rightlink as pointer to parent page is possible during
index build? It would allow to simplify code significantly, because of no
more need to maintain in-memory structures for parents memorizing.
--
With best regards,
Alexander Korotkov.
On 14.07.2011 23:41, Alexander Korotkov wrote:
Do you think using rightlink as pointer to parent page is possible during
index build? It would allow to simplify code significantly, because of no
more need to maintain in-memory structures for parents memorizing.
I guess, but where do you store
On Wed, Jul 13, 2011 at 5:59 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
One thing that caught my eye is that when you empty a buffer, you load the
entire subtree below that buffer, down to the next buffered or leaf level,
into memory. Every page in that subtree is kept
On 14.07.2011 11:33, Alexander Korotkov wrote:
On Wed, Jul 13, 2011 at 5:59 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
One thing that caught my eye is that when you empty a buffer, you load the
entire subtree below that buffer, down to the next buffered or leaf level,
Hi,
Looking at the performance test results again on the wiki page
(http://wiki.postgresql.org/wiki/Fast_GiST_index_build_GSoC_2011#Testing_results),
the patch can be summarized like this: it reduces the number of disk
accesses, at the cost of some extra CPU work.
Is it possible to switch
Hi!
On Wed, Jul 13, 2011 at 12:33 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Is it possible to switch to the new buffering method in the middle of an
index build? We could use the plain insertion method until the index grows
to a certain size (effective_cache_size?),
On Wed, Jul 13, 2011 at 12:40 PM, Alexander Korotkov
aekorot...@gmail.comwrote:
On Wed, Jul 13, 2011 at 12:33 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Is it possible to switch to the new buffering method in the middle of an
index build? We could use the plain
On 12.07.2011 11:34, Alexander Korotkov wrote:
New version of patch with a little more refactoring and comments.
Great! The README helps tremendously to understand this, thanks for that.
One thing that caught my eye is that when you empty a buffer, you load
the entire subtree below that
On Fri, Jul 8, 2011 at 6:18 PM, Tom Lane t...@sss.pgh.pa.us wrote:
For test purposes, you could turn off synchronize_seqscans to prevent
that.
Thanks, it helps. I'm rerunning tests now.
--
With best regards,
Alexander Korotkov.
New version of patch with a little more refactoring and comments.
--
With best regards,
Alexander Korotkov.
gist_fast_build-0.6.0.patch.gz
Description: GNU Zip compressed data
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
I found that results of previous tests with USNOA2 were not fully correct.
Tables for tests with various parameters contained tuples in different
order. I assumed that query
CREATE TABLE tab2 AS (SELECT * FROM tab1);
creates tab2 as exact copy of tab1, i.e. tab2 contain tuples in same order
as
Alexander Korotkov aekorot...@gmail.com writes:
I found that results of previous tests with USNOA2 were not fully correct.
Tables for tests with various parameters contained tuples in different
order. I assumed that query
CREATE TABLE tab2 AS (SELECT * FROM tab1);
creates tab2 as exact copy
New version of patch with readme and some bugfixes.
Some new tests with fast build parameters variations are on the wiki page.
While I doubt to interpret some results of tests, because it looks strange
for me. I particular I can't explain why decrease of buffer size affects
index quality so much
1 - 100 of 136 matches
Mail list logo
