On Tue, Sep 13, 2011 at 5:00 PM, Stefan Keller wrote:
> 2011/9/6 Alexander Korotkov :
>> GiST use serial numbers of operations for concurrency. In current
>> implementation xlog record ids are used in capacity of that numbers. In
>> unlogged table no xlog records are produced. So, we haven't seria
I'm on the way to open a ticket for hash indexes (adding WAL support) anyway:
May I open a ticket for adding GiST support to unlogged tables ?
Stefan
2011/9/14 Stefan Keller :
> Robert,
>
> 2011/9/6 Alexander Korotkov :
>> GiST use serial numbers of operations for concurrency. In current
>> imple
Robert,
2011/9/6 Alexander Korotkov :
> GiST use serial numbers of operations for concurrency. In current
> implementation xlog record ids are used in capacity of that numbers. In
> unlogged table no xlog records are produced. So, we haven't serial numbers
> of operations. AFAIK, it's enough to pr
Thanks for congratulations!
Tnanks to Heikki for mentoring and his work on patch!
--
With best regards,
Alexander Korotkov.
My congratulations too, Alexander ! Hope to work on SP-GiST together !
Oleg
On Thu, 8 Sep 2011, Heikki Linnakangas wrote:
On 06.09.2011 01:18, Alexander Korotkov wrote:
Small bugfix: in gistBufferingFindCorrectParent check that downlinkoffnum
doesn't exceed maximal offset number.
I've commi
On Thu, Sep 8, 2011 at 10:59 AM, Heikki Linnakangas
wrote:
> On 06.09.2011 01:18, Alexander Korotkov wrote:
>>
>> Small bugfix: in gistBufferingFindCorrectParent check that downlinkoffnum
>> doesn't exceed maximal offset number.
>
> I've committed the patch now, including that fix. Thanks for a gr
On 06.09.2011 01:18, Alexander Korotkov wrote:
Small bugfix: in gistBufferingFindCorrectParent check that downlinkoffnum
doesn't exceed maximal offset number.
I've committed the patch now, including that fix. Thanks for a great
GSoC project!
--
Heikki Linnakangas
EnterpriseDB http://ww
Hi!
> Unlogged tables seems to me to follow a similar goal. Obviously GiST
> indexes are not supported there.
> Do you know the technical reason?
>
GiST using serial numbers of operations for concurrency. In current
implementation xlog record ids are used in capacity of that numbers. In
unlogged
Hi,
Unlogged tables seems to me to follow a similar goal. Obviously GiST
indexes are not supported there.
Do you know the technical reason?
Do you see some synergy in your work on fast GiST index building and
unlogged tables?
Yours, Stefan
2011/9/6 Alexander Korotkov :
> Small bugfix: in gistBuf
Small bugfix: in gistBufferingFindCorrectParent check that downlinkoffnum
doesn't exceed maximal offset number.
--
With best regards,
Alexander Korotkov.
gist_fast_build-0.14.3.patch.gz
Description: GNU Zip compressed data
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or
On 05.09.2011 14:10, Heikki Linnakangas wrote:
On 01.09.2011 12:23, Alexander Korotkov wrote:
On Thu, Sep 1, 2011 at 12:59 PM, Heikki Linnakangas<
heikki.linnakan...@enterprisedb.com> wrote:
So I changed the test script to generate the table as:
CREATE TABLE points AS SELECT random() as x, ra
On 01.09.2011 12:23, Alexander Korotkov wrote:
On Thu, Sep 1, 2011 at 12:59 PM, Heikki Linnakangas<
heikki.linnakan...@enterprisedb.com> wrote:
So I changed the test script to generate the table as:
CREATE TABLE points AS SELECT random() as x, random() as y FROM
generate_series(1, $NROWS);
T
On Thu, Sep 1, 2011 at 12:59 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> So I changed the test script to generate the table as:
>
> CREATE TABLE points AS SELECT random() as x, random() as y FROM
> generate_series(1, $NROWS);
>
> The unordered results are in:
>
>
On 30.08.2011 13:38, Alexander Korotkov wrote:
On Tue, Aug 30, 2011 at 1:08 PM, Heikki Linnakangas<
heikki.linnakan...@enterprisedb.com> wrote:
Thanks. Meanwhile, I hacked together my own set of test scripts, and let
them run over the weekend. I'm still running tests with ordered data, but
he
On Tue, Aug 30, 2011 at 9:29 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> On 30.08.2011 13:29, Alexander Korotkov wrote:
>
>> On Tue, Aug 30, 2011 at 1:13 PM, Heikki Linnakangas<
>> heikki.linnakangas@**enterprisedb.com>
>> wrote:
>>
>> So, over 50% of the CPU time is s
On 30.08.2011 13:29, Alexander Korotkov wrote:
On Tue, Aug 30, 2011 at 1:13 PM, Heikki Linnakangas<
heikki.linnakan...@enterprisedb.com> wrote:
So, over 50% of the CPU time is spent in choosing a block from the
temporary files. That should be pretty easy to improve..
Yes, probably we can jus
On 30.08.2011 13:38, Alexander Korotkov wrote:
On Tue, Aug 30, 2011 at 1:08 PM, Heikki Linnakangas<
heikki.linnakan...@enterprisedb.com> wrote:
Thanks. Meanwhile, I hacked together my own set of test scripts, and let
them run over the weekend. I'm still running tests with ordered data, but
he
On 30.08.2011 13:29, Alexander Korotkov wrote:
On Tue, Aug 30, 2011 at 1:13 PM, Heikki Linnakangas<
heikki.linnakan...@enterprisedb.com> wrote:
So, over 50% of the CPU time is spent in choosing a block from the
temporary files. That should be pretty easy to improve..
Yes, probably we can jus
On Tue, Aug 30, 2011 at 1:08 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
>
> Thanks. Meanwhile, I hacked together my own set of test scripts, and let
> them run over the weekend. I'm still running tests with ordered data, but
> here are some preliminary results:
>
>
On Tue, Aug 30, 2011 at 1:13 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> So, over 50% of the CPU time is spent in choosing a block from the
> temporary files. That should be pretty easy to improve..
>
Yes, probably we can just remove free blocks sorting.
--
With bes
On 30.08.2011 12:08, Heikki Linnakangas wrote:
What's going on here? This data set was large enough to not fit in RAM,
the table was about 8.5 GB in size (and I think the index is even larger
than that), and the box has 4GB of RAM. Does the buffering only help
with even larger indexes that exceed
On 26.08.2011 17:18, Alexander Korotkov wrote:
On Thu, Aug 25, 2011 at 11:08 PM, Heikki Linnakangas<
heikki.linnakan...@enterprisedb.com> wrote:
Could you share the test scripts, patches and data sets etc. needed to
reproduce the tests you've been running? I'd like to try them out on a test
se
On 26.08.2011 17:18, Alexander Korotkov wrote:
On Thu, Aug 25, 2011 at 11:08 PM, Heikki Linnakangas<
heikki.linnakan...@enterprisedb.com> wrote:
Could you share the test scripts, patches and data sets etc. needed to
reproduce the tests you've been running? I'd like to try them out on a test
se
On Thu, Aug 25, 2011 at 10:53 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
>
>> In the tests on the first version of patch I found index quality of
>> regular
>> build much better than it of buffering build (without neighborrelocation).
>> Now it's similar, though it's beca
On Thu, Aug 25, 2011 at 11:08 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> Could you share the test scripts, patches and data sets etc. needed to
> reproduce the tests you've been running? I'd like to try them out on a test
> server.
>
1) I've updated links to the datase
On 24.08.2011 16:57, Alexander Korotkov wrote:
I've added some testing results to the wiki page:
http://wiki.postgresql.org/wiki/Fast_GiST_index_build_GSoC_2011
There are not all the results I planned for the first chunk because it takes
more time than I expect.
Some notes about it.
Now I see tw
I've added some testing results to the wiki page:
http://wiki.postgresql.org/wiki/Fast_GiST_index_build_GSoC_2011
There are not all the results I planned for the first chunk because it takes
more time than I expect.
Some notes about it.
Now I see two causes which accelerate regular build of GiST i
On Wed, Aug 17, 2011 at 11:11 AM, Alexander Korotkov
wrote:
> On Tue, Aug 16, 2011 at 11:15 PM, Heikki Linnakangas <
> heikki.linnakan...@enterprisedb.com> wrote:
>
>> On 16.08.2011 22:10, Heikki Linnakangas wrote:
>>
>>> Here's an version of the patch with a bunch of minor changes:
>>>
>>
>> And
On Tue, Aug 16, 2011 at 11:15 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> On 16.08.2011 22:10, Heikki Linnakangas wrote:
>
>> Here's an version of the patch with a bunch of minor changes:
>>
>
> And here it really is, this time with an attachment...
Thanks a lot. I'm go
On 16.08.2011 21:46, Alexander Korotkov wrote:
On Tue, Aug 16, 2011 at 9:43 PM, Heikki Linnakangas<
heikki.linnakan...@enterprisedb.com> wrote:
Why is there ever a buffer on the root node? It seems like a waste of time
to load N tuples from the heap into the root buffer, only to empty the
buff
On Tue, Aug 16, 2011 at 9:43 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> Why is there ever a buffer on the root node? It seems like a waste of time
> to load N tuples from the heap into the root buffer, only to empty the
> buffer after it fills up. You might as well pull
Why is there ever a buffer on the root node? It seems like a waste of
time to load N tuples from the heap into the root buffer, only to empty
the buffer after it fills up. You might as well pull tuples directly
from the heap.
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
On Tue, Aug 16, 2011 at 4:04 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> I can see that that's equal to the formula given in the paper, log_B(M/4B),
> but I couldn't see any explanation for that formula in the paper. Your
> explanation makes sense, but where did it come
Looking at the calculation of levelStep:
+ /*
+* Calculate levelStep by available amount of memory. We should be able
to
+* load into main memory one page of each underlying node buffer (which
+* are in levelStep below). That give constraint over
+* mainten
I found that I forgot to remove levelstep and buffersize from reloptions.c.
Updated patch is attached.
--
With best regards,
Alexander Korotkov.
gist_fast_build-0.14.1.patch.gz
Description: GNU Zip compressed data
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To mak
Hi!
Thank you for your notes.
On Fri, Aug 12, 2011 at 7:04 PM, Robert Haas wrote:
> On Thu, Aug 11, 2011 at 6:21 AM, Alexander Korotkov
> wrote:
> > [ new patch ]
>
> Some random comments:
>
> - It appears that the "noFollowFight" flag is really supposed to be
> called "noFollowRight".
>
Fixed
On Thu, Aug 11, 2011 at 6:21 AM, Alexander Korotkov
wrote:
> [ new patch ]
Some random comments:
- It appears that the "noFollowFight" flag is really supposed to be
called "noFollowRight".
- In gist_private.h you've written "halt-filled" where you really mean
"half-filled".
- It seems like you
On Fri, Aug 12, 2011 at 12:23 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> I think it would also be fairly simple to decrease levelstep and/or adjust
> buffersize on-the-fly. The trick would be in figuring out the heuristics on
> when to do that.
>
I would be simple to de
On 11.08.2011 23:30, Alexander Korotkov wrote:
On Thu, Aug 11, 2011 at 2:28 PM, Heikki Linnakangas<
heikki.linnakan...@enterprisedb.com> wrote:
On 10.08.2011 22:44, Alexander Korotkov wrote:
Manual and readme updates.
Thanks, I'm reviewing these now.
Do we want to expose the level-step a
On Thu, Aug 11, 2011 at 2:28 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> On 10.08.2011 22:44, Alexander Korotkov wrote:
>
>> Manual and readme updates.
>>
>
> Thanks, I'm reviewing these now.
>
> Do we want to expose the level-step and buffersize parameters to users?
> T
On 10.08.2011 22:44, Alexander Korotkov wrote:
Manual and readme updates.
I went through these, and did some editing and rewording. Attached is an
updated README, and an updated patch of the doc changes. Let me know if
I screwed up something.
--
Heikki Linnakangas
EnterpriseDB http://
On 10.08.2011 22:44, Alexander Korotkov wrote:
Manual and readme updates.
Thanks, I'm reviewing these now.
Do we want to expose the level-step and buffersize parameters to users?
They've been useful during testing, but I'm thinking we should be able
to guess good enough values for them autom
On Wed, Aug 10, 2011 at 11:45 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> unloadNodeBuffers() is now dead code.
>
processEmptyingStack calls it.
LEAF_PAGES_STATS_* are unused now.
Removed.
> Should avoid calling smgrnblocks() on every tuple, the overhead of that
> co
On Thu, Aug 11, 2011 at 10:21 AM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> Split of an internal node works like this:
>
> 1. Gather all the existing tuples on the page, plus the new tuple being
> inserted.
> 2. Call picksplit on the tuples, to divide them into pages
> 3.
Split of an internal node works like this:
1. Gather all the existing tuples on the page, plus the new tuple being
inserted.
2. Call picksplit on the tuples, to divide them into pages
3. Go through all tuples on the buffer associated with the page, and
divide them into buffers on the new pages
On 10.08.2011 13:19, Alexander Korotkov wrote:
Hi!
Here is last verion of the patch.
List of changes:
1) Neighbor relocation and prefetch were removed. They will be supplied as
separate patches.
unloadNodeBuffers() is now dead code.
2) Final emptying now using standart lists of all buffers b
Manual and readme updates.
--
With best regards,
Alexander Korotkov.
gist_fast_build-0.12.0.patch.gz
Description: GNU Zip compressed data
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hac
Hi!
Here is last verion of the patch.
List of changes:
1) Neighbor relocation and prefetch were removed. They will be supplied as
separate patches.
2) Final emptying now using standart lists of all buffers by levels.
3) Automatic switching again use simple comparison of index size and
effective_ca
On Mon, Aug 8, 2011 at 1:23 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
>
> 2) Neighbor relocation was added.
>>
>
> Ok. I think we're going to need some sort of a heuristic on when to enable
> neighbor relocation. If I remember the performance tests correctly, it
> improve
On 07.08.2011 22:28, Alexander Korotkov wrote:
There is last version of patch. There is the list of most significant
changes in comparison with your version of patch:
1) Reference counting of path items was added. It should helps against
possible accumulation of path items.
Ok.
2) Neighbor re
Hi!
There is last version of patch. There is the list of most significant
changes in comparison with your version of patch:
1) Reference counting of path items was added. It should helps against
possible accumulation of path items.
2) Neighbor relocation was added.
3) Subtree prefetching was added
Uhh, my bad, really stupid bug. Many thanks.
--
With best regards,
Alexander Korotkov.
On Wed, Aug 3, 2011 at 8:31 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> On 03.08.2011 11:18, Alexander Korotkov wrote:
>
>> I found that in previous version of patch I missed Pag
On 03.08.2011 11:18, Alexander Korotkov wrote:
I found that in previous version of patch I missed PageSetLSN
and PageSetTLI, but huge amount of WAL is still here. Also I found that huge
amount of WAL appears only with -O2. With -O0 amount of WAL is ok, but
messages "FATAL: xlog flush request BFF
On Wed, Aug 3, 2011 at 4:18 AM, Alexander Korotkov wrote:
> I found that in previous version of patch I missed PageSetLSN
> and PageSetTLI, but huge amount of WAL is still here. Also I found that huge
> amount of WAL appears only with -O2. With -O0 amount of WAL is ok, but
> messages "FATAL: xlog
I found that in previous version of patch I missed PageSetLSN
and PageSetTLI, but huge amount of WAL is still here. Also I found that huge
amount of WAL appears only with -O2. With -O0 amount of WAL is ok, but
messages "FATAL: xlog flush request BFF11148/809A600 is not satisfied ---
flushed only t
On 02.08.2011 20:06, Alvaro Herrera wrote:
Excerpts from Heikki Linnakangas's message of mar ago 02 11:59:24 -0400 2011:
On 02.08.2011 15:18, Simon Riggs wrote:
On Tue, Aug 2, 2011 at 12:43 PM, Heikki Linnakangas
wrote:
On 02.08.2011 14:36, Simon Riggs wrote:
Actually I think we can append
On 02.08.2011 15:18, Simon Riggs wrote:
On Tue, Aug 2, 2011 at 12:43 PM, Heikki Linnakangas
wrote:
On 02.08.2011 14:36, Simon Riggs wrote:
Actually I think we can append the new information to the end of the page
split record, so that an old version server can read WAL generated by new
version
Hi!
I'm now working on adding features to your version of patch. Current version
is attached. Somehow this version produce huge amount of WAL and that makes
it slow. Though count and avg. length of WAL records is similar to that of
non-buffering build.
test=# create table points as (select point(
On Tue, Aug 2, 2011 at 12:43 PM, Heikki Linnakangas
wrote:
> On 02.08.2011 14:36, Simon Riggs wrote:
>>
>> On Tue, Aug 2, 2011 at 12:03 PM, Heikki Linnakangas
>> wrote:
>>>
>>> If we change the WAL record, we have to make it so that the new version
>>> can
>>> still read the old format, which co
On 02.08.2011 14:36, Simon Riggs wrote:
On Tue, Aug 2, 2011 at 12:03 PM, Heikki Linnakangas
wrote:
If we change the WAL record, we have to make it so that the new version can
still read the old format, which complicates the implementation a bit.
Neverthelss, I'm leaning towards option 1.
We
On Tue, Aug 2, 2011 at 12:03 PM, Heikki Linnakangas
wrote:
> On 01.08.2011 13:44, Heikki Linnakangas wrote:
>>
>> On 01.08.2011 13:13, Simon Riggs wrote:
>>>
>>> Did you want me to write the patch for 9.0?
>>
>> I'm looking at it now.
>
> So, in 9.0, we currently leave the rightlink and NSN invali
On 01.08.2011 13:44, Heikki Linnakangas wrote:
On 01.08.2011 13:13, Simon Riggs wrote:
Did you want me to write the patch for 9.0?
I'm looking at it now.
So, in 9.0, we currently leave the rightlink and NSN invalid when
replaying a page split. To set them correctly, we'd need the old
right
On Mon, Aug 1, 2011 at 3:34 PM, Heikki Linnakangas
wrote:
> On 01.08.2011 17:26, Simon Riggs wrote:
>>
>> On Mon, Aug 1, 2011 at 2:29 PM, Heikki Linnakangas
>> wrote:
>>>
>>> I believe we code acquire the locks in right order already, and the patch
>>> I
>>> posted fixes the premature release of
On 01.08.2011 17:26, Simon Riggs wrote:
On Mon, Aug 1, 2011 at 2:29 PM, Heikki Linnakangas
wrote:
I believe we code acquire the locks in right order already, and the patch I
posted fixes the premature release of locks at page split.
Your patch is good, but it does rely on the idea that we're
On Mon, Aug 1, 2011 at 2:29 PM, Heikki Linnakangas
wrote:
> On 01.08.2011 14:35, Simon Riggs wrote:
>>
>> On Mon, Aug 1, 2011 at 11:44 AM, Heikki Linnakangas
>> wrote:
>>
Does the order of locking of the buffers matter? I'm sure it does.
>>>
>>> Yep.
>>
>> Do you mean that the BlockNumbers
On 01.08.2011 14:35, Simon Riggs wrote:
On Mon, Aug 1, 2011 at 11:44 AM, Heikki Linnakangas
wrote:
Does the order of locking of the buffers matter? I'm sure it does.
Yep.
Do you mean that the BlockNumbers are already in correct sequence, or
that you will be adding this code to redo?
I j
On Mon, Aug 1, 2011 at 11:44 AM, Heikki Linnakangas
wrote:
>> Does the order of locking of the buffers matter? I'm sure it does.
>
> Yep.
Do you mean that the BlockNumbers are already in correct sequence, or
that you will be adding this code to redo?
--
Simon Riggs http://ww
On Mon, Aug 1, 2011 at 3:25 PM, Simon Riggs wrote:
> On Mon, Aug 1, 2011 at 11:47 AM, Thom Brown wrote:
> > On 1 August 2011 11:44, Heikki Linnakangas
> > wrote:
> >> On 01.08.2011 13:13, Simon Riggs wrote:
> >>> And what does NSN stand for? :-)
> >>
> >> Hmm, I don't know actually know what NS
On 1 August 2011 12:25, Simon Riggs wrote:
> On Mon, Aug 1, 2011 at 11:47 AM, Thom Brown wrote:
>> On 1 August 2011 11:44, Heikki Linnakangas
>> wrote:
>>> On 01.08.2011 13:13, Simon Riggs wrote:
And what does NSN stand for? :-)
>>>
>>> Hmm, I don't know actually know what NSN is an acronym
On Mon, Aug 1, 2011 at 11:47 AM, Thom Brown wrote:
> On 1 August 2011 11:44, Heikki Linnakangas
> wrote:
>> On 01.08.2011 13:13, Simon Riggs wrote:
>>> And what does NSN stand for? :-)
>>
>> Hmm, I don't know actually know what NSN is an acronym for :-).
>
> Node Sequence Number.
Do you have a r
On 1 August 2011 11:44, Heikki Linnakangas
wrote:
> On 01.08.2011 13:13, Simon Riggs wrote:
>> And what does NSN stand for? :-)
>
> Hmm, I don't know actually know what NSN is an acronym for :-).
Node Sequence Number.
--
Thom Brown
Twitter: @darkixion
IRC (freenode): dark_ixion
Registered Linux
On 01.08.2011 13:13, Simon Riggs wrote:
On Mon, Aug 1, 2011 at 10:38 AM, Heikki Linnakangas
wrote:
Attached is a patch for that for 9.1/master. The 9.0 GiST replay code was
quite different, it will require a separate patch.
Hmm, I was assured no changes would be required for Hot Standby for
On Mon, Aug 1, 2011 at 10:38 AM, Heikki Linnakangas
wrote:
> On 27.07.2011 17:43, Alexander Korotkov wrote:
>>
>> OK, thanks. I also found behaviour of GiST(without patch) with streaming
>> replication that seems strange for me. On master there are only few
>> rightlinks are InvalidBlockNumber whi
On 27.07.2011 17:43, Alexander Korotkov wrote:
OK, thanks. I also found behaviour of GiST(without patch) with streaming
replication that seems strange for me. On master there are only few
rightlinks are InvalidBlockNumber while on slave there are a lot of them. I
hack gevel for getting index stru
On Fri, Jul 29, 2011 at 1:10 AM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> gistFindCorrectParent() should take care of that.
>
OK, now it seems that I understood. I need to verify amount memory needed
for paths because it seems that they tends to accumulate. Also I need to
On 28.07.2011 23:57, Alexander Korotkov wrote:
On Tue, Jul 26, 2011 at 9:24 PM, Heikki Linnakangas<
heikki.linnakan...@enterprisedb.com> wrote:
Let me know if you have questions on the approach taken.
I realized that approach which comes as replace to loaded-subtrees keeping
is unclear to m
On Tue, Jul 26, 2011 at 9:24 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
>
> Let me know if you have questions on the approach taken.
I realized that approach which comes as replace to loaded-subtrees keeping
is unclear to me. We save paths between node buffers. But those
On 27.07.2011 17:43, Alexander Korotkov wrote:
1(l:1) blk: 324 numTuple: 129 free: 2472b(69.71%) rightlink:4294967295
(InvalidBlockNumber)
1(l:2) blk: 242 numTuple: 164 free: 932b(88.58%)
rightlink:4294967295 (InvalidBlockNumber)
2(l:2) blk: 525 numTuple: 121 free: 2824
On Wed, Jul 27, 2011 at 6:05 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> Dunno, both approaches seem reasonable to me. There's no rule against
> changing WAL record structure across major releases, if that's what you were
> worried about.
>
OK, thanks. I also found beha
On 27.07.2011 15:29, Alexander Korotkov wrote:
I found a problem in WAL with this patch. I use simplified insert algorithm
in my patch which don't insert downlink one by one but insert them at once.
Thus FollowRight flag is leaving uncleared when redoing from WAL, because
only one flag can be cle
I found a problem in WAL with this patch. I use simplified insert algorithm
in my patch which don't insert downlink one by one but insert them at once.
Thus FollowRight flag is leaving uncleared when redoing from WAL, because
only one flag can be cleared by one WAL record. Do you think modification
On Tue, Jul 26, 2011 at 9:24 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> That was a quite off-the-cuff remark, so I took the patch and culled out
> loaded-tree bookkeeping. A lot of other changes fell off from that, so it
> took me quite some time to get it working again
On 22.07.2011 12:38, Alexander Korotkov wrote:
Patch with my try to detect ordered datasets is attached. The implemented
idea is desribed below.
Index tuples are divided by chunks of 128. On each chunk we measure how much
leaf pages where index tuples was inserted don't match those of previous
ch
Hi!
Patch with my try to detect ordered datasets is attached. The implemented
idea is desribed below.
Index tuples are divided by chunks of 128. On each chunk we measure how much
leaf pages where index tuples was inserted don't match those of previous
chunk. Based on statistics of several chunks w
Hi!
New version of patch is attached. There are following changes.
1) Since proposed tchnique is not always a "fast" build, it was renamed
everywhere in the patch to "buffering" build.
2) Parameter "buffering" now has 3 possible values "yes", "no" and "auto".
"auto" means automatic switching from
Fri, Jul 15, 2011 at 12:53 AM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> On 14.07.2011 23:41, Alexander Korotkov wrote:
>
>> Do you think using "rightlink" as pointer to parent page is possible
>> during
>> index build? It would allow to simplify code significantly, becaus
On 14.07.2011 11:33, Alexander Korotkov wrote:
On Wed, Jul 13, 2011 at 5:59 PM, Heikki Linnakangas<
heikki.linnakan...@enterprisedb.com> wrote:
One thing that caught my eye is that when you empty a buffer, you load the
entire subtree below that buffer, down to the next buffered or leaf level,
On Wed, Jul 13, 2011 at 5:59 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> One thing that caught my eye is that when you empty a buffer, you load the
> entire subtree below that buffer, down to the next buffered or leaf level,
> into memory. Every page in that subtree is k
On 14.07.2011 23:41, Alexander Korotkov wrote:
Do you think using "rightlink" as pointer to parent page is possible during
index build? It would allow to simplify code significantly, because of no
more need to maintain in-memory structures for parents memorizing.
I guess, but where do you store
Do you think using "rightlink" as pointer to parent page is possible during
index build? It would allow to simplify code significantly, because of no
more need to maintain in-memory structures for parents memorizing.
--
With best regards,
Alexander Korotkov.
On Thu, Jul 14, 2011 at 12:42 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> Pinning a buffer that's already in the shared buffer cache is cheap, I
> doubt you're gaining much by keeping the private hash table in front of the
> buffer cache.
>
Yes, I see. Pinning a lot of b
On 12.07.2011 11:34, Alexander Korotkov wrote:
New version of patch with a little more refactoring and comments.
Great! The README helps tremendously to understand this, thanks for that.
One thing that caught my eye is that when you empty a buffer, you load
the entire subtree below that buffe
On Wed, Jul 13, 2011 at 12:40 PM, Alexander Korotkov
wrote:
> On Wed, Jul 13, 2011 at 12:33 PM, Heikki Linnakangas <
> heikki.linnakan...@enterprisedb.com> wrote:
>
>> Is it possible to switch to the new buffering method in the middle of an
>> index build? We could use the plain insertion method u
Hi!
On Wed, Jul 13, 2011 at 12:33 PM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> Is it possible to switch to the new buffering method in the middle of an
> index build? We could use the plain insertion method until the index grows
> to a certain size (effective_cache_size?
Hi,
Looking at the performance test results again on the wiki page
(http://wiki.postgresql.org/wiki/Fast_GiST_index_build_GSoC_2011#Testing_results),
the patch can be summarized like this: it reduces the number of disk
accesses, at the cost of some extra CPU work.
Is it possible to switch to
New version of patch with a little more refactoring and comments.
--
With best regards,
Alexander Korotkov.
gist_fast_build-0.6.0.patch.gz
Description: GNU Zip compressed data
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http:/
On Fri, Jul 8, 2011 at 6:18 PM, Tom Lane wrote:
> For test purposes, you could turn off synchronize_seqscans to prevent
> that.
Thanks, it helps. I'm rerunning tests now.
--
With best regards,
Alexander Korotkov.
Alexander Korotkov writes:
> I found that results of previous tests with USNOA2 were not fully correct.
> Tables for tests with various parameters contained tuples in different
> order. I assumed that query
> CREATE TABLE tab2 AS (SELECT * FROM tab1);
> creates tab2 as exact copy of tab1, i.e. tab
I found that results of previous tests with USNOA2 were not fully correct.
Tables for tests with various parameters contained tuples in different
order. I assumed that query
CREATE TABLE tab2 AS (SELECT * FROM tab1);
creates tab2 as exact copy of tab1, i.e. tab2 contain tuples in same order
as tab1
New version of patch with readme and some bugfixes.
Some new tests with fast build parameters variations are on the wiki page.
While I doubt to interpret some results of tests, because it looks strange
for me. I particular I can't explain why decrease of buffer size affects
index quality so much (i
1 - 100 of 136 matches
Mail list logo
