Re: [HACKERS] Bug in new buffering GiST build code

2012-06-05 Thread Alexander Korotkov
On Tue, Jun 5, 2012 at 2:00 AM, Alexander Korotkov aekorot...@gmail.comwrote: On Mon, May 28, 2012 at 1:46 AM, Alexander Korotkov aekorot...@gmail.comwrote: On Sat, May 26, 2012 at 12:33 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Alexander, do you still have the

Re: [HACKERS] Bug in new buffering GiST build code

2012-06-05 Thread Heikki Linnakangas
On 05.06.2012 09:45, Alexander Korotkov wrote: On Tue, Jun 5, 2012 at 2:00 AM, Alexander Korotkovaekorot...@gmail.comwrote: On Mon, May 28, 2012 at 1:46 AM, Alexander Korotkovaekorot...@gmail.comwrote: On Sat, May 26, 2012 at 12:33 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com

Re: [HACKERS] Bug in new buffering GiST build code

2012-06-04 Thread Alexander Korotkov
On Mon, May 28, 2012 at 1:46 AM, Alexander Korotkov aekorot...@gmail.comwrote: On Sat, May 26, 2012 at 12:33 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Alexander, do you still have the test environments and data lying around that you used for GiST buffering testing

Re: [HACKERS] Bug in new buffering GiST build code

2012-05-30 Thread Heikki Linnakangas
On 28.05.2012 00:46, Alexander Korotkov wrote: On Sat, May 26, 2012 at 12:33 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Attached is a patch to replace the path stacks with a hash table. With this patch, the index build time in my test case dropped from 59 minutes to

Re: [HACKERS] Bug in new buffering GiST build code

2012-05-30 Thread Alexander Korotkov
On Wed, May 30, 2012 at 1:01 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I also spotted and fixed another little oversight: the temporary file didn't get deleted after the index build. I've one note not directly related to buffering build. While I debugging buffering

Re: [HACKERS] Bug in new buffering GiST build code

2012-05-30 Thread Tom Lane
Alexander Korotkov aekorot...@gmail.com writes: I've one note not directly related to buffering build. While I debugging buffering GiST index build, backend was frequently crashed. After recovery partially built index file was remain. Do we have some tool to detect such dead files? If not,

Re: [HACKERS] Bug in new buffering GiST build code

2012-05-27 Thread Alexander Korotkov
On Sat, May 26, 2012 at 12:33 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I think we should rewrite the way we track the parents completely. Rather than keep a path stack attached to every node buffer, let's just maintain a second hash table that contains the parent of

Re: [HACKERS] Bug in new buffering GiST build code

2012-05-25 Thread Heikki Linnakangas
On 22.05.2012 01:09, Alexander Korotkov wrote: Hi! On Tue, May 22, 2012 at 12:56 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: The management of the path stacks is a bit complicated, anyway. I'll think about this some more tomorrow, maybe we can make it simpler, knowing

Re: [HACKERS] Bug in new buffering GiST build code

2012-05-21 Thread Heikki Linnakangas
On 18.05.2012 20:34, Alexander Korotkov wrote: On Fri, May 18, 2012 at 8:27 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: After fixing that, however, I'm now getting another error, much later in the build process: ERROR: failed to re-find parent for block 123002

Re: [HACKERS] Bug in new buffering GiST build code

2012-05-21 Thread Heikki Linnakangas
On 18.05.2012 20:34, Alexander Korotkov wrote: On Fri, May 18, 2012 at 8:27 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: After fixing that, however, I'm now getting another error, much later in the build process: ERROR: failed to re-find parent for block 123002

Re: [HACKERS] Bug in new buffering GiST build code

2012-05-21 Thread Alexander Korotkov
Hi! On Tue, May 22, 2012 at 12:56 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: After staring at graphs built from gist trees for the whole day, I think I finally understand what's wrong: There's a thinko in the way we maintain the parent paths during insertions. It

[HACKERS] Bug in new buffering GiST build code

2012-05-18 Thread Heikki Linnakangas
I bumped into a bug in the new buffering GiST build code. I did this: create table gisttest (t text); insert into gisttest select a||'fooo' from generate_series(1,1000) a; create index i_gisttest on gisttest using gist (t collate C) WITH

Re: [HACKERS] Bug in new buffering GiST build code

2012-05-18 Thread Alexander Korotkov
On Fri, May 18, 2012 at 8:27 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: After fixing that, however, I'm now getting another error, much later in the build process: ERROR: failed to re-find parent for block 123002 STATEMENT: create index i_gisttest on gisttest using