Tom Lane wrote:
I wrote:
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
samples %symbol name
350318533 98.8618 mergepreread
9718220.2743 tuplesort_gettuple_common
4136740.1167 tuplesort_heap_siftup
I don't have enough memory to really reproduce this, but I've come close
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
LOG: begin index sort: unique = f, workMem = 8024000, randomAccess = f
LOG: switching to external sort with 28658 tapes: CPU 4.18s/13.96u sec
elapsed 32.43 sec
LOG: finished writing run 1 to tape 0: CPU 173.56s/3425.85u sec elapsed
3814.82
Tom Lane wrote:
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
LOG: begin index sort: unique = f, workMem = 8024000, randomAccess = f
LOG: switching to external sort with 28658 tapes: CPU 4.18s/13.96u sec
elapsed 32.43 sec
LOG: finished writing run 1 to tape 0: CPU 173.56s/3425.85u sec
On Fri, 2006-03-10 at 09:31 -0500, Tom Lane wrote:
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
LOG: begin index sort: unique = f, workMem = 8024000, randomAccess = f
LOG: switching to external sort with 28658 tapes: CPU 4.18s/13.96u sec
elapsed 32.43 sec
LOG: finished writing run 1
Simon Riggs [EMAIL PROTECTED] writes:
On Fri, 2006-03-10 at 09:31 -0500, Tom Lane wrote:
I'll look into it, but I was already wondering if we shouldn't bound the
number of tapes somehow. It's a bit hard to believe that 28000 tapes is
a sane setting.
I thought you had changed the memory
I'll look into it, but I was already wondering if we shouldn't bound
the number of tapes somehow. It's a bit hard to believe that 28000
tapes is a sane setting.
Well, since they are not actually tapes, why not?
I wonder what the OS does when we repeatedly open and close those files
Zeugswetter Andreas DCP SD [EMAIL PROTECTED] writes:
I'll look into it, but I was already wondering if we shouldn't bound
the number of tapes somehow. It's a bit hard to believe that 28000
tapes is a sane setting.
Well, since they are not actually tapes, why not?
I wonder what the OS
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
samples %symbol name
350318533 98.8618 mergepreread
9718220.2743 tuplesort_gettuple_common
4136740.1167 tuplesort_heap_siftup
I don't have enough memory to really reproduce this, but I've come close
enough that I believe I see
I wrote:
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
samples %symbol name
350318533 98.8618 mergepreread
9718220.2743 tuplesort_gettuple_common
4136740.1167 tuplesort_heap_siftup
I don't have enough memory to really reproduce this, but I've come close
enough that I
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
samples %symbol name
103520432 47.9018 inlineApplySortFunction
33382738 15.4471 comparetup_index
25296438 11.7054 tuplesort_heap_siftup
10089122 4.6685 btint4cmp
8395676 3.8849 LogicalTapeRead
2873556 1.3297
Tom Lane wrote:
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
samples %symbol name
103520432 47.9018 inlineApplySortFunction
33382738 15.4471 comparetup_index
25296438 11.7054 tuplesort_heap_siftup
10089122 4.6685 btint4cmp
8395676 3.8849 LogicalTapeRead
2873556 1.3297
Tom Lane wrote:
I wrote:
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
samples %symbol name
24915704 96.2170 ltsReleaseBlock
We probably need to tweak things so this doesn't get called during the
final merge pass. Looking at it now.
I've committed a fix for this into CVS
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
CREATE INDEX on a 1,8B row table (5 int columns - index created on the
first row about 300M distinct values):
before: 11h 51min
after: 3h 11min(!)
Cool. Does it seem to be I/O bound now? Would you be willing to do it
over with oprofile turned
On Wed, 2006-03-08 at 10:45 -0500, Tom Lane wrote:
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
CREATE INDEX on a 1,8B row table (5 int columns - index created on the
first row about 300M distinct values):
before: 11h 51min
after: 3h 11min(!)
Cool. Does it seem to be I/O bound
Tom Lane wrote:
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
CREATE INDEX on a 1,8B row table (5 int columns - index created on the
first row about 300M distinct values):
before: 11h 51min
after: 3h 11min(!)
Cool. Does it seem to be I/O bound now? Would you be willing to do it
I wrote:
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
samples %symbol name
24915704 96.2170 ltsReleaseBlock
We probably need to tweak things so this doesn't get called during the
final merge pass. Looking at it now.
I've committed a fix for this into CVS HEAD --- please try it
Tom Lane wrote:
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
forgot to mention that this is 8.1.3 compiled from source.
See the discussion starting here:
http://archives.postgresql.org/pgsql-hackers/2006-02/msg00590.php
I was following this thread - and it was partly a reason why I'm
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
samples %symbol name
24915704 96.2170 ltsReleaseBlock
3872651.4955 LogicalTapeRead
1687250.6516 inlineApplySortFunction
Hmm ... the comment in ltsReleaseBlock sez
/*
* Insert blocknum into array, preserving
On Sun, 2006-03-05 at 15:15 -0500, Tom Lane wrote:
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
samples %symbol name
24915704 96.2170 ltsReleaseBlock
3872651.4955 LogicalTapeRead
1687250.6516 inlineApplySortFunction
Hmm ... the comment in ltsReleaseBlock sez
Hi all!
while playing on a new box i noticed that postgresql does not seem to be
able to cope with very large settings for maintenance_work_mem.
For a test I created a single table with 5 integer columns containing
about 1,8B rows 8(about 300M distinct values in the column I want to index):
Stefan Kaltenbrunner wrote:
Hi all!
while playing on a new box i noticed that postgresql does not seem to be
able to cope with very large settings for maintenance_work_mem.
For a test I created a single table with 5 integer columns containing
about 1,8B rows 8(about 300M distinct values
On 3/4/06, Stefan Kaltenbrunner [EMAIL PROTECTED] wrote:
forgot to mention that this is 8.1.3 compiled from source. Further
testing shows that not only CREATE INDEX has some issue with large
maintenance_work_mem settings :
what does it show:
cat /proc/sys/kernel/shmmax
?
depesz
hubert depesz lubaczewski wrote:
On 3/4/06, Stefan Kaltenbrunner [EMAIL PROTECTED] wrote:
forgot to mention that this is 8.1.3 compiled from source. Further
testing shows that not only CREATE INDEX has some issue with large
maintenance_work_mem settings :
what does it show:
cat
Stefan Kaltenbrunner wrote:
hubert depesz lubaczewski wrote:
On 3/4/06, Stefan Kaltenbrunner [EMAIL PROTECTED] wrote:
forgot to mention that this is 8.1.3 compiled from source. Further
testing shows that not only CREATE INDEX has some issue with large
maintenance_work_mem settings :
what
Stefan Kaltenbrunner wrote:
hubert depesz lubaczewski wrote:
On 3/4/06, Stefan Kaltenbrunner [EMAIL PROTECTED] wrote:
forgot to mention that this is 8.1.3 compiled from source. Further
testing shows that not only CREATE INDEX has some issue with large
maintenance_work_mem settings :
what
Stefan Kaltenbrunner wrote:
foo=# set maintenance_work_mem to 200;
SET
foo=# VACUUM ANALYZE verbose;
INFO: vacuuming information_schema.sql_features
ERROR: invalid memory alloc request size 204798
Just an FYI, I reported a similar problem on my 8.0.0 database a few
weeks ago. I
Michael Paesold wrote:
Stefan Kaltenbrunner wrote:
hubert depesz lubaczewski wrote:
On 3/4/06, Stefan Kaltenbrunner [EMAIL PROTECTED] wrote:
forgot to mention that this is 8.1.3 compiled from source. Further
testing shows that not only CREATE INDEX has some issue with large
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
not that I think it is related to the problem at all. It looks like I'm
hitting the MaxAllocSize Limit in src/include/utils/memutils.h.
just tried to increase this limit to 4GB (from the default 1GB) and this
seems to help a fair bit.
s/help a
Tom Lane wrote:
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
not that I think it is related to the problem at all. It looks like I'm
hitting the MaxAllocSize Limit in src/include/utils/memutils.h.
just tried to increase this limit to 4GB (from the default 1GB) and this
seems to help a
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
Tom Lane wrote:
The sorting code probably needs a defense to keep it from trying to
exceed MaxAllocSize for the SortObject array; AFAIR there is no such
consideration there now, but it's easily added. I'm not sure where your
VACUUM failure is
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
forgot to mention that this is 8.1.3 compiled from source.
See the discussion starting here:
http://archives.postgresql.org/pgsql-hackers/2006-02/msg00590.php
I was following this thread - and it was partly a reason why I'm playing
with
31 matches
Mail list logo
