On Sat, Apr 22, 2006 at 14:20:32 -0700,
daveg <[EMAIL PROTECTED]> wrote:
> On Sat, Apr 22, 2006 at 01:49:25PM -0700, David Fetter wrote:
> > On Sat, Apr 22, 2006 at 01:14:42PM -0700, David Gould wrote:
> >
> > > To avoid running out of swap and triggering the oom killer we have
> > > had to redu
Tom,
On 3/21/06 3:06 PM, "Tom Lane" <[EMAIL PROTECTED]> wrote:
> The real problem we are facing with a whole lot of our optimization
> issues (not only sorting) is that it's not all that trivial to get
> credible experimental results that we can expect will hold up across
> a range of usage scena
On Tue, 2006-03-21 at 17:47 -0500, Tom Lane wrote:
> I'm fairly unconvinced about Simon's underlying premise --- that we
> can't make good use of work_mem in sorting after the run building phase
> --- anyway.
We can make good use of memory, but there does come a point in final
merging where too
"Luke Lonergan" <[EMAIL PROTECTED]> writes:
> Experiment should take but a minute to validate or disprove the hypothesis.
Only if you're prepared to trust the results of one experiment on one
platform with a not-very-large amount of data. Otherwise it's going to
take quite a few minutes ...
The
Tom,
On 3/21/06 2:47 PM, "Tom Lane" <[EMAIL PROTECTED]> wrote:
> I'm fairly unconvinced about Simon's underlying premise --- that we
> can't make good use of work_mem in sorting after the run building phase
> --- anyway. If we cut back our memory usage then we'll be forcing a
> significantly mor
Martijn van Oosterhout writes:
> There is one way to guarentee the memory is released to the OS after
> completion. Make the allocator allocate work_mem bytes using mmap()
> rather than malloc(). munmap() will then definitly return the memory to
> the OS. Unfortunatly, the coding required would pr
On Tue, Mar 21, 2006 at 08:05:50PM +, Simon Riggs wrote:
> > Point 2 is actually a serious flaw in Simon's proposal, because there
> > is no portable way to make malloc return freed memory to the OS. Some
> > mallocs will do that ... in some cases ... but many simply don't ever
> > move the br
On Sat, 2006-03-18 at 13:21 -0800, Luke Lonergan wrote:
> In short, faster performance through more aggressive runtime compilation. A
> JIT for the database kernel. It's not like I'm on shaky ground here - other
> commercial DBMS have done it for over a decade.
I think what Luke may be referrin
On Fri, 2006-03-17 at 09:46 -0500, Tom Lane wrote:
> "Qingqing Zhou" <[EMAIL PROTECTED]> writes:
> > So what's the difference between these two strategy?
> > (1) Running time: do they use the same amount of memory? Why option 2 is
> > better than 1?
> > (2) Idle time: after sort done, option 1 will
Thomas Hallgren wrote:
Luke Lonergan wrote:
Tom,
On 3/17/06 9:59 PM, "Tom Lane" <[EMAIL PROTECTED]> wrote:
This would buy what exactly?
I guess you didn't read the other 80% of the post.
In short, faster performance through more aggressive runtime
compilation. A
JIT for the database k
Luke Lonergan wrote:
Tom,
On 3/17/06 9:59 PM, "Tom Lane" <[EMAIL PROTECTED]> wrote:
This would buy what exactly?
I guess you didn't read the other 80% of the post.
In short, faster performance through more aggressive runtime compilation. A
JIT for the database kernel. It's not like I'm o
Tom,
On 3/17/06 12:18 PM, "Tom Lane" <[EMAIL PROTECTED]> wrote:
> One user with ability to enter arbitrary SQL commands can *always* blow
> your resource planning away. Blaming such things on work_mem is
> seriously misguided.
Agreed - that's why we need to split this discussion into the two ca
Tom,
On 3/17/06 9:59 PM, "Tom Lane" <[EMAIL PROTECTED]> wrote:
> This would buy what exactly?
I guess you didn't read the other 80% of the post.
In short, faster performance through more aggressive runtime compilation. A
JIT for the database kernel. It's not like I'm on shaky ground here - o
"Luke Lonergan" <[EMAIL PROTECTED]> writes:
> We've looked at using the open source llvm compiler to create an
> intermediate representation of the plan, then generate machine code and
> dispatch for execution.
This would buy what exactly?
regards, tom lane
--
Csaba,
On 3/17/06 7:07 AM, "Csaba Nagy" <[EMAIL PROTECTED]> wrote:
> It worths a look at how apache Derby does with query planning, where a
> planned query is actually a compiled Java class, i.e. the executable
> byte code which will run to fetch the results, created and compiled by
> the planner
On Fri, Mar 17, 2006 at 04:45:17PM -0500, Tom Lane wrote:
> Hannu Krosing <[EMAIL PROTECTED]> writes:
> > So perhaps we could keep the shaded_work_mem in actual shared memory,
> > and alloc it to processes from there ?
>
> No, that's utterly not reasonable, both from an allocation point of view
>
Hannu Krosing <[EMAIL PROTECTED]> writes:
> So perhaps we could keep the shaded_work_mem in actual shared memory,
> and alloc it to processes from there ?
No, that's utterly not reasonable, both from an allocation point of view
(you'd have to make shared memory enormous, and not all platforms will
Ühel kenal päeval, R, 2006-03-17 kell 09:46, kirjutas Tom Lane:
> "Qingqing Zhou" <[EMAIL PROTECTED]> writes:
> > So what's the difference between these two strategy?
> > (1) Running time: do they use the same amount of memory? Why option 2 is
> > better than 1?
> > (2) Idle time: after sort done,
Josh Berkus writes:
> Mind you, I'm also thinking that on enterprise installations with
> multi-department use of the database, the fact that work_mem is
> inalienably USERSET is also an allocation problem. One user with a SET
> command can blow all of your resource planning away.
One user w
Tom,
> My own thoughts about the problems with our work_mem arrangement are
> that the real problem is the rule that we can allocate work_mem per sort
> or hash operation; this makes the actual total memory use per backend
> pretty unpredictable for nontrivial queries. I don't know how to fix
> t
> My own thoughts about the problems with our work_mem arrangement are
> that the real problem is the rule that we can allocate work_mem per sort
> or hash operation; this makes the actual total memory use per backend
> pretty unpredictable for nontrivial queries. I don't know how to fix
> this th
"Qingqing Zhou" <[EMAIL PROTECTED]> writes:
> So what's the difference between these two strategy?
> (1) Running time: do they use the same amount of memory? Why option 2 is
> better than 1?
> (2) Idle time: after sort done, option 1 will return all 1024 to the OS and
> 2 will still keep 512?
Poin
On Fri, 2006-03-17 at 13:29 +0800, Qingqing Zhou wrote:
> "Simon Riggs" <[EMAIL PROTECTED]> wrote
> >
> Interesting, I understand that shared_work_mem is process-wise,
> allocate-when-use, request-may-or-may-not-get-it (as you have pointed out,
> this may make planner in a hard situation if we are
"Simon Riggs" <[EMAIL PROTECTED]> wrote
>
> We would only look at dynamically changing work_mem in those few
> restricted cases where we track that against the work_mem limit. If we
> hit that limit, we would make a request to the central pool: "Can I be
> allotted another 2MB please?" (etc). The
24 matches
Mail list logo
