On Wed, Dec 21, 2016 at 4:25 AM, Heikki Linnakangas wrote:
> In the meanwhile, Robert committed the cap on the number of tapes. Since
> that's in, I'm not sure if the pause/resume part of this is worthwhile.
> Maybe it is.
I rebased my parallel tuplesort patch on top of what you committed a
few d
On 12/21/2016 03:22 PM, Robert Haas wrote:
On Wed, Dec 21, 2016 at 7:25 AM, Heikki Linnakangas wrote:
For now, barring objections, I'm going to commit the first patch. It seems
like a worthwhile simplification in any case, especially for Peter's
Parallel tuplesort patch set.
Well, it removes
On Wed, Dec 21, 2016 at 7:25 AM, Heikki Linnakangas wrote:
> For now, barring objections, I'm going to commit the first patch. It seems
> like a worthwhile simplification in any case, especially for Peter's
> Parallel tuplesort patch set.
Well, it removes more code than it adds. That's definitel
On 10/12/2016 05:30 PM, Heikki Linnakangas wrote:
On 10/10/2016 10:31 PM, Peter Geoghegan wrote:
On Sun, Oct 9, 2016 at 11:52 PM, Heikki Linnakangas wrote:
Ok, good. I think we're in agreement on doing this, then, even if we don't
agree on the justification :-).
Agreed. :-)
Attached are ne
On 10/10/2016 10:31 PM, Peter Geoghegan wrote:
On Sun, Oct 9, 2016 at 11:52 PM, Heikki Linnakangas wrote:
Ok, good. I think we're in agreement on doing this, then, even if we don't
agree on the justification :-).
Agreed. :-)
Attached are new patch versions. Rebased them over current head, f
On Sun, Oct 9, 2016 at 11:52 PM, Heikki Linnakangas wrote:
> Regardless of the number of tapes, the memory used for the tape buffers,
> while building the initial runs, is wasted. It's not entirely wasted when
> you have multiple merge passes, because without the buffer you need to read
> the last
Here are freshly rebased versions of these patches. No big changes, but
edited some comments slightly.
- Heikki
>From 324706fb2e15facbc5686e98ee26bbdc5017366a Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas
Date: Mon, 10 Oct 2016 10:59:38 +0300
Subject: [PATCH 1/2] Simplify tape block format
On 10/09/2016 03:27 AM, Peter Geoghegan wrote:
You shouldn't really waste 8% of the budget with low work_mem settings
with my cap patch applied, because the new cap never limits the number
of tapes. IIRC, the cap of 500 tapes doesn't start to matter until you
have about 1GB of work_mem. So, if th
Apologies for the delayed response to this.
On Tue, Oct 4, 2016 at 3:47 AM, Heikki Linnakangas wrote:
> One of the patches in Peter Geoghegan's Parallel tuplesort patch set [1] is
> to put a cap on the number of tapes to use for the sort.
> That amounts to about 8% of the available memory. That'
On 10/04/2016 07:06 PM, Claudio Freire wrote:
On Tue, Oct 4, 2016 at 7:47 AM, Heikki Linnakangas wrote:
1. The first patch changes the way we store the logical tapes on disk.
Instead of using indirect blocks, HFS style, the blocks form a linked list.
Each block has a trailer, with the block num
On Tue, Oct 4, 2016 at 7:47 AM, Heikki Linnakangas wrote:
> 1. The first patch changes the way we store the logical tapes on disk.
> Instead of using indirect blocks, HFS style, the blocks form a linked list.
> Each block has a trailer, with the block numbers of the previous and next
> block of th
On 10/04/2016 05:58 PM, Simon Riggs wrote:
On 4 October 2016 at 12:47, Heikki Linnakangas wrote:
Why not just make each new run start at a block boundary?
That way we waste on average BLCKSZ/2 disk space per run, which is
negligible but we avoid any need to have code to read back in the last
b
On 4 October 2016 at 12:47, Heikki Linnakangas wrote:
>> Why not just make each new run start at a block boundary?
>> That way we waste on average BLCKSZ/2 disk space per run, which is
>> negligible but we avoid any need to have code to read back in the last
>> block.
>
>
> Hmm. You'd still have
On 10/04/2016 02:09 PM, Simon Riggs wrote:
On 4 October 2016 at 11:47, Heikki Linnakangas wrote:
When we finish writing an initial run to a tape, we keep the tape buffers
around. That's the reason we need to reserve that memory. But there's no
fundamental reason we have to do that. As I sugges
On 4 October 2016 at 11:47, Heikki Linnakangas wrote:
> When we finish writing an initial run to a tape, we keep the tape buffers
> around. That's the reason we need to reserve that memory. But there's no
> fundamental reason we have to do that. As I suggested in [2], we could flush
> the buffers
15 matches
Mail list logo