On Mon, Apr 22, 2024 at 11:13 AM Peter Geoghegan wrote:
> I'm pretty sure that I could fix this by simply removing the
> assertion. But I need to think about it a bit more before I push a
> fix.
>
> The test case you've provided proves that _bt_preprocess_keys's
> new no-op path isn't just used
On Mon, Apr 22, 2024 at 4:00 AM Alexander Lakhin wrote:
> Please look at another case, where a similar Assert (but this time in
> _bt_preprocess_keys()) is triggered:
> CREATE TABLE t (a text, b text);
> INSERT INTO t (a, b) SELECT 'a', repeat('b', 100) FROM generate_series(1,
> 500) g;
> CREATE
Hello Peter,
07.04.2024 20:18, Peter Geoghegan wrote:
On Sun, Apr 7, 2024 at 1:00 PM Alexander Lakhin wrote:
SELECT * FROM t WHERE a < ANY (ARRAY[1]) AND b < ANY (ARRAY[1]);
TRAP: failed Assert("so->numArrayKeys"), File: "nbtutils.c", Line: 560, PID:
3251267
I immediately see what's up
On Thu, Apr 18, 2024 at 2:13 AM Donghang Lin wrote:
> It's triggered when a scankey's strategy is set to invalid. While for a
> descending ordered column,
> the strategy needs to get fixed to its commute strategy. That doesn't work if
> the strategy is invalid.
The problem is that
Hi Peter
There seems to be an assertion failure with this change in HEAD
TRAP: failed Assert("leftarg->sk_attno == rightarg->sk_attno"), File:
"../../src/backend/access/nbtree/nbtutils.c", Line: 3246, PID: 1434532
It can be reproduced by:
create table t(a int);
insert into t select 1 from
On Sun, Apr 7, 2024 at 9:57 PM Peter Geoghegan wrote:
> On Sun, Apr 7, 2024 at 9:50 PM Tom Lane wrote:
> > those first two Asserts are redundant with the "if" as well.
>
> I'll get rid of those other two assertions as well, then.
Done that way.
--
Peter Geoghegan
On Sun, Apr 7, 2024 at 9:50 PM Tom Lane wrote:
> If you're doing that, then surely
>
> if (j != (BTEqualStrategyNumber - 1) ||
> !(xform[j].skey->sk_flags & SK_SEARCHARRAY))
> {
> ...
> }
>
Peter Geoghegan writes:
> The assertions in question are arguably redundant. There are very
> similar assertions just a little earlier on, as we initially set up
> the array stuff (right before _bt_compare_scankey_args is called).
> I'll just remove the "Assert(xform[j].ikey == array->scan_key)"
On Sun, Apr 7, 2024 at 9:25 PM Tom Lane wrote:
> Perhaps this'd help:
>
> -Assert(xform[j].ikey == array->scan_key);
> +Assert(array && xform[j].ikey == array->scan_key);
>
> If that doesn't silence it, I'd be prepared to just dismiss the
> warning.
Peter Geoghegan writes:
> On Sun, Apr 7, 2024 at 8:48 PM Tom Lane wrote:
>> Coverity pointed out something that looks like a potentially live
>> problem in 5bf748b86:
>> ... which certainly makes it look like array->scan_key could be
>> a null-pointer dereference.
> But the
On Sun, Apr 7, 2024 at 8:48 PM Tom Lane wrote:
>
> Coverity pointed out something that looks like a potentially live
> problem in 5bf748b86:
>
> /srv/coverity/git/pgsql-git/postgresql/src/backend/access/nbtree/nbtutils.c:
> 2950 in _bt_preprocess_keys()
> 2944 * need
Coverity pointed out something that looks like a potentially live
problem in 5bf748b86:
/srv/coverity/git/pgsql-git/postgresql/src/backend/access/nbtree/nbtutils.c:
2950 in _bt_preprocess_keys()
2944 * need to make sure that we don't throw away
an array
2945
On Sun, Apr 7, 2024 at 1:00 PM Alexander Lakhin wrote:
> Please look at an assertion failure (reproduced starting from 5bf748b86),
> triggered by the following query:
> CREATE TABLE t (a int, b int);
> CREATE INDEX t_idx ON t (a, b);
> INSERT INTO t (a, b) SELECT g, g FROM generate_series(0, 999)
Hello Peter,
03.04.2024 22:53, Peter Geoghegan wrote:
On Mon, Apr 1, 2024 at 6:33 PM Peter Geoghegan wrote:
Note: v18 doesn't have any adjustments to the costing, as originally
planned. I'll probably need to post a revised patch with improved (or
at least polished) costing in the next few
On Mon, Mar 18, 2024 at 9:25 AM Matthias van de Meent
wrote:
> I was thinking about a more unified processing model, where
> _bt_preprocess_keys would iterate over all keys, including processing
> of array keys (by merging and reduction to normal keys) if and when
> found. This would also reduce
On Thu, Mar 7, 2024 at 10:42 AM Benoit Tigeot wrote:
> I am not up to date with the last version of patch but I did a regular
> benchmark with version 11 and typical issue we have at the moment and the
> result are still very very good. [1]
Thanks for providing the test case. It was definitely
On Sat, 16 Mar 2024 at 01:12, Peter Geoghegan wrote:
>
> On Fri, Mar 8, 2024 at 9:00 AM Matthias van de Meent
> wrote:
> > I've attached v14, where 0001 is v13, 0002 is a patch with small
> > changes + some large comment suggestions, and 0003 which contains
> > sorted merge join code for
On Sat, 16 Mar 2024 at 01:12, Peter Geoghegan wrote:
>
> On Fri, Mar 8, 2024 at 9:00 AM Matthias van de Meent
> wrote:
> > I've attached v14, where 0001 is v13, 0002 is a patch with small
> > changes + some large comment suggestions, and 0003 which contains
> > sorted merge join code for
On Wed, Mar 6, 2024 at 4:51 PM Matthias van de Meent
wrote:
> To clarify, what I mean here is that merging the changes of both the
> SAOPs changes and the removal of arrayKeyData seems to increase the
> patch size and increases the maximum complexity of each component
> patch's review.
Removing
On Wed, Mar 6, 2024 at 4:46 PM Matthias van de Meent
wrote:
> On Wed, 6 Mar 2024 at 01:50, Peter Geoghegan wrote:
> > I think that there is supposed to be a closing parenthesis here? So
> > "... (such as those described in ") might
> > perform...".
>
> Correct.
>
> > FWM, if that's what you
Hello,
I am not up to date with the last version of patch but I did a regular
benchmark with version 11 and typical issue we have at the moment and the
result are still very very good. [1]
In term of performance improvement the last proposals could be a real game
changer for 2 of our biggest
On Wed, 6 Mar 2024 at 22:46, Matthias van de Meent
wrote:
>
> On Wed, 6 Mar 2024 at 01:50, Peter Geoghegan wrote:
> > At one point Heikki suggested that I just get rid of
> > BTScanOpaqueData.arrayKeyData (which has been there for as long as
> > nbtree had native support for SAOPs), and use
> >
On Wed, 6 Mar 2024 at 01:50, Peter Geoghegan wrote:
>
> On Mon, Mar 4, 2024 at 3:51 PM Matthias van de Meent
> wrote:
> > > +that searches for multiple values together. Queries that use certain
> > > +SQL constructs to search for rows matching any
> > > value
> > > +out of a list
On Sat, 2 Mar 2024 at 02:30, Peter Geoghegan wrote:
>
> On Thu, Feb 15, 2024 at 6:36 PM Peter Geoghegan wrote:
> > Attached is v11, which now says something like that in the commit
> > message.
>
> Attached is v12.
Some initial comments on the documentation:
> +that searches for multiple
On Mon, Jan 22, 2024 at 4:13 PM Matthias van de Meent
wrote:
> Attached 2 patches: v11.patch-a and v11.patch-b. Both are incremental
> on top of your earlier set, and both don't allocate additional memory
> in the merge operation in non-assertion builds.
>
> patch-a is a trivial and clean
On Mon, Jan 22, 2024 at 4:13 PM Matthias van de Meent
wrote:
> On Fri, 19 Jan 2024 at 23:42, Peter Geoghegan wrote:
> And this is where I disagree with your (percieved) implicit argument
> that this should be and always stay this way.
I never said that, and didn't intend to imply it. As I
On Fri, 19 Jan 2024 at 23:42, Peter Geoghegan wrote:
>
Thank you for your replies so far.
> On Thu, Jan 18, 2024 at 11:39 AM Matthias van de Meent
> wrote:
> > I would agree with you if this was about new APIs and features, but
> > here existing APIs are being repurposed without changing them.
On Thu, Jan 18, 2024 at 11:39 AM Matthias van de Meent
wrote:
> > I'm not going to break out the planner changes, because they're *not*
> > independent in any way.
>
> The planner changes depend on the btree changes, that I agree with.
> However, I don't think that the btree changes depend on the
On Tue, 16 Jan 2024 at 03:03, Peter Geoghegan wrote:
>
> On Mon, Jan 15, 2024 at 2:32 PM Matthias van de Meent
> wrote:
> > Can you pull these planner changes into their own commit(s)?
> > As mentioned upthread, it's a significant change in behavior that
> > should have separate consideration
On Mon, Jan 15, 2024 at 2:32 PM Matthias van de Meent
wrote:
> Can you pull these planner changes into their own commit(s)?
> As mentioned upthread, it's a significant change in behavior that
> should have separate consideration and reference in the commit log. I
> really don't think it should be
On Thu, 28 Dec 2023 at 18:28, Peter Geoghegan wrote:
>
> On Sat, Dec 9, 2023 at 10:38 AM Peter Geoghegan wrote:
> > Attached is v8, which pretty much rips all of this stuff out.
>
> Attached is v9, which I'm posting just to fix bitrot. The patch
> stopped cleanly applying against HEAD due to
On Mon, Dec 4, 2023 at 7:25 PM Peter Geoghegan wrote:
> 2. The optimization that has _bt_checkkeys skip non-required scan keys
> that are *only* required in the direction *opposite* the current scan
> direction -- this can work even without any precheck from
> _bt_readpage.
>
> Note that this
On Mon, Nov 27, 2023 at 5:39 AM Heikki Linnakangas wrote:
> - +1 on the general idea. Hard to see any downsides if implemented right.
Glad you think so. The "no possible downside" perspective is one that
the planner sort of relies on, so this isn't just a nice-to-have -- it
might actually be a
On Tue, Nov 28, 2023 at 3:52 PM Peter Geoghegan wrote:
> While I still think that we need heuristics that apply speculative
> criteria to decide whether or not going to the next page directly
> (when we have a choice at all), that doesn't mean that the v7
> heuristics can't be improved on, with a
On Tue, Nov 28, 2023 at 9:19 AM Peter Geoghegan wrote:
> > I'm not convinced this is a problem we have to solve. It's possible it
> > only affects cases that are implausible in practice (the script forces a
> > particular scan type, and maybe it would not be picked in practice). But
> > maybe
On Tue, Nov 28, 2023 at 4:29 AM Tomas Vondra
wrote:
> I haven't looked at the code, but I decided to do a bit of blackbox perf
> and stress testing, to get some feeling of what to expect in terms of
> performance improvements, and see if there happen to be some unexpected
> regressions. Attached
On 21/11/2023 04:52, Peter Geoghegan wrote:
Attached is v7.
First, some high-level reactions before looking at the patch very closely:
- +1 on the general idea. Hard to see any downsides if implemented right.
- This changes the meaning of amsearcharray==true to mean that the
ordering is
On Wed, 8 Nov 2023 at 02:53, Peter Geoghegan wrote:
>
> On Tue, Nov 7, 2023 at 4:20 AM Matthias van de Meent
> wrote:
> > On Tue, 7 Nov 2023 at 00:03, Peter Geoghegan wrote:
> > > I should be able to post v6 later this week. My current plan is to
> > > commit the other nbtree patch first (the
On Fri, 10 Nov 2023 at 00:58, Peter Geoghegan wrote:
> On Tue, Nov 7, 2023 at 5:53 PM Peter Geoghegan wrote:
> > If you end up finding a bug in this v6, it'll most likely be a case
> > where nbtree fails to live up to that. This project is as much about
> > robust/predictable performance as
On Tue, Nov 7, 2023 at 5:53 PM Peter Geoghegan wrote:
> If you end up finding a bug in this v6, it'll most likely be a case
> where nbtree fails to live up to that. This project is as much about
> robust/predictable performance as anything else -- nbtree needs to be
> able to cope with
On Tue, 7 Nov 2023 at 00:03, Peter Geoghegan wrote:
>
> On Mon, Nov 6, 2023 at 1:28 PM Matthias van de Meent
> wrote:
> > I'm planning on reviewing this patch tomorrow, but in an initial scan
> > through the patch I noticed there's little information about how the
> > array keys state machine
On Mon, Nov 6, 2023 at 1:28 PM Matthias van de Meent
wrote:
> I'm planning on reviewing this patch tomorrow, but in an initial scan
> through the patch I noticed there's little information about how the
> array keys state machine works in this new design. Do you have a more
> toplevel description
On Sat, 21 Oct 2023 at 00:40, Peter Geoghegan wrote:
>
> On Sun, Oct 15, 2023 at 1:50 PM Peter Geoghegan wrote:
> > Attached is v4, which applies cleanly on top of HEAD. This was needed
> > due to Alexandar Korotkov's commit e0b1ee17, "Skip checking of scan
> > keys required for directional scan
On Sun, Oct 15, 2023 at 1:50 PM Peter Geoghegan wrote:
> Attached is v4, which applies cleanly on top of HEAD. This was needed
> due to Alexandar Korotkov's commit e0b1ee17, "Skip checking of scan
> keys required for directional scan in B-tree".
>
> Unfortunately I have more or less dealt with
On Thu, Sep 28, 2023 at 5:32 PM Peter Geoghegan wrote:
> On Sun, Sep 17, 2023 at 4:47 PM Peter Geoghegan wrote:
> > Attached is v2, which makes all array key advancement take place using
> > the "next index tuple" approach (using binary searches to find array
> > keys using index tuple values).
On Sun, Sep 17, 2023 at 4:47 PM Peter Geoghegan wrote:
> Attached is v2, which makes all array key advancement take place using
> the "next index tuple" approach (using binary searches to find array
> keys using index tuple values).
Attached is v3, which fixes bitrot caused by today's bugfix
On Wed, Jul 26, 2023 at 6:41 AM Peter Geoghegan wrote:
> > MDAM seems to require exponential storage for "scan key operations"
> > for conditions on N columns (to be precise, the product of the number
> > of distinct conditions on each column); e.g. an index on mytable
> > (a,b,c,d,e,f,g,h) with
On Mon, Jul 31, 2023 at 12:24 PM Alena Rybakina
wrote:
> I noticed that you are going to add CNF->DNF transformation at the index
> construction stage. If I understand correctly, you will rewrite
> restrictinfo node,
> change boolean "AND" expressions to "OR" expressions, but would it be
>
On Thu, Jul 27, 2023 at 10:00 AM Matthias van de Meent
wrote:
> My idea is not quite block nested loop join. It's more 'restart the
> index scan at the location the previous index scan ended, if
> heuristics say there's a good chance that might save us time'. I'd say
> it is comparable to the
Hi, all!
CNF -> DNF conversion
=
Like many great papers, the MDAM paper takes one core idea, and finds
ways to leverage it to the hilt. Here the core idea is to take
predicates in conjunctive normal form (an "AND of ORs"), and convert
them into disjunctive normal form (an
On Thu, 27 Jul 2023 at 16:01, Peter Geoghegan wrote:
>
> On Thu, Jul 27, 2023 at 7:59 AM Matthias van de Meent
> wrote:
> > > Basically, the patch that added that feature had to revise the index
> > > AM API, in order to support a mode of operation where scans return
> > > groupings rather than
On Thu, Jul 27, 2023 at 7:59 AM Matthias van de Meent
wrote:
> > Basically, the patch that added that feature had to revise the index
> > AM API, in order to support a mode of operation where scans return
> > groupings rather than tuples. Whereas this patch requires none of
> > that. It makes
On Thu, 27 Jul 2023 at 06:14, Peter Geoghegan wrote:
>
> On Wed, Jul 26, 2023 at 12:07 PM Matthias van de Meent
> wrote:
> > We could cache the last accessed leaf page across amrescan operations
> > to reduce the number of index traversals needed when the join key of
> > the left side is highly
On Wed, 26 Jul 2023 at 15:42, Peter Geoghegan wrote:
>
> On Wed, Jul 26, 2023 at 5:29 AM Matthias van de Meent
> > I'm not sure I understand. MDAM seems to work on an index level to
> > return full ranges of values, while "skip scan" seems to try to allow
> > systems to signal to the index to
On Wed, Jul 26, 2023 at 12:07 PM Matthias van de Meent
wrote:
> We could cache the last accessed leaf page across amrescan operations
> to reduce the number of index traversals needed when the join key of
> the left side is highly (but not necessarily strictly) correllated.
That sounds like
On Wed, 26 Jul 2023 at 15:42, Peter Geoghegan wrote:
>
> On Wed, Jul 26, 2023 at 5:29 AM Matthias van de Meent
> wrote:
> > Considering that it caches/reuses the page across SAOP operations, can
> > (or does) this also improve performance for index scans on the outer
> > side of a join if the
On Wed, Jul 26, 2023 at 5:29 AM Matthias van de Meent
wrote:
> Considering that it caches/reuses the page across SAOP operations, can
> (or does) this also improve performance for index scans on the outer
> side of a join if the order of join columns matches the order of the
> index?
It doesn't
On Tue, 25 Jul 2023 at 03:34, Peter Geoghegan wrote:
>
> I've been working on a variety of improvements to nbtree's native
> ScalarArrayOpExpr execution. This builds on Tom's work in commit
> 9e8da0f7.
Cool.
> Attached patch is still at the prototype stage. I'm posting it as v1 a
> little
58 matches
Mail list logo
