ll commit
> it before next commitfest.
Hi!
I've took a glance at the patch, here's what catches my eye in comments:
corrdinate, dimenstions, descinding, stoty.
I'll try to provide meaningful review next week.
Best regards, Andrey Borodin.
the heapallindexed check doesn't actually
> care about index structure at all.
Seems like it was not a big deal of patching, I've fixed those bits (see
attachment).
I've done only simple tests as for now, but I'm planning to do better testing
before next CF.
Thanks for mentioning
t; . Seems like we
have common view with Peter Geoghegan and Anastasia that found drawback will be
fixed before next CF.
If there is no objections, I'll put "needs review" to move.
Best regards, Andrey Borodin.
> 30 нояб. 2017 г., в 23:07, Andrey Borodin <x4...@yandex-team.ru> написал(а):
>
> Seems like it was not a big deal of patching, I've fixed those bits (see
> attachment).
> I've done only simple tests as for now, but I'm planning to do better testing
> before next CF.
I have two concerns about this:
1. Does this affect the performance of the database when PTRACK is not enabled?
2. What is the cost of having PTRACK enabled?
Best regards, Andrey Borodin.
ary at
least to validate the idea. I'd sign up as co-mentor for this. Moreover this
will attract students from huge pool of ML hackers.
Also, I'm planning to add my previous-year project about sorting. Or add
something WAL-G-related under Postgres umbrella (probably, WAL-scanning for
delta-backup
Hi hackers!
Here is the patch that deletes pages during GiST VACUUM.
> 12 нояб. 2017 г., в 23:20, Andrey Borodin <x4...@yandex-team.ru> написал(а):
>
> If author and community do not object, I want to continue work on
> Konstantin's patch.
==Purpose==
Long story sh
earned index prototype
Expected outcome: Design for integration of learned indexes into PostgreSQL
(possibly, using index-as-extension technology), prototype implementation and
benchmarks.
Best regards, Andrey Borodin.
lot of magic numbers) to merge this
spikes into one event does not look promising to me. But this is just my
superstition, chances are that you can tune your algorithm into serious
advancement. But please publish benchmarks, whatever result will be.
Best regards, Andrey Borodin.
ir first NBlock/32 replacements).
>
I do not understand where these numbers come from...
Best regards, Andrey Borodin.
buffers.
Best regards, Andrey Borodin.
> 27 апр. 2018 г., в 23:23, Kefan Yang <starord...@gmail.com> написал(а):
>
> I think Slack would be great for our communication
Here's Slack invite app http://postgres-slack.herokuapp.com/
Should we create a channel for this project?
Best regards, Andrey Borodin.
Hi!
> 26 апр. 2018 г., в 0:12, Kefan Yang написал(а):
>
> My name is Kefan Yang. I am so excited that my proposal ‘Sorting Algorithm
> and Benchmarking 2018’ has been accepted.
Welcome! I'm also glad you've chosen this project.
> I see on the Wiki page that you can
ckend doesn't justify a
> separate usage count bump each time.
Upper in this thread Yura suggested that usages should not create equal bump
each time. He effectively suggested log scale of usages, thus many consecutive
usages will be taken into account but not dramatically more important than just
few recent usages.
Best regards, Andrey Borodin.
and decompress functions. If compress
function is omited, IndexOnlyScan is enabled for opclass without any extra
change.
https://github.com/postgres/postgres/commit/d3a4f89d8a3e500bd7c0b7a8a8a5ce1b47859128
<https://github.com/postgres/postgres/commit/d3a4f89d8a3e500bd7c0b7a8a8a5ce1b47859128>
e concept of
scan that you propose looks interesting.
I have few questions:
1. Charts are measured in percents of pgbench TPS, right?
2. For example, is 97% actually 3% degrade?
3. The results are obtained on actual "sort of TPC-B" script?
Best regards, Andrey Borodin.
to
restore database from backups and inspect it, if necessary. Though first
occurrence of this error was beyond recovery window.
Best regards, Andrey Borodin.
Hi, Ivan!
> 19 марта 2018 г., в 15:32, Andrey Borodin <x4...@yandex-team.ru> написал(а):
> I was looking into this patch mainly because I was reviewing other checksums
> patch in different thread. But the purpose of this patch seems viable for me.
> After looking into
Hi, Thomas!
> 17 мая 2018 г., в 9:40, Thomas Munro <thomas.mu...@enterprisedb.com>
> написал(а):
>
> On Wed, Mar 7, 2018 at 7:50 PM, Andrey Borodin <x4...@yandex-team.ru> wrote:
>> Here's new version of GiST VACUUM patch set aimed at CF 2018-09.
>
> Hi A
Hi!
> 30 апр. 2018 г., в 23:15, Andres Freund <and...@anarazel.de> написал(а):
> On 2018-04-30 15:39:08 +0500, Andrey Borodin wrote:
>> I suspect that performance benefits can be not that big or even
>> marginal, if we do not extend comprehensive eviction strateg
> 30 апр. 2018 г., в 0:48, Andres Freund <and...@anarazel.de> написал(а):
>
> On 2018-04-25 11:31:12 +0500, Andrey Borodin wrote:
>>
>> 1. Teaching BgWriter to used data from eviction strategy to aggressively
>> flush data to disk (instead of ++next_to_clea
Hi, Joshua!
> 2 мая 2018 г., в 22:05, Joshua D. Drake <j...@commandprompt.com> написал(а):
>
> Who is coordinating GSOC this year?
Stephen Frost is org admin.
Best regards, Andrey Borodin.
0] )
Best regards, Andrey Borodin.
[0]
https://www.postgresql.org/message-id/7780a07b-4d04-41e2-b228-166b41d07...@yandex-team.ru
Hi!
> 28 дек. 2017 г., в 16:37, Andrey Borodin <x4...@yandex-team.ru> написал(а):
> Here is new version of the patch for GiST VACUUM.
> There are two main changes:
> 1. During rescan for page deletion only know to be recently empty pages are
> rescanned.
> 2. I've re-i
hat it already has
implementation.
I'd suggest to do not 1\1 split for these pages, to keep tree lower. Like
fillfactor/(1-fillfactor) split. Also, this will make splits less frequent.
Best regards, Andrey Borodin.
think that patch is ready for committer.
Best regards, Andrey Borodin.
The new status of this patch is: Ready for Committer
Hello!
The patch does not apply currently.
Anastasia, can you, please, rebase the patch?
Best regards, Andrey Borodin.
e may as well fully embrace
> non-determinism.
I think that determinism across platforms is not that important as determinism
across runs.
Thanks for the amcheck! It is very useful.
Best regards, Andrey Borodin.
Hello everyone!29 нояб. 2017 г., в 22:50, Shubham Barai <shubhambara...@gmail.com> написал(а): I have fixed formatting style. Please take a look at updated patch.Here's rebased patch. Every issue has been addressed, so I'm marking this patch as ready for committer.Best regards, Andrey B
s.
Thanks for working on this feature, hope to see it in 11.
Best regards, Andrey Borodin.
level description in
> src/backend/access/gist/README. Given this description, it would be much
> easier to review the patches.
Yes, that's definitely true. Please find README patch attached.
Best regards, Andrey Borodin.
0003-Update-README-with-info-on-new-GiST-VACUUM.patch
Description: Binary data
I feel sorry for the noise, switching this patch there and back. But the patch
needs rebase again. It still applies with -3, but do not compile anymore.
Best regards, Andrey Borodin.
The new status of this patch is: Waiting on Author
> 21 янв. 2018 г., в 3:36, Peter Geoghegan <p...@bowt.ie> написал(а):
>
> On Wed, Jan 17, 2018 at 12:45 AM, Andrey Borodin <x4...@yandex-team.ru> wrote:
>> Unfortunately, amcheck_next does not work currently on HEAD (there are
>> problems with AllocSetContextCre
Hello, Alexander!
> 16 янв. 2018 г., в 21:42, Andrey Borodin <x4...@yandex-team.ru> написал(а):
> Please find README patch attached.
Here's v2 version. Same code, but x2 comments. Also fixed important typo in
readme BFS->DFS. Feel free to ping me any time with questions.
Best
w versions that must be removed.
[install]check[-world] passed :)
From my POV, patch is in a good shape.
I think it is time to make the patch ready for committer again.
Best regards, Andrey Borodin.
ugh there are tests in cube and seg for that, if your patch passes
check-world, than this behavior is not affected.
Thank you for the patch!
Best regards, Andrey Borodin.
on our
> projects page here:
I've added project for amcheck enhancement to wiki, I hope it's not too late.
Currently, only I'm listed as a mentor (same for WAL-G project too), but, I
hope, we will find backup mentor if necessary.
Best regards, Andrey Borodin.
and Postgres Pro already
have various flavours of similar feature. Maybe they are planning to publish
that?
Best regards, Andrey Borodin.
ization this
year.
https://summerofcode.withgoogle.com/organizations/6357583019900928/
<https://summerofcode.withgoogle.com/organizations/6357583019900928/>
Time to attract some great students :)
Best regards, Andrey Borodin.
s this, patch looks good. Please, add it to the following commitfest so
that work on the patch could be tracked.
Best regards, Andrey Borodin.
l to some callers.
Bloom filter behaves deterministically, but differently. This does not ensures
any thing, but probably will give something with hight probability.
Thanks!
Best regards, Andrey Borodin.
Hi, David!
> 7 февр. 2018 г., в 18:39, David Steele <da...@pgmasters.net> написал(а):
>
> Hi Andrey,
>
> On 1/21/18 5:34 AM, Andrey Borodin wrote:
>> Hello, Alexander!
>>> 16 янв. 2018 г., в 21:42, Andrey Borodin <x4...@yandex-team.ru> написал
Hi everyone!
> 10 февр. 2018 г., в 20:45, Andrey Borodin <x4...@yandex-team.ru> написал(а):
>
> I'm planning to provide review
>
So, I was looking into the patch.
The patch adds:
1. Ability to specify collation provider (with version) in --locale for initdb
and cre
wiki.postgresql.org/wiki/Index-only_scans
I do not know if it is possible to take into account this optimization in cost
estimation.
Does Limit node take cost of scanning into startup cost?
Thanks!
Best regards, Andrey Borodin.
eads mentioned by
Magnus [2,3]
Or do you mean extracting "worker+lancher" for reuse for other purposes?
Best regards, Andrey Borodin.
[0]
https://www.postgresql.org/message-id/flat/E2B195BF-7AA1-47AF-85BE-0E936D157902%40endpoint.com#e2b195bf-7aa1-47af-85be-0e936d157...@endpoint.com
&
Hi hackers!
> 19 дек. 2017 г., в 15:58, Andrey Borodin <x4...@yandex-team.ru> написал(а):
> Here is the patch that deletes pages during GiST VACUUM.
Here is new version of the patch for GiST VACUUM.
There are two main changes:
1. During rescan for page deletion only know to be re
ll be a little bit easier to
read.
Best regards, Andrey Borodin.
[0]
https://github.com/postgres/postgres/blob/master/contrib/pageinspect/heapfuncs.c#L439
Hi!
> 21 дек. 2017 г., в 5:51, Michael Paquier
> написал(а):
>
> On Thu, Dec 21, 2017 at 7:35 AM, Robert Haas wrote:
>> On Wed, Dec 20, 2017 at 3:45 PM, Tomas Vondra
>> wrote:
Isn't more effective hold this
e()). +1 was a
tough thing for me to understand before looking around and reading those
comments.
4. I could not understand purpose of this expression
page[BLCKSZ - 1] & 0X00FF
Happy New Year :) Keep up good work.
Best regards, Andrey Borodin.
Hi, Thomas!29 июля 2018 г., в 14:28, Thomas Munro <thomas.mu...@enterprisedb.com> написал(а):On Fri, Jul 6, 2018 at 5:27 AM, Andrey Borodin <x4...@yandex-team.ru> wrote:Here is v3 version of the patch. I've fixed some comments and added some words to docs.Hi again Andrey,Cf
Hi!
Thank you!
> 29 июля 2018 г., в 14:04, Thomas Munro
> написал(а):
>
> On Tue, Jul 24, 2018 at 6:04 AM, Andrey Borodin wrote:
>>> 21 июля 2018 г., в 17:11, Andrey Borodin написал(а):
>>> <0001-Physical-GiST-scan-in-VACUUM-v13.patch>
>>
>>
rn BOX *box_copy(const BOX *box);
I'm not sure in architectural point of view: supporting two ways (list and
heap) to store result seems may be a bit heavy, but OK. At least, it has
meaningful benefits.
I think the patch is ready for committer.
Best regards, Andrey Borodin.
Hi! Thanks for looking into the patch!
> 30 июля 2018 г., в 18:39, Heikki Linnakangas написал(а):
>
> On 29/07/18 14:47, Andrey Borodin wrote:
>> Fixed both problems. PFA v14.
>
> Thanks, took a really quick look at this.
>
> The text being added to README is outdat
writing bad writes.
But adding LSNs, and whole regular PageHeader is quite easy in this patch. Do
you think we should really go that way?
Putting SLRUs into usual shared buffers and WAL-logging looks like a good idea,
but a lot of work.
Best regards, Andrey Borodin.
Hi!
> 5 авг. 2018 г., в 16:18, Heikki Linnakangas написал(а):
>
> On 31/07/18 23:06, Andrey Borodin wrote:
>>> On a typical GiST index, what's the ratio of leaf vs. internal
>>> pages? Perhaps an array would indeed be better.
> >
>> Typical GiST has ar
Thanks, Thomas!
> 30 июля 2018 г., в 3:58, Thomas Munro
> написал(а):
>
> On Sun, Jul 29, 2018 at 10:50 PM, Andrey Borodin wrote:
>> Thanks! The problem appeared with commit 701fd0b [0] which dropped
>> validation rules checked in failed test. Here's the patch with
Hi!
PFA v16.
> 5 авг. 2018 г., в 21:45, Andrey Borodin написал(а):
>> 5 авг. 2018 г., в 16:18, Heikki Linnakangas написал(а):
>>
>> Hmm. A ListCell is 16 bytes, plus the AllocChunk header, 16 bytes. 32
>> bytes per internal page in total, while a bitmap consumes
mething, but why this should not be in access methods?
You can extend AM to control it's segment size and ability to truncate unneeded
pages. This may to be useful, for example, in LSM tree implementation or
something similar.
Best regards, Andrey Borodin.
_ARRAY_ELEMENT_ERROR),
+errmsg("A cube cannot have more than %d
dimensions.",
+ CUBE_MAX_DIM)));
dur = ARRPTR(ur);
Best regards, Andrey Borodin.
> 28 авг. 2018 г., в 14:18, Alexander Korotkov
> написал(а):
>
> OK, but I think cube_c_f8() and cube_c_f8_f8() also need to be
> revised. Also, I think this behavior should be covered by regression
> tests.
True. Also there's one case in cube_subset.
Best regar
od will stay safe forever.
But now it is.
Best regards, Andrey Borodin.
ry hard to test. Also, this makes impossible to use two
archiving system simultaneously for transit period.
Best regards, Andrey Borodin.
> 28 авг. 2018 г., в 17:07, Stephen Frost написал(а):
>
> Greetings,
>
> * Andrey Borodin (x4...@yandex-team.ru) wrote:
>>> 28 авг. 2018 г., в 14:08, Stephen Frost написал(а):
>>> * David Steele (da...@pgmasters.net <mailto:da...@pgmasters.net>) wrote:
he attached patch applied, the above messages becomes as
> follows. (And index can be built being a bit sparse by fill
> factor.)
We are passing freespace everywhere. Also, we pass GistInsertState, and
GistState.
Maybe let's put GistState into GistInsertState, GistState already has free
space, and pass just GistInsertState everywhere?
Best regards, Andrey Borodin.
Hi!
> 19 июля 2018 г., в 23:26, Andrey Borodin написал(а):
>
> I'm working on triggering left split during vacuum. Will get back when done.
> Thanks!
Here's patch including some messy hacks to trigger NSN and FollowRight jumps
during VACUUM.
To trigger FollowRight GiST some
Hi!
> 16 июля 2018 г., в 21:24, Andrey Borodin написал(а):
>
I was checking WAL replay of new scheme to log page deletes and found a bug
there (incorrect value of deleted downlink in WAL record). Here's fixed patch
v10.
Also I've added support to WAL identification for new record, don
Hi, Tomas!
> 15 июля 2018 г., в 1:20, Tomas Vondra
> написал(а):
>
> So I doubt it's this, but I've tweaked the scripts to also set this GUC
> and restarted the tests on both machines. Let's see what that does.
Do you observe any different results?
Thanks!
Best regards, Andrey Borodin.
didn't found one.
Should it be one-time test script or something "make check-world"-able?
Best regards, Andrey Borodin.
cycle, without memorizing anything.
Actually, you do not even need to rescan rightlinks: there may be no splits to
the left when no pages are deleted.
If you think it is proper way to go - OK, I'll prepare better version of
attached diff (by omitting tail recursion and adding more comments).
Best regards, Andrey Borodin.
gist_phisical_vacuum_v1.diff
Description: Binary data
ave their
> own associated scariness.
Thanks, that's exactly what I'm thinking about where to store this xid.
Here's v9 of the patch, it uses pd_prune_xid, but it is abstracted to
GistPageGetDeleteXid() \ GistPageSetDeleteXid() so that we can change the way
we store it easily.
Best regards,
er one
very important topic. When we use CREATE INDEX CONCURRENTLY we pay for less
intrusive lock by scanning data twice. What is the price of RENAME
CONCURRENTLY? Should this mode be default?
Best regards, Andrey Borodin.
erTuple->allTheSame)
+ if (out.nNodes != 0 && out.nNodes != nNodes)
+ elog(ERROR, "inconsistent inner_consistent results for
allTheSame inner tuple");
+
+ if (out.nNodes) // few lines before you compare with 0
Best regards, Andrey Borodin.
Hi!
> 21 июля 2018 г., в 17:11, Andrey Borodin написал(а):
>
> <0001-Physical-GiST-scan-in-VACUUM-v13.patch>
Just in case, here's second part of patch series with actual page deletion.
I was considering further decreasing memory footprint by using bloom filters
in
19.07.2018, 15:20, "Heikki Linnakangas" :On 19/07/18 13:52, Andrey Borodin wrote: Hi! 19 июля 2018 г., в 1:12, Heikki Linnakangas <hlinn...@iki.fi> написал(а): Yeah, please, I think this is the way to go. Here's v11 divided into proposed steps.Thanks, on
physical scan only
reads internal pages. I can omit that bitmap, if I'll scan everything.
Also, I can replace emptyLeafs bitmap with array\list, but I do not really
think it will be big.
Anyway I propose focusing on first step.
Best regards, Andrey Borodin.
0002-Delete-pages-during-GiST-V
page before the split, we see all the leaf tuples on that page. If
> we scan the single page after the split, it means that we start the scan
> after the split, and we will see both leaf pages as we continue the scan.
Yes, only page 0 may change type, and page 0 cannot split to left.
I'm working on triggering left split during vacuum. Will get back when done.
Thanks!
Best regards, Andrey Borodin.
> 14 июля 2018 г., в 0:28, Heikki Linnakangas написал(а):
>
> On 13/07/18 21:28, Andrey Borodin wrote:
>>> 13 июля 2018 г., в 18:25, Heikki Linnakangas
>>> написал(а):
>>> Looking at the second patch, to scan the GiST index in physical
>>> order
> 14 июля 2018 г., в 14:39, Heikki Linnakangas написал(а):
>
> On 14/07/18 10:26, Andrey Borodin wrote:
>> This is tradeoff between complex concurrency feature and possibility
>> of few dead tuples left after VACUUM. I want to understand: is it
>> something da
100)::int from generate_series(1,1e7);
it brings something like 2-4% of performance improvement on my laptop.
Is there a reason why we do not use __builtin_prefetch? Have anyone tried to
use cache prefetching?
Best regards, Andrey Borodin.
> 5 сент. 2018 г., в 13:29, Kyotaro HORIGUCHI
> написал(а):
>
> We don't preserve it for Andrey's use case.
Just my 2 cents: that was a hacky use case for development reasons. I think
that removing fillfactor is good idea and your latest patch looks good from my
POV.
Best reg
s somewhat a downside. B-tree is kind of a rocket
science. Chances are you have some kind of roadmap of B-tree things to
implement in nearest years?
Best regards, Andrey Borodin.
[0]
https://www.postgresql.org/message-id/20160814171131.21390.75752.pgcf%40coridan.postgresql.org
Hi!
> 13 апр. 2018 г., в 17:36, Andrey Borodin написал(а):
>
> Here's V2, with basic set of tests.
> Currently, I'm investigating what to document and more places to tests.
Here is v3 version of the patch. I've fixed some comments and added some words
to docs.
Best regards, An
.
"ERROR: including column does not support ASC/DESC options" and "ERROR:
including column does not support NULLS FIRST/LAST options".
IMHO it is not a bug. CREATE INDEX ON some_table(id+0) INCLUDE (id); or some
similar tricks will work anyway.
Best regards, Andrey Borodin.
egards, Andrey Borodin.
[0] https://commitfest.postgresql.org/14/1215/
lit (page merge) concurrency like in Lanin
and Shasha's paper, but it is just too complex for little purpose. Will add
comments on possible orphan pages.
Many thanks! I hope to post updated patch series this week.
Best regards, Andrey Borodin.
least
> would be good.
Added explanatory comment between WAL-logging downlink removal and marking
pages deleted.
Thank you for reviewing the patch!
Best regards, Andrey Borodin.
0002-Physical-GiST-scan-during-VACUUM-v5.patch
Description: Binary data
0001-Delete-pages-during-GiST-VACUUM-v5.patch
Description: Binary data
> 12 июля 2018 г., в 20:40, Heikki Linnakangas написал(а):
>
> On 12/07/18 19:06, Andrey Borodin wrote:
>>> 11 июля 2018 г., в 0:07, Heikki Linnakangas
>>> написал(а):
>>> This seems misplaced. This code deals with internal pages, and as
>>> far
s lossy");
Some small typos:
usefull -> useful
nearest-neighbour -> nearest-neighbor (or do we use e.g. "colour"?)
Best regards, Andrey Borodin.
> 4 июля 2018 г., в 19:22, Tom Lane написал(а):
>
> Andrey Borodin writes:
>> There is bunch of code in current GiST implementation checking for
>> GistTupleIsInvalid(). PostgreSQL since 9.1 do not create invalid tuples.
>> Should we support this tuples
ber the
> current LSN position at the beginning of the vacuum, and compare with that.
> The B-tree code uses the "cycle ID" for similar purposes.
>
> Do we still need the separate gistvacuumcleanup() pass, if we scan the index
> in physical order in the bulkdelete pa
hich was required before 11.
I think, explain results will be identical.
Best regards, Andrey Borodin.
ck could be removed from loop. We do not expect that
it's state will change during execution, do we?
Besides this, I think the patch is ready for committer.
Best regards, Andrey Borodin.
list.
We were discussing this with Shubham back in July, chosen some approach that
seemed better, but I can't remember what was that...
Best regards, Andrey Borodin.
is ready for committer.
Best regards, Andrey Borodin.
The new status of this patch is: Ready for Committer
ike implement utility for offline checksum enabling, without
WAL-logging, surely.
Thanks for the patch!
Best regards, Andrey Borodin.
-log page whenever it is checksummed by worker? Even if
the checksum was correct?
Best regards, Andrey Borodin.
e provide backporting for such features?
Best regards, Andrey Borodin.
seems outside of the patch scope and I believe it's better avoid messing with
SSI internals without strong reason.
Best regards, Andrey Borodin.
will prevent this kind of problems with enabling
fastupdate.
Best regards, Andrey Borodin.
lse
implement more algorithms.
amcheck. June - implement checks for Gin (like b-tree in b-tree, resembles
existing amcheck), July - checks for Hash, BRIN and SP-GiST, August - RUM,
patch, commitfest, reviews.
Best regards, Andrey Borodin.
1 - 100 of 609 matches
Mail list logo