Re: nbtsort.c performs unneeded (though harmless) truncation

2018-05-04 Thread Peter Geoghegan
On Fri, May 4, 2018 at 2:39 AM, Teodor Sigaev <teo...@sigaev.ru> wrote: > Thank you, pushed. Thanks. -- Peter Geoghegan

Re: pageinspect get_raw_page() option to return invalid pages

2018-05-04 Thread Peter Geoghegan
onally, > or only if the item wasn't found in cache. There's some coherency > differences, obviously. +1. This would be a good option for amcheck, too. -- Peter Geoghegan

Re: pageinspect get_raw_page() option to return invalid pages

2018-05-04 Thread Peter Geoghegan
page. Therefore I was more thinking to falling back to smgrread() or > such. I'm not envisaging anything specific just yet. It would be nice if amcheck had an option that bypassed shared_buffers, because users want that. That's all. -- Peter Geoghegan

Re: pageinspect get_raw_page() option to return invalid pages

2018-05-04 Thread Peter Geoghegan
e to focus on making this accessible to backup tools, that should ideally work without needing to acquire any MVCC snapshot. Probably from a front-end utility. We'd need to export at least some operator class functionality to make that work, though. -- Peter Geoghegan

Re: [HACKERS] Clock with Adaptive Replacement

2018-05-09 Thread Peter Geoghegan
On Thu, May 3, 2018 at 5:16 PM, Peter Geoghegan <p...@bowt.ie> wrote: > On Thu, May 3, 2018 at 12:37 PM, Robert Haas <robertmh...@gmail.com> wrote: >> On Wed, May 2, 2018 at 3:06 PM, Vladimir Sitnikov >> <sitnikov.vladi...@gmail.com> wrote: >>>

Re: [SPAM] Re: Local partitioned indexes and pageinspect

2018-05-09 Thread Peter Geoghegan
little point in changing > the error handling, still it would be good to get test coverage. That's > not really a bug though as in all those cases we don't get errors like > "could not open file" or such. So we could also let things as they are > now. Now that the relkind iss

Re: Compiler warnings with --enable-dtrace

2018-05-11 Thread Peter Geoghegan
to make it more accessible, though it's already quite accessible. [1] https://lwn.net/Articles/753601/ [2] https://github.com/iovisor/bcc/blob/master/tools/trace_example.txt -- Peter Geoghegan

Re: Compiler warnings with --enable-dtrace

2018-05-12 Thread Peter Geoghegan
On Fri, May 11, 2018 at 11:46 PM, Christoph Berg <m...@debian.org> wrote: > Re: Peter Geoghegan 2018-05-12 >

Re: Postgres 11 release notes

2018-05-12 Thread Peter Geoghegan
DD COLUMN sets a default value for a column". Referencing the fact that Postgres was previously able to do this with a NULL default doesn't seem to add anything. -- Peter Geoghegan

Re: Postgres 11 release notes

2018-05-11 Thread Peter Geoghegan
On Fri, May 11, 2018 at 12:17 PM, Peter Geoghegan <p...@bowt.ie> wrote: > * Suggest replacement sort item be phrased as: "Remove the > configuration parameter replacement_sort_tuples. The > replacement selection sort algorithm is no longer used." Also, it should be moved

Re: Postgres 11 release notes

2018-05-11 Thread Peter Geoghegan
ter replacement_sort_tuples. The replacement selection sort algorithm is no longer used." -- Peter Geoghegan

Re: Fix for FETCH FIRST syntax problems

2018-05-20 Thread Peter Geoghegan
other debating something that's inconsequential? FWIW, I am neutral on the important question of whether or not this patch should actually be back-patched. -- Peter Geoghegan

Re: Compiler warnings with --enable-dtrace

2018-05-24 Thread Peter Geoghegan
to do this stuff without the user having to read a tedious explanation of how the different parts fit together. -- Peter Geoghegan

Re: Add CONTRIBUTING.md

2018-06-09 Thread Peter Geoghegan
don't you start a thread about it? I'd actually even suggest > backpatching it. PostGIS has one of these too, which covers several languages, including SQL. Perhaps it would be worth tightening up the indentation of SQL files while we're at it. -- Peter Geoghegan

Re: JIT documentation fixes

2018-06-09 Thread Peter Geoghegan
ve. That change was also left out. If I'm mistaken in how I interpreted the sentence, then Andres should follow up. Thanks -- Peter Geoghegan

Re: JIT documentation fixes

2018-06-09 Thread Peter Geoghegan
On Sat, Jun 9, 2018 at 10:40 AM, Daniel Gustafsson wrote: > Thanks, I’m honoured to get the first commit. Keep them coming. :-) -- Peter Geoghegan

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-06-18 Thread Peter Geoghegan
el as they are matched/killed. It should be safe to avoid rechecking anything other than the heap TID values. [1] http://pgeoghegan.blogspot.com/2017/07/postgresql-index-bloat-microscope.html [2] https://www.postgresql.org/message-id/CAH2-Wzmf6intNY1ggiNzOziiO5Eq=DsXfeptODGxO=2j-i1...@mail.gmail.com -- Peter Geoghegan

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-06-18 Thread Peter Geoghegan
(unless perhaps you assume that that problem is solved by something else, such as zheap). The mechanism that Andrey describes is rather unlike VACUUM as we know it today, but that's the whole point. -- Peter Geoghegan

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2018-06-15 Thread Peter Geoghegan
on't quite see a clear path to making this patch useful. But, as I said, I have an open mind about what the next step should be. [1] https://wiki.postgresql.org/wiki/Key_normalization#Avoiding_unnecessary_unique_index_enforcement -- Peter Geoghegan

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2018-06-18 Thread Peter Geoghegan
e we at least know that the deleting transaction committed there. That is, they could be recently dead or dead, and it may be worth going to the extra trouble of checking which when we know that it's one of the two possibilities. -- Peter Geoghegan

Re: Removing "Included attributes in B-tree indexes" section from docs

2018-06-18 Thread Peter Geoghegan
On Mon, Jun 18, 2018 at 10:21 AM, Alvaro Herrera wrote: > One which includes at least half a working day in a different timezone. > You asked mid-afternoon on a Friday in a timezone pretty far west. It was 11 am PST. I'll make a note about this. It won't happen again. -- Peter Geoghegan

Re: Invisible Indexes

2018-06-18 Thread Peter Geoghegan
uld > become a new unreserved keyword. > So, do we want this feature? If we do I'll go ahead and prepare a patch. I know that it's definitely a feature that I want. Haven't thought about the syntax, though. -- Peter Geoghegan

Re: Invisible Indexes

2018-06-18 Thread Peter Geoghegan
ic collector stats masking a problem, the possibility that there are very important queries that do use the index but are only run very infrequently, and so on. -- Peter Geoghegan

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-06-18 Thread Peter Geoghegan
l/root/build/../source/src/backend/access/nbtree/nbtree.c:1074:3: warning: ‘blkno’ may be used uninitialized in this function [-Wmaybe-uninitialized] cleanup_block(info, stats, blkno); ^ I think that they're both harmless, though. -- Peter Geoghegan

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-06-18 Thread Peter Geoghegan
On Mon, Jun 18, 2018 at 2:54 PM, Peter Geoghegan wrote: > On Sun, Jun 17, 2018 at 9:39 PM, Andrey V. Lepikhov > wrote: >> Patch '0001-retail-indextuple-deletion' introduce new function >> amtargetdelete() in access method interface. Patch >> '0002-quick-vacuum-strategy'

Re: Invisible Indexes

2018-06-18 Thread Peter Geoghegan
eature. Oracle has a similar feature. -- Peter Geoghegan

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-06-18 Thread Peter Geoghegan
On Mon, Jun 18, 2018 at 4:05 PM, Peter Geoghegan wrote: > Finally, doing things this way would let you delete multiple > duplicates in one shot, as I described in an earlier e-mail. Only a > single descent of the tree is needed to delete quite a few index > tuples, provided that the

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-06-18 Thread Peter Geoghegan
On Mon, Jun 18, 2018 at 4:05 PM, Peter Geoghegan wrote: > IOW, the approach you've taken in bttargetdelete() isn't quite correct > because you imagine that it's okay to occasionally "lose" the index > tuple that you originally found, and just move on. That needs to be >

Re: Duplicate Item Pointers in Gin index

2018-06-13 Thread Peter Geoghegan
s issue is that gin indexes > could return a correct result without no assertion failure even if it > somewhat corrupted. So maybe having amcheck for gin indexes would > resolve part of problems. That seems like a really good idea. As you know, I have misgivings about this area. -- Peter Geoghegan

Re: Locking B-tree leafs immediately in exclusive mode

2018-06-13 Thread Peter Geoghegan
an also imagine someone seeing something that I missed. -- Peter Geoghegan

Re: Locking B-tree leafs immediately in exclusive mode

2018-06-14 Thread Peter Geoghegan
ue. I think we will need an option for that. > I'm not yet sure if this option should be user-visible or > auto-decision based on table access method used. I agree. -- Peter Geoghegan

Making all nbtree entries unique by having heap TIDs participate in comparisons

2018-06-14 Thread Peter Geoghegan
z-ktrqiaa13xg1gne461yowra-s-yccqptyfrpkta...@mail.gmail.com [3] https://wiki.postgresql.org/wiki/Key_normalization#Suffix_truncation_of_normalized_keys -- Peter Geoghegan 0001-Ensure-nbtree-leaf-tuple-keys-are-always-unique.patch Description: Binary data

Removing "Included attributes in B-tree indexes" section from docs

2018-06-15 Thread Peter Geoghegan
umns isn't appropriate. It seems sufficient to only mention this once, in the CREATE INDEX docs. Attached patch shows what I have in mind -- the total removal of this top-level section. -- Peter Geoghegan From 293f1fc93771fa6be65867f53aa507fb30137230 Mon Sep 17 00:00:00 2001 From: Peter Geoghegan Dat

Re: Removing "Included attributes in B-tree indexes" section from docs

2018-06-17 Thread Peter Geoghegan
ection was not well considered in the first place, I thought that the question was clear cut, and I doubted that anyone else would follow up at all. -- Peter Geoghegan

Re: Invisible Indexes

2018-06-19 Thread Peter Geoghegan
e -- not just the current session. -- Peter Geoghegan

Re: Fast default stuff versus pg_upgrade

2018-06-19 Thread Peter Geoghegan
't seem like anything we can't fix with acceptable risk. I agree with both points. -- Peter Geoghegan

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-06-19 Thread Peter Geoghegan
aybe that alone will be enough to make the patch worth committing; "opportunistic microvacuuming" can come later, if at all. -- Peter Geoghegan

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2018-06-19 Thread Peter Geoghegan
to the heap tuple via an index scan if there are no index tuples that stick around until "recently dead" heap tuples become "fully dead"? How can you avoid keeping around both old and new index tuples at the same time? -- Peter Geoghegan

Re: MERGE SQL statement for PG12

2018-06-19 Thread Peter Geoghegan
On Tue, Jun 19, 2018 at 4:06 AM, Pavan Deolasee wrote: > I would like to resubmit the MERGE patch for PG12. The past discussions > about the patch can be found here [1] [2]. FWIW, I'm really glad that you're going to work on this for v12. -- Peter Geoghegan

Re: Time to put context diffs in the grave

2018-06-19 Thread Peter Geoghegan
the only one that doesn't use context diffs and also doesn't hate them with a passion? I don't get it. > There's already a (perhaps underdocumented) way to make regression > diffs look the way you want in local runs. Actually, it is documented. -- Peter Geoghegan

Re: SP-GiST failing to complete SP-GiST index build

2018-05-27 Thread Peter Geoghegan
an infinite * picksplit loop. */ CHECK_FOR_INTERRUPTS(); Perhaps the problem is in the range type SP-GiST opclass support code - it could have this exact picksplit infinite loop problem. That's perfectly consistent with what we see here. -- Peter Geoghegan

Re: SP-GiST failing to complete SP-GiST index build

2018-05-27 Thread Peter Geoghegan
On Sun, May 27, 2018 at 2:09 PM, Peter Geoghegan <p...@bowt.ie> wrote: > On Sun, May 27, 2018 at 12:45 PM, Jonathan S. Katz <jk...@postgresql.org> > wrote: >> Next, see bad.sql. 1.2MM sparsely clustered rows inserted, GiST indexes >> builds in about 30s on my ma

Re: New committers announced at PGCon 2018

2018-06-01 Thread Peter Geoghegan
Thanks! Congratulations to the other 6 new committers. -- Peter Geoghegan (Sent from my phone)

Re: Code of Conduct plan

2018-06-05 Thread Peter Geoghegan
s of enforcement, etc. Naturally, the rules across disparate groups vary widely for all kinds of reasons. Formalizing and being more transparent about how this works seems like the opposite of paternalism to me. -- Peter Geoghegan

Re: SP-GiST failing to complete SP-GiST index build

2018-05-27 Thread Peter Geoghegan
l screwed in this situation. It does finish eventually, but in about > 50x longer than GiST. I imagine the index's query performance would be > equally awful. Can you think of some way of side-stepping the issue? It's unfortunate that SP-GiST is potentially so sensitive to input order. -- Peter Geoghegan

Re: SP-GiST failing to complete SP-GiST index build

2018-05-27 Thread Peter Geoghegan
On Sun, May 27, 2018 at 5:24 PM, Peter Geoghegan <p...@bowt.ie> wrote: > On Sun, May 27, 2018 at 5:10 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: >> Instrumenting the test case suggests that getQuadrant pretty much always >> returns 1, resulting in a worst-case unbal

Re: Bulk Insert into PostgreSQL

2018-07-01 Thread Peter Geoghegan
rting about 500 million records. Periodic pstack > runs on Linux showed that the backend was busy in btree operations. I didn't > pursue the cause due to other businesses, but there might be something to be > improved. What kind of data was indexed? Was it a bigserial primary key, or so

Re: Allow cancelling VACUUM of nbtrees with corrupted right links

2018-06-27 Thread Peter Geoghegan
ral, page deletion is the most complicated part of nbtree concurrency, by far (if we just had the basic L, the concurrency aspects would be far easier to grasp). Doing better in _bt_unlink_halfdead_page() seems extremely difficult, and very unlikely to be worthwhile. -- Peter Geoghegan

Re: Allow cancelling VACUUM of nbtrees with corrupted right links

2018-06-27 Thread Peter Geoghegan
as needed. You're probably looking at a test-case that doesn't involve a multi-level deletion, where the leaf and target page are the same. Not sure if you want to reproduce the multi-level deletion case independently. -- Peter Geoghegan

Re: Allow cancelling VACUUM of nbtrees with corrupted right links

2018-06-27 Thread Peter Geoghegan
On Wed, Jun 27, 2018 at 1:11 PM, Andres Freund wrote: > Well, I don't really want to generally do better. Just be able to check > for interrupts ;) That's what I meant. :-) -- Peter Geoghegan

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-06-27 Thread Peter Geoghegan
m unsure of that, though. -- Peter Geoghegan

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-06-27 Thread Peter Geoghegan
between _bt_delitems_delete() and _bt_delitems_vacuum(), and the considerations for hot standby? I think that that's another TODO list item for this patch. -- Peter Geoghegan

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-06-22 Thread Peter Geoghegan
On Fri, Jun 22, 2018 at 12:43 PM, Peter Geoghegan wrote: > On Fri, Jun 22, 2018 at 4:24 AM, Andrey V. Lepikhov > wrote: >> According to your feedback, i develop second version of the patch. >> In this version: >> 1. High-level functions index_beginscan(), index_rescan() n

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-06-22 Thread Peter Geoghegan
at my patch has are also problems for your patch. One patch cannot get committed without the other, so they are already the same project. As a bonus, my patch will probably improve the best case performance for your patch, since multi-deletions will now have much better locality of access. -- Peter Geoghegan

Removing obsolete comment block at the top of nbtsort.c.

2018-06-24 Thread Peter Geoghegan
t presorted SERIAL-like inputs, especially when parallelism is used. Back in 1999, there was no WAL-logging. -- Peter Geoghegan From 2a77b947e07dadef953019334062580cad560836 Mon Sep 17 00:00:00 2001 From: Peter Geoghegan Date: Sun, 24 Jun 2018 11:41:16 -0700 Subject: [PATCH] Remove obsolete comment

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2018-06-19 Thread Peter Geoghegan
s of that technology. I don't understand, but okay. I can provide feedback once a design for delete marking is available. -- Peter Geoghegan

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2018-07-02 Thread Peter Geoghegan
On Thu, Jun 14, 2018 at 11:44 AM, Peter Geoghegan wrote: > I attach an unfinished prototype of suffix truncation, that also > sometimes *adds* a new attribute in pivot tuples. It adds an extra > heap TID from the leaf level when truncating away non-distinguishing > attributes during

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-07-02 Thread Peter Geoghegan
On Mon, Jul 2, 2018 at 9:28 AM, Peter Geoghegan wrote: >> Execution time of last "VACUUM test;" command on my notebook was: >> >> with bulk deletion: 1.6 s; >> with Quick Vacuum Strategy: 5.2 s; >> with Quick Vacuum Strategy & TID sorting: 0.6 s. >

Re: Fix to not check included columns in ANALYZE on indexes

2018-06-30 Thread Peter Geoghegan
ng any index at all, which Tom referred to in passing. That's why I understand his remarks in this way. -- Peter Geoghegan

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-07-02 Thread Peter Geoghegan
assing the lowest heap TID in the list of TIDs to kill to _bt_search() and _bt_binsrch(). You might have to work through several extra B-Tree leaf pages per bttargetdelete() call without this (you'll move right multiple times within bttargetdelete()). -- Peter Geoghegan

nbtsort.c performs unneeded (though harmless) truncation

2018-04-30 Thread Peter Geoghegan
BTreeTupleGetNAtts() assertions. This isn't a correctness issue, and the extra overhead of unneeded truncation should be negligible, but what we have now seems confusing to me. -- Peter Geoghegan 0001-Don-t-truncate-away-non-key-attributes-for-leftmost-.patch Description: Binary data

Re: [HACKERS] Clock with Adaptive Replacement

2018-05-01 Thread Peter Geoghegan
l > that using wall-clock time is the right way to solve that problem > because it leads to edge effects I agree that wall-clock time is a bad approach, actually. If we were to use wall-clock time, we'd only do so because it can be used to discriminate against a thing that we actually care about in an approximate, indirect way. If there is a more direct way of identifying correlated accesses, which I gather that there is, then we should probably use it. -- Peter Geoghegan

Re: Sort performance cliff with small work_mem

2018-05-02 Thread Peter Geoghegan
32 kB of memory left after that >> (including, if it went below 0), then we bump availMem back up to 32 kB. So >> we'd always reserve 32 kB to hold the tuples, even if that means that we >> exceed 'work_mem' slightly. > > Sounds very reasonable. +1 -- Peter Geoghegan

Re: Sort performance cliff with small work_mem

2018-05-02 Thread Peter Geoghegan
e minimum additional amount of memory should be big enough that you have some chance of using all 1024 SortTuples (the whole memtuples array). -- Peter Geoghegan

Re: Sort performance cliff with small work_mem

2018-05-02 Thread Peter Geoghegan
maybe a new preprocessor constant): state->memtupsize = Max(1024, ALLOCSET_SEPARATE_THRESHOLD / sizeof(SortTuple) + 1); The ALLOCSET_SEPARATE_THRESHOLD part can become a static assertion. -- Peter Geoghegan

Re: Sort performance cliff with small work_mem

2018-05-02 Thread Peter Geoghegan
oint that the sort starts, though. -- Peter Geoghegan

Re: Sort performance cliff with small work_mem

2018-05-02 Thread Peter Geoghegan
rting extremely wide tuples. -1 from me. What about the case where only some tuples are massive? -- Peter Geoghegan

Re: [HACKERS] Clock with Adaptive Replacement

2018-04-26 Thread Peter Geoghegan
or index pages and visibility map > pages ought to show up as fiery hot on workloads where the index or > visibility map, as the case may be, is heavily used. I don't think that internal index pages and the visibility map are the problem, since there is so few of them, and they are accessed at least hundreds of times more frequently than the leaf pages. -- Peter Geoghegan

Re: [HACKERS] Clock with Adaptive Replacement

2018-04-25 Thread Peter Geoghegan
lem is it that we waste memory bandwidth copying to and from OS cache, particularly on large systems? Might that be the bigger problem, but also one that can be addressed incrementally? I think that I may be repeating some of what Andrey said, in another way (not sure of that). -- Peter Geoghegan

Re: Fsync request queue

2018-04-30 Thread Peter Geoghegan
ero pg_stat_bgwriter.buffers_backend_fsync in the wild after the compaction queue stuff was added/backpatched? -- Peter Geoghegan

Re: Local partitioned indexes and pageinspect

2018-04-29 Thread Peter Geoghegan
that generalizes from that won't have a problem, but it would be nice if it had a friendlier message. I didn't change amcheck to account for this myself because I thought it was possible that somebody else would want to use a more general solution. -- Peter Geoghegan

Re: BufFileSize() doesn't work on a "shared" BufFiles

2018-04-30 Thread Peter Geoghegan
called BufFileSeek() > or BufFileRead() on the shared BufFile handle before calling > BufFileAppend(), it would get confused. Seems worth fixing. -- Peter Geoghegan

ignore_system_indexes affects DROP SCHEMA ... CASCADE reported number of objects dropped

2018-05-03 Thread Peter Geoghegan
CADE; ! NOTICE: drop cascades to 17 other objects --- 676,679 -- \set VERBOSITY terse DROP SCHEMA collate_tests CASCADE; ! NOTICE: drop cascades to 20 other objects -- Peter Geoghegan

Re: ignore_system_indexes affects DROP SCHEMA ... CASCADE reported number of objects dropped

2018-05-03 Thread Peter Geoghegan
On Thu, May 3, 2018 at 3:18 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Peter Geoghegan <p...@bowt.ie> writes: >> Why should the drop cascade to 63 objects rather than 62 because I've >> set ignore_system_indexes=on? > > Indeed, that seems weird. Maybe tweak the

Re: ignore_system_indexes affects DROP SCHEMA ... CASCADE reported number of objects dropped

2018-05-03 Thread Peter Geoghegan
On Thu, May 3, 2018 at 4:14 PM, Peter Geoghegan <p...@bowt.ie> wrote: > On Thu, May 3, 2018 at 3:18 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: >> Indeed, that seems weird. Maybe tweak the test scripts so you can see >> all the objects cascaded to, and then find out wh

Re: [HACKERS] Clock with Adaptive Replacement

2018-05-03 Thread Peter Geoghegan
esn't justify a > separate usage count bump each time. I don't have time to check this out just now, but it seems like an excellent idea. It would be nice if it could be enhanced further, so you get some idea of what the blocks are without having to decode them yourself using tools like pageinspect. -- Peter Geoghegan

Re: ignore_system_indexes affects DROP SCHEMA ... CASCADE reported number of objects dropped

2018-05-03 Thread Peter Geoghegan
oth > will be reported as deletion targets. And it's not surprising that > disabling indexscans on pg_depend changes the visitation order. I also noticed that there are cases where we see less helpful (though still technically correct) HINT messages about which other object the user may prefer to drop. -- Peter Geoghegan

Re: [HACKERS] Clock with Adaptive Replacement

2018-04-28 Thread Peter Geoghegan
e of a problem with miscounting the number of hits based on accidents around how things are structured in the executor -- we need to be more careful about recognizing whether or not block hits are logically distinct. Particularly with stuff like nested loop joins. -- Peter Geoghegan

Re: ignore_system_indexes affects DROP SCHEMA ... CASCADE reported number of objects dropped

2018-05-03 Thread Peter Geoghegan
er on a view. I'm too tired to figure out whether or not this is truly cause for concern right now, though -- Peter Geoghegan

Re: Faster inserts with mostly-monotonically increasing values

2017-12-31 Thread Peter Geoghegan
ler, the costs associated with index descent becomes higher. FWIW, I think that int4 indexes have to be extremely large before they exceed 4 levels (where the leaf level counts as a level). My conservative back-of-an-envelope estimate is 9 billion index tuples. -- Peter Geoghegan

Re: Contributing with code

2017-12-31 Thread Peter Geoghegan
est and most important parts of successfully contributing to PostgreSQL. It's usually essential to understand in detail why the thing that you're thinking of working on doesn't already exist. The TODO list seems to suggest almost the opposite, and as such is a trap for inexperienced hackers. -- Peter Geoghegan

Re: TODO list (was Re: Contributing with code)

2017-12-31 Thread Peter Geoghegan
nto maintaining it to a high standard then it probably would have happened already. The fact that it hasn't happened tells us plenty. -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-10 Thread Peter Geoghegan
is a > join done one partition at a time, but a leader-wise logical tape set > is not done one leader at a time. If there's another meaning to the > affix -wise, I'm not familiar with it. Don't we just mean "a single > logical tapeset managed by the leader"? Yes, we do. Will change. > There's a lot here I haven't grokked yet, but I'm running out of > mental energy so I think I'll send this for now and work on this some > more when time permits, hopefully tomorrow. The good news is that the things that you took issue with were about what I expected you to take issue with. You seem to be getting through the review of this patch very efficiently. -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-10 Thread Peter Geoghegan
On Wed, Jan 10, 2018 at 1:31 PM, Robert Haas <robertmh...@gmail.com> wrote: >> Can we actually call it max_parallel_maintenance_workers instead? >> I mean we don't have work_mem_maintenance. > > Good point. WFM. -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-09 Thread Peter Geoghegan
is the main reason why external sorts can be faster than internal sorts -- this happens fairly frequently these days, especially with CREATE INDEX, where being able to write out the index as it merges on-the-fly helps a lot.) -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-10 Thread Peter Geoghegan
ot; comes from the idea of having a leader-wise space following concatenating worker tapes (who have original/worker-wise space). We must apply an offset to get from a worker-wise offset to a leader-wise offset. This made more sense in an earlier version. I overlooked this during recent self review. -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-10 Thread Peter Geoghegan
rial case too, but much less so. That's really not why he did it. > I assume the author credit will be "Peter > Geoghegan, Rushabh Lathia" in that order, but let me know if anyone > thinks that isn't the right idea. "Peter Geoghegan, Rushabh Lathia" seems right. Tho

Re: Unimpressed with pg_attribute_always_inline

2018-01-08 Thread Peter Geoghegan
her gdb > knows where all the variables are. In my experience, inlining hurts > both of those things, which is why I'm saying that forcing inlining > even in non-optimized builds is a bad idea. Isn't that an argument against inlining in general, rather than forcing inlining in particular? -- Peter Geoghegan

Re: [HACKERS] GUC for cleanup indexes threshold.

2018-01-06 Thread Peter Geoghegan
an for waiting to hear other people's views before proceeding. It really needs to be properly discussed. -- Peter Geoghegan

Re: [HACKERS] GUC for cleanup indexes threshold.

2018-01-06 Thread Peter Geoghegan
the most convenient way of using L's technique to defer recycling until it is definitely safe. We only need to make sure that _bt_page_recyclable() cannot become confused by XID wraparound to fix this problem -- that's it. -- Peter Geoghegan

Re: let's not complain about harmless patch-apply failures

2018-01-16 Thread Peter Geoghegan
generally not helpful unless used thoughtfully. -- Peter Geoghegan

Re: let's not complain about harmless patch-apply failures

2018-01-16 Thread Peter Geoghegan
ty about detecting problems mechanically. Rebasing a patch without conflicts (including seeing a warning about offsets) does not mean that your patch didn't become broken in some subtle, harmful way. Mechanical detection is only useful to the extent that it guides and augments human oversight. -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-17 Thread Peter Geoghegan
atch, and check its progress, to make sure that what I add is comparable to what ultimately gets committed for parallel query. -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-18 Thread Peter Geoghegan
perhaps only secondarily about the question of ripping out parallel_leader_participation entirely. > Can you please elaborate what part of optimizer are you talking about > where without leader participation partial path will always lose to a > serial sequential scan path? See my rema

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-18 Thread Peter Geoghegan
e" comment within InitializeParallelDSM() should instead say something like "Serialize indexes-pending-reindex state". Other than that, looks good to me. It's a simple patch with a clear purpose. -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-18 Thread Peter Geoghegan
On Thu, Jan 18, 2018 at 10:27 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Jan 18, 2018 at 1:14 PM, Peter Geoghegan <p...@bowt.ie> wrote: >> That seems pretty far fetched. > > I don't think it is, and there are plenty of other examples. All you > need

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-18 Thread Peter Geoghegan
make parallel_leader_participation=off allow a degenerate parallel CREATE INDEX in the next version. I think that it will make parallel_leader_participation less useful, with no upside, but there doesn't seem to be much more that I can do about that. -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-15 Thread Peter Geoghegan
On Sun, Jan 14, 2018 at 8:25 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Sun, Jan 14, 2018 at 1:43 AM, Peter Geoghegan <p...@bowt.ie> wrote: >> On Sat, Jan 13, 2018 at 4:32 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> Yeah, but this would mean

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-20 Thread Peter Geoghegan
t there is this issue with nworkers_launched until very recently. But then, that field has absolutely no comments. -- Peter Geoghegan

Re: [HACKERS] WIP: Covering + unique indexes.

2018-01-20 Thread Peter Geoghegan
ry and on slave. > Also, I've checked bt_index_parent_check() on master. I fixed that recently. It should be fine now. -- Peter Geoghegan

  1   2   3   4   5   6   >