Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-11-03 Thread Tomas Vondra
Hi, On 11/02/2017 06:45 PM, Alvaro Herrera wrote: > Tomas Vondra wrote: > >> Unfortunately, I think we still have a problem ... I've been wondering >> if we end up producing correct indexes, so I've done a simple test. > > Here's a proposed patch that should fix this

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-31 Thread Tomas Vondra
On 10/31/2017 11:44 PM, Tomas Vondra wrote: > ... > Unfortunately, I think we still have a problem ... I've been wondering > if we end up producing correct indexes, so I've done a simple test. > > 1) create the table as before > > 2) let the insert + vacuum run

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-31 Thread Tomas Vondra
t where a = 0; count --- 9062 (1 row) test=# set enable_bitmapscan = off; SET test=# select count(*) from brin_test where a = 0; count --- 9175 (1 row) Attached is a SQL script with commands I used. You'll need to copy the commands into multiple

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-31 Thread Tomas Vondra
(which didn't occur to me before as a possible issue at all, so that's for pointing that out). But that's a significantly more limited issue to fix than all the parallel-unsafe bits. Now, I agree this is somewhat more limited than I hoped for, but OTOH it still solves the issue I initially aimed

Re: [HACKERS] Re: Anyone have experience benchmarking very high effective_io_concurrency on NVME's?

2017-10-31 Thread Tomas Vondra
me much more modest - not because the device could not handle more, but because of the prefetch/processing ratio reached the optimal value. But all this is actually per-process, if you can run multiple backends (particularly when doing bitmap index scans), I'm sure you'll see the queues being more f

Re: [HACKERS] WIP: BRIN multi-range indexes

2017-10-30 Thread Tomas Vondra
resql.org/message-id/5d78b774-7e9c-c94e-12cf-fef51cc89b1a%402ndquadrant.com -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 0001-Pass-all-keys-to-BRIN-consistent-function-at-once.patch.gz Description: appl

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-30 Thread Tomas Vondra
roblems? Will look. > Not sure, but I can confirm Michael's findings - I've been unable to reproduce the issue on 1a4be103a5 even after 20 minutes, and on 24992c6db9 it failed after only 2. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Supp

[HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-29 Thread Tomas Vondra
FWIW I can reproduce this on REL_10_STABLE, but not on REL9_6_STABLE. So it seems to be due to something that changed in the last release. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent

[HACKERS] PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-29 Thread Tomas Vondra
only the value (offset) changes. The stack trace always looks exactly the same - see the attachment. At first it seemed the idxrel is always the index on 'e' (i.e. the UUID column), but it seems I also got failures on the other indexes. regards -- Tomas Vondra http://www.2ndQuadrant

Re: [HACKERS] WIP: BRIN bloom indexes

2017-10-27 Thread Tomas Vondra
hi, On 10/28/2017 02:41 AM, Nico Williams wrote: > On Fri, Oct 27, 2017 at 10:06:58PM +0200, Tomas Vondra wrote: >>> + * We use an optimisation that initially we store the uint32 values >>> directly, >>> + * without the extra hashing step. And only later fillin

Re: [HACKERS] WIP: BRIN bloom indexes

2017-10-27 Thread Tomas Vondra
> > Modulo of one 64bit result by two coprime numbers (`nbits` and `nbits-1`) > gives two good-quality functions usable for double hashing. > OK, thanks for the suggestion. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Tra

Re: [HACKERS] WIP: BRIN bloom indexes

2017-10-27 Thread Tomas Vondra
Hi, On 10/27/2017 07:17 PM, Nico Williams wrote: > On Thu, Oct 19, 2017 at 10:15:32PM +0200, Tomas Vondra wrote: > > A bloom filter index would, indeed, be wonderful. > > Comments: > > + * We use an optimisation that initially we store the uint32 values directly, &g

Re: [HACKERS] WIP: BRIN bloom indexes

2017-10-27 Thread Tomas Vondra
hi, On 10/27/2017 09:34 AM, Simon Riggs wrote: > On 27 October 2017 at 07:20, Robert Haas <robertmh...@gmail.com> wrote: >> On Thu, Oct 19, 2017 at 10:15 PM, Tomas Vondra >> <tomas.von...@2ndquadrant.com> wrote: >>> Let's see a query like this: >>> &g

Re: [HACKERS] Burst in WAL size when UUID is used as PK while full_page_writes are enabled

2017-10-27 Thread Tomas Vondra
ill be created but merely that the inserts will touch a different (possibly existing) leaf page. That's a direct consequence of the inherent UUID randomness. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Serv

Re: [HACKERS] WIP: BRIN multi-range indexes

2017-10-25 Thread Tomas Vondra
Apparently I've managed to botch the git format-patch thing :-( Attached are both patches (the first one adding BRIN bloom indexes, the other one adding the BRIN multi-range). Hopefully I got it right this time ;-) regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL

[HACKERS] WIP: BRIN multi-range indexes

2017-10-25 Thread Tomas Vondra
nly works with float8 (and also timestamp data types) for now, but it should be straightforward to add support for additional data types. Most of that will be about adding catalog definitions anyway. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 S

[HACKERS] CUBE seems a bit confused about ORDER BY

2017-10-19 Thread Tomas Vondra
| (11399, 377),(11360, 294) (15 rows) Seems like a bug somewhere in gist_cube_ops, I guess? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgres

[HACKERS] WIP: BRIN bloom indexes

2017-10-19 Thread Tomas Vondra
; indexes are more vulnerable to "wide ranges". But perhaps this is a non-issue, as the bloom index are meant for cases when minmax indexes don't work. And when minmax indexes work, they work better than bloom. So people are unlikely to build both of them at the same time. regards -- T

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-17 Thread Tomas Vondra
On 10/17/2017 03:16 PM, Robert Haas wrote: > On Sat, Oct 14, 2017 at 2:14 PM, Tomas Vondra > <tomas.von...@2ndquadrant.com> wrote: >> I propose is to add a new cursor option (PARALLEL), which would allow >> parallel plans for that particular user-defined cursor. Attach

Re: [HACKERS] SIGSEGV in BRIN autosummarize

2017-10-17 Thread Tomas Vondra
e compilation options you can run pg_config, look for CONFIGURE line and then just modify add --prefix option. And after `make install` you can add it to $PATH and start the server using those binaries. $ export PATH=/path/to/alternative/binaries/bin:$PATH $ which pg_ctl $ pg_ctl -D $DATADIR stop $ pg_

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-16 Thread Tomas Vondra
On 10/16/2017 01:13 PM, Amit Kapila wrote: > On Sat, Oct 14, 2017 at 11:44 PM, Tomas Vondra > <tomas.von...@2ndquadrant.com> wrote: >> Hi, >> >> >> I propose is to add a new cursor option (PARALLEL), which would allow >> parallel plans for tha

Re: [HACKERS] SIGSEGV in BRIN autosummarize

2017-10-15 Thread Tomas Vondra
uum-brin-summarize.gz > Thanks, but I'm not sure that'll help, at this point. We already know what happened (corrupted memory), we don't know "how". And core files are mostly just "snapshots" so are not very useful in answering that :-( regards -- Tomas Vondra

Re: [HACKERS] SIGSEGV in BRIN autosummarize

2017-10-14 Thread Tomas Vondra
rtypmod -1 :varcollid 0 :varlevelsup 0 :varnoold 1 :varoattno 1 :location > 146} {CONST :consttype 1184 :consttypmod -1 :constcollid 0 :constlen 8 > :constbyv"... > cur_relname = 0x298cd68 > "cdrs_eric_msc_sms_2017_10_14_startofcharge_idx" > __func__ =

Re: [HACKERS] Extended statistics is not working on Vars hidden under a RelabelType

2017-10-14 Thread Tomas Vondra
t on a test that's rather extreme (statistics on empty table). So again, likely well within noise, and on larger tables it'll get even less significant. But of course - it's not free. It's a bit more work we need to do. But if you don't need multi-column statistics, don't create them. If your queries are al

[HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-14 Thread Tomas Vondra
tXSuiyXBM3y%3DA%3DeJzmg%40mail.gmail.com -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c index 76d6cf1..ffaa096 100644 --- a

Re: [HACKERS] Extended statistics is not working on Vars hidden under a RelabelType

2017-10-13 Thread Tomas Vondra
consider the plans were likely generated on a assert-enabled build and on a laptop (both of which adds quite a bit of noise to occasional timings). The patch has no impact on planning time (at least I've been unable to measure any). regards -- Tomas Vondra http://www.2ndQuadr

Re: [HACKERS] Parallel Bitmap Heap Scans segfaults due to (tbm->dsa==NULL) on PostgreSQL 10

2017-10-12 Thread Tomas Vondra
On 10/12/2017 02:40 PM, Dilip Kumar wrote: > On Thu, Oct 12, 2017 at 4:31 PM, Tomas Vondra > <tomas.von...@2ndquadrant.com> wrote: >> Hi, >> >> It seems that Q19 from TPC-H is consistently failing with segfaults due >> to calling tbm_prepare_shared_iterat

[HACKERS] Parallel Bitmap Heap Scans segfaults due to (tbm->dsa==NULL) on PostgreSQL 10

2017-10-12 Thread Tomas Vondra
* query.sql * plan.txt * backtrace.txt 2) details for the "minimal" query triggering the issue: * query-minimal.sql * plan-minimal.txt * backtrace-minimal.txt regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Trai

[HACKERS] Re: Extended statistics is not working on Vars hidden under a RelabelType

2017-10-10 Thread Tomas Vondra
ther node. > Thanks for noticing this. The patch seems fine to me. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] PATCH : Generational memory allocator (was PATCH: two slab-like memory allocators)

2017-09-24 Thread Tomas Vondra
added a brief note into reorderbuffer.c mentioning that it uses SlabContext and GenerationContext. As I explained, I don't think we should include details about how we tested the patch or whatever here. regard -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7

Re: [HACKERS] VACUUM and ANALYZE disagreeing on what reltuples means

2017-09-24 Thread Tomas Vondra
relying on autovacuum that won't happen, as it only runs on tables with certain number of dead tuples. So you'd have to be running VACUUM in a loop or something (but not VACUUM ANALYZE, because that works fine) from a script, or something like that. That being said, fixing a bug is always a g

Re: [HACKERS] VACUUM and ANALYZE disagreeing on what reltuples means

2017-09-24 Thread Tomas Vondra
On 09/06/2017 09:45 AM, Haribabu Kommi wrote: > > > On Tue, Jul 25, 2017 at 9:33 PM, Tomas Vondra > <tomas.von...@2ndquadrant.com <mailto:tomas.von...@2ndquadrant.com>> wrote: > > On 7/25/17 12:55 AM, Tom Lane wrote: > > Tomas

Re: [HACKERS] user-defined numeric data types triggering ERROR: unsupported type

2017-09-23 Thread Tomas Vondra
On 09/21/2017 04:24 PM, Tom Lane wrote: > Tomas Vondra <tomas.von...@2ndquadrant.com> writes: >> [ scalarineqsel may fall over when used by extension operators ] > > I concur with your thought that we could have > ineq_histogram_selectivity fall back to a "mid buck

Re: [HACKERS] Boom filters for hash joins (was: A design for amcheck heapam verification)

2017-09-19 Thread Tomas Vondra
On 09/19/2017 06:03 PM, Peter Geoghegan wrote: > On Tue, Sep 19, 2017 at 6:28 AM, Tomas Vondra > <tomas.von...@2ndquadrant.com> wrote: >> The patch is fairly simple, and did not try to push the bloom filters to >> scan nodes or anything like that. It might be a meaningf

Re: [HACKERS] Boom filters for hash joins (was: A design for amcheck heapam verification)

2017-09-19 Thread Tomas Vondra
speculative. I suspect that this could be >> true, and it seems worth investigating that framing of the problem >> first. > > ISTR Tomas Vondra doing some experiments with this a few years ago and > finding that it was, in fact, a problem. > You seem to have better memory t

[HACKERS] user-defined numeric data types triggering ERROR: unsupported type

2017-09-19 Thread Tomas Vondra
t is, we get 50% estimate, because that's what scalarineqsel uses when it ineq_histogram_selectivity can't compute selectivity from a histogram for some reason. Wouldn't it make it more sense to use the default 33% estimate here? regards -- Tomas Vondra http://www.2ndQuadrant

[HACKERS] valgrind vs. shared typmod registry

2017-09-16 Thread Tomas Vondra
ing tests under valgrind not too long ago and I don't recall such failures, so perhaps something broke it in the past few days. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services ==18897== Conditional jump

Re: [HACKERS] PATCH : Generational memory allocator (was PATCH: two slab-like memory allocators)

2017-09-14 Thread Tomas Vondra
On 09/14/2017 04:21 PM, Simon Riggs wrote: > On 14 August 2017 at 01:35, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: >> Hi, >> >> Attached is a rebased version of the Generational context, originally >> submitted with SlabContext (which was already comm

Re: [HACKERS] Hooks to track changed pages for backup purposes

2017-09-13 Thread Tomas Vondra
; > Well, in some not so rare cases users encrypt backups and send to S3. > And there is no system with CPUs that can handle that WAL parsing. > Currently, I'm considering mocking prototype for wal-g, which works > exactly this. > Why couldn't there be a system with enough CPU pow

Re: [HACKERS] Patches that don't apply or don't compile: 2017-09-12

2017-09-13 Thread Tomas Vondra
ree, that was clearly a mistake, I had to add Daniel to CC. Sorry I > didn't do that. I've returned all affected patches back to "Needs > Review". On the bright side while doing this I've noticed that many > patches were already updated by their authors. > Yeah. regards

Re: [HACKERS] Patches that don't apply or don't compile: 2017-09-12

2017-09-12 Thread Tomas Vondra
whether the failures are legitimate or not, and eliminate as many false positives as possible. Once we are happy with the accuracy, we can enable it again. kind regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2017-09-12 Thread Tomas Vondra
Attached is an updated version of the patch, dealing with fallout of 821fb8cdbf700a8aadbe12d5b46ca4e61be5a8a8 which touched the SGML documentation for CREATE STATISTICS. regards On 09/07/2017 10:07 PM, Tomas Vondra wrote: > Hi, > > Attached is an updated version of the patch, fixing t

Re: [HACKERS] The case for removing replacement selection sort

2017-09-11 Thread Tomas Vondra
s say. So I think you're right it to 1B rows may break this assumption, and make it perform worse. But perhaps the fact that we're testing with multiple work_mem values, and with smaller data sets (100k or 1M rows) makes this a non-issue? regards -- Tomas Vondra http://www.2ndQuadran

Re: [HACKERS] Automatic testing of patches in commit fest

2017-09-11 Thread Tomas Vondra
after each new submission, disrupting the CF process. Making the testing reliable may even require establishing some sort of policy (say, always send a complete patch, not just one piece). > > I hope this observation will change your mind :) > Not sure. But I'm mostly just a passenger here,

Re: [HACKERS] Remove 1MB size limit in tsvector

2017-09-11 Thread Tomas Vondra
d not really matter on indexed tsvectors. So the question is how realistic that benchmark actually is. How likely are we to do queries on fts directly, not through a GIN/GiST index? Particularly in performance-sensitive cases? regards -- Tomas Vondra http://www.2ndQuadrant.com P

Re: [HACKERS] Automatic testing of patches in commit fest

2017-09-11 Thread Tomas Vondra
, as (a) there's nothing wrong with the patch, and (b) it's not clear what to do to fix the problem. So -1 to this for now, until we make this part smart enough. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Service

Re: [HACKERS] Polyphase merge is obsolete

2017-09-11 Thread Tomas Vondra
Hi, I planned to do some benchmarking on this patch, but apparently the patch no longer applies. Rebase please? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing

Re: [HACKERS] The case for removing replacement selection sort

2017-09-10 Thread Tomas Vondra
On 09/11/2017 02:22 AM, Peter Geoghegan wrote: > On Sun, Sep 10, 2017 at 5:07 PM, Tomas Vondra > <tomas.von...@2ndquadrant.com> wrote: >> I'm currently re-running the benchmarks we did in 2016 for 9.6, but >> those are all sorts with a single column (see the attached scri

Re: [HACKERS] The case for removing replacement selection sort

2017-09-10 Thread Tomas Vondra
ly testing the largest/smallest work_mem values? So that we could get some numbers now, possibly running the complete test suite later. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services sort-bench.sh Descript

Re: [HACKERS] WIP: Separate log file for extension

2017-09-09 Thread Tomas Vondra
L is started without the collector, and an extension attempts to use LOG_DESTINATION_CSVLOG? Or will it start a separate collector for each such extension? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training &

Re: [HACKERS] segment size depending *_wal_size defaults (was increasing the default WAL segment size)

2017-09-09 Thread Tomas Vondra
it'd not be hard to GUC support. > It's not hard - it's just a lot of copy-pasting of infrastructure code. Incidentally, I already have a patch doing that, as we had to add that support to XL, and I can submit it to PostgreSQL. Might save some boring coding. regards -- Tomas Vondra

Re: [HACKERS] psql: new help related to variables are not too readable

2017-09-09 Thread Tomas Vondra
On 09/09/2017 01:24 AM, Tom Lane wrote: > Tomas Vondra <tomas.von...@2ndquadrant.com> writes: >> The translator has exactly the same context in both cases, and the >> struggle to wrap it at 80 characters will be pretty much the same too. > > Really? With the old way,

Re: [HACKERS] psql: new help related to variables are not too readable

2017-09-08 Thread Tomas Vondra
a matter of > taste and I expect there should be half a dozen different opinions on > the matter of formatting. > FWIW, +1 to extra lines from me - I find it way more readable, as it clearly separates the items. You're right this is also a matter of taste to some degree, so I understand Eri

Re: [HACKERS] [PATCH] Pageinspect - add functions on GIN and GiST indexes from gevel

2017-09-07 Thread Tomas Vondra
e index, not individual pages. Why should it be part of pageinspect? For example we have pgstattuple extension, which seems like a better match. Or just create a new extension - if the code is valuable, surely we can live one more extension instead of smuggling it in inside pageinspect. regards -- To

Re: [HACKERS] The case for removing replacement selection sort

2017-09-07 Thread Tomas Vondra
t replacement selection actually did do > better with on 9.6. I clearly remember Tomas Vondra doing lots of > benchmarking, showing some benefit with RS with a work_mem of 8MB or > less. As I said in my introduction on this thread, we weren't wrong to > add replacement_sort_tuples to 9.6, give

Re: [HACKERS] GnuTLS support

2017-09-07 Thread Tomas Vondra
options, just like you can't set ssl=on when you build without OpenSSL support. But perhaps we could simply not include the inactive options into the config file, no? I don't see how breaking the TLS config into separate files improves the situation - instead of dealing with GUCs in a single file

Re: [HACKERS] Remove 1MB size limit in tsvector

2017-09-07 Thread Tomas Vondra
but also page cache). Can you share the benchmarks, so that others can retry running them? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@pos

Re: [HACKERS] Hooks to track changed pages for backup purposes

2017-09-07 Thread Tomas Vondra
re-read the WAL segments from disk (which may be a lot of I/O, particularly when done from multiple processes). >From this POV, the idea to collect this information on the backup system (WAL archive) by pre-processing the arriving WAL segments seems like the most promising. It moves the work to anot

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2017-09-07 Thread Tomas Vondra
Hi, Attached is an updated version of the patch, fixing the issues reported by Adrien Nayrat, and also a bunch of issues pointed out by valgrind. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 0001

Re: [HACKERS] error-handling in ReorderBufferCommit() seems somewhat broken

2017-08-29 Thread Tomas Vondra
On 08/30/2017 02:19 AM, Andres Freund wrote: > On 2017-08-30 02:11:10 +0200, Tomas Vondra wrote: >> >> I'm not really following your reasoning. You may very well be right that >> the BeginInternalSubTransaction() example is not quite valid on postgres >> core, but I

Re: [HACKERS] error-handling in ReorderBufferCommit() seems somewhat broken

2017-08-29 Thread Tomas Vondra
On 08/30/2017 01:34 AM, Andres Freund wrote: > Hi, > > On 2017-08-30 01:27:34 +0200, Tomas Vondra wrote: >> I've been investigating some failures in test_decoding regression tests, >> and it seems to me the error-handling in ReorderBufferCommit() is >> somewhat

[HACKERS] error-handling in ReorderBufferCommit() seems somewhat broken

2017-08-29 Thread Tomas Vondra
oes not support subtransactions and so the BeginInternalSubTransaction() call in the try branch always fails, triggering the issue. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2017-08-26 Thread Tomas Vondra
ashes. > Thanks for the report, this is clearly a bug. I don't think we need to test the stxkind, but rather a missing check that the requested type is actually built. > Unfotunately, I don't have the knowledge to produce a patch :/ > > Small fix in documentation, patch attached.

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

2017-08-15 Thread Tomas Vondra
On 08/15/2017 09:55 PM, Robert Haas wrote: On Tue, Aug 15, 2017 at 3:42 PM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: What I think we should not do is interpret the bitmasks (omitting some of the information) assuming all the bits were set correctly. I'm still co

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

2017-08-15 Thread Tomas Vondra
On 08/15/2017 07:54 PM, Robert Haas wrote: On Tue, Aug 15, 2017 at 9:59 AM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: I don't think so -- the "committed" and "invalid" meanings are effectively canceled when the "frozen" mask is present. I mean,

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

2017-08-15 Thread Tomas Vondra
ght (not) have happened. Or at least make the filtering optional. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] PATCH : Generational memory allocator (was PATCH: two slab-like memory allocators)

2017-08-13 Thread Tomas Vondra
ideas of other useful test cases are welcome. regards [1] https://www.postgresql.org/message-id/20170227111732.vrx5v72ighehwpkf%40alap3.anarazel.de [2] https://www.postgresql.org/message-id/20160706185502.1426.28143%40wrigleys.postgresql.org -- Tomas Vondra http://www

Re: [HACKERS] VACUUM and ANALYZE disagreeing on what reltuples means

2017-07-25 Thread Tomas Vondra
On 7/25/17 5:04 PM, Tom Lane wrote: Tomas Vondra <tomas.von...@2ndquadrant.com> writes: On 7/25/17 12:55 AM, Tom Lane wrote: I think the planner basically assumes that reltuples is the live tuple count, so maybe we'd better change VACUUM to get in step. Attached is a patch that (I

Re: [HACKERS] VACUUM and ANALYZE disagreeing on what reltuples means

2017-07-25 Thread Tomas Vondra
On 7/25/17 12:55 AM, Tom Lane wrote: Tomas Vondra <tomas.von...@2ndquadrant.com> writes: It seems to me that VACUUM and ANALYZE somewhat disagree on what exactly reltuples means. VACUUM seems to be thinking that reltuples = live + dead while ANALYZE apparently believes that reltuples

[HACKERS] VACUUM and ANALYZE disagreeing on what reltuples means

2017-07-24 Thread Tomas Vondra
both on 9.6 and 10. I haven't checked older versions, but I guess those are affected too. The question is - which of the reltuples definitions is the right one? I've always assumed that "reltuples = live + dead" but perhaps not? regards -- Tomas Vondra http://www.2nd

Re: [HACKERS] Pluggable storage

2017-06-26 Thread Tomas Vondra
nty of references in the MySQL "Restrictions and Limitations" section of the manual: https://downloads.mysql.com/docs/mysql-reslimits-excerpt-5.7-en.pdf regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Traini

Re: [HACKERS] Pluggable storage

2017-06-23 Thread Tomas Vondra
s pattern and things like prefetch would not be very efficient, I think. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make ch

Re: [HACKERS] Pluggable storage

2017-06-22 Thread Tomas Vondra
to get API that supports all kinds of pluggable storage implementations. But I guess that'll take some time. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Pluggable storage

2017-06-22 Thread Tomas Vondra
d < (1 << MaxHeapTuplesPerPageBits)) But perhaps there are some other issues? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To m

Re: [HACKERS] Parallel Aggregation support for aggregate functions that use transitions not implemented for array_agg

2017-06-18 Thread Tomas Vondra
as part of the "serialize" function, and benefit from that while combining that in the gather, then sure, that should be a huge win. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via

Re: [HACKERS] WIP: Data at rest encryption

2017-06-18 Thread Tomas Vondra
won't work for you. But then there's eCryptfs, for example: https://en.wikipedia.org/wiki/ECryptfs regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] TPC-H Q20 from 1 hour to 19 hours!

2017-06-11 Thread Tomas Vondra
chives. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mail

Re: [HACKERS] TPC-H Q20 from 1 hour to 19 hours!

2017-05-25 Thread Tomas Vondra
Hi, On 5/25/17 6:03 AM, Robert Haas wrote: On Thu, Apr 6, 2017 at 4:37 PM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: Which brings me to the slightly suspicious bit. On 9.5, there's no difference between GROUP and GROUP+LIKE cases - the estimates are exactly the same in both

Re: [HACKERS] WITH clause in CREATE STATISTICS

2017-05-12 Thread Tomas Vondra
hose did not make it into pg10. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] CTE inlining

2017-05-04 Thread Tomas Vondra
as if executed exactly once". I may be misunderstanding what other people proposed in this thread, but I think the plan was to only inline CTEs where we know it won't change the results, etc. So e.g. CTEs with volatile functions would not get inlined, which includes nextval() for example.

Re: [HACKERS] CTE inlining

2017-05-04 Thread Tomas Vondra
function, and in my understanding the plan was not to inline CTEs with volatile functions (or CTEs doing writes etc.). That is, we'd guarantee the same results as we get now. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA

Re: [HACKERS] WITH clause in CREATE STATISTICS

2017-05-03 Thread Tomas Vondra
clause? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/p

Re: [HACKERS] CTE inlining

2017-05-03 Thread Tomas Vondra
y is happy either. +1 for option 1. And while I would not like if we had to combine it with a backwards compatibility GUC which enables the old behavior to get it merged I still personally would prefer that over option 2 and 3. > Andreas > +1 to what Andreas says -- Tomas Vondra

Re: [HACKERS] CTE inlining

2017-05-02 Thread Tomas Vondra
On 5/2/17 11:23 PM, Merlin Moncure wrote: \On Tue, May 2, 2017 at 12:05 PM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: On 5/2/17 6:34 PM, David Fetter wrote: On Tue, May 02, 2017 at 02:40:55PM +0200, Andreas Karlsson wrote: On 05/02/2017 04:38 AM, Craig Ringer wrote: On

Re: [HACKERS] CTE inlining

2017-05-02 Thread Tomas Vondra
sure this is not a battle you'd like to fight. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] CTE inlining

2017-05-02 Thread Tomas Vondra
ealize OFFSET 0 is no-op? In that case replacing CTE optimization fence with "OFFSET 0" would be akin to painting yourself into a corner, waiting for the pain to dry, walking over to another corner and painting yourself into that one. cheers -- Tomas Vondra http://www

Re: [HACKERS] CTE inlining

2017-05-01 Thread Tomas Vondra
e multiple CTEs in the query with different fencing behavior, and it would be just as difficult to investigate. So no more planner-affecting GUCs, please, particularly if we expect regular users to use them. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Developmen

Re: [HACKERS] CTE inlining

2017-04-30 Thread Tomas Vondra
Say, being able to define indexes on them, but that's really a separate topic. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) T

Re: [HACKERS] CTE inlining

2017-04-30 Thread Tomas Vondra
le attempts at this in the past, none of them succeeded. But perhaps we could at least propagate some of the CTE features, so that the outside query can benefit from that (e.g. when the CTE is sorted, we could set the sortkeys). That wouldn't break the fence thing, but it would

Re: [HACKERS] to-do item for explain analyze of hash aggregates?

2017-04-24 Thread Tomas Vondra
On 04/24/2017 10:55 PM, Jeff Janes wrote: On Mon, Apr 24, 2017 at 12:13 PM, Tomas Vondra <tomas.von...@2ndquadrant.com <mailto:tomas.von...@2ndquadrant.com>> wrote: On 04/24/2017 08:52 PM, Andres Freund wrote: ... I've wanted that too. It's not impossible at al

Re: [HACKERS] to-do item for explain analyze of hash aggregates?

2017-04-24 Thread Tomas Vondra
n't that be possible? We probably can't use exactly the same approach as Hash, because hashjoins use custom hash table while hashagg uses dynahash IIRC. But why couldn't measure the amount of memory by looking at the memory context, for example? regards -- Tomas Vondra http://

Re: Do we need multi-column frequency/histogram stats? WAS Re: [HACKERS] Statistics "dependency"

2017-04-23 Thread Tomas Vondra
he other types of statistics actually track correlation between values in the columns, not just "column A implies column B". regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hack

Re: [HACKERS] Statistics "dependency"

2017-04-23 Thread Tomas Vondra
sh. There are multiple statistics for dependency stored, hence "dependencies". I don't like it, but its the best term I can see at present. That is a good point, actually. It should probably be plural. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreS

Re: [HACKERS] WITH clause in CREATE STATISTICS

2017-04-20 Thread Tomas Vondra
index on t (a) where a < 100 with (fillfactor=10); ERROR: syntax error at or near "with" LINE 1: create index on t (a) where a < 100 with (fillfactor=10); ^ test=# create index on t (a) with (fillfactor=10) where a < 100;

Re: [HACKERS] pg_statistic_ext.staenabled might not be the best column name

2017-04-12 Thread Tomas Vondra
On 04/12/2017 03:36 PM, David Rowley wrote: "stakind" seems like a better name. I'd have personally gone with "statype" but pg_statistic already thinks stakind is better. +1 to stakind -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development,

Re: [HACKERS] pg_stats_ext view does not seem all that useful

2017-04-10 Thread Tomas Vondra
), because pg_statistic_ext would simply return the whole bytea value. But since then the view kinda lost the purpose and no one really noticed. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-04-10 Thread Tomas Vondra
ally not adding a new GUC parameter, you're adding a constant which is then used as a max value for max for the two existing parallel GUCs. I think this is fine. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training &am

Re: [HACKERS] Performance issue with postgres9.6

2017-04-07 Thread Tomas Vondra
ollect profiles from that one backend using 'perf'. I assume you're using the same hardware / machine for the tests? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing

Re: [HACKERS] increasing the default WAL segment size

2017-04-06 Thread Tomas Vondra
On 04/06/2017 11:45 PM, David Steele wrote: On 4/6/17 5:05 PM, Tomas Vondra wrote: On 04/06/2017 08:33 PM, David Steele wrote: On 4/5/17 7:29 AM, Simon Riggs wrote: 2. It's not clear to me the advantage of being able to pick varying filesizes. I see great disadvantage in having too many

Re: [HACKERS] increasing the default WAL segment size

2017-04-06 Thread Tomas Vondra
e actually have any infrastructure for that? Or do you plan to add some new animals with different WAL segment sizes? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing

  1   2   3   4   5   6   7   8   9   10   >