Re: scan-build plpython stuff

2017-11-24 Thread John Naylor
Peter, I built plpython with scan-build using Python 2.7.12 and Clang 3.8. On master, I got 13 warnings, and with your patches only one warning (report attached). Make installcheck passes. Let me know if I can test anything else. -John Naylor Title: plpy_spi.c Bug

Re: scan-build plpython stuff

2017-11-28 Thread John Naylor
On 11/28/17, Peter Eisentraut <peter.eisentr...@2ndquadrant.com> wrote: > On 11/24/17 08:40, John Naylor wrote: >> I built plpython with scan-build using Python 2.7.12 and Clang 3.8. On >> master, I got 13 warnings, and with your patches only one warning >> (r

Re: pearltidy source code has been removed (pgindent)

2017-12-14 Thread John Naylor
On 12/15/17, Jordan Deitch wrote: > Hi hackers, > > I am unable to build pgindent as it appears the pearltidy source has been > removed from sourceforge: I found it here: https://cpan.metacpan.org/authors/id/S/SH/SHANCOCK/Perl-Tidy-20090616.tar.gz

Re: WIP: a way forward on bootstrap data

2018-05-06 Thread John Naylor
might be) in the relevant header file. You'd need some improvement in readability to justify that additional code, though. I don't think this example quite passes (it's pretty obvious locally what the letters refer to), but others may feel differently. -John Naylor

Re: unused_oids script is broken with bsd sed

2018-04-26 Thread John Naylor
On 4/26/18, Tom Lane <t...@sss.pgh.pa.us> wrote: > John Naylor <jcnay...@gmail.com> writes: >> For those following along, these scripts still assume we're in the >> catalog directory. I can hack on that part tomorrow if no one else >> has. > > I didn't touc

inconsistency and inefficiency in setup_conversion()

2018-04-28 Thread John Naylor
Taking a close look at the result of setup_conversion(), wrong or at least confusing comments are applied to the functions. Consider this family of conversions: select conproc, conname from pg_conversion where conproc = 'utf8_to_win'::regproc order by oid; conproc | conname

Re: WIP: a way forward on bootstrap data

2018-04-28 Thread John Naylor
imple substitution mechanism, but I find the above a bit harder to reason about. It also has the added disadvantage that there is whitespace in the BKI annotation, so it can no longer be parsed with the current setup. That could be overcome with additional complexity, of course, but then

Re: unused_oids script is broken with bsd sed

2018-05-04 Thread John Naylor
to do so, but it got stuck on the back burner. I'm happy to report that, running non-parallel make on my old laptop goes from over 5 minutes to about 5 seconds, when touching a catalog file. -John Naylor

Re: master fails to build on Windows

2018-05-18 Thread John Naylor
oes the attached patch fix the issue? (This may not be the right permanent fix, but hopefully it will confirm my suspicion) -John Naylor diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm index 4ad1f8f..a33e68e 100644 --- a/src/tools/msvc/Solution.pm +++ b/src/tools/msvc/Solution.pm @@ -501,7

Re: inconsistency and inefficiency in setup_conversion()

2018-05-18 Thread John Naylor
esql.org/message-id/7408.1525812528%40sss.pgh.pa.us [2] https://www.postgresql.org/docs/devel/static/system-catalog-initial-data.html#SYSTEM-CATALOG-OID-REFERENCES -John Naylor From 686dd18ae238e1660ed3b2a7f6071cd5af20369d Mon Sep 17 00:00:00 2001 From: John Naylor <jcnay...@gmail.com> Dat

generating bootstrap entries for array types

2018-05-19 Thread John Naylor
-id/CAJVSVGW-D7OobzU%3DdybVT2JqZAx-4X1yvBJdavBmqQL05Q6CLw%40mail.gmail.com -John Naylor From 8c1b345b54bddd560df0e15b534c960b532cdade Mon Sep 17 00:00:00 2001 From: John Naylor <jcnay...@gmail.com> Date: Sat, 19 May 2018 15:10:51 +0700 Subject: [PATCH v1] Generate bootstrap entries for array types Ad

FindDefinedSymbol() is subtly broken

2018-05-19 Thread John Naylor
style -remove a duplicate comment -John Naylor From be3cbb1499e026355697b1aff890a3049cdef4ac Mon Sep 17 00:00:00 2001 From: John Naylor <jcnay...@gmail.com> Date: Sat, 19 May 2018 13:14:58 +0700 Subject: [PATCH] Update FindDefinedSymbol() to match current practice. Once upon a time, headers

Re: missing toast table for pg_policy

2018-06-15 Thread John Naylor
testing. On my machine, initdb takes at least: HEAD ~1040 ms patch ~1070 ms with my addenda ~1075 ms A little slower, but within the noise of variation. -John Naylor > Joe > > -- > Crunchy Data - http://crunchydata.com > PostgreSQL Support for Secure Enterprises > Consulting, Tr

Re: inconsistency and inefficiency in setup_conversion()

2018-07-02 Thread John Naylor
I've attached v4, which is a rebase plus some comment revisions. -John Naylor From 4d1cb1d40c5c79c732e5433e95f8560fb41e20bd Mon Sep 17 00:00:00 2001 From: John Naylor Date: Mon, 2 Jul 2018 12:52:07 +0700 Subject: [PATCH v4 1/2] Add pg_language lookup. This didn't seem worth doing before

Re: missing toast table for pg_policy

2018-06-19 Thread John Naylor
ternate implementation starts to make it ugly, so I haven't updated it for now. -John Naylor > Michael > diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h index ee54487..f259890 100644 --- a/src/include/catalog/toasting.h +++ b/src/include/catalog/toasting.h @

Re: inconsistency and inefficiency in setup_conversion()

2018-05-02 Thread John Naylor
On 4/28/18, Tom Lane <t...@sss.pgh.pa.us> wrote: > John Naylor <jcnay...@gmail.com> writes: >> Solution #1 - As alluded to in [1], turn the conversions into >> pg_proc.dat and pg_conversion.dat entries. Teach genbki.pl to parse >> pg_wchar.h to map conversion name

Re: MCV lists for highly skewed distributions

2017-12-31 Thread John Naylor
which showcases the opposite problem: For more uniform distributions, there are too many MCVs. Not relevant to your problem, but if I have time I'll try my hand at testing an approach suggested in that thread at the same time I test your patch and see how it interacts. -John Naylor

Re: MCV lists for highly skewed distributions

2017-12-29 Thread John Naylor
is also a good idea, but I haven't thought it through. If you don't go this route, I would move this section back out of the loop as well. -John Naylor

Re: WIP: a way forward on bootstrap data

2017-12-21 Thread John Naylor
so, no need to skip non-data values explicitly. The code knows where to find the schema. :-) Thanks for pushing 1 and 2, BTW. -John Naylor

Re: MCV lists for highly skewed distributions

2018-01-07 Thread John Naylor
a few times and eyeballed the results, of which I give a typical example. I used a low stat target to make it easier to see the general picture. Suggestions welcome. -John Naylor test_analyze_highly_skewed_v1.sql Description: application/sql

Re: WIP: a way forward on bootstrap data

2018-01-15 Thread John Naylor
utils/fmgroids.h into a static file that just #includes catalog/pg_proc_d.h. Thoughts? And I'm curious, what is "_d" intended to convey? (While I'm thinking outloud, I'm beginning to think that these headers lie outside the scope of genbki.pl, and belong in a separate script.) -John Naylor

Re: MCV lists for highly skewed distributions

2018-01-19 Thread John Naylor
ther for highly uniform or highly non-uniform distributions. [1] https://www.postgresql.org/message-id/flat/32261.1496611829%40sss.pgh.pa.us#32261.1496611...@sss.pgh.pa.us -John Naylor

Re: MCV lists for highly skewed distributions

2018-01-20 Thread John Naylor
ps://www.postgresql.org/message-id/flat/32261.1496611829%40sss.pgh.pa.us#32261.1496611...@sss.pgh.pa.us -John Naylor

stricter MCV tests for uniform distributions (was Re: MCV lists for highly skewed distributions)

2018-01-22 Thread John Naylor
(Starting a new thread so as not to distract review) On 1/21/18, Dean Rasheed <dean.a.rash...@gmail.com> wrote: > On 21 January 2018 at 07:26, John Naylor <jcnay...@gmail.com> wrote: >> I spent a few hours hacking on this, and it turns out calculating the >>

Re: WIP: a way forward on bootstrap data

2018-01-14 Thread John Naylor
eakage from this patch set; we don't need to take on rearrangements that > aren't buying anything. I don't have a convincing rebuttal, so I'll withdraw it. -- [1] https://www.postgresql.org/message-id/25254.1248533810%40sss.pgh.pa.us -John Naylor

Re: WIP: a way forward on bootstrap data

2018-01-12 Thread John Naylor
ependency tracking is certainly broken until such time as I can shave that yak and test. 2. Keeping the oid symbols with the data entries required some Makefile trickery to make them visible to .c files outside the backend (patch 0007). It builds fine, but the dependency tracking might have bugs. -

Re: WIP: a way forward on bootstrap data

2018-01-13 Thread John Naylor
typeoids{ $bki_values{prorettype}}; + $bki_values{prorettype} = $rettypeoid + if defined($rettypeoid); This is all in patch 0011. -John Naylor

Re: MCV lists for highly skewed distributions

2018-02-04 Thread John Naylor
ng, including of planner estimates on real data, is still needed of course, but I think this is definitely in the right direction. -John Naylor test_mcvstats_v1.sql Description: application/sql

Re: prokind column (was Re: [HACKERS] SQL procedures)

2018-02-25 Thread John Naylor
t touches src/include/catalog/, > so we ought to apply it at a point where that list of patches is short > and there's lots of time for people to redo them. Hence, end of the > dev cycle is the right time. I agree. -John Naylor

Re: missing toast table for pg_policy

2018-07-18 Thread John Naylor
large > object data is handled as user data by pg_upgrade, which would cause > failures." Sounds good to me. -John Naylor

Re: missing toast table for pg_policy

2018-07-17 Thread John Naylor
that was seen here? It would be nice > to see this patch committed but the reason here should be more > explicit about why this cannot happen. I'll copy what I wrote upthread last month: On 6/19/18, John Naylor wrote: > On 2/20/18, Michael Paquier wrote: >> Regression tests of pg_upgra

Re: missing toast table for pg_policy

2018-07-14 Thread John Naylor
it might be useful to update Joe's patch. The attached was rebased over the new regression test, passes the pg_upgrade test, and has a draft commit message. -John Naylor From e414effdcf03b3005390ab1671f4425c9e3036f2 Mon Sep 17 00:00:00 2001 From: John Naylor Date: Sat, 14 Jul 2018 15:09:44 +070

Re: Allow to specify a index name as ANALYZE parameter

2018-09-09 Thread John Naylor
he above, I thought it appropriate to mark the patch Waiting on Author. -John Naylor > Thanks, > > -- > Yugo Nagata > >

Re: pg_ugprade test failure on data set with column with default value with type bit/varbit

2018-09-09 Thread John Naylor
On 8/1/18, Paul Guo wrote: > Thanks. I updated the patch as attached. > > Double-checked those tests passed. I've verified make check-world passes. I've marked it Ready for Committer. -John Naylor

Re: MCV lists for highly skewed distributions

2018-01-19 Thread John Naylor
e class size in the sample, as denoted by gamma in the Haas and Stokes paper on page 7. -John Naylor

Re: MCV lists for highly skewed distributions

2018-03-13 Thread John Naylor
kept in the MCV list, its population frequency is +* assumed to equal its sample frequency, and the distribution of the +* value's count in the sample is a hypergeomtric distribution with + * the following standard deviation. +*/ The part after "and" doesn't seem to follow from the first part. Would you mind clarifying? -John Naylor

Re: WIP: a way forward on bootstrap data

2018-03-15 Thread John Naylor
On 3/15/18, Tom Lane <t...@sss.pgh.pa.us> wrote: > John Naylor <jcnay...@gmail.com> writes: >> It didn't take that long to rebase the remaining parts of the >> patchset, so despite what I said above I went ahead and put them in >> version 10 (attached), this time

Re: WIP: a way forward on bootstrap data

2018-04-05 Thread John Naylor
$values{aggfnoid} = lookup_procname($values{aggfnoid}); + } aggfnoid is in pg_aggregate, and pg_am already had a regproc lookup. Do you remember the intent here? -John Naylor

Re: WIP: a way forward on bootstrap data

2018-04-05 Thread John Naylor
can try a draft tomorrow to see how it looks, unless you see an obvious downside. -John Naylor

Re: WIP: a way forward on bootstrap data

2018-04-07 Thread John Naylor
ahead and wrapped it with an EXPOSE_TO_CLIENT_CODE macro. All your additional patches apply still apply over it. Your SGML patch can only apply if my doc patch is not applied, but I've included it anyway for the sake of no surprises. I'll check back in 24 hours to see if everything still applies. -John Naylor

Re: WIP: a way forward on bootstrap data

2018-04-05 Thread John Naylor
postgres.sql file which would load into a development schema so one could query the bootstrap data in a database without running initdb. I could update it at a future point. -John Naylor

Re: WIP: a way forward on bootstrap data

2018-04-05 Thread John Naylor
re shared, > so it ought to be able to figure out where to route the descriptions. Fairly trivial (attached), and shouldn't be too hard to integrate into the series. -John Naylor diff --git a/src/backend/catalog/README.data b/src/backend/catalog/README.data index 22ad0f2..2c05fab 100644 --- a/src

Re: WIP: a way forward on bootstrap data

2018-04-08 Thread John Naylor
I wrote: > I'll check back in 24 hours to see if everything still applies. There were a couple more catalog changes that broke patch context, so attached is version 16. -John Naylor v16-bootstrap-data-conversion.tar.gz Description: GNU Zip compressed data

Re: WIP: a way forward on bootstrap data

2018-04-08 Thread John Naylor
and initial commits, and to all who participated in previous discussion. -John Naylor

Re: Documentation for bootstrap data conversion

2018-04-09 Thread John Naylor
hat now that the conversion is done, it'd be better > to remove that special case. Would you send a patch for that? Sure, attached. -John Naylor diff --git a/src/include/catalog/reformat_dat_file.pl b/src/include/catalog/reformat_dat_file.pl index bbceb16..038ba7b 100644 --- a/src/include/

Re: WIP: a way forward on bootstrap data

2018-04-05 Thread John Naylor
a new one in a couple days, to give a chance for further review and discussion of the defaults. I didn't feel the need to respond to the other messages, but yours and Andres' points are well taken. -John Naylor diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl index f6be50

Re: Documentation for bootstrap data conversion

2018-04-09 Thread John Naylor
preferred style is to refer to "bootstrap" relations rather than "bootstrapped" relations. The attached patch makes code comments more like the docs in this regard. -John Naylor diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c index 59cd4b1..3da7861

Re: Verbosity of genbki.pl

2018-04-09 Thread John Naylor
quot;, and I guess a second one for fmgroids.h and related files. >> >> I don't have a strong preference. Opinions? > > Second point, pls. I'd like to see some stage done The attached patch does #2. -John Naylor diff --git a/src/backend/catalog/Catalog.pm b/src/backend/catalog/Ca

Re: WIP: a way forward on bootstrap data

2018-04-06 Thread John Naylor
nd in some > other places like AddDefaultValues, is to be able to finger the location > of the bad tuple by filename and line number, but I have no idea whether > it's practical to annotate the tuples with that while reading the .dat > files. Any thoughts? Done (patch 0007). So far onl

remove quoting hacks and simplify bootscanner.l

2018-04-17 Thread John Naylor
. Currently, Catalog.pm, genbki.pl, and bootscanner.l all have different ideas on how to parse and format array types. Patch 0003 rips all that out and does it once and for all in Catalog.pm. Change in postgres.bki: Array types now look like '_foo'. -John Naylor From

Re: Documentation for bootstrap data conversion

2018-04-19 Thread John Naylor
On 4/18/18, Tom Lane <t...@sss.pgh.pa.us> wrote: > John Naylor <jcnay...@gmail.com> writes: >> and dug through a bit to find cases where 'catalog' is clearly a >> better term. Most of these are in the pg_*.h/.dat file boilerplate >> comments, which would be ea

Re: Documentation for bootstrap data conversion

2018-04-16 Thread John Naylor
lso be referring to an index or toast table, so it's probably best to leave well enough alone for most cases. -John Naylor

Re: unused_oids script is broken with bsd sed

2018-04-25 Thread John Naylor
est, since they're run at the same time anyway (see catalog/Makefile), and we've already read all the data. -John Naylor

lingering references to V0 calling convention

2018-04-23 Thread John Naylor
of the configure options for --disable-floatN-byval, since this thread proposed something a bit more invasive: https://www.postgresql.org/message-id/flat/10862.1519228208%40sss.pgh.pa.us#10862.1519228...@sss.pgh.pa.us -John Naylor diff --git a/doc/src/sgml/plhandler.sgml b/doc/src/sgml/plhandler.sgml index

Re: WIP: a way forward on bootstrap data

2018-04-17 Thread John Naylor
OID(4066) BKI_WITHOUT_OIDS BKI_SCHEMA_MACRO -- with Patch 0002: 5 lines > 80 chars 102 CATALOG(pg_shseclabel,3592,SharedSecLabel) SHARED_RELATION ROWTYPE_OID(4066) WITHOUT_OIDS SCHEMA_MACRO -John Naylor From ff1384643e47ef423ab13cfd847f12c0969029f1 Mon Sep 17 00:00:00 2001 From: John Naylor

Re: WIP: a way forward on bootstrap data

2018-03-27 Thread John Naylor
ess. I think I'll consolidate all documentation patches into one, at the end of the series for maximum flexibility. I liked the idea of spreading the doc changes over the patches, but there is not a huge amount of time left. -John Naylor --- /home/john/pgdev/postgresql/src/include/catalog/reformat_dat_fi

Re: WIP: a way forward on bootstrap data

2018-03-26 Thread John Naylor
E into a new file, maybe README.data, which will contain code snippets and some example scenarios. I'll include the example pg_proc.prokind merger among those. -John Naylor

Re: WIP: a way forward on bootstrap data

2018-03-26 Thread John Naylor
nse on all points. I’m not optimistic about creating a new sgml doc on time, but I’ll keep it in mind. -John Naylor

Re: WIP: a way forward on bootstrap data

2018-03-25 Thread John Naylor
ic rearrangements and comment/commit message edits. Open items: -Test MSVC. -Arrange for rewrite_dat.pl to run when perltidy does. -I was a bit cavalier about when to use =/:= in the Makefiles. Not sure if there's a preferred project style for when the choice doesn't really matter. -Maybe document

Re: WIP: a way forward on bootstrap data

2018-03-31 Thread John Naylor
isting Makefile in which to put convenience targets for formatting. On the other hand, it kind of makes sense to have the files describing the schema (.h) and the contents (.dat) in the same directory. I'm inclined to leave things as they are for that reason. -John Naylor v13-bootstrap-data-conve

Re: JIT compiling with LLVM v12.2

2018-03-29 Thread John Naylor
, maybe that's also useful information. :-) -John Naylor diff --git a/src/backend/jit/README b/src/backend/jit/README index bfed319..7924127 100644 --- a/src/backend/jit/README +++ b/src/backend/jit/README @@ -13,12 +13,12 @@ the CPU that just handles that expression, yielding a speedup

Re: pgsql: Add documentation for the JIT feature.

2018-03-29 Thread John Naylor
jitted. No one agrees on that, but it seems very common to use "JIT" as a verb. See the LLVM docs: https://llvm.org/docs/DebuggingJITedCode.html -John Naylor

Re: MCV lists for highly skewed distributions

2018-03-18 Thread John Naylor
t from the stats target, and use a hard-coded maximum like 1000. That might allow us to bump the max stat target without hurting planning time. Perhaps next cycle. -John Naylor [1] https://www.postgresql.org/message-id/CAEZATCVHEEg%2BCrP%2B-0JUsVeNPu5rs_S23oJVeH4VF%3DfgDwhfLQ%40mail.gmail.com #!/u

Re: WIP: a way forward on bootstrap data

2018-03-22 Thread John Naylor
;> and change it to '\0' for schemapg.h. > >> /* One of the ATTRIBUTE_IDENTITY_* constants below, or '\0' */ >> char attidentity BKI_DEFAULT(""); > > That definitely seems like a hack --- why not BKI_DEFAULT('\0') ? Hmm, yes, the way I had it, the comment is a mystery. I'll switch it around. -John Naylor

Re: WIP: a way forward on bootstrap data

2018-03-22 Thread John Naylor
chartypdelim BKI_DEFAULT(\054); Which I hope is fine. Were you thinking of this comment in pg_attribute.h? We use the double-quoted empty string for postgres.bki and change it to '\0' for schemapg.h. /* One of the ATTRIBUTE_IDENTITY_* constants below, or '\0' */ charattidentity BKI_DEFAULT(""); -John Naylor

Re: WIP: a way forward on bootstrap data

2018-03-22 Thread John Naylor
f pg_proc.h with both formats as I understand them. I turned a couple comments into multi-line comments to demonstrate. I think without spaces it's just as hard to read as with multiple annotations. I'd vote for spaces, but then again I'm not th

Re: WIP: a way forward on bootstrap data

2018-03-22 Thread John Naylor
none) */ And thought "only a machine could be so precisely awkward" -John Naylor

Re: WIP: a way forward on bootstrap data

2018-03-22 Thread John Naylor
On 3/21/18, Tom Lane <t...@sss.pgh.pa.us> wrote: > John Naylor <jcnay...@gmail.com> writes: >> [ v11-bootstrap-data-conversion.tar.gz ] > > I've done a round of review work on this, focusing on the Makefile > infrastructure. I found a bunch of problems with para

Re: MCV lists for highly skewed distributions

2018-03-19 Thread John Naylor
1 Ran. unif. dist. (N=60k, Nd=5000) | 14 | 1 | 0 (38 rows) -John Naylor MCV_queries_truncated_mean.sql Description: application/sql

Re: WIP: a way forward on bootstrap data

2018-03-03 Thread John Naylor
Thanks for taking a look. On 3/3/18, Tom Lane <t...@sss.pgh.pa.us> wrote: > John Naylor <jcnay...@gmail.com> writes: >> Version 8, rebased against 76b6aa41f41d. > > I took a preliminary look through this, without yet attempting to execute > the script agains

Re: WIP: a way forward on bootstrap data

2018-03-03 Thread John Naylor
I wrote: > I'll submit a > preliminary patch soon to get some of those items out of the way. I've attached a patch that takes care of these cleanups so they don't clutter the patch set. -John Naylor From d97a8b2e5fa4977e656d1aca7ee7bf1289ecbd40 Mon Sep 17 00:00:00 2001 From: John Naylor

Re: MCV lists for highly skewed distributions

2018-03-06 Thread John Naylor
alent, right? Just a label for convenience to make sure you ran the same number of tests? -John Naylor

Re: unused_oids script is broken with bsd sed

2018-04-25 Thread John Naylor
ing to use Catalog.pm for that, it seems more convenient to expose toast and index oids directly rather than in strings formatted specifically for the bki file, as in the attached. Thoughts? -John Naylor diff --git a/src/backend/catalog/Catalog.pm b/src/backend/catalog/Catalog.pm index 6305a2b..601d68

Re: unused_oids script is broken with bsd sed

2018-04-25 Thread John Naylor
still assume we're in the catalog directory. I can hack on that part tomorrow if no one else has. -John Naylor

Re: WIP: Avoid creation of the free space map for small tables

2018-10-08 Thread John Naylor
On 10/7/18, Tom Lane wrote: > John Naylor writes: >> On 10/6/18, Thomas Munro wrote: >>> On Sat, Oct 6, 2018 at 7:47 AM John Naylor wrote: >>>> A while back, Robert Haas noticed that the space taken up by very >>>> small tables is dominated b

WIP: Avoid creation of the free space map for small tables

2018-10-05 Thread John Naylor
.pgh.pa.us -- I'll add this to the November commitfest. -John Naylor fsmtest.sql Description: application/sql From 77c85f633f915bd247c554b691a134fac1f32316 Mon Sep 17 00:00:00 2001 From: John Naylor Date: Sat, 6 Oct 2018 00:35:33 +0700 Subject: [PATCH v1] Avoid creation of the free space map for sm

Re: WIP: Avoid creation of the free space map for small tables

2018-10-14 Thread John Naylor
ot a relation, I had to reimplement that separately, but the logic is basically the same. It works under streaming replication. I've also attached a couple SQL scripts which, when the aforementioned DEBUG1 calls are enabled, show what the heap insert code is doing for different sce

Re: WIP: Avoid creation of the free space map for small tables

2018-10-15 Thread John Naylor
k if some other backend has extended a block for us while * we were waiting on the lock. */ targetBlock = GetPageWithFreeSpace(relation, len + saveFreeSpace); By the time we get to this call, we likely wouldn't trigger the logic to try every block, but I don't think we can guarantee that. We could add a boolean parameter that means "consider trying every block", but I don't think the FSM code should have so much state passed to it. Thanks for reviewing, -John Naylor

Re: WIP: Avoid creation of the free space map for small tables

2018-10-16 Thread John Naylor
callers. > [proposal for a cache of blocks to try] That's interesting. I'll have to do some reading elsewhere in the codebase, and then I'll follow up. Thanks, -John Naylor From 0dafd220e437ed1cf9ed77e41cf3090365a63798 Mon Sep 17 00:00:00 2001 From: John Naylor Date: Tue, 16 Oct 2018 17:10:14 +0

Re: WIP: Avoid creation of the free space map for small tables

2018-10-22 Thread John Naylor
buffer, whether from an existing block, or by extension, we clear the local map. Once this is in shape, I'll do some performance testing. -John Naylor From 529fa1f57946d70736b2304c2883213e45f7c077 Mon Sep 17 00:00:00 2001 From: John Naylor Date: Mon, 22 Oct 2018 13:39:25 +0700 Subject: [PATCH v6]

ERROR's turning FATAL in BRIN regression tests

2018-10-19 Thread John Naylor
osed the connection unexpectedly + This probably means the server terminated abnormally + before or while processing the request. + connection to server was lost I'm thoroughly stumped -- anyone have an idea where to look next? Thanks, -John Naylor

Re: WIP: Avoid creation of the free space map for small tables

2018-10-13 Thread John Naylor
On 10/13/18, Amit Kapila wrote: > On Sat, Oct 6, 2018 at 12:17 AM John Naylor wrote: >> -For normal mode, I taught fsm_set_and_search() to switch to a >> non-extending buffer call, but the biggest missing piece is WAL >> replay. >> > > fsm_set_and_search() &

Re: inconsistency and inefficiency in setup_conversion()

2018-10-13 Thread John Naylor
Attached is v6, a simple rebase. -John Naylor From e37cb80ab8e7baaa5231fc3b8dbc9d96ec253018 Mon Sep 17 00:00:00 2001 From: John Naylor Date: Sat, 13 Oct 2018 19:28:08 +0700 Subject: [PATCH v6 1/2] Add pg_language lookup. This didn't seem worth doing before, but an upcoming commit will add 88

Re: WIP: Avoid creation of the free space map for small tables

2018-10-23 Thread John Naylor
FSM in shared buffers exactly makes up for the 3 other blocks of heap that are checked when free space runs out. I can run additional tests if there's interest. -John Naylor fsm-copy-test.sql Description: application/sql

Re: PostgreSQL Limits and lack of documentation about them.

2018-10-31 Thread John Naylor
63 characters + Can this be increased with recompiling, if not conveniently? + Maximum Indexed Columns + 32 + Can be increased by recompiling PostgreSQL How about the max number of included columns in a covering index? > I'm not so sure about detailing limits of GUCs since the limits of > those are mentioned in pg_settings. Maybe we could just have a link to that section in the docs. -- -John Naylor

Re: PostgreSQL Limits and lack of documentation about them.

2018-10-31 Thread John Naylor
header + toast table oid + chunk_id + logical size + compressed size. This is one area where visual diagrams would be nice. -John Naylor

Re: [HACKERS] generated columns

2018-11-01 Thread John Naylor
and atthasmissing). A bit crazier, what if "d = dropped" was another allowed value in attmaterialized -- we could then get rid of attisdropped as well. That has obvious disadvantages, but the broader idea is that this design may have use cases we haven't thought of yet. Thoughts? -John Naylor

Re: WIP: Avoid creation of the free space map for small tables

2018-10-31 Thread John Naylor
at all. This isn't anything to get excited about, but it would be nice to investigate, at least so it can be documented. A cursory dig hasn't found the cause, but I'll keep doing that as time permits. -John Naylor From 428b56e12e3f5d2bd8fc81c9fb5fe7169e9da580 Mon Sep 17 00:00:00 2001 From: John Naylor Dat

Re: WIP: Avoid creation of the free space map for small tables

2018-11-04 Thread John Naylor
in a fairly different way. Were you referring to performance or just functionality? Because the threshold works on the standby, but I don't know about the performance there. -John Naylor fsm-copy-test-v2.sql Description: application/sql

Re: "Writing" output lines during make

2018-11-04 Thread John Naylor
ing output, perhaps from a > Perl library. I would like to suppress them but I don't know how. Those should be gone as of pg11. Otherwise, those messages come from in backend/catalog/genkbi.pl -John Naylor

Re: WIP: Avoid creation of the free space map for small tables

2018-11-05 Thread John Naylor
first 3 pages are all wasted space, that's the amount of space that would have gone to the FSM anyway, and the relation will likely grow beyond the threshold soon, at which point the free space will become visible again. -John Naylor

Re: WIP: Avoid creation of the free space map for small tables

2018-11-02 Thread John Naylor
rger than required, and then test inserting into X number of pages, to simulate a threshold of X. I increased X until I saw a regression. That's where the 32 came from, sorry if that was misleading, in my head it was obvious. I'd be happy test other scenarios. I'm not sure how to test redo -- seems more difficult to get meaningful results than the normal case. -John Naylor

Re: PostgreSQL Limits and lack of documentation about them.

2018-11-07 Thread John Naylor
here are better ways to format all this, but trying to detail out > what the content should be first. The language seems fine to me. -John Naylor

doc fix for pg_stat_activity.backend_type

2018-11-11 Thread John Naylor
Hi all, Commit fc70a4b0df3 added backend_type to pg_stat_activity, but the documentation omitted "logical replication launcher". Patch attached. -John Naylor diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index add71458e2..ddf607bb17 100644 --- a/do

Re: doc fix for pg_stat_activity.backend_type

2018-11-12 Thread John Naylor
On 11/13/18, Amit Kapila wrote: > On Tue, Nov 13, 2018 at 5:38 AM Michael Paquier > wrote: >> >> On Mon, Nov 12, 2018 at 09:42:45PM +0700, John Naylor wrote: >> > Looks like it. A quick search revealed "parallel worker" and "logical >> &g

Re: doc fix for pg_stat_activity.backend_type

2018-11-13 Thread John Naylor
On 11/13/18, Amit Kapila wrote: > On Tue, Nov 13, 2018 at 12:04 PM John Naylor wrote: >> >> On 11/13/18, Amit Kapila wrote: >> > On Tue, Nov 13, 2018 at 5:38 AM Michael Paquier >> > wrote: >> >> >> >> On Mon, Nov 12, 2018 at 09:42:45PM

Re: PostgreSQL Limits and lack of documentation about them.

2018-11-12 Thread John Naylor
ggestions for a few days, I'll mark it ready for committer. -John Naylor

Re: doc fix for pg_stat_activity.backend_type

2018-11-13 Thread John Naylor
On 11/13/18, Amit Kapila wrote: > On Tue, Nov 13, 2018 at 3:37 PM John Naylor wrote: >> >> On 11/13/18, Amit Kapila wrote: >> > >> > Don't you need to remove background worker? >> >> It's handled in pgstat_get_backend_desc(), so I assumed not

Re: doc fix for pg_stat_activity.backend_type

2018-11-12 Thread John Naylor
On 11/12/18, Amit Kapila wrote: > On Mon, Nov 12, 2018 at 1:20 PM John Naylor wrote: >> >> Hi all, >> >> Commit fc70a4b0df3 added backend_type to pg_stat_activity, but the >> documentation omitted "logical replication launcher". Patch attached. >>

Re: [RFC] Removing "magic" oids

2018-11-15 Thread John Naylor
On 11/16/18, Andres Freund wrote: > On 2018-11-15 17:25:21 +0700, John Naylor wrote: >> I don't see an advantage to having a different range, but maybe it >> should error out if $maxoid reaches FirstBootstrapObjectId. > > Hm. Not sure I really see the point. Note we d

  1   2   >