Re: Statistics Import and Export

2024-05-16 Thread Corey Huinker
> > Can you explain what you did with the > SECTION_NONE/SECTION_DATA/SECTION_POST_DATA over v19-v21 and why? > Initially, I got things to work by having statistics import behave like COMMENTs, which meant that they were run immediately after the table/matview/index/constraint that created the

Re: Virtual generated columns

2024-04-29 Thread Corey Huinker
On Mon, Apr 29, 2024 at 4:24 AM Peter Eisentraut wrote: > Here is a patch set to implement virtual generated columns. > I'm very excited about this! > The main feature patch (0005 here) generally works but has a number of > open corner cases that need to be thought about and/or fixed, many

Re: documentation structure

2024-04-28 Thread Corey Huinker
> > I've splitted it to7 patches. > each patch split one into separate new files. > Seems like a good start. Looking at the diffs of these, I wonder if we would be better off with a func/ directory, each function gets its own file in that dir, and either these files above include the individual

Re: documentation structure

2024-04-18 Thread Corey Huinker
> > Yeah, we can't expect everyone wanting to call a built-in function to > know how they would define an equivalent one themselves. In that case I > propos marking it up like this: > > format ( > formatstr text > , formatarg "any" > , ... ) > text > Looks good, but I guess I

Re: documentation structure

2024-04-18 Thread Corey Huinker
> > I havent dealt with variadic yet, since the two styles are visually > different, not just markup (... renders as [...]). > > The two styles for variadic are the what I call caller-style: > >concat ( val1 "any" [, val2 "any" [, ...] ] ) >format(formatstr text [, formatarg "any" [, ...]

Re: improve performance of pg_dump --binary-upgrade

2024-04-18 Thread Corey Huinker
> > One downside of this approach is the memory usage. This was more-or-less > > Bar-napkin math tells me in a worst-case architecture and braindead byte alignment, we'd burn 64 bytes per struct, so the 100K tables cited would be about 6.25MB of memory. The obvious low-memory alternative would

Re: documentation structure

2024-04-17 Thread Corey Huinker
> > And it's very inconsistent. For example, some functions use > tags for optional parameters, others use square brackets, and some use > VARIADIC to indicate variadic parameters, others use > ellipses (sometimes in tags or brackets). Having just written a couple of those functions, I wasn't

Re: documentation structure

2024-04-17 Thread Corey Huinker
> > > This sounds to me like it would be a painful exercise with not a > > lot of benefit in the end. > > Maybe we could _verify_ the contents of func.sgml against pg_proc. > All of the functions redefined in catalog/system_functions.sql complicate using pg_proc.dat as a doc generator or source

Importing Extended Statistics

2024-04-12 Thread Corey Huinker
I'm creating this new thread separate from the existing Statistics Export/Import thread to keep the original thread focused on that patch. Assuming that the function signature for pg_set_attribute_stats() remains the same (regclass, attname, inherited, version, ...stats...), how would we design

Re: Statistics Import and Export

2024-04-06 Thread Corey Huinker
> > > > I think it'll be a serious, serious error for this not to be > SECTION_DATA. Maybe POST_DATA is OK, but even that seems like > an implementation compromise not "the way it ought to be". > We'd have to split them on account of when the underlying object is created. Index statistics would

Re: Statistics Import and Export

2024-04-04 Thread Corey Huinker
> > For a partitioned table this value has to be true. For a normal table when > setting this value to true, it should at least make sure that the table has > at least one child. Otherwise it should throw an error. Blindly accepting > the given value may render the statistics unusable. Prologue of

Re: Statistics Import and Export

2024-04-03 Thread Corey Huinker
> > > As far as that goes, it shouldn't be that hard to deal with, at least > not for "soft" errors which hopefully cover most input-function > failures these days. You should be invoking array_in via > InputFunctionCallSafe and passing a suitably-set-up ErrorSaveContext. > (Look at

Re: Statistics Import and Export

2024-04-02 Thread Corey Huinker
> > Yeah, but that problem exists no matter what. I haven't read enough > of the patch to find where it's determining that, but I assume there's > code in there to intuit the statistics storage type depending on the > table column's data type and the statistics kind. > Correct. It borrows a lot

Re: Statistics Import and Export

2024-04-02 Thread Corey Huinker
> > side to know the element type anyway. So, I apologize for sending > us down a useless side path. We may as well stick to the function > signature as shown in the v15 patch --- although maybe variadic > any is still worthwhile so that an unrecognized field name doesn't > need to be a hard

Re: Statistics Import and Export

2024-04-02 Thread Corey Huinker
I have refactored pg_set_relation_stats to be variadic, and I'm working on pg_set_attribute_sttats, but I'm encountering an issue with the anyarray values. Jeff suggested looking at anyarray_send as a way of extracting the type, and with some extra twiddling we can get and cast the type. However,

Re: Statistics Import and Export

2024-04-02 Thread Corey Huinker
Here's a one-liner patch for disabling update of pg_class relpages/reltuples/relallviible during a binary upgrade. This was causting pg_upgrade tests to fail in the existing stats import work. From 322db8c9e8796ce673f7d7b63bd64e4c9403a144 Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Mon, 1

Re: Statistics Import and Export

2024-04-01 Thread Corey Huinker
> > If we are envisioning that the function might emit multiple warnings > per call, a useful definition could be to return the number of > warnings (so zero is good, not-zero is bad). But I'm not sure that's > really better than a boolean result. pg_dump/pg_restore won't notice > anyway, but

Re: Statistics Import and Export

2024-04-01 Thread Corey Huinker
> > > I still think that we could just declare the function strict, if we > use the variadic-any approach. Passing a null in any position is > indisputable caller error. However, if you're allergic to silently > doing nothing in such a case, we could have pg_set_attribute_stats > check each

Re: Statistics Import and Export

2024-04-01 Thread Corey Huinker
> > Ah, yeah, you could change the function to have more parameters, > given the assumption that all calls will be named-parameter style. > I still suggest that my proposal is more robust for the case where > the dump lists parameters that the receiving system doesn't have. > So what's the

Re: Statistics Import and Export

2024-04-01 Thread Corey Huinker
> > I think pg_upgrade could check for the existence of extended statistics > in any database and adjust the analyze recommdnation wording > accordingly. > +1

Re: Statistics Import and Export

2024-04-01 Thread Corey Huinker
> > That's not what I suggested at all. The function parameters would > be declared anyarray, but the values passed to them would be coerced > to the correct concrete array types. So as far as the coercion rules > are concerned this'd be equivalent to the variadic-any approach. > +1 > > >

Re: Statistics Import and Export

2024-03-31 Thread Corey Huinker
> > IIRC, "variadic any" requires having at least one variadic parameter. > But that seems fine --- what would be the point, or even the > semantics, of calling pg_set_attribute_stats with no data fields? > If my pg_dump run emitted a bunch of stats that could never be imported, I'd want to know.

Re: Statistics Import and Export

2024-03-31 Thread Corey Huinker
> > > I concur with the plan of extracting data from pg_stats not > pg_statistics, and with emitting a single "set statistics" > call per attribute. (I think at one point I'd suggested a call > per stakind slot, but that would lead to a bunch of UPDATEs on > existing pg_attribute tuples and hence

Re: Statistics Import and Export

2024-03-31 Thread Corey Huinker
> > I wonder if the right answer to that is "let's enhance the I/O > functions for those types". But whether that helps or not, it's > v18-or-later material for sure. > That was Stephen's take as well, and I agreed given that I had to throw the kitchen-sink of source-side oid mappings (attname,

Re: Statistics Import and Export

2024-03-31 Thread Corey Huinker
On Sun, Mar 31, 2024 at 2:41 PM Tom Lane wrote: > Corey Huinker writes: > > Having given this some thought, I'd be inclined to create a view, > > pg_stats_missing, with the same security barrier as pg_stats, but looking > > for tables that lack stats on at least one

Re: Statistics Import and Export

2024-03-31 Thread Corey Huinker
> > That will make it *really* hard for any form of automation or drivers of > this. The information needs to go somewhere where such tools can easily > consume it, and an informational message during runtime (which is also > likely to be translated in many environments) is the exact opposite of

Re: Statistics Import and Export

2024-03-30 Thread Corey Huinker
> > I didn't have any specific proposal in mind, was just trying to think > outside the box. > What if we added a separate resection SECTION_STATISTICS which is run following post-data?

Re: Statistics Import and Export

2024-03-30 Thread Corey Huinker
> > I'm getting late into this discussion and I apologize if I've missed this > being discussed before. But. > > Please don't. > > That will make it *really* hard for any form of automation or drivers of > this. The information needs to go somewhere where such tools can easily > consume it, and an

Re: Statistics Import and Export

2024-03-29 Thread Corey Huinker
> > (I've not read the patch yet, but I assume the switch works like > other pg_dump filters in that you can apply it on the restore > side?) > Correct. It follows the existing --no-something pattern.

Re: Statistics Import and Export

2024-03-29 Thread Corey Huinker
On Fri, Mar 29, 2024 at 7:34 PM Jeff Davis wrote: > On Fri, 2024-03-29 at 18:02 -0400, Stephen Frost wrote: > > I’d certainly think “with stats” would be the preferred default of > > our users. > > I'm concerned there could still be paths that lead to an error. For > pg_restore, or when loading

Re: Statistics Import and Export

2024-03-29 Thread Corey Huinker
> > > There's still a failure in the pg_upgrade TAP test. One seems to be > ordering, so perhaps we need to ORDER BY the attribute number. Others > seem to be missing relstats and I'm not sure why yet. I suggest doing > some manual pg_upgrade tests and comparing the before/after dumps to > see if

Re: Statistics Import and Export

2024-03-27 Thread Corey Huinker
> > 1) The docs say this: > > >The purpose of this function is to apply statistics values in an >upgrade situation that are "good enough" for system operation until >they are replaced by the next ANALYZE, usually via >autovacuum This function is used by >pg_upgrade and

Re: Statistics Import and Export

2024-03-27 Thread Corey Huinker
> > > > +\gexec > > Why do we need to construct the command and execute? Can we instead > execute the function directly? That would also avoid ECHO magic. > We don't strictly need it, but I've found the set-difference operation to be incredibly useful in diagnosing problems. Additionally, the

Re: Statistics Import and Export

2024-03-21 Thread Corey Huinker
> > > > But ideally we'd just make it safe to dump and reload stats on your own > tables, and then not worry about it. > That is my strong preference, yes. > > > Not off hand, no. > > To me it seems like inconsistent data to have most_common_freqs in > anything but descending order, and we

Re: Statistics Import and Export

2024-03-21 Thread Corey Huinker
> > How about just some defaults then? Many of them have a reasonable > default, like NULL or an empty array. Some are parallel arrays and > either both should be specified or neither (e.g. > most_common_vals+most_common_freqs), but you can check for that. > +1 Default NULL has been implemented

Re: Statistics Import and Export

2024-03-21 Thread Corey Huinker
On Thu, Mar 21, 2024 at 2:29 AM Jeff Davis wrote: > On Tue, 2024-03-19 at 05:16 -0400, Corey Huinker wrote: > > v11 attached. > > Thank you. > > Comments on 0001: > > This test: > >+SELECT >+format('SELECT pg_catalog.pg_set_attribute_stats(

Re: Statistics Import and Export

2024-03-19 Thread Corey Huinker
more cleanly - pg_set_*_stats function now have all of their parameters in the same order as the table/view they pull from From 5c63ed5748eb3817d193b64329b57dc590e0196e Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Mon, 11 Mar 2024 14:18:39 -0400 Subject: [PATCH v11 1/2] Create

Re: Statistics Import and Export

2024-03-18 Thread Corey Huinker
> > > > > From testrun/pg_dump/002_pg_dump/log/regress_log_002_pg_dump, search > for the "not ok" and then look at what it tried to do right before > that. I see: > > pg_dump: error: prepared statement failed: ERROR: syntax error at or > near "%" > LINE 1: ..._histogram => %L::real[])

Re: Statistics Import and Export

2024-03-17 Thread Corey Huinker
ailable in the patches attached. From ba411ce31c25193cf05bc4206dcb8f2bf32af0c7 Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Mon, 11 Mar 2024 14:18:39 -0400 Subject: [PATCH v10 1/2] Create pg_set_relation_stats, pg_set_attribute_stats. These functions will be used by pg_dump/restore and p

Re: Statistics Import and Export

2024-03-15 Thread Corey Huinker
79c11bd22d2ec Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Mon, 11 Mar 2024 14:18:39 -0400 Subject: [PATCH v9 1/2] Create pg_set_relation_stats, pg_set_attribute_stats. These functions will be used by pg_dump/restore and pg_upgrade to convey relation and attribute statistics from the source d

Re: Statistics Import and Export

2024-03-13 Thread Corey Huinker
> > Note that there's two different things we're talking about here- the > lock on the relation that we're analyzing and then the lock on the > pg_statistic (or pg_class) catalog itself. Currently, at least, it > looks like in the three places in the backend that we open > StatisticRelationId, we

Re: Statistics Import and Export

2024-03-12 Thread Corey Huinker
> > No, we should be keeping the lock until the end of the transaction > (which in this case would be just the one statement, but it would be the > whole statement and all of the calls in it). See analyze.c:268 or > so, where we call relation_close(onerel, NoLock); meaning we're closing > the

Re: Statistics Import and Export

2024-03-11 Thread Corey Huinker
> > > In which case we're failing nearly silently, yes, there is a null > returned, > > but we have no idea why there is a null returned. If I were using this > > function manually I'd want to know what I did wrong, what parameter I > > skipped, etc. > > I can see it both ways and don't feel super

Re: Statistics Import and Export

2024-03-11 Thread Corey Huinker
The second one has the feature that if any one attribute fails, then the whole update fails, except, of course, for the in-place update of pg_class. This avoids having an explicit transaction block, but we could get that back by having restore wrap the list of commands in a transaction block (an

Re: Statistics Import and Export

2024-03-10 Thread Corey Huinker
On Sun, Mar 10, 2024 at 11:57 AM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Fri, Mar 8, 2024 at 12:06 PM Corey Huinker > wrote: > > > > Anyway, here's v7. Eagerly awaiting feedback. > > Thanks for working on this. It looks useful to

Re: Statistics Import and Export

2024-03-08 Thread Corey Huinker
> > Perhaps it's just me ... but it doesn't seem like it's all that many > parameters. > It's more than I can memorize, so that feels like a lot to me. Clearly it's not insurmountable. > > I am a bit concerned about the number of locks on pg_statistic and the > > relation itself, doing

Re: Statistics Import and Export

2024-03-07 Thread Corey Huinker
ily depends on how this is received. Also, I'm still seeking confirmation that I can create a pg_dump TOC entry with a chain of commands (e.g. BEGIN; ... COMMIT; ) or if I have to fan them out into multiple entries. Anyway, here's v7. Eagerly awaiting feedback. From 9bc7200b8a67efefe20453e0c48aed8a

Re: Statistics Import and Export

2024-03-07 Thread Corey Huinker
> > > > BEGIN; > > LOCK TABLE schema.relation IN SHARE UPDATE EXCLUSIVE MODE; > > LOCK TABLE pg_catalog.pg_statistic IN ROW UPDATE EXCLUSIVE MODE; > > SELECT pg_import_rel_stats('schema.relation', ntuples, npages); > > SELECT pg_import_pg_statistic('schema.relation', 'id', ...); > > SELECT

Re: Statistics Import and Export

2024-02-29 Thread Corey Huinker
> > > That’s certainly a fair point and my initial reaction (which could > certainly be wrong) is that it’s unlikely to be an issue- but also, if you > feel you could make it work with an array and passing all the attribute > info in with one call, which I suspect would be possible but just a bit

Re: Statistics Import and Export

2024-02-29 Thread Corey Huinker
> > > > Having looked through this thread and discussed a bit with Corey > off-line, the approach that Tom laid out up-thread seems like it would > make the most sense overall- that is, eliminate the JSON bits and the > SPI and instead export the stats data by running queries from the new >

Re: Document efficient self-joins / UPDATE LIMIT techniques.

2024-02-15 Thread Corey Huinker
> > > As for whether it's commonplace, when I was a consultant I had a number > > of customers that I had who bemoaned how large updates caused big > > replica lag, basically punishing access to records they did care about > > in order to properly archive or backfill records they don't care about.

Re: Document efficient self-joins / UPDATE LIMIT techniques.

2024-02-13 Thread Corey Huinker
On Tue, Feb 13, 2024 at 11:51 AM Joel Jacobson wrote: > On Tue, Feb 13, 2024, at 10:28, Laurenz Albe wrote: > > On Mon, 2024-02-12 at 12:24 -0500, Corey Huinker wrote: > >> > Do you plan to add it to the commitfest? If yes, I'd set it "ready > for committer

Re: Statistics Import and Export

2024-02-12 Thread Corey Huinker
> > Also, it says "statistics are replaced" but it's quite clear if that > applies only to matching statistics or if all stats are deleted first > and then the new stuff is inserted. (FWIW remove_pg_statistics clearly > deletes all pre-existing stats). > All are now deleted first, both in the

Re: Document efficient self-joins / UPDATE LIMIT techniques.

2024-02-12 Thread Corey Huinker
> > Do you plan to add it to the commitfest? If yes, I'd set it "ready for > committer". > > Commitfest entry reanimated.

Re: Document efficient self-joins / UPDATE LIMIT techniques.

2024-02-12 Thread Corey Huinker
ot; before between is unnecessary and had better be removed, > but > I'll defer to the native speaker. > The "in" is more common when spoken. Removed. From a6b57bf3a88c5df614b5dede99af3e99fe8e8089 Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Mon, 12 Feb 2024 11:32:49

Re: Document efficient self-joins / UPDATE LIMIT techniques.

2024-02-03 Thread Corey Huinker
ty. Because the join syntax is subtly different between UPDATE and DELETE, I've kept code examples in both, but the detailed explanation is in UPDATE under the anchor "update-limit" and the DELETE example links to it. From 298c812838491408e6910f7535067ea147abe5fc Mon Sep 17 00:00:00 2001 From

Re: Statistics Import and Export

2024-02-02 Thread Corey Huinker
On Mon, Jan 22, 2024 at 1:09 AM Peter Smith wrote: > 2024-01 Commitfest. > > Hi, This patch has a CF status of "Needs Review" [1], but it seems > there were CFbot test failures last time it was run [2]. Please have a > look and post an updated version if necessary. > > == > [1]

Re: Statistics Import and Export

2023-12-29 Thread Corey Huinker
> > But maybe it's enough to just do what you did - if we get an MCELEM > slot, can it ever contain anything else than array of elements of the > attribute array type? I'd bet that'd cause all sorts of issues, no? > > Thanks for the explanation of why it wasn't working for me. Knowing that the

Re: Statistics Import and Export

2023-12-28 Thread Corey Huinker
On Wed, Dec 27, 2023 at 10:10 PM Tom Lane wrote: > Corey Huinker writes: > > Export functions was my original plan, for simplicity, maintenance, etc, > > but it seemed like I'd be adding quite a few functions, so the one view > > made more sense for an initial version. Also

Re: Statistics Import and Export

2023-12-27 Thread Corey Huinker
> > Yeah, this is pretty much what I meant by "functional" interface. But if > I said maybe the format implemented by the patch is maybe too close to > how we store the statistics, then this has exactly the same issue. And > it has other issues too, I think - it breaks down the stats into >

Re: Statistics Import and Export

2023-12-27 Thread Corey Huinker
> > > As mentioned already, we'd also need some sort of > version identifier, and we'd expect the load_statistics() functions > to be able to transform the data if the old version used a different > representation. I agree with the idea that an explicit representation > of the source table

Re: Statistics Import and Export

2023-12-27 Thread Corey Huinker
On Mon, Dec 25, 2023 at 8:18 PM Tomas Vondra wrote: > Hi, > > I finally had time to look at the last version of the patch, so here's a > couple thoughts and questions in somewhat random order. Please take this > as a bit of a brainstorming and push back if you disagree some of my > comments. > >

Re: Statistics Import and Export

2023-12-15 Thread Corey Huinker
On Fri, Dec 15, 2023 at 3:36 AM Andrei Lepikhov wrote: > On 13/12/2023 17:26, Corey Huinker wrote:> 4. I don't yet have a > complete vision for how these tools will be used > > by pg_upgrade and pg_dump/restore, the places where these will provide > > the biggest win for u

Re: Statistics Import and Export

2023-11-01 Thread Corey Huinker
> > > > Maybe I just don't understand, but I'm pretty sure ANALYZE does not > derive index stats from column stats. It actually builds them from the > row sample. > That is correct, my error. > > > * now support extended statistics except for MCV, which is currently > > serialized as an

Re: Document efficient self-joins / UPDATE LIMIT techniques.

2023-10-31 Thread Corey Huinker
aid in discovery. A user might miss the technique for update if it's only documented in delete, and even if they did see it there, they might not realize that it works for both UPDATE and DELETE. We could make reference links from one to the other, but that seems like extra work for the reader. From c6

Re: SQL:2011 application time

2023-08-31 Thread Corey Huinker
> > The PERIOD patch is not finished and includes some deliberately-failing > tests. I did make some progress here finishing ALTER TABLE ADD PERIOD. > If it's ok with you, I need PERIODs for System Versioning, and planned on developing a highly similar version, albeit closer to the standard. It

Re: Statistics Import and Export

2023-08-31 Thread Corey Huinker
> > > Thanks. I think this may be used with postgres_fdw to import > statistics directly from the foreigns server, whenever possible, > rather than fetching the rows and building it locally. If it's known > that the stats on foreign and local servers match for a foreign table, > we will be one

Re: Document efficient self-joins / UPDATE LIMIT techniques.

2023-08-31 Thread Corey Huinker
On Wed, Jun 28, 2023 at 2:20 PM Corey Huinker wrote: > This patch adds a few examples to demonstrate the following: > Bumping so CF app can see thread. >

Statistics Import and Export

2023-08-31 Thread Corey Huinker
pg_stats_export is a view that aggregates pg_statistic data by relation oid and stores all of the column statistical data in a system-indepdent (i.e. no oids, collation information removed, all MCV values rendered as text) jsonb format, along with the relation's relname, reltuples, and relpages

Document efficient self-joins / UPDATE LIMIT techniques.

2023-06-28 Thread Corey Huinker
This patch adds a few examples to demonstrate the following: * The existence of the ctid column on every table * The utility of ctds in self joins * A practical usage of SKIP LOCKED The reasoning for this is a bit long, but if you're interested, keep reading. In the past, there has been a

Re: Should heapam_estimate_rel_size consider fillfactor?

2023-06-14 Thread Corey Huinker
> > So maybe we should make table_block_relation_estimate_size smarter to > also consider the fillfactor in the "no statistics" branch, per the > attached patch. > I like this a lot. The reasoning is obvious, the fix is simple,it doesn't upset any make-check-world tests, and in order to get a

Re: Adding SHOW CREATE TABLE

2023-05-19 Thread Corey Huinker
> > I think the ONLY place we should have this is in server side functions. > More than ten years ago I did some work in this area (see below), but it's > one of those things that have been on my ever growing personal TODO list > > See >

Re: Large files for relations

2023-05-09 Thread Corey Huinker
On Wed, May 3, 2023 at 1:37 AM Thomas Munro wrote: > On Wed, May 3, 2023 at 5:21 PM Thomas Munro > wrote: > > rsync --link-dest > > I wonder if rsync will grow a mode that can use copy_file_range() to > share blocks with a reference file (= previous backup). Something > like --copy-range-dest.

Re: Note new NULLS NOT DISTINCT on unique index tutorial page

2023-04-17 Thread Corey Huinker
On Wed, Apr 12, 2023 at 10:40 AM David Gilman wrote: > The SQL Language part of the docs has a brief page on unique indexes. > It doesn't mention the new NULLS NOT DISTINCT functionality on unique > indexes and this is a good place to mention it, as it already warns > the user about the

Re: Add SHELL_EXIT_CODE to psql

2023-04-05 Thread Corey Huinker
On Fri, Mar 24, 2023 at 5:21 PM Corey Huinker wrote: > > > On Tue, Mar 21, 2023 at 3:33 PM Corey Huinker > wrote: > >> >>> As you had it, any nonzero result would prevent backtick substitution. >>> I'm not really sure how much thought went into the exis

Re: Thoughts on using Text::Template for our autogenerated code?

2023-04-03 Thread Corey Huinker
> > Yeah, it's somewhat hard to believe that the cost/benefit ratio would be > attractive. But maybe you could mock up some examples of what the input > could look like, and get people on board (or not) before writing any > code. > > tl;dr - I tried a few things, nothing that persuades myself let

Re: Thoughts on using Text::Template for our autogenerated code?

2023-03-30 Thread Corey Huinker
> > I don't think that's remotely a starter. Asking people to install an old > and possibly buggy version of a perl module is not something we should do. > > I think the barrier for this is pretty high. I try to keep module > requirements for the buildfarm client pretty minimal, and this could

Re: Thoughts on using Text::Template for our autogenerated code?

2023-03-30 Thread Corey Huinker
> > I think many of those could just be replaced by multi-line strings and/or > here > documents to get most of the win. > I agree that a pretty good chunk of it can be done with here-docs, but template files do have attractive features (separation of concerns, syntax highlighting, etc) that made

Thoughts on using Text::Template for our autogenerated code?

2023-03-30 Thread Corey Huinker
Is there a barrier to us using non-core perl modules, in this case Text::Template? I think it would be a tremendous improvement in readability and maintainability over our current series of print statements, some multiline, some not. The module itself works like this

Re: Autogenerate some wait events code and documentation

2023-03-30 Thread Corey Huinker
On Wed, Mar 29, 2023 at 8:51 AM Drouvot, Bertrand < bertranddrouvot...@gmail.com> wrote: > Hi, > > On 3/29/23 11:44 AM, Drouvot, Bertrand wrote: > > > > > Looking forward to your feedback, > > Just realized that more polishing was needed. > > Done in V2 attached. > > Regards, > > -- > Bertrand

Re: Using Ephemeral Named Relation like a temporary table

2023-03-28 Thread Corey Huinker
On Wed, Mar 29, 2023 at 12:54 AM Yugo NAGATA wrote: > Hello, > > > Temporary tables are often used to store transient data in > batch processing and the contents can be accessed multiple > times. However, frequent use of temporary tables has a problem > that the system catalog tends to bloat. I

Re: CAST(... ON DEFAULT) - WIP build on top of Error-Safe User Functions

2023-03-28 Thread Corey Huinker
On Tue, Mar 28, 2023 at 3:25 PM Isaac Morland wrote: > On Mon, 19 Dec 2022 at 17:57, Corey Huinker > wrote: > >> >> Attached is my work in progress to implement the changes to the CAST() >> function as proposed by Vik Fearing. >> >> CAST(expr AS typena

Re: CAST(... ON DEFAULT) - WIP build on top of Error-Safe User Functions

2023-03-28 Thread Corey Huinker
On Tue, Mar 28, 2023 at 2:53 PM Gregory Stark (as CFM) wrote: > On Tue, 3 Jan 2023 at 14:16, Tom Lane wrote: > > > > Vik Fearing writes: > > > > > I don't think we should add that syntax until I do get it through the > > > committee, just in case they change something. > > > > Agreed. So this

Re: Add SHELL_EXIT_CODE to psql

2023-03-24 Thread Corey Huinker
On Tue, Mar 21, 2023 at 3:33 PM Corey Huinker wrote: > >> As you had it, any nonzero result would prevent backtick substitution. >> I'm not really sure how much thought went into the existing behavior, >> but I am pretty sure that it's not part of this patch's

Re: Make ON_ERROR_STOP stop on shell script failure

2023-03-24 Thread Corey Huinker
On Fri, Mar 24, 2023 at 2:16 PM Corey Huinker wrote: > > > On Fri, Mar 24, 2023 at 11:07 AM Peter Eisentraut < > peter.eisentr...@enterprisedb.com> wrote: > >> On 20.03.23 19:31, Greg Stark wrote: >> > So I took a look at this patch. The conflict is with

Re: Make ON_ERROR_STOP stop on shell script failure

2023-03-24 Thread Corey Huinker
On Fri, Mar 24, 2023 at 11:07 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 20.03.23 19:31, Greg Stark wrote: > > So I took a look at this patch. The conflict is with 2fe3bdbd691 > > committed by Peter Eisentraut which added error checks for pipes. > > Afaics the behaviour

Re: doc: add missing "id" attributes to extension packaging page

2023-03-23 Thread Corey Huinker
> > TBH I'm a bit afraid that people will immediately start complaining > about the failing docs builds after this got applied since it forces > them to add ids to all varlistenries in a variablelist if they add one, > which can be perceived as quite a burden (also committers and reviewers > will

Re: Add n_tup_newpage_upd to pg_stat table views

2023-03-22 Thread Corey Huinker
> > > * No more dedicated struct to carry around the type of an update. > > We just use two boolean arguments to the pgstats function instead. The > struct didn't seem to be adding much, and it was distracting to track > the information this way within heap_update(). > That's probably a good

Re: Add SHELL_EXIT_CODE to psql

2023-03-21 Thread Corey Huinker
> > > As you had it, any nonzero result would prevent backtick substitution. > I'm not really sure how much thought went into the existing behavior, > but I am pretty sure that it's not part of this patch's charter to > change that. > > regards, tom lane > The changes all

Re: Add SHELL_EXIT_CODE to psql

2023-03-20 Thread Corey Huinker
On Mon, Mar 20, 2023 at 1:01 PM Tom Lane wrote: > Corey Huinker writes: > > 128+N is implemented. > > I think this mostly looks OK, but: > > * I still say there is no basis whatever for believing that the result > of ferror() is an exit code, errno code, or anything

Re: Add SHELL_EXIT_CODE to psql

2023-03-17 Thread Corey Huinker
On Fri, Mar 10, 2023 at 3:49 PM Tom Lane wrote: > I'm okay with adopting bash's rule, but then it should actually match > bash --- signal N is reported as 128+N, not -N. > 128+N is implemented. Nonzero pclose errors of any kind will now overwrite an existing exit_code. I didn't write a formal

Re: Add SHELL_EXIT_CODE to psql

2023-03-02 Thread Corey Huinker
On Thu, Mar 2, 2023 at 1:35 PM Tom Lane wrote: > Corey Huinker writes: > > [ v9-0003-Add-psql-variables-SHELL_ERROR-and-SHELL_EXIT_COD.patch ] > > I took a brief look through this. I'm on board with the general > concept, but I think you've spent too much time on an ultimatel

Re: verbose mode for pg_input_error_message?

2023-02-24 Thread Corey Huinker
On Thu, Feb 23, 2023 at 4:47 PM Nathan Bossart wrote: > On Thu, Feb 23, 2023 at 11:30:38AM -0800, Nathan Bossart wrote: > > Will post a new version shortly. > > As promised... > > -- > Nathan Bossart > Amazon Web Services: https://aws.amazon.com Looks good to me, passes make check-world.

Re: Disable vacuuming to provide data history

2023-02-24 Thread Corey Huinker
On Thu, Feb 23, 2023 at 6:04 AM wrote: > Hey, > > It depnends on scenario, but there is many use cases that hack data > change from somebody with admin privileges could be disaster. > That is the place where data history could come with help. Some basic > solution would be trigger which writes

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

2023-02-24 Thread Corey Huinker
On Thu, Feb 23, 2023 at 2:39 PM Andres Freund wrote: > Hi, > > On 2023-02-23 13:56:56 -0500, Tom Lane wrote: > > Corey Huinker writes: > > > My not-ready-for-16 work on CAST( ... ON DEFAULT ... ) involved making > > > FuncExpr/IoCoerceExpr/Arra

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

2023-02-23 Thread Corey Huinker
On Wed, Feb 22, 2023 at 5:47 PM Andres Freund wrote: > Hi, > > On 2023-02-22 16:34:44 -0500, Tom Lane wrote: > > I wrote: > > > Andres Freund writes: > > >> Maybe it's worth sticking a StaticAssert() for the struct size > > >> somewhere. > > > > > Indeed. I thought we had one already. > > > >

Re: Add SHELL_EXIT_CODE to psql

2023-02-22 Thread Corey Huinker
On Wed, Feb 22, 2023 at 4:18 PM Thomas Munro wrote: > On Tue, Jan 31, 2023 at 9:23 AM Corey Huinker > wrote: > >> Unfortunately, there is a fail in FreeBSD > https://cirrus-ci.com/task/6466749487382528 > >> > >> Maybe, this patch is need to b

Re: Add SHELL_EXIT_CODE to psql

2023-02-22 Thread Corey Huinker
> > > > The patch set seem to be in a good shape and pretty stable for quite a > while. > Could you add one more minor improvement, a new line after variables > declaration? > As you wish. Attached. > > After this changes, I think, we make this patch RfC, shall we? > > Fingers crossed. From

Re: psql memory leaks

2023-02-21 Thread Corey Huinker
On Mon, Feb 20, 2023 at 9:56 PM Kyotaro Horiguchi wrote: > I noticed that \bind is leaking memory for each option. > > =# SELECT $1, $2, $3 \ bind 1 2 3 \g > > The leaked memory blocks are comming from > psql_scan_slash_option(). The attached small patch resolves that > issue. I looked through

Re: Improving inferred query column names

2023-02-11 Thread Corey Huinker
On Sat, Feb 11, 2023 at 3:47 PM Vladimir Churyukin wrote: > For backwards compatibility I guess you can have a GUC flag controlling > that behavior that can be set into backwards compatibility mode if required. > The previous functionality can be declared deprecated and removed (with > the flag)

  1   2   3   4   >